Managing on-demand AWS infrastructure for maximum cost optimization

On-demand IT infrastructure is a great option for cost-conscious companies that want enterprise-class technology without investing in physical hardware or dedicated space in a datacenter. And Amazon Web Services (AWS) is as a leader in the on-demand infrastructure category with plenty of options for cost optimization.

In a previous post, I looked at various service and pricing options available for both Amazon S3 storage and EC2 compute resources. Picking the model that best supports your project’s needs, at the most attractive price point, is the first step in cost optimization of your AWS services.

However, many AWS clients – regardless of the plan they choose – find themselves paying for resources on a per-use basis. To truly get the most from your infrastructure budget, you also need to manage on-demand resource use and your overall AWS account structure.

In this post, I’ll look at some operational tactics that will help further optimize your costs for on-demand infrastructure through AWS.

Consolidate multiple AWS accounts

One of the best cost optimization tactics is also among the most obvious. If you have a minimum of two AWS accounts, you can usually save money if you consolidate your accounts.

Consolidating accounts gives you more control over all your resources. Of course, you'll still have to pay for each resource, but having a single account does let you take advantage of volume discounts, most notably in S3 data transfer and storage volume. Data transfer and volume pricing is based on a progressive discount, so combining all that activity under a single account is usually a good call.

Also remember that the 650 hours of Free Usage Tier you receive with each account is not cumulative; if you were to consolidate three accounts into one, that would be 1,300 in Free Usage Tier hours (2 x 650) you’d lose. So this is a factor you should consider before deciding to consolidate numerous accounts.

Use auto-scaling to create and destroy instances

This tactic not only saves money but also enables a redundancy plan by automatically removing unusable resources. An example scenario of this could be two different servers, A and B, being used for the same website. If server A crashes and becomes unusable, the auto-scaling service will create a server C and automatically destroy server A.  Generally, auto-scaling involves the following steps:

1. Create a dependable image from your running instance.
2. Use that image to create more instances on-demand when the resource demands of your website start to grow.
3. Once demand drops, the instances are no longer necessary, and those new machines are automatically destroyed.

Do note that the use of such services requires some additional maintenance:

• An external database
• External cache
• External session control. This is not mandatory but recommended.

Use Elastic IP addresses wisely

AWS tries to push a very small limit of static IP addresses (which it brands as Elastic IP addresses) to its users. These are typically used to mask the failure of an instance and remap to another resource. By default, we can use only five of them in an AWS account. If we want more, we have to ask, and provide a good reason.

If you want to use static IPs, make sure that all of them are being used. Otherwise, you will be charged. That’s right – you are charged only if you do not use all your static IPs. So, before allocating a new static IP, make sure that it will be used.

Terminate any unused on-demand AWS resources

The entire idea of on-demand computing is that you only pay for storage and compute that you need. And if you aren’t using a resource, you don’t need it. And many environments don’t need to have all resources available 24/7.

We pay for running instances. Stopped instances don't charge you. However, you still have to pay for any other resources related to the stopped instances.

Let's say that you have a staging environment on an EC2 instance that is Linux, m5.xlarge, in Canada. The price per hour for an on-demand instance is $0.214. Leaving your instance running the whole month, you’ll have to pay about $160. If you get a Reserved Instance, the price goes to $98.55 with no upfront charge. Sounds pretty good. But there are additional ways to pay less.

Assume that the company works from Monday to Friday, 9 a.m. to 5 p.m., and by policy nobody can make changes to the staging server outside those hours. From that scenario, we actually are using the instance only about 160 hours per month.

Instead of having a Reserved Instance, we can just have the machine down when we are not using it and pay just $34.24 per month.

Of course, manually starting and stopping instances daily can be more costly in staff hours than simply running the resource continuously. Fortunately, this process can be automated. 

A few more cost-optimization tips

Here are a few additional pointers on how to manage your AWS resources for maximum cost efficiency.

• Running your cronjobs on an instance is expensive. Lambda calls can do the same job using an external resource that will be live only for the necessary time. Instead of having an EC2 instance to handle your website and also some heavy cronjobs, you can use Lambda calls on strong instances and have a smaller server running 24/7/365 only to host your website.
• Local databases are cheaper on a single vertical scaled instance. You might be able to save money by migrating to an external database and scale horizontally behind a load balancer.
• Have a whole database to store client visits? It can be much cheaper if you use Amazon’s NoSQL DynamoDB service with SQS calls.
• Inbound traffic is free. If you download a 10GB backup file, you have to pay for such traffic. But if you want to move the same 10GB file from an EC2 instance to a S3 bucket, it's free. So use internal traffic whenever possible.

Conclusion

On-demand infrastructure is a great way to get the most out of your IT budget, but you must manage your pay-as-you resource usage to get the full benefits of this approach. By using these tips to cost optimize your AWS services, you’ll realize some significant monthly savings, particularly if you handle high demand for data and compute resources.