When it comes to optimizing cloud spend, the knee-jerk reaction when perusing a bill is to focus on big consumption peaks. While it's only natural, those larger numbers could actually be taking attention away from a range of subtler issues, ones that when totaled up can really bust a cloud budget.
The following are six less-than-obvious issues that may be impacting your bill, accompanied by tips to help you bring cloud costs back to earth and ensure your organization runs as efficiently as possible.
1. Picking Up the Trail?
Paying for AWS CloudTrail? If so, you've got an opportunity to lower costs right off the bat.
While the initial trail in a region won't cost you, it's best just to keep one at the AWS Organization. That's because these are automatically generated in member accounts. New trails are in addition to ones that already exist, so you can eliminate those separate ones for quick savings.
Further, at the AWS Organization level you can apply and enforce event logging strategy organization-wide because configuration propagates to all accounts. So, verify that the configuration of your Organization trail reflects how you want all trails to be moving forward.
2. Got Policies?
If cloud storage costs are consistently increasing, you might not have the correct object lifecycle policies. The right ones automate the transitioning of data to different storage tiers, while deleting via predefined rules. This aligns storage expenses with data value and accessibility needs, effectively preventing overpayment.
If you're not doing this, you could create a build-up of data, a continually growing log store, and far too many snapshots. This raises storage needs, which is especially costly when you've got older or infrequently accessed data in high-cost, high-performant tiers. Transitioning or expiring objects after 30-90 days is a good rule of thumb. With costs increasing, this is an area that should be closely examined.
3. Setting Limits
Organizational policies can control how cloud users may access, use, and manage cloud resources. They're also particularly effective for ensuring employees don't spin up services they should not. What's more, failing to restrict instance types and regions opens a cloud infrastructure to greater security concerns. And when control is lacking, cybercriminals have an increased likelihood of avoiding attention.
Therefore, be sure to limit instance types and regions to only ones that you use. By doing so, people won't be able to spin up instances, whether intentionally or mistakenly, in Europe when an organization's cloud resources are in the U.S.
4. Sky-High APIs
Third-party services like New Relic and Datadog are able to scan accounts and report on cloud consumption. Still, there's a cost to the related API requests, which you'll find in the GetMetricData API SKU. If you don't keep tabs, you could rack up a steep tab from third-party API calls.
For instance, you may have an account utilizing a lot of resources on Cloudwatch. This could be the result of an API call occurring every minute, a frequency you don't typically need. To gain control, have third-party providers tweak the frequency and reduce the metrics being used on specific work.
5. Looking at Your Logging
Logging is vital for monitoring and troubleshooting. Even so, doing too much of it can add to your cloud bill.
Like those aforementioned API calls, determine if your frequency of logging, and the metrics you're culling, is overkill. Logging should never exceed more than 20% of your total cloud bill. If it does, review how teams are using your logs to figure out what frequency and metrics make the most sense.
Pay particular attention to non-production logging because it doesn't generate revenue. In these cases, you often don't need the same history and drill-down details you would for production accounts. Also, when something breaks in a non-production environment, you just have to switch logs on and off, whereas production environments require additional information for troubleshooting.
6. Cross-Checking Choices?
When technology choices aren't cross-checked, they can evolve into major cloud bill problems and performance issues. For instance, should a team purchase a multi-year reserved instance — but the company suddenly decides it will go serverless in six months — there's a risk of ending up with a load of virtual machines that are of no use.
Making cloud-related decisions in a vacuum can severely impact the greater good and bottom line. Be sure that cloud infrastructure choices are cross-checked against engineering strategy and goals.
Bust Your Bill, not Your Budget
Reviewing your bill deeper can bust those sneaky issues before they bust your budget. Tackling those big consumption peaks is imperative, but remember, the sum of minor issues could be accruing and taking an equal toll. So don't accept your bill at face value, continually roll up your sleeves, ask questions, and demand answers.
If you notice S3 costs are steadily increasing, ask which SKUs are causing it. If it's the result of data transfer, then find out from employees if this was known or unexpected. The rise in expenditure could be perfectly legitimate, but you'll never be sure, and it will only continue to build, unless you get to the bottom of it.
This approach can instill in your organization a cost optimization culture, one in which everyone understands how they affect and can minimize a cloud bill. With a full team owning this responsibility, cloud costs will be better controlled and resources more effectively optimized.
About the author:
Sam Clark is a senior technical account manager at DoiT International. He possesses more than 25 years of experience in the technology sector, with a dedicated focus on cloud expertise spanning the last decade. Specializing in optimizing cloud operations and cost efficiency, he champions the principles of FinOps via webinars, podcasts and blog posts.
