In this post I will investigate an unexpected CloudWatch charge on my April 2022 AWS bill, and explain how to interpret the bill and find the resources responsible.
Table of Contents
- Introduction
- The April 2022 Bill
- CloudWatch Alarm Dashboard
- CloudTrail Event History
- Charge Explanations
- Summary
Introduction
My April 2022 AWS bill has arrived. The total wasn’t unusual – £4.16 is a pretty standard charge for me at the moment, most of which is S3. Then I took a closer look at the services and found an unexpected cost for CloudWatch, which is usually zero.
But not this month:
While $0.30 isn’t bank-breaking, it is unexpected and worth investigating. More importantly, nothing should be running in EU London! And there were no CloudWatch changes at all on my March 2022 bill. So what’s going on here?
Let’s start with the bill itself.
The April 2022 Bill
Looking at the bill, the rows with unexpected CloudWatch charges all mention alarms. Since nothing else has generated any charges, let’s take a closer look at all of the rows referring to alarms.
$0.00 Per Alarm Metric Month – First 10 Alarm Metrics – 10.000 Alarms
The AWS Always Free Tier includes ten CloudWatch alarms.
$0.10 Per Alarm Metric Month (Standard Resolution) – EU (Ireland) – 2.000002 Alarms
In EU Ireland, each standard resolution alarm after the first ten costs $0.10. The bill says there are twelve alarms in EU Ireland
$0.10 Per Alarm Metric Month (Standard Resolution) – EU (London) – 1.000001 Alarms
CloudWatch standard resolution alarms also cost $0.10 in EU London. As all my free alarms are seemingly in EU Ireland, the one in EU London costs a further $0.10.
So the bill is saying I have thirteen alarms – twelve in EU Ireland and one in EU London. Let’s open CloudWatch and see what’s going on there.
CloudWatch Alarm Dashboard
It seems I have thirteen CloudWatch alarms. Interesting, because I could only remember the four security alarms I set up in February.
CloudWatch says otherwise. This is my current EU Ireland
Closer inspection finds eight alarms with names like:
TargetTracking-table/Rides-ProvisionedCapacityHigh-a53f2f67-9477-45a6-8197-788d2c7462b3TargetTracking-table/Rides-ProvisionedCapacityLow-a36cf02f-7b3c-4fb0-844e-cf3d03fa80a9
Two of these are constantly In Alarm, and all have Last State Update values on 2022-03-17. The alarm names led me to suspect that DynamoDB was involved, and this was confirmed by viewing the Namespace and Metric Name values in the details of one of the alarms:
At this point I had an idea of what was going on. To be completely certain, I wanted to check my account history for 2022-03-17. That means a trip to CloudTrail!
CloudTrail Event History
CloudTrail’s Event History shows the last 90 days of management events. I entered a date range of 2022-03-17 00:00 > 2022-03-18 00:01 into the search filter, and it didn’t take long to start seeing some familiar-looking Resource Names:
Alongside the TargetTracking-table resource names linked to monitoring.amazonaws.com, there are also rows on the same day for other Event Sources including:
dynamodb.amazonaws.comapigateway.amazonaws.comlambda.amazonaws.comcognito-idp.amazonaws.com
I now know with absolute certainty where the unexpected CloudWatch alarms came from. Let me explain.
Charge Explanations
So far I’ve reviewed my bills, found the CloudWatch alarms and established what was happening in my account when they were added. Now I’ll explain how this all led to charges on my bill.
The $0.20 EU Ireland Charge
When I was recently studying for the Developer Associate certification, I followed an AWS tutorial on how to Build a Serverless Web Application with AWS Lambda, Amazon API Gateway, AWS Amplify, Amazon DynamoDB, and Amazon Cognito. This was to top up my serverless knowledge before the exam.
The third module involves creating a DynamoDB table for the application. A table that I provisioned with auto-scaling for read and write capacity:
These auto-scaling policies rely on CloudWatch alarms to function, as demonstrated by some of the alarm conditions:
The DynamoDB auto-scaling created eight CloudWatch alarms. Four for Read Capacity Units:
ConsumedReadCapacityUnits > 42 for 2 datapoints within 2 minutesConsumedReadCapacityUnits < 30 for 15 datapoints within 15 minutesProvisionedReadCapacityUnits > 1 for 3 datapoints within 15 minutesProvisionedReadCapacityUnits < 1 for 3 datapoints within 15 minutes
And four for Write Capacity Units:
ConsumedWriteCapacityUnits > 42 for 2 datapoints within 2 minutesConsumedWriteCapacityUnits < 30 for 15 datapoints within 15 minutesProvisionedWriteCapacityUnits > 1 for 3 datapoints within 15 minutesProvisionedWriteCapacityUnits < 1 for 3 datapoints within 15 minutes
These eight alarms joined the existing four. The first ten were free, leaving two accruing charges.
This also explains why two alarms are always In Alarm – the criteria for scaling in are being met but the DynamoDB table can’t scale down any further.
I could have avoided this situation by destroying the resources after finishing the tutorial. The final module of the tutorial covers this. Instead I decided to keep everything around so I could take a proper look at everything under the hood.
No resources accrued any charges in March, so I left everything in place during April. I’ll go into why there was nothing on the March bill shortly, but first…
The $0.10 EU London Charge
Remember when I said that I shouldn’t be running anything in EU London? Turns out I was!
I found a very old CloudWatch alarm from 2020. It’s been there ever since. Never alerting so I didn’t know it was there. Included in the Always Free tier, so never costing me anything or triggering an AWS Budget alert. Appearing on my bill, but always as a free entry so never drawing attention.
When I exceeded my ten free CloudWatch alarms, the one in EU London became chargeable for the first time. A swift delete later and that particular problem is no more.
No CloudWatch Charge On The March 2022 Bill
That only leaves the question of why there were no CloudWatch charges on my March 2022 bill, despite there being thirteen alarms on my account for almost half of that month:
I wanted to understand what was going on, so I reached out to AWS Support.
In what must have been a first for them, I asked why no money had been billed for CloudWatch in March:
On my April 2022 bill I was charged $0.30 for CloudWatch. $0.20 in Ireland and $0.10 in London. I understand why.
What I want to understand is why I didn’t see a charge for them on my March 2022 bill. The alerts were added to the account on March 17th, so from that moment on I had thirteen alerts which is three over the free tier.
Can I get confirmation on why they don’t appear on March but do on April please?
I soon received a reply from AWS Support that explained the events in full:
…although you enabled all 13 Alarms in March, the system only calculated a pro-rated usage value, since the Alarms were only enabled on 17th March. The pro-rated Alarm usage values only amounted to 7.673 Alarms in the EU (Ireland) region, and 1.000003 Alarms in the EU (London) region.
The total pro-rated Alarm usage calculated for March (8.673003 Alarms) is thus within the 10 Alarm Free Tier threshold and thus incurred no charges, whereas in April the full 13 Alarm usage came into play for the entire month…
To summarise, I hadn’t been charged for the alarms in March because they’d only been on my account for almost half a month. Thanks for the help folks!
Summary
In this post I investigated an unexpected CloudWatch charge on my April 2022 AWS bill. I showed what the bill looked like, demonstrated how to find the resources generating the charges and explained how those resources came to be on my AWS account.
If this post has been useful, please feel free to follow me on the following platforms for future updates:
Thanks for reading ~~^~~