
AWS Cost Explorer
tap-aws-cost-explorer (meltanolabs variant)
AWS Cost Explorer reports what you spend on Amazon Web Services, broken down by account, service, usage type, region, cost allocation tag and cost category.
This tap extracts daily or monthly costs through the Cost Explorer API. Run against an AWS Organizations management account, it covers every member account in the organization.
Prerequisites
Access key: In the AWS IAM console, create a user with an access key, and give it these permissions:
| Permission | Used by |
|---|---|
| ce:GetCostAndUsage | Every cost* stream |
| organizations:ListAccounts | accounts |
To cover every account in an organization, create the user, or the role in Role ARN,
in the organization's management account. Costs are then split out by member account
in the linked_account column. A member account's credentials only return that account's
costs, and can't list the organization's accounts, so deselect accounts.
Cost: AWS bills each Cost Explorer API request, at $0.01 per request at the time of
writing. The tap makes one request per page of results for each selected cost* stream.
Available Streams
Standard Streams
| Stream | Description | Replication | Replication Key | Primary Key |
|---|---|---|---|---|
| cost | Total cost | Incremental | time_period_start | time_period_start, metric_name |
| cost_by_account_service | Cost by account and service | Incremental | time_period_start | time_period_start, linked_account, service, metric_name |
| cost_by_account_usage_type | Cost by account and usage type | Incremental | time_period_start | time_period_start, linked_account, usage_type, metric_name |
| cost_by_account_region | Cost by account and region | Incremental | time_period_start | time_period_start, linked_account, region, metric_name |
| cost_by_service_usage_type | Cost by service and usage type | Incremental | time_period_start | time_period_start, service, usage_type, metric_name |
| accounts | Accounts in the AWS organization | Full Table | — | id |
Custom Reports
Cost Explorer can group costs by at most two things at a time, so each grouping is its own
stream. Add streams for other groupings with the Reports setting. Each report is grouped
by up to two dimensions (such as SERVICE, LINKED_ACCOUNT, USAGE_TYPE, REGION,
INSTANCE_TYPE or RECORD_TYPE), cost allocation tags (TAG) or cost categories
(COST_CATEGORY), and can be filtered:
[
{
"name": "cost_by_team",
"group_by": [
{"type": "TAG", "key": "team"},
{"type": "DIMENSION", "key": "SERVICE"}
],
"filter": {"Dimensions": {"Key": "RECORD_TYPE", "Values": ["Usage"]}}
}
]
Group columns are named after the key in snake case, with tags prefixed tag_ and cost
categories cost_category_, so this report has tag_team and service columns. A tag must
be activated as a cost allocation tag in the AWS Billing console before costs can be grouped
by it.
Limitations
- 14 months of history. Cost Explorer serves the last 14 months of costs, or 38 months of monthly costs if multi-year data is enabled in the account.
- Two groupings per stream. Each stream is grouped by at most two dimensions, tags or cost categories.
- Untagged costs have no tag value. Costs without the tag, or outside every cost category, have an empty tag or cost category column.
Data recovery / backfill
Each cost* stream keeps a bookmark: the latest time_period_start it has loaded. Each run
starts Lookback Days before the bookmark, because AWS revises recent costs as usage is
processed and while a month's bill is finalized. Rows that are read again replace the
earlier ones, by primary key. accounts is full table and re-syncs completely on each run.
To backfill, reset the pipeline state and set Start Date to the first day you need, within the last 14 months. Lowering Start Date alone has no effect once a bookmark exists.
cost and other cost* streams
Support incremental replication using time_period_start. Each row is one metric, for one
time period and group.
| Field | Type | Description |
|---|---|---|
| time_period_start | date | First day of the period — replication key |
| time_period_end | date | Day after the last day of the period |
| estimated | boolean | Whether AWS still considers the amount an estimate |
| linked_account | string | AWS account ID (streams grouped by account) |
| service | string | AWS service, e.g. Amazon Simple Storage Service (streams grouped by service) |
| usage_type | string | Usage type, e.g. USE1-TimedStorage-ByteHrs (streams grouped by usage type) |
| region | string | AWS region, e.g. us-east-1 (streams grouped by region) |
| metric_name | string | The metric, e.g. UnblendedCost |
| amount | number | The amount |
| amount_unit | string | The unit, e.g. USD, or N/A for UsageQuantity |
accounts
Accounts in the AWS organization. Join id to linked_account to name the accounts in
cost streams.
| Field | Type | Description |
|---|---|---|
| id | string | AWS account ID |
| arn | string | Account ARN |
| email | string | Email address of the account's root user |
| name | string | Account name |
| state | string | Account state, e.g. ACTIVE or SUSPENDED |
| status | string | Account status, the older form of state |
| joined_method | string | How the account joined the organization: INVITED or CREATED |
| joined_timestamp | datetime | When the account joined the organization |
Settings
Access Key ID
AWS access key ID, from the IAM console. Use a key in the organization's management account to cover every member account.
Secret Access Key
AWS secret access key for the access key ID.
Session Token
AWS session token. Only needed for temporary credentials.
Role ARN
IAM role to assume after signing in with the access key, for example a read-only billing role in the organization's management account. Leave empty to use the access key's own permissions.
Start Date
First day of costs to extract, e.g. 2025-11-01. Cost Explorer serves the last 14
months.
End Date
Day to stop at, exclusive. Leave empty to extract costs up to yesterday.
Lookback Days
Days before the bookmark to extract again on each run, to pick up costs AWS has revised.
Granularity
Time period of each row.
Metrics
Cost metrics to extract: AmortizedCost, BlendedCost, NetAmortizedCost,
NetUnblendedCost, NormalizedUsageAmount, UnblendedCost and UsageQuantity.
Reports
Extra cost streams, each grouped by up to two dimensions, tags or cost categories. Each
report has a name, a group_by list of {"type": ..., "key": ...} entries, and an
optional Cost Explorer filter. See the description above for an example.
AWS Cost Explorer connector is available on Meltano. It is built, maintained, supported, and tested by Meltano.