AWS Cost Explorer
A
AWS Cost Explorer

tap-aws-cost-explorer (meltanolabs variant)

AWS Cost Explorer reports what you spend on Amazon Web Services, broken down by account, service, usage type, region, cost allocation tag and cost category.

This tap extracts daily or monthly costs through the Cost Explorer API. Run against an AWS Organizations management account, it covers every member account in the organization.

Prerequisites

Access key: In the AWS IAM console, create a user with an access key, and give it these permissions:

| Permission | Used by | |---|---| | ce:GetCostAndUsage | Every cost* stream | | organizations:ListAccounts | accounts |

To cover every account in an organization, create the user, or the role in Role ARN, in the organization's management account. Costs are then split out by member account in the linked_account column. A member account's credentials only return that account's costs, and can't list the organization's accounts, so deselect accounts.

Cost: AWS bills each Cost Explorer API request, at $0.01 per request at the time of writing. The tap makes one request per page of results for each selected cost* stream.

Available Streams

Standard Streams

| Stream | Description | Replication | Replication Key | Primary Key | |---|---|---|---|---| | cost | Total cost | Incremental | time_period_start | time_period_start, metric_name | | cost_by_account_service | Cost by account and service | Incremental | time_period_start | time_period_start, linked_account, service, metric_name | | cost_by_account_usage_type | Cost by account and usage type | Incremental | time_period_start | time_period_start, linked_account, usage_type, metric_name | | cost_by_account_region | Cost by account and region | Incremental | time_period_start | time_period_start, linked_account, region, metric_name | | cost_by_service_usage_type | Cost by service and usage type | Incremental | time_period_start | time_period_start, service, usage_type, metric_name | | accounts | Accounts in the AWS organization | Full Table | — | id |

Custom Reports

Cost Explorer can group costs by at most two things at a time, so each grouping is its own stream. Add streams for other groupings with the Reports setting. Each report is grouped by up to two dimensions (such as SERVICE, LINKED_ACCOUNT, USAGE_TYPE, REGION, INSTANCE_TYPE or RECORD_TYPE), cost allocation tags (TAG) or cost categories (COST_CATEGORY), and can be filtered:

[
  {
    "name": "cost_by_team",
    "group_by": [
      {"type": "TAG", "key": "team"},
      {"type": "DIMENSION", "key": "SERVICE"}
    ],
    "filter": {"Dimensions": {"Key": "RECORD_TYPE", "Values": ["Usage"]}}
  }
]

Group columns are named after the key in snake case, with tags prefixed tag_ and cost categories cost_category_, so this report has tag_team and service columns. A tag must be activated as a cost allocation tag in the AWS Billing console before costs can be grouped by it.

Limitations

  • 14 months of history. Cost Explorer serves the last 14 months of costs, or 38 months of monthly costs if multi-year data is enabled in the account.
  • Two groupings per stream. Each stream is grouped by at most two dimensions, tags or cost categories.
  • Untagged costs have no tag value. Costs without the tag, or outside every cost category, have an empty tag or cost category column.

Data recovery / backfill

Each cost* stream keeps a bookmark: the latest time_period_start it has loaded. Each run starts Lookback Days before the bookmark, because AWS revises recent costs as usage is processed and while a month's bill is finalized. Rows that are read again replace the earlier ones, by primary key. accounts is full table and re-syncs completely on each run.

To backfill, reset the pipeline state and set Start Date to the first day you need, within the last 14 months. Lowering Start Date alone has no effect once a bookmark exists.

cost and other cost* streams

Support incremental replication using time_period_start. Each row is one metric, for one time period and group.

| Field | Type | Description | |---|---|---| | time_period_start | date | First day of the period — replication key | | time_period_end | date | Day after the last day of the period | | estimated | boolean | Whether AWS still considers the amount an estimate | | linked_account | string | AWS account ID (streams grouped by account) | | service | string | AWS service, e.g. Amazon Simple Storage Service (streams grouped by service) | | usage_type | string | Usage type, e.g. USE1-TimedStorage-ByteHrs (streams grouped by usage type) | | region | string | AWS region, e.g. us-east-1 (streams grouped by region) | | metric_name | string | The metric, e.g. UnblendedCost | | amount | number | The amount | | amount_unit | string | The unit, e.g. USD, or N/A for UsageQuantity |

accounts

Accounts in the AWS organization. Join id to linked_account to name the accounts in cost streams.

| Field | Type | Description | |---|---|---| | id | string | AWS account ID | | arn | string | Account ARN | | email | string | Email address of the account's root user | | name | string | Account name | | state | string | Account state, e.g. ACTIVE or SUSPENDED | | status | string | Account status, the older form of state | | joined_method | string | How the account joined the organization: INVITED or CREATED | | joined_timestamp | datetime | When the account joined the organization |

Settings

Access Key ID

AWS access key ID, from the IAM console. Use a key in the organization's management account to cover every member account.

Secret Access Key

AWS secret access key for the access key ID.

Session Token

AWS session token. Only needed for temporary credentials.

Role ARN

IAM role to assume after signing in with the access key, for example a read-only billing role in the organization's management account. Leave empty to use the access key's own permissions.

Start Date

First day of costs to extract, e.g. 2025-11-01. Cost Explorer serves the last 14 months.

End Date

Day to stop at, exclusive. Leave empty to extract costs up to yesterday.

Lookback Days

Days before the bookmark to extract again on each run, to pick up costs AWS has revised.

Granularity

Time period of each row.

Metrics

Cost metrics to extract: AmortizedCost, BlendedCost, NetAmortizedCost, NetUnblendedCost, NormalizedUsageAmount, UnblendedCost and UsageQuantity.

Reports

Extra cost streams, each grouped by up to two dimensions, tags or cost categories. Each report has a name, a group_by list of {"type": ..., "key": ...} entries, and an optional Cost Explorer filter. See the description above for an example.

Meltano Cloud Connector

AWS Cost Explorer connector is available on Meltano. It is built, maintained, supported, and tested by Meltano.

Why Meltano?
Expert supportDirect access to the team that built and maintains Meltano Cloud. Same-day responses during UK business hours. When something breaks, we fix it fast because we know exactly how it works.
Rigorously testedEvery connector goes through comprehensive testing and quality checks before production. Daily monitoring catches issues before they hit your pipelines. We don't just wrap open-source taps and hope for the best. We validate, we test, we maintain.
No maintenance overheadAPI changes. Connector updates. Schema drift. Breaking changes from upstream sources. We handle it all. Your team focuses on using data. Our team focuses on making sure it's there when you need it.
Access to Meltano Slack communityJoin 5,500+ data engineers and analytics practitioners. The community is active, helpful, and always on. Good for quick questions, sharing patterns, and learning what others are building.