Datacomponents
Datacomponents hold configuration for dataplugins, and are the building blocks for constructing pipelines. One dataplugin may be referenced by many datacomponents, each with a different set of properties for the dataplugin settings. One pipeline may reference multiple datacomponents.
Objects
Datacomponent
| Path | Type | Format | Description |
|---|---|---|---|
id |
String |
Version 4 UUID | The datacomponent ID |
created |
String |
ISO 8601 timestamp | When the datacomponent was created |
lastModified |
String |
ISO 8601 timestamp | When the datacomponent was last modified |
name |
String |
The datacomponent name | |
dataPlugin |
String |
Create / update with dataplugin fullyQualifiedName |
|
properties |
Properties |
The datacomponent properties, defined by the dataplugin settings |
{
"id" : "431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5",
"created" : "2024-12-16T10:01:59.073001",
"lastModified" : "2024-12-16T10:01:59.27244",
"name" : "extractors/tap-matatika-sit",
"properties" : { },
"commands" : { },
"dataPlugin" : "extractors/tap-matatika-sit--matatika",
"_embedded" : {
"dataplugin" : {
"id" : "f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"pluginType" : "EXTRACTOR",
"name" : "tap-matatika-sit",
"namespace" : "tap_matatika_sit",
"variant" : "matatika",
"label" : "Matatika SIT",
"description" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs",
"logoUrl" : "/assets/images/datasource/tap-matatika-sit.svg",
"hidden" : false,
"docs" : "https://meltano.com/plugins/extractors/spreadsheets-anywhere.html",
"pipUrl" : "git+https://github.com/ets/tap-spreadsheets-anywhere.git",
"repo" : "https://github.com/ets/tap-spreadsheets-anywhere",
"executable" : "tap-spreadsheets-anywhere",
"capabilities" : [ "DISCOVER", "CATALOG", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "tables",
"aliases" : [ ],
"label" : "Tables",
"value" : "[{\"path\":\"https://raw.githubusercontent.com/Matatika/matatika-examples/master/example_data\",\"name\":\"gitflixusers\",\"pattern\":\"GitFlixUsers.csv\",\"start_date\":\"2021-01-01T00:00:00Z\",\"key_properties\":[\"id\"],\"format\":\"csv\"}]",
"options" : [ ],
"kind" : "ARRAY",
"description" : "A setting in Matatika SIT that allows users to view and manage tables of data.",
"hidden" : false,
"sensitive" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : true,
"requires" : [ ],
"fullDescription" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs\n\n## Settings\n\n\n### Tables\n\nA setting in Matatika SIT that allows users to view and manage tables of data.",
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994"
},
"update dataplugin" : {
"href" : "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"type" : "PUT"
}
}
}
},
"managed" : false,
"draft" : false,
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"update datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"delete datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
}
}
}
Properties
For each setting s in the dataplugin settings:
| Path | Type | Description |
|---|---|---|
s.name |
s.kind |
Refer to s.description |
Requests
- View all datacomponents in a workspace
- View a datacomponent
- Initialise a new datacomponent in a workspace
- Create or update a datacomponent in a workspace
- Update a datacomponent
- Delete a datacomponent
View all datacomponents in a workspace
GET
/api/workspaces/{workspace-id}/datacomponents
Returns all datacomponents in the workspace {workspace-id}.
Prerequisites
- Workspace
{workspace-id}must exist
Request
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/datacomponents' -i -X GET \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests)
import requests
url = "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/datacomponents"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Datacomponent collection with HAL links.
{
"_embedded" : {
"datacomponents" : [ {
"id" : "aa0f1935-22c1-49c4-9262-9b7e460bf0df",
"created" : "2024-12-16T10:01:53.027677",
"lastModified" : "2024-12-16T10:01:53.027678",
"name" : "Warehouse",
"properties" : {
"password" : "jK3f7G3C_X49iCdOK_x4I7daOu",
"dbname" : "wysmawf",
"default_target_schema" : "analytics",
"port" : "5432",
"host" : "sharp-banana.postgres.database.azure.com",
"user" : "wysmawf"
},
"commands" : { },
"dataPlugin" : "loaders/target-postgres--matatika",
"_embedded" : {
"dataplugin" : {
"id" : "a2f063d1-2e66-4b72-b6fe-4d988dd2b07e",
"pluginType" : "LOADER",
"name" : "target-postgres",
"namespace" : "postgres_transferwise",
"variant" : "matatika",
"label" : "Postgres Warehouse",
"description" : "Postgres Warehouse is a data warehousing solution built on top of the Postgres database management system.\n\nPostgres Warehouse is designed to handle large volumes of data and complex queries, making it an ideal solution for businesses that need to store and analyze large amounts of data. It provides a number of features that are specifically tailored to data warehousing, such as columnar storage, parallel processing, and support for advanced analytics. Additionally, Postgres Warehouse is highly scalable, allowing businesses to easily add more resources as their data needs grow. Overall, Postgres Warehouse is a powerful and flexible data warehousing solution that can help businesses make better decisions by providing them with the insights they need to succeed.\n### Prerequisites\nThe process of obtaining the required settings for connecting to a Postgres Warehouse may vary depending on the specific setup and configuration of the database. However, here are some general ways to obtain each of the required settings:\n\n- User: The user is typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the username.\n- Password: The password is also typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the password.\n- Host: The host is the server where the database is located. You can ask the database administrator or check the database documentation to find out the host name or IP address.\n- Port: The port is the number that the database listens on for incoming connections. The default port for Postgres is 5432, but it may be different depending on the configuration. You can ask the database administrator or check the database documentation to find out the port number.\n- Database Name: The database name is the name of the specific database you want to connect to. You can ask the database administrator or check the database documentation to find out the database name.\n- Default Target Schema: The default target schema is the schema that you want to use as the default when connecting to the database. This may be set up by the database administrator or you may need to create it yourself. You can ask the database administrator or check the database documentation to find out the default target schema.",
"logoUrl" : "/assets/logos/loaders/postgres.png",
"hidden" : false,
"docs" : "https://www.matatika.com/data-details/target-postgres/",
"pipUrl" : "git+https://github.com/Matatika/[email protected]",
"repo" : "git+https://github.com/Matatika/[email protected]",
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "user",
"aliases" : [ "username" ],
"label" : "User",
"options" : [ ],
"kind" : "STRING",
"description" : "The username used to connect to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"protected" : false
}, {
"name" : "password",
"aliases" : [ ],
"label" : "Password",
"options" : [ ],
"kind" : "STRING",
"description" : "The password used to authenticate the user.",
"hidden" : false,
"sensitive" : true,
"required" : "true",
"protected" : false
}, {
"name" : "host",
"aliases" : [ "address" ],
"label" : "Host",
"options" : [ ],
"kind" : "STRING",
"description" : "The hostname or IP address of the Postgres Warehouse server.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"protected" : false
}, {
"name" : "port",
"aliases" : [ ],
"label" : "Port",
"value" : "5432",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The port number used to connect to the Postgres Warehouse server.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"protected" : false
}, {
"name" : "dbname",
"aliases" : [ "database" ],
"label" : "Database Name",
"options" : [ ],
"kind" : "STRING",
"description" : "The name of the database to connect to.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"protected" : false
}, {
"name" : "default_target_schema",
"aliases" : [ ],
"label" : "Default Target Schema",
"value" : "analytics",
"options" : [ ],
"kind" : "STRING",
"description" : "The default schema to use when writing data to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"required" : "true",
"protected" : false
}, {
"name" : "ssl",
"aliases" : [ ],
"label" : "SSL",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to use SSL encryption when connecting to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"protected" : false,
"value_post_processor" : "STRINGIFY"
}, {
"name" : "batch_size_rows",
"aliases" : [ ],
"label" : "Batch Size Rows",
"value" : "100000",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The number of rows to write to the Postgres Warehouse in each batch.",
"hidden" : false,
"sensitive" : false,
"protected" : false
}, {
"name" : "underscore_camel_case_fields",
"aliases" : [ ],
"label" : "Underscore Camel Case Fields",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to convert field names from camel case to underscore-separated format.",
"hidden" : true,
"sensitive" : false,
"protected" : false
}, {
"name" : "flush_all_streams",
"aliases" : [ ],
"label" : "Flush All Streams",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to flush all streams to the Postgres Warehouse before closing the connection.",
"hidden" : true,
"sensitive" : false,
"protected" : false
}, {
"name" : "parallelism",
"aliases" : [ ],
"label" : "Parallelism",
"value" : "0",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The number of threads to use when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"protected" : false
}, {
"name" : "parallelism_max",
"aliases" : [ ],
"label" : "Max Parallelism",
"value" : "16",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum number of threads to use when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"protected" : false
}, {
"name" : "default_target_schema_select_permission",
"aliases" : [ ],
"label" : "Default Target Schema Select Permission",
"options" : [ ],
"kind" : "STRING",
"description" : "The permission level required to select data from the default target schema.",
"hidden" : true,
"sensitive" : false,
"protected" : false
}, {
"name" : "schema_mapping",
"aliases" : [ ],
"label" : "Schema Mapping",
"options" : [ ],
"kind" : "STRING",
"description" : "A mapping of source schema names to target schema names.",
"hidden" : true,
"sensitive" : false,
"protected" : false
}, {
"name" : "add_metadata_columns",
"aliases" : [ ],
"label" : "Add Metadata Columns",
"value" : "true",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to add metadata columns to the target table.",
"hidden" : true,
"sensitive" : false,
"protected" : false
}, {
"name" : "hard_delete",
"aliases" : [ ],
"label" : "Hard Delete",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to perform hard deletes when deleting data from the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"protected" : false
}, {
"name" : "data_flattening_max_level",
"aliases" : [ ],
"label" : "Data Flattening Max Level",
"value" : "10",
"options" : [ ],
"kind" : "INTEGER",
"description" : "The maximum level of nested data structures to flatten when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"protected" : false
}, {
"name" : "primary_key_required",
"aliases" : [ ],
"label" : "Primary Key Required",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not a primary key is required for the target table.",
"hidden" : false,
"sensitive" : false,
"protected" : false
}, {
"name" : "validate_records",
"aliases" : [ ],
"label" : "Validate Records",
"value" : "false",
"options" : [ ],
"kind" : "BOOLEAN",
"description" : "Whether or not to validate records before writing them to the Postgres Warehouse.",
"hidden" : false,
"sensitive" : false,
"protected" : false
}, {
"name" : "temp_dir",
"aliases" : [ ],
"label" : "Temporary Directory",
"options" : [ ],
"kind" : "STRING",
"description" : "The directory to use for temporary files when writing data to the Postgres Warehouse.",
"hidden" : true,
"sensitive" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : "Postgres Warehouse is a data warehousing solution built on top of the Postgres database management system.\n\nPostgres Warehouse is designed to handle large volumes of data and complex queries, making it an ideal solution for businesses that need to store and analyze large amounts of data. It provides a number of features that are specifically tailored to data warehousing, such as columnar storage, parallel processing, and support for advanced analytics. Additionally, Postgres Warehouse is highly scalable, allowing businesses to easily add more resources as their data needs grow. Overall, Postgres Warehouse is a powerful and flexible data warehousing solution that can help businesses make better decisions by providing them with the insights they need to succeed.\n### Prerequisites\nThe process of obtaining the required settings for connecting to a Postgres Warehouse may vary depending on the specific setup and configuration of the database. However, here are some general ways to obtain each of the required settings:\n\n- User: The user is typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the username.\n- Password: The password is also typically created when the database is set up. You can ask the database administrator or check the database documentation to find out the password.\n- Host: The host is the server where the database is located. You can ask the database administrator or check the database documentation to find out the host name or IP address.\n- Port: The port is the number that the database listens on for incoming connections. The default port for Postgres is 5432, but it may be different depending on the configuration. You can ask the database administrator or check the database documentation to find out the port number.\n- Database Name: The database name is the name of the specific database you want to connect to. You can ask the database administrator or check the database documentation to find out the database name.\n- Default Target Schema: The default target schema is the schema that you want to use as the default when connecting to the database. This may be set up by the database administrator or you may need to create it yourself. You can ask the database administrator or check the database documentation to find out the default target schema.\n\n## Settings\n\n\n### User\n\nThe username used to connect to the Postgres Warehouse.\n\n### Password\n\nThe password used to authenticate the user.\n\n### Host\n\nThe hostname or IP address of the Postgres Warehouse server.\n\n### Port\n\nThe port number used to connect to the Postgres Warehouse server.\n\n### Database Name\n\nThe name of the database to connect to.\n\n### Default Target Schema\n\nThe default schema to use when writing data to the Postgres Warehouse.\n\n### Batch Size Rows\n\nThe number of rows to write to the Postgres Warehouse in each batch.\n\n### Primary Key Required\n\nWhether or not a primary key is required for the target table.\n\n### Validate Records\n\nWhether or not to validate records before writing them to the Postgres Warehouse.",
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/dataplugins/a2f063d1-2e66-4b72-b6fe-4d988dd2b07e"
},
"update dataplugin" : {
"href" : "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/dataplugins/a2f063d1-2e66-4b72-b6fe-4d988dd2b07e",
"type" : "PUT"
}
}
}
},
"managed" : true,
"draft" : false,
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/datacomponents/aa0f1935-22c1-49c4-9262-9b7e460bf0df"
},
"update datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/aa0f1935-22c1-49c4-9262-9b7e460bf0df"
},
"delete datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/aa0f1935-22c1-49c4-9262-9b7e460bf0df"
}
}
}, {
"id" : "68cb548e-c4ed-4596-a444-9f8722a7e4b1",
"created" : "2024-12-16T10:01:53.099991",
"lastModified" : "2024-12-16T10:01:53.099991",
"name" : "dbt",
"properties" : { },
"commands" : {
"compile" : {
"args" : "compile",
"description" : "Generates executable SQL from source model, test, and analysis files. Compiled SQL files are written to the target/ directory."
},
"docs-generate" : {
"args" : "docs generate",
"description" : "Generate documentation artifacts for your project."
},
"test" : {
"args" : "test",
"description" : "Runs tests on data in deployed models."
},
"seed" : {
"args" : "seed",
"description" : "Load data from csv files into your data warehouse."
},
"run" : {
"args" : "run",
"description" : "Compile SQL and execute against the current target database."
},
"deps" : {
"args" : "deps",
"description" : "Pull the most recent version of the dependencies listed in packages.yml"
},
"clean" : {
"args" : "clean",
"description" : "Delete all folders in the clean-targets list (usually the dbt_modules and target directories.)"
},
"snapshot" : {
"args" : "snapshot",
"description" : "Execute snapshots defined in your project."
}
},
"dataPlugin" : "transformers/dbt--dbt-labs",
"_embedded" : {
"dataplugin" : {
"id" : "3aa40b6c-7dbf-44cb-a67d-d25f84648b40",
"pluginType" : "TRANSFORMER",
"name" : "dbt",
"namespace" : "dbt",
"variant" : "dbt-labs",
"label" : "dbt",
"description" : " Power your project transformations with dbt™, a SQL-first transformation tool that enables analytics engineers to develop transformations with code.\n\n***Version Control and CI/CD***\n\nUse Matatika to deploy and promote changes between dev, UAT, and production environments.\n\n***Test and Document***\n\nUse Matatika to develop and test every model prior to production release, and share dynamically generated documentation with all stakeholders.\n\n***Develop***\n\nWrite modular data transformations in .sql – Matatika together with dbt handles the chore of dependency management. ",
"logoUrl" : "/assets/images/transformer/dbt.png",
"hidden" : false,
"docs" : "https://www.matatika.com/data-details/dbt/",
"pipUrl" : "dbt-core~=1.3.0 dbt-postgres~=1.3.0 dbt-snowflake~=1.3.0\n",
"repo" : "https://github.com/dbt-labs/dbt-core",
"capabilities" : [ ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "project_dir",
"aliases" : [ ],
"value" : "$MELTANO_PROJECT_ROOT/transform",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"protected" : false
}, {
"name" : "profiles_dir",
"aliases" : [ ],
"value" : "$MELTANO_PROJECT_ROOT/transform/profile",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"env" : "DBT_PROFILES_DIR",
"protected" : false
}, {
"name" : "target",
"aliases" : [ ],
"value" : "$MELTANO_LOAD__DIALECT",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"protected" : false
}, {
"name" : "source_schema",
"aliases" : [ ],
"value" : "$MELTANO_LOAD__TARGET_SCHEMA",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"protected" : false
}, {
"name" : "target_schema",
"aliases" : [ ],
"value" : "analytics",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"protected" : false
}, {
"name" : "models",
"aliases" : [ ],
"value" : "$MELTANO_TRANSFORM__PACKAGE_NAME $MELTANO_EXTRACTOR_NAMESPACE my_meltano_project",
"options" : [ ],
"kind" : "STRING",
"hidden" : false,
"sensitive" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : {
"compile" : {
"args" : "compile",
"description" : "Generates executable SQL from source model, test, and analysis files. Compiled SQL files are written to the target/ directory."
},
"docs-generate" : {
"args" : "docs generate",
"description" : "Generate documentation artifacts for your project."
},
"test" : {
"args" : "test",
"description" : "Runs tests on data in deployed models."
},
"seed" : {
"args" : "seed",
"description" : "Load data from csv files into your data warehouse."
},
"run" : {
"args" : "run",
"description" : "Compile SQL and execute against the current target database."
},
"deps" : {
"args" : "deps",
"description" : "Pull the most recent version of the dependencies listed in packages.yml"
},
"clean" : {
"args" : "clean",
"description" : "Delete all folders in the clean-targets list (usually the dbt_modules and target directories.)"
},
"snapshot" : {
"args" : "snapshot",
"description" : "Execute snapshots defined in your project."
}
},
"matatikaHidden" : false,
"requires" : [ {
"id" : "e6c1ad3d-ebf5-4c4a-b129-f68156b47555",
"pluginType" : "FILE",
"name" : "files-dbt",
"namespace" : "dbt",
"variant" : "matatika",
"description" : " Files dbt is a file bundle that automatically configures your project to run transforms with dbt.\nThe bundle includes template project configuration:\n\n- transform/models (directory)\n- transform/profile/profiles.yml\n- transform/dbt_project.yml\n- transform/.gitignore\n- transform/macros/centralize_test_failures.sql\n",
"hidden" : false,
"pipUrl" : "git+https://github.com/Matatika/[email protected]",
"repo" : "https://github.com/Matatika/files-dbt",
"capabilities" : [ ],
"select" : [ ],
"update" : {
"transform/profile/profiles.yml" : "true"
},
"vars" : { },
"settings" : [ ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : false,
"requires" : [ ],
"fullDescription" : " Files dbt is a file bundle that automatically configures your project to run transforms with dbt.\nThe bundle includes template project configuration:\n\n- transform/models (directory)\n- transform/profile/profiles.yml\n- transform/dbt_project.yml\n- transform/.gitignore\n- transform/macros/centralize_test_failures.sql\n"
} ],
"fullDescription" : " Power your project transformations with dbt™, a SQL-first transformation tool that enables analytics engineers to develop transformations with code.\n\n***Version Control and CI/CD***\n\nUse Matatika to deploy and promote changes between dev, UAT, and production environments.\n\n***Test and Document***\n\nUse Matatika to develop and test every model prior to production release, and share dynamically generated documentation with all stakeholders.\n\n***Develop***\n\nWrite modular data transformations in .sql – Matatika together with dbt handles the chore of dependency management. ",
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/dataplugins/3aa40b6c-7dbf-44cb-a67d-d25f84648b40"
},
"update dataplugin" : {
"href" : "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/dataplugins/3aa40b6c-7dbf-44cb-a67d-d25f84648b40",
"type" : "PUT"
}
}
}
},
"managed" : true,
"draft" : false,
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/datacomponents/68cb548e-c4ed-4596-a444-9f8722a7e4b1"
},
"update datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/68cb548e-c4ed-4596-a444-9f8722a7e4b1"
},
"delete datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/68cb548e-c4ed-4596-a444-9f8722a7e4b1"
}
}
}, {
"id" : "431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5",
"created" : "2024-12-16T10:01:59.073001",
"lastModified" : "2024-12-16T10:01:59.27244",
"name" : "extractors/tap-matatika-sit",
"properties" : { },
"commands" : { },
"dataPlugin" : "extractors/tap-matatika-sit--matatika",
"_embedded" : {
"dataplugin" : {
"id" : "f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"pluginType" : "EXTRACTOR",
"name" : "tap-matatika-sit",
"namespace" : "tap_matatika_sit",
"variant" : "matatika",
"label" : "Matatika SIT",
"description" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs",
"logoUrl" : "/assets/images/datasource/tap-matatika-sit.svg",
"hidden" : false,
"docs" : "https://meltano.com/plugins/extractors/spreadsheets-anywhere.html",
"pipUrl" : "git+https://github.com/ets/tap-spreadsheets-anywhere.git",
"repo" : "https://github.com/ets/tap-spreadsheets-anywhere",
"executable" : "tap-spreadsheets-anywhere",
"capabilities" : [ "DISCOVER", "CATALOG", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "tables",
"aliases" : [ ],
"label" : "Tables",
"value" : "[{\"path\":\"https://raw.githubusercontent.com/Matatika/matatika-examples/master/example_data\",\"name\":\"gitflixusers\",\"pattern\":\"GitFlixUsers.csv\",\"start_date\":\"2021-01-01T00:00:00Z\",\"key_properties\":[\"id\"],\"format\":\"csv\"}]",
"options" : [ ],
"kind" : "ARRAY",
"description" : "A setting in Matatika SIT that allows users to view and manage tables of data.",
"hidden" : false,
"sensitive" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : true,
"requires" : [ ],
"fullDescription" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs\n\n## Settings\n\n\n### Tables\n\nA setting in Matatika SIT that allows users to view and manage tables of data.",
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994"
},
"update dataplugin" : {
"href" : "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"type" : "PUT"
}
}
}
},
"managed" : false,
"draft" : false,
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"update datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"delete datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/datacomponents?page=0&size=20"
}
},
"page" : {
"size" : 20,
"totalElements" : 3,
"totalPages" : 1,
"number" : 0
}
}
View a datacomponent
GET
/api/datacomponents/{datacomponent-id}
Returns the datacomponent {datacomponent-id}.
Prerequisites
- Datacomponent
{datacomponent-id}must exist
Request
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5' -i -X GET \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests)
import requests
url = "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Datacomponent with HAL links.
{
"id" : "431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5",
"created" : "2024-12-16T10:01:59.073001",
"lastModified" : "2024-12-16T10:01:59.27244",
"name" : "extractors/tap-matatika-sit",
"properties" : { },
"commands" : { },
"dataPlugin" : "extractors/tap-matatika-sit--matatika",
"_embedded" : {
"dataplugin" : {
"id" : "f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"pluginType" : "EXTRACTOR",
"name" : "tap-matatika-sit",
"namespace" : "tap_matatika_sit",
"variant" : "matatika",
"label" : "Matatika SIT",
"description" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs",
"logoUrl" : "/assets/images/datasource/tap-matatika-sit.svg",
"hidden" : false,
"docs" : "https://meltano.com/plugins/extractors/spreadsheets-anywhere.html",
"pipUrl" : "git+https://github.com/ets/tap-spreadsheets-anywhere.git",
"repo" : "https://github.com/ets/tap-spreadsheets-anywhere",
"executable" : "tap-spreadsheets-anywhere",
"capabilities" : [ "DISCOVER", "CATALOG", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "tables",
"aliases" : [ ],
"label" : "Tables",
"value" : "[{\"path\":\"https://raw.githubusercontent.com/Matatika/matatika-examples/master/example_data\",\"name\":\"gitflixusers\",\"pattern\":\"GitFlixUsers.csv\",\"start_date\":\"2021-01-01T00:00:00Z\",\"key_properties\":[\"id\"],\"format\":\"csv\"}]",
"options" : [ ],
"kind" : "ARRAY",
"description" : "A setting in Matatika SIT that allows users to view and manage tables of data.",
"hidden" : false,
"sensitive" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : true,
"requires" : [ ],
"fullDescription" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs\n\n## Settings\n\n\n### Tables\n\nA setting in Matatika SIT that allows users to view and manage tables of data.",
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994"
},
"update dataplugin" : {
"href" : "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"type" : "PUT"
}
}
}
},
"managed" : false,
"draft" : false,
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"update datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"delete datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
}
}
}
Initialise a new datacomponent in a workspace
POST
/api/workspaces/{workspace-id}/datacomponents
Initialises a new datacomponent in the workspace {workspace-id}.
Request
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/datacomponents' -i -X POST \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests)
import requests
url = "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/datacomponents"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("POST", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
datacomponent with HAL links.
{
"id" : "431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5",
"created" : "2024-12-16T10:01:58.863096946",
"lastModified" : "2024-12-16T10:01:58.863097146",
"properties" : { },
"commands" : { },
"managed" : false,
"draft" : false,
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"create datacomponent" : {
"href" : "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
}
}
}
Create or update a datacomponent in a workspace
PUT
/api/workspaces/{workspace-id}/datacomponents/{datacomponent-id}
Creates or updates the datacomponent {datacomponent-id} in the workspace {workspace-id}.
Request
Body
Datacomponent resource.
{
"name" : "extractors/tap-matatika-sit",
"dataPlugin" : "extractors/tap-matatika-sit"
}
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5' -i -X PUT \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json' \
-d '{
"name" : "extractors/tap-matatika-sit",
"dataPlugin" : "extractors/tap-matatika-sit"
}'
Python (requests)
import requests
url = "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
data = {
"name" : "extractors/tap-matatika-sit",
"dataPlugin" : "extractors/tap-matatika-sit"
}
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("PUT", url, headers=headers, data=data)
print(response.text.encode('utf8'))
Response
200 OK / 201 Created
Datacomponent with HAL links.
{
"id" : "431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5",
"created" : "2024-12-16T10:01:59.073001",
"lastModified" : "2024-12-16T10:01:59.073002",
"name" : "extractors/tap-matatika-sit",
"properties" : { },
"commands" : { },
"dataPlugin" : "extractors/tap-matatika-sit--matatika",
"_embedded" : {
"dataplugin" : {
"id" : "f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"pluginType" : "EXTRACTOR",
"name" : "tap-matatika-sit",
"namespace" : "tap_matatika_sit",
"variant" : "matatika",
"label" : "Matatika SIT",
"description" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs",
"logoUrl" : "/assets/images/datasource/tap-matatika-sit.svg",
"hidden" : false,
"docs" : "https://meltano.com/plugins/extractors/spreadsheets-anywhere.html",
"pipUrl" : "git+https://github.com/ets/tap-spreadsheets-anywhere.git",
"repo" : "https://github.com/ets/tap-spreadsheets-anywhere",
"executable" : "tap-spreadsheets-anywhere",
"capabilities" : [ "DISCOVER", "CATALOG", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "tables",
"aliases" : [ ],
"label" : "Tables",
"value" : "[{\"path\":\"https://raw.githubusercontent.com/Matatika/matatika-examples/master/example_data\",\"name\":\"gitflixusers\",\"pattern\":\"GitFlixUsers.csv\",\"start_date\":\"2021-01-01T00:00:00Z\",\"key_properties\":[\"id\"],\"format\":\"csv\"}]",
"options" : [ ],
"kind" : "ARRAY",
"description" : "A setting in Matatika SIT that allows users to view and manage tables of data.",
"hidden" : false,
"sensitive" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : true,
"requires" : [ ],
"fullDescription" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs\n\n## Settings\n\n\n### Tables\n\nA setting in Matatika SIT that allows users to view and manage tables of data.",
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994"
},
"update dataplugin" : {
"href" : "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"type" : "PUT"
}
}
}
},
"managed" : false,
"draft" : false,
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"update datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"delete datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
}
}
}
Update a datacomponent
PUT
/api/datacomponents/{datacomponent-id}
Updates the datacomponent {datacomponent-id}.
Prerequisites
- Datacomponent
{datacomponent-id}must exist
Request
Body
Datacomponent resource.
{
"name" : "extractors/tap-matatika-sit",
"dataPlugin" : "extractors/tap-matatika-sit"
}
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5' -i -X PUT \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json' \
-d '{
"name" : "extractors/tap-matatika-sit",
"dataPlugin" : "extractors/tap-matatika-sit"
}'
Python (requests)
import requests
url = "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
data = {
"name" : "extractors/tap-matatika-sit",
"dataPlugin" : "extractors/tap-matatika-sit"
}
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("PUT", url, headers=headers, data=data)
print(response.text.encode('utf8'))
Response
200 OK
Datacomponent with HAL links.
{
"id" : "431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5",
"created" : "2024-12-16T10:01:59.073001",
"lastModified" : "2024-12-16T10:01:59.27244",
"name" : "extractors/tap-matatika-sit",
"properties" : { },
"commands" : { },
"dataPlugin" : "extractors/tap-matatika-sit--matatika",
"_embedded" : {
"dataplugin" : {
"id" : "f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"pluginType" : "EXTRACTOR",
"name" : "tap-matatika-sit",
"namespace" : "tap_matatika_sit",
"variant" : "matatika",
"label" : "Matatika SIT",
"description" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs",
"logoUrl" : "/assets/images/datasource/tap-matatika-sit.svg",
"hidden" : false,
"docs" : "https://meltano.com/plugins/extractors/spreadsheets-anywhere.html",
"pipUrl" : "git+https://github.com/ets/tap-spreadsheets-anywhere.git",
"repo" : "https://github.com/ets/tap-spreadsheets-anywhere",
"executable" : "tap-spreadsheets-anywhere",
"capabilities" : [ "DISCOVER", "CATALOG", "STATE" ],
"select" : [ ],
"update" : { },
"vars" : { },
"settings" : [ {
"name" : "tables",
"aliases" : [ ],
"label" : "Tables",
"value" : "[{\"path\":\"https://raw.githubusercontent.com/Matatika/matatika-examples/master/example_data\",\"name\":\"gitflixusers\",\"pattern\":\"GitFlixUsers.csv\",\"start_date\":\"2021-01-01T00:00:00Z\",\"key_properties\":[\"id\"],\"format\":\"csv\"}]",
"options" : [ ],
"kind" : "ARRAY",
"description" : "A setting in Matatika SIT that allows users to view and manage tables of data.",
"hidden" : false,
"sensitive" : false,
"protected" : false
} ],
"variants" : [ ],
"commands" : { },
"matatikaHidden" : true,
"requires" : [ ],
"fullDescription" : "Test extractor based on tap-spreadsheets-anywhere used during Matatika SIT runs\n\n## Settings\n\n\n### Tables\n\nA setting in Matatika SIT that allows users to view and manage tables of data.",
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994"
},
"update dataplugin" : {
"href" : "https://app.matatika.com/api/workspaces/2edda86a-e24c-4b32-bcac-bd66c73dedb9/dataplugins/f70f6b69-a44a-43db-a5c6-4cf0511b2994",
"type" : "PUT"
}
}
}
},
"managed" : false,
"draft" : false,
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"update datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
},
"delete datacomponent" : {
"href" : "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
}
}
}
Delete a datacomponent
DELETE
/api/datacomponents/{datacomponent-id}
Deletes the datacomponent {datacomponent-id}.
Request
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5' -i -X DELETE \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests)
import requests
url = "https://app.matatika.com/api/datacomponents/431e8d4b-4c7a-4d4a-bcce-c5f9ff4427c5"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("DELETE", url, headers=headers)
print(response.text.encode('utf8'))
Response
204 No Content
No response body provided.