Jobs

A job is an arbitrary task with some stored state, pertaining to the governing workspace. Typically, jobs are orchestrated by pipeline operations, but can also represent tasks for the user to complete.

Objects

Job

Path	Type	Format	Description
`id`	`String`	Version 4 UUID	The job ID
`created`	`String`	ISO 8601 timestamp	The instant at which the job was created
`type`	`String`	Job Type	The descriptor for the process undertaken by the job
`exitCode`	`Integer`	Process exit status	The job exit code
`status`	`String`	Job Status	The job status
`startTime`	`String`	ISO 8601 timestamp	The instant at which the job run started
`endTime`	`String`	ISO 8601 timestamp	The instant at which the job run ended

{
  "id" : "daac0008-7394-4d2b-9326-a58b9744da4a",
  "created" : "2024-12-16T09:56:50.965667",
  "type" : "PIPELINE_RUN",
  "maxAttempts" : 0,
  "attempt" : 0,
  "commitId" : "356272a0d4450afbc4f337d62e562b07178c7c3b",
  "exitCode" : 0,
  "status" : "COMPLETE",
  "startTime" : "2024-12-16T09:57:04.009",
  "endTime" : "2024-12-16T09:57:43.129",
  "_embedded" : {
    "pipeline" : {
      "id" : "2180a2e8-f75f-4319-8a91-f47046535967",
      "status" : "READY",
      "name" : "SIT-Generated Pipeline [2024-12-16T09:54:29.465456]",
      "timeout" : 0,
      "maxRetries" : 0,
      "created" : "2024-12-16T09:54:29.753328",
      "lastModified" : "2024-12-16T09:54:29.753328",
      "properties" : { },
      "dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
      "actions" : [ ],
      "triggeredBy" : [ ],
      "repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-12-16T09:54:29.465456].yml"
    },
    "profile" : {
      "id" : "auth0|5eb0327cbfd7490bff55feeb",
      "name" : "[email protected]",
      "handle" : "@sit+prod",
      "email" : "[email protected]"
    }
  },
  "_links" : {
    "self" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a"
    },
    "delete job" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a",
      "type" : "DELETE"
    },
    "logs" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a/logs?sequence=0",
      "type" : "GET"
    }
  }
}

Formats

Job Status

String

Value	Description
`QUEUED`	The job is queued
`RUNNING`	The job is running
`COMPLETE`	The job finished with no errors
`ERROR`	The job finished with errors
`STOPPED`	The job timed out or was manually stopped

Job Type

String

Value	Description
`WORKSPACE_INIT`	A system task to create a Meltano project in a workspace repository - automatically run when a workspace is created
`PIPELINE_CONFIG`	A system task to configure the Meltano project and publish datasets with reference to a pipeline - automatically run when a pipeline is created, or a pipeline with a `status` of `FAILED` is updated
`PIPELINE_VERFIY`	A system task to isplay and test the configuration of a pipeline
`PIPELINE_RUN`	A system task to run a pipeline to load data into the workspace default datastore, or some other destination external to the platform - manually run by the user or automatically run on the defined `schedule`
`PROFILE_COLLABORATE`	A user task to send an invitation
`PROFILE_IMPORT`	A user task to create a pipeline

Requests

View all running or completed jobs for a workspace
View all running or completed jobs for a pipeline
View a job
View the logs of a job
Create a job from a pipeline
Stop a job
Delete a job

View all running or completed jobs for a workspace

GET

/api/workspaces/{workspace-id}/jobs

Returns all running or completed jobs for the workspace {workspace-id}.

Prerequisites

Workspace {workspace-id} must exist

Request

Example Snippets

cURL

curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/workspaces/c7f80322-55d0-4c00-bdea-d12dabadd39b/jobs' -i -X GET \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'

Python (requests)

import requests

url = "https://app.matatika.com/api/workspaces/c7f80322-55d0-4c00-bdea-d12dabadd39b/jobs"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("GET", url, headers=headers)

print(response.text.encode('utf8'))

Response

200 OK

Job collection with HAL links.

{
  "_embedded" : {
    "jobs" : [ {
      "id" : "ce8b52dd-f864-4dd4-b9d6-e69ba138c064",
      "created" : "2024-12-16T09:54:02.232675",
      "type" : "PROFILE_IMPORT",
      "maxAttempts" : 0,
      "attempt" : 0,
      "status" : "QUEUED",
      "_embedded" : {
        "profile" : {
          "id" : "auth0|5eb0327cbfd7490bff55feeb",
          "name" : "[email protected]",
          "handle" : "@sit+prod",
          "email" : "[email protected]"
        }
      },
      "_links" : {
        "self" : {
          "href" : "https://app.matatika.com/api/jobs/ce8b52dd-f864-4dd4-b9d6-e69ba138c064"
        },
        "delete job" : {
          "href" : "https://app.matatika.com/api/jobs/ce8b52dd-f864-4dd4-b9d6-e69ba138c064",
          "type" : "DELETE"
        },
        "logs" : {
          "href" : "https://app.matatika.com/api/jobs/ce8b52dd-f864-4dd4-b9d6-e69ba138c064/logs?sequence=0",
          "type" : "GET"
        },
        "withdraw job" : {
          "href" : "https://app.matatika.com/api/jobs/ce8b52dd-f864-4dd4-b9d6-e69ba138c064/stopped",
          "type" : "PUT"
        }
      }
    }, {
      "id" : "05037764-c49d-41d9-b4ef-aefcfd2f4000",
      "created" : "2024-12-16T09:54:02.233503",
      "type" : "PROFILE_COLLABORATE",
      "maxAttempts" : 0,
      "attempt" : 0,
      "status" : "QUEUED",
      "_embedded" : {
        "profile" : {
          "id" : "auth0|5eb0327cbfd7490bff55feeb",
          "name" : "[email protected]",
          "handle" : "@sit+prod",
          "email" : "[email protected]"
        }
      },
      "_links" : {
        "self" : {
          "href" : "https://app.matatika.com/api/jobs/05037764-c49d-41d9-b4ef-aefcfd2f4000"
        },
        "delete job" : {
          "href" : "https://app.matatika.com/api/jobs/05037764-c49d-41d9-b4ef-aefcfd2f4000",
          "type" : "DELETE"
        },
        "logs" : {
          "href" : "https://app.matatika.com/api/jobs/05037764-c49d-41d9-b4ef-aefcfd2f4000/logs?sequence=0",
          "type" : "GET"
        },
        "withdraw job" : {
          "href" : "https://app.matatika.com/api/jobs/05037764-c49d-41d9-b4ef-aefcfd2f4000/stopped",
          "type" : "PUT"
        }
      }
    }, {
      "id" : "e97edf4f-341c-4780-a5e1-c2edbd863bfc",
      "created" : "2024-12-16T09:54:07.08463",
      "type" : "WORKSPACE_INIT",
      "maxAttempts" : 0,
      "attempt" : 0,
      "commitId" : "3594e83a59d8143ba8a7257c507e5a4e034df4c9",
      "exitCode" : 0,
      "status" : "COMPLETE",
      "startTime" : "2024-12-16T09:54:21.092",
      "endTime" : "2024-12-16T09:54:25.749",
      "_embedded" : {
        "profile" : {
          "id" : "auth0|5eb0327cbfd7490bff55feeb",
          "name" : "[email protected]",
          "handle" : "@sit+prod",
          "email" : "[email protected]"
        }
      },
      "_links" : {
        "self" : {
          "href" : "https://app.matatika.com/api/jobs/e97edf4f-341c-4780-a5e1-c2edbd863bfc"
        },
        "delete job" : {
          "href" : "https://app.matatika.com/api/jobs/e97edf4f-341c-4780-a5e1-c2edbd863bfc",
          "type" : "DELETE"
        },
        "logs" : {
          "href" : "https://app.matatika.com/api/jobs/e97edf4f-341c-4780-a5e1-c2edbd863bfc/logs?sequence=0",
          "type" : "GET"
        }
      }
    } ]
  },
  "_links" : {
    "self" : {
      "href" : "https://app.matatika.com/api/workspaces/c7f80322-55d0-4c00-bdea-d12dabadd39b/jobs?page=0&size=20"
    }
  },
  "page" : {
    "size" : 20,
    "totalElements" : 3,
    "totalPages" : 1,
    "number" : 0
  }
}

View all running or completed jobs for a pipeline

GET

/api/pipelines/{pipeline-id}/jobs

Returns all running or completed jobs for the pipeline {pipeline-id}.

Prerequisites

Pipeline {pipeline-id} must exist

Request

Example Snippets

cURL

curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/pipelines/2180a2e8-f75f-4319-8a91-f47046535967/jobs' -i -X GET \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'

Python (requests)

import requests

url = "https://app.matatika.com/api/pipelines/2180a2e8-f75f-4319-8a91-f47046535967/jobs"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("GET", url, headers=headers)

print(response.text.encode('utf8'))

Response

200 OK

Job collection with HAL links.

{
  "_embedded" : {
    "jobs" : [ {
      "id" : "daac0008-7394-4d2b-9326-a58b9744da4a",
      "created" : "2024-12-16T09:56:50.965667",
      "type" : "PIPELINE_RUN",
      "maxAttempts" : 0,
      "attempt" : 0,
      "status" : "QUEUED",
      "_embedded" : {
        "pipeline" : {
          "id" : "2180a2e8-f75f-4319-8a91-f47046535967",
          "status" : "READY",
          "name" : "SIT-Generated Pipeline [2024-12-16T09:54:29.465456]",
          "timeout" : 0,
          "maxRetries" : 0,
          "created" : "2024-12-16T09:54:29.753328",
          "lastModified" : "2024-12-16T09:54:29.753328",
          "properties" : { },
          "dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
          "actions" : [ ],
          "triggeredBy" : [ ],
          "repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-12-16T09:54:29.465456].yml"
        },
        "profile" : {
          "id" : "auth0|5eb0327cbfd7490bff55feeb",
          "name" : "[email protected]",
          "handle" : "@sit+prod",
          "email" : "[email protected]"
        }
      },
      "_links" : {
        "self" : {
          "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a"
        },
        "delete job" : {
          "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a",
          "type" : "DELETE"
        },
        "logs" : {
          "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a/logs?sequence=0",
          "type" : "GET"
        },
        "withdraw job" : {
          "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a/stopped",
          "type" : "PUT"
        }
      }
    }, {
      "id" : "886c8e69-121b-4c4d-a275-62a1b942f01d",
      "created" : "2024-12-16T09:56:08.021023",
      "type" : "PIPELINE_VERIFY",
      "maxAttempts" : 0,
      "attempt" : 0,
      "commitId" : "356272a0d4450afbc4f337d62e562b07178c7c3b",
      "exitCode" : 0,
      "status" : "COMPLETE",
      "startTime" : "2024-12-16T09:56:21.059",
      "endTime" : "2024-12-16T09:56:50.204",
      "_embedded" : {
        "pipeline" : {
          "id" : "2180a2e8-f75f-4319-8a91-f47046535967",
          "status" : "READY",
          "name" : "SIT-Generated Pipeline [2024-12-16T09:54:29.465456]",
          "timeout" : 0,
          "maxRetries" : 0,
          "created" : "2024-12-16T09:54:29.753328",
          "lastModified" : "2024-12-16T09:54:29.753328",
          "properties" : { },
          "dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
          "actions" : [ ],
          "triggeredBy" : [ ],
          "repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-12-16T09:54:29.465456].yml"
        },
        "profile" : {
          "id" : "auth0|5eb0327cbfd7490bff55feeb",
          "name" : "[email protected]",
          "handle" : "@sit+prod",
          "email" : "[email protected]"
        }
      },
      "_links" : {
        "self" : {
          "href" : "https://app.matatika.com/api/jobs/886c8e69-121b-4c4d-a275-62a1b942f01d"
        },
        "delete job" : {
          "href" : "https://app.matatika.com/api/jobs/886c8e69-121b-4c4d-a275-62a1b942f01d",
          "type" : "DELETE"
        },
        "logs" : {
          "href" : "https://app.matatika.com/api/jobs/886c8e69-121b-4c4d-a275-62a1b942f01d/logs?sequence=0",
          "type" : "GET"
        }
      }
    }, {
      "id" : "d1045777-e8e8-4350-b57b-94f7919c8021",
      "created" : "2024-12-16T09:54:29.998777",
      "type" : "WORKSPACE_CONFIG",
      "maxAttempts" : 0,
      "attempt" : 0,
      "commitId" : "2182027d73406f993ca18fa02fc4af998e4a2684",
      "exitCode" : 0,
      "status" : "COMPLETE",
      "startTime" : "2024-12-16T09:54:43.771",
      "endTime" : "2024-12-16T09:56:06.786",
      "_embedded" : {
        "pipeline" : {
          "id" : "2180a2e8-f75f-4319-8a91-f47046535967",
          "status" : "READY",
          "name" : "SIT-Generated Pipeline [2024-12-16T09:54:29.465456]",
          "timeout" : 0,
          "maxRetries" : 0,
          "created" : "2024-12-16T09:54:29.753328",
          "lastModified" : "2024-12-16T09:54:29.753328",
          "properties" : { },
          "dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
          "actions" : [ ],
          "triggeredBy" : [ ],
          "repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-12-16T09:54:29.465456].yml"
        },
        "profile" : {
          "id" : "auth0|5eb0327cbfd7490bff55feeb",
          "name" : "[email protected]",
          "handle" : "@sit+prod",
          "email" : "[email protected]"
        }
      },
      "_links" : {
        "self" : {
          "href" : "https://app.matatika.com/api/jobs/d1045777-e8e8-4350-b57b-94f7919c8021"
        },
        "delete job" : {
          "href" : "https://app.matatika.com/api/jobs/d1045777-e8e8-4350-b57b-94f7919c8021",
          "type" : "DELETE"
        },
        "logs" : {
          "href" : "https://app.matatika.com/api/jobs/d1045777-e8e8-4350-b57b-94f7919c8021/logs?sequence=0",
          "type" : "GET"
        }
      }
    } ]
  },
  "_links" : {
    "self" : {
      "href" : "https://app.matatika.com/api/pipelines/2180a2e8-f75f-4319-8a91-f47046535967/jobs?page=0&size=20"
    }
  },
  "page" : {
    "size" : 20,
    "totalElements" : 3,
    "totalPages" : 1,
    "number" : 0
  }
}

View a job

GET

/api/jobs/{job-id}

Returns the job {job-id}.

Prerequisites

Job {job-id} must exist

Request

Example Snippets

cURL

curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a' -i -X GET \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'

Python (requests)

import requests

url = "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("GET", url, headers=headers)

print(response.text.encode('utf8'))

Response

200 OK

Job with HAL links.

{
  "id" : "daac0008-7394-4d2b-9326-a58b9744da4a",
  "created" : "2024-12-16T09:56:50.965667",
  "type" : "PIPELINE_RUN",
  "maxAttempts" : 0,
  "attempt" : 0,
  "commitId" : "356272a0d4450afbc4f337d62e562b07178c7c3b",
  "exitCode" : 0,
  "status" : "COMPLETE",
  "startTime" : "2024-12-16T09:57:04.009",
  "endTime" : "2024-12-16T09:57:43.129",
  "_embedded" : {
    "pipeline" : {
      "id" : "2180a2e8-f75f-4319-8a91-f47046535967",
      "status" : "READY",
      "name" : "SIT-Generated Pipeline [2024-12-16T09:54:29.465456]",
      "timeout" : 0,
      "maxRetries" : 0,
      "created" : "2024-12-16T09:54:29.753328",
      "lastModified" : "2024-12-16T09:54:29.753328",
      "properties" : { },
      "dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
      "actions" : [ ],
      "triggeredBy" : [ ],
      "repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-12-16T09:54:29.465456].yml"
    },
    "profile" : {
      "id" : "auth0|5eb0327cbfd7490bff55feeb",
      "name" : "[email protected]",
      "handle" : "@sit+prod",
      "email" : "[email protected]"
    }
  },
  "_links" : {
    "self" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a"
    },
    "delete job" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a",
      "type" : "DELETE"
    },
    "logs" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a/logs?sequence=0",
      "type" : "GET"
    }
  }
}

View the logs of a job

GET

/api/jobs/{job-id}/logs?sequence={sequence}

Returns the logs of the job {job-id}.

Prerequisites

Job {job-id} must exist

Request

Query Parameters

Query Parameter	Format	Default Value	Description
`sequence`	Unsigned integer	`0`	The line number in the logs to read from

Headers

Accept

Media Type(s)	Description
`text/plain` `/`	Sets the response content type format to plain text
`application/stream+json` `application/x-ndjson`	Sets the response content type format to NDJSON

Example Snippets

cURL

curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a/logs?sequence=0' -i -X GET \
    -H 'Accept: application/x-ndjson' \
    -H 'Content-Type: application/json'

Python (requests)

import requests

url = "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a/logs?sequence=0"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("GET", url, headers=headers)

print(response.text.encode('utf8'))

Response

200: The job logs in the format specified by associated request Accept header

{
  "jobId" : "daac0008-7394-4d2b-9326-a58b9744da4a",
  "level" : "INFO",
  "message" : "Begin: daac0008-7394-4d2b-9326-a58b9744da4a",
  "sequence" : 0,
  "timestamp" : "2024-12-16T09:56:50.965Z"
}

204: No response body provided.

Create a job from a pipeline

POST

/api/pipelines/{pipeline-id}/jobs

Creates a new job from the pipeline {pipeline-id}.

Prerequisites

Pipeline {pipeline-id} must exist and not already be running

Request

Example Snippets

cURL

curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/pipelines/2180a2e8-f75f-4319-8a91-f47046535967/jobs' -i -X POST \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'

Python (requests)

import requests

url = "https://app.matatika.com/api/pipelines/2180a2e8-f75f-4319-8a91-f47046535967/jobs"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("POST", url, headers=headers)

print(response.text.encode('utf8'))

Response

201 Created

Job with HAL links.

{
  "id" : "daac0008-7394-4d2b-9326-a58b9744da4a",
  "created" : "2024-12-16T09:56:50.965667",
  "type" : "PIPELINE_RUN",
  "maxAttempts" : 0,
  "attempt" : 0,
  "status" : "QUEUED",
  "_embedded" : {
    "pipeline" : {
      "id" : "2180a2e8-f75f-4319-8a91-f47046535967",
      "status" : "READY",
      "name" : "SIT-Generated Pipeline [2024-12-16T09:54:29.465456]",
      "timeout" : 0,
      "maxRetries" : 0,
      "created" : "2024-12-16T09:54:29.753328",
      "lastModified" : "2024-12-16T09:54:29.753328",
      "properties" : { },
      "dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
      "actions" : [ ],
      "triggeredBy" : [ ],
      "repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-12-16T09:54:29.465456].yml"
    },
    "profile" : {
      "id" : "auth0|5eb0327cbfd7490bff55feeb",
      "name" : "[email protected]",
      "handle" : "@sit+prod",
      "email" : "[email protected]"
    }
  },
  "_links" : {
    "self" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a"
    },
    "delete job" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a",
      "type" : "DELETE"
    },
    "logs" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a/logs?sequence=0",
      "type" : "GET"
    },
    "withdraw job" : {
      "href" : "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a/stopped",
      "type" : "PUT"
    }
  }
}

Stop a job

PUT

/api/jobs/{job-id}/stopped

Stops the execution of the job {job-id}.

Prerequisites

Job {job-id} must exist
Job {job-id} must have status RUNNING

Request

Example Snippets

cURL

curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/jobs/6ae6afd4-87ea-4680-9908-c8adbacc2856/stopped' -i -X PUT \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'

Python (requests)

import requests

url = "https://app.matatika.com/api/jobs/6ae6afd4-87ea-4680-9908-c8adbacc2856/stopped"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("PUT", url, headers=headers)

print(response.text.encode('utf8'))

Response

202 Accepted

Job stop acceptance message.

{
  "message" : "Job stop requested"
}

Delete a job

DELETE

/api/jobs/{job-id}

Deletes and stops the execution of the job {job-id}.

Prerequisites

Job {job-id} must exist

Example Snippets

cURL

curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a' -i -X DELETE \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'

Python (requests)

import requests

url = "https://app.matatika.com/api/jobs/daac0008-7394-4d2b-9326-a58b9744da4a"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("DELETE", url, headers=headers)

print(response.text.encode('utf8'))

Response

204 No Content

No response body provided.