Jobs
A job is an arbitrary task with some stored state, pertaining to the governing workspace. Typically, jobs are orchestrated by pipeline operations, but can also represent tasks for the user to complete.
Objects
Job
Path | Type | Format | Description |
---|---|---|---|
id |
String |
Version 4 UUID | The job ID |
created |
String |
ISO 8601 timestamp | The instant at which the job was created |
type |
String |
Job Type | The descriptor for the process undertaken by the job |
exitCode |
Integer |
Process exit status | The job exit code |
status |
String |
Job Status | The job status |
startTime |
String |
ISO 8601 timestamp | The instant at which the job run started |
endTime |
String |
ISO 8601 timestamp | The instant at which the job run ended |
{
"id" : "bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"created" : "2024-10-24T15:10:06.223303",
"type" : "PIPELINE_RUN",
"maxAttempts" : 0,
"attempt" : 0,
"commitId" : "1e81ff378befa5bdc8a517bbbb77f1c897f84f00",
"exitCode" : 0,
"status" : "COMPLETE",
"startTime" : "2024-10-24T15:10:17.537",
"endTime" : "2024-10-24T15:10:50.218",
"_embedded" : {
"pipeline" : {
"id" : "2600d33f-21ac-4001-9bd6-d81b05f26ed9",
"status" : "READY",
"name" : "SIT-Generated Pipeline [2024-10-24T16:07:59.292658]",
"timeout" : 0,
"maxRetries" : 0,
"created" : "2024-10-24T15:07:59.71135",
"lastModified" : "2024-10-24T15:07:59.711351",
"properties" : { },
"dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
"actions" : [ ],
"triggeredBy" : [ ],
"repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-10-24T16:07:59.292658].yml"
},
"profile" : {
"id" : "auth0|5eb0327cbfd7490bff55feeb",
"name" : "[email protected]",
"handle" : "@sit+prod",
"email" : "[email protected]"
}
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020"
},
"delete job" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"type" : "DELETE"
},
"logs" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020/logs?sequence=0",
"type" : "GET"
}
}
}
Formats
Job Status
String
Value | Description |
---|---|
QUEUED |
The job is queued |
RUNNING |
The job is running |
COMPLETE |
The job finished with no errors |
ERROR |
The job finished with errors |
STOPPED |
The job timed out or was manually stopped |
Job Type
String
Value | Description |
---|---|
WORKSPACE_INIT |
A system task to create a Meltano project in a workspace repository - automatically run when a workspace is created |
PIPELINE_CONFIG |
A system task to configure the Meltano project and publish datasets with reference to a pipeline - automatically run when a pipeline is created, or a pipeline with a status of FAILED is updated |
PIPELINE_VERFIY |
A system task to isplay and test the configuration of a pipeline |
PIPELINE_RUN |
A system task to run a pipeline to load data into the workspace default datastore, or some other destination external to the platform - manually run by the user or automatically run on the defined schedule |
PROFILE_COLLABORATE |
A user task to send an invitation |
PROFILE_IMPORT |
A user task to create a pipeline |
Requests
- View all running or completed jobs for a workspace
- View all running or completed jobs for a pipeline
- View a job
- View the logs of a job
- Create a job from a pipeline
- Stop a job
- Delete a job
View all running or completed jobs for a workspace
GET
/api/workspaces/{workspace-id}/jobs
Returns all running or completed jobs for the workspace {workspace-id}
.
Prerequisites
- Workspace
{workspace-id}
must exist
Request
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/workspaces/7dd2d604-60c3-44aa-8534-ee26daf05e22/jobs' -i -X GET \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests
)
import requests
url = "https://app.matatika.com/api/workspaces/7dd2d604-60c3-44aa-8534-ee26daf05e22/jobs"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Job collection with HAL links.
{
"_embedded" : {
"jobs" : [ {
"id" : "d695f559-18fd-482c-8892-47b0f70ac5fa",
"created" : "2024-10-24T15:07:37.220812",
"type" : "PROFILE_IMPORT",
"maxAttempts" : 0,
"attempt" : 0,
"status" : "QUEUED",
"_embedded" : {
"profile" : {
"id" : "auth0|5eb0327cbfd7490bff55feeb",
"name" : "[email protected]",
"handle" : "@sit+prod",
"email" : "[email protected]"
}
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/jobs/d695f559-18fd-482c-8892-47b0f70ac5fa"
},
"delete job" : {
"href" : "https://app.matatika.com/api/jobs/d695f559-18fd-482c-8892-47b0f70ac5fa",
"type" : "DELETE"
},
"logs" : {
"href" : "https://app.matatika.com/api/jobs/d695f559-18fd-482c-8892-47b0f70ac5fa/logs?sequence=0",
"type" : "GET"
},
"withdraw job" : {
"href" : "https://app.matatika.com/api/jobs/d695f559-18fd-482c-8892-47b0f70ac5fa/stopped",
"type" : "PUT"
}
}
}, {
"id" : "bc0a51c2-f57a-416a-a722-04991a1b6d54",
"created" : "2024-10-24T15:07:37.221972",
"type" : "PROFILE_COLLABORATE",
"maxAttempts" : 0,
"attempt" : 0,
"status" : "QUEUED",
"_embedded" : {
"profile" : {
"id" : "auth0|5eb0327cbfd7490bff55feeb",
"name" : "[email protected]",
"handle" : "@sit+prod",
"email" : "[email protected]"
}
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/jobs/bc0a51c2-f57a-416a-a722-04991a1b6d54"
},
"delete job" : {
"href" : "https://app.matatika.com/api/jobs/bc0a51c2-f57a-416a-a722-04991a1b6d54",
"type" : "DELETE"
},
"logs" : {
"href" : "https://app.matatika.com/api/jobs/bc0a51c2-f57a-416a-a722-04991a1b6d54/logs?sequence=0",
"type" : "GET"
},
"withdraw job" : {
"href" : "https://app.matatika.com/api/jobs/bc0a51c2-f57a-416a-a722-04991a1b6d54/stopped",
"type" : "PUT"
}
}
}, {
"id" : "1885cfe7-634d-4735-90f6-3f2635163f1b",
"created" : "2024-10-24T15:07:38.991494",
"type" : "WORKSPACE_INIT",
"maxAttempts" : 0,
"attempt" : 0,
"commitId" : "bb7967868cba66c8e8fa978893be7121000cd0e2",
"exitCode" : 0,
"status" : "COMPLETE",
"startTime" : "2024-10-24T15:07:49.865",
"endTime" : "2024-10-24T15:07:53.862",
"_embedded" : {
"profile" : {
"id" : "auth0|5eb0327cbfd7490bff55feeb",
"name" : "[email protected]",
"handle" : "@sit+prod",
"email" : "[email protected]"
}
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/jobs/1885cfe7-634d-4735-90f6-3f2635163f1b"
},
"delete job" : {
"href" : "https://app.matatika.com/api/jobs/1885cfe7-634d-4735-90f6-3f2635163f1b",
"type" : "DELETE"
},
"logs" : {
"href" : "https://app.matatika.com/api/jobs/1885cfe7-634d-4735-90f6-3f2635163f1b/logs?sequence=0",
"type" : "GET"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/workspaces/7dd2d604-60c3-44aa-8534-ee26daf05e22/jobs?page=0&size=20"
}
},
"page" : {
"size" : 20,
"totalElements" : 3,
"totalPages" : 1,
"number" : 0
}
}
View all running or completed jobs for a pipeline
GET
/api/pipelines/{pipeline-id}/jobs
Returns all running or completed jobs for the pipeline {pipeline-id}
.
Prerequisites
- Pipeline
{pipeline-id}
must exist
Request
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/pipelines/2600d33f-21ac-4001-9bd6-d81b05f26ed9/jobs' -i -X GET \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests
)
import requests
url = "https://app.matatika.com/api/pipelines/2600d33f-21ac-4001-9bd6-d81b05f26ed9/jobs"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Job collection with HAL links.
{
"_embedded" : {
"jobs" : [ {
"id" : "bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"created" : "2024-10-24T15:10:06.223303",
"type" : "PIPELINE_RUN",
"maxAttempts" : 0,
"attempt" : 0,
"status" : "QUEUED",
"_embedded" : {
"pipeline" : {
"id" : "2600d33f-21ac-4001-9bd6-d81b05f26ed9",
"status" : "READY",
"name" : "SIT-Generated Pipeline [2024-10-24T16:07:59.292658]",
"timeout" : 0,
"maxRetries" : 0,
"created" : "2024-10-24T15:07:59.71135",
"lastModified" : "2024-10-24T15:07:59.711351",
"properties" : { },
"dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
"actions" : [ ],
"triggeredBy" : [ ],
"repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-10-24T16:07:59.292658].yml"
},
"profile" : {
"id" : "auth0|5eb0327cbfd7490bff55feeb",
"name" : "[email protected]",
"handle" : "@sit+prod",
"email" : "[email protected]"
}
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020"
},
"delete job" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"type" : "DELETE"
},
"logs" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020/logs?sequence=0",
"type" : "GET"
},
"withdraw job" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020/stopped",
"type" : "PUT"
}
}
}, {
"id" : "69826a49-5b2d-4c3a-9d06-61983cd4f9cd",
"created" : "2024-10-24T15:09:23.559991",
"type" : "PIPELINE_VERIFY",
"maxAttempts" : 0,
"attempt" : 0,
"commitId" : "1e81ff378befa5bdc8a517bbbb77f1c897f84f00",
"exitCode" : 0,
"status" : "COMPLETE",
"startTime" : "2024-10-24T15:09:37.545",
"endTime" : "2024-10-24T15:10:02.932",
"_embedded" : {
"pipeline" : {
"id" : "2600d33f-21ac-4001-9bd6-d81b05f26ed9",
"status" : "READY",
"name" : "SIT-Generated Pipeline [2024-10-24T16:07:59.292658]",
"timeout" : 0,
"maxRetries" : 0,
"created" : "2024-10-24T15:07:59.71135",
"lastModified" : "2024-10-24T15:07:59.711351",
"properties" : { },
"dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
"actions" : [ ],
"triggeredBy" : [ ],
"repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-10-24T16:07:59.292658].yml"
},
"profile" : {
"id" : "auth0|5eb0327cbfd7490bff55feeb",
"name" : "[email protected]",
"handle" : "@sit+prod",
"email" : "[email protected]"
}
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/jobs/69826a49-5b2d-4c3a-9d06-61983cd4f9cd"
},
"delete job" : {
"href" : "https://app.matatika.com/api/jobs/69826a49-5b2d-4c3a-9d06-61983cd4f9cd",
"type" : "DELETE"
},
"logs" : {
"href" : "https://app.matatika.com/api/jobs/69826a49-5b2d-4c3a-9d06-61983cd4f9cd/logs?sequence=0",
"type" : "GET"
}
}
}, {
"id" : "2c832db5-5849-4a5d-b7f8-09150b06eafa",
"created" : "2024-10-24T15:08:00.414087",
"type" : "WORKSPACE_CONFIG",
"maxAttempts" : 0,
"attempt" : 0,
"commitId" : "6b8ff84700dec255d9f815fe9b33470e5cb780e7",
"exitCode" : 0,
"status" : "COMPLETE",
"startTime" : "2024-10-24T15:08:11.745",
"endTime" : "2024-10-24T15:09:18.911",
"_embedded" : {
"pipeline" : {
"id" : "2600d33f-21ac-4001-9bd6-d81b05f26ed9",
"status" : "READY",
"name" : "SIT-Generated Pipeline [2024-10-24T16:07:59.292658]",
"timeout" : 0,
"maxRetries" : 0,
"created" : "2024-10-24T15:07:59.71135",
"lastModified" : "2024-10-24T15:07:59.711351",
"properties" : { },
"dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
"actions" : [ ],
"triggeredBy" : [ ],
"repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-10-24T16:07:59.292658].yml"
},
"profile" : {
"id" : "auth0|5eb0327cbfd7490bff55feeb",
"name" : "[email protected]",
"handle" : "@sit+prod",
"email" : "[email protected]"
}
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/jobs/2c832db5-5849-4a5d-b7f8-09150b06eafa"
},
"delete job" : {
"href" : "https://app.matatika.com/api/jobs/2c832db5-5849-4a5d-b7f8-09150b06eafa",
"type" : "DELETE"
},
"logs" : {
"href" : "https://app.matatika.com/api/jobs/2c832db5-5849-4a5d-b7f8-09150b06eafa/logs?sequence=0",
"type" : "GET"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/pipelines/2600d33f-21ac-4001-9bd6-d81b05f26ed9/jobs?page=0&size=20"
}
},
"page" : {
"size" : 20,
"totalElements" : 3,
"totalPages" : 1,
"number" : 0
}
}
View a job
GET
/api/jobs/{job-id}
Returns the job {job-id}
.
Prerequisites
- Job
{job-id}
must exist
Request
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020' -i -X GET \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests
)
import requests
url = "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Job with HAL links.
{
"id" : "bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"created" : "2024-10-24T15:10:06.223303",
"type" : "PIPELINE_RUN",
"maxAttempts" : 0,
"attempt" : 0,
"commitId" : "1e81ff378befa5bdc8a517bbbb77f1c897f84f00",
"exitCode" : 0,
"status" : "COMPLETE",
"startTime" : "2024-10-24T15:10:17.537",
"endTime" : "2024-10-24T15:10:50.218",
"_embedded" : {
"pipeline" : {
"id" : "2600d33f-21ac-4001-9bd6-d81b05f26ed9",
"status" : "READY",
"name" : "SIT-Generated Pipeline [2024-10-24T16:07:59.292658]",
"timeout" : 0,
"maxRetries" : 0,
"created" : "2024-10-24T15:07:59.71135",
"lastModified" : "2024-10-24T15:07:59.711351",
"properties" : { },
"dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
"actions" : [ ],
"triggeredBy" : [ ],
"repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-10-24T16:07:59.292658].yml"
},
"profile" : {
"id" : "auth0|5eb0327cbfd7490bff55feeb",
"name" : "[email protected]",
"handle" : "@sit+prod",
"email" : "[email protected]"
}
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020"
},
"delete job" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"type" : "DELETE"
},
"logs" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020/logs?sequence=0",
"type" : "GET"
}
}
}
View the logs of a job
GET
/api/jobs/{job-id}/logs?sequence={sequence}
Returns the logs of the job {job-id}
.
Prerequisites
- Job
{job-id}
must exist
Request
Query Parameters
Query Parameter | Format | Default Value | Description |
---|---|---|---|
sequence |
Unsigned integer | 0 |
The line number in the logs to read from |
Headers
Accept
Media Type(s) | Description |
---|---|
text/plain */* |
Sets the response content type format to plain text |
application/stream+json application/x-ndjson |
Sets the response content type format to NDJSON |
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020/logs?sequence=0' -i -X GET \
-H 'Accept: application/x-ndjson' \
-H 'Content-Type: application/json'
Python (requests
)
import requests
url = "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020/logs?sequence=0"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200
: The job logs in the format specified by associated requestAccept
header
{
"jobId" : "bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"level" : "INFO",
"message" : "Begin: bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"sequence" : 1,
"timestamp" : "2024-10-24T15:10:18.065Z"
}
204
: No response body provided.
Create a job from a pipeline
POST
/api/pipelines/{pipeline-id}/jobs
Creates a new job from the pipeline {pipeline-id}
.
Prerequisites
- Pipeline
{pipeline-id}
must exist and not already be running
Request
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/pipelines/2600d33f-21ac-4001-9bd6-d81b05f26ed9/jobs' -i -X POST \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests
)
import requests
url = "https://app.matatika.com/api/pipelines/2600d33f-21ac-4001-9bd6-d81b05f26ed9/jobs"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("POST", url, headers=headers)
print(response.text.encode('utf8'))
Response
201 Created
Job with HAL links.
{
"id" : "bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"created" : "2024-10-24T15:10:06.223303",
"type" : "PIPELINE_RUN",
"maxAttempts" : 0,
"attempt" : 0,
"status" : "QUEUED",
"_embedded" : {
"pipeline" : {
"id" : "2600d33f-21ac-4001-9bd6-d81b05f26ed9",
"status" : "READY",
"name" : "SIT-Generated Pipeline [2024-10-24T16:07:59.292658]",
"timeout" : 0,
"maxRetries" : 0,
"created" : "2024-10-24T15:07:59.71135",
"lastModified" : "2024-10-24T15:07:59.711351",
"properties" : { },
"dataComponents" : [ "tap-matatika-sit", "Warehouse", "dbt" ],
"actions" : [ ],
"triggeredBy" : [ ],
"repositoryPath" : "pipelines/SIT-Generated Pipeline [2024-10-24T16:07:59.292658].yml"
},
"profile" : {
"id" : "auth0|5eb0327cbfd7490bff55feeb",
"name" : "[email protected]",
"handle" : "@sit+prod",
"email" : "[email protected]"
}
},
"_links" : {
"self" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020"
},
"delete job" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020",
"type" : "DELETE"
},
"logs" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020/logs?sequence=0",
"type" : "GET"
},
"withdraw job" : {
"href" : "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020/stopped",
"type" : "PUT"
}
}
}
Stop a job
PUT
/api/jobs/{job-id}/stopped
Stops the execution of the job {job-id}
.
Prerequisites
- Job
{job-id}
must exist - Job
{job-id}
must have statusRUNNING
Request
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/jobs/f994edaa-df14-4e54-88a5-b95d4e14974d/stopped' -i -X PUT \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests
)
import requests
url = "https://app.matatika.com/api/jobs/f994edaa-df14-4e54-88a5-b95d4e14974d/stopped"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("PUT", url, headers=headers)
print(response.text.encode('utf8'))
Response
202 Accepted
Job stop acceptance message.
{
"message" : "Job stop requested"
}
Delete a job
DELETE
/api/jobs/{job-id}
Deletes and stops the execution of the job {job-id}
.
Prerequisites
- Job
{job-id}
must exist
Example Snippets
cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020' -i -X DELETE \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
Python (requests
)
import requests
url = "https://app.matatika.com/api/jobs/bc78a5fe-3693-43ea-ac56-0a6afcef7020"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("DELETE", url, headers=headers)
print(response.text.encode('utf8'))
Response
204 No Content
No response body provided.