Executing workflows via API
Last updated
Was this helpful?
Last updated
Was this helpful?
A workflow can be executed via an API call, which allows merging workflows with bigger processes, running analytics pipelines from external applications and further integrations that enable a wide range of use cases.
CARTO Workflows translates the analytical pipelines designed in the canvas to cloud native SQL queries that are executed directly on the data warehouse. This generated SQL code has different forms:
The code displayed in the contains all the control structures and creation of intermediate tables for each node in the workflow.
A stored procedure that is created in , executed when a workflow execution is triggered via API call. This stored procedure might have input parameters that are used later in the code, that are created from the marked as 'Parameters'.
The ability of workflows to generate SQL code that can be executed via an API call represents a significant advancement in integrating complex analytical processes with external systems. This capability not only streamlines the process of executing workflows in response to external triggers but also opens up a myriad of possibilities for automation and scalability within data analysis projects.
To enable API access for an existing workflow, click on the three dots in the upper-right corner and find 'API'. Click on 'Enable API access' and you will see a dialog screen that looks like this:
This is an example of an API call that would trigger the execution of a workflow:
GET example:
POST example:
There are a few things that are worth taking into consideration:
The response to that call will look like the following:
You can use that externalId to make a request to check the status of the execution, like:
The above is just an example that is using a specific API URL (gcp-us-east1.api.carto.com) and connection name. Please make sure that you use the same API endpoint that you obtained from the UI and you used to create the job in the first place.
The response to that API call is a JSON that contains metadata about the query execution, including a status object with information about the execution status.
The following is an example of a failed execution due to incorrect parameter types:
While this would be an example of a successful execution:
The location of the temporary table is returned when the job is created, but the content of the table won't be complete until the execution is complete.
API executions of the same workflow, but with different queryParameters will produce different output tables. On the other hand, API executions with same queryParameters will yield the same result, as the same output table will be reused.
Whenever you want to propagate changes in your workflow to the corresponding stored procedure that is executed with an API call, you need to update it.
For that, just click on the chip in the top header that says 'API enabled', which will open the API endpoint modal. Wait for a couple of seconds while CARTO checks for changes in the workflow and you will see this:
Click on 'Update' to sync the workflow that is executed via API with the current state of the workflow in the UI.
If you need to prevent API access for a workflow, just click on 'Disable API access' in the API endpoint modal and confirm:
It is actually a GET or POST request to CARTO's , which can be used to create jobs that run queries asynchronously.
The query is a CALL statement for a stored procedure. Workflows will generate a stored procedure in your schema/dataset. The generated SQL is the same that you will obtain by using the .
The queryParameters object in the payload contains values for some variables that are passed as inputs to the stored procedure. These variables are the ones marked as 'Parameter' in the '' menu.
The provided API call is authorized with an that has the specific grants necessary to run that query through same connection that was used to create the workflow.
The provided example uses curl to illustrate the POST request that triggers the execution of the workflow. This example call should be easily adapted to other methods in different languages, like the library in Python.
Get more information about using the job endpoint in the .
In order to define which node of your workflow will be used as output of the API call, you need to use the component. This component will ensure that the content of the node connected to it is stored in a temporary table, which location is returned in the "workflowOutputTableName" object in the response of the API call.
This behavior can be controlled with the in your Workflows UI. Setting it off will always force re-execution of the workflow, even if consecutive API calls with same queryParameters are performed.
