For the complete documentation index, see llms.txt. This page is also available as Markdown.

lds

This module contains functions and procedures that make use of location data services, such as geocoding, reverse geocoding, isolines and routing computation.

For manual installations of the CARTO Analytics Toolbox, after installing it for the first time, and before using any LDS function you need to call the SETUP procedure to configure the LDS functions. It also optionally sets default credentials.

GEOCODE_TABLE

GEOCODE_TABLE(input_table, address_column [, geom_column] [, country] [, options] [, api_base_url, api_access_token])

Description

Geocodes an input table by adding an user defined column geom_column with the geographic coordinates (latitude and longitude) corresponding to a given address column. This procedure also adds a carto_geocode_metadata column with additional information of the geocoding result in JSON format. If the input table already contains a geometry column with the name geom_column, only those rows with NULL values in it will be geocoded.

The table is geocoded sequentially, one batch of rows at a time. The default batch size depends on the geocoding provider configured for your account: 200 rows with TomTom, which is processed through the provider's asynchronous batch API, and 75 rows with any other provider. Use the carto_batch_size option to override it.

Redshift batches are much smaller than the ones used on other warehouses because the provider request is assembled inside a VARCHAR value, and its 64 KB ceiling bounds how many addresses fit into a single call.

Input parameters

  • input_table: VARCHAR(MAX) name of the table to be geocoded. Please make sure you have enough permissions to alter this table, as this procedure will add two columns to it to store the geocoding result. It must have the form [schema_name].[table_name]. Notice that a table name expressed as [database_name].[schema_name].[table_name] is not valid, and the function will not handle it correctly.

  • address_column: VARCHAR(MAX) name of the column from the input table that contains the addresses to be geocoded.

  • geom_column (optional): VARCHAR(MAX) column name for the output geometry column. Defaults to 'geom'.

  • country (optional): VARCHAR(MAX) name of the country in ISO 3166-1 alpha-2. There is no default: when it is omitted or empty, an empty country is sent to the provider. See Always set the country below.

  • options (optional): VARCHAR(MAX) containing a valid JSON with the different options. In addition to the geocoding service options described in the table below, the following CARTO options are accepted:

    CARTO option
    Description

    carto_force_geocode

    a BOOL (false by default) that requests every row again, including rows that already have a non-null value in geom_column. It replaces values the caller already had, so it is the one option here that spends quota on rows that were already resolved.

    carto_force_rebuild

    a BOOL (false by default) that discards the partial table described in Resuming an interrupted job below and starts the job again. It never replaces a value already in the table, so only the rows still pending are requested.

    carto_batch_size

    an integer that overrides the number of rows sent to the provider in each batch. It must be 1 or greater — lower values fail with Option "carto_batch_size" must be a positive number. It defaults to the per-provider value described above (200 with TomTom, 75 otherwise). Lower it when a batch fails because the request is too large, which happens with unusually long addresses; a value above it is reduced to it and reported in the summary rather than rejected, which is also what keeps a batch inside the 64 KB VARCHAR ceiling mentioned above.

    carto_max_status_attempts

    an integer (120 by default) that bounds how many status checks each TomTom async batch gets before the run fails with did not complete. Each healthy check long-polls, so the default allows hours per batch; raise it only for extremely large batches on a slow provider tier.

    If no options are indicated then 'default' values would be applied.

    Provider
    Option
    Description

    All

    language

    A VARCHAR that specifies the language of the geocoding in RFC 4647 format.

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Always set the country

Geocoding large tables

Geocoding millions of rows is a long-running operation, and a single CALL can be cut short before it finishes — by your cluster's statement_timeout, by a WLM query monitoring rule, or by cancelling the query. A large table is therefore expected to need several sequential runs.

This is normal operation, not a failure: calling it again with exactly the same arguments continues where the previous call stopped, paying only for what is still pending. The contract is to re-run the same CALL until no rows remain to geocode:

That count is what remains once a run has finished. The resolved addresses are merged into the table only when the run completes, so to watch a job that is still running count the rows of its partial table instead — see Resuming an interrupted job below.

Re-run until that count stops going down. What remains at that point are addresses the provider could not resolve, and further runs will not recover them.

Resuming an interrupted job

While a job is unfinished its results live in a table named <input_table>_partial_<id> next to your input, for example my_table_partial_1a2b3c4d, where <id> stands for the parameters of the run. The resolved geometries are merged into your table only when the run completes, so a running job is watched by counting the rows of that partial table. If a run is interrupted — cancelled, disconnected or stopped by an error — run the exact same call again: it continues from that partial table and geocodes only the work still pending, paying again for at most one in-flight batch. With TomTom, a provider job that keeps failing its status checks stops the run with an error naming the job and batch, and the next run asks only for the rows still without a geometry.

Every set of parameters keeps its own partial table. Do not run the same call twice at the same time: concurrent runs work through the same internal tables and will collide. Drop the partial table of a job you abandon — it holds results you have already paid for. A table named without its schema keeps its partial table in the session as a temporary table: it is never resumed, and nothing is left once the session ends. Name the schema to resume a job.

Changing any parameter — a different country, other provider options — starts a new job with a partial table of its own. carto_batch_size is not part of that identity, which is what makes an oversized batch recoverable: re-run the same call with a smaller one and every address already paid for is kept.

Examples

Additional examples

GEOCODE_REVERSE_TABLE

Description

Reverse-geocodes an input table by adding an user defined column address_column with the addresses for a given point location column. If the input table already contains a column with the name address_column, only those rows with NULL values in it will be reverse-geocoded. An interrupted run keeps every address it had already resolved, and the next one only spends quota on the geometries still missing one — see Resuming an interrupted job below. Do not run the same call twice at the same time: concurrent runs work through the same internal tables and will collide.

The table is reverse-geocoded sequentially, one batch of rows at a time, with a default batch size of 100 rows, which is also the maximum. Use the carto_batch_size option to lower it.

Input parameters

  • input_table: VARCHAR(MAX) name of the table to be reverse-geocoded. Please make sure you have enough permissions to alter this table, as this procedure will add two columns to it to store the geocoding result.

  • geom_column (optional): GEOMETRY column name for the geometry column that contains the points to be reverse-geocoded. Defaults to 'geom'.

  • address_column: VARCHAR(MAX) name of the column where the computed addresses will be stored. It defaults to 'address', and it is created on the input table if it doesn't exist.

  • language (optional): VARCHAR(MAX) language in which results should be returned. Defaults to ''. The effect and interpretation of this parameter depends on the LDS provider assigned to your account.

  • options (optional): VARCHAR(MAX) containing a valid JSON with the different options. The following CARTO options are accepted:

    CARTO option
    Description

    carto_batch_size

    an integer that overrides the number of points sent to the provider in each batch. It must be 1 or greater — lower values fail with Option "carto_batch_size" must be a positive number. It defaults to 100. Reverse geocoding requests are fixed-size points, so lowering it is only needed if the provider times out on full batches.

    carto_force_geocode

    a BOOL (false by default) that requests an address for every geometry again, including rows that already have one. It replaces the addresses already in the address column, and spends quota on all of them.

    carto_force_rebuild

    a BOOL (false by default) that discards the partial table described in Resuming an interrupted job below and starts the job again. It never replaces a value already in the table, so only the rows still pending are requested.

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Reverse geocoding large tables

Reverse-geocoding millions of rows is a long-running operation, and a single CALL can be cut short before it finishes — by your cluster's statement_timeout, by a WLM query monitoring rule, or by cancelling the query. A large table is therefore expected to need several sequential runs.

This is normal operation, not a failure: calling it again with exactly the same arguments continues where the previous call stopped, paying only for what is still pending. The contract is to re-run the same CALL until no rows remain to reverse-geocode:

That count is what remains once a run has finished. The resolved addresses are merged into the table only when the run completes, so to watch a job that is still running count the rows of its partial table instead — see Resuming an interrupted job below.

Re-run until that count stops going down. What remains at that point are geometries the provider could not resolve, and further runs will not recover them.

Resuming an interrupted job

While a job is unfinished its results live in a table named <input_table>_partial_<id> next to your input, for example my_table_partial_1a2b3c4d, where <id> stands for the parameters of the run. The resolved addresses are merged into your table only when the run completes, so a running job is watched by counting the rows of that partial table. If a run is interrupted — cancelled, disconnected or stopped by an error — run the exact same call again: it continues from that partial table and reverse-geocodes only the work still pending, paying again for at most one in-flight batch.

Every set of parameters keeps its own partial table. Do not run the same call twice at the same time: concurrent runs work through the same internal tables and will collide. Drop the partial table of a job you abandon — it holds results you have already paid for. A table named without its schema keeps its partial table in the session as a temporary table: it is never resumed, and nothing is left once the session ends. Name the schema to resume a job. Only distinct geometries are sent to the provider, so a table where the same point repeats costs one request for all of them.

Changing any parameter — a different language, other provider options — starts a new job with a partial table of its own. carto_batch_size is not part of that identity, which is what makes an oversized batch recoverable: re-run the same call with a smaller one and every geometry already paid for is kept.

Examples

CREATE_ISOLINES

Description

Calculates the isolines (polygons) from given origins (points) in a table or query. It creates a new table with the columns of the input table or query except the geom_column plus the isolines in the column geom (if the input already contains a geom column, it will be overwritten). It calculates isolines sequentially in chunks of N rows, N being the optimal batch size for this datawarehouse and the specific LDS provider that you are using.

Note that The term isoline is used here in a general way to refer to the areas that can be reached from a given origin point within the given travel time or distance (depending on the range_type parameter).

Isolines are requested sequentially, one batch of origins at a time. The default batch size depends on the isolines provider configured for your account: 10 origins with TravelTime, 100 with TomTom, 50 with HERE and 20 with Mapbox. Use the carto_batch_size option to override it.

Input parameters

  • input: VARCHAR(MAX) name of the input table or query.

  • output_table: VARCHAR(MAX) qualified name of the output table, e.g. <my-schema>.<my-output-table>. The process will fail if the table already exists, unless carto_force_rebuild is set.

  • geom_column: VARCHAR(MAX) column name for the origin geometry column.

  • mode: VARCHAR(MAX) type of transport. The supported modes depend on the provider:

    • HERE: 'walk', 'car', 'truck', 'taxi', 'bus', 'private_bus'.

    • TomTom: 'walk', 'car', 'bike', 'motorbike', 'truck', 'taxi', 'bus', 'van'.

    • TravelTime: 'walk', 'car', 'bike', 'public_transport', 'coach', 'bus', 'train', 'ferry'.

    • Mapbox: 'walk', 'car', 'bike'.

  • range: INT range of the isoline in seconds (for range_type 'time') or meters (for range_type 'distance').

  • range_type: VARCHAR(MAX) type of range. Supported: 'time' (for isochrones), 'distance' (for isodistances).

  • options (optional): VARCHAR(MAX) containing a valid JSON with the different options. In addition to the provider options described in the table below, the following CARTO options are accepted:

    CARTO option
    Description

    carto_batch_size

    an integer that overrides the number of origins sent to the provider in each batch. It must be 1 or greater — lower values fail with Option "carto_batch_size" must be a positive number It defaults to 10 with TravelTime, 100 with TomTom, 50 with HERE and 20 with Mapbox. Isoline responses carry full polygon geometries, so the response size is the practical limit: lower it if large batches fail or time out; a value above it is reduced to it and reported in the summary rather than rejected.

    carto_keep_orig_geom

    a boolean, false by default. When true the output keeps the geometry each result was computed from, in a orig_geom column. When false that column is absent.

    carto_force_rebuild

    a boolean, false by default. When true the job starts from scratch: the output table and the partial table holding any unfinished work for these parameters are dropped, and every result is requested and paid for again. A table under the output name that this procedure did not produce is never dropped — the call fails instead.

    Valid provider options are described in the table below. If no options are indicated then 'default' values would be applied.

    Provider
    Option
    Description

    HERE

    arrival_time

    A VARCHAR that specifies the time of arrival. If the value is set, a reverse isoline is calculated. If set to "any", then time-dependent effects will not be taken into account. It cannot be used in combination with departure_time. Supported: "any", "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>".

    HERE

    departure_time

    Default: "any". A VARCHAR that specifies the time of departure. If set to "any", then time-dependent effects will not be taken into account. It cannot be used in combination with arrival_time. Supported: "any", "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>".

    HERE

    optimize_for

    Default: "balanced". A VARCHAR that specifies how isoline calculation is optimized. Supported: "quality" (calculation of isoline focuses on quality, that is, the graph used for isoline calculation has higher granularity generating an isoline that is more precise), "performance" (calculation of isoline is performance-centric, quality of isoline is reduced to provide better performance) and "balanced" (calculation of isoline takes a balanced approach averaging between quality and performance).

    HERE

    routing_mode

    Default: "fast". A VARCHAR that specifies which optimization is applied during isoline calculation. Supported: "fast" (route calculation from start to destination optimized by travel time. In many cases, the route returned by the fast mode may not be the route with the fastest possible travel time. For example, the routing service may favor a route that remains on a highway, even if a faster travel time can be achieved by taking a detour or shortcut through an inconvenient side road) and "short" (route calculation from start to destination disregarding any speed information. In this mode, the distance of the route is minimized, while keeping the route sensible. This includes, for example, penalizing turns. Because of that, the resulting route will not necessarily be the one with minimal distance).

    TomTom

    departure_time

    Default: "now". A VARCHAR that specifies the time of departure. Supported: "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>".

    TomTom

    traffic

    Default: false. A BOOLEAN that specifies if all available traffic information will be taken into consideration. Supported: true and false.

    TravelTime

    level_of_detail

    A JSON string. In the most typical case, you will want to use a string in the form { scale_type: 'simple_numeric', level: -N }, with N being the detail level (-8 by default). Higher Ns (more negative levels) will simplify the polygons more but will reduce performance. There are other ways of setting the level of detail. Check the [https://docs.traveltime.com/api/reference/isochrones#arrival_searches-level_of_detail](TravelTime docs) for more info.

    TravelTime

    departure_time

    Default: "now". A STRING that specifies the time of departure. Supported: "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>Z".

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Resuming an interrupted job

While a job is unfinished its results live in a table named <output_table>_partial_<id> next to your output, where <id> stands for the parameters of the run. The output table appears only when the run completes, so a table under the output name is always a finished result. If a run is interrupted — cancelled, disconnected or stopped by an error — run the exact same call again: it continues from that partial table and requests, and pays for, only the work still pending. The error names the batch the run stopped on, so no work is ever dropped silently.

Every set of parameters keeps its own partial table. Do not run the same call twice at the same time: concurrent runs work through the same internal tables and will collide. Drop the partial table of a job you abandon — it holds results you have already paid for.

Once the job has finished the same call fails rather than write over the result; carto_force_rebuild runs it again from scratch. When the input is a query its text is part of the parameters, so editing it — even cosmetically — starts a new job: pass it unchanged, or point at a table or view. carto_batch_size is not part of that identity, which is what makes an oversized batch recoverable: re-run the same call with a smaller one and everything already paid for is kept.

Examples

CREATE_ROUTES

Description

Calculates the routes (line strings) between given origins and destinations (points) in a query. It creates a new table with the columns of the input query with the resulting route in the user defined column geom_column (if the input already contains a column named geom_column, it will be overwritten) and a carto_routing_metadata column with the response of the service provider except for the route geometry.

Note that routes are calculated using the external LDS provider assigned to your CARTO account. Currently TomTom, HERE, and TravelTime are supported.

Routes are calculated sequentially, one batch at a time, with a default batch size of 10 routes, which is also the maximum. Use the carto_batch_size option to lower it.

Input parameters

  • input: VARCHAR(MAX) name of the input query, which must have columns named origin and destination of type GEOMETRY and containing points. If a column named waypoints is also present, it should contain a VARCHAR with the coordinates of the desired intermediate points with the format "lon1,lat1:lon2,lat2...".

  • output_table: VARCHAR(MAX) qualified name of the output table, e.g. <my-schema>.<my-output-table>. The process will fail if the table already exists, unless carto_force_rebuild is set.

  • geom_column: VARCHAR(MAX) column name for generated routes geometry column.

  • mode: VARCHAR(MAX) type of transport. The supported modes depend on the provider:

    • TomTom: 'car', 'pedestrian', 'bicycle', 'motorcycle', 'truck', 'taxi', 'bus', 'van'.

    • HERE: 'car', 'truck', 'pedestrian', 'bicycle', 'scooter', 'taxi', 'bus', 'privateBus'.

    • TravelTime: 'cycling', 'driving', 'walking', 'public_transport', 'coach', 'bus', 'train', 'ferry', 'driving+train', 'driving+ferry', 'cycling+ferry', 'cycling+public_transport'.

  • options (optional): VARCHAR(MAX) containing a valid JSON with optional parameters. This is intended for advanced use: additional parameters can be passed directly to the Routing provider by placing them in this JSON string. To find out what your provider is, check the GET_LDS_QUOTA_INFO function. In addition to the provider parameters described in the table below, the following CARTO options are accepted:

    CARTO option
    Description

    carto_batch_size

    an integer that overrides the number of routes sent to the provider in each batch. It must be 1 or greater — lower values fail with Option "carto_batch_size" must be a positive number. It defaults to 10. Lower it when a batch fails because the request is too large, which happens with long waypoint lists.

    carto_force_rebuild

    a boolean, false by default. When true the job starts from scratch: the output table and the partial table holding any unfinished work for these parameters are dropped, and every result is requested and paid for again. A table under the output name that this procedure did not produce is never dropped — the call fails instead.

    The following are some of the most common parameters for each provider:

    • TomTom:

      • avoid: Specifies something that the route calculation should try to avoid when determining the route. Possible values (several of them can be used at the same time):

        • tollRoads

        • motorways

        • ferries

        • unpavedRoads

        • carpools

        • alreadyUsedRoads

        • borderCrossings

        • tunnels

        • carTrains

        • lowEmissionZones

      • routeType: Specifies the type of optimization used when calculating routes. Possible values: fastest, shortest, short eco, thrilling

      • traffic: Set to true true to consider all available traffic information during routing. Set to false otherwise

      • departAt: The date and time of departure at the departure point. It should be specified in RFC 3339 format with an optional time zone offset.

      • arriveAt: The date and time of arrival at the destination point. It should be specified in RFC 3339 format with an optional time zone offset.

      • vehicleMaxspeed: Maximum speed of the vehicle in kilometers/hour.

    • HERE

      • avoid: Elements or areas to avoid. Information about avoidance can be found here

      • departureTime: The date and time of departure at the departure point. It should be specified in RFC 3339 format with an optional time zone offset.

      • arrivalTime: The date and time of arrival at the destination point. It should be specified in RFC 3339 format with an optional time zone offset.

      • language: The language to use. Supported language codes can be found here

    • TravelTime

      • snap_penalty: Controls whether walking time and distance from the departure location to the nearest road and from the nearest road to the arrival location are included in the route. Possible values: enabled (walking segments are added to the total travel time and distance), disabled (journey effectively starts and ends at the nearest points on the road network). Defaults to disabled for driving modes and enabled for other modes.

    For more advanced usage, check the documentation of your provider's routing API.

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Resuming an interrupted job

While a job is unfinished its results live in a table named <output_table>_partial_<id> next to your output, where <id> stands for the parameters of the run. The output table appears only when the run completes, so a table under the output name is always a finished result. If a run is interrupted — cancelled, disconnected or stopped by an error — run the exact same call again: it continues from that partial table and requests, and pays for, only the work still pending. The error names the provider job and the batch the run stopped on, so no work is ever dropped silently.

Every set of parameters keeps its own partial table. Do not run the same call twice at the same time: concurrent runs work through the same internal tables and will collide. Drop the partial table of a job you abandon — it holds results you have already paid for.

Once the job has finished the same call fails rather than write over the result; carto_force_rebuild runs it again from scratch. When the input is a query its text is part of the parameters, so editing it — even cosmetically — starts a new job: pass it unchanged, or point at a table or view. carto_batch_size is not part of that identity, which is what makes an oversized batch recoverable: re-run the same call with a smaller one and everything already paid for is kept.

Examples

GEOCODE

Description

Geocodes an address into a point with its geographic coordinates (latitude and longitude).

Input parameters

  • address: VARCHAR(MAX) input address to geocode.

  • country (optional): VARCHAR name of the country in ISO 3166-1 alpha-2. Defaults to ''.

  • options (optional): VARCHAR(MAX) containing a valid JSON with the different options. Valid options are described in the table below. If no options are indicated then 'default' values would be applied.

    Provider
    Option
    Description

    All

    language

    A VARCHAR that specifies the language of the geocoding in RFC 4647 format.

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Return type

GEOMETRY

Constraints

This function performs requests to the CARTO Location Data Services API. Redshift makes parallel requests depending on the number of records you are processing, potentially hitting the limit of the number of requests per seconds allowed for your account. The payload size of these requests depends on the number of records and could cause a timeout in the external function, with the error message External function timeout. The limit is around 500 records but could vary with the provider. To avoid this error, please try geocoding smaller volumes of data or using the procedure GEOCODE_TABLE instead. This procedure manages concurrency and payload size to avoid exceeding this limit.

Examples

GEOCODE_REVERSE

Description

Performs a reverse geocoding of the point received as input.

Input parameters

  • geom: GEOMETRY input point for which to obtain the address.

  • language (optional): VARCHAR(MAX) language in which results should be returned.

  • options (optional): VARCHAR(MAX) containing a valid JSON with the different options. No options are allowed currently, so this value will not be taken into account.

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Return type

VARCHAR(MAX)

Constraints

This function performs requests to the CARTO Location Data Services API. Redshift makes parallel requests depending on the number of records you are processing, potentially hitting the limit of the number of requests per seconds allowed for your account. The payload size of these requests depends on the number of records and could cause a timeout in the external function, with the error message External function timeout. The limit is around 500 records but could vary with the provider. To avoid this error, please try processing smaller volumes of data.

Examples

ISOLINE

Description

Calculates the isoline polygon from a given point.

Input parameters

  • origin: GEOMETRY origin point of the isoline.

  • mode: VARCHAR(MAX) type of transport. The supported modes depend on the provider:

    • HERE: 'walk', 'car', 'truck', 'taxi', 'bus', 'private_bus'.

    • TomTom: 'walk', 'car', 'bike', 'motorbike', 'truck', 'taxi', 'bus', 'van'.

    • TravelTime: 'walk', 'car', 'bike', 'public_transport', 'coach', 'bus', 'train', 'ferry'.

    • Mapbox: 'walk', 'car', 'bike'.

  • range: INT range of the isoline in seconds (for range_type 'time') or meters (for range_type 'distance').

  • range_type: VARCHAR(MAX) type of range. Supported: 'time' (for isochrones), 'distance' (for isodistances).

  • options (optional): VARCHAR(MAX) containing a valid JSON with the different options. Valid options are described in the table below. If no options are indicated then 'default' values would be applied.

    Provider
    Option
    Description

    HERE

    arrival_time

    A VARCHAR that specifies the time of arrival. If the value is set, a reverse isoline is calculated. If set to "any", then time-dependent effects will not be taken into account. It cannot be used in combination with departure_time. Supported: "any", "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>".

    HERE

    departure_time

    Default: "now". A VARCHAR that specifies the time of departure. If set to "any", then time-dependent effects will not be taken into account. It cannot be used in combination with arrival_time. Supported: "any", "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>".

    HERE

    optimize_for

    Default: "balanced". A VARCHAR that specifies how isoline calculation is optimized. Supported: "quality" (calculation of isoline focuses on quality, that is, the graph used for isoline calculation has higher granularity generating an isoline that is more precise), "performance" (calculation of isoline is performance-centric, quality of isoline is reduced to provide better performance) and "balanced" (calculation of isoline takes a balanced approach averaging between quality and performance).

    HERE

    routing_mode

    Default: "fast". A VARCHAR that specifies which optimization is applied during isoline calculation. Supported: "fast" (route calculation from start to destination optimized by travel time. In many cases, the route returned by the fast mode may not be the route with the fastest possible travel time. For example, the routing service may favor a route that remains on a highway, even if a faster travel time can be achieved by taking a detour or shortcut through an inconvenient side road) and "short" (route calculation from start to destination disregarding any speed information. In this mode, the distance of the route is minimized, while keeping the route sensible. This includes, for example, penalizing turns. Because of that, the resulting route will not necessarily be the one with minimal distance).

    TomTom

    departure_time

    Default: "now". A VARCHAR that specifies the time of departure. If set to "any", then time-dependent effects will not be taken into account. Supported: "any", "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>".

    TomTom

    traffic

    Default: true. A BOOLEAN that specifies if all available traffic information will be taken into consideration. Supported: true and false.

    TravelTime

    level_of_detail

    A JSON string. In the most typical case, you will want to use a string in the form { scale_type: 'simple_numeric', level: -N }, with N being the detail level (-8 by default). Higher Ns (more negative levels) will simplify the polygons more but will reduce performance. There are other ways of setting the level of detail. Check the [https://docs.traveltime.com/api/reference/isochrones#arrival_searches-level_of_detail](TravelTime docs) for more info.

    TravelTime

    departure_time

    Default: "now". A STRING that specifies the time of departure. Supported: "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>Z".

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Return type

GEOMETRY

Constraints

This function performs requests to the CARTO Location Data Services API. Redshift makes parallel requests depending on the number of records you are processing, potentially hitting the limit of the number of requests per seconds allowed for your account. The payload size of these requests depends on the number of records and could cause a timeout in the external function, with the error message External function timeout. The limit is around 500 records but could vary with the provider. To avoid this error, please try processing smaller volumes of data.

Examples

GET_LDS_QUOTA_INFO

Description

Returns statistics about the LDS quota. LDS quota is an annual quota that defines how much geocoding and isolines you can compute. Each geocoded row or computed isolines counts as one LDS quota unit. The single element in the result of GET_LDS_QUOTA_INFO will show your LDS quota for the current annual period (annual_quota), how much you’ve spent (used_quota), and which LDS providers are in use.

Input parameters

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Return type

VARCHAR(MAX)

Examples

CREATE_H3_ISOLINES

Description

Calculates H3 isolines (isochrones represented as H3 cells) from given origins (points) in a table or query. Unlike the standard CREATE_ISOLINES procedure which returns polygon geometries, this procedure returns an array of H3 cells with travel time information for each cell.

It creates a new table with the columns of the input table or query except the geom_column, with each H3 cell unnested into a separate row containing h3, h3_min, h3_max, and h3_mean as scalar values. It calculates H3 isolines sequentially in chunks of N rows, N being the optimal batch size for this datawarehouse.

The output table will contain a column named carto_isoline_metadata with error information for each isoline result. Rows with errors will have NULL values in the H3 columns.

Isolines are requested sequentially, one batch of origins at a time, with a default batch size of 10 origins. Use the carto_batch_size option to override it.

Input parameters

  • input: VARCHAR(MAX) name of the input table or query.

  • output_table: VARCHAR(MAX) qualified name of the output table, e.g. <my-schema>.<my-output-table>. The process will fail if the table already exists, unless carto_force_rebuild is set.

  • geom_column: VARCHAR(MAX) column name for the origin geometry column.

  • mode: VARCHAR(MAX) type of transport. Supported modes for TravelTime: 'walk', 'car', 'bike', 'public_transport', 'coach', 'bus', 'train', 'ferry'.

  • range: INT range of the isoline in seconds (for range_type 'time') or meters (for range_type 'distance'). Valid range for time: 60-36000 (1 minute to 10 hours).

  • range_type: VARCHAR(MAX) type of range. Currently only 'time' is supported for H3 isolines.

  • resolution: INT H3 resolution level for the output cells. Valid range: 6-12.

  • options (optional): VARCHAR(MAX) containing a valid JSON with the different options. In addition to the provider options described in the table below, the following CARTO options are accepted:

    CARTO option
    Description

    carto_batch_size

    an integer that overrides the number of origins sent to the provider in each batch. It must be 1 or greater — lower values fail with Option "carto_batch_size" must be a positive number It defaults to 10. Lower it if large batches fail or time out; a value above the largest batch CARTO sends to the provider is reduced to it and reported in the summary rather than rejected.

    carto_keep_orig_geom

    a boolean, false by default. When true the output keeps the geometry each result was computed from, in a orig_geom column. When false that column is absent.

    carto_force_rebuild

    a boolean, false by default. When true the job starts from scratch: the output table and the partial table holding any unfinished work for these parameters are dropped, and every result is requested and paid for again. A table under the output name that this procedure did not produce is never dropped — the call fails instead.

    Valid provider options are described in the table below. If no options are indicated then 'default' values would be applied.

    Provider
    Option
    Description

    TravelTime

    level_of_detail

    A JSON string. In the most typical case, you will want to use a string in the form { scale_type: 'simple_numeric', level: -N }, with N being the detail level (-8 by default). Higher Ns (more negative levels) will simplify the results more but will reduce performance. Check the TravelTime docs for more info.

    TravelTime

    departure_time

    Default: "now". A STRING that specifies the time of departure. Supported: "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>Z".

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Output Table Schema

The output table will contain all columns from the input table except geom_column, with each H3 cell as a separate row:

Column
Type
Description

h3

VARCHAR

H3 cell index.

h3_min

INT

Minimum travel time in seconds for this H3 cell.

h3_max

INT

Maximum travel time in seconds for this H3 cell.

h3_mean

INT

Average travel time in seconds for this H3 cell.

carto_isoline_metadata

VARCHAR(MAX)

Error information, or NULL if successful.

Resuming an interrupted job

While a job is unfinished its results live in a table named <output_table>_partial_<id> next to your output, where <id> stands for the parameters of the run. The output table appears only when the run completes, so a table under the output name is always a finished result. If a run is interrupted — cancelled, disconnected or stopped by an error — run the exact same call again: it continues from that partial table and requests, and pays for, only the work still pending. The error names the batch the run stopped on, so no work is ever dropped silently.

Every set of parameters keeps its own partial table. Do not run the same call twice at the same time: concurrent runs work through the same internal tables and will collide. Drop the partial table of a job you abandon — it holds results you have already paid for.

Once the job has finished the same call fails rather than write over the result; carto_force_rebuild runs it again from scratch. When the input is a query its text is part of the parameters, so editing it — even cosmetically — starts a new job: pass it unchanged, or point at a table or view. carto_batch_size is not part of that identity, which is what makes an oversized batch recoverable: re-run the same call with a smaller one and everything already paid for is kept.

Examples

CREATE_ROUTING_MATRIX

Description

Calculates the routes (line strings) between given origins and destinations (points) in two table. It creates a new table with the cross join of the origin and destination geom columns adding the distance or time between the different routes. A carto_routing_matrix_metadata column will also be added containing pottential errors thrown by the service provider for particular origin-destination combinations.

Note that routes are calculated using the external LDS provider assigned to your CARTO account. Currently TomTom and TravelTime are supported.

The matrix is requested in batches of origins x destinations pairs. The default batch dimensions depend on the routing provider configured for your account: 20 x 20 with TomTom and 1 x 400 with TravelTime. Use the carto_origins_batch_size and carto_destinations_batch_size options described in the table below to override them.

Input parameters

  • origins_table: VARCHAR name of the origins input table, or a query whose result is used as origins.

  • origins_geom_column: VARCHAR column name for the origin geometry column.

  • destinations_table: VARCHAR name of the destinations input table, or a query whose result is used as destinations.

  • destinations_geom_column: VARCHAR column name for the destination geometry column.

  • output_table: VARCHAR(MAX) qualified name of the output table, e.g. <my-schema>.<my-output-table>. The process will fail if the table already exists, unless carto_force_rebuild is set.

  • geom_column: VARCHAR column name for the generated geography column that will contain the resulting routes.

  • mode: VARCHAR type of transport. The supported modes depend on the provider:

    • TomTom: 'car', 'truck', 'pedestrian'.

    • TravelTime: 'cycling', 'driving', 'driving+train', 'driving+public_transport', 'public_transport', 'walking', 'coach', 'bus', 'train', 'ferry', 'driving+ferry', 'cycling+ferry', 'cycling+public_transport'.

  • options (optional): VARCHAR containing a valid JSON with optional parameters. This is intended for advanced use: additional parameters can be passed directly to the Routing provider by placing them in this JSON string. To find out what your provider is, check the GET_LDS_QUOTA_INFO function. The following are some of the most common parameters for each provider:

    Provider
    Option
    Description

    TomTom

    avoid

    Default: []. An ARRAY that specifies something that the route calculation should try to avoid when determining the route. Supported: ["tollRoads"], ["unpavedRoads"].

    TomTom

    departAt

    Default: "any". A STRING that specifies the time of departure. Supported: "now", "any" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>".

    TomTom

    routeType

    Default: "fastest". A STRING that specifies the type of optimization used when calculating routes. Supported: "fastest", "shortest".

    TomTom

    traffic

    Default: historical. A STRING that decides how traffic is considered for computing routes. Supported: historical and live. live may not be used in conjunction with departAt=any.

    TomTom

    vehicleMaxSpeed

    Default: 0. A NUMBER that specifies the maximum speed of the vehicle in kilometers/hour. Supported: a value in the range [0, 250]. A value of 0 means that an appropriate value for the vehicle will be determined and applied during route planning.

    TravelTime

    departure_time

    Default: "now". A STRING that specifies the time of departure. Supported: "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>Z".

    TravelTime

    travel_time

    Default: 14400 (4 hours). A NUMBER that specifies the maximum travel time in seconds. Supported: a value in the range [0, 14400].

    For more advanced usage, check the documentation of your provider's matrix routing API.

  • api_base_url (optional): VARCHAR url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR an API Access Token that is allowed to use the LDS API.

    The following are options provided by CARTO in order to adjust the procedure performance:

    CARTO option
    Description

    carto_origins_batch_size

    Default: TomTom: 20, TravelTime: 1. A NUMBER that specifies the number of origins rows to process in each batch. Increasing this value in the case of Traveltime can cause some origins-destinations combinations to be skipped when the origin is not found. A value above the default is reduced to it and reported in the summary rather than rejected.

    carto_destinations_batch_size

    Default: TomTom: 20, TravelTime: 400. A NUMBER that specifies the number of destinations rows to process in each batch. A value above the default is reduced to it and reported in the summary rather than rejected.

    carto_max_status_attempts

    an integer, 120 by default, bounding how many status checks a provider job may take before the run stops. A job is also abandoned after 3 consecutive failed checks.

    carto_force_rebuild

    a boolean, false by default. When true the job starts from scratch: the output table and the partial table holding any unfinished work for these parameters are dropped, and every result is requested and paid for again. A table under the output name that this procedure did not produce is never dropped — the call fails instead.

    In the case of TomTom the next requirements must be met:

    • If departAt=now or departAT=dateTime then:

      • The size of the matrix should not be larger than 2500.

      • The number of origins should not be larger than 1000.

      • The number of destinations should not be larger than 1000.

      • The bounding box should not be larger than 400km X 400km. Example: 2X1000, 5X500, 100X25, etc.

    • If departAt=any then:

      • The size of the matrix can be as large as 100M.

      • The number of origins should not be larger than 10000.

      • The number of destinations should not be larger than 10000.

Return type

The results are stored in the table named <output_table>, which contains the following columns:

  • origin_geom: VARCHAR the origin geometry.

  • destination_geom: VARCHAR the destination geometry.

  • route_distance: INT the distance of the route in meters.

  • route_duration: INT the duration of the route in seconds.

  • carto_routing_matrix_metadata: VARCHAR possible errors thrown by the service provider for particular origin-destination combinations.

When generating the output table origin-destination combinations, duplicated or null geometries will not be processed.

Resuming an interrupted job

While a job is unfinished its results live in a table named <output_table>_partial_<id> next to your output, where <id> stands for the parameters of the run. The output table appears only when the run completes, so a table under the output name is always a finished result. If a run is interrupted — cancelled, disconnected or stopped by an error — run the exact same call again: it continues from that partial table and requests, and pays for, only the work still pending. The error names the provider job and the batch the run stopped on, so no work is ever dropped silently.

Every set of parameters keeps its own partial table. Do not run the same call twice at the same time: concurrent runs work through the same internal tables and will collide. Drop the partial table of a job you abandon — it holds results you have already paid for.

Once the job has finished the same call fails rather than write over the result; carto_force_rebuild runs it again from scratch. When the input is a query its text is part of the parameters, so editing it — even cosmetically — starts a new job: pass it unchanged, or point at a table or view. The two batch sizes are not part of that identity, but they are not a way to shrink the batches of a job already under way either: a resumed run follows the grid the interrupted run laid out, and carto_origins_batch_size and carto_destinations_batch_size passed to it only change the quota estimate. Only carto_force_rebuild lays the grid out again, requesting every result once more, so a job that fails on an oversized batch has to be rebuilt with smaller sizes rather than resumed.

Examples

Examples with TomTom specific parameters:

Example with batch sizes:

H3_ISOLINE

Description

Calculates H3 isolines (isochrones represented as H3 cells) from a given point. Unlike the standard ISOLINE function which returns a polygon geometry, this function returns an array of H3 cells with travel time information for each cell.

Input parameters

  • origin: GEOMETRY origin point of the isoline.

  • mode: VARCHAR(MAX) type of transport. Supported modes for TravelTime: 'walk', 'car', 'bike', 'public_transport', 'coach', 'bus', 'train', 'ferry'.

  • range: INT range of the isoline in seconds (for range_type 'time') or meters (for range_type 'distance'). Valid range for time: 60-36000 (1 minute to 10 hours).

  • range_type: VARCHAR(MAX) type of range. Currently only 'time' is supported for H3 isolines.

  • resolution: INT H3 resolution level for the output cells. Valid range: 6-12.

  • options (optional): VARCHAR(MAX) containing a valid JSON with the different options. Valid options are described in the table below. If no options are indicated then 'default' values would be applied.

    Provider
    Option
    Description

    TravelTime

    level_of_detail

    A JSON string. In the most typical case, you will want to use a string in the form { scale_type: 'simple_numeric', level: -N }, with N being the detail level (-8 by default). Higher Ns (more negative levels) will simplify the results more but will reduce performance. Check the TravelTime docs for more info.

    TravelTime

    departure_time

    Default: "now". A STRING that specifies the time of departure. Supported: "now" and date-time as "<YYYY-MM-DD>T<hh:mm:ss>Z".

  • api_base_url (optional): VARCHAR(MAX) url of the API where the customer account is stored.

  • api_access_token (optional): VARCHAR(MAX) an API Access Token that is allowed to use the LDS API.

Return type

SUPER

Each element in the array is an object representing an H3 cell within the isochrone:

Field
Type
Description

h3

VARCHAR

H3 cell index.

h3_min

INT

Minimum travel time in seconds to reach this H3 cell.

h3_max

INT

Maximum travel time in seconds to reach this H3 cell.

h3_mean

INT

Average travel time in seconds to reach this H3 cell.

Constraints

This function performs requests to the CARTO Location Data Services API. Redshift makes parallel requests depending on the number of records you are processing, potentially hitting the limit of the number of requests per seconds allowed for your account. To avoid this error, please try processing smaller volumes of data or use the CREATE_H3_ISOLINES procedure instead.

Examples

Last updated

Was this helpful?