lds
BETA
This module contains functions and procedures that make use of location data services, such as geocoding, reverse geocoding, isolines and routing computation.
For manual installations of the CARTO Analytics Toolbox, after installing it for the first time, and before using any LDS function you need to call the SETUP
procedure to configure the LDS and gateway functions. It also optionally sets default credentials.
Credentials
Your CARTO account has monthly quotas assigned for each LDS service that are used up by every call to the LDS functions and procedures in the Analytics Toolbox.
Accordingly, the use of these functions and procedures requires providing authorization credentials to prevent fraudulent usage. Two parameters are needed:
api_base_url
The API base url is simply the address through which you can access all the services of your CARTO account, and it depends on the region or premises where your account is located. Usually it will be this one:https://gcp-us-east1.api.carto.com
.api_access_token
This token is an API Access Token that is allowed to use the LDS API. You must keep this secret! Anyone that has access to this token can use up the LDS quota assigned to your account.
Both the API base url and your API access token can be accessed through the developers section of the CARTO user interface. The API base url is displayed inside this section while for the API access token you will have to create a new API access token allowing the LDS API.
For more information about CARTO for developers, please check our documentation for Developers.
If default credentials are provided to the SETUP
procedure, you can pass NULL values for the credentials in the LDS functions, and the credentials configured during SETUP will be used.
tip
To check that everything works correctly, without spending any credits, make a call to the GET_LDS_QUOTA_INFO
procedure. You can enter the following in the GCP Bigquery console having selected the project where the Analytics Toolbox is installed:
Note that if you haven't set up default LDS credentials with SETUP
you'll need to provide them when calling the LDS fucntions:
You should get a JSON response like this, with the available services and the quotas:
LDS quota is an annual quota that defines how much geocoding and isolines you can compute. Each geocoded row or computed isolines counts as one LDS quota unit. The single element in the result of LDS_QUOTA_INFO will show your LDS quota for the current annual period (annual_quota), how much you’ve spent (used_quota), and which LDS providers are in use.
This also will allow you to verify that you have the right credentials, the AT is installed correctly and the service is working.
GEOCODE_TABLE
warning
This function consumes LDS quota. Each call consumes as many units of quota as the number of rows of your input table. Before running, we recommend checking the size of the data to be geocoded and your available quota using the GET_LDS_QUOTA_INFO
function.
Description
Geocodes an input table by adding an user defined column geom_column
with the geographic coordinates (latitude and longitude) of a given address column. This procedure also adds a carto_geocode_metadata
column with additional information of the geocoding result in JSON format. It geocodes sequentially the table in chunks of 100.
api_base_url
:STRING
|NULL
url of the API where the customer account is stored. If default credentials have been configured withSETUP
NULL can be passed to use them.api_access_token
:STRING
|NULL
an API Access Token that is allowed to use the LDS API. If default credentials have been configured withSETUP
NULL can be passed to use them.input_table
:STRING
name of the table to be geocoded. Please make sure you have enough permissions to alter this table, as this procedure will add two columns to it to store the geocoding result.address_column
:STRING
name of the column from the input table that contains the addresses to be geocoded.geom_column
:STRING
|NULL
column name for the geometry column. Defaults to'geom'
. Set toNULL
to use the default value.country
:STRING
|NULL
name of the country in ISO 3166-1 alpha-2. Defaults to''
. Set toNULL
to use the default value.options
:STRING
|NULL
containing a valid JSON with the different options. In addition to the options targeted at the geocoding service described below, a boolean optioncarto_force_geocode
(false by default) can be used to force geocoding rows that already have a non-null value ingeom_column
. Valid geocoding service options are described in the table below. If no options are indicated then 'default' values would be applied.
If the input table already contains a geometry column with the name defined by geom_column
, only those rows with NULL
values will be geocoded.
If you plan to repeat the geocoding process, bear in mind that if you drop columns from your table you won't be able to create columns with the same name for a period of time (7 days) because BigQuery reserves the deleted columns names for time travel purposes. So, for example, instead of dropping the geom
column to re-geocode all rows, update the table and set it to NULL
.
Examples
Additional examples
GEOCODE_REVERSE_TABLE
warning
This function consumes LDS quota. Each call consumes as many units of quota as the number of rows your input table has. Before running, we recommend checking the size of the data to be geocoded and your available quota using the GET_LDS_QUOTA_INFO
function.
Description
Reverse-geocodes an input table by adding an user defined column address_column
with the address coordinates corresponding to a given point location column. It geocodes sequentially the table in chunks of 100 rows.
api_base_url
:STRING
|NULL
url of the API where the customer account is stored. If default credentials have been configured withSETUP
NULL can be passed to use them.api_access_token
:STRING
|NULL
an API Access Token that is allowed to use the LDS API. If default credentials have been configured withSETUP
NULL can be passed to use them.input_table
:STRING
name of the table to be reverse-geocoded. Please make sure you have enough permissions to alter this table, as this procedure will add two columns to it to store the geocoding result.geom_column
:STRING
|NULL
column name for the geometry column that contains the points to be reverse-geocoded. Defaults to'geom'
.address_column
:STRING
name of the column where the computed addresses will be stored. It defaults to'address'
, and it is created on the input table if it doesn't exist.language
:STRING
|NULL
language in which results should be returned. Defaults to''
. The effect and interpretation of this parameter depends on the LDS provider assigned to your account.options
:STRING
|NULL
containing a valid JSON with the different options. No options are allowed currently, so this value will not be taken into account.
If the input table already contains a column with the name defined by address_column
, only those rows with NULL values will be reverse-geocoded.
If you plan to repeat the reverse-geocoding process, bear in mind that if you drop columns from your table you won't be able to create columns with the same name for a period of time (7 days) because BigQuery reserves the deleted columns names for time travel purposes. So, for example, instead of dropping the address_column
column to re-process all rows, update the table and set it to NULL
.
Examples
CREATE_ISOLINES
warning
This function consumes LDS quota. Each call consumes as many units of quota as the number of rows your input table or query has. Before running, we recommend checking the size of data for which isolines will be created and your available quota using the GET_LDS_QUOTA_INFO
function.
Description
Calculates the isolines (polygons) from given origins (points) in a table or query. It creates a new table with the columns of the input table or query except the geom_column
plus the isolines in the column geom
(if the input already contains a geom
column, it will be overwritten). It calculates isolines sequentially in chunks of N rows, N being the optimal batch size for this datawarehouse and the specific LDS provider that you are using.
The output table will contain a column named carto_isoline_metadata
with error information for each isoline result. Rows with errors will have a NULL geom
.
Note that The term isoline is used here in a general way to refer to the areas that can be reached from a given origin point within the given travel time or distance (depending on the range_type
parameter).
api_base_url
:STRING
|NULL
url of the API where the customer account is stored. If default credentials have been configured withSETUP
NULL can be passed to use them.api_access_token
:STRING
|NULL
an API Access Token that is allowed to use the LDS API. If default credentials have been configured withSETUP
NULL can be passed to use them.input
:STRING
name of the input table or query.output_table
:STRING
name of the output table. It will raise an error if the table already exists.geom_column
:STRING
column name for the origin geometry column.mode
:STRING
type of transport. The supported modes depend on the provider:HERE
: 'walk', 'car', 'truck', 'taxi', 'bus', 'private_bus'.TomTom
: 'walk', 'car', 'bike', 'motorbike', 'truck', 'taxi', 'bus', 'van'.TravelTime
: 'walk', 'car', 'bike', 'public_transport', 'coach', 'bus', 'train', 'ferry'.Mapbox
: 'walk', 'car', 'bike'.
range_value
:INT64
range of the isoline in seconds (forrange_type
'time') or meters (forrange_type
'distance').range_type
:STRING
type of range. Supported: 'time' (for isochrones), 'distance' (for isodistances).options
:STRING
|NULL
containing a valid JSON with the different options. Valid options are described in the table below. If no options are indicated then 'default' values would be applied.
warning
Before running, we recommend checking your provider using the GET_LDS_QUOTA_INFO
function. Notice that some of the parameters are provider dependant. Please contact your CARTO representative if you have questions regarding the service provider configured in your organization.
Examples
Additional examples
CREATE_ROUTES
warning
This function consumes LDS quota. Each call consumes as many units of quota as the number of rows your input query has. Before running, we recommend checking the size of the data used to create the routes and your available quota using the GET_LDS_QUOTA_INFO
function.
Description
Calculates the routes (line strings) between given origins and destinations (points) in a query. It creates a new table with the columns of the input query with the resulting route in the user defined column geom_column
(if the input already contains a column named geom_column
, it will be overwritten) and a carto_routing_metadata
column with the response of the service provider except for the route geometry. It calculates routes sequentially in chunks of 100 rows.
Note that routes are calculated using the external LDS provider assigned to your CARTO account. Currently TomTom and HERE are supported.
api_base_url
:STRING
|NULL
url of the API where the customer account is stored. If default credentials have been configured withSETUP
NULL can be passed to use them.api_access_token
:STRING
|NULL
an API Access Token that is allowed to use the LDS API. If default credentials have been configured withSETUP
NULL can be passed to use them.input
:STRING
name of the input query, which must have columns namedorigin
anddestination
of typeGEOGRAPHY
and containing points. If a column namedwaypoints
is also present, it should contain a STRING with the coordinates of the desired intermediate points with the format"lon1,lat1:lon2,lat2..."
.output_table
:STRING
name of the output table. It will raise an error if the table already exists.geom_column
:STRING
column name for the generated geography column that will contain the resulting routes.mode
:STRING
type of transport. The supported modes depend on the provider:TomTom
: 'car', 'pedestrian', 'bicycle', 'motorcycle', 'truck', 'taxi', 'bus', 'van'.HERE
: 'car', 'truck', 'pedestrian', 'bicycle', 'scooter', 'taxi', 'bus', 'privateBus'.
options
:STRING
|NULL
containing a valid JSON with optional parameters. This is intended for advanced use: additional parameters can be passed directly to the Routing provider by placing them in this JSON string. To find out what your provider is, check theGET_LDS_QUOTA_INFO
function. The following are some of the most common parameters for each provider:TomTom
:avoid
: Specifies something that the route calculation should try to avoid when determining the route. Possible values (several of them can be used at the same time):tollRoads
motorways
ferries
unpavedRoads
carpools
alreadyUsedRoads
Avoids
This
borderCrossings
tunnels
carTrains
lowEmissionZones
routeType
: Specifies the type of optimization used when calculating routes. Possible values:fastest
,shortest
,short
eco
,thrilling
traffic
: Set to truetrue
to consider all available traffic information during routing. Set tofalse
otherwisedepartAt
: The date and time of departure at the departure point. It should be specified in RFC 3339 format with an optional time zone offset.arriveAt
: The date and time of arrival at the destination point. It should be specified in RFC 3339 format with an optional time zone offset.vehicleMaxspeed
: Maximum speed of the vehicle in kilometers/hour.
HERE
avoid
: Elements or areas to avoid. Information about avoidance can be found heredepartureTime
: The date and time of departure at the departure point. It should be specified in RFC 3339 format with an optional time zone offset.arrivalTime
: The date and time of arrival at the destination point. It should be specified in RFC 3339 format with an optional time zone offset.language
: The language to use. Supported language codes can be found here
For more advanced usage, check the documentation of your provider's routing API.
warning
Before running, we recommend checking your provider using the GET_LDS_QUOTA_INFO
function. Notice that some of the parameters are provider dependant. Please contact your CARTO representative if you have questions regarding the service provider configured in your organization.
Examples
Example with TomTom specific parameters:
GEOCODE
warning
This function consumes LDS quota. Each call consumes one unit of quota. Before running, check the size of the data to be geocoded and make sure you store the result in a table to avoid misuse of the quota. To check the information about available and consumed quota use the function GET_LDS_QUOTA_INFO
.
We recommend using this function only with an input of up to 10 records. In order to geocode larger sets of addresses, we strongly recommend using the GEOCODE_TABLE
procedure. Likewise, in order to materialize the results in a table.
Description
Geocodes an address into a point with its geographic coordinates (latitude and longitude).
api_base_url
:STRING
|NULL
url of the API where the customer account is stored. If default credentials have been configured withSETUP
NULL can be passed to use them.api_access_token
:STRING
|NULL
an API Access Token that is allowed to use the LDS API. If default credentials have been configured withSETUP
NULL can be passed to use them.address
:STRING
input address to geocode.country
:STRING
|NULL
name of the country in ISO 3166-1 alpha-2. Defaults to''
. Set toNULL
to use the default value.options
:STRING
|NULL
containing a valid JSON with the different options.
Return type
GEOGRAPHY
Constraints
This function performs requests to the CARTO Location Data Services API. BigQuery makes parallel requests depending on the number of records you are processing, potentially hitting the limit of the number of requests per seconds allowed for your account. The payload size of these requests depends on the number of records and could cause a timeout in the external function, with the error message External function timeout
. The limit is around 500 records but could vary with the provider. To avoid this error, please try geocoding smaller volumes of data or using the procedure GEOCODE_TABLE
instead. This procedure manages concurrency and payload size to avoid exceeding this limit.
Examples
GEOCODE_REVERSE
warning
This function consumes LDS quota. Each call consumes one unit of quota. Before running, check the size of the data to be reverse geocoded and make sure you store the result in a table to avoid misuse of the quota. To check the information about available and consumed quota use the function [GET_LDS_QUOTA_INFO
](lds.md#get_ lds_quota_info)
We recommend using this function only with an input of up to 10 records. In order to reverse-geocode larger sets of locations, we strongly recommend using the GEOCODE_REVERSE_TABLE
procedure. Likewise, in order to materialize the results in a table.
Description
Performs a reverse geocoding of the point received as input.
api_base_url
:STRING
|NULL
url of the API where the customer account is stored. If default credentials have been configured withSETUP
NULL can be passed to use them.api_access_token
:STRING
|NULL
an API Access Token that is allowed to use the LDS API. If default credentials have been configured withSETUP
NULL can be passed to use them.geom
:GEOGRAPHY
input point for which to obtain the address.language
:STRING
|NULL
language in which results should be returned. Defaults to''
. The effect and interpretation of this parameter depends on the LDS provider assigned to your account.options
:STRING
|NULL
containing a valid JSON with the different options. No options are allowed currently, so this value will not be taken into account.
Return type
STRING
Constraints
This function performs requests to the CARTO Location Data Services API. BigQuery makes parallel requests depending on the number of records you are processing, potentially hitting the limit of the number of requests per seconds allowed for your account. The payload size of these requests depends on the number of records and could cause a timeout in the external function, with the error message External function timeout
. The limit is around 500 records but could vary with the provider. To avoid this error, please try processing smaller volumes of data.
Examples
ISOLINE
warning
This function consumes LDS quota. Each call consumes one unit quota. Before running, check the size of the data and make sure you store the result in a table to avoid misuse of the quota. To check the information about available and consumed quota use the function GET_LDS_QUOTA_INFO
.
We recommend using this function only with an input of up to 10 records. In order to calculate isolines for larger sets of locations, we strongly recommend using the CREATE_ISOLINES
procedure. Likewise, in order to materialize the results in a table.
Description
Calculates the isoline polygon from a given point.
api_base_url
:STRING
|NULL
url of the API where the customer account is stored. If default credentials have been configured withSETUP
NULL can be passed to use them.api_access_token
:STRING
|NULL
an API Access Token that is allowed to use the LDS API. If default credentials have been configured withSETUP
NULL can be passed to use them.origin
:GEOGRAPHY
origin point of the isoline.mode
:STRING
type of transport. The supported modes depend on the provider:HERE
: 'walk', 'car', 'truck', 'taxi', 'bus', 'private_bus'.TomTom
: 'walk', 'car', 'bike', 'motorbike', 'truck', 'taxi', 'bus', 'van'.TravelTime
: 'walk', 'car', 'bike', 'public_transport', 'coach', 'bus', 'train', 'ferry'.Mapbox
: 'walk', 'car', 'bike'.
range_value
:INT
range of the isoline in seconds (forrange_type
'time') or meters (forrange_type
'distance').range_type
:STRING
type of range. Supported: 'time' (for isochrones), 'distance' (for isodistances).options
STRING
|NULL
containing a valid JSON with the different options. Valid options are described in the table below. If no options are indicated then 'default' values would be applied.
warning
Before running, we recommend checking your provider using the GET_LDS_QUOTA_INFO
function. Notice that some of the parameters are provider dependant. Please contact your CARTO representative if you have questions regarding the service provider configured in your organization.
Return type
GEOGRAPHY
Constraints
This function performs requests to the CARTO Location Data Services API. Bigquery makes parallel requests depending on the number of records you are processing, potentially hitting the limit of the number of requests per seconds allowed for your account. The payload size of these requests depends on the number of records and could cause a timeout in the external function, with the error message External function timeout
. The limit is around 500 records but could vary with the provider. To avoid this error, please try processing smaller volumes of data.
Examples
GET_LDS_QUOTA_INFO
Description
Returns statistics about the LDS quota. LDS quota is an annual quota that defines how much geocoding and isolines you can compute. Each geocoded row or computed isolines counts as one LDS quota unit. The single element in the result of GET_LDS_QUOTA_INFO will show your LDS quota for the current annual period (annual_quota), how much you’ve spent (used_quota), and which LDS providers are in use.
api_base_url
:STRING
|NULL
url of the API where the customer account is stored. If default credentials have been configured withSETUP
NULL can be passed to use them.api_access_token
:STRING
|NULL
an API Access Token that is allowed to use the LDS API. If default credentials have been configured withSETUP
NULL can be passed to use them.
Return type
STRING
Example
Last updated