Core module

If you are not a CARTO customer you can still use the core modules of the Analytics Toolbox. These modules are available to all BigQuery authenticated users through the carto-os and carto-os-eu projects. These projects are deployed in the US and EU multi-regions, respectively, and you may choose one or the other depending on the location of your data.

If you need to deploy the core module in a different project, follow the instructions below:

Installation

We can divide the process into three steps: preparation, setup and installation. The setup must be done only the first time, then the installation must be done every time you want to install a new version of the packages.

In this guide we will use Google Cloud Shell to setup and install the toolbox. Please open the GCP console and select the project to install the toolbox, then use the “>_” button (top right) to “Activate Cloud Shell”.

Preparation

You will need a GCP project to install the Toolbox, as well as a storage bucket in the same project to store the JavasScript libraries needed. Users of the Toolbox will need permissions to read both the BigQuery dataset (where the functions and procedures will be installed) and the bucket in order to run the CARTO Analytics Toolbox.

We will set the project and bucket names as well as the location where the toolbox will be created (should be the same as the bucket) as Cloud Shell environment variables:

• PROJECT: Project id where the toolbox dataset will be created

• REGION: Location of the BigQuery dataset that will be created to install the Analytics Toolbox

• BUCKET: Name of the bucket to store the JavasScript libraries needed by the Toolbox (please omit any protocol prefix like gs://)

Set these variables by executing the following in Cloud Shell (after replacing the appropriate values):

export PROJECT="<my-project>"
export REGION="<my-region>"
export BUCKET="<my-bucket>"

After a while without using the Cloud Shell you may need to reconnect it; if that happens, you will need to set the environment variables again.

Setup

This step is only required before the first installation. Activate the Cloud Shell in the target project and make sure the environment variables from the preparation step above are set.

Before starting the process make sure the target GCP project exists and that it is the correct one by executing the following:

# Check project existence
gcloud projects describe $PROJECT

Then, create a BigQuery dataset named carto, where the Toolbox will be installed:

# Create dataset "carto"
bq mk --location=$REGION --description="CARTO dataset" -d $PROJECT:carto

Installation

To install the Analytics Toolbox in the carto dataset we will use the this installation package and follow the instructions below. Please note that this process should be repeated every time a new version of the Toolbox is available.

Access the Cloud Shell and set the environment variables as described in the preparation step above, then run the following commands:

# Download package
wget https://storage.googleapis.com/carto-analytics-toolbox-core/bigquery/carto-analytics-toolbox-core-bigquery-latest.zip
unzip carto-analytics-toolbox-core-bigquery-latest.zip

# Enter the directory
cd $(unzip -Z -1 carto-analytics-toolbox-core-bigquery-latest.zip | head -1)

# Prepare SQL code
sed -e 's!@@BUCKET@@!'"$BUCKET"'!g'  modules.sql > modules_rep.sql

# Copy libs to bucket
gsutil -m cp -r libs/ gs://$BUCKET/carto/

# Install the functions and procedures
bq --location=$REGION --project_id=$PROJECT query --use_legacy_sql=false --max_statement_results=10000 --format=prettyjson < modules_rep.sql

This will remove all the previous functions and procedures in the “carto” dataset.

Check the installed version

Execute the following in the BigQuery console, in the same project where the Toolbox was installed:

SELECT carto.VERSION_CORE();

You can also check all the installed routines (functions and procedures) with:

SELECT * FROM carto.INFORMATION_SCHEMA.ROUTINES;

Congratulations! You have successfully installed the CARTO Analytics Toolbox in your BigQuery project. Now you can start using the functions.