Google BigQuery
Last updated
Last updated
CARTO can connect to your BigQuery Data Warehouse, allowing you to use your data for building Maps and Workflows. There are two ways to set up a connection to Google BigQuery:
Sign in with Google: connect your own Google account and use all the Google BigQuery permissions that you have access to, with the possibility of enforcing viewer credentials. This is the preferred authentication mechanism. This method is also called OAuth 2.0.
Service Account: these are a set of credentials (a key in JSON format) generated in Google Cloud, representing a set of permissions for a database or a project, not associated with an individual.
Please make sure that your credentials (regardless of the method used) have the permissions for CARTO to run. For more information, see Required BigQuery Permissions.
In both methods you will need to indicate a billing project. All queries performed by CARTO will use the billing account associated with the selected billing project. We recommend you review the different BigQuery pricing models, and more importantly, configure specific limits in BigQuery to avoid any unexpected charges.
CARTO is a fully cloud-native platform that runs queries on your behalf to power maps, workflows, etc. We never create or maintain any copies of your data.
To connect CARTO and BigQuery using your Google account simply click the Continue with Google button. This will open a Google login flow that will request the necessary scopes for CARTO to connect to your BigQuery data.
After allowing CARTO to access your Google BigQuery data, you will see a form where you'll specify the remaining details for this connection:
Name: This will be the name used to identify this connection across CARTO. It needs to be unique and there are special format rules: 3-50 characters long, containing only lowercase letters and numbers. Dashes and underscores are allowed if they're not leading or trailing.
Billing project: All queries performed by CARTO will run against this Google Cloud Platform project, and its associated billing account.
When using OAuth-based connections (such as this "Continue with Google"), you might be asked to reconnect at any time. It could happen, for example, after a few months or after changing your password. The reason is that this type of connections are linked to your Google account consent to CARTO, which you can also revoke at any moment.
Please make sure your Google account has adequate permissions for CARTO, at least at the billing project level. Learn more at Required BigQuery Permissions.
Connections to Google BigQuery using OAuth can be set up to require viewer credentials. This means that when the connection is shared, other users trying to access it will have to provide their own credentials to use it, instead of using the credentials (token) of the user that created the connection.
For more information, see Requiring viewing credentials for shared connections.
If you select Connect using a Service Account, you'll see a form where you'll specify the details for this connection:
Name: This will be the name used to identify this connection across CARTO. It needs to be unique and there are special format rules: 3-50 characters long, containing only lowercase letters and numbers. Dashes and underscores are allowed if they're not leading or trailing.
Service account key: The credentials file in JSON format. Please read the following instructions to learn how to create a service account and a service account key file in Google Cloud.
Billing project: All queries performed by CARTO will run against this Google Cloud Platform project, and its associated billing account.
Please make sure the Service Account has adequate permissions for CARTO, at least at the billing project level. Learn more at Required BigQuery Permissions.
When creating the connection, CARTO will check that you have a minimum set of permissions that will allow the connection to operate with CARTO. These checks are performed at the Billing-project level.
You can then granularly specify a different set of permissions for each resource. For example, the connection could have edit permissions in some tables but read-only in others. Please note that you can give limited and granular permissions to resources in completely different projects than the billing project. We call this Resource level.
For each area + resource combination, connection credentials must have at least the “Minimum permissions” to work. Some optional features may require additional permissions to work as expected.
For the best experience in CARTO, we advise you to set up the “Recommended role”:
bigquery.dataEditor
bigquery.user
CARTO requires the following permissions at the billing project to connect to BigQuery:
| Where | Recommended role | Minimum permissions required |
|---|---|---|
Billing project (as specified in the connection) | bigquery.dataEditor bigquery.user | bigquery.jobs.list bigquery.jobs.create resourcemanager.projects.get |
CARTO requires the following permissions for each BigQuery resource in order to operate with those resources, such as projects, datasets, or tables.
| Where | Recommended role | Minimum permissions required |
|---|---|---|
Listing projects, datasets and tables in CARTO (Data Explorer) | bigquery.dataEditor bigquery.user | resourcemanager.projects.get resourcemanager.projects.list bigquery.tables.list |
Projects, datasets, and tables used for map visualization (Builder) | bigquery.dataEditor bigquery.user | bigquery.jobs.create bigquery.tables.list |
Projects, datasets, and tables used for spatial analysis (Workflows) | bigquery.dataEditor bigquery.user | bigquery.jobs.create bigquery.jobs.list bigquery.tables.list bigquery.tables.create bigquery.datasets.create bigquery.datasets.get |
Projects, datasets, and tables used in custom applications (CARTO for Developers) | bigquery.dataEditor bigquery.user | bigquery.jobs.create |
You can also check our (more generic) guide about why CARTO requires each permission, with examples on setting different connections for different teams.
Analytics Toolbox location: This setting controls the location of the Analytics Toolbox used in SQL queries generated by Workflows components, Builder SQL Analyses, 'Create Tileset', 'Geocode Table' and 'Enrich Data' functionalities. By default, CARTO will automatically determine the corresponding AT Location based on the actual region of the data.
Data Observatory location: This settings controls the location of the Data Observatory subscriptions. This setting will be observed by Data Explorer, Workflows and Enrichment to access your data subscriptions.
By default, a specific project for your account (created automatically and maintained by CARTO) will be used. For example carto-data.ac_xxxxxxxx
Workflows temp. location: This setting controls the location (project.dataset) where Workflows will create temporal tables for each node. By default, it's a carto dataset that will be created in the connection's project during the execution of a workflow. Learn more about it here.
Data Transfer Version Info: This setting is only necessary for Scheduling Workflows with 'Sign in with Google' connections. Learn about how to generate the code here.
Max number of concurrent queries: This setting controls the maximum number of simultaneous queries that CARTO will send to BigQuery using this connection.
Max query timeout: This setting controls the maximum allowed duration of queries that CARTO runs in BigQuery using this connection.
If you're using the cloud version of CARTO (SaaS), CARTO will connect to BigQuery using a set of static IPs for each region. Check this guide to find the IPs you need to allow for your specific region.
