Access data in BigQuery
Last updated
Was this helpful?
Last updated
Was this helpful?
In order to make the Data Observatory subscriptions accessible in your BigQuery account, first of all CARTO needs to create and register your subscriptions in your CARTO account. Once this first step has been done, Admin users of your CARTO organization will be able to see the list of active subscriptions in the Data Observatory section in Settings.
Once your subscriptions have been created, the CARTO team will proceed with the data transfers so to make the data accessible directly on your BigQuery account. Please contact your CARTO representative if you need to start this process on demand (e.g. public data subscriptions).
CARTO hosts all Data Observatory data in BigQuery's US Multi-region. Automatically we share access to your subscriptions hosted in that location to any of your BigQuery connections. However, in order to be able to enrich your data hosted in a different location with your Data Observatory subscriptions, CARTO needs to transfer the data directly to your BigQuery account in your specific location.
In order to make your subscriptions available in your BigQuery account, CARTO leverages the native private sharing mechanisms in the Google Cloud Platform, in this case the Analytics Hub.
CARTO will create and maintain a listing in the Analytics Hub with the data from all your Data Observatory subscriptions, and we will provide you with a link to access such listing. We will privately share access to that listing based on the GCP user credentials (individual Google account, Google group, service account) that you share with us.
Once you access the listing in the Analytics Hub, you should click on '+ Subscribe' . That will create a linked dataset in a BigQuery project of your choice.
Once you have selected the destination project and the linked dataset name for your listing, then click on 'Save'. The linked dataset will then become available from your BigQuery console within the selected project.
In order to use the data from your Data Observatory subscriptions that have been transferred directly to your BigQuery account you should not use the Data Observatory tabs in Data Explorer, Workflows or Builder, but directly your own BigQuery connections.
You need to ensure that the destination project that you have selected when creating the linked dataset is accessible from your BigQuery connections.
CARTO hosts all Data Observatory data in BigQuery's US Multi-region. If your BigQuery account is hosted in that specific region, you will have access to your Data Observatory subscriptions automatically and you will be able to enrich your data with your subscriptions without any further action. When you create a connection in CARTO to your BigQuery account in the US Multi-region location, we automatically share you access to the carto-data.account_id project in BigQuery where your subscription data is available. You can pin the carto-data.account_id dataset in your BigQuery account to ensure it is always available from your BigQuery connections in CARTO.
This access method is recommended only for users with CARTO SaaS accounts (not CARTO Self-Hosted deployments) connecting to BigQuery in US Multi-region.
If you have set up a connection of type OAuth (using the “Sign in with Google” option), you will be granted permissions to query your Data Observatory subscriptions using the BigQuery console while you are logged in with the same Google user.
On the other hand, if your have set up a connection using a Service Account instead, you will be granted permissions to query your Data Observatory subscriptions from any BigQuery client while you are logged in using such Service Account.
