Visualizing Data Observatory datasets
Data Observatory datasets can be visualized from the CARTO Workspace using Builder. You can easily do so by clicking on the Create map action in the subscription’s detail page, available from the Data Observatory section of the Data Explorer:
Or by adding a new Data Observatory source to an existing map:
Those datasets whose size is within platform limits will be visualized in full. Bigger datasets will be applied a spatial filter (a buffer around the centroid of the most populated city of the dataset’s country), but this filter can be modified at your own will through the provided SQL query (check out this tutorial for more details). These datasets will require a tileset to be visualized in full. Please refer to the Creating Data Observatory tilesets section to learn more.
Choosing a connection
You will be asked to select the connection that will be used to create a map with your Data Observatory subscription. The chosen connection will be used to retrieve the necessary data to add the layer to the map. Currently, CARTO Data Warehouse, BigQuery and Snowflake connections are supported; Redshift and Databricks support is coming soon.
In order to be able to use a Snowflake connection to create a map, the data first needs to be imported into your database. This import process is performed by our engineering team on a request basis.
To request it, go to the subscription’s page, click on the Create map button and choose the desired Snowflake connection. You will be asked to request access to the dataset.
Once we receive your request, we will get in touch with you to coordinate the import process. The data will be imported into a schema called
CARTO that will be created in the Snowflake database you have set up in your Snowflake connection. Finally, you will be able to create a map using such connection.
Some of the spatial datasets offered in the Data Observatory are massive (a few TB), either due to their global coverage, such as WorldPop or NASADEM, or their fine granularity, such as ACS Sociodemographics at census block group level, and their visualization requires the creation of a tileset.
From the Workspace
When a Data Observatory subscription requires a tileset to be visualized in full, a Create tileset option will be available from the subscription’s page.
Clicking on this option will display a wizard that will guide you through the tileset creation process:
- Choose the name and location of the output tileset. Please note that you should have writing permissions in the destination dataset.
- Select the zoom range for which the tileset will be created.
- Select the attributes (data columns) that will be included in the tileset.
- Launch tileset creation.
Once the tileset creation has been launched, you can check its status using the processing jobs menu available at the top right of the page.
Finally, once the tileset is created, you can add it as a layer to any of your maps as with any other tileset. Please refer to this guide for more information.
From your data warehouse
You can also create tilesets directly from your data warehouse using SQL commands. This option is currently only available for BigQuery using any of the available procedures in the Analytics Toolbox. You can create two types of tilesets:
- Simple: choose this type if you are wanting to visualize the original features of the dataset. To get started, find here a set of examples on how to create simple tilesets using the
- Aggregation: choose this type when your dataset is composed of points and you want to see them aggregated. To get started, find here a set of examples on how to create aggregation tilesets using the
Both of the aforementioned procedures take as input a SQL query specifying the data that you want to transform into a tileset. If the case of Data Observatory datasets, you can use the example query provided as part of the Access in functionality (learn more here). We recommend that in order to minimize the cost and avoid reaching BigQuery internal limits, you only include in your input query the data columns that you strictly need for your visualization.