Visualizing Data Observatory datasets

Creating maps

Data Observatory datasets can be visualized from the CARTO Workspace using Builder. You can easily do so by clicking on the Create map action in the subscription’s detail page, available from the Data Observatory section of the Data Explorer:

Or by adding a new Data Observatory source to an existing map:

Those datasets whose size is within platform limits will be visualized in full. Bigger datasets will be applied a spatial filter (a buffer around the centroid of the most populated city of the dataset’s country), but this filter can be modified at your own will through the provided SQL query. These datasets will require a tileset to be visualized in full. Please refer to the Creating Data Observatory tilesets section to learn more.

Choosing a connection

You will be asked to select the connection that will be used to create a map with your Data Observatory subscription. The chosen connection will be used to retrieve the necessary data to add the layer to the map. Currently, CARTO Data Warehouse, BigQuery and Snowflake connections are supported; Redshift and Databricks support is coming soon.

In order to be able to use a Snowflake connection to create a map, the data first needs to be imported into your database. This import process is performed by our engineering team on a request basis.

To request it, go to the subscription’s page, click on the Create map button and choose the desired Snowflake connection. You will be asked to request access to the dataset.

Once we receive your request, we will get in touch with you to coordinate the import process. The data will be imported into a schema called CARTO that will be created in the Snowflake database you have set up in your Snowflake connection. Finally, you will be able to create a map using such connection.

If you would like to access more than one of your Data Observatory subscriptions from your Snowflake database, it is not necessary to request access for each of them individually, as we can import several datasets at once during the same process.

Creating tilesets

Some of the spatial datasets offered in the Data Observatory are massive (a few TB), either due to their global coverage, such as WorldPop or NASADEM, or their fine granularity, such as ACS Sociodemographics at census block group level, and their visualization requires the creation of a tileset.

From the Workspace

When a Data Observatory subscription requires a tileset to be visualized in full, a Create tileset option will be available from the subscription’s page.

Clicking on this option will display a wizard that will guide you through the tileset creation process:

  1. Choose the name and location of the output tileset. Please note that you should have writing permissions in the destination dataset.

Tileset creation through the Workspace interface is currently available for CARTO Data Warehouse, BigQuery, Snowflake, Redshift and PostgreSQL connections. Support for Databricks connections will come soon.

Finally, once the tileset is created, you can add it as a layer to any of your maps as with any other tileset. Please refer to this guide for more information.

Once the tileset creation has been launched, you can check its status using the processing jobs menu available at the top right of the page.

2. Launch tileset creation.

3. Select the attributes (data columns) that will be included in the tileset.

4. Select the zoom range for which the tileset will be created.

We have created a collection of ready-to-use Data Observatory tilesets from public datasets that are directly available in the BigQuery project carto-do-public-tilesets. Visit this page for a gallery of visualizations and the full list of available tilesets.

From your data warehouse

You can also create tilesets directly from your data warehouse using SQL commands. This option is currently only available for BigQuery using any of the available procedures in the Analytics Toolbox. You can create two types of tilesets:

  • Simple: choose this type if you are wanting to visualize the original features of the dataset. To get started, find here a set of examples on how to create simple tilesets using the CREATE_TILESET procedure.

  • Aggregation: choose this type when your dataset is composed of points and you want to see them aggregated. To get started, find here a set of examples on how to create aggregation tilesets using the CREATE_POINT_AGGREGATION_TILESET procedure.

Both of the aforementioned procedures take as input a SQL query specifying the data that you want to transform into a tileset. If the case of Data Observatory datasets, you can use the example query provided as part of the Access in functionality (learn more here). We recommend that in order to minimize the cost and avoid reaching BigQuery internal limits, you only include in your input query the data columns that you strictly need for your visualization.

Last updated