Analytics Toolbox for Snowflake

Analytics Toolbox for Snowflake

Tilesets

Tilesets enable to process and visualize very large spatial datasets (millions or even billions of rows) stored in your Snowflake database.

How it works

The tiler procedures will process your data and create a complete tileset out of it. All the map tiles for the specified zoom range will be stored in a Snowflake table in GeoJSON format. Each individual tile is a row in this table, with the tile coordinates and the corresponding geometry data stored in different columns:

Row z x y data
1 16 45340 24576 {“geometry”:{“coordinates”:[-138.50516208988,…
2 16 45292 24576 {“geometry”:{“coordinates”:[-164.87444768784,…

Visualizing and publishing your tilesets is straight-forward using Builder, the map making tool integrated into the CARTO Workspace.

The integration of tilesets with custom web map applications is also possible with CARTO Maps API, which will connect to Snowflake using your connection’s credentials to fetch and serve the tiles.

Tileset types and procedures

We currently support the creation of aggregation tilesets, which encode aggregations over the input features, and simple tilesets, which instead encode all the input features as is. In a nutshell, you should use simple tilesets for visualizing a dataset of world rivers, but use an aggregation tileset to visualize a heatmap of the trees distribution in New York City.

We provide the following set of procedures to create tilesets:

  1. carto.CREATE_SIMPLE_TILESET

    • This procedure creates a simple tileset. You should use it if you have a dataset with any geography type (point, line, or polygon) and you want to visualize it at an appropriate zoom level.
    • The geographies will be represented exactly as stored in BigQuery, which means that if they are too small to be visible at a certain zoom level they won’t be included in the tiles at that zoom level.
    • The values associated with each feature are the same as the ones available in the source dataset.
  2. carto.CREATE_POINT_AGGREGATION_TILESET

    • Use this procedure if you have a point dataset (or anything that can be converted to points, such as polygon centroids) and you want to see it aggregated.
    • The points will be aggregated into cells. Each feature or cell represents all the points that fall under it, so the associated properties available for visualization are generated by aggregating the values in the source dataset.

Benefits

The tiler is:

  • Convenient – It can be run directly as SQL commands in Snowflake. The data never leaves Snowflake so you won’t have to worry about security and additional ETLs.
  • Fast – It benefits from the massive scalability capabilities of Snowflake and can process hundreds of millions of rows in a few minutes.
  • Scalable – This solution works well for 1M points or 100B points.
  • Cost-effective – Since Snowflake separates storage from computing, the actual cost of hosting these tilesets is very low. Additionally, since the tiling process runs on-demand, you’ll only pay for that processing and you won’t need to have a warehouse available 24/7.