# Importing rasters with raster-loader Raster data plays a critical role in geospatial applications, enabling the visualization and analysis of spatial patterns, trends, and relationships. CARTO provides end-to-end support for raster data, allowing users to store, analyze, and render raster datasets directly in their cloud data warehouses. This documentation outlines the complete process of preparing raster data, ensuring it meets CARTO’s specifications, and importing it into supported cloud data warehouses, including Google BigQuery, Snowflake, and Databricks using the [`raster-loader`](https://pypi.org/project/raster-loader/0.2.0/) python utililty. The tables that result from importing raster files with `raster-loader` follow the [**Raquet**](https://github.com/CartoDB/raquet) specification.
## Preparing your raster data Before importing raster data into your cloud data warehouse using CARTO `raster-loader` , it must meet specific format, tiling, and projection requirements to ensure compatibility with CARTO. **Required specifications:** * **Format:** Cloud Optimized GeoTIFF (COG) * **Tiling Schema:** Google Maps Tiling Schema ### Setting up the environment To process and prepare raster data efficiently, certain software dependencies must be installed, particularly **Python** and **GDAL** for geospatial data manipulation. **Check Python installation** Ensure that Python 3 is installed on your system. Run the following command to verify: ```bash python3 --version ``` If Python is not installed, download and install it from [Python.org](https://www.python.org/). **Set up a virtual environment (Recommended)** Using a virtual environment helps manage dependencies and prevents conflicts with system-wide packages. Run the following commands to create and activate a virtual environment: ```bash python3 -m venv carto_raster_env source carto_raster_env/bin/activate # For Linux/macOS carto_raster_env\Scripts\activate # For Windows ``` **Install GDAL (Python bindings):** Once the virtual environment is activated, install GDAL using `pip`: ```bash pip install GDAL ``` This will install the necessary GDAL bindings for Python, allowing you to manipulate and process raster data. ### Inspecting raster metadata Use `gdalinfo` to get information and metadata about your file that will be useful for debugging and preparation. ```bash gdalinfo raster.tif ``` ### NODATA values NODATA values represent missing or invalid data within a raster file which can be inspected using `gdalinfo` command. These values are automatically ignored by CARTO to ensure accurate analysis and visualization. Defining NODATA values properly is crucial, as they will not be displayed on the map and will be excluded from analytical queries. ```bash gdal_translate -a_nodata 0 input.tif output_nodata.tif ``` ### Reprojecting raster data It is advisable to reproject your raster to EPSG:4326 before converting it into a Cloud Optimized GeoTiff. This is easily done with `gdalwarp` ```bash gdalwarp -wm 1024 -multi -s_srs EPSG:5070 -t_srs EPSG:4326 input.tif output_4326.tif ``` ### Generating a Cloud Optimized Geotiff CARTO requires that raster files are in the Cloud Optimized GeoTIFF (COG) format. For this, use GDAL's [`gdalwarp`](https://gdal.org/programs/gdalwarp.html#gdalwarp) tool to transform our raster to this projection using the Google Maps tiling scheme, as per below example: ```bash gdalwarp -of COG \ -co TILING_SCHEME=GoogleMapsCompatible \ -co COMPRESS=DEFLATE -co OVERVIEWS=IGNORE_EXISTING -co ADD_ALPHA=NO \ -co RESAMPLING=NEAREST \ raster_4326.tif raster_cog.tif ``` #### Resampling methods Reprojecting a raster during the conversion to COG can introduce artifacts, especially when interpolating pixel values. The `RESAMPLING=NEAREST` method is used to avoid these distortions by assigning the value of the nearest pixel rather than interpolating between multiple pixels. This method is particularly useful for categorical data (such as land cover classifications), where preserving exact pixel values is essential to maintain data integrity. Other resampling methods exist, such as `BILINEAR` or `CUBIC`, but they are better suited for continuous data like elevation models or temperature maps, where smooth transitions between pixels are desirable. #### Overviews Raster overviews, also known as pyramids, are lower-resolution copies of the original raster data stored within the file. Overviews allow CARTO to display raster data efficiently at different zoom levels by loading a lower-resolution version of the raster when the user is zoomed out, reducing processing and loading times. By including the `-co OVERVIEWS=IGNORE_EXISTING` option, you ensure that overviews are generated correctly, allowing CARTO to request the appropriate resolution dynamically based on the zoom level. Without overviews, CARTO would have to load the full-resolution raster even at distant zoom levels, leading to slow rendering performance. #### Alpha band In terms of adding an alpha band to your COG, `-co ADD_ALPHA=NO` is the safer general option. However, in some cases it's advisable to convert your NO\_DATA values to an alpha band and use `-co ADD_ALPHA=YES` instead. {% hint style="info" %} `gdalwarp` supports many other options when creating a COG. Take a look at the [**complete documentation of the COG driver**](https://gdal.org/drivers/raster/cog.html#creation-options) to see all of them. {% endhint %} *** ## Importing raster data into your data warehouse This section outlines the available methods for importing raster data into CARTO when using Google BigQuery, Snowflake and Databricks as your data warehouse. The method you choose will depend on the file size, the level of control required during the upload process and the data warehouse provider. Available import methods: * [Import interface](#using-carto-import-interface): Best suited for smaller raster files (≤5GB) where advanced configuration is not necessary. Only for BigQuery and Snowflake. * [Raster Loader](#using-carto-raster-loader-python-utility): Recommended for larger raster files or cases where more control is needed during the upload process. {% hint style="warning" %} Please note that importing files —including rasters—into **Databricks** is not supported via the CARTO import interface. {% endhint %} ### Using CARTO import interface Recommended for files smaller than 5GB. This is the most straightforward approach but has limitations in terms of file size and complexity. Currently, it is only supported for **Google BigQuery** and **Snowflake**—**not supported for Databricks**. ### Using CARTO Raster Loader The [CARTO Raster Loader](https://raster-loader.readthedocs.io/en/latest/user_guide/cli.html#using-the-raster-loader-cli) is a Python utility that can import a COG raster file to **Google BigQuery**, **Snowflake** and **Databricks** as a CARTO raster table. #### Installation The `raster-loader` library can be installed from `pip` like: ```bash pip install raster-loader ``` Installation within a virtual environment is highly recommended. ```bash python -m venv rasterenv source rasterenv/bin/activate pip install raster-loader ``` #### **Authentication** Before uploading rasters to BigQuery, ensure you have the **gcloud SDK installed**. If not, install it from . Then, authenticate with Google Cloud by executing the following command: ```bash gcloud auth application-default login ``` For Snowflake and Databricks, authentication is performed during the uploading command itself. #### **Uploading raster data to BigQuery** Before you can upload a raster file, you need to have set up the following in BigQuery: * A GCP project * A BigQuery dataset To use the bigquery utilities, use the `carto bigquery` command. Find a complete guide and reference at the [**Raster Loader documentation**](https://raster-loader.readthedocs.io/en/latest/user_guide/cli.html#using-the-raster-loader-with-bigquery)**.** The basic command to upload a COG to BigQuery as a CARTO raster table is: ```bash carto bigquery upload \ --file_path /path/to/raster.tif \ --project my-gcp-project \ --dataset my-bigquery-dataset \ --table my-table \ --overwrite ``` #### **Uploading raster data to Snowflake** Before you can upload a raster file, you need to have set up the following in Snowflake: * A Snowflake account * A Snowflake database * A Snowflake schema To use the snowflake utilities, use the `carto snowflake` command. Find a complete guide and reference at the [**Raster Loader documentation**](https://raster-loader.readthedocs.io/en/latest/user_guide/cli.html#using-the-raster-loader-with-snowflake)**.** The basic command to upload a COG to Snowflake as a CARTO raster table is: ```bash carto snowflake upload \ --file_path /path/to/raster.tif \ --database my-database \ --schema my-schema \ --table my-table \ --account my-snowflake-account \ --username my-username \ --password my-password \ --overwrite ``` #### **Uploading raster data to Databricks** Before you can upload a raster file, you need to have set up the following in Databricks: * A Databricks server hostname * A Databricks cluster id * A Databricks token To use the databricks utilities, use the `carto databricks` command. Find a complete guide and reference at the [**Raster Loader documentation**](https://raster-loader.readthedocs.io/en/latest/user_guide/cli.html#using-the-raster-loader-with-databricks)**.** The basic command to upload a COG to Databricks as a CARTO raster table is: ``` carto databricks upload \ --file_path /path/to/my/raster/file.tif \ --catalog my-databricks-catalog \ --schema my-databricks-schema \ --table my-databricks-table \ --server-hostname my-databricks-server-hostname \ --cluster-id my-databricks-cluster-id \ --token my-databricks-token \ --overwrite ``` #### Advanced options **Options for raster bands** By default, Raster Loader will upload the first band in the raster file, but it's possible to specify a different band with a command like: ```bash --band 2 ``` Uploading multiple bands, with (optionally) custom names is supporting by concatenating both the bands to include and the label if required. ```bash --band 1 \ --band 2 \ --band_name red \ --band_name green ``` **Options for very large files** For large raster files, you can use the `--chunk_size` flag to specify the number of rows to upload at once. The default chunk size is **1000** rows. For example, the following command uploads the raster in chunks of 2000 rows: ```bash --chunk_size 2000 ``` For large raster files, you also have the option to enable the `--compress` flag which enables compression of the band data using gzip compression which can significantly reduce storage size. ```bash --compress ``` ## Analyzing and visualizing raster data Once your raster data is stored in your cloud data warehouse, you can analyze and visualize it using the CARTO platform. Below are the available options: * [**Using Analytics Toolbox for Raster Analysis**](https://docs.carto.com/data-and-analysis/analytics-toolbox-overview)**:** Perform advanced spatial analysis on raster data using SQL-based functions within your data warehouse using Raster modules for [BigQuery](https://docs.carto.com/data-and-analysis/analytics-toolbox-for-bigquery/sql-reference/raster), and [Snowflake](https://docs.carto.com/data-and-analysis/analytics-toolbox-for-snowflake/sql-reference/raster). * [**Processing Raster Data with Workflows**](https://docs.carto.com/carto-user-manual/workflows/components/raster-operations)**:** Build low-code geospatial analysis pipelines that integrate raster data with other datasets and processing tools for BigQuery and Snowflake. * [**Visualizing Raster Data in CARTO Builder**](https://docs.carto.com/carto-user-manual/maps/layers/raster)**:** Add and style raster layers directly in CARTO Builder for interactive map exploration and presentation. * [**Rendering Raster Data with CARTO and Deck.gl**](https://docs.carto.com/carto-for-developers/carto-for-deck.gl)**:** Use Deck.gl for high-performance client-side rendering of raster layers in custom applications. For more details, visit the linked sections or explore CARTO’s documentation. {% hint style="warning" %} **Databricks** **does not yet support analytics capabilities** either as part of the Analytics Toolbox or Workflows. Only visualization is supported. {% endhint %} --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.carto.com/carto-user-manual/data-explorer/importing-data/importing-rasters.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.