Release Notes

October 28th, 2024 (v1.15.0)

Module lds

Improvement

• Added async batch routing services support for TomTom, providing a 2x-3x preformance improvement in processing speed.

Module statistics

New

• New TIME_SERIES_CLUSTERING procedure, which expects a table containing time series (indexed by a time step column and a unique series identifier) and can group the time series in as many clusters as the user requests.

• New SPACETIME_HOTSPOTS_CLASSIFICATION procedure. This procedure is designed to analyze spatio-temporal data in order to identify and categorize locations based on their hotspot or coldspot status over time. Utilizing z-score values generated by the Space-Time Getis-Ord function.

September 24th, 2024 (v1.14.0)

Module tiler

New

• Add tile_resolution to all tileset creation options. This new setting allows controlling the size of the tiles when rendered in a map. Higher values produce larger tiles.

Module statistics

Fix

• Update python version from 3.8 to 3.9 on CREATE_SPATIAL_COMPOSITE_SUPERVISED and CREATE_SPATIAL_COMPOSITE_UNSUPERVISED procedures, as version 3.8 is getting deprecated.

August 23rd, 2024 (v1.13.1)

Module statistics

Fix

• Fixed an issue with GETIS_ORD_SPACETIME_H3 and GETIS_ORD_SPACETIME_QUADBIN procedures, that were returning wrong results when “time_freq” was passed as lower case.

July 2nd, 2024 (v1.13.0)

Module raster

New

• Adjustments to use only full-resolution data when the raster table has overviews (blocks of resolution smaller than the block_resolution stated in the metadata)

• We can now specify per-band no-data values for raster datasets. Rasters bands also now have a ‘colorInterp’ metadata attribute with possible values of red, green, blue, alpha or unknown.

Improvement

• Replaced some of the IFF logic checks for NODATA values with NULLIF and arithmetic operations. This allows Snowflake's planner to better optimize queries and improves performance.

• Optimized raster procedures (RASTER_VALUES, RASTER_AGG_VALUES) to manage bigger data sources.

• New options in RASTER_AGG_VALUES: groupby_vector_columns and groupby_raster_columns to have finer control over the aggregation.

Module import

Fix

• Import functions in AT Gateway now raise an exception when there is an error. This replaces the previous status message.

May 22nd, 2024 (v1.12.0)

Module h3

Improvement

• Avoid memory limit exceeded in H3_POLYFILL_TABLE

Module statistics

Fix

• Include all numeric types for unsupervised composite score validation in CREATE_SPATIAL_COMPOSITE_UNSUPERVISED.

Module http_request

New

• We have a new module named “http_request”. It allows making requests to external endpoints through the function HTTP_REQUEST

Module import

New

• We have a new module named “import”. It allows importing tables for a given URL containing a file in any of the formats supported by the CARTO Import API. The procedure in charge of this import is IMPORT_URL.

April 18th, 2024 (v1.11.0)

Module raster

New

• Added new RASTER_VALUES and RASTER_AGG_VALUES procedures:

  • RASTER_VALUES obtains pixel values from a raster where it overlaps with an optional vector data input. It accepts SQL expressions, including operations between bands.

  • RASTER_AGG_VALUES obtains aggregated values from a raster's pixel values where it intersects with a vector data input. It accepts any SQL aggregation expression.

Module clustering

Fix

• Update ST_CLUSTERKMEANS in order to avoid generating fewer clusters than requested. The maximum number of clusters now will be equal to the number of different points passed.

Module h3

Improvement

• Functions in the h3 module now use Snowflake's native H3 functions internally. This change improves performance as it leverages native Snowflake optimization capabilities.

Module lds

Fix

• GEOCODE_TABLE not working properly for “geom_column” parameters value different from “geom”.

March 21st, 2024 (v1.10.0)

Module lds

New

• All external functions (including those used by the procedures in the lds module) now use a single, unified endpoint. This also simplifies the manual installation of the Analytics Toolbox.

• GEOCODE_TABLE now uses async batch geocoding with TomTom, boosting the geocoding speed considerably. Also matching by rows is applied ensuring that the address that reaches the TomTom endpoints is as similar as possible as the table value.

Module statistics

Fix

• Updated output table schema for the procedures CREATE_SPATIAL_COMPOSITE_SUPERVISED and CREATE_SPATIAL_COMPOSITE_UNSUPERVISED.

February 16th, 2024 (v1.9.0)

Module raster

New

• Add RASTER_VALUE and RASTER_VALUE_TABLE procedures: work with CARTO raster tables, with support for custom operations, including multi-band and aggregation expressions.

• Raster loader v0.5.0: adding support for Snowflake.

Module statistics

Fix

• Fixes in CREATE_SPATIAL_COMPOSITE_UNSUPERVISED, GETIS_ORD_SPACETIME procedures.

• Add warning to Moran's I procedures when no cells have neighbors.

Module lds

Fix

• Sanitize queries in GEOCODE_TABLE so addresses with special characters are geocoded.

Module quadbin

Fix

• Fix QUADBIN_TOPARENT so it can work with views.

Module random

Fix

• Fix ST_GENERATEPOINTS to accept column names and increase performance.

January 18th, 2024 (v1.8.0)

Module quadbin

New

• New QUADBIN_DISTANCE function that returns the distance (number of cells) between two quadbin cells.

Fix

• Improved precision when calculating cell indexes from latitudes close to the poles (±90º).

• QUADBIN functions now return NULL when NULL parameters are passed.

Module transformations

New

• New ST_POINTONSURFACE function.

Module tiler

Fix

• Fixed a problem that caused CREATE_SIMPLE_TILESET procedure crashing for zooms higher than 13.

• Improved tile precision when creating tilesets from geographies containing points with latitudes close to the poles (±90º) using CREATE_SIMPLE_TILESET or CREATE_POINT_AGGREGATION_TILESET.

Module data

Fix

• Fixed an issue when using AVG aggregation, that was not correctly computed when “data_query” contained NULL values.

• Output tables are now correctly prepared to work with Snowflake's Search Optimization Service.

Module lds

Fix

• Output tables are now correctly prepared to work with Snowflake's Search Optimization Service.

Module statistics

New

• New CREATE_SPATIAL_COMPOSITE_SUPERVISED, CREATE_SPATIAL_COMPOSITE_UNSUPERVISED and CRONBACH_ALPHA_COEFFICIENT procedures.

Fix

• Replaced “index” as column name in the output tables with “h3” or “quadbin”, for easier use of the results in Builder for Moran’s, GETIS_ORD_* and GWR_GRID procedures.

• Output tables are now correctly prepared to work with Snowflake's Search Optimization Service.

December 21, 2023 (v1.7.1)

Module statistics

Fix

• Change output types of GWR_GRID from Variant to the corresponding ones

• Added EXECUTE AS CALLER option to GETIS_ORD_SPACETIME_QUADBIN and GETIS_ORD_SPACETIME_H3 procedures. This fixes a previous issue when trying to use the procedures with a role different than the owner.

Module retail

New

• Introduced the new retail module for Snowflake, aimed to solve specific retail analytics use cases.

• New COMMERCIAL_HOTSPOTS procedure, which can be used to locate hotspot areas by calculating a combined Getis-Ord Gi* statistic over several variables.

November 13, 2013 (v1.7.0)

Module lds

New

• Add SETUP procedure. This procedure allows configuring credentials and endpoints for a custom installation of the Analytics Toolbox. Check this documentation for more information about this topic.

• Replace LDS_QUOTA_INFO by GET_LDS_QUOTA_INFO function. This new function allows checking your currently consumed quota with a single query, without the need to store the result as a table.

Module statistics

New

• Added LOCAL_MORANS_I_H3, LOCAL_MORANS_I_QUADBIN procedures. These procedures allows computing the spatial autocorrelation locally.

• Added GWR_GRID procedure. This procedure allows computing a Geographically Weighed Regression on an spatial index grid.

• Added GETIS_ORD_SPACETIME_H3, GETIS_ORD_SPACETIME_QUADBIN procedures. These procedures allow computing the space temporal Getis-Ord Gi* statistic for each H3 index and each datetime timestamp according to the method described in this paper.

Fix

• Added missing EXECUTE AS CALLER command in the procedures of this module, which was causing an error when the role that ran the procedure was not the owner.

October 13, 2023 (v1.6.1)

Module lds

Improvement

• New batch service for CREATE_ISOLINES. Used automatically by the procedure. No changes required.

September 8, 2023 (v1.6.0)

Module statistics

New

• The new version of the following procedures now expects a table as input, overcoming previous performance limitations:

  • GETIS_ORD_H3

  • GETIS_ORD_QUADBIN

  • MORANS_I_H3

  • MORANS_I_QUADBIN

This is a breaking change from the previous version of the functions that expected an ARRAY as input. Please make sure that you make the necessary changes if you are using this functions.

Module lds

New

• We have added optional parameters: api_base_url, lds_token to every function/procedure

Fix

• Fix CREATE_ROUTES fails if geom_column is not named geom

July 14, 2023 (v1.4.0)

Module lds

New

• We are now allowing users to send any of the parameter options supported by the isolines service provider in CREATE_ISOLINES.

• We have improved our logging of results in GEOCODE_TABLE so more information about processed and geocoded rows is provided to the user. Also the new option “carto_force_geocode” allows the re-geocoding of already geocoded rows.

June 6, 2023 (v1.3.0)

Module lds

New

• We have created a new procedure named CREATE_ROUTES, which calculates the routes (returned as linestrings) between a set of given origins and destinations (points) in a query. It supports different transport modes and advanced options as the other functions of the lds module.

May 5, 2023 (v1.2.0)

Module statistics

New

We have added a new module named “statistics” in the Analytics Toolbox for Snowflake which has a set of geostatistics functions to compute the Getis-Ord Gi* statistic and the Moran’s I spatial autocorrelation:

• Getis-Ord Gi* functions:

  • GETIS_ORD_QUADBIN

  • GETIS_ORD_H3

  • P_VALUE

• Moran’s I functions:

  • MORANS_I_QUADBIN

  • MORANS_I_H3

Module h3

New

We have incorporated two new functions to the “h3” module of the Analytics Toolbox, namely H3_CENTER, which returns the center of an H3 cell as a GEOGRAPHY point, and H3_RESOLUTION which returns the H3 cell resolution as an integer.

Module transformations

New

We have added support for the ST_CONVEXHULL function, which computes the convex hull of the input geographies (the smallest convex geography that covers all the input ones)

April 4, 2023 (v1.1.1)

Module lds

Improvement

• We have improved the performance in CREATE_ISOLINES, benefiting now from parallelizing requests to the API.

Change

• We have updated the LDS module to be aligned with the new usage quota model, working as an annual budget and combining consumption across all supported LDS services.

March 7, 2023 (v1.1.0)

Module lds

New

• We have added object type support for the options parameter also maintaining string type support as until now in GEOCODE_TABLE, GEOCODE_REVERSE_TABLE, CREATE_ISOLINES, , GEOCODE, GEOCODE_REVERSE, and ISOLINE.

January 31, 2023 (v1.0.0)

Module tiler

Improvement

• Update default zoom values: geo, quadbin, h3 (12, 12, 6).

• Fix spatial index tiler with empty properties.

December 22, 2022

Module lds

Improvement

• We have added the possibility to configure more options as parameters when executing the functions to CREATE_ISOLINES. These new options, which depend on the LDS service provider, allow the user to configure more transportation modes such as truck or bike, the possibility of specifying departure or arrival times allowing the creation of reverse isolines, and other options like different routing modes.

• Added new confidence/relevance metadata to the results of the geocoding function GEOCODE_TABLE.

Module quadbin

New

• Quadbin/quadkey conversion functions have been added to ease the conversion from quadbin to quadkey and from quadkey to quadbin indexes.

Module h3

• The H3_BOUNDARY function for invalid geographies has been fixed.

November 1, 2022

All modules

Feature

• Create release workflows

October 24, 2022

Module processing

Fix

• Prevent error in ST_VORONOIPOLYGONS, ST_VORONOILINES, ST_VORONOIPOLYGONS, ST_DELAUNAYLINES when points where too close together by rounding input coordinates to 5 decimal places

October 7, 2022

Module clustering

Feature

• Move ST_CLUSTERKMEANS function to core.

Module random

Feature

• Move ST_GENERATEPOINTS function to core.

October 4, 2022

Module lds

Improvement

• CREATE_ISOLINES output is ordered by geometry to be compliant with Snowflake Search Optimization Service.

September 30, 2022

Module lds

Improvement

• Rename LDS_API_URL to API_BASE_URL.

September 28, 2022

Module data

Improvement

• Optimize DATAOBS_ENRICH_GRID, DATAOBS_ENRICH_GRID_RAW when the enrich data is a grid compatible with the input.

• Remove support for s2, geohash in all enrich procedures.

• Optimize performance of all enrich result tables by ordering or clusterizing them.

September 19, 2022

Module tiler

Improvement

• Set version field in tilers metadata compliant with AT version.

August 11, 2022

Module tiler

Improvement

• Unify extra_metadata into metadata in tiler metadata.

July 13, 2022

Module data

Feature

• Add quadbin support to module.

July 12, 2022

Module tiler

Fix

• Fix quoting and letter case in spatial index tiler.

July 8, 2022

Module tiler

Fix

• Fix wrong uppercase in spatial index tiler.

Improvement

• Use core functions to convert H3 from int to string.

July 7, 2022

Module h3

Fix

• Correctly handle large polygons in H3_POLYFILL.

• Fixed wrong uppercase for quadbin and h3 tile ids

June 24, 2022

Module quadbin

Feature

• Add QUADBIN_BBOX function.

• Add QUADBIN_BOUNDARY function.

• Add QUADBIN_CENTER function.

• Add QUADBIN_FROMGEOGPOINT function.

• Add QUADBIN_FROMLONGLAT function.

• Add QUADBIN_FROMZXY function.

• Add QUADBIN_ISVALID function.

• Add QUADBIN_KRING function.

• Add QUADBIN_KRING_DISTANCES function.

• Add QUADBIN_POLYFILL function.

• Add QUADBIN_RESOLUTION function.

• Add QUADBIN_SIBLINGS function.

• Add QUADBIN_TOCHILDREN function.

• Add QUADBIN_TOPARENT function.

• Add QUADBIN_TOZXY function.

June 23, 2022

Module tiler

Improvement

• Add CREATE_SPATIAL_INDEX_TILESET procedure.

June 15, 2022

Module data

Improvement

• ENRICH_ procedures require all the enrich data query rows to be of the same geometry kind (points/lines/polygons) .

• Various optimizations for performance and preventing out-of-memory errors

Improvement

• Errors used to be reported with a procedure result message. Now they raise exceptions.

May 19, 2022

Module lds

Improvement

• Add optional language parameter to GEOCODE_REVERSE function.

May 6, 2022

Module data

Improvement

• Errors used to be reported with a procedure result message. Now they raise exceptions.

April 28, 2022

Module data

Feature

• Allow using tables as input, not just queries.

• Add in place enrichment.

April 21, 2022

Module tiler

Improvement

• Add support for non-uppercase column names

April 20, 2022

Module tiler

Improvement

• Errors used to be reported with a procedure result message. Now they raise exceptions.

April 7, 2022

Module transformations

Feature

• Add ST_BUFFER function.

March 25, 2022

Module tiler

Improvement

• Run the simple tiler in a single query.

• Add fraction_dropped_per_zoom to the metadata.

Fix

• Fix empty properties causing errors.

• Fix drop_fraction_as_needed in the simple tiler.

• Fix problems with GeometryCollections received after ST_Simplify.

March 21, 2022

Module transformations

Improvement

• ST_CONCAVEHULL now allows arrays with one/two points as input.

February 24, 2022

Module tiler

Feature

• Add CREATE_SIMPLE_TILESET procedure.

February 18, 2022

Module lds

Feature

• Add GEOCODE_TABLE procedure.

• Add GEOCODE function.

• Add GEOCODE_REVERSE function.

• Add ISOLINE function.

• Add LDS_QUOTA_INFO function.

February 16, 2022

Module tiler

Feature

• Create tiler module.

• Add CREATE_POINT_AGGREGATION_TILESET procedure.

January 10, 2022

Module data

Feature

• Add DATAOBS_SUBSCRIPTIONS procedure.

• Add DATABOS_SUBSCRIPTION_VARIABLES procedure.

Improvement

• Read spatial_catalog tables in source.

December 15, 2021

Module data

Improvement

• Missing subscriptions are reported by name in DATAOBS_ENRICH_* procedures

December 10, 2021

Module random

Improvement

• ST_GENERATEPOINTS now uses a spherically uniform distribution. Previously used to by uniform on projection.

December 3, 2021

Module accessors

Improvement

• Deployment schema “carto” instead of “accessors”.

• Rename ST_ENVELOPE function to ST_ENVELOPE_ARR.

Removed

• Remove VERSION function.

Module constructors

Improvement

• Deployment schema “carto” instead of “constructors”.

Removed

• Remove VERSION function.

Module h3

Improvement

• Deployment schema “carto” instead of “h3”.

• Rename ST_ASH3 function to H3_FROMGEOGPOINT.

• Rename LONGLAT_ASH3 function to H3_FROMLONGLAT.

• Rename ST_ASH3_POLYFILL function to H3_POLYFILL.

• Rename ST_BOUNDARY function to H3_BOUNDARY.

• Rename ISVALID function to H3_ISVALID.

• Rename COMPACT function to H3_COMPACT.

• Rename UNCOMPACT function to H3_UNCOMPACT.

• Rename TOPARENT function to H3_TOPARENT.

• Rename TOCHILDREN function to H3_TOCHILDREN.

• Rename ISPENTAGON function to H3_ISPENTAGON.

• Rename DISTANCE function to H3_DISTANCE.

• Rename KRING function to H3_KRING.

• Rename KRING_DISTANCES function to H3_KRING_DISTANCES.

• Rename HEXRING function to H3_HEXRING.

Removed

• Remove VERSION function.

Module measurements

Improvement

• Deployment schema “carto” instead of “measurements”.

Removed

• Remove ST_ANGLE, already present in Snowflake.

• Remove ST_AZIMUTH, already present in Snowflake.

• Remove VERSION function.

Module placekey

Improvement

• Deployment schema “carto” instead of “placekey”.

• Rename H3_ASPLACEKEY function to PLACEKEY_FROMH3.

• Rename PLACEKEY_ASH3 function to PLACEKEY_TOH3.

• Rename ISVALID function to PLACEKEY_ISVALID.

Removed

• Remove VERSION function.

Module processing

Improvement

• Deployment schema “carto” instead of “processing”.

Removed

• Remove VERSION function.

Module s2

Improvement

• Deployment schema “carto” instead of “s2”.

• Rename ID_FROMHILBERTQUADKEY function to S2_FROMHILBERTQUADKEY.

• Rename HILBERTQUADKEY_FROMID function to S2_TOHILBERTQUADKEY.

• Rename LONGLAT_ASID function to S2_FROMLONGLAT.

• Rename ST_ASID function to S2_FROMGEOGPOINT.

• Rename ST_BOUNDARY function to S2_BOUNDARY.

Removed

• Remove VERSION function.

Module transformations

Improvement

• Deployment schema “carto” instead of “transformations”.

Removed

• Remove VERSION function.

Module clustering

Improvement

• Deployment schema “carto” instead of “clustering”.

Removed

• Remove VERSION function.

Module data

Improvement

• Deployment schema “carto” instead of “data”.

Removed

• Remove VERSION function.

Module random

Improvement

• Deployment schema “carto” instead of “random”.

Removed

• Remove VERSION function.

November 25, 2021

Module clustering

Improvement

• Deployment schema “carto” instead of “clustering”.

• Remove VERSION function.

Module data

Improvement

• Deployment schema “carto” instead of “data”.

• Remove VERSION function.

Module random

Improvement

• Deployment schema “carto” instead of “random”.

• Remove VERSION function.

November 24, 2021

Module data

Feature

• Add DATAOBS_ENRICH_POINTS procedure.

• Add DATAOBS_ENRICH_POINTS_RAW procedure.

• Add DATAOBS_ENRICH_POLYGON procedure.

• Add DATAOBS_ENRICH_POLYGON_RAW procedure.

• Add DATAOBS_ENRICH_GRID procedure.

• Add DATAOBS_ENRICH_GRID_RAW procedure.

November 5, 2021

Module data

Changes

• Fields named dimension, total, intersection and input_area are now __carto_dimension, __carto_total and __carto_intersection and __carto_input_area. Also the column _carto_enrichment_ is now __carto_enrichment. This affects all the raw enrichment procedures: ENRICH_POINTS_RAW, ENRICH_POLYGONS_RAW, ENRICH_GRID_RAW.

Fix

• User provided queries can now have columns named dimension, total, intersection, input_area, _nonglobal, which could have collided previously with internal columns. All internal columns are now prefixed with __carto_. This affects all the enrichment procedures: ENRICH_POINTS, ENRICH_POLYGONS, ENRICH_GRID, ENRICH_POINTS_RAW, ENRICH_POLYGONS_RAW, ENRICH_GRID_RAW.

November 4, 2021

Module data

Feature

• Create data module.

• Add VERSION function.

• Add ENRICH_POINTS procedure.

• Add ENRICH_POINTS_RAW procedure.

• Add ENRICH_POLYGON procedure.

• Add ENRICH_POLYGON_RAW procedure.

• Add ENRICH_GRID procedure.

• Add ENRICH_GRID_RAW procedure.

September 22, 2021

Module h3

Feature

• Add KRING_DISTANCES function.

Improvement

• Review HEXRING, KRING functions.

September 14, 2021

Module s2

Changes

• Compute ST_BOUNDARY from WKT.

August 24, 2021

Module h3

Fix

• Support GEOMETRYCOLLECTION from ST_ASH3_POLYFILL.

June 2, 2021

Module h3

Improvement

• Reduce bundle size for every function.

May 26, 2021

Module processing

Feature

• Create processing module.

• Add ST_VORONOIPOLYGONS function.

• Add ST_VORONOILINES function.

• Add ST_DELAUNAYPOLYGONS function.

• Add ST_DELAUNAYLINES function.

• Add ST_POLYGONIZE function.

• Add VERSION function.

May 24, 2021

Module clustering

Feature

• Create clustering module.

• Add VERSION function.

• Add ST_CLUSTERKMEANS function.

Module random

Feature

• Create random module.

• Add ST_GENERATEPOINTS function.

• Add VERSION function.

May 21, 2021

Module accessors

Feature

• Create accessors module.

• Add ST_ENVELOPE function.

• Add VERSION function.

May 20, 2021

Module constructors

Feature

• Create constructors module.

• Add ST_BEZIERSPLINE function.

• Add ST_MAKEELLIPSE function.

• Add ST_MAKEENVELOPE function.

• Add ST_TILEENVELOPE function.

• Add VERSION function.

Module measurements

Feature

• Create measurements module.

• Add ST_ANGLE function.

• Add ST_AZIMUTH function.

• Add ST_MINKOWSKIDISTANCE function.

• Add VERSION function.

Module transformations

Feature

• Create transformations module.

• Add ST_CENTERMEAN function.

• Add ST_CENTERMEDIAN function.

• Add ST_CENTEROFMASS function.

• Add ST_CONCAVEHULL function.

• Add ST_DESTINATION function.

• Add ST_GREATCIRCLE function.

• Add ST_LINE_INTERPOLATE_POINT function.

• Add VERSION function.

April 16, 2021

Module placekey

Feature

• Create placekey module.

• Add H3_ASPLACEKEY function.

• Add PLACEKEY_ASH3 function.

• Add ISVALID function.

• Add VERSION function.

April 12, 2021

Module s2

Feature

• Create s2 module.

• Add ID_FROMHILBERTQUADKEY function.

• Add HILBERTQUADKEY_FROMID function.

• Add LONGLAT_ASID function.

• Add ST_ASID function.

• Add ST_BOUNDARY function.

• Add VERSION function.

April 7, 2021

Module h3

Feature

• Create h3 module.

• Add ST_ASH3 function.

• Add LONGLAT_ASH3 function.

• Add ST_ASH3_POLYFILL function.

• Add ST_BOUNDARY function.

• Add ISVALID function.

• Add COMPACT function.

• Add UNCOMPACT function.

• Add TOPARENT function.

• Add TOCHILDREN function.

• Add ISPENTAGON function.

• Add DISTANCE function.

• Add KRING function.

• Add HEXRING function.

• Add VERSION function.