Release notes
Last updated
Was this helpful?
Last updated
Was this helpful?
Module raster
Fix
Added a fix to keep the original geometry column from the vector input in procedure.
Module lds
Fix
Added a fix to have better handling of null values in all procedures.
Module statistics
Improvement
Added a new KMEANS_INIT_METHOD option to procedure.
Module clustering
Fix
Added a fix to prevent the procedure from throwing an error when encountering null values in Geography columns
Module lds
Fix
Module h3
Fix
Module quadbin
Fix
Module raster
Improvement
Module statistics
Fix
Module lds
Improvement
Added a verification of AT Gateway parameters during the SETUP process to ensure availability of resources.
Fix
Module cpg
Fix
Module data
Fix
Module statistics
New
Module lds
Fix
Module data
Fix
Module data
Fix
Module lds
Improvement
Added async batch routing services support for TomTom, providing a 2x-3x preformance improvement in processing speed.
Module cpg
Fix
Module retail
Fix
Module statistics
Fix
Module tiler
New
Module statistics
New
Fix
Module telco
Fix
Module statistics
Fix
Module lds
Fix
Module raster
New
Adjustments to use only full-resolution data when the raster table has overviews (blocks of resolution smaller than the block_resolution stated in the metadata)
We can now specify per-band no-data values for raster datasets. Rasters bands also now have a ‘colorInterp’ metadata attribute with possible values of red, green, blue, alpha or unknown.
Improvement
Module import
Fix
Import functions in AT Gateway now raise an exception when there is an error. This replaces the previous status message.
Module statistics
New
Module telco
New
Module quadbin
Improvement
Module http_request
New
Module import
New
Module raster
New
RASTER_VALUES obtains pixel values from a raster where it overlaps with an optional vector data input. It accepts SQL expressions, including operations between bands.
RASTER_AGG_VALUES obtains aggregated values from a raster's pixel values where it intersects with a vector data input. It accepts any SQL aggregation expression.
Module clustering
Fix
Module lds
Fix
Module lds
New
All external functions (including those used by the procedures in the lds module) now use a single, unified endpoint. This also simplifies the manual installation of the Analytics Toolbox.
Module data
New
Module statistics
Fix
Module random
Fix
Module retail
Fix
Module statistics
Fix
Module quadbin
New
Fix
Improved precision when calculating cell indexes from latitudes close to the poles (±90º).
Module h3
Fix
Module transformations
New
Module tiler
Fix
Module data
Fix
Fixed an issue when using AVG aggregation, that was not correctly computed when “data_query” contained NULL values.
Module lds
Fix
Module raster
Improvement
Module statistics
Fix
Module raster
New
Deprecated
Previous versions of RASTER procedures in beta
Module retail
Improvement
The previous version of the Twin Areas procedures are still available as:
Module cpg
Improvement
The previous version of the Twin Areas procedures are still available as:
Module tiler
Fix
Module data
Improvement
Deprecated
Old quadint/quadkey support in all procedures
Module lds
New
Fix
Module data
Change
Change location of spatial catalog tables: This is only an internal change. Now the catalog is hosted along with the Data Observatory subscriptions. This change decouples the Analytics Toolbox location from the Data Observatory location, providing more flexibility for future developments.
Module statistics
Fix
Module retail
Fix
Fix bugs in kring decay procedures: KRING_DECAY, KRING_DECAY_GRID. These procedure are used internally by other procedures. They work finding cells that are within the K-Ring of each cell in a grid, and assigns a weight based on the distance. This helps model changes in a variable across neighboring cells.\
Module cpg
Change
Module statistics
New
We have added new PCAMIX procedures:
Module retail
Improvement
Module lds
Improvement
Module statistics
New
Added _TABLE procedures for some statistic functions. The _TABLE version of the procedures expect a table as input, overcoming performance limitations of the functions that receive an array as input:
Module retail
Fix
New
Module lds
Fix
Fix GEOCODE_TABLE not iterating through input table rows.
Module cpg
New
Module statistics
Fix
Module h3
New
Improvement
Module quadbin
New
Improvement
Module lds
New
Fix
Module data
New
Fix
Module retail
New
Fix
Module statistics
New
Fix
Module quadbin
Change
Module lds
New
Module statistics
New
Fix
Renamed internal function names to avoid possible conflicts in the future.
Module cpg
Fix
Module tiler
New
Module statistics
New
Set of procedures to compute Area of Applicability of a model built with BigQuery ML:
Fix
We have improved the management of temporary tables in all procedures to compute spatial composites (scores)
Module raster (beta)
Improvement
Store ‘nodata’ value in metadata
Add resolution level to metadata
Fix bands metadata aggregation
Module statistics
New
Module lds
Change
Fix
Module raster (beta)
Improvement
Note
The raster module of the Analytics Toolbox for BigQuery v1.1.0 will work only with raster-loader v0.3.0, which pre-computes and stores metadata in the raster table and homogenizes the generic/quadbin raster table format. So, if you update your Analytics Toolbox version to the most recent v1.1.0, please ensure you are using the raster-loader v0.3.0.
Module cpg
New
Fix
Module lds
Improvement
Raster module (beta)
New
Note that Analytics Toolbox for BigQuery v1.0.0 will work only with raster-loader v0.2.0, to use the latest version of rater-loader please ensure you also have access to the latest version of the Analytics Toolbox
Routing module (beta)
Improvement
Module retail
Fix
Module statistics
New
Module tiler
Improvement
Update default zoom values: geo, quadbin, h3 (12, 12, 6).
Fix spatial index tiler with empty properties.
Module lds
Improvement
Module cpg
New
Module retail
Improvement
Module data
Fix
Module quadbin
New
Module retail
Fix
__KRING_DECAY_GRID was aggregating the value of the moving window pivoting index instead of neighbour indexes.
Module h3
Improvement
Add linestrings and points support to function H3_POLYFILL.
Module retail
New
Add FIND_TWIN_AREAS_WEIGHTED procedure.
All modules
New
Create release workflows
Module s2
New
Module transformations
Fix
Fix ST_BUFFER crashing with geographies close to the poles.
Module tiler
Improvement
Partition tables in spatial index tilesets.
Module cpg
New
Add CUSTOMER_SEGMENTATION_ANALYSIS_DATA procedure.
Add GENERATE_TRADE_AREAS procedure.
Add RUN_CUSTOMER_SEGMENTATION procedure.
Module lds
Fix
CREATE_ISOLINES output is stored in the column geom instead of __iso_geom.
Improvement
CREATE_ISOLINES output is clustered by geometry.
Module processing
Fix
Prevent error in ST_VORONOIPOLYGONS, ST_VORONOILINES, ST_VORONOIPOLYGONS, ST_DELAUNAYLINES when points where too close together by rounding input coordinates to 5 decimal places.
Module clustering
New
Move ST_CLUSTERKMEANS function to core.
Module random
New
Move ST_GENERATEPOINTS function to core.
Module lds
Improvement
Rename LDS_API_URL to API_BASE_URL.
Module retail
Improvement
LINEAR_REG model is now supported.
Module tiler
Improvement
Fix bug in __BUILD_REVENUE_MODEL_DATA_GRID: Output table now contains competitor_count.
Module tiler
Improvement
Set version field in tilers metadata compliant with AT version.
Module tiler
Improvement
Hotfix: set generate_feature_id to false by default due a bug in the generation.
Module data
Improvement
Optimize DATAOBS_ENRICH_GRID, DATAOBS_ENRICH_GRID_RAW when the enrich data is a grid compatible with the input.
Remove support for s2, geohas and quadbinh in all enrich procedures.
Optimize all enrichment result tables by default by clusterizing by the geography or spatial index column.
Module s2
New
Add S2_CENTER function.
Module retail
Improvement
Return shap_values in PREDICT_REVENUE_AVERAGE.
Module tiler
New
Add generate_feature_id parameter in CREATE_SIMPLE_TILESET.
Module retail
Improvement
Index parameter admits any type in PREDICT_REVENUE_AVERAGE.
Index from revenue_model_data is not casted in PREDICT_REVENUE_AVERAGE to reduce query data processing.
Module tiler
Fix
Fixed bug when parsing zoom columns in geography tilers.
Module tiler
Improvement
Unify extra_metadata into metadata in tiler metadata.
Module h3
Fix
Apply make_valid in H3_BOUNDARY.
Module tiler
Fix
Fixed bug in metadata CREATE_SPATIAL_INDEX_TILESET.
Module retail
Improvement
Check fixed model options in BUILD_REVENUE_MODEL procedure.
Module retail
New
Add BUILD_CANNIBALIZATION_DATA procedure.
Add CANNIBALIZATION_OVERLAP procedure.
Module lds
New
Add GEOCODE_TABLE procedure.
Add LDS_QUOTA_INFO procedure.
Add GEOCODE_REVERSE_TABLE procedure.
Add CREATE_ISOLINES procedure.
Module statistics
Improvement
Added HH, LL, LH, HL detection in LOCAL_MORANS_I functions.
Module tiler
Fix
Fixed bug when passing multiline input to CREATE_SPATIAL_INDEX_TILESET.
Module data
New
Add quadbin support to module.
Module retail
New
Add quadbin support
Module tiler
New
Support of quadgrid as alias in CREATE_POINT_AGGREGATION_TILESET.
Improvement
Use core functions to convert H3 from int to string.
Module h3
Fix
Correctly handle large polygons in H3_POLYFILL.
Fixed wrong uppercase for quadbin and h3 tile ids
Module statistics
New
Add SMOOTHING_MRF_QUADBIN procedure.
Add LOCAL_MORANS_I_QUADBIN function.
Add MORANS_I_QUADBIN function.
Add GETIS_ORD_QUADBIN function.
Quadbin support in GWR_GRID function.
Module tiler
Fix
Fixed tablename quoting in BigQuery tiler
Module retail
Fix
Fix quadkey support in FIND_WHITESPACE_AREAS
Module retail
Fix
Fixing bugs when not providing store variables to BUILD_REVENUE_MODEL_DATA
Module quadbin
New
Add QUADBIN_BBOX function.
Add QUADBIN_BOUNDARY function.
Add QUADBIN_CENTER function.
Add QUADBIN_FROMGEOGPOINT function.
Add QUADBIN_FROMLONGLAT function.
Add QUADBIN_FROMZXY function.
Add QUADBIN_ISVALID function.
Add QUADBIN_KRING function.
Add QUADBIN_KRING_DISTANCES function.
Add QUADBIN_POLYFILL function.
Add QUADBIN_RESOLUTION function.
Add QUADBIN_SIBLING function.
Add QUADBIN_TOCHILDREN function.
Add QUADBIN_TOPARENT function.
Add QUADBIN_TOZXY function.
Module data
Fix
Fix BQ inplace enrichment with uppercase column names.
Module tiler
Improvement
Support H3 integer in CREATE_SPATIAL_INDEX_TILESET.
Module data
Fix
Fix order of __KRING_DECAY parameters.
Module tiler
Fix
Fix global stats in CREATE_SPATIAL_INDEX_TILESET.
Improvement
Check input resolution limits in CREATE_SPATIAL_INDEX_TILESET.
Module data
Fix
Report missing subscription geographies in DATAOBS_ENRICH_ procedures.
Module data
Fix
Fix stability issues in GRIDIFY_ENRICH procedure.
Module retail
New
Add store features and competitors stores features into BUILD_REVENUE_MODEL_DATA procedure.
Fix
Fixing bugs calling other routines
Module tiler
New
Add spatial index tileset generator CREATE_SPATIAL_INDEX_TILESET.
Module tiler
Fix
Bug fixing when the geometry early collapse
Module data
New
Add GRIDIFY_ENRICH procedure.
Module retail
New
Add FIND_TWIN_AREAS procedure.
Module data
New
Allow using tables as input, not just queries.
Add in place enrichment.
Allow using tables as input.
Accept NULL as output.
Module retail
Fix
Fix BUILD_REVENUE_MODEL_DATA k-ring buffer computation.
Module transformations
Improvement
ST_CONCAVEHULL now allows arrays with one/two points as input.
Module retail
New
Add COMMERCIAL_HOTSPOTS procedure.
Module statistics
New
Add SMOOTHING_MRF_H3 procedure.
Add SMOOTHING_MRF_QUADKEY procedure.
Module statistics
New
Add VARIOGRAM function.
Add ORDINARY_KRIGING function.
Add ORDINARY_KRIGING_TABLE procedure.
Module h3
New
Add H3_CENTER function.
Add H3_RESOLUTION function.
Module retail
New
Add FIND_WHITESPACE_AREAS procedure.
Module retail
New
Remove features importance tables from BUILD_REVENUE_MODEL procedure.
Module statistics
New
Add LOCAL_MORANS_I_H3 function.
Add LOCAL_MORANS_I_QUADKEY function.
Improvement
Improve MORANS_I_H3 and MORANS_I_QUADKEY implementation.
Module statistics
Fix
Replace STDDEV by STDDEV_POP in GETIS_ORD_H3 and GETIS_ORD_QUADKEY.
Module retail
New
Add BUILD_REVENUE_MODEL_DATA procedure.
Add BUILD_REVENUE_MODEL procedure.
Add PREDICT_REVENUE_AVERAGE procedure.
Module accessors
Improvement
Deployment schema "carto" instead of "accessors".
Removed
Remove VERSION function.
Module constructors
Improvement
Deployment schema "carto" instead of "constructors".
Removed
Remove VERSION function.
Module geohash
Improvement
Deployment schema "carto" instead of "geohash".
Removed
Remove VERSION function.
Module h3
Improvement
Deployment schema "carto" instead of "h3".
Rename ST_ASH3 function to H3_FROMGEOGPOINT.
Rename LONGLAT_ASH3 function to H3_FROMLONGLAT.
Rename ST_ASH3_POLYFILL function to H3_POLYFILL.
Rename ST_BOUNDARY function to H3_BOUNDARY.
Rename ISVALID function to H3_ISVALID.
Rename COMPACT function to H3_COMPACT.
Rename UNCOMPACT function to H3_UNCOMPACT.
Rename TOPARENT function to H3_TOPARENT.
Rename TOCHILDREN function to H3_TOCHILDREN.
Rename ISPENTAGON function to H3_ISPENTAGON.
Rename DISTANCE function to H3_DISTANCE.
Rename KRING function to H3_KRING.
Rename KRING_DISTANCES function to H3_KRING_DISTANCES.
Rename HEXRING function to H3_HEXRING.
Removed
Remove VERSION function.
Module measurements
Improvement
Deployment schema "carto" instead of "measurements".
Removed
Remove ST_ANGLE, already present in Bigquery.
Remove VERSION function.
Module placekey
Improvement
Deployment schema "carto" instead of "placekey".
Rename H3_ASPLACEKEY function to PLACEKEY_FROMH3.
Rename PLACEKEY_ASH3 function to PLACEKEY_TOH3.
Rename ISVALID function to PLACEKEY_ISVALID.
Removed
Remove VERSION function.
Module processing
Improvement
Deployment schema "carto" instead of "processing".
Removed
Remove VERSION function.
Module s2
Improvement
Deployment schema "carto" instead of "s2".
Rename ID_FROMHILBERTQUADKEY function to S2_FROMHILBERTQUADKEY.
Rename HILBERTQUADKEY_FROMID function to S2_TOHILBERTQUADKEY.
Rename LONGLAT_ASID function to S2_FROMLONGLAT.
Rename ST_ASID function to S2_FROMGEOGPOINT.
Rename ID_FROMTOKEN function to S2_FROMTOKEN.
Rename TOKEN_FROMID function to S2_TOTOKEN.
Rename ID_FROMUINT64REPR function to S2_FROMUINT64REPR.
Rename UINT64REPR_FROMID function to S2_TOUINT64REPR.
Rename ST_BOUNDARY function to S2_BOUNDARY.
Removed
Remove VERSION function.
Module transformations
Improvement
Deployment schema "carto" instead of "transformations".
Removed
Remove VERSION function.
Module clustering
Improvement
Deployment schema "carto" instead of "clustering".
Removed
Remove VERSION function.
Module data
Improvement
Deployment schema "carto" instead of "data".
Removed
Remove VERSION function.
Module random
Improvement
Deployment schema "carto" instead of "random".
Removed
Remove VERSION function.
Module routing
Improvement
Deployment schema "carto" instead of "routing".
Removed
Remove VERSION function.
Module statistics
Improvement
Deployment schema "carto" instead of "statistics".
Removed
Remove VERSION function.
Module tiler
Improvement
Deployment schema "carto" instead of "tiler".
Removed
Remove VERSION function.
Module tiler
Improvement
Add support for legacy project names that contain dots or other special characters.
Module statistics
New
Add KNN function.
Add GFUN function.
Add LOF function.
Add LOF_TABLE procedure.
Module tiler
Fix
Escape single-quoted properties in the CREATE_TILESET metadata when obtaining the geometry type.
Module statistics
New
Add GWR_GRID function.
Module tiler
Fix
Use zoom_min_column as zoom_min and zoom_max_column as zoom_max when provided.
Improvement
Optimize simplification method instead of the native BigQuery.
Module data
Improvement
Fields named dimension, total, intersection and input_area are now __carto_dimension, __carto_total and __carto_intersection and __carto_input_area. Also the column _carto_enrichment_ is now __carto_enrichment. This affects all the raw enrichment procedures: ENRICH_POINTS_RAW, ENRICH_POLYGONS_RAW, ENRICH_GRID_RAW, DATAOBS_ENRICH_POINTS_RAW, DATAOBS_ENRICH_POLYGONS_RAW, DATAOBS_ENRICH_GRID_RAW.
Fix
User provided queries can now have columns named dimension, total, intersection, input_area, _nonglobal, which could have collided previously with internal columns. All internal columns are now prefixed with __carto_. This affects all the enrichment procedures: ENRICH_POINTS, ENRICH_POLYGONS, ENRICH_GRID, DATAOBS_ENRICH_POINTS, DATAOBS_ENRICH_POLYGONS, DATAOBS_ENRICH_GRID, ENRICH_POINTS_RAW, ENRICH_POLYGONS_RAW, ENRICH_GRID_RAW, DATAOBS_ENRICH_POINTS_RAW, DATAOBS_ENRICH_POLYGONS_RAW, DATAOBS_ENRICH_GRID_RAW.
Module data
Improvement
Add associated_geography_slug column to DATAOBS_SUBSCRIPTIONS.
Some column names have changed in DATAOBS_SUBSCRIPTIONS:
table is now dataset_table
associated_geography is now associated_geography_table
Some column names have changed in DATAOBS_SUBSCRIPTION_VARIABLES:
geography_slug is now associated_geography_slug.
Some column names have changed in DATAOBS_SAMPLES:
table is now dataset_sample_table.
In DATAOBS_SUBSCRIPTION_VARIABLES, the column associated_geography_slug is now NULL for geography datasets.
Fix
The output of DATAOBS_SUBSCRIPTIONS and DATAOBS_SUBSCRIPTION_VARIABLES now omits items that are not Data Observatory products.
Module data
Fix
Enrichment results could mix input rows and values, specially for large inputs due to multiple evaluation of nondeterministic ROW_NUMBERS expression.
Improvement
New API, with consistent naming of procedures `
Module s2
Improvement
Rename functions ID_FROMUINT64REPR, UINT64REPR_FROMID to follow convention.
Module statistics
New
Create statistics module.
Add GETIS_ORD_H3 function.
Add GETIS_ORD_QUADKEY function.
Add MORANS_I_H3 function.
Add MORANS_I_QUADKEY function.
Add VERSION function.
Module h3
Improvement
Review HEXRING, KRING functions.
Change KRING_INDEXED to KRING_DISTANCES.
Module tiler
Fix
Fix support for DATE in CREATE_TILESET.
Fix naming of internal variables to avoid name collisions.
New
Add fraction_dropped_per_zoom in tileset metadata.
Module data
Changes
Support sub_ (table) subscriptions in:
DATAOBS_ENRICH_POINTS
DATAOBS_ENRICH_POINTS_WITH_MEASURES
DATAOBS_ENRICH_POLYGONS_WITH_AGGREGATION
DATAOBS_ENRICH_POLYGONS_WITH_MEASURES
DATAOBS_ENRICH_GRID
Module s2
Fix
Avoid keeping planar shape in spherical coordinates in ST_BOUNDARY.
Module data
Changes
Search for tables in DATAOBS_SAMPLES, DATAOBS_SUBSCRIPTIONS, DATAOBS_SUBSCRIPTIONS_VARIABLES.
Module data
Improvement
DATAOBS_ENRICH_* procedures are now optimized to reduce the amount of data processed by taking advantage of clusterization by geography in the Data Observatory.
The input_id_column parameter has been removed from the following procedures:
DATAOBS_ENRICH_POINTS
DATAOBS_ENRICH_POINTS_WITH_MEASURES
DATAOBS_ENRICH_POLYGONS_WITH_AGGREGATION
DATAOBS_ENRICH_POLYGONS_WITH_MEASURES
ENRICH_POINTS
ENRICH_POINTS_WITH_MEASURES
ENRICH_POLYGONS_WITH_AGGREGATION
ENRICH_POLYGONS_WITH_MEASURES
Module data
Improvement
ENRICH_GRID and DATAOBS_ENRICH_GRID procedures take input_query and input_index_column parameters instead of the indices array.
DATAOBS_ENRICH_* procedures now report missing subscriptions in a more user-friendly way.
Module routing
Improvement
Pass network to JS UDF natively
Support custom speed over LineStrings for GENERATE_NETWORK function.
Support custom speed over LineStrings for FIND_SHORTEST_PATH function.
Support custom speed over LineStrings for DISTANCE_MAP function.
Module tiler
Fix
Fix support for TIMESTAMP,DATETIME in CREATE_TILESET and CREATE_SIMPLE_TILESET.
Reduce parallelization to skip the recent error: "Not enough resources for query planning - too many subqueries or query is too complex"
Module h3
Fix
Support GEOMETRYCOLLECTION from ST_ASH3_POLYFILL.
Module data
New
Add DATAOBS_ENRICH_GRID procedure.
Add ENRICH_GRID procedure.
Module h3
New
Add KRING_INDEXED function.
Module geohash
New
Create geohash module.
Add VERSION function.
Add ST_BOUNDARY function.
Module tiler
Improvement
Checking if the output table exists before running the tiler.
Module data
Improvement
DATAOBS_ENRICH_* procedures take single source parameter with default carto-customers project.
Module data
New
Add DATAOBS_SAMPLES procedure.
Improvement
DATAOBS_SUBSCRIPTIONS and DATAOBS_SUBSCRIPTION_VARIABLES take single source parameter with default carto-customers project. Views with sub_ prefix are supported.
Module routing
New
Create routing module.
Add GENERATE_NETWORK function.
Add GENERATE_NETWORK_TABLE procedure.
Add FIND_SHORTEST_PATH function.
Add FIND_SHORTEST_PATH_FROM_NETWORK function.
Add FIND_SHORTEST_PATH_FROM_NETWORK_TABLE procedure.
Add DISTANCE_MAP function.
Add DISTANCE_MAP_FROM_NETWORK function.
Add DISTANCE_MAP_FROM_NETWORK_TABLE procedure.
Add VERSION function.
Module data
Fix
Fix DATAOBS_SUBSCRIPTIONS and DATAOBS_SUBSCRIPTION_VARIABLES not working with subscriptions in which the provider id contained underscores.
Module tiler
New
Add new automatic tileset generator CREATE_TILESET.
Module data
New
Add DATAOBS_SUBSCRIPTIONS procedure.
Add DATAOBS_SUBSCRIPTION_VARIABLES procedure.
Improvement
Change spatial catalog structure using two tables.
Improve enrichment procedures:
Use public geographies instead of subscription views
Remove unnecessary variables from ENRICH_POLYGONS_WITH_MEASURES
Make input_id_column always mandatory
Add input_area consistently
Module data
New
Create data module.
Add VERSION function.
Add DATAOBS_ENRICH_POINTS procedure.
Add DATAOBS_ENRICH_POINTS_WITH_MEASURES procedure.
Add DATAOBS_ENRICH_POLYGONS_WITH_AGGREGATION procedure.
Add DATAOBS_ENRICH_POLYGONS_WITH_MEASURES procedure.
Add ENRICH_POINTS procedure.
Add ENRICH_POINTS_WITH_MEASURES procedure.
Add ENRICH_POLYGONS_WITH_AGGREGATION procedure.
Add ENRICH_POLYGONS_WITH_MEASURES procedure.
Module s2
New
Add TOKEN_FROMID function.
Add ID_FROMTOKEN function.
Add ID_FROM_UINT64REPR function.
Add UINT64REPR_FROM_ID function.
Module tiler
Improvement
Use semantic versioning.
Add features_count and geometry labels to the tilesets.
Module clustering
Improvement
Change ST_CLUSTERKMEANS returning cluster and geom
Module accessors
New
Create accessors module.
Add ST_ENVELOPE function.
Add VERSION function.
Module processing
New
Create processing module.
Add ST_VORONOIPOLYGONS function.
Add ST_VORONOILINES function.
Add ST_DELAUNAYPOLYGONS function.
Add ST_DELAUNAYLINES function.
Add ST_POLYGONIZE function.
Add VERSION function.
Module transformations
New
Add ST_CONCAVEHULL function.
Module clustering
New
Create clustering module.
Add ST_CLUSTERKMEANS function.
Add VERSION function.
Module constructors
New
Add ST_BEZIERSPLINE function.
Add ST_MAKEELLIPSE function.
Module measurements
New
Create measurements module.
Add ST_ANGLE function.
Add ST_AZIMUTH function.
Add ST_MINKOWSKIDISTANCE function.
Module transformations
New
Rename module to transformations.
Add ST_CENTERMEAN function.
Add ST_CENTERMEDIAN function.
Add ST_CENTEROFMASS function.
Add ST_DESTINATION function.
Add ST_GREATCIRCLE function.
Add ST_LINE_INTERPOLATE_POINT function.
Module tiler
Removed
Remove quota & project checking.
Module constructors
New
Create constructors module.
Add ST_MAKEENVELOPE function.
Add ST_TILEENVELOPE function.
Add VERSION function.
Module transformations
New
Create transformation module.
Add ST_BUFFER function.
Add VERSION function.
Module random
Improvement
ST_GENERATEPOINTS now uses a spherically uniform distribution. Previously used to by uniform on projection.
Module h3
Improvement
Use hexadecimal as default type instead of int for h3 indexes.
Fix
Fix ST_BOUNDARY generating error when not able to parse geometry.
Module placekey
Improvement
Placekey conversions works with hexadecimal h3 indexes instead of int.
Module random
New
Create random module.
Add ST_GENERATEPOINTS function.
Add VERSION function.
Module h3
New
Create h3 module.
Add ST_ASH3 function.
Add LONGLAT_ASH3 function.
Add ST_ASH3_POLYFILL function.
Add ST_BOUNDARY function.
Add ISVALID function.
Add COMPACT function.
Add UNCOMPACT function.
Add TOPARENT function.
Add TOCHILDREN function.
Add ISPENTAGON function.
Add DISTANCE function.
Add KRING function.
Add HEXRING function.
Add VERSION function.
Module placekey
New
Create placekey module.
Add H3_ASPLACEKEY function.
Add PLACEKEY_ASH3 function.
Add ISVALID function.
Add VERSION function.
Module s2
New
Create s2 module.
Add ID_FROMHILBERTQUADKEY function.
Add HILBERTQUADKEY_FROMID function.
Add LONGLAT_ASID function.
Add ST_ASID function.
Add ST_BOUNDARY function.
Add VERSION function.
Module tiler
New
Add QUOTA procedure in order to display the monthly quota and quota available.
Implement support for layer name.
Add a max limit to the max_tile_size_kb property. Adds a new option (skip_validation) to circunvent this limit, which also allows going over the max partition limit.
Add a limit to the max amount of expected tiles at the highest zoom level for Simple Tilesets. It can be disabled via the skip_validation option of modified (for tests using max_tiles_at_maxzoom).
Add new drop_fraction_as_needed strategy initial implementation. Also adds internal debug and fraction_drop_per_zoom_level options to help debugging.
Add new metadada input parameter to set the properties name, description, legends and also extra_metadata.
Add zoom_min_column and zoom_max_column to control the zoom level at which particular features are visible.
Improvement
Change the default and max partition limit to 3999.
Make aggregation tilesets to use the quadkey module UDF LONGLAT_ASQUADINTLIST_RESOLUTION in order to generate quadkey aggregations.
Removed
Removed zoom_step option.
Fix
Fix bug that left empty the generator options in the metadata cell.
Fix CREATE_SIMPLE_TILESET failing with single column tables.
Fix collision with frequency column name.
Module tiler
New
Add tileset type to metadata and BigQuery labels.
Add the prefix "aggregation_" to aggregation options: type, resolution and placement.
Improvement
Moved to the Advanced Spatial Extension.
Change "max_tile_size_strategy" default to "throw_error", matching the other defaults and improve the exception error message in CreatePointAggregationTileset.
Rename adapting to BigQuery conventions, using UPPER CASE and separating words with underscore "_".
Set up limits for the procedures based on a limits table in the config project. The users can only execute the tiler from the "billing_project" and they have a maximum amount of processed bytes per month.
Removed
Remove compression levels 10-12 to reduce library size.
Remove ST prefix from functions not using geographies.
Fixed an issue with and procedure to avoid losing already calculated results when the process is interrupted when using TomTom as provider.
Fixed an issue with , , and that was not working with geographies containing holes.
Fixed an issue with , and that was not working with geographies containing holes.
Added support for compressed raster tables in and .
Fixed an issue with the procedure that was returning a “Resources exceeded during query execution error” when “scaling” was set to “RANKING”.
Fixed an issue with that prevented working with duplicate routes.
Fixed an issue with and procedures that pevented them from working on a VPC environment.
Fixed an issue with , and procedures procedures that pevented them from working on a VPC environment. .
Added procedures for aggregation & smoothing of spatial index datasets.
Fixed an issue that produced redundant calls in procedure.
Fixed a problem with that prevented component in Workflows from working with more than one component per workflow.
Fixed a problem with and that caused an issue when min, max, or count were used as aggregation methods for weight variables.
Modify procedures to overwrite internal temp tables and avoid crashing when called twice in the same session.
Modify , , FIND_TWIN_AREAS_V1 (discontinued version), and FIND_TWIN_AREAS_WEIGHTED_V1 (discontinued version) procedures to overwrite internal temp tables and and avoid crashing when called twice in the same session.
Modify , , , and SMOOTHING_MRF_QUADINT (discontinued version) procedures to overwrite internal temp tables and avoid crashing when called twice in the same session.
Add tile_resolution to all creation options. This new setting allows controlling the size of the tiles when rendered in a map. Higher values produce larger tiles.
Add and procedures for detecting anomalous space-time regions in the data. This new capability allows for uncovering unusual patterns in historical spatial or space-time data and for accurately identifying emerging anomalous patterns in space and time.
Modify and procedures to overwrite internal temp tables and not crash when called twice in the same session.
Fix input format of terrain_raster_band in procedure.
Fixed an issue with and procedures that were returning wrong results when “time_freq” was passed as lower case.
Fixed a bug where there was more than one matching row in the target for each row in the source. now use ST_DISTINCT to solve this issue
Optimized raster procedures (, ) to manage bigger data sources. Additionally, the option intersect_center is supported in both procedures to use the centroid of the pixel instead of the contour, in order to speed up the extraction/aggregation for relatively big polygons.
New options in : groupby_vector_columns and groupby_raster_columns to have finer control over the aggregation.
New procedure, which expects a table containing time series (indexed by a time step column and a unique series identifier) and can group the time series in as many clusters as the user requests.
New procedure. This procedure is designed to analyze spatio-temporal data in order to identify and categorize locations based on their hotspot or coldspot status over time. Utilizing z-score values generated by the Space-Time Getis-Ord function.
New procedure, that finds the path between the transmitter (Tx) and receiver (Rx), the obstacles that exist along the propagation path, i.e. those that intersect with the propagation line, and then it evaluates the impact of each obstacle's height
New procedure, an optimised version of the nominal TELCO_PATH_PROFILE procedure which only accepts as inputs a clutter and a terrain elevation rasters loaded using CARTO raster loader or Import API.
New : This procedure implements a statistical propagation model that describes large-scale propagation path loss over distance at all relevant frequencies in a certain outdor scenario.
New : This new procedure implements the empirical propagation model Extended HATA for radio propagation path loss for the range of frequencies 30 MHz - 3000 MHz in an outdoor scenario.
Improved and performance with big polygons on high-resolution cells.
We have a new module named “http_request”. It allows making requests to external endpoints through the function
We have a new module named “import”. It allows importing tables for a given URL containing a file in any of the formats supported by the CARTO Import API. The procedure in charge of this import is .
Added new and procedures:
Update in order to avoid generating fewer clusters than requested. The maximum number of clusters now will be equal to the number of different points passed.
not working properly for “geom_column” parameter's value different from “geom”.
now uses async batch geocoding with TomTom, boosting the geocoding speed considerably. Also matching by rows is applied ensuring that the address that reaches the TomTom endpoints is as similar as possible as the table value.
Add procedure. This new procedure enables users to enrich polygons with a provided data query by taking into account the ratio of a specified feature between the intersected and original areas in a way to weigh the contribution of each feature in the enrichment. Note that the original methods only use the geographic area of intersection for this weight.
Fix output table schema on the SQL Reference for the procedures and .
Add support in the procedure for supporting ‘NUMERIC’, ‘BIGNUMERIC’.
now guarantees to return the exact number of points requested.
Fixed aggregation column name suffix in .
Set null weights to 0 to standarize results in all and .
Add warning to procedures when no cells have neighbors.
New function that returns the distance (number of cells) between two quadbin cells.
Fixed an issue with the procedure that was not working due to a broken function reference.
New function that returns a point contained within a given polygon.
Support adding properties from BIGNUMERIC columns in the procedure.
Fixed a problem with , that was not parsing vars metadata correctly and producing a crash.
Fixed a problem when using “Default” aggregation in * procedures.
Fixed a problem that caused crash when a table contains multiple points very close to each other.
Support aggregation method in and procedures.
Added a normalization constant for * procedures
New , and procedures: updated raster format, introduced performance optimizations and support for custom operations, including multi-band and aggregation expressions.
: supports new raster format, upload multiple bands and performance optimizations.
Updated procedures to also work with categorical data.
Updated procedures to also work with categorical data.
Fixed a bug in procedure that was causing issues for generating higher zoom level tiles.
Updated procedures to also work with categorical data.
The previous version of the procedure is still available as
Add procedure. This procedure helps configuring the Analytics Toolbox in custom installations. Check about installing the Analytics Toolbox in your own BigQuery project for more information.
Add LDS functions: , , . This functions complement the already existing procedures. They allow geocoding a single address or getting a single isoline without creating a result table.
Replace LDS_QUOTA_INFO procedure with function. This new function allows checking your currently consumed quota with a single query, without the need to store the result as a table.
Correctly handle addresses with quotes in .
Use temporal tables to avoid existing issues in .
Adapt : The algorithm used for computing supervised composite scores has been changed from BOOSTED_TREE_REGRESSOR to to avoid overfitting.
: to create the data used by the PCAMIX method
: to train a PCA model using the PCAMIX method
: to predict the PC scores on the trained/new data
Improved performance in the procedure.
New batch service for . Used automatically by the procedure. No changes required.
Fix default options for that were causing errors when using
now supports
We have added the new procedures and to allow users create scores to analyze their network of merchants/point-of-sale based on a business performance metrics or geospatial attributes to analyze their suitability for the CPG brands.
We have fixed order of return values in and functions.
We have added a new function that enables different polyfill modes: center, intersects, contains.
We have improved the performance of by an average factor of 60x.
We have added a new function that enables different polyfill modes: center, intersects, contains.
We have improved the performance of by an average factor of 10x.
We are now allowing users to send any of the parameter options supported by the isolines service provider in .
We have improved our logging of results in so more information about processed and geocoded rows is provided to the user. Also the new option “carto_force_geocode” allows the re-geocoding of already geocoded rows.
We have applied some fixes to the function so the metadata column can be successfully created and informed.
We have added the new procedures and . They allow users to enrich polygons or grids based on spatial indexes (e.g. H3, Quadbin) with features from Data Observatory subscriptions by taking into account the ratio of a specified feature between the intersected and original areas in a way to weight the contribution of each feature in the enrichment. Note that the original methods only use the geographic area of intersection for this weight.
We have updated the procedure so the default value for the “decay” function is taken into consideration.
We have added the possibility of passing custom polygons as trade areas in and .
We added exceptions to both and procedures to ensure that the parameter “origin_query” includes only one row (the origin cell).
We are now allowing to use Numeric and Bignumeric as predictors data types types in the procedures: , and .
We modified the bucketization options in and in to better fit potential use cases covered by each procedure.
We have updated the formula used in so it gets standardized. In addition a new implementation of has been added in order to increase performance.
We have created a new procedure named , which calculates the routes (returned as linestrings) between a set of given origins and destinations (points) in a query. It supports different transport modes and advanced options as the other functions of the lds module.
We have added new functions in the statistics module to perform space-time cluster analysis, namely for quadbin indexes and for H3 indexes.
We have changed the input type and defined default values for some internal parameters that affect and . Also added some missing brackets to them.
Changed code to improve scalability in .
We have introduced a better way to handle large tables and drop intermediate tables in the function.
We have added metadata_partition option to , , , . A metadata_partition label is added to the table when the partition is not NULL. This optimizes tilesets for its use within CARTO.
: This procedure computes the Area of Applicability (AOA) of a Bigquery ML model. It generates a metric which tells the user where the results from a Machine Learning (ML) model can be trusted when the predictions are extrapolated outside the training space (i.e. where the estimated cross-validation performance holds).
: This procedure derives cross validation (CV) folds based on.
We have added a new mandatory parameter called correlation_var when scoring_method = FIRST_PC in the
We have released a new version of the , with the following improvements:
We allow for the use of ‘nodata’ metadata in
We have released experimental rendering capabilities for the raster data in deck.gl:
We have released a new set of procedures within the statistics module to enable users to (also known as or indexes) derived from a combination of different features. We have included 3 different procedures:
: to compute a spatial composite score as the residuals of a regression model which is used to detect areas of under- and over-prediction.
: to perform an aggregation of individual variables, scaled and weighted accordingly, into a spatial composite score.
: to measure the internal consistency of the variables used to derive the spatial composite score.
We have updated the to be aligned with the new model, working as an annual budget and combining consumption across all supported LDS services.
now supports null values in the input address column, ignoring them when running the geocoding process to provide results only for the informed addresses.
We have made the faster by retrieving metadata from the table and avoiding creating again the original table.
We have improved the by removing pixel_geog column and cluster results by quadbin in the quadbin raster format.
We have enhanced the outcome by preserving the input columns, renaming centroid_quadbin_block and centroid_quadbin_pixel to block_quadbin and quadbin, clustering the results by quadbin in quadbin raster and removing few non-informative columns (e.g. geog and myoffset).
We have released a new set of functions to enable users to perform the merchant universe matching analysis. It includes 2 different functions, which performs a fuzzy match between two different POI datasets based on location and name similarity (i.e. one dataset should include current merchants and the other one the complete universe of all potential merchants), and that generates report-like tables summarizing market penetration based on the results of the analysis.
We have implemented a fix in the .
From this version, we support line break (\n) and other escaped sequences in addresses in .
New module to operate with raster data in BigQuery. This module includes the procedures and to obtain the values of the raster in one or multiple geographies, respectively. It also includes the procedure to obtain metadata associated with the raster file.
New enhanced version of the routing module with procedures to calculate origin-destination matrices and to compute isolines around a set of locations, both supporting multiple transportation modes (car, bike, and walk). These new functions run on top of CARTO’s road network (derived from OSM segments) that is available as a public subscription in the Data Observatory. Please note that these improvements imply breaking changes with the previous version of the routing module.
Fix in procedure / example.
Fix in when sum is 1.
Add function (Inverse Distance Weighted).
We have added the possibility to configure more options as parameters when executing the functions to . These new options, which depend on the LDS service provider, allow the user to configure more transportation modes such as truck or bike, the possibility of specifying departure or arrival times allowing the creation of reverse isolines, and other options like different routing modes. Please note that these improvements imply breaking changes with the previous version of the CREATE_ISOLINES functions.
Added new confidence/relevance metadata to the results of the geocoding function . Please note that this improvement has implied breaking changes with the previous versions of the GEOCODE_TABLE and functions.
New function to the CPG module, that allows users to identify which locations (e.g. merchants, stores) are more similar to a chosen location (e.g. top performant) based on the characteristics of their surrounding areas (or trade areas).
Update in and to support buffer, kring and isoline methods. Please note that these improvements have implied some breaking changes in the aforementioned functions invalidating their previous version.
We have implemented a fix in the weighted averages computation in the procedure.
Quadbin/quadkey conversion functions have been added to ease the conversion from to quadkey and from quadkey to quadbin indexes.
Add function.
Add function.