# Configure your own buckets (Helm)

{% hint style="info" %}
This documentation only applies to **advanced Orchestrated container deployments** using **Kubernetes** and **Helm**
{% endhint %}

## **1.** Overview

Every CARTO Self-Hosted installation requires **two cloud storage buckets** to handle data and assets used by the platform:

<table><thead><tr><th>Purpose</th><th width="155.40625">Is Mandatory?</th><th width="320.15625">Description</th><th>Example contents</th></tr></thead><tbody><tr><td><strong>1. Import Bucket</strong></td><td>Yes</td><td>Used to upload and import datasets into CARTO.</td><td><code>.csv</code>, <code>.geojson</code>, <code>.zip</code></td></tr><tr><td><strong>2. Thumbnails Bucket</strong></td><td>Yes</td><td>Stores generated map thumbnails and customization assets (logos, markers, etc.).</td><td><code>.png</code> images</td></tr><tr><td><strong>3. Export Bucket</strong></td><td>No, this bucket is optional</td><td>Used for exporting data from your data warehouse (BigQuery, Snowflake, Redshift, or Amazon RDS). Create this only if you plan to use data export features.</td><td><code>.csv</code>, <code>.json</code>, <code>.parquet</code></td></tr></tbody></table>

You can create and use your own storage buckets in any of the following supported storage providers:

* [Google Cloud Storage](https://cloud.google.com/storage)
* [AWS S3](https://aws.amazon.com/s3/)
* [Azure Blob Storage](https://azure.microsoft.com/es-es/products/storage/blobs/)

2. (Optional) Create the **data export bucket**. This bucket has to be created in different storage providers depending on your data warehouse:
   * BigQuery: [Google Cloud Storage](https://cloud.google.com/storage)
   * Snowflake: [AWS S3](https://aws.amazon.com/s3/)
   * Redshift: [AWS S3](https://aws.amazon.com/s3/)
   * Amazon RDS: [AWS S3](https://aws.amazon.com/s3/)

## 2. Pre-requisites

#### 2.1. Create the required buckets

1. **Import Bucket**
2. **Thumbnails Bucket**
3. **Export Bucket** (optional)

> There are no naming constraints for these buckets.

## 3. Configuration notes

### 3.1. Thumbnails bucket access

* Thumbnails objects (`.png`) can be **public** (default) or **private**.
* To make them private, set:

  ```
  WORKSPACE_THUMBNAILS_PUBLIC="false"
  ```
* ⚠️ Some features (branding, custom markers) require **public access**.\
  To keep the bucket private while allowing these features:
  * Enable **public objects**
  * Allow **ACLs** (non-uniform permissions)
  * Disable **server-side encryption**

### 3.2. Export bucket provider requirements

| Data Warehouse              | Required Storage Provider |
| --------------------------- | ------------------------- |
| **BigQuery**                | Google Cloud Storage      |
| **Snowflake**               | AWS S3                    |
| **Redshift**                | AWS S3                    |
| **Amazon RDS (PostgreSQL)** | AWS S3                    |
| **Oracle**                  | AWS S3                    |

### 3.3. CORS Configuration

**Required for Import and Thumbnails Buckets**

<table><thead><tr><th width="189.984375">Setting</th><th>Value</th></tr></thead><tbody><tr><td>Allowed origins</td><td><code>*</code></td></tr><tr><td>Allowed methods</td><td><code>GET, PUT, POST</code></td></tr><tr><td>Allowed headers (common)</td><td><code>Content-Type, Content-MD5, Content-Disposition, Cache-Control</code></td></tr><tr><td>GCS extra headers</td><td><code>x-goog-content-length-range, x-goog-meta-filename</code></td></tr><tr><td>Azure extra headers</td><td><code>Access-Control-Request-Headers, X-MS-Blob-Type</code></td></tr><tr><td>Max age</td><td><code>3600</code></td></tr></tbody></table>

CORS configuration location:

* **GCS / S3:** Bucket level
* **Azure Blob:** Storage account level

For more details, refer to:

* Google Cloud Storage CORS Setup
* AWS S3 CORS Setup
* Azure Blob CORS Setup

### 3.4. Authentication requirements

The buckets access from the Carto platform require authentication configuration. You'll find below the authentication methods available for each provider:

| Provider       | Auth Method                       |
| -------------- | --------------------------------- |
| **GCS**        | Service Account Key               |
| **AWS S3**     | Access Key ID + Secret Access Key |
| **Azure Blob** | Access Key                        |

{% hint style="info" %}
If you can't setup Service Account Keys, Access Keys or Secret Access Key due to security constraints or other reasons you can setup GCP Workload Identity or EKS Pod Identity using the following guides:

* [Use Workload Identity in GCP (Helm)](https://docs.carto.com/carto-self-hosted/guides/guides-helm/use-workload-identity-in-gcp)
* [Use EKS Pod Identity in AWS (Helm)](https://docs.carto.com/carto-self-hosted/guides/guides-helm/use-eks-pod-identity-in-aws)
  {% endhint %}

## 4. Setup per provider

### 4.1 Import and Thumbnails buckets

{% tabs %}
{% tab title="Google Cloud Storage" %}
In order to use Google Cloud Storage custom buckets you need to:

1. Add the following lines to your <mark style="color:orange;">customizations.yaml</mark> and replace the `<values>` with your own settings:

```yaml
appConfigValues:
  storageProvider: "gcp"
  workspaceImportsBucket: <import_bucket_name>
  workspaceImportsPublic: <false|true>
  workspaceThumbnailsBucket: <thumbnails_bucket_name>
  workspaceThumbnailsPublic: <false|true>
  thumbnailsBucketExternalURL: <public or authenticated external bucket URL>
  googleCloudStorageProjectId: <gcp_project_id>
```

> Note that thumbnailsBucketExternalURL could be <https://storage.googleapis.com/\\>\<thumbnails\_bucket\_name>/ for public access or <https://storage.cloud.google.com/\\>\<thumbnails\_bucket\_name>/ for authenticated access.

2. Select a **Service Account** that will be used by the application to interact with the buckets. There are two options:
   1. Using a custom Service Account, that will be used not only for the buckets, but for the services deployed by CARTO as well. If you are using Workload Identity, that's your option.
   2. Using a dedicated Service Account **only for the buckets**
3. Grant the selected Service Account with the role `roles/iam.serviceAccountTokenCreator` in the GCP project where it was created.

> ⚠️ We don't recommend granting this role at project IAM level, but instead at the Service Account permissions level (IAM > Service Accounts > `your_service_account` > Permissions).

4. Grant the selected Service Account with the role `roles/storage.admin` to the buckets created.
5. **\[OPTIONAL]** Pass your GCP credentials as secrets: **This is only required if you are going to use a dedicated Service Account only for the buckets**

   * **Option 1: Automatically create the secret:**

     ```yaml
     appSecrets:
       googleCloudStorageServiceAccountKey:
         value: |
           <REDACTED>
     ```

   > `appSecrets.googleCloudStorageServiceAccountKey.value` should be in plain text, preserving the multiline and correctly tabulated.

   * **Option 2: Using existing secret:** Create a secret running the command below, after replacing the `<PATH_TO_YOUR_SECRET.json>` value with the path to the file of the Service Account:

     ```bash
     kubectl create secret generic \
       [-n my-namespace] \
       mycarto-google-storage-service-account \
       --from-file=key=<PATH_TO_YOUR_SECRET.json>
     ```

     Add the following lines to your <mark style="color:orange;">customizations.yaml</mark>, without replacing any value:

     ```yaml
     appSecrets:
       googleCloudStorageServiceAccountKey:
         existingSecret:
           name: mycarto-google-storage-service-account
           key: key
     ```

{% endtab %}

{% tab title="AWS S3" %}
In order to use AWS S3 custom buckets you need to:

1. Create an IAM user and generate a programmatic key ID and secret.
2. Grant this user with read/write access permissions over the buckets. If server-side encryption is enabled, the user must be granted with permissions over the KMS key used.
3. Add the following lines to your <mark style="color:orange;">customizations.yaml</mark> and replace the `<values>` with your own settings:

```yaml
appConfigValues:
  storageProvider: "s3"
  workspaceImportsBucket: <import_bucket_name>
  workspaceImportsPublic: <false|true>
  workspaceThumbnailsBucket: <thumbnails_bucket_name>
  workspaceThumbnailsPublic: <false|true>
  thumbnailsBucketExternalURL: <external bucket URL>
  awsS3Region: <s3_buckets_region>
```

> Note that thumbnailsBucketExternalURL should be https\://\<thumbnails\_bucket\_name>.s3.amazonaws.com/

4. Pass your AWS credentials as secrets by using one of the options below:
   * **Option1: Automatically create a secret**

     Add the following lines to your <mark style="color:orange;">customizations.yaml</mark> replacing it with your access key values:

     ```yaml
     appSecrets:
       awsAccessKeyId:
         value: "<REDACTED>"
       awsAccessKeySecret:
         value: "<REDACTED>"
     ```

     > `appSecrets.awsAccessKeyId.value` and `appSecrets.awsAccessKeySecret.value` should be in plain text
   * **Option 2: Using an existing secret**

     Create a secret running the command below, after replacing the `<REDACTED>` values with your key values:

     ```bash
     kubectl create secret generic \
       [-n my-namespace] \
       mycarto-custom-s3-secret \
       --from-literal=awsAccessKeyId=<REDACTED> \
       --from-literal=awsSecretAccessKey=<REDACTED>
     ```

     > Use the same namespace where you are installing the helm chart

     Add the following lines to your <mark style="color:orange;">customizations.yaml</mark>, without replacing any value:

     ```yaml
     appSecrets:
       awsAccessKeyId:
         existingSecret:
           name: mycarto-custom-s3-secret
           key: awsAccessKeyId
       awsAccessKeySecret:
         existingSecret:
           name: mycarto-custom-s3-secret
           key: awsSecretAccessKey
     ```

{% endtab %}

{% tab title="Azure Blob Storage" %}
In order to use Azure Storage buckets you need to:

1. Generate an Access Key, from the storage account's Security properties.
2. Add the following lines to your <mark style="color:orange;">customizations.yaml</mark> and replace the `<values>` with your own settings:

```yaml
appConfigValues:
  storageProvider: "azure-blob"
  azureStorageAccount: <storage_account_name>
  workspaceImportsBucket: <import_bucket_name>
  workspaceImportsPublic: <false|true>
  workspaceThumbnailsBucket: <thumbnails_bucket_name>
  thumbnailsBucketExternalURL: <external bucket URL>
  workspaceThumbnailsPublic: <false|true>
```

> Note that thumbnailsBucketExternalURL should be https\://\<azure\_resource\_group>.blob.core.windows.net/\<thumbnails\_bucket\_name>/

3. Pass your credentials as secrets by using one of the options below:
   * **Option 1: Automatically create the secret:** Add the following lines to your `customizations.yaml` replacing it with your access key value:

     ```yaml
     appSecrets:
       azureStorageAccessKey:
         value: "<REDACTED>"
     ```

     > `appSecrets.azureStorageAccessKey.value` should be in plain text
   * **Option 2: Using existing secret:** Create a secret running the command below, after replacing the `<REDACTED>` values with your key values:

     ```bash
     kubectl create secret generic \
       [-n my-namespace] \
       mycarto-custom-azure-secret \
       --from-literal=azureStorageAccessKey=<REDACTED>
     ```

     > Use the same namespace where you are installing the helm chart

     Add the following lines to your <mark style="color:orange;">customizations.yaml</mark>, without replacing any value:

     ```yaml
     appSecrets:
       awsAccessKeyId:
         existingSecret:
           name: mycarto-custom-azure-secret
           key: azureStorageAccessKey
     ```

{% endtab %}
{% endtabs %}

## 5. Data export bucket

### 5.1 BigQuery

To enable exporting data from BigQuery on CARTO Self-Hosted platform these are the required steps:

1. Grant read/write permissions to the service account used by your CARTO Self-Hosted installation on the GCS bucket created in the [pre-requisites](#pre-requisites).
2. Define the name the bucket that will be used for exporting data in your <mark style="color:orange;">customizations.yaml</mark> file:

```
appConfigValues:
    workspaceExportsBucket: <YOUR_EXPORTS_BUCKET>
```

### 5.2 Snowflake and Redshift

Snowflake and Redshift require an AWS S3 bucket to export data from CARTO platform. These are the needed steps for allowing exporting data from CARTO Self-Hosted in these providers:

1. Create an IAM user and generate a programmatic key ID and secret. If server-side encryption is enabled, the user must be granted with permissions over the KMS key used.

{% hint style="info" %}
If you've already configured the Import and Thumbnails buckets using AWS S3, you can use the same user you already created for these buckets.
{% endhint %}

1. Create an AWS IAM role with the following settings:

   1. Trusted entity type: `Custom trust policy`.
   2. Custom trust policy: Make sure to replace `<your_aws_user_arn>`.

   ```json
   {
     "Version": "2012-10-17",
     "Statement": [
         {
             "Effect": "Allow",
             "Principal": {
                 "AWS": "<your_aws_user_arn>"
             },
             "Action": [
                 "sts:AssumeRole",
                 "sts:TagSession"
             ]
         }
     ]
   }
   ```

   3. Add permissions: Create a new permissions' policy, replacing `<your_aws_s3_bucket_name>`.

   ```json
   {
      "Version": "2012-10-17",
      "Statement": [
          {
              "Effect": "Allow",
              "Action": "s3:ListBucket",
              "Resource": "arn:aws:s3:::<your_aws_s3_bucket_name>"
          },
          {
              "Effect": "Allow",
              "Action": "s3:*Object",
              "Resource": "arn:aws:s3:::<your_aws_s3_bucket_name>/*"
          }
      ]
   }
   ```
2. Update your <mark style="color:orange;">customizations.yaml</mark> file with the following values:

```yaml
appConfigValues:
    awsExportBucket: <BUCKET_NAME>
    awsExportBucketRegion: <REGION>
    exportAwsRoleArn: <ROLE_ARN>
```

Pass your AWS credentials as secrets:

* **Option 1: Automatically create the secret:**

  ```yaml
  appSecrets:
    exportAwsSecretAccessKey:
      value: <REDACTED>
    exportAwsAccessKeyId:
      value: <REDACTED>
  ```

> `appSecrets.exportAwsSecretAccessKey.value` and `appSecrets.exportAwsAccessKeyId.value` be in plain text, preserving the multiline and correctly tabulated.

* **Option 2: Using existing secret:** Create a secret running the command below:

  ```bash
  kubectl create secret generic \
    [-n my-namespace] \
    mycarto-export-aws-access-key \
    --from-literal=key-id=<ACCESS_KEY_ID>
    --from-literal=key-secret=<ACCESS_KEY_SECRET>
  ```

  Add the following lines to your <mark style="color:orange;">customizations.yaml</mark>, without replacing any value:

  ```yaml
  appSecrets:
    exportAwsSecretAccessKey:
      existingSecret:
        name: mycarto-export-aws-access-key
        key: key-id
    exportAwsAccessKeyId:
      existingSecret:
        name: mycarto-export-aws-access-key-id
        key: key-secret
  ```

## Configure data exports in Amazon RDS for PostgreSQL

The bucket to export data from Amazon RDS for PostgreSQL can be configured from the CARTO platform UI. Once your Self-Hosted installation is finished, you can check in the following documentation [how to configure your S3 bucket integration for Amazon RDS for PostgreSQL](https://docs.carto.com/carto-user-manual/settings/advanced-settings/configuring-s3-bucket-integration-for-rds-for-postgresql-exports-in-builder).