Skip to main content

Cloud file storage

Overview​

Cloud file storage is a reusable connection to a cloud storage bucket or container that MessageGears uses for optimized bulk file operations. Once configured, a cloud file storage location can be selected anywhere Accelerator needs a place to stage or retrieve files, including:

MessageGears supports three file storage types: Google Cloud Storage, Amazon S3, and Azure Blob Storage.

Whichever type you choose, Accelerator performs the same four operations against the storage location: it writes staged files, reads them back, lists the contents of the bucket or container, and deletes files it no longer needs. The identity you configure must therefore be able to read, write, list, and delete objects — the provider tabs below spell out the exact permissions.

Let Accelerator manage file retention

Accelerator cleans up after itself — for example, a daily job removes FastCache files once no Blueprint references them. Some files are intentionally long-lived: FastCache files backing active Blueprints stay in storage for as long as the Blueprint uses them, and Audience Recording output remains until you've ingested it. Don't apply object expiration or retention rules that automatically delete objects from the bucket, container, or path Accelerator uses — they can remove files Accelerator or your downstream processes still need.

Networking

If your bucket or container restricts network access (a storage firewall, VPC/VNet rules, or private endpoints), you will need to allow traffic from your MessageGears environment before Test Connection will succeed. Contact MessageGears Support for the egress IP addresses for your environment.

Creating a new file storage​

  1. Navigate to Admin > System configuration > File Storage.
  2. Click Add New File Storage.
  3. Enter a unique Name. This is how the file storage will be referenced elsewhere in Accelerator (Database Connections, FastCache Settings, Audience Recording, etc.), so it must not match the name of an existing file storage.
  4. Select a File Storage Type. The remaining fields change depending on the type selected (see the tabs below).
  5. Optionally check Default Client to authenticate using the credentials of the environment Accelerator runs in (for example, an AWS instance role, GCP attached service account, or Azure managed identity) instead of entering keys or secrets. When checked, the credential fields for the selected type are not required — you supply only the bucket or container details. This option applies mainly to self-hosted deployments where the Accelerator host itself has been granted access to the storage.
  6. Fill out the remaining fields for the selected file storage type.
  7. Click Test Connection to confirm Accelerator can authenticate and reach the bucket or container before saving.
  8. Click Save.
note

Secret fields (AWS Secret, GCS Credentials, Azure Account Key, Azure Client Secret) are stored encrypted and are never displayed again after saving. When editing an existing file storage, these fields appear blank — leave them blank to keep the stored value, or enter a new value to replace it.

Storage types​

Select your storage provider:

Fields

  • Bucket (required) - The name of the GCS bucket Accelerator will read from and write to.
  • Path - An optional folder path within the bucket.
  • Credentials (required) - The GCP service account credentials used to authenticate to the bucket. Paste the full contents of the service account's JSON key file.

Setting up in Google Cloud

Complete these steps in the Google Cloud Console before creating the file storage in Accelerator:

  1. Create a bucket (or choose an existing one): go to Cloud Storage > Buckets > Create. The default bucket settings work — in particular, keep uniform bucket-level access enabled (Accelerator never sets per-object ACLs). Note the bucket name — you'll enter it in the Bucket field.

  2. Create a dedicated service account: go to IAM & Admin > Service Accounts > Create Service Account. Give it a recognizable name such as messagegears-file-storage.

  3. Grant the service account access to the bucket: on the bucket's Permissions tab, click Grant Access, add the service account as a principal, and assign the Storage Object Admin (roles/storage.objectAdmin) role. This grants the object read, write, list, and delete permissions Accelerator needs, scoped to that bucket only.

    If you prefer a custom role, it must include these permissions: storage.objects.get, storage.objects.create, storage.objects.list, and storage.objects.delete.

  4. Create a JSON key for the service account: on the service account's Keys tab, click Add Key > Create new key > JSON, and download the key file. Paste the entire contents of this file into the Credentials field in Accelerator.

You should now have everything the Accelerator form asks for:

Accelerator fieldWhere it comes from
BucketThe bucket name from step 1
PathOptional — a folder within the bucket, if you want Accelerator confined to one
CredentialsThe full contents of the JSON key file from step 4
warning

Treat the JSON key file like a password — it grants access to your bucket. Share it only through a secure channel and delete downloaded copies once the file storage is configured.