Google Cloud Storage

Get started with the Cloud Storage target!

Obtain Cloud Storage credentials

For the Cloud Storage target to work correctly, you need to create credentials in your Google Cloud Console. The instructions to create these credentials are below.

Enable the Google Cloud Storage target

Head to your flow, and go to the target options.

Enable Cloud Storage targetEnable Cloud Storage target

Enable Cloud Storage target

Select the Cloud Storage target, and you'll be prompted to provide your credentials.

Provide Cloud Storage credentialsProvide Cloud Storage credentials

Provide Cloud Storage credentials

Bucket path prefix

The path prefix is the path under which data will be uploaded. This field can include the following parameters:

ParameterValue
{tenant}The user id this job is running for
{date}The current date formatted as YYYYMM
{flow_id}The flow id this job ran for
{env_id}The environment this job ran for
{job_id}The internal hotglue job id

For example, a valid bucket path prefix is data/invoices/{tenant}

How to get your Cloud Storage credentials

Create a Service Account

Navigate to IAM in your Google Cloud Console, and select Create service account

Create service accountCreate service account

Create service account

Name the service account and give it a description, like the following, then select Create and continue:

Fill service account detailsFill service account details

Fill service account details

Now you can assign the necessary permissions. In this case, I will use the Storage Object Creator role. You could also use a different role with less granular permissions, such as the Storage Admin role, but this is not recommended.

Assign Storage Object Creator roleAssign Storage Object Creator role

Assign Storage Object Creator role

Generate the key file

Finally, select Done and navigate to the newly created service account, and head to the Keys section.

Add Key to Service AccountAdd Key to Service Account

Add Key to Service Account

Select "Create new key" and choose JSON type

This will generate a JSON key to your local machine

Finally, open the JSON file that was downloaded. This file contains the necessary credentials:

Generated GCS credentialsGenerated GCS credentials

Generated GCS credentials

You can copy these credentials to the matching fields in hotglue to configure your GCS target! πŸŽ‰


Did this page help you?