Cloud Bucket Download Demo

Demo going over how to download a file from Google Cloud Storage.

This demo uses the csvtojson node package library.
However, you may use any package that you prefer; or even a user built solution.

Note

This version uses JS modules (i.e. "import" instead of "require").

Make sure your package.json contains the lines below. Also, make sure your JS files have the .mjs extentions

Note

Other properties removed for brevity.

"type": "module"
"scripts": {
  "test": "node --experimental-vm-modules node_modules/jest/bin/jest.js",
},

Setup

In order to use this demo as a template, perform the following actions

Click the "Use this template" button in GitHub.
Select a name for the repo under your user workspace.
Navigate to Cloud Shell.
Open the termninal and clone the repository.

Important

Make sure you clone the repository under your user workspace in GitHub, not the "CIT-Data-Driven-Cloud-Apps" organization.

git clone <REPO_URL>

Open the folder in Cloud Shell VSCode.
Open the terminal and run npm install to download all the dependent packages.

Testing

Integration

To test downloading a file currently in your Cloud Storage bucket, do the following.

Connect to your Google Cloud project.
```
gcloud config set project <PROJECT_ID>
```
Upload a CSV file to your bucket.

There is a students.csv file in this project that you may use.
However, make sure you test with other CSV files; modifying the code as needed.

Open tests/integration/file-download.test.mjs
Update the bucket and name variables to your bucket and file name, respectively.
Run npm run test:integration in the terminal.

End-to-End (e2e)

To test the trigger of uploading a file to Cloud Storage you will need to do the following.

Create a bucket in Cloud Storage.
Deploy your function by running the following command in the terminal.

Replacing YOUR_REGION and YOUR_BUCKET_NAME with the region your bucket is located in and the name of your bucket, respectively.

gcloud functions deploy gcs-upload-trigger \
  --gen2 \
  --runtime=nodejs22 \
  --region=YOUR_REGION \
  --source=. \
  --entry-point=onFileUpload \
  --trigger-event-filters="type=google.cloud.storage.object.v1.finalized" \
  --trigger-event-filters="bucket=YOUR_BUCKET_NAME"

Once the function is deployed, navigate to Cloud Storage, and upload a CSV file to the bucket.
Next, navigate to the Cloud Run, click your function, then click the "Logs" navigation item under "Observability".
You should see the output from the index.mjs console.dir(jsonData) statement.

CIT-Data-Driven-Cloud-Apps/cloud-bucket-download-demo

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Languages