Google BigQuery data classification integration

📘

Draft note (remove before publishing)

Based on the HubSpot articles "Integration for Google BigQuery" (classification parts only) and "Data Classification for Google BigQuery - Predicting Cost". Add screenshots of the integration type selection, the GCP IAM grant and the Data Classifier panel.

Automate data classification by integrating MineOS with Google BigQuery

How It Works

MineOS connects to your Google Cloud project, samples the data in your BigQuery tables or views, and classifies the personal data it finds. Results are added to the data types of the BigQuery data source, according to the likelihood of each data type found.

Note: This article covers data classification. Setting up BigQuery for privacy requests is covered in the Google BigQuery DSR integration article.

Choose the right connection type

Before You Start

• Make sure you have access to Google Cloud Platform and Google BigQuery, and know the Project ID you want to scan.
• For data classification only, the connecting user or service account needs the BigQuery Data Viewer and BigQuery Job User roles.
• If your organization enforces a session timeout for Google Cloud scopes, mark the MineOS app as trusted and exempt trusted apps from session timeout. Otherwise the integration expires and has to be reconnected.
• [Optional] Enable Preview mode to view sample PII values alongside classification results. You must enable Preview mode before running the scan.

Setting Up

1. Log in to MineOS. On the left sidebar, click Inventory → Data Sources. Click Add Data Source, search for Google BigQuery and click Add.
2. Open the Google BigQuery data source → Integrations tab → Request Handling section → select Integration, and choose your connection type.
3. Mark the Scan this source using Data Classifier checkbox.
4. Connect your account:
   • OAuth types: click Connect and follow the on-screen instructions to grant the required permissions.
   • Service account types:
     a. In the Google Cloud Console, open IAM & Admin and choose the organization you want to scan from the top bar.
     b. On the IAM page, click Grant access.
     c. Copy the principal account shown in MineOS and paste it into New principals.
     d. Assign the BigQuery Data Viewer and BigQuery Job User roles.
5. Enter your Google Cloud Project ID and click Test & Save.

You're done! MineOS has been connected to your BigQuery project. Go to the Data Classifier panel on the data source page to start the scan.

Note: If you receive an authentication error, check that the principal has the right roles.

Good to Know

• Partitioned tables aren't currently supported for data classification.
• Tables with column-level access control that prevents MineOS access are skipped. If you see the error "Access Denied: BigQuery: User has neither fine-grained reader nor masked get permission to get data protected by policy tag", assign MineOS the additional fine-grained access roles for those columns.

Predicting the Cost of a Scan

Scanning BigQuery tables runs queries in your Google Cloud project, which BigQuery bills to your GCP account. This is separate from your MineOS subscription, and applies to tables, not views.

MineOS keeps this cost low by scanning a statistically representative sample of each table instead of the whole table:

• Table row counts are read from BigQuery metadata, so no counting queries are run.
• Each table is sampled at a calculated percentage, so BigQuery processes only part of the table's data.
• Each table has a hard cap of 2 TB of processed data. If a query approaches it, it's canceled at no cost.

Typical costs:

• Small tables (under 1 GB) often fall under BigQuery's 10 MB minimum per query, so the cost is negligible.
• Medium tables (1–100 GB): sampling typically processes 1–5% of the data.
• Large tables (over 100 GB): sampling usually processes less than 1% of the data.
• For example, a full scan of a 1 TB table costs about $5, while a MineOS sample of the same table costs about $0.05–0.25.

Partitioned data can cost slightly more to sample, because BigQuery's processing blocks don't span partitions.

To keep costs predictable:

• Start with a few tables or a single project before scaling to your whole environment.
• Exclude archived or unneeded tables from the scan scope.
• Track actual costs in your GCP billing console during and after the scan.
• If you have concerns about specific large tables, contact your Customer Success Manager; the sampling can be adjusted for your environment.

Permissions

MineOS requests the following scope when you connect with OAuth:

Need Help?

If you need any help with this integration, please contact your Customer Success Manager. They will be happy to assist you!