Amazon S3 data classification integration

📘

Draft note (remove before publishing)

Migrated from the HubSpot article "Amazon AWS S3 Integration". Add screenshots of the AWS role setup and the MineOS integration form.

Automate data classification by integrating MineOS with Amazon S3

How It Works

MineOS connects to your AWS account through an IAM role you create, then scans objects in the S3 buckets you choose to detect and map the types of personal data stored in them.

• Data classification: MineOS extracts text from supported files and classifies the personal data it finds, reporting the data types and where they were found.
• You choose the buckets: a bucket regex controls exactly which buckets are scanned.
• Read-only access: the role only needs read access to your buckets.

Note: This integration supports data classification only.

Before You Start

• You need permission to create IAM roles in your AWS account.
• Have your 12-digit AWS account ID ready.
• [Optional] Enable Preview mode to view sample PII values alongside classification results. You must enable Preview mode before running the scan.

Setting Up

Step 1: Create a role for MineOS in AWS

1. Log in to your AWS account and go to IAM → Roles → Create Role.
2. Under Trusted entity type, choose AWS account.
3. Choose Another AWS account and enter the account ID 595379680669.
4. Under Options, mark Require External ID and enter an External ID of your choice. Keep it; you'll need it in MineOS.
5. Click Next. Under Add Permissions, add the AmazonS3ReadOnlyAccess policy.
6. Click Next, add a Role Name and Description, and click Create Role.

Note: The integration requires the s3:ListAllMyBuckets permission. Encrypted buckets also require kms:Decrypt, which isn't part of the AmazonS3ReadOnlyAccess policy; encrypted buckets are skipped without it. If you already have a role for MineOS, add the policy to the existing role.

Step 2: Connect in MineOS

1. Log in to MineOS. On the left sidebar, click Inventory → Data Sources. Click Add Data Source, search for Amazon S3 and click Add.
2. Open the Amazon S3 data source → Integrations tab → Request Handling section → select Integration.
3. Fill in your External ID, Account ID, Role Name and Bucket Regex. For example:
   • .+ scans all the buckets the role can access.
   • prod-.+ scans all buckets that start with prod-.
   • .+-data$ scans all buckets that end with -data.
4. Mark the Scan this source using Data Classifier checkbox.
5. Click Test your integration. The bucket names that match your regex appear; check that they're the buckets you intended.
6. Click Test & Save.

You're done! MineOS has been connected to your Amazon S3 buckets.

To add more buckets or regexes, click + Create Instance and enter another regex. You can reuse the same External ID, Account ID and Role Name.

Supported File Types

MineOS extracts text from the following file types and classifies it:

• Apache Avro (.avro), with limits on maximum block size, file size and number of columns
• Apache Parquet (.parquet)
• .csv and .tsv
• PDF (up to 30 MB)
• Text files
• Microsoft Word, Excel and PowerPoint (up to 30 MB)
• Image files, using OCR

Note: "Requester pays" buckets aren't supported. Objects in the Glacier Flexible Retrieval or Glacier Deep Archive storage classes are skipped; Glacier Instant Retrieval is supported. Compressed objects (gzip) aren't currently supported.

Need Help?

If you need any help with this integration, please contact your Customer Success Manager. They will be happy to assist you!