Custom Integration for DSR

The custom integration allows you to extend the DSR handling workflow with your own code. This is especially useful for applying custom business logic, handling internal home-brew systems or preventing external connections to internal databases. This is extremely easy to implement with your own API endpoint (such as a cloud function) or even no-code automation tools, such as our own Integration Builder.

The custom integration allows you to register API endpoints that will be called as part of the DSR workflow. Just like Mine's built-in integrations, custom integration supports any processing type (Delete, Copy), allows previewing user data, and supports advanced operations such as handling failures and performing long-running async operations.

The custom integration supports the following operations. Each operation requires an endpoint URL in order to be used:

Note: while Preview is optional, at least one of the copy or delete endpoints has to be defined.

To see specific implementation guides on each of the operations, use the following links:

• Handling Delete Operations
• Handling Copy Operations
• Handling Preview Operations

Adding Custom Integrations

Custom integration is available for custom systems in your data inventory. To use more than one custom integration, simply add multiple custom systems.

When multiple custom integrations are used, the integrationId field is used for updating the status of a specific integration.

Setting Up in the Portal

1. Go to Data Inventory → Data Sources → Add data source.
2. Click + New custom source and give your source a name.
3. From the Data Sources list, click the new source to open its settings.
4. Scroll down and enable Handle this data source in privacy requests, then select Integration.
5. Enter the endpoint URLs for the operations you want to support (Delete, Copy, and/or Preview). All fields are optional, but at least one of Delete or Copy must be filled in.
6. Optionally, configure an API Key header for authentication (see Security section below).
7. Click Save, then click Test Integration to verify the setup.

Request Payload

Every time MineOS makes an HTTP call to your endpoint, regardless of the operation, the following JSON payload is sent as the request body:

{
    // useful for logging
    "traceId": "123456789abcdefghijklm",
    // used for the CustomIntegrationStatus endpoint
    "integrationId": "0c36xnykgewwbzfukh1jkq",
    // indicates this is just a test event, actual processing should be skipped
    "isTest": true,
    // the request object is not always available:
    // its available when handling a DSR but not when handling a user search (preview)
    "request": {
        // used for the CustomIntegrationStatus endpoint
        "id": "ABCDEFGHIJKLMNOPQRSTUV",
        // type of request. Possible values: Delete,GetCopy,DoNotSell,Undetermined,RightToEdit,DoNotMail
        "type": "Delete",
        // the channel from which the request was sent. Possible values: Form,MineApp,Api,EmailForwarding,Manual
        "source": "Form",
        // the main domain of the workspace to which the request was sent
        "domain": "test.com",
        "createdAt": "2023-07-28T11:35:21.7528109Z",
        // type of request V2. The possible values can be viewed in the Portal under DSR Handling -> DSR setup tab
        // for backwards compatibility the id is the same as the "type" field lowercased.
        "requestType": {
            "id": "delete",
            "name": "Delete"
        }
    },
    "userInfo": {
        "name": "Test User",
        "email": "[email protected]",
        "isVerified": false,
        "countryOfResidence": "TestCountry",
        // fields set through UpdateMetadata endpoint
        "customFields": {
            "testCustomKey": "Custom value"
        }
    }
}

Logging And Storing Information

At any time while processing a request, you might want to store information on the request:

1. Logging/documenting important business information, for example: the type of customer, the underlying system etc. This can be done with the Add a note API. In the UI this information is visible in the notes section on the right side.
2. Storing metadata/state information, for example: internal user Id, URI of related resources or other metadata about the handling flow. This can be done with the Update metadata API. This information is not visible in the UI but can be retrieved with the Get metadata API.

Security

When receiving requests to your endpoint, it's best to perform a security check before processing the event. There are a number of supported security mechanisms; you can use any one of them, or a combination of more than one:

1. API Key

The requests can include a header with a name and value of your choice. By setting a secret value as a custom header, you can verify in your code that the header contains the value you would expect, similar to using an API Key. This is the best and easiest option.

To configure the API Key header in the portal:

1. Open your custom integration settings in Data Inventory.
2. Under Custom request headers, enter a Header name and Header value.
3. Save the integration.

MineOS will attach this header to every outgoing request (Delete, Copy, and Preview). Your endpoint should reject any request that does not include the expected header and value.

2. IP Whitelisting

We use static IPs that you can whitelist in your firewall / ACL. See the list of IPs here.

3. Verifying Signature

This is another option instead of the API Key. Each request MineOS makes to your endpoint contains a HTTP header called: x-mine-signature. This header contains the event signature, which can be verified to make sure the request originated from your MineOS account.

Your verification key is displayed in the custom integration settings panel in the portal.

See the example code for verifying the signature:

var key = "YOUR VERIFICATION KEY";
var data = "JSON PAYLOAD OF THE EVENT";

var hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
var rawSignature = hmac.ComputeHash(Encoding.UTF8.GetBytes(data));
var signature = BitConverter.ToString(rawSignature).Replace("-", "").ToLower();

var key = "YOUR VERIFICATION KEY";
var data = "JSON PAYLOAD OF THE EVENT";

var crypto = require('crypto');
var signature = crypto.createHmac('SHA256', key).update(data).digest('hex');

To test whether your implementation is correct, you can use an online HMAC calculator.

4. VPN Tunneling

The custom system integration supports MineOS VPN Tunneling so you can securely connect to endpoints running in your internal network.

Operation Guides

• Handling Delete Operations
• Handling Copy Operations
• Handling Preview Operations
• Local Development and Testing