data-manager-api-event-ingestion

Files (1)

• SKILL.md

SKILL.md (first lines)

# Data Manager API Event Ingestion

## Implementation Workflow

### Prerequisites

-   **Authentication & Library Installation**: If you need to set up access to
    the Data Manager API or install the client and utility libraries, refer to
    the `data-manager-api-setup` skill.

### Step 1: Identify Use Case & Read Documentation

-   **Determine Destination Account Type**: [CRITICAL] If it's not
    explicitly stated, STOP and CLARIFY with the user where the data is being
    sent (e.g., Google Ads, Floodlight, Google Analytics)
    BEFORE generating any code. Do not assume Google Ads by default. This maps
    to the `account_type` field of the `operating_account` in the `Destination`,
    and also determines valid event identifiers and requirements.
-   **Read Documentation**: [CRITICAL] Follow the
    [Send events guide](https://developers.google.com/data-manager/api/devguides/events/send-events.md.txt)
    to implement the integration, as steps for configuring and sending the
    request may vary between destinations.

### Step 2: Retrieve Code Sample

> [!IMPORTANT]
> If writing or updating an ingestion script, ALWAYS retrieve the
> relevant code sample to use as a reference:

| Language | Sample |
| :--- | :--- |
| **Python** | [`ingest_events.py`](https://github.com/googleads/data-manager-python/blob/main/samples/events/ingest_events.py) |
| **Java** | [`IngestEvents.java`](https://github.com/googleads/data-manager-java/blob/main/data-manager-samples/src/main/java/com/google/ads/datamanager/samples/IngestEvents.java) |
| **PHP** | [`ingest_events.php`](https://github.com/googleads/data-manager-php/blob/main/samples/events/ingest_events.php) |
| **Node** | [`ingest_events.ts`](https://github.com/googleads/data-manager-node/blob/main/samples/events/ingest_events.ts) |
| **.NET**| [`IngestEvents.cs`](https://github.com/googleads/data-manager-dotnet/blob/main/samples/IngestEvents.cs) |

### Step 3: Retrieve Migration Guides

> [!IMPORTANT]
> If refactoring code to upgrade from another Google API, ALWAYS
> extract the full contents of the relevant field mapping guide.

#### Google Ads

*   **Google Ads API Offline Conversions**:
    [Google Ads Offline Conversions Migration Field Mappings](https://developers.google.com/data-manager/api/devguides/events/google-ads/offline/upgrade/field-mappings.md.txt)
*   **Google Ads API Store Sales**:
    [Google Ads Store Sales Migration Field Mappings](https://developers.google.com/data-manager/api/devguides/events/google-ads/store-sales/upgrade/field-mappings.md.txt)

#### Google Analytics

*   **Measurement Protocol (Google Analytics)**:
    [Google Analytics Measurement Protocol Migration Field Mappings](https://developers.google.com/data-manager/api/devguides/events/analytics/measurement-protocol/upgrade/field-mappings.md.txt)

#### Campaign Manager 360 (CM360)

*   **Campaign Manager 360 API Offline Conversions**:
    [Campaign Manager 360 Offline Conversions Migration Field Mappings](https://developers.google.com/data-manager/api/devguides/events/cm360/offline/upgrade/field-mappings.md.txt)

### Step 4: Implementation

Implement the ingestion logic using the following checkpoints:

-   [ ] **Initialize Client**: Instantiate the Data Manager client
    (`IngestionServiceClient`).
-   [ ] **Define Destinations**: Build the `Destination` object using the
    `product_destination_id` and the appropriate account configurations:
    `operating_account` (target account receiving data), `login_account` (if
    authenticating using a manager account or a data partner account), and
    `linked_account` (if you're a data partner accessing the account via a
    partner link to a manager account). **STRONGLY RECOMMENDED**: Refer to the
    [Configure destinations and headers](https://developers.google.com/data-manager/api/devguides/concepts/destinations.md.txt)
    guide for more details on configuring destinations.
-   [ ] **Prepare Event Data**: Use the utility library helpers to format and
    normalize user identifiers correctly.
-   [ ] **Construct Payload**: Build the request payload
    (`IngestEventsRequest`) containing the destinations, event records, and
    consent permissions.
-   [ ] **Support Validation**: Support sending the `validate_only` boolean
    option on the `IngestEventsRequest` to allow developers to validate schemas
    without actually uploading data.
-   [ ] **Send Request**: Execute `ingest_events` and record the returned
    `request_id` for later diagnostics.
-   [ ] **Check for Ingestion Warnings**: If any non-required field had a
    validation failure, the response from `ingest_events` will also
    include `field_warnings`, a list of `FieldWarning` objects detailing the
    issues.
-   [ ] **Retrieve Request Status**: Check the status of the ingestion request
    using diagnostics. Since request processing is asynchronous, a
    successful ingestion response (HTTP 200 OK returning a `request_id`) only
    indicates the payload was received. To check if the records actually
    succeeded, partially succeeded, or failed to process, query the
    `client.retrieve_request_status` endpoint using the `request_id`. Skipping
    this step is a common user mistake.

## Formatting

*   Fetch the [Format user data](https://developers.google.com/data-manager/api/devguides/concepts/formatting.md.txt)
    guide and use that as the source of truth for formatting and
    normalization rules.

*   Use the utility library to format, hash, and encrypt user data
    (emails, phone numbers, addresses).

    **Python Example:**

    ```python
    from google.ads.datamanager_util import Formatter
    from google.ads.datamanager_util.format import Encoding

    formatter: Formatter = Formatter()

    processed_email: str = formatter.process_email_address(
        email, Encoding.HEX
    )
    ```

## Critical Gotchas

From KUNDURUHIMANEESHREDDY/dataos, MIT.

More from KUNDURUHIMANEESHREDDY/dataos