Data Transforms
A Data Transform is a saved, named Data Mapping that you run over Entities that already exist. Each Entity is both the source and the target of the mapping, so a Data Transform can copy, combine, look up or recalculate attribute values in place.
Data Transforms are managed in Operator and belong to an Entity Type. You run one from a DataLab action, or while re-indexing an Entity Type.
This article will cover how to manage Data Transform records and how to run them. You can read about the mapping itself, including every mapping function, in Data Mapping.
Data model
Below is the data model for a Data Transform.
Operator fills both schemas from the Entity Type's own Data Schema when you create a Data Transform. You can read about the schema structure in Entity Type: Data Schema.
Requesting records
You can request every Data Transform in the account, the ones belonging to one Entity Type, or a single one. Lists are sorted by name, include the mapping and both schemas, and leave out Data Transforms whose Entity Type has been deleted.
Access is checked against the Data Transform permissions of the requesting user.
Get a list of Data Transforms
Get a list of Data Transforms for an Entity Type
Get Data Transform by ID
The same request scoped to an Entity Type returns a 404 error when the Data Transform belongs to a different Entity Type.
Get Data Transform by ID for an Entity Type
Create and update records
You can create a record by performing a POST request with the ID as "0". If you specify an existing ID then it'll update the record. Both return the saved record.
When updating, only the properties you include are changed. The mapping and each schema are replaced as a whole. The Entity Type is set on create: from the URL when you use the Entity Type route, otherwise from "EntityType.ID" in the body.
Create or update Data Transform
Create or update Data Transform for an Entity Type
Delete a record
We don't provide any bulk deletion but you can delete by record ID. Deleting an ID that does not exist still succeeds.
Delete Data Transform
Delete Data Transform for an Entity Type
Running a Data Transform
Running a Data Transform maps each Entity onto itself. The mapping output is merged into the Entity, so attributes the mapping does not write keep their values. A mapping cannot change an Entity's ID, Entity Type or audit values.
The DataLab "data-transform" action runs a Data Transform over the Entities a DataLab query finds, named by its ID or its name. It also adds the attributes of "DestinationDataSchema" to the Data Schema of each Entity Type it changes. You can read about it in DataLab actions.
You can also run a Data Transform over every Entity of an Entity Type while re-indexing it. Every Entity the mapping changes is saved again, and "reSaveAll" saves the rest as well. The request starts a Pending Action, which you can follow as described in Pending Actions.