Synchronize After Merge
Description
Synchronize After Merge applies inserts, updates, and deletes to a database table based on the flag column generated by the Merge Rows (diff) step. Use this step to complete a data synchronization workflow — for example, keeping a target reporting table in sync with a source system by automatically inserting new records, updating changed ones, and deleting removed ones. It reads the flag value (identical, changed, new, or deleted) for each row and performs the corresponding database operation. The input data must have been processed by Merge Rows (diff) with sorted streams before reaching this step.
Configurations
| Field Name | Description |
|---|---|
| Step Name | Name of the step. This name has to be unique in a single workflow. |
| Connection The database connection to which data is written | |
| Target schema | The name of the Schema for the table to which data is written. This is important for data sources that allow for table names with periods in them. |
| Target table | Name of the table in which you want to do the insert/update/delete. |
| Commit size | The number of rows to change before running a commit. |
| Use batch update | |
| Tablename is defined in a field | |
| Key Lookup table | Allows you to specify a list of field values and comparators. You can use the following comparators: =, <>, <, <=, >, >=, LIKE, BETWEEN, IS NULL, IS NOT NULL.Note: Click Get fields to retrieve a list of fields from the input stream(s). |
| Update Fields | Allows you to specify all fields in the table you want to insert/update including the keys. Avoid updates on certain fields (e.g. primary keys or non-null constrained fields) by specifying N in the update column. Note: Click Get Update fields to retrieve a list of update fields from the input stream(s). |
| SQL button | Click SQL to generate the SQL to create the table and indexes for correct operation. |
| Advanced Tab: | |
| Operation fieldname | This is a required field. This field is used by the step to obtain an operation flag for the current row. Default from the Merge Rows (diff) step: "flagfield" |
| Insert when value equal | Specify the value of the Operation fieldname which signifies that an Insert should be carried out. Default from the Merge Rows (diff) step: "new" |
| Update when value equal | Specify the value of the Operation fieldname which signifies that an Update should be carried out. Default from the Merge Rows (diff) step: "changed" |
| Delete when value equal | Specify the value of the Operation fieldname which signifies that a Delete should be carried out. Default from the Merge Rows (diff) step: "deleted" |
| Perform lookup | Performs a lookup when deleting or updating. If the lookup field is not found, then an exception is thrown. This option can be used as an extra check if you wish to check updates/deletes prior to their execution. Default from the Merge Rows (diff) step: Not applicable |