Skip to main content

​
Overview

At RWS Integration it’s possible to parallelize extractions to increase the range of data that can be extracted in a single integration. Parallelization allows you to extract data from multiple divided segments simultaneously by configuring a list of parameter values that will be used to make parallel API calls.

​
When to use

Use parallelization when:
  • The API to be extracted has data divided into multiple “channels” or segments based on a parameter
  • You need to extract data from multiple specific values of a parameter (e.g., multiple store IDs, account IDs, region codes)
  • The API returns different datasets based on parameter values

​
How it works

RWS Integration will parallelize the extractions by the number of parameter values configured in the ‘Extract’ parallelization section. Each value in your configured list will trigger a separate, parallel extraction request to the API.

​
Example

Consider an API that returns order data from multiple stores. The API requires a storeId parameter that filters orders by store. Scenario: You need to extract orders from 3 stores with IDs: abc123, def456, and ghi789. Configuration: In the parallelization section, configure:
  • Parameter: storeId
  • Parameter list: abc123,def456,ghi789 (separated by commas without spaces)
Result: RWS Integration will trigger 3 separate, parallel extraction requests—one for each store ID in your list. This allows you to extract data from all three stores simultaneously in a single integration run.

​
Benefits

  • Simplicity: Allow you to configure a list of parameter values to be extracted in a single integration
  • Scalability: Easily add or remove parameter values without creating new integrations