Author: Mohammad Mazhar Ansari
As we know MuleSoft provides For Each, Parallel For Each and Batch Processing to process a list of records. In this blog, we will compare all of them to see which use cases are suitable to use these.
What is For Each Scope?
- The Foreach scope splits a collection into elements and processes them iteratively through the processors embedded in the scope, then returns the original message to the flow.
What is Parallel For Each Scope?
- The Parallel For Each scope enables you to process a collection of messages by splitting the collection into parts that are simultaneously processed in separate routes within the scope of any limitation configured for concurrent-processing. After all messages are processed, the results are aggregated following the same order they were in before the split and then the flow continues.
What is Batch Job?
- Mule allows you to process messages in batches. You can initiate a Batch Job scope, which splits messages into individual records, performs actions upon each record, and then reports on the results and potentially pushes the processed output to other systems or queues.
Comparison Between For Each, Parallel For Each and Batch Processing:
| For Each | Parallel For Each | Batch Processing | |
| Execution Support | Mule 3.x Onwards | Mule 4.2 Onwards | Mule 3.x Onwards |
| Graphical Support | Mule 3.x Onwards | Mule 4.3 Onwards | Mule 3.x Onwards |
| Execution Pattern | Synchronous | Synchronous | Asynchronous |
| Execution Order | Sequential | Parallel | Parallel |
| Record Grouping | Possible using Batch Size | Not Possible | Possible using Batch Aggregator |
| Error Handling | In exception scenarios stops processing if error not handled | In exception scenarios does not stop processing however raise MULE:COMPOSITE_ROUTE error type | The Behaviour can be configured |
| Suitable for? | Sequential Processing | Parallel Processing With Synchronous | Parallel Processing With Asynchronous |
| # of Records ? | Small Data | Medium Data | Large Data |
| Output | Original Payload, Custom Logic required to get each record processing output | Accumulated Payload | Original Payload |
For Each Use Cases:
- Sequential Processing Required
- Synchronous Processing Required
- Small Data Set
- Processing of records in Batch required
- Process records only if previous records are processed successfully
Parallel For Each Use Cases:
- Synchronous Processing Required with parallelism
- Medium Data Set
- Accumulated Output Required
- Process records irrespective of previous records status
Parallel For Batch Job Cases:
- Asynchronous Processing Required
- Ordering of process records not needed
- Large Data set
- Processing logic is complex and filtering is optional
- Process records irrespective of previous records status
Conclusion:
In general the # of records and behavior (Sync or Async) determines which option to choose. However for Medium # of records choosing between Parallel For Each and Batch Job mostly govern whether we want accumulated output or not. But if you are choosing Parallel For Each just because your use case requires accumulated output just keep in mind that if Accumulated output is large it can cause JVM Out of Memory issues.
To know more about our services visit: savannahw2.sg-host.com.
