MuleSoft For Each, Parallel For Each, and Batch Processing Comparison

Author: Mohammad Mazhar Ansari

As we know MuleSoft provides For Each, Parallel For Each and Batch Processing to process a list of records. In this blog, we will compare all of them to see which use cases are suitable to use these.

What is For Each Scope?

  • The Foreach scope splits a collection into elements and processes them iteratively through the processors embedded in the scope, then returns the original message to the flow.

What is Parallel For Each Scope?

  • The Parallel For Each scope enables you to process a collection of messages by splitting the collection into parts that are simultaneously processed in separate routes within the scope of any limitation configured for concurrent-processing. After all messages are processed, the results are aggregated following the same order they were in before the split and then the flow continues.

What is Batch Job?

  • Mule allows you to process messages in batches. You can initiate a Batch Job scope, which splits messages into individual records, performs actions upon each record, and then reports on the results and potentially pushes the processed output to other systems or queues.

Comparison Between For Each, Parallel For Each and Batch Processing:

For EachParallel For EachBatch Processing
Execution SupportMule 3.x OnwardsMule 4.2 OnwardsMule 3.x Onwards
Graphical SupportMule 3.x OnwardsMule 4.3 OnwardsMule 3.x Onwards
Execution PatternSynchronousSynchronousAsynchronous
Execution OrderSequentialParallelParallel
Record GroupingPossible using Batch SizeNot PossiblePossible using Batch Aggregator
Error HandlingIn exception scenarios stops processing if error not handledIn exception scenarios does not stop processing however raise MULE:COMPOSITE_ROUTE error typeThe Behaviour can be configured
Suitable for?Sequential ProcessingParallel Processing With SynchronousParallel Processing With Asynchronous
# of Records ?Small DataMedium DataLarge Data
OutputOriginal Payload, Custom Logic required to get each record processing outputAccumulated PayloadOriginal Payload

For Each Use Cases:

  • Sequential Processing Required
  • Synchronous Processing Required
  • Small Data Set
  • Processing of records in Batch required
  • Process records only if previous records are processed successfully

Parallel For Each Use Cases:

  • Synchronous Processing Required with parallelism
  • Medium Data Set
  • Accumulated Output Required
  • Process records irrespective of previous records status

Parallel For Batch Job Cases:

  • Asynchronous Processing Required
  • Ordering of process records not needed
  • Large Data set
  • Processing logic is complex and filtering is optional
  • Process records irrespective of previous records status

Conclusion:

In general the # of records and behavior (Sync or Async) determines which option to choose. However for Medium # of records choosing between Parallel For Each and Batch Job mostly govern whether we want accumulated output or not. But if you are choosing Parallel For Each just because your use case requires accumulated output just keep in mind that if Accumulated output is large it can cause JVM Out of Memory issues.

To know more about our services visit: savannahw2.sg-host.com.

We use cookies on this site to enhance your user experience. For a complete overview of how we use cookies, please see our privacy policy.