Author: Rithul Kumar
Objective
Learn about
i) Mule’s Parallel Foreach Processing Capability
ii) how it works and its application in this introductory article.
Overview
Mulesoft has the capability to process a collection of objects in an iterative manner. This is useful for modifying records inside a collection. It has already provided concepts like Batch Process and Foreach to achieve this. In this blog, we will talk about a new component introduced in Mule 4.2.0 called Parallel-foreach.
Prerequisite:
This post assumes that you are familiar with Foreach scope and Batch Scope in Mule. Please ensure that you have Mule 4.2.0 runtime and Anypoint Studio 7+ in your system.
Note:Currently the component is not available in Mule Palette, we need to add the modifications directly in the configuration xml. So, don’t worry, if you see the component as not recognized in the Mule Flow.
1.1 What does Parallel Foreach do?
· Like Foreach Scope in Mule, the Parallel-Foreach scope splits the payload into elements and processes them one by one using the components you place in the scope.
· The payload inside the scope is each of the split elements on each iteration.
· But unlike the sequential processing in Foreach scope, here the processing happens parallelly. So here each iteration is given to a specific thread from thread pool.
· And the output of this scope is the collection of the results of each route and the aggregated results will be in the same order they were, prior to the split.
1.2 Format of Parallel Foreach
As currently, the parallel-foreach component is not available, please modify it directly in the configuration xml page.
1.3 Configurations for Parallel Foreach
1.4 What is the output format of the parallel foreach scope?
[
{
“inboundAttachmentNames”:[],
“exceptionPayload”: null,
“inboundPropertyNames”: [],
“outboundAttachmentNames”: [],
“payload”: “beta”,
“outboundPropertyNames”:[],
“attributes”: null
}
]
The output will be an array of objects where each object represents an iteration.
1.5 Error Handling in Parallel Foreach
· Parallel Foreach will continue to process all the routes even if an exception occurs in an iteration and will invoke the error handler only after all the routes completes its processing with aggregated results and errors from each iteration.
· Throws
MULE:COMPOSITE_ROUTING
1.6 Format of Error Message
{
“inboundAttachmentNames”: [],
“exceptionPayload”: null,
“inboundPropertyNames”: [],
“outboundAttachmentNames”: [],
“payload”: { -> This gives the aggregated result
“failures”: { -> The aggregated failure messages are listed
“1”: { -> This indicates in which iteration the error occurred
“errorType”: { -> Error Details
“parentErrorType”: {
“parentErrorType”: null,
“identifier”: “ANY”,
“namespace”: “MULE”
},
“identifier”: “EXPRESSION”,
“namespace”: “MULE”
},
“childErrors”: [],
“errorMessage”: null,
“cause”: cause details over here,
“description”: error desciption over here
}
},
“results”: { -> This gives the aggregated success results.
“0”: {
“inboundAttachmentNames”: [],
“exceptionPayload”: null,
“inboundPropertyNames”: [],
“outboundAttachmentNames”: [],
“payload”: “apple”,
“outboundPropertyNames”: [],
“attributes”: null
}
}
},
“outboundPropertyNames”: [],
“attributes”: null
}
1.7 How is it different from Foreach?
| Foreach | Parallel Foreach |
| The split collection is processed sequentially. | The split collection is processed parallelly. |
| Single thread manages each iteration. After one iteration completes, the same thread moves to the next iteration | Multi thread processing, each iteration is given to available thread and all the threads execute parallelly. |
| Counter variables exist. | Counter variable not available |
| Attributes modifications from the previous iteration will be available to the next iteration. | Each route will have access to data given before the start of the scope. There is no persistence of modifications across routes. |
| The output of the scope will be same as the input | The output will be an aggregated result from each iteration. |
| Execution gets interrupted when an exception is thrown, and error handler is invoked | Execution will continue even if an exception occurs and error handler is invoked only after all the iteration routes has completed its processing |
Conclusion
Mule 4 have added a lot of new ideas and Parallel Foreach will increase the performance of the already existing foreach scope for many situations due to its parallel processing capability
References
Mule Documentation on Parallel-Foreach