Tune Service Tasks Performance by Throttling

Overview

Each automated Workflow Step/Activity in Straatos queues and processes tasks  in parallel. However not all tasks that are queued in a step/activity are processed at the same time. Straatos manages the throughput of each automated step/activity by adjusting the number of “ready-for-process” tasks and number of processing threads in parallel by “throttling. .  

Both these parameters are configurable by configuring the “number of instances” and “number of threads” parameter in a workflow step/activity.

Why do we need to throttle

When an automated service task is used to send information and data to an external service, and that receiving service has a limit to the amount of tasks it can accept and process. In this case, the number of tasks can be limited so the external service is not overloaded.

Configuration task Throttles and Threads

The throttle can be configured separately for each workflow step. In the Process Designer, click on an activity, then click on the 'Set maximum number of threads' icon.

In the pop up box, enter the settings:

Throttle maximum number of instances in activity (0 means don’t throttle):

This throttle can be used if you want to limit the number of “ready-to-process” tasks in a particular workflow step. For example, in a very busy environment, there may be 1000 tasks (each tasks contained an image to be processed) queued in the automated workflow step/activity that is sending documents to a OCR cloud service. However, the OCR service can only accept 20 documents. Therefore if the parameter is set to 0 value, Straatos will attempt to send all 1000 tasks (images) to the OCR cloud service resulting in an error.

Maximum threads processed in parallel

This limits the number of processing threads servicing a queue of tasks in a workflow step/activity. Threads works in conjunction with the number of instances (“ready-to-process” tasks ) configured in the throttle parameter.

For synchronous execution (for example in JavaScript steps), this has the same effect as limiting the number of instances (when throttle is set to 0 (no throttling)). By default this value is set to 10. (Number of configured processing threads should not be larger than the number of instances in any given situation).

In asynchronous processing (when a step/activity is not expecting a return result from the external service) , the workflow step will   trigger  10 processing threads  at a   time and is ready  for the next processing cycle once the last one has ended. . If  the asynchronous service  returns with the result, a processing thread from the configured number of thread pool is consumed  to manage  the incoming request. This helps to optimize  the amount of computing resources used for each workstep  and hence the ability to  manage bottlenecks and maximise throughput. Most Straatos Service Tasks are processing documents asynchronously behind the scene and hence limiting the throttle will not affect the performance significantly, except for large number of tasks processed in parallel. In most cases, if limiting the tasks is required, the throttle option should be used.