Effectively Managing Nintex Workflow History

It's common practice to exploit Nintex workflows to further extent. Sometimes due to limited technologies or limited control over SharePoint environment, most of the organisations decide to use Nintex workflow as a ETL tool. Also used for replacing C# code with  Nintex workflow. The Nintex workflow with a robust design works fine but as it's built over SharePoint workflow engine, a small issue in worker process or timer job can hamper the workflow execution.

A badly designed workflow can soon overload workflow history list which can deteriorates the performance of the workflow.

Lets assume a use case where Server Access management process is implemented using Nintex workflow. To grant access to a server the request should be approved by two users, an appropriate permission should be setup to secure the metadata and update the request metadata during request life cycle.

To achieve the above use case, the nintex workflow will have 2 request approval activities, 2 Update list item activities, 2 Set permission activities, 2 pause activities (for safe delay) and may be some error handling technique to log additional data into history list.

With the above workflow design (Consisting Request approval, Pause for and Log into history list) below number of entries will be logged in workflow history list.

Workflow Activity	Number of entries logged in history list
2 Request Approval (On Approval or Rejection)	2
2 Pause for	4
Error Log or Informative Data	2

As you can see a single instance of workflow execution logs 8 entries in Workflow History list for simple process. Assuming on a average 5 requests are processed in a day by workflow and 100 are processed in a month the entries created in history list will be 800 per month and for a year it will be 9,600 entries and 48,000 entries in 5 years which is enough to slow down the workflow. 

We are talking about only one simple workflow and in a scenario where a same workflow history list shared with other running workflows. The list entries can grow exponentially in a short span.

Nintex today does not provide any functionality to switch to a new workflow history list in case items created in workflow history list exceeds the threshold value. In my experience larger the number of items in history list slower the workflow execution, Higher the number of workflow failures.

So Do we have any solution which can be put in place by developers/designer to ensure that their workflows are running efficiently?

"Information management policy" can be used to define the retention policy. Based on the auditing requirements a retention policy can be defined on a workflow history list which can then either delete the items or archive them to a different list.