After setting up all the prerequisites from Part I and Part II, it is time to create the Orchestrator runbooks that will provide the auto-remediation for active incidents and problems:

Incident Management Automation: This runbook will be triggered by the SQL-related Incidents which were auto-generated in SCSM by the SCOM Alerts. The “fix” for the generating issues, covered in Part I, will be enclosed in a PowerShell Script activity. The result is that an email will be sent to the Incident Owner, notifying him that a problem had been detected and Orchestrator fixed it.

Problem Management Automation: The counters in this runbook will keep track of the number of instances for which an incident was created. If the same incident is triggered and fixed 5 times, a problem will automatically be generated and the runbook will attach any existing and future similar incidents to it. This goes by the principle that a recurring issue should be identified as a problem, not as an incident. Once the problem is marked as “Resolved” or “Closed”, the counter is reset.

Additionally, from Orchestrator’s perspective, this post will also demonstrate why “Counters” are not a very practical solution in most situations, mainly because they are global and not runbook-specific. This idea will be resumed in Part IV, where a more advanced replacement for counters will be implemented.

Before starting the main runbook, besides completing Part I and Part II and installing the IPs, three prerequisites need to be prepared. These include creating two child-runbooks, which might seem out-of-context at this point, but will make sense once they are integrated in the main runbook.
Continue reading →