Automating Incident & Problem management – Part III: Using Orchestrator to remediate the issue (Counter edition)

After setting up all the prerequisites from Part I and Part II, it is time to create the Orchestrator runbooks that will provide the auto-remediation for active incidents and problems:

Incident Management Automation: This runbook will be triggered by the SQL-related Incidents which were auto-generated in SCSM by the SCOM Alerts. The “fix” for the generating issues, covered in Part I, will be enclosed in a PowerShell Script activity. The result is that an email will be sent to the Incident Owner, notifying him that a problem had been detected and Orchestrator fixed it.

Problem Management Automation: The counters in this runbook will keep track of the number of instances for which an incident was created. If the same incident is triggered and fixed 5 times, a problem will automatically be generated and the runbook will attach any existing and future similar incidents to it. This goes by the principle that a recurring issue should be identified as a problem, not as an incident. Once the problem is marked as “Resolved” or “Closed”, the counter is reset.

SCSM Incident automation, SCSM Problem Automation

Additionally, from Orchestrator’s perspective, this post will also demonstrate why “Counters” are not a very practical solution in most situations, mainly because they are global and not runbook-specific. This idea will be resumed in Part IV, where a more advanced replacement for counters will be implemented.

Before starting the main runbook, besides completing Part I and Part II and installing the IPs, three prerequisites need to be prepared. These include creating two child-runbooks, which might seem out-of-context at this point, but will make sense once they are integrated in the main runbook.
Continue reading

Advertisements

While modifying a runbook, I deleted some of my activities / links

Bad news: There is no “Undo” button. So what do you do when this happens?

You have two options:

  1. Since you deleted part of your runbook, you are probably still in “check out” mode with it. So, in order to see how the runbook was before check-out, log on from a different machine, to Orchestrator Runbook Designer. Select everything inside the runbook and copy it to a new runbook, then start modifying again.
  2. If you just want everything restored in the same runbook, verify the time when you deleted your activities / links (let’s say it was 10:00 a.m.), then go to your Orchestrator database, and run the following query:

use Orchestrator

Select * from OBJECTS where Deleted = 1 and LastModified > ‘2013-11-22 09:55:00.000‘ and LastModified < ‘2013-11-22 10:05:00.000

Update OBJECTS set Deleted = 0 where Deleted = 1 and LastModified > ‘2013-11-22 09:55:00.000‘ and LastModified < ‘2013-11-22 10:05:00.000

In case you deleted the entire runbook, or worse, an entire folder of runbooks, you should check this post, which covers well the rest of the available undeleting tricks.

Child runbooks in Orchestrator: Automating an uniquely named,fully patched VM with basic apps installed.

I always see people comparing PowerShell with Orchestrator, or thinking of Orchestrator as a GUI for PowerShell. From my point of view the real power of Orchestrator lies in its capabilities to integrate at different levels to multiple systems and bring them together in a simple “runbook”.

Finally in this post, we can bring together all the previous runbooks and get to experiment with child runbooks. For this all the previous runbooks have to be ready:

Another prerequisite for this runbook is having a VM template (with Windows 7 in my case) ready for deployment.

Scenario

The main idea, when creating these little runbooks, was to have a VM that:

  • Was auto generated
  • Has an unique name that complies with a specific standard
  • Is fully patched
  • Has a minimum of applications installed

Following this scenario, we easily get to connect to Active Directory, Exchange, SCCM and SCVMM. And we could go even further, by integrating this runbook with SCSM, for change management (future post).

Continue reading

Orchestrating Software Updates and Applications Deployment

This scenario continues the idea of the automatically generated virtual machine, adding automatic patching and basic application deployment. All the pieces will fit together in the next post.

With this example I want to point out that you don’t necessarily have to do everything in Orchestrator. Since it relies a lot on SCCM, there is a lot of flexibility in this runbook, it’s up to you how much do you automate in Orchestrator and what it remains to be done in SCCM.

20131110 - 0 0 Small Picture

Continue reading

Invalid Runbook in SCSM

Changing parameters in a runbook after it was synced in SCSM may change its state to “Invalid”. To fix this you need to go in SCSM and delete the synced runbook and run the Orchestrator connector again. If your SCSM machine is slow ( like mine :) )  you need to wait a little until the “Delete” action appears in the console, on the first instance you get only “Refresh”

More on this topic: Runbook States in Service Manager
> V.

Generate Unique VM Name

This small runbook is a demonstration on how to use counters and variables and some basic smart link conditions. It also emphasizes one important difference between Opalis and SC Orchestrator (spoiler: cycles are not allowed! More about this on this cool post: http://www.sc-orchestrator.eu/index.php/scoblog/49-cycles-in-pipleline-mode)

The scenario for this runbook: You want to create a VM but you want the name to be auto-generated following some basic naming convention and you want to make sure that is unique. You also might have users manually creating VMs using the same naming conventions so you need to check this carefully. The runbook will look like this:

Unique VM Name

 

Continue reading