Sustainable Powershell - Cmdlets

Since we're automating work, and (currently) a lot of the work involves Windows ecosystems, the natural tool of choice is Powershell. Let's look at some general guidelines for writing sustainable Powershell cmdlets.

For the purpose of this post, I'll use functions and cmdlets interchangeably as they can both be considered atomic work units.

Powershell Cmdlet Guidelines

Cmdlets shall:

Use the Verb-Noun Naming Scheme

Using this scheme forces you to think more about what your functions/cmdlets should be doing. It's the decided 'best practice' from Microsoft and when you import a module that violates this scheme, you get a nice nagging message from Powershell telling you exactly that.

Protip: Not that you should use it, but you can pass the -DisableNameChecking parameter to Import-Module to suppress the warning. You should know, however, that if you need to suppress the warning, you're probably developing your cmdlets the wrong way.

Use Camel Case for Capitalization

For instance, Get-DetailsFromReallyLongName.

The corollary to this is use descriptive names. The year is 2013. We have plenty of disk to store code with long names. Use it.

Cmdlets Do Not Hold State

 Cmdlets do stuff, they do not hold things. If you're familiar with RESTful interfaces and/or stateless architecture, move to the next point because you already understand.

For the rest of us: A cmdlet should not hold onto a value. That is, it should not store something in a global/environment variable. It can check some environment variables if it needs to make a decision, but it should not communicate with the user/other cmdlets via manipulating those variables.

If you need data, put it in/get it from a data source, such as a database, the Windows registry, or even a flat text file. Don't store it in memory (e.g. a variable) because you'll create problems for yourself that distract you from the real work.

Use "Approved" Verbs

Personally, I'm a little bitter that the creators of Powershell drew a line around what we can and cannot use when writing our own modules but that doesn't change the fact that we have to deal with it. And besides, it does help to keep you in the right mindset when writing functions/cmdlets.

When writing modules, your functions need to start with 'approved' verbs. To get a list of those verbs, use the command Get-Verb or see Microsoft's writeup.

Do One Thing. Do It Well.

See also the Unix Philosophy of software development. Forty years of tireless development and crushing testing environments have made this philosophy even more relevant.

Use Command-Line Parameters

When thinking of how to pass information to your cmdlet, think in the same way you write your cmdlets/functions: simple and atomic. Don't pass an object representing information to your cmdlet (because that breeds complexity, which leaves you prone to errors). Instead, simply pass the information.

For example, if a cmdlet needs information about a server, don't create an object just to represent that server. Instead, have your cmdlet accept the text version of that information.

Don't Do This
Create-NewServer -serverDetails complicatedObject 
Where complicatedObject is an object or a hash with values describing the server.

Instead, Do This
Create-NewServer -ServerName worker01 -IP 192.168.10.1 -Netmask 255.255.255.0

Writing your function will take more time and the incantations will be more verbose. B-U-T in the future when (not 'if', but 'when') you need to update that function, making the required changes will be so much easier than updating code that expects more complex input.

In summary, this is the state we're in today. The environment will change and constraints will improve but, in the meanwhile, following these guidelines will reduce (not eliminate!) tomorrow's maintenance cost of the software you write today.  

Sustainable Automation

When you automate technology (at a meaningful scale, anyway), you invariably wind up writing code. I've yet to see some automation technology that was all point-and-click. Chef, Puppet, Bash, Python, Powershell.... all of them languages.

Since we're writing code for our automation, we might as well write sustainable code. I can sum up the entire point like this:

Writing code has

1. An initial cost. The cost, in time, dollars, or other measurement to write the code and 
2. A maintenance cost. The cost you incur to keep the code relevant.

If you're lucky, your code will survive long enough to accrue a maintenance cost. If that is the case, it is nearly certain that the maintenance cost will be overwhelmingly larger than the initial cost. Therefore, we always seek to reduce the maintenance cost.

Or, in more interesting terms (and I cannot take credit for this version), write your software as if it will be maintained by a homicidal maniac who knows where you sleep.

Now that we know what to do (reduce maintenance cost), we need some direction on how to do it. The answer, as is most often the case, is simplicity. How do we keep things simple?

I like to borrow a few notes from the Unix Philosophy, specifically 'keep things modular' (Small is Beautiful). Regardless of technology, you'll be modularizing your work, whether they be in functions, recipes, manifests, whatever. Think of these as your atomic work units, where the keyword here is atomic. Your units should do one thing, and they should do it well. If you want to add functionality to a unit, make another unit and have your first one call it.

Why? Why not just add functionality to the existing work units? It's easier, there are fewer units to juggle, and, yes, it takes less time.

Less time?

Less inital time, yes. But we're not working to minimize initial time. We accrue the initial time only once. The maintenance time (i.e. cost), however, we accrue every time we need to update our code. Every. Time. And god save us if we have to refactor our codebase. Forget progress, you'll be doing well just to get back to normal.

This is a quick (albeit contrived) example of my bigger point: Always strive to make things easier for the guy in the future. After all, it just may be you.

Every Job is a Custom Job

No matter how much we standardize, how repeatable or automated our processes are, we will never be able to deliver the exact same product/datacenter/service twice.

This isn't a problem experienced by many other domains (not to such a degree, anyway). I can order a pair of speakers from a manufacturer and they will be the exact same speakers that my neighbor orders. Likewise with most food at the store, even at restaurants. Sure, you can ask them to hold the onions on your burger but that level of customization is not what's experienced by developers/deployers/devops/what-have-you.

The customer will always have some indigenous nuance about their particular system - legacy software/hardware, limited budget, upper management has already decided on a particular product before you arrived - that will preclude you from delivering the *exact* same product every time.

The challenge, then (and what makes the work interesting), is working this reality into your automation processes.

New-Datacenter?

Note: This post is to facilitate conversation on the topic of making a new Windows datacenter. This is far from a howto. If you came here looking for that, you will be disappointed.

In the beginning, there was nothing. Then there was the command line.

But more seriously, I've been playing around with the idea of bootstrapping a Windows data center. Assume you have up to some hypervisor. Since we're making a Windows ecosystem, we can assume VMware products. How do you go from "a few ESXi hypervisors" to "a fully-capable Windows datacenter", complete with all the services you expect from Microsoft?

The Dream

A one-touch solution to deploy an entire Windows datacenter.

The Way Forward

The immediate answer is automation. Powershell automation, specifically. But where does the Powershell magic run from? Can't run on the bare hypervisors. Okay, so we'll need a bootstrapping Powershell.... server (a BPS?), of sorts. So assume we have a few hypervisors and a single Windows host (not on the hypervisor since that hasn't been configured yet). 

We'll also need some binaries. So assume all binaries are also on this BPS. Let's also throw some templates in there, just to get a vCenter started up. This is getting long... for the sake of brevity, let's just say we have a set of distinct scripts that, on their own, will give us all the components we need for our Windows datacenter.

How do we tie them all together?

Devil, something something, Details

This is a bootstrapped environment, so we don't have the luxuries of Active Directory or DFS or anything else that makes life easier. We have a bunch of blank Windows installations. First challenge - we have to get the binaries & scripts to the blank VMs. We could set up a share on our BPS, but our blank VMs don't know how to get to it, and we can't even use a Workflow from our BPS because, with no server certs, WinRM will only use the 'default' configuration, which limits you to a single hop for cred-forwarding.

The only thing I've come up with is to set up a minimal IIS installation and have the blank VMs download the necessary files over HTTP. We can use some simpler 3rd party web servers, but then we'd be introducing non-native products, which only complicates things.

Assuming we can transfer the data (binaries) and the instructions(scripts) to the blank VMs, how do we choreograph the installation of the products that make up our datacenter? I'm imagining something like a Master Powershell Workflow run from the BPS that knows the order of operations. 

This is getting complicated. And I'm concerned that it's unnecessarily so.

Externalize Away the Setbacks

And all of this is to say nothing about sustainment. You can create a bunch of stuff. Nice. Now how about configuration management? I don't know either. Maybe this has all been done before and I'm just not aware of it. Sounds like more research...

Welcome to Powershell Workflows. Please Follow These Rules

The most powerful feature introduced in Powershell 3.0 is The Workflow, no doubt. However, workflows are not something exclusive to Powershell. In fact, workflows (that is, the Windows Workflow Foundation) have been around since .NET 3.0 (recall that Powershell 3.0 requires .NET 4.0). When you are using workflows in Powershell, you have left the well-known comforts of Powershell. You are now a guest in the Windows Workflow Foundation.

Activites, not Commands
In each workflow you write, the individual lines are no longer commands. They are individual Activities, as per the Windows Workflow vocabulary. As such, they run in their own process. The implications here are great. For the sake of brevity, I'm only covering the cases that have occupied most of my time.

No Module Imports
Import-Module is one of the (many) disallowed cmdlets in workflows. In order to get around this, we have been given the -PSRequiredModule parameter.

Not this:
Import-Module AwesomeSauce
Import-Module RequiredModule
$goods = Get-OutTheGreat

This:
$goods = Get-OutTheGreat -PSRequiredModules 'AwesomeSauce','RequiredModule'

Snap-In Complications
Add-PSSnapin is also disallowed. This is immediately a problem for fans of PowerCLI, as it's written as a Snap-In, not a module. The best workaround I've found (though it's kludgy) is to wrap the necessary work in an InlineScript.

Not this:
Add-PSSnapin vmware.vimautomation.core
Get-VM

This:
InlineScript{
    Add-PSSnapin vmware.vimautomation.core
    Get-VM
}

No Implied Parameters
When using workflows, you must explicitly spell out each parameter for every cmdlet you use. This can be a bit fun when learning the name of that parameter you've been taking for granted. The new ISE can be helpful with discovering cmdlet parameters.

Not this:
Get-VM BigGiantVM

This:
Get-VM -Name BigGiantVM

Verbose Workflows
One overlooked feature of workflows is the Activity Common Parameters. These are parameters you pass to individual activities. My favorite so far is -DisplayName. With this parameter, you can label the activities in your workflow so they appear more human friendly, such as "Connecting to remote host AwesomeHost" instead of "Workflow: Line 12, Character 8" while executing. 

Creativity Under Constraints
I've found myself complaining about these details along the way. But actually, they may be forcing me to write better code. By disallowing the easy (and unsustainable) path, I'm reducing what used to be complex scripts/functions into more atomic, modular functions/workflows. This is the beginning of your own personal framework. I say embrace the changes. Besides, it's not like we have much of a choice.

I Just Want Powershell Remoting!

In some bootstrapping environments, you don't have fancy things like Active Directory domains or valid DNS entries. You have to connect to machines with IP addresses and no machines have certificates yet. This is the scenario where Workflows can matter most and it's where most Powershell Remoting howto's have failed me. However, there is a fast and (relatively) easy way to have Powershell Remoting (and, thus, Workflows) in such arid, hostile environments.

The Situtation
You're logged into machine Foo. You want to execute remote Powershell scripts (or get a Powershell session) on machine Bar. There is no AD environment and you can only address machines by IP.

The Details
Foo's IP: 192.168.0.1
Bar's IP: 192.168.0.2

Do this
On both machines, in a Powershell session with Admin rights:
Enable-PSRemoting -Force

This enables the WinRM service, pokes a hole in the Windows Firewall, and does a few other things to enable remote Powershelling.

 Then...

Set-Item WSMan:\localhost\Client\TrustedHosts -Value '192.168.0.*' 

This adds the network 192.168.0.* to the list of Trusted Hosts, which is a requirement when you use Remote Powershell with 'default' authentication in WinRM.

You could even run the above commands in an OS template, so that all new Windows machines in your virtual environment will accept remote Powershell sessions.

Test it
On Foo, run:
Enter-PSSession -ComputerName 192.168.0.2 -Credential $(Get-Credential) 

The other requirement when using Remote Powershell with 'default' authentication is that you specify explicit credentials (no pass-through creds). So enter your credentials at the prompt and enjoy your new remote Powershell session. You can also now run Workflows in your environment, so you can get back to bootstrapping your datacenter!