Tuesday, February 23, 2010

Episode #83: Faster. Higher. Stronger.

Tim goes for Gold:

The Olympics are in full swing. The world's finest athletes are doing everything they can to shave fractions of a second; and not get caught doping. This episode we try to shave off precious seconds from our tasks, and not get caught with our (Irish) coffee at work (again).

We all have those commands that we use regularly, and accessing those commands more quickly would save some time. With PoweShell we can do just that by using Set-Alias.

PS C:\> Set-Alias -Name ss -Value Select-String

The Name and Value paramters are positional, so we don't have to use the parameter names. This command does the same thing:

PS C:\> Set-Alias ss Select-String

We have created our alias, now how can we use it? If we wanted to search all the files in a directory for the word "test" we had to use this command:

PS C:\> gci | Select-String test

...but now we can use this shorter command:

PS C:\> gci | ss test

The second command is half the length of the first command command. That is a nice efficiency gain.

What if we regularly checked the event log to see the five latest items? Obviously a short command would save us some time, but see what happens when we try to create an alias.

PS C:\> Set-Alias g5 Get-WinEvent -MaxEvents 5
Set-Alias : A parameter cannot be found that matches parameter name 'MaxEvents'.

That didn't work, but we can do it, we just need to use a function.

PS C:\> function Get-Last5Events { Get-WinEvent -MaxEvents 5 }
PS C:\> Get-Last5Events

TimeCreated ProviderName Id Message
----------- ------------ -- -------
2/23/2010 8:12:1... Service Control ... 7000 The Diagnostic S...
2/23/2010 8:12:1... Microsoft-Window... 135 The Diagnostic P...
2/23/2010 8:12:1... Service Control ... 7000 The Diagnostic S...
2/23/2010 8:12:1... Microsoft-Window... 135 The Diagnostic P...
2/23/2010 8:11:4... Service Control ... 7036 The Computer Bro...

The name we picked for our function is a bit long, so let's use Set-Alias to create an alias for the function.

PS C:\> Set-Alias g5 Get-Last5Events

So we've shaved a few seconds off of our commands, now on to the doping.

In PowerShell we can use snap-ins and modules to extend the shell. There are modules and snap-ins for managing Active Directory, Group Policy, Diagnostics, Exchange 2007 and 2010, SharePoint 2010, IIS 7, VMware, and many more servers and services.

Snap-ins load sets of cmdlets and providers. Modules, which are only available in v2, can include cmdlets, providers, functions, variables, aliases, and much more. Modules are easier to create than snap-ins and they appear destined to replace snapins as the main way to extend PowerShell. You programmers can think of modules as a "class" while a snap-in is just a collection of functions.

Here is how a snap-in is loaded:

PS C:\> Add-PSSnapin VMware.VimAutomation.Core

Wildard characters can be used in the module name. I use it since I can't ever remeber to type VMware.VimAutomation.Core, so I just type *vmware*. Let's see what cmdlets have been added.

PS C:\> Get-Command -Module *vmware*

CommandType Name Definition
----------- ---- ----------
Cmdlet Add-VMHost Add-VMHost [-Name] <String> ...
Cmdlet Add-VMHostNtpServer Add-VMHostNtpServer [-NtpSer...
Cmdlet Apply-VMHostProfile Apply-VMHostProfile [-Entity...
Cmdlet Connect-VIServer Connect-VIServer [-Server] <...

Now, let's load a module and then see what new cmdlets are available:

PS C:\> Import-Module GroupPolicy
PS C:\> Get-Command -Module GroupPolicy

CommandType Name Definition
----------- ---- ----------
Cmdlet Backup-GPO Backup-GPO -Guid <Guid> -Pat...
Cmdlet Copy-GPO Copy-GPO -SourceGuid <Guid> ...
Cmdlet Get-GPInheritance Get-GPInheritance [-Target] ...
Cmdlet Get-GPO Get-GPO [-Guid] <Guid> [[-Do...

We have these new aliases, functions and cmdlets, but we don't want to go through the same setup everytime. We can automatically load these by editing our profile, but where is it?

There are four profiles, but we typically only work with the one stored in $profile. This file may not exist, so you may need to create it.

PS C:\> $profile

We can edit the file with good ol' notepad.

PS C:\> notepad $profile

Once we have it open we can add all of our aliases, functions, snap-ins and modules.

Set-Alias -Name ss -Value Select-String
function Get-Last5Events { Get-WinEvent -MaxEvents 5 }
Add-PSSnapin VMware.VimAutomation.Core
Get-Command -Module GroupPolicy
Write-Host "Welcome to Tim's shell..."

This is a good spot to add those commands and functions that aren't built into PowerShell such as Test-Hash and Get-Netstat.

Let's see how the other shells compete.

Hal's Been There, Done That

"See how other shells compete"? Kid, Unix shells were medalling in the Olympic Aliasing event before Powershell was even a gleam in some demented Microsoft developer's eye.

Setting up aliases in bash is straightforward. Here's how I make it easy to access the history command in my shell:

$ alias h=history
$ h
501 alias h=history
502 h
$ type h
h is aliased to `history'

Notice that the type command will tell you when a given command is an alias (or a shell built-in, or a regular command, or...), and thus is preferrable to commands like which.

Aside from using aliases to shorten commands that I use frequently, I'll often use aliases as a way of making sure I get the program I want, even when I type the wrong thing:

alias more=less
alias mail=mutt

And, yes, I still regularly use mutt to read my email. You got a problem with that?

Aliases can always have multiple arguments, you just need to be careful with your quoting:

alias clean='rm -f *~ .*~'
alias l='ls -AF'
alias tless='less +G'

Aliases can do pretty much any shell code you can imagine, but they do have one shortcoming: any variables you put into an alias are expanded when the alias is read during the start-up of your shell. You can't generally have variables that get interpolated when the alias is executed. For example, I wanted to create an alias that did a "cd" to whatever directory the last file I accessed lives in. But that requires run-time interpolation in order to know the last file accessed, so I can't do it in an alias.

However, you can use shell functions for this:

$ function f { cd `dirname $_`; }
$ cp ~/.bashrc ~/stuff/testing/src/clkf/aliases/bashrc.example
$ f
$ pwd

I called the function "f" for "follow"-- as in, "copy that file to a directory and then follow it over there". It's hugely useful.

You'll notice that the function uses the magical bash builtin variable "$_", which is always set to the last argument of the previous command. So my function only works if the last argument of the last command is a file name, but that's good enough for my purposes. If I had specified a directory instead of a file name in the cp command, then I could have just used "cd !$" instead of "f":

$ cp ~/.bashrc ~/stuff/testing/src/clkf/aliases
$ cd !$
cd ~/stuff/testing/src/clkf/aliases

If you're wondering what the heck that "!$" is all about, you need to go back and check out Episode #14.

The best place to put your aliases and shell functions is in your ~/.bashrc file. That way, they'll get read every time you fire off a shell.

Hmmm, I hope this isn't another one of those "easy for Unix (and Powershell), hard for CMD.EXE" kinds of things. Ed always gets so cranky about those...

Ed's Not Overly Cranky Today
Cranky? Moi? Nevah. Well, mostly never. Ok... sometimes.

Despite my non-cranky demeanor, support for aliases in the cmd.exe shell isn't particularly strong. In this corner of the shell-o-sphere, we typically apply two common approaches to mimicking aliases: doskey macros and small batch files. Let's look at each.

Didja shudder just a little bit when I mentioned doskey? Yeah, it's very old skool, but it can help us get some work done. To define a macro, you could simply run:
C:\> doskey <MacroName>=<Macro>
For example, if you want to display your running processes by simply running "ps", you could run:
C:\> doskey ps=wmic process list brief
C:\> ps

HandleCount Name Priority ProcessId ThreadCount WorkingSetSize
0 System Idle Process 0 0 1 24576
426 System 8 4 97 581632
That simple macro is nice, but it's got a big limitation we've got to overcome by expanding our macro knowledge. To see this limitation, let's try running our new macro and searching its output for cmd.exe:
C:\> ps | find "cmd.exe"
HandleCount Name Priority ProcessId ThreadCount WorkingSetSize
0 System Idle Process 0 0 1 24576
426 System 8 4 97 581632
Doh! We only got the full output of our ps macro, without our search taking effect. What happened? Well, the macro substitution by default just ignores anything that follows it on the command line, unless we define the macro to end with $*, which holds the remainder of the command line typed in after the macro. By putting it at the end of our macro, everything typed after the macro will be executed after macro expansion. So, a better ps would be:
C:\> doskey ps=wmic process list brief $*
C:\> ps | find "cmd.exe"
23 cmd.exe 8 676 1 1462272
28 cmd.exe 8 4008 1 2785280
That's what Momma likes.

Next, to more closely mimic what Hal does above with shell history by simply running the "history" command or the "h" command, you could use:
C:\> doskey history=doskey /history $*
C:\> doskey h=doskey /history $*

C:\> h
Note that I did use $* here, in case someone wants to pipe our output to another command or to redirect it into a file. If you just run the macro with nothing after it, $* contains nothing, so it executes as you'd expect... no harm, no foul. You can even split up the command line options following your macro, referring to each component as $1 up to $9, using each independently. And, if you want to have multiple commands in a macro, enter them as command1$Tcommand2, rather than using & between them. But, if you really want to use &, modern versions of Windows allow you to define the macro as command1 ^& command2. Likewise, if you want to do any piping in doskey macros, make sure you use a ^|.

If you ever want to list all of the macros you've created, run:
C:\> doskey /macros
It should be noted that these macros take precedent over any normal commands you type at cmd.exe, so you could really make your shell useless with them. For example, if you run:

c:\> doskey dir=echo No Way Dude!

c:\> dir
No Way Dude!
You've lost access to the dir command. To undefine a macro, simply use doskey to redefine a macro name with nothing after the equals sign:
C:\> doskey dir=
C:\> dir
Volume in drive C has no label.
Volume Serial Number is 442A-03DE
Now, these little toy macros we've defined above are cute, but let's get into some macros that really save us some serious time. Faithful readers of this blog (Hi Mom!) know that many of my episodes include certain constructs, like making a command run forever or invoking delayed variable expansion. Let's look at some macros for those:
C:\> doskey forever=for /L %z in (1,0,2) do @$*

C:\> forever ipconfig /displaydns & ping -n 2 > nul
Here, I've created a macro called "forever" that invokes a FOR /L loop, set to count from 1 to 2 in steps of zero. That way, it'll run forever. In the DO clause of my loop, I turn off the display of commands (@) and run whatever follows the invocation of forever. Note that I have to define a variable for my FOR loop, and I've chosen %z. That's because I want to minimize the chance I'll have a collision with any variables I might choose for my command to forever-ize. If I had used a %i in my macro, I'd really be thrown for a loop (no pun intended) if I used %i in my follow-up command. With FOR /F loops that have multiple tokens, variables are dynamically allocated alphabetically, so I hang out at the end of the alphabet here to lower the chance of collision.

You could put your ping delay inside the forever macro, but I find it best to keep it out, giving me more flexibility as I define my commands. Note that the command I'm running after forever here will run ipconfig to dump the DNS cache, followed by a one-second delay introduced by pinging myself twice (first ping happens immediately, followed by another ping one second later).

And, to perform delayed variable expansion (described in Episode #12), I could define a macro of:
C:\> doskey dve=cmd.exe /v:on /c "$*"
Note that you can put stuff in a macro _after_ the $* remainder of your command-line invocation, as I'm putting in a close quotation mark here to make sure that my full command gets executed in the context of delayed variable expansion. Now, I can do crazy stuff like this from Episode 58, filling the screen with random numbers to look a little like the Matrix:
C:\> dve for /L %i in (1,0,2) do @set /a !random!
It's important to note that any environment variables whose value changes in your command must be referred to using !variable_name!, instead of the more traditional %variable_name%.

Ahhh... "But wait," you say. After running dve in this example, I manually typed out a FOR /L loop that was simply my "forever" macro from before. Couldn't I just do a dve followed by a forever? Let's try it:
C:\> dve forever set /a !random!
'forever' is not recognized as an internal or external command, operable program
or batch file.
No sir. You can't nest these suckers. Also, they have a problem if they aren't the first command included on your command line. Consider:
C:\> echo hello & ps
'ps' is not recognized as an internal or external command,
operable program or batch file.
So, we've got some pretty serious limitations here*, but, as Tim points out, they can shave off a precious few seconds, especially for things we often start our command-lines with, such as the forever FOR loop or delayed variable expansion.

Wanna see a really whacked out macro? This one was provided by Brian Dyson, a cmd.exe warrior like no other (and a long-lost twin of my friend Mike Poor, but that is a story for another day). Brian showed me a macro in which he emulates the !command feature of bash, which Hal alludes to above. Check out this amazing action, which I quote from Brian:

For the hard-core DOSKEY macro lover, a '!' DOSKEY macro that acts (somewhat like) Bash, running the last command:
c:\> doskey !=doskey /history ^| findstr /r /c:"^$*" ^| findstr /rv /c:"^!" ^>
"%TEMP%\history.txt" ^&^& ((for /f "tokens=*" %a in (%TEMP%\history.txt) do
@set _=%a) ^&^& call echo.%_^% ^& call call ^%_^%) ^& if exist "%TEMP%\history.txt"
del "%TEMP%\history.txt"
Here we get the history and pipe it into a `findstr` command searching
for commands that begin with the arguments to `!`. We remove any
previous `!` command and redirect everything into a temporary file. (I
couldn't find a work-around for the temporary file). If the final
findstr was successful, parse through the temporary file and set '_' to
the last command (like Bash). If this was successful, then echo out the
command and call it via double `call`. Finally, clean up the temporary
history file.

Dude! Nice. Now, to invoke Brian's macro, you have to run ! followed by a space, followed by a command or letters and it will invoke the last instance of whatever previous command started with the letters you type. Check it out:
C:\> ! di
Volume in drive C has no label.
Volume Serial Number is 442A-03DE

Directory of c:\
That space between the ! macro and the first part of the command we want to run is very important. Without it, cmd.exe tries to find a command called !di and bombs. With it, Brian's macro kicks in and expands it to the most recent command that starts with those letters.

Note that a given set of macros only applies to the shell in which it is created. Your macros aren't carried to other currently running shells, nor do they even apply to child shells that you spawn. If you exit your shell, they are gone. If you want to make your macros permanent, you should first define them, as we show above and put all those definitions in a file. Then, you can export them into a file, by running:
C:\> doskey /macros > macros.txt
You can call the file anything you want, but I like calling it macros.txt because it's easy to remember. Then, at any time, you can import these macros by running:
C:\> doskey /macrofile=macros.txt
If you want to apply your macros to every cmd.exe you launch going forward, you can place your macros.txt in a convenient place on your system (such as your common user home directory or even in system32). Then, put a command like the following into any of your autostart locations:

%systemroot%\system32\doskey.exe /macrofile=%systemroot%\system32\macros.txt

For macros, I typically put them in the autostart entry associated with the command shell itself, namely the Autorun Registry key at HKCU\Software\Microsoft\Command Processor. You can define this key by running:
C:\> reg add "hkcu\software\microsoft\command processor" /v Autorun /t reg_sz /d
"%systemroot%\system32\doskey.exe /macrofile=%systemroot%\system32\macros.txt"
Be careful! If you already have a command set to autorun via this key, you may want to append the command to your already-existing one by simply inserting an & between the two commands. The reg command will prompt you to confirm or reject the overwrite if you already have something there.

*To get around some (but not all) of the limitations of doskey macros, you could alternatively use small bat files placed in %systemroot%\system32 to kick off familiar commands. For example, you could run:
C:\> echo @wmic process list brief > %systemroot%\system32\ps.bat
C:\> ps
HandleCount Name Priority ProcessId ThreadCount WorkingSetSize
0 System Idle Process 0 0 1 24576
404 System 8 4 97 774144
The advantages of doing your command-line shrinkage with bat files is that you can now run them as little commands themselves, anywhere in your command invocation:
C:\> echo hello & ps
HandleCount Name Priority ProcessId ThreadCount WorkingSetSize
0 System Idle Process 0 0 1 24576
404 System 8 4 97 745472
The downside of this approach is that your bat file must contain whole commands, not just the start of a command, like we can do with macros. That's why my forever and dve examples above work so well as macros and not as bat files. They are the starter of something else, but not whole commands to themselves, as is the ps and history examples we've touched on here.

So, back to Tim's juicing metaphor. You get to pick your poison with cmd.exe and alias-like behavior. While both methods have some limitations, shell jocks can use macros or bat files to shave off a few precious seconds and boost your performance.

Seth (our Go-To-Mac Guy) Shoots & Scores:
Saving time? Who wants to save time? I thought the whole point of CLI tricks was to make things as painful and as drawn out as possible? Oh wait, that's the job of some of the curlers in Vancouver.

When you start talking about an alias on a Mac, the first thing any Mac user is going to think of is a GUI pointer, otherwise known in Mac land as an Alias. These have existed long before the days of OS X and while not as powerful as say Unix Symbolic Links, they do have some nice features. For example, if you move a target file of an alias, the alias will still work as long as the target is on the same file system. Sadly though, they don't work from the command line. Use good old ln for that.

But I get off topic. I'm disappointed in Hal; for all his talk about being quick on the draw and not letting the boss see you enjoying that wonderful concoction of roasted bean water and barley, he doesn't take his shell fu to the next level.

Hal left us with:
$ function f { cd `dirname $_`; }
$ cp ~/.bashrc ~/stuff/testing/src/clkf/aliases/bashrc.example
$ f
$ pwd
Great! But who's to say we can't use aliases?
$ alias lastdir='function f { cd `dirname $_`; }; f; echo Your working directory is now $PWD'
$ cp ~/.bashrc ~/stuff/testing/src/clkf/aliases/bashrc.example
$ lastdir
Your working directory is now /home/hal/stuff/testing/src/clkf/aliases
Now you might ask, what's the point of wrapping a function into an alias, it's extra unneeded text in an already beautiful simple command! True, but it shows the flexibility of one line commands ( I love the semi-colon) and saves us an extra step when we load it into our Mac ~/.bash_profile file.

Aliases also allow us to use variables, you just have to remember to single quote and not double quote the command. Otherwise it will resolve the variable when you set the alias.
$ pwd
$ alias shellfu="echo O Canada! Our $HOSTNAME and native $PWD; echo True patriot love in all they sons $SHELL"
$ shellfu
O Canada! Our HomePC and native /Users/seth
True patriot love in all they sons /bin/bash
$ cd ~/Desktop
$ shellfu
O Canada! Our HomeMac and native /Users/seth
True patriot love in all they sons /bin/bash

So even though we changed the current working directory, our alias is reporting the path that was active when we set the alias. But when we single quote the command:
$ pwd
$ alias shellfu='echo O Canada! Our $HOSTNAME and native $PWD; echo True patriot love in all they sons $SHELL'
$ shellfu
O Canada! Our HomeMac and native /Users/seth
True patriot love in all they sons /bin/bash
$ cd ~/Desktop
$ shellfu
O Canada! Our HomeMac and native /Users/seth/Desktop
True patriot love in all they sons /bin/bash

Ok, so what have we learned? First, you can use variables in aliases. Secondly, you can use multiple arguments and commands in aliases. Let's make this even more interesting.

Suppose it's 2 am. You've just gotten word that a host is down on your network. What's the first thing you do? Personally I ping it.
$ ping -c 3 curlingrocks.ohcanada.com
PING curlingrocks.ohcanada.com ( 56 data bytes
64 bytes from icmp_seq=0 ttl=128 time=0.991 ms
64 bytes from icmp_seq=1 ttl=128 time=0.558 ms
64 bytes from icmp_seq=2 ttl=128 time=0.403 ms

Ok, so I know that is the correct IP address for that server, so DNS is working and the host is responding to pings. I'm not sure what this server does or where it lives on my company network, but it's really important and the Boss wants it back up ASAP. So what do I do, run some more commands to see what services are running, etc. Probably faster than trying to dig through crummy documentation.

So with my nice little scan alias:

$ alias scan='hping --count 3 --fast --rawip $_; nslookup $_; echo Results 
from $HOSTNAME to $_; traceroute $_; nmap -n -A -PN $_'

After I ping it, I can immediately find out lots of other information about the host without having to wait by just typing in "scan". Now, the reason that the above works, and we don't have to get into functions like Hal did, is because the last argument in each command is the IP that we want and it doesn't change. $_ in the last nmap command is actually referring to the last argument of the traceroute command and so on. If this wasn't the case, we'd have to use a function like Hal showed us or assign our target IP it's own variable.

$ alias scan='TARGET=($_); echo Results from $HOSTNAME to $TARGET; hping 
--count 3 --fast --rawip $TARGET; nslookup $TARGET; traceroute $TARGET;
nmap -n -A -PN $TARGET'

Also remember if you alias your favorite command, and happen to name it something else useful‚ top perhaps. You can always ignore the alias with a backslash.

$ \top

Talk about an easy way to disguise the up'ers!