Episode #68: Help Fu

Tim hits lead off:

This week we get help with help. Everyone needs help, and some in different ways than others. But we aren't going to discuss Ed's problem with crunchy peanut butter or Hal's issue with furry animals with tails. Anyway, back to the issue at hand.

Instead of jumping right in to getting help on commands, let's use PowerShell to find commands by using the Get-Command cmdlet. This command will retrieve all the cmdlets, aliases, and functions available to us. In order to use this command effectively, let's go over a bit of the very basics of PowerShell. As you all probably know, the nomenclature for PowerShell cmdlets is verb-noun. Microsoft's approved verb list defines the verbs so you know which verb to use and to ensure consistent use by cmdlet developers. That way the verbs are consistent so "get" is always used instead of a random choice of get, retrieve, grab, or obtain. The noun part of the name identifies the entity on which the action is performed.

Ed and Hal discussed the command history in episode #27, so we'll use that as the focus of our research.

Let's see a list of all the available nouns.

PS C:\> Get-Command -Type cmdlet | Sort-Object noun | Group-Object noun

Count Name      Group
----- ----      -----
2 Acl       {Set-Acl, Get-Acl}
5 Alias     {Import-Alias, Get-Alias, Set-Alias, Export-...
...
4 History   {Invoke-History, Clear-History, Get-History,...
...

You probably could have already guessed, but the noun we want to use is "History." To get a lits of the commands available using that noun we can use this command:

PS C:\> Get-Command -Noun history

CommandType   Name               Definition
-----------   ----               ----------
Cmdlet        Add-History        Add-History [[-InputObject] ...
Cmdlet        Clear-History      Clear-History [[-Id] <Int32[...
Cmdlet        Get-History        Get-History [[-Id] <Int64[]>...
Cmdlet        Invoke-History     Invoke-History [[-Id] <Strin...

The parameter is not case sensitive and will accept wildcards so "hIST*" would returned similar results. Similarly, we could see all the available commands using the "get" verb using this command.

PS C:\> Get-Command -Verb Get

Back to the subject at hand...

Now we have a list of cmdlets we are interested in. Let's see how we can use the command by using the Get-Help cmdlet.

PS C:\> Get-Help Get-History

NAME
Get-History

SYNOPSIS
Gets a list of the commands entered during the current session.

SYNTAX
Get-History [[-Id] ] [[-Count] ] []
...

We have a bit of a problem here, the basic help isn't super useful since it doesn't provide a good description of the parameters or any examples of the cmdlet's use. However, it does provide syntax and a list of parameters. To see the full help including examples and parameter descriptions use this command:

PS C:\> Get-Help Get-History -Full

To see just the examples:

PS C:\> Get-Help Get-History -Examples

As you have probably noticed in past episodes there are a lot of aliases available in PowerShell. The Get-Alias cmdlet can be used to "get" the list of aliases. Specifically, we can see the aliases for the commands using the History noun.

PS C:\> Get-Alias | Where-Object {$_.Definition -like "*history"}

CommandType     Name                            Definition
-----------     ----                            ----------
Alias           clhy                            Clear-History
Alias           ghy                             Get-History
Alias           h                               Get-History
Alias           history                         Get-History
Alias           ihy                             Invoke-History
Alias           r                               Invoke-History

The Get-Command cmdlet can be used to get the same results.

PS C:\> Get-Command -Type Alias | Where-Object {$_.Definition -like "*history"}

That's about it for PowerShell, let's see what Ed has for us with the Windows command line: Get-Help -FuMaster Ed -Subject Windows

Ed's Output:

You guys with your fancy help capabilities in your shells are a real hoot. Yes, your shells include intuitive and easy-to-access help features. Where’s the fun in that? Or, more to the point, where’s the fu in that? I’ve spent many countless hours sitting at a cmd.exe trying to reverse engineer… uh… I mean conducting detailed design recovery of cmd.exe functionality to try to figure out what the heck the little bugger was really doing. Ah… good times.

In all seriousness though, cmd.exe offers an embarrassingly paltry amount of help for using it and its commands. Really, it’s kind of stinky. Let’s explore the few skimpy options we have.

First off, we can run the help command, followed by the command we want to analyze, as in:

C:\> help findstr

The good news? Well, we’ll see a brief description of findstr functionality, as well as its command flags.

The bad news? Well, where to start? How about:

Bad News #1) The help command doesn’t provide help for all of the commands available in cmd.exe. It only covers a little less than 80 of them.

To get a list of the commands that help has information about, simply run:

C:\> help

Some of the most interesting and useful commands of all, like netstat, netsh, reg, sc, and wmic, result in this unhappy little message:

C:\> help reg
This command is not supported by the help utility.  Try "x /?".

Bad News #2) The output of the help command doesn’t include any examples or much detail at all. It’s just really basic usage, without any interesting fu included.

Bad News #3) Note the little suggestion in the output of my command in Bad News #1: Windows tells us to try running the command with a /? argument for help. As an old Unix guy (shhhh… don’t tell anyone), I’m very hesitant to run a command that I don’t understand just so I can get help. It scares me to do so without knowing what just running the command might do. And, here’s a dirty little secret of the help command itself. It actually runs the command you are trying to get help about! Yes, check this out…

First, start Task Manager, move to the Processes tab, and sort it alphabetically by clicking on the Image Name column.

Now, in a separate cmd.exe, run the following command, which will get help for the findstr command, repeatedly:

C:\> for /L %i in (1,0,2) do @help findstr

Look in your Task Manager window… see it? Every once in a while, the findstr command pops up! So “help findstr” is actually invoking the findstr command itself to scrape out its help options. Yowza. That’s just painful for me.

Bad News #4: Well, with only around 80 commands covered by the help command, we often have to turn to that old stand by, “[command] /?” for more help. But the help included here is spotty at best as well, with major inconsistencies between commands and some commands supporting context-specific help for particular sub-areas of the command. For example, “wmic /?” does offer some help, but there is a wmic-specific option for getting more detailed help:

C:\> wmic /?:full

Besides wmic, none of the other commands I know about support this /?:full option.

Also, some commands have specific help for certain contexts within the command. Consider the following examples:

C:\> wmic process get /?

This shows you a list of all the process attributes wmic supports.

Or, try this:

C:\> wmic process call /?

This one shows you a list of all the process methods you can call from within wmic.

Or, how about this one:

C:\> net /?

And then:

C:\> net use /?

Likewise, we can run “reg /?” to get an overview of the reg command, followed by a “reg query /?” to get more details about the specific syntax for querying the registry.

Want another bewildering inconsistency? Try running this:

C:\> netstat /?

Note that the output actually shows the command flags with dashes instead of slashes (i.e., “netstat –na” instead of “netstat /na”). Thankfully, the actual netstat command on modern Windows boxes lets you use dashes or slashes in its flags.

So, how can we wrap our heads around all of this inconsistency? I’ve found that mastering Linux involves learning certain general principles and then seeing how they are applied throughout the operating system. Whereas, in Windows, mastering the command line involves memorizing a bunch of complex, counter intuitive, and often inconsistent options scattered throughout the operating system without rhyme or reason. So, why bother? Because, once mastered, the Windows command-line is incredibly useful in analyzing systems at a fine-grained level. Also, cmd.exe is pretty much everywhere, pre-installed, so there’s a lot less hit-and-miss than you get with installs of PowerShell. At least, there is for now… this is changing as PowerShell gets more widely deployed.

The bottom line here? When I’m actually looking for help with a command, I apply the following process. I start by trying “help [command]”. If that provides no satisfaction, I proceed to “[command] /?”. If that doesn’t give me what I want, I try to look for context-specific help with “[command] [option] /?”. And, if that doesn’t get me where I need to go, I turn to the web to research commands. One of my absolutely favorite sites for researching Windows (as well as Linux and Unix) commands is the awesome ss64 site. It includes a detailed list of most Windows commands, including their various flags, example usage, and mapping to rough equivalents in Linux, OS X (sometimes), and PowerShell (occasionally). That’s awesome. And, finally, there’s Microsoft’s own command line reference, worth a check from time to time.

Hal cleans up:

There are actually a number of different help systems available on a typical Unix system. First there's the standard on-line manual pages accessed via the "man" command. The Unix manual pages include not only information about programs that you can run from the command line, but also documentation on programming APIs, device interfaces, and even the format of important configuration files on the system.

The trick sometimes is getting to the right manual page. For example, suppose I wanted some information on how to use the chmod() system call in my C program. If I just typed "man chmod", I'd get the manual page for the chmod program and not the documentation on the system call. To distinguish these two different manual pages, the Unix manual has traditionally been organized into sections. Sections 1 (user commands) and 8 (administrator commands) are devoted to command-line tools, while sections 2 (system calls) and 3 (library routines) are devoted to programming APIs. FYI, section 4 is device interfaces, section 5 is configuration file formats, section 6 is games, and section 7 is "miscellaneous" but includes useful tidbits like the table of ASCII values ("man ascii") and additional details on various IPC mechanisms. When a Unix document refers to "chmod(2)" it means "the documentation on the chmod system call in section 2 of the manual".

But how does on pull up the chmod(2) manual page instead of the default chmod(1) page? The man command takes a "-s" option to specify the section: "man -s 2 chmod". But on many Unix variants you can simply drop the "-s" and just type "man 2 chmod".

Suppose, however, that I didn't know which section the chmod() system call was documented in. One of the most useful features of the man command is the "-k" (keyword search) option:

$ man -k chmod
chmod (1)            - change file mode bits
chmod (2)            - change permissions of a file
fchmod (2)           - change permissions of a file
fchmodat (2)         - change permissions of a file relative to a directory f...

You don't have to search on command names, of course. Any related keyword will work. For example, if you're new to Unix and don't know how to rename files:

$ man -k rename
dpkg-name (1)        - rename Debian packages to full package names
lvrename (8)         - rename a logical volume
mmove (1)            - move or rename an MSDOS file or subdirectory
mren (1)             - rename an existing MSDOS file
mv (1)               - move (rename) files
prename (1)          - renames multiple files
rename (1)           - renames multiple files
rename (2)           - change the name or location of a file
rename.ul (1)        - Rename files
renameat (2)         - rename a file relative to directory file descriptors
vgrename (8)         - rename a volume group

By the way, the "apropos" and "whatis" commands are equivalent to "man -k". However, all of these commands operate on special "databases" of information that has been extracted from the manual pages themselves via the "makewhatis" command (aka "mandb" on Linux). Very often your system will have a cron job that builds the whatis databases automatically every week or so, but on some Unix systems you have to build the databases manually (or create your own cron job).

Another tricky issue with the Unix on-line manual is that sometimes you have multiple repositories of manual pages. For example, you might have the standard OS manual pages under /usr/share/man and manual pages for third-party software in /usr/local/man. The man command lets you use the "-M" option to specify a path to use for finding manual pages, but far it's far easier to set MANPATH in your .bashrc file:

export MANPATH=/usr/share/man:/usr/local/man

MANPATH works just like PATH or LD_LIBRARY_PATH. In the above example, the man command will check the system manual page directory first and then look in /usr/local/man if it doesn't find the manual page you're requesting.

The man command is also the source of one of the most famous jokes in Unix:

$ man 'furry animals with tails'
No manual entry for furry animals with tails

OK, so the traditional joke is "man rear", but you get the idea.

Sadly, man pages seem to be dieing off. A lot of Open Source projects have either very scanty or completely non-existent manual pages. It's actually pretty tough to write a good manual page, both in terms of content and also because you need to know nroff/troff formatting commands to format the text. But writing (and reading) manual pages properly is still an important Unix skill IMHO.

Another source of on-line documentation on Unix is the Free Software Foundation's "info" system. The most complete documentation on packages like emacs, gcc, GNU tar, et al is found in the info system-- the manual pages are mostly just stubs extracted from the docs in the info pages. Run the command "info info" to get more information on how to navigate the curses-based info interface. Frankly, I think the user interface for the info system is terrible, but it was mostly designed to be run from within emacs. I suppose that it makes sense if you're one of those people who essentially lives inside of emacs and never touches an actual terminal window.

But the info system never really caught on outside of the FSF. With traditional manual pages being marginalized as well, the main documentation interface these days seems to be the "-h" or "--help" options supported by many commands. Typically, one or both of these options will generate at least a terse summary of command options:

$ info --help
Usage: info [OPTION]... [MENU-ITEM...]

Read documentation in Info format.

Options:
      --apropos=STRING         look up STRING in all indices of all manuals.
  -d, --directory=DIR          add DIR to INFOPATH.
      --dribble=FILENAME       remember user keystrokes in FILENAME.
  -f, --file=FILENAME          specify Info file to visit.
  -h, --help                   display this help and exit.
      --index-search=STRING    go to node pointed by index entry STRING.
  -n, --node=NODENAME          specify nodes in first visited Info file.
  -o, --output=FILENAME        output selected nodes to FILENAME.
  -R, --raw-escapes            output "raw" ANSI escapes (default).
      --no-raw-escapes         output escapes as literal text.
      --restore=FILENAME       read initial keystrokes from FILENAME.
  -O, --show-options, --usage  go to command-line options node.
      --subnodes               recursively output menu items.
  -w, --where, --location      print physical location of Info file.
      --vi-keys                use vi-like and less-like key bindings.
      --version                display version information and exit.

The first non-option argument, if present, is the menu entry to start from;
it is searched for in all `dir' files along INFOPATH.
If it is not present, info merges all `dir' files and shows the result.
Any remaining arguments are treated as the names of menu
items relative to the initial node visited.

Examples:
  info                       show top-level dir menu
  info emacs                 start at emacs node from top-level dir
  info emacs buffers         start at buffers node within emacs manual
  info --show-options emacs  start at node with emacs' command line options
  info --subnodes -o out.txt emacs  dump entire manual to out.txt
  info -f ./foo.info         show file ./foo.info, not searching dir

Email bug reports to bug-texinfo@gnu.org,
general questions and discussion to help-texinfo@gnu.org.
Texinfo home page: http://www.gnu.org/software/texinfo/

Not as useful as a traditional manual page, IMHO, but at least it's something. The biggest problem is that with the documentation being tied up in each individual command's "--help" output, there's no "man -k" equivalent for doing global keyword searches if you're not sure what command you're looking for.

Our friend Jeff Haemer is at it again, with some more "helpful" pointers on his blog. Man, I can't believe I forgot the "help" command, but Jeff will tell you all about it, plus some other neat shell trickery.