Tuesday, June 22, 2010

Episode #101: Third-Party Party

Ed Genially Sets Up:

Sometimes, doing things at the Windows cmd.exe shell can be really tough as we've seen several times recently here, here, and even here. And, don't even get me started about Episode #100! What a debacle.

But, on the other hand, sometimes this venerable lil' shell gives us some useful functionality, easily accessible. Yes, believe it or not, there are instances where some things are really straight-forward and fun on Windows, and are inscrutably ugly on Linux. I spend a considerable amount of time thinking about such things to throw Hal into a tizzy. Consider, if you will, the topic of gathering information about third-party products installed on a box.

On Windows, we can turn to our faithful friend, wmic:

C:\> wmic product list brief
Caption IdentifyingNumber Name
Vendor Version
Microsoft .NET Framework 3.5 SP1 {CE2CDD62-0124-36CA-84D3-9F4DCF5C5BD9} Micros
oft .NET Framework 3.5 SP1 Microsoft Corporation 3.5.30729
VMware Tools {FE2F6A2C-196E-4210-9C04-2B1BC21F07EF} VMware
Tools VMware, Inc. 8.2.4.7509

Your output will include the product's name, Vendor, and version number, along with a unique identifying number for each product. That version number is typically quite explicit, giving you an item you can research to see if the given program is vulnerable.

On some systems, it can take a while for this command to run. But, when it finishes, you should have a nice list of most of the third-party products installed on the machine. Note that this list includes programs installed using a standard Windows installer. If someone simply copies a program somewhere, it won't find that.

We can get more detail, including the InstallDate and InstallLocation by running:
C:\> wmic product list full

Description=Microsoft .NET Framework 3.5 SP1
IdentifyingNumber={CE2CDD62-0124-36CA-84D3-9F4DCF5C5BD9}
InstallDate=20091203
InstallLocation=
InstallState=5
Name=Microsoft .NET Framework 3.5 SP1
PackageCache=c:\Windows\Installer\52814a.msi
SKUNumber=
Vendor=Microsoft Corporation
Version=3.5.30729


Description=VMware Tools
IdentifyingNumber={FE2F6A2C-196E-4210-9C04-2B1BC21F07EF}
InstallDate=20100404
InstallLocation=
InstallState=5
Name=VMware Tools
PackageCache=C:\Windows\Installer\2ed60.msi
SKUNumber=
Vendor=VMware, Inc.
Version=8.2.4.7509
Want even prettier output? You can get it in HTML format in a file called C:\products.htm by running:
C:\> wmic /output:c:\products.htm product list full /format:hform.xsl
Open that sucker in a browser and behold the results:
C:\> products.htm
Or, if you prefer a csv file, you could run:
C:\> wmic /output:c:\products.csv product list full /format:csv.xsl
Now that I've set this up... let's see what my beloved fellow bloggers have up their command-line sleeves.

Update: Reader John Allison writes in:

Just wanted to let you know that Windows 7 has changed how the WMIC “/format” switch works. You can no longer append “.xsl” when specifying the CSV or HFORM format. If you do you get an error “Invalid XSL format (or) file name.” If you just use the keywords CSV or HFORM it works fine. Thanks for all the hard work you put in on this blog and keep it up.

Good stuff, John! Thank you for that.

Uptade: Reader Rickard Uddenberg writes in:

To use the /format switch you have to move the localisation-files to correct forlder. They are in the C:\Windows\System32\wbem\en-US, but since i have sv-SE i have to create that folder and copy all files to it... (or at least the XML-files). With out that copy it doesn't work, and you get the infamous "Invalid XSL format (or) file name.".

Tim rolls up his sleeves:

Sometimes I feel bad for Ed. Episode 99 was brutal and I can't even begin to describe how I would have done the cmd portion in Episode 100. Fortunately, it was pretty easy for him this week, and even better (for me) since my portion is just a rip off of Ed's fu.

As we have previously discussed, the PowerShell equivalent of wmic is Get-WmiObject (alias gwmi). Most wmic classes require a win32_ prefix in PowerShell. Now that we have that knowledge, let's rip off Ed's portion do some PowerShell.

PS C:\> Get-WmiObject win32_product

IdentifyingNumber : {DBBC72B2-2442-4B8B-9D70-2ED6C0322916}
Name : VMware vSphere PowerCLI
Vendor : VMware, Inc.
Version : 4.0.1.2164
Caption : VMware vSphere PowerCLI
We can get even more detail by piping the output into Format-List *. If we splat the Format-List cmdlet we get every property, not just the common ones, but the output is huge. This is one reason why I like to export the results into CSV.

PS C:\> gwmi win32_product | Export-Csv product.csv
There are two benefits to exporting to CSV. First, we can use something like Excel to create a pretty report. Second, we can easily import the results to quickly manipulate the objects. Here is how we do the import:

PS C:\> Import-Csv product.csv | select Name
The really cool thing is that we can treat the imported objects the same as the original command output. In my tests it was nearly 100 times faster to import the CSV (0.13 seconds) than to re-run the Get-WmiObject command (10.3 seconds). This is handy if you want to filter in different ways or hand off your results for someone else to work with.

Ed exported a few different ways, and we can do the same thing. The two most common are HTML and XML. The output of ConvertTo-Xml and ConvertTo-Html writes the output to standard output. If you want to save the output as a file you also need to use Out-File or output redirection (>).

PS C:\> gwmi win32_product | ConvertTo-Html | products.html
PS C:\> gwmi win32_product | ConvertTo-Xml > products.xml
So Ed and I have had a pretty easy week. This is pretty hard for Hal, but I don't have much pity for Hal, he makes our lives miserable.

And Hal has to take out the garbage:

Ed just wouldn't let this one alone, so I figured I'd give him his tawdry little moment of glory. Yep, managing third-party software on different Unix variants ranges from the trivial to the impossible. Let's survey the carnage, shall we?

Debian systems have a very active group of package maintainers and nearly all of the Open Source third-party packages you would want to use are included in the Debian repositories. That means that on Debian systems you can mostly rely on the built-in package management tools to tell you about third-party software. As we've covered in Episode #90 you can use "dpkg --list" to get information about all installed packages on the system.

In general, Red Hat systems have relatively less packages available in the core OS repositories. But Red Hat admins should really get to know Dag Wieers' RPM repository, which as of this writing contains over 95,000 packages in RPM format. Using the repository couldn't be simpler, since the necessary configuration files are themselves incorporated into an easy to install RPM:

rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm

The above command installs the necessary configuration files for obtaining x86_64 architecture RPMs for RHEL 5.x releases. Similar commands for other processor architectures and OS revisions can be found on Dag's installation and configuration page. Once you've got the appropriate configuration files installed, then you can "yum install" extra third-party packages to your heart's content.

But the question posed by this week's Episode is how to inventory those packages once you've installed them. I haven't found a way for yum to directly tell me what packages have been installed from a specific repo, but I have come up with the following hack:

# yum --disablerepo=rpmforge list extras
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
* addons: mirror.5ninesolutions.com
* base: mirrors.usc.edu
* extras: mirrors.cat.pdx.edu
* updates: mirrors.usc.edu
Extra Packages
VMware-server.x86_64 2.0.2-203138 installed
libpfm.x86_64 3.2-0.060926.4.el5 installed
mpack.x86_64 1.6-2.el5.rf installed
pfmon.x86_64 3.2-0.060926.5.el5 installed
rpmforge-release.x86_64 0.3.6-1.el5.rf installed

Here I'm explicitly telling yum to ignore the Dag's rpmforge repo and then asking it to "list extras"-- meaning list all packages that cannot be associated with any of the remaining repos that we are using. This does get me a listing of several packages I've installed from Dag's site, but it also lists other packages like VMware-server that I've downloaded and installed directly from the vendor site.

BSD systems generally use some variant of the ports system for handling Open Source third-party software. "pkg_info -a" will give you tons of information about installed packages-- exactly how much info depends on which BSD variant you're using. FreeBSD gives you lots of information because a typical base install includes dozens if not hundreds of packages out of the ports tree. At the opposite end of the spectrum, OpenBSD handles the core install separately from the ports mechanism, so "pkg_info -a" only lists packages that you've explicitly loaded from the ports.

So far I've only really been talking about Open Source third-party packages. The problem with commercial software is that there's no standard for how such software ends up being installed. Some vendors might give you their software in your system's native package management format-- e.g., the VMware example we saw above. But other vendors might use a proprietary installation framework or even require you to build their package from source. In these cases, there really isn't a good way to tell whether a given software package is installed other than knowing the standard installation directory and manually auditing your systems.

Similarly, if you download software from the 'net in source code form and install it on your systems, then you won't be able to track that software using your built-in package management system. If you have a software package that's only available in source code form and which you need to use widely in your environment, you might consider making your own installation package in whatever format your system(s) use natively. While it's extra work to create such a package, it does make distributing the software around your network a lot simpler and pays long-term dividends when it comes to tracking and upgrading the software.

Seth (Our Go-to-Mac Guy) adds:
So the nice thing about having a Mac is that you can get all of the power of 'nix without all the chaos of something that isn't centrally managed; it's similar to RedHat in that regard. With its BSD underbelly, the Mac has similar commands to gather data about installed software as well as having some Apple-provided tools.

For the rest of Seth's insights on Mac 3rd Party command line, check out his posting here.