Tuesday, September 22, 2009

Episode #61: Just Sit Right Back & You'll Hear a Tale... or a Tail...

Ed muses whimsically:

I'm sure every self-respecting geek has contemplated the scenario. I know I think about it all the time. You're trapped on a desert island, surrounded by nothing but coconut trees, sand, water, and 50 beautiful babes... all living in rustic harmony in your lavish hut. Oh, and you also have your laptop computer, with a power supply and Internet connection. Now, the question before the house, of course, is as follows:

When stranded on a desert island, if you could only have a single command in your operating system, what would it be and how would you use it?

Yes, it's a dilemma that has no doubt puzzled philosophers for ages. I'd like to weigh in on it, and see Hal's thoughts on the matter as well.

For my Windows usage, I'd have to go with WMIC, the Windows Management Instrumentation Command-Line tool. While it's just a command, it opens up whole worlds to us for interacting with our Windows boxen. Built-into Windows XP Pro and later, WMIC can be used to query information from machines and update it using a syntax known as the WMIC Query Language (WQL), which I described in an article a while back.

WMIC can be used as a replacement for numerous other commands, and in many instances it provides even more information than the commands it subsumes. For instance, you could supplant the Service Controller command (sc) that we discussed in Episode #57 with:

C:\> wmic service list full

Or, you can replace the tasklist command with:

C:\> wmic process list full

The taskkill command functionality can be mimicked with:

C:\> wmic process where processid="[pid]" delete

You can get lists of users, including many settings for their account and their SIDs with:

C:\> wmic useraccount list full

Those are the things I most often use WMIC for: interacting with services, processes, and user accounts based on variations of those commands. I've written a lot about WMIC in the past, but I've come up with some new uses for it that I'd like to talk about here. And, getting back to our little desert island fantasy... I mean... scenario, let's talk about some additional WMIC use cases.

Suppose, on this desert island, you wanted to see if a given Windows machine was real or virtual. Perhaps you had hacked into another box on the island or you had this question about your own system. WMIC can provide insight into the answer, especially if VMware is in use:

C:\> wmic bios list full | find /i "vmware"
SerialNumber=VMware-00 aa bb cc dd ee ff aa-bb cc dd ee ff 00 aa bb

VMware detection, in a single command! I'm sure the babes will like that one. Here, I'm querying the bios of my Windows machine, looking for the string "VMware" in a case-insensitive fashion (/i). If you see output, you are running inside a VMware guest machine. Also, you'll get the serial number of that VMware install, which might be useful to you.

Perhaps, with all that spare time on your island paradise, you will start to contemplate the fine-grained inner workings of your Windows box, thinking about the order that various drivers are loaded. Wanna see that info? Use this:

C:\> wmic loadorder list full

On a desert island, I'm sure you'll need to know a lot of details about your hard drive, including the number of heads, cylinders, and sectors (so you can make a new hard drive from coconut shells when your existing one fails, of course). To get that information, run:

C:\> wmic diskdrive list full

At some point, you may need to write up a little command-line script that checks the current screen resolution on your default monitor. There must be a distinct need on desert islands for pulling this information (perhaps just to impress the babes), which can be obtained with:

C:\> wmic desktopmonitor where name="Default Monitor" get screenheight,screenwidth
ScreenHeight ScreenWidth
665 1077

Now, suppose you are conducting a detailed forensics investigation to determine who among your cadre of babes stole the coconut cream pie. The answer might lie in the creation, modification, or last accessed time of a given directory on your hard drive. You can get that information by running:

C:\> wmic fsdir where (name="c:\\tmp") get installdate,lastaccessed,lastmodified
InstallDate LastAccessed LastModified

20090913044801.904300-420 20090914051243.852518-420 20090913073338.075232-420

Note that the path to the directory in this one must use \\ in place of each backslash. The first backslash is an escape, and the second is the real backslash. You have to do this for any where clauses of wmic that have a backslash in them. Also, note that fsdir works only for directories, not files. Still, that should help you crack the case of the missing coconut cream pie!

There are thousands of other uses for WMIC, which can be explored by simply running "wmic /?". As you can see, it is an ideal tool for an intrepid geek in a tropic island nest.

No phone! No lights! No motor cars!
Not a single luxury...
Like Robinson Caruso...
Except for WMIC. :)

Hal's been on the island far too long:

When Ed proposed this question I thought it was kind of unfair for me to get to choose a command plus have all the functionality of the Unix shell. And that got me thinking, just how much could I accomplish using the shell itself with no other external commands? This is not as idle a question as it might first appear: there have been times when I've had to recover hosed systems without much more than the built-in functionality in my shell.

First let's inventory our resources. Built into the bash shell we have cd for navigating the directory tree, echo and printf for outputting data, read for reading in data, and a few miscellaneous commands like kill, umask, and ulimit. We also have several different kinds of loops and conditional statements, plus the test operator for doing different kinds of comparisons. This actually turns out to be a lot of functionality.

Starting off simply, we can create a simple ls command with the echo built-in:

$ cd /usr/local
$ echo *
bin cronjobs depot etc games include lib lib64 libexec lost+found man sbin share src


But that output is kind of ugly, and would be hard to read if the directory contained more items. So we could make pretty output with an ugly loop:

$ i=0; for f in *; do printf '%-20s' $f; (( ++i % 4 )) || echo; done; \
(( $i % 4 )) && echo

bin cronjobs depot etc
games include lib lib64
libexec lost+found man sbin
share src

I'm using printf to output the data in columns (though you'll note that my columns sort left to right rather than up and down like the normal ls command), and a counter variable $i to output a newline after the fourth column.

Emulating the cat command is straightforward too:

$ while read l; do echo $l; done </etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6

We can also use this idiom as a simple version of the cp command by just redirecting the output into a new file. Unfortunately, there's no unlink operator built into the shell, so I can't do either rm or mv (though you can use ">file" to zero-out a file). There's also no way to do the ln command in the shell, nor to emulate commands like chown, chmod, and touch that update the meta-data associated with a file.

However, since bash has a pattern matching operator, we can emulate grep very easily:

$ while read l; do [[ $l =~ ^127 ]] && echo $l; done </etc/hosts
127.0.0.1 localhost.localdomain localhost

In a similar vein, we can also count lines ala "wc -l":

$ i=0; while read l; do ((i++)); done </etc/hosts; echo $i
4

While our cat emulator works fine for small files, what if we had a longer file and wanted something like more or less that would show us one screenful at a time:

$ i=0; \
while read -u 3 l; do
echo $l;
((++i % 23)) || read -p 'More: ';
done 3</etc/passwd

root:x:0:0:root:/root:/bin/bash
[... 21 lines not shown ...]
smmsp:x:51:51::/var/spool/mqueue:/sbin/nologin
More:

After every 23 lines, I use the read command to display the "More: " prompt and wait for the user to hit newline. Since I'm going to be reading the user's input on the standard input, I have to read the file the user wants to view on a different file descriptor. At the end of the loop I'm associating the /etc/passwd file with file descriptor 3, and at the top of the loop I use "read -u 3" to read my input from this file descriptor. Thank you bash, and your amazingly flexible output redirection routines.

Since we have for loops, creating our own version of the head command is also easy:

$ for ((i=0; $i<10; i++)); do read l; echo $l; done </etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
news:x:9:13:news:/etc/news:

If I have the head command, I suppose it's a moral imperative that I also produce something like tail:

$ i=0; while read l; do a[$i]=$l; i=$(( ($i+1)%10 )); done </etc/passwd; \
for ((j=0; $j<10; j++)); do echo ${a[$(( ($j+$i)%10 ))]}; done

webalizer:x:67:67:Webalizer:/var/www/usage:/sbin/nologin
pcap:x:77:77::/var/arpwatch:/sbin/nologin
hsqldb:x:96:96::/var/lib/hsqldb:/sbin/nologin
xfs:x:43:43:X Font Server:/etc/X11/fs:/sbin/nologin
gdm:x:42:42::/var/gdm:/sbin/nologin
sabayon:x:86:86:Sabayon user:/home/sabayon:/sbin/nologin
radiusd:x:95:95:radiusd user:/:/bin/false
mailman:x:41:41:GNU Mailing List Manager:/usr/lib/mailman:/sbin/nologin
tomcat:x:91:91:Tomcat:/usr/share/tomcat5:/bin/sh
avahi-autoipd:x:100:103:avahi-autoipd:/var/lib/avahi-autoipd:/sbin/nologin

In the first loop I'm using an array as a circular buffer to hold the last 10 lines read. After the first loop exhausts the file, I use a second loop to output the lines stored in the array.

The idea of reading the contents of a file into an array suggested this nasty hack to emulate the sort command:

$ n=0; while read l; do a[$n]=$l; ((n++)); done <myfile; \
n=$(($n-1)); \
for ((i=0; $i<$n; i++)); do
s=$i;
for ((j=$((i+1)); $j<=$n; j++)); do
[[ ${a[$s]} < ${a[$j]} ]] || s=$j;
done;
t=${a[$i]}; a[$i]=${a[$s]}; a[$s]=$t;
done; \
for ((i=0; $i<=$n; i++)); do echo ${a[$i]}; done

1
1
10
10
2
3
4
5
6
7
8
9

Yep, the middle, nested loops are actually a selection sort implemented in the shell. You'll notice that the sort here is an alphabetic sort. We could produce a numeric sort using "-lt" instead of "<" inside the "[[ ... ]]" clause in the innermost loop.

You'll also notice that I put some duplicate values in my test input file. Hey, if you're going to do "sort" you've got to also do "uniq". Here's a numeric sort plus some mods to the final loop to emulate uniq:

$ n=0; while read l; do a[$n]=$l; ((n++)); done <myfile; \
n=$(($n-1)); \
for ((i=0; $i<$n; i++)); do
s=$i;
for ((j=$((i+1)); $j<=$n; j++)); do
[[ ${a[$s]} -lt ${a[$j]} ]] || s=$j;
done;
t=${a[$i]}; a[$i]=${a[$s]}; a[$s]=$t;
done; \
for ((i=0; $i<=$n; i++)); do
[[ "X$l" == "X${a[$i]}" ]] || echo ${a[$i]}; l=${a[$i]};
done

1
2
3
4
5
6
7
8
9
10


With the help of the IFS variable, we can do something similar to the cut command:

$ IFS=":"; \
while read uname x uid gid gecos home shell; do
echo $uname $uid;
done </etc/passwd

root 0
bin 1
daemon 2
...

And since bash has a substitution operator, I can even emulate "sed s/.../.../":

$ while read l; do echo ${l//root/toor}; done </etc/passwd
toor:x:0:0:toor:/toor:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
...

I couldn't resist exploiting /proc on my Linux box to generate a simple ps listing:

$ printf "%-10s %5s %5s   %s\n" UID PID PPID CMD; \
for d in /proc/[0-9]*; do
cmd=$(cat $d/cmdline | tr \\000 ' ');
while read label value rest; do
case $label in
Name:) name=$value;;
Pid:) pid=$value;;
PPid:) ppid=$value;;
Uid:) uid=$value;;
esac;
done <$d/status;
[[ -z "$cmd" ]] && cmd=$name;
printf "%-10s %5s %5s %s\n" $uid $pid $ppid "$cmd";
done

UID PID PPID CMD
0 1 0 init [3]
0 10 1 watchdog/2
0 10994 87 kjournald
0 11 1 migration/3
0 11058 1 /usr/lib/vmware/bin/vmware-vmx -...

This is obviously skirting pretty close to our "no scripting" rule, but I actually was able to type this in on the command line. I suspect that there may be information available under /proc that would also enable me to emulate some functionality of other commands like netstat and ifconfig, and possibly even df, but this episode is already getting too long.

Before I finish, however, I wanted to show one more example of how we could create our own simple find command. This one definitely wanders far into scripting territory, since it involves creating a small recursive function to traverse directories:

$ function traverse { 
cd $1;
for i in .[^.]* *; do
$(filetest $i) && echo "$1/$i";
[[ -d $i && -r $i && ! -h $i ]] && (traverse "$1/$i");
done;
}

$ function filetest { [[ -d $1 ]]; }
$ traverse /etc
traverse /etc
/etc/.java
/etc/.java/.systemPrefs
/etc/acpi
/etc/acpi/ac.d
/etc/acpi/battery.d
/etc/acpi/events
/etc/acpi/resume.d
/etc/acpi/start.d
/etc/acpi/suspend.d
/etc/alternatives
...

Specify a directory and the traverse function will walk the entire directory tree, calling the filetest function you define on each object it finds. If the filetest function resolves to true, then traverse will echo the pathname of the object it called filetest on. In the example above, filetest is true if the object is a directory, so our example is similar to "find /etc -type d".