Episode #29: Finding the Time

Hal Says:

When analyzing a file system after an incident, it's often useful to search for recently modified files. Yes, an attacker who's broken root could have reset the timestamps on the files they modified, but I've found that in many cases they don't bother. That means you can do something like this:

# find / -mtime -2 -ls

That will give you a detailed listing of all files that have been modified within the last two days.

This example points up a shortcoming of find, however-- the finest granularity you have for time-based searches is in terms of some number of days. But what if your IDS or firewall logs can tell you exactly when the attack occurred down to the second? Wouldn't it be nice if you could have your find command show you only the files that have been modified since that exact point in time?

Turns out there's a clever little idiom for doing precisely this:

# touch -t 200904291446.53 /tmp/timestamp
# find -newer /tmp/timestamp -ls

The touch command allows the root user to explicitly set the timestamp on a file. The format is YYYYMMDDhhmm.ss (the seconds are optional), so in the example above we're setting the timestamp to 2:46:53pm on Apr 29, 2009. Once you've set the timestamp appropriately, it's a simple matter of using "-newer" with find to locate all files that have been modified more recently than that date.

Ed responds:
I really hate it when Hal pulls something out of his... uh... ear that is trivially easy to do in Linux with bash but is a real pain in the... uh... neck in Windows cmd.exe. But, saddled with this shell, we must make do.

One option we have for getting files' last modification dates and times in Windows cmd.exe is to use the dir command as follows:

C:\> dir /a /tw /o-d

That'll display the contents of the current directory, with all files regardless of their attributes (/a), showing the timestamp (/t) of the last written time (w), ordered (/o) in reverse (-) by date (d). OK... it's a start. The most recently written-to files (i.e., "modified") are at the top of the list.

But, Hal's find command is doing recursion through the Linux file system. What happens when we add a happy little /s to dir to make it recurse through all of c:\?

C:\> dir /a /s /tw /o-d c:\

In the words of Steve Jobs, it's a bag of hurt. It just sucks. It sucks horrifically bad. You see, the contents of each directory are displayed, and those files in each directory are sorted by modified date, as we'd like... but it's done on a directory-by-directory basis, without regard to when each directory was altered. Doh! Adding a /b (for bare form of output) doesn't help us, because it that'll leave off the timestamp information that we crave.

Often in cmd.exe, if we can't get the file information we'd like from the dir command because of all of its baked-in limitations, we can get far closer to what we want with a FOR loop.

So, let's try this again, this time with a FOR loop. We'll use a FOR /R loop, because it recurses through the file system for us, finding files. How about this:

C:\> for /r %i in (*) do @echo %~ti, %~fi

This loop will recurse (/r) through all subdirectories (*) of our current directory (cd into c:\ if you want to go from the top), with an iterator variable of %i taking on the name of each file that we find. In our do clause, we turn off display of commands (@), and echo to standard output %~ti. That reference will display the timestamp associated with %i (that is, our file). As it turns out, the timestamp displayed here is the last modified time. We then display a comma (you'll see why shortly), followed by %~fi, which is a way to reference the full path of %i.

The output here will show modified_time, full_path_to_file, for all files in our given directory and its subdirectories.

(BTW, with the FOR /R, our iterator variable will only take on the value of file names... if you want files and directories, you could use FOR /D /R... yup... both /D and /R together in the same FOR loop).

You might think that we could sort them using the sort command. Unfortunately, Windows cmd.exe built-in sort is an alphanumeric sort, and cannot in anyway understand the dates we want to sort by. Ouch.

So, what can we do? Well, we could dump our output into a csv file thusly:

C:\> for /r %i in (*) do @echo %~ti, %~fi >> file_list.csv

Now you can see why I put that comma in between the timestamp and full file path. Now, we could open up that csv file in a spreadsheet, and sort it however we want. I know that's kinda cheating with respect to our rules of using only built-in features, but we simply lack a good sort program in cmd.exe. Thus, we often have to create csv's and use a spreadsheet.

But, wait... Hal threw us a bone with his follow-up command, which is looking for a file with a very particular date and timestamp during a forensics investigation. We can look for that easily in the output of our FOR /R loop, as follows:

C:\> for /r %i in (*) do @echo %~ti, %~fi | find "04/28/2009 02:06 PM"

Mr. Bucket mentioned to me that the forfiles command is worthy of a mention in this article. My first thought was... "That's a great tool, but it's only in the Resource Kits." I've used it before in investigations, and carry it around with me on a USB token along with a ton of other useful tools. But, Mr. Bucket's bringing it up did make me double check... and, lo and behold, forfiles is built-in to Vista and Windows 2008 Server! That's scrumptious! The best news I've had in the last 30 minutes, at the very least. Sorry, but it's not built into earlier versions of Windows, though, but is a separate download from the Resource Kits.

Using forfiles can really help us out here, and, on Vista and 2008, we've got it built in. Here's how we can satisfy Hal's original hankerin':

C:\> forfiles /p c:\ /s /d -2

This one will display files inside of c:\ (the /p means "path"), recursing subdirectories (/s), displaying those with a last modification date earlier than the current date minus two days. Note that the /d option can be used with a date as well, instead of a number of days, with the date expressed in MM/DD/YYYY format. Instead of a -, you can also use a + in front of the number or date to show all files modified after that given date.

So, we have many options... dir for really simple stuff, FOR /R and FOR /D /R for more complicated stuff, and forfiles if we're on Vista or 2008. It's nice to have options in life. :)

Episode #28: Environment-a-list

Ed starts out:

At a cmd.exe prompt, you can list all environment variables using the "set" command by itself:

C:\> set

Pretty simple. By default, there are a typically a lot of variables set, upwards of two or three dozen.

Another property of set involves the listing of groups of environment variables that start with a substring. For example, to see all of your environment variables that start with the letters pr, you could type:

C:\> set pr

It's kinda weird, but environment variables in Windows cmd.exe are case insensitive.

We can refer to any one of these variables in a command by using the %[var]%. Thus, we can see the value of the username variable by running:

C:\> echo %username%

Or, we can see the value using the set command with:

C:\> set username

Prior to Vista, Windows didn't include a built-in whoami command. The closest thing we have is this "set username". However, be aware that, depending on how your shell is instantiated, you may or may not have this variable set. For example, this variable is typically not set when Metasploit launches a shell on a compromised box. Windows Vista does include a whoami command, but that's a story for another article. We must keep focus when performing command line kung fu.

Other environment variables besides username that are worthwhile include our path variable:

C:\> set path
Path=C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH

The output here will show you both your path, plus the pathext variable (remember, set [string] shows all variables that start with string). The pathext variable is particularly interesting, because it tells us the priority of execution on the machine for various file name suffixes. By default, if I type in "command", but don't specify a suffix, command.com will be run first. If that doesn't exist, the system tries command.exe. Then, it moves onto command.bat and so forth. That behavior can be used for some evil stuff by creating a backdoor called cmd.com or ipconfig.com and inserting it somewhere in the user's path.

Windows, by the way, includes an implicit "." at the very beginning of every user's path. That's a huge bummer from a security perspective, because users can be tricked into running a backdoor out of their current directory when they try to invoke some other command. Also, note that Windows doesn't seem to be too proud that "." is in the path, because it doesn't show it... but, it behaves as though it's there allright.

Other useful environment variables include %systemroot%, which tells you which partition and folder your primary Windows software is located in:

C:\> echo %systemroot%

Often when I'm handling an incident, I see attackers who assume the system root is on C:\. But, sometimes, it's not, and attackers get wrapped around the axel trying to figure out why their tweaks to the OS aren't having an impact. Whenever I'm doing a penetration test and I compromise a Windows box, I quickly display my systermroot and username variables, because that helps to orient me to the target machine.

Also, the prompt environment variable is called, simply enough, prompt. It's default value is $P$G, which indicates your current working directory ($P) followed by a greater than sign ($G). You could really freak yourself out by running:

C:\> set prompt=$$
$ ls

You now have a Linux wanna-be for your shell. I tell ya, when I see that $ prompt, and I just want to type "ls". Alas, no dice. :)

Hal responds:

It's interesting to me how even something as simple as environmental variables points out the differences between the Windows and Unix command-line philosophies. For example, instead of packing all kinds of functionality into a single command like Windows does with "set", Unix handles this by providing a toolkit of different programs that all interact with each other. For example, you use the "env" command to dump all of your current environment variable settings. If you want a specific subset, you pipe the output of "env" into "grep":

$ env | grep PATH
MANPATH=/usr/man:/usr/share/man:/usr/openwin/man:/usr/dt/man:/usr/local/man
CDPATH=.:/home/hal
LD_LIBRARY_PATH=/usr/lib:/usr/openwin/lib:/usr/dt/lib:/usr/local/lib
PATH=/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/sbin:/usr/sbin:...

This seems cumbersome-- you have to use two commands and a pipe instead of using just one "set" command with different parameters. But the power of this idiom is that you use the same knowledge of "grep" to search any kind of textual data in Unix, whereas under Windows you have to learn the particular idiosyncracies of a whole host of separate command line interfaces that each only deal with very specific types of data.

Anyway, returning to Ed's examples, you can display the value of a particular variable using "echo", and we typically set new values using "export" so that the variable setting will be passed along to any subshells we spawn:

$ echo $PATH
PATH=/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/sbin:/usr/sbin:...
$ export PATH=.:$PATH
$ echo $PATH
PATH=.:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/sbin:/usr/sbin:...
$ export PATH=${PATH/\.:/}
$ echo $PATH
PATH=/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/sbin:/usr/sbin:...

Here I'm using the current value of $PATH in the "export" expression so that I can prepend "." (the current working directory) at the front of my search path. Obviously, as Ed points out above, this is a terrible idea from a security perspective, so I then use the same variable substitution operator we used back in Episode 26 to remove the leading dot. Again, I'm leveraging prior knowledge of basic shell building blocks to solve a new problem.

As Ed points out, Unix does have the "whoami" command, plus a number of other mechanisms for figuring out what user you are that provide different types of information:

$ whoami
hal
$ who am i
hal      pts/2        2009-04-26 14:44 (:0.0)
$ id
uid=1000(hal) gid=1000(hal) groups=4(adm),20(dialout),24(cdrom),...

Also, the your login program will generally set the $LOGNAME and $HOME environment variables, and your shell may also set the $USER and/or $USERNAME variables as well.

You can change your prompt by setting the $PS1 environment variable. For example, the default on my Ubunty machine is:

$ export PS1='\u@\h:\w\$ '
hal@elk:~$

There are a whole host of different escape sequences you can use here: "\u" is your user ID, "\h" is the unqualified host name, and "\w" is the current working directory. So if we wanted to emulate the standard Windows command prompt in our Unix shell we could use:

hal@elk:~$ export PS1='\w> '
~> cd /tmp
/tmp>

Of course, it doesn't look exactly like the Windows prompt, because when you're in your home directory, the directory is listed as "~", not $HOME. However, you can use the magic $PWD variable in your prompt instead of "\w" to get a more Windows-like experience:

/tmp> export PS1='$PWD> '
/tmp> cd
/home/hal> 

The emulation isn't perfect because Unix uses forward slashes in directory paths, not backslashes. I present the following for the truly psychotic:

/home/hal> export PS1='C:${PWD//\//\\\\}> '
C:\home\hal> cd /tmp
C:\tmp> 

You think having a "$" prompt will confuse the Windows folks? Try putting the above recipe into your /etc/profile file next April Fool's Day and watch your Unix users freak out.

By the way, you'll notice that the variable is $PS1. There's also $PS2:

C:\home\hal> export PS1='$ '
$ export PS2='more> '
$ for i in *
more> do
more> ...

Believe it or not, there are even $PS3 (the prompt used by the "select" command) and $PS4 (usually set to "+", this is the value printed before each command in an execution trace as with "set -x").

Paul Throws His Hat In:

One of the environment variables that I've encountered in the past that has really helped me is "LD_LIBRARY_PATH". The variable stores the path that will be used by programs to find software libraries. For example, if you've got a bunch of custom libraries in /opt/local/lib you could run the following command:

$ export LD_LIBRARY_PATH="/opt/local/lib:/usr/local/lib:/usr/lib:/lib"

This comes in handy when you are using OS X and something such as MacPorts is installed in /opt and has the libraries you are looking for. Yes, these are in fact the libraries you are looking for...

Episode #27: You'll Be History

Ed starts with:

In Episode #14, I touched briefly on how a cmd.exe user could hit the F7 key to view shell history, and we received a lot of positive feedback on that from folks. But, there's some more aspects of cmd.exe history that I use all the time that I think are worth bringing up here.

In particular, if you want to get access to your shell history at the command line itself so you can dump it to a file or manipulate it (rather than just displaying it on the screen so you can scroll through it with F7), you could run the following command:

C:\> doskey /history

Dude! The 80's just called, and they want their doskey command back. :)

Seriously, though, it remains a very useful command. Running doskey this way will show all of the commands typed into the current cmd.exe session. Unfortunately, you won't see history for other cmd.exe's on the screen or those invoked in the past. Still, when handling an incident, running this command with the output redirected to a file can be very useful evidence. If the bad guy was kind enough to leave a cmd.exe on the console screen of the machine he controlled the GUI on, you can quickly dump the history to an evidence file. Alternatively, if you, the incident handler or forensics analyst, are typing commands at a cmd.exe and want a record of what you've typed, invoking:

C:\> doskey /history > \\evidence_server\share\mycommands.txt

...can be very helpful to shoot your results to an evidence server somewhere.

Oh, and note that cmd.exe "telescopes" commands down, so that if you run the same command multiple times in a row, the history will only show the command once. For example:

C:\> dir > nul

C:\> ipconfig > nul

C:\> ipconfig > nul

C:\> ipconfig > nul

C:\> hostname > nul

C:\> doskey /history
dir
ipconfig
hostname
doskey /history

As we can see, the ipconfig command only appears in history once, even though we ran it three times in a row, due to this telescoping down effect.

By default, the command history stores 50 commands, but is configurable on a user-by-user basis. To change it at the command line for the current user, you could run:

C:\> reg add hkcu\console\^%systemroot^%_system32_cmd.exe /v HistoryBufferSize
        /t reg_dword /d 0x400

That last number there is a hexadecimal value for the number of commands you want it to remember. 0x400 will make it remember the last 1024 commands. This new history depth will only take effect when you launch new cmd.exe's going forward. But, you don't have to logoff or reboot for it to take effect. Just launch another cmd.exe, and you should have your new history depth (which can be viewed in the GUI by right clicking the title bar of a cmd.exe window, selecting "Properties", and going to the Options tab to view Command History Buffer Size).

Oh, and another thing about cmd.exe history... I often see bashketeers get a little freaked out by how cmd.exe handles shell history when you scroll through it with the up-arrow and down-arrow keys. As we all know, you can hit the up-arrow key in cmd.exe to scroll back to a previous command in modern Windows machines. When you find the command you want, you can hit Enter to re-run that command. No problem there. The issue that puzzles some people is what happens after you've run that command from your history, and you then hit the up arrow key again. Let me illustrate it with this screen shot.

Here, I've just hit 1 followed by Enter, to seed my history with some bogus commands so we can track things easily. Because there is no program called 1.exe, my shell tells me so. I do this for 2, 3, 4, and 5 as well.

Then, I hit the up arrow key to scroll back to my 3 command. I hit Enter on that. If I hit the up arrow again, of course I get the 3 again.

Now for the interesting part. What happens when I hit the up arrow again? If you were a bash guy, you might think that the system would display the number 5, because before you did 3, you did 5. And, you'd be wrong in a cmd.exe shell, which would display the number 2. What? Why 2?

Well, cmd.exe maintains a shell history pointer, and when you scroll back into your history and then run a command from it, this pointer jumps back to your earlier command. All up or down arrow movements from that point are relative to that older command, not your most recent commands.

This used to flummox me, with my ancient Unix background. But, you know what? I'll share a dirty little secret with you if you don't tell anyone. I actually have grown to like (and... gulp... prefer) this behavior. You see, there is a certain locality of command execution from history, and, if I execute a command that was 20 commands ago, there is a reasonably good probability that I'll want to run another command from 19 commands ago after it.

So, cmd.exe history scrolling isn't broken... it's just different. And, it's different for a reason. You might not agree with the reason, but at least there is one. So many things in Windows are different for no reason whatsoever, you know... but don't get me started about tracert. ;)

Hal Responds:

I was really happy when Ed brought us back around to command-line history again, because there was a bunch of stuff that I wanted to put into Episode #14, but decided not to in the interests of space. But there's no stopping me now...

While "doskey /history" is an obvious and intuitive way to access your command history, those wacky Unix folks have decided to force you to use the obscure "history" command instead (that was sarcasm for those of you who missed it because you were trying to remember the silly cmd.exe syntax):

# history
[...]
 1001  /etc/init.d/httpd restart
 1002  vi /etc/httpd/conf/httpd.conf
 1003  vi /etc/httpd/conf.d/ssl.conf
 1004  /etc/init.d/httpd restart
 1005  ps -ef | grep http
 1006  ps -ef | grep http
 1007  history

Oh, and notice that the bash history saves all of the commands you've typed, even the repeats. I think that's more useful-- especially from a forensic perspective-- than removing repeated commands, but I'm sure lord high doskey knows better than me.

See the numbers next to the commands in the history output above? You can use "!<n>" to execute the previous command with number <n>. For example, "!1001" would execute "/etc/init.d.http restart" again (as would "!1004"). I often use this when I'm debugging a problem with a server process like Apache-- try to start the server and fail, enter a bunch of commands to fix the problem, "!<whatever>" to try and start the server, lather, rinse, repeat. In other words, "!<n>" is useful when you are constantly re-running the same command with an arbitrary number of other commands in between attempts.

By default, the command shell keeps your last 500 commands, but you can change this by setting the HISTSIZE variable in your .bashrc or .profile:

export HISTSIZE=1024

Wow, you can even set it in decimal! Crazy! What will those wacky Unix folks think of next?

Episode #26: Renaming Files With Regular Expressions

Hal Says:

I admit it, I'm a fan of the CommandLineFu site (hey, it's a community, not a competition), and like trolling through it occasionally for interesting ideas. This post by vgagliardi shows a cool trick for renaming a bunch of files using regular expressions and the substitution operator in bash. For example, suppose I wanted to convert spaces to underscores in all file names in the directory:

$ for f in *; do mv -- "$f" "${f// /_}"; done

I realize that syntax looks a little crazy. The general form is "${variable/pattern/substitution}", but in this case we have an extra "/" at the front of "pattern", which means "replace all instances" rather than only replacing the first instance.

By the way, you can use the standard Unix regular expression syntax for your substitution pattern. For example, here's a loop to remove all characters from file names except for alphanumeric characters, dot, hypen, and underscore:

$ for f in *; do mv -- "$f" "${f//[^-_.A-Za-z0-9]/}"; done

In this case "[^...]" means match any characters not in the specified set, and we're performing a null substitution.

Did you notice that we're also using the "--" argument to the "mv" command, just in case one of our file names happens to start with a "-"? These files are typically a huge pain in Unix. What if we wanted to replace all the "-" characters at the beginning of a file name with underscores?

$ for f in *; do mv -- "$f" "${f/#-/_}"; done

As you can see, starting the pattern with "#" means "match at the front of the string. Or we can match at the end of the string with "%":

$ find docroot -type f -name \*.htm | \
     while read f; do mv -- "$f" "${f/%.htm/.html}"; done

Here we're using "find" to locate all of the *.htm files in our web docroot and then piping the output into a while loop that renames all these files to be *.html files instead.

There are a couple of problems with the method that I'm using here: (1) if multiple files map to the same name, you'll end up clobbering all but the last instance of that file, and (2) you get errors if your substitution doesn't actually modify the file name because the "mv" command refuses to rename a file to itself. We can fix both of these problems with a little extra logic in the loop. Let's return to our first example of converting spaces to underscores:

$ for f in *; do n="${f// /_}"; [ -f "$n" ] || mv -- "$f" "$n"; done

First we assign the new file name to the variable $n. Then we check to see if a file named "$n" exists-- the "mv" command after the "||" is only executed if there is no "$n" file.

I admit that I usually use the Perl rename program for renaming large numbers of files, because (a) the syntax is much more terse, and (b) I love Perl. But this program isn't always available on all the different flavors of Unix that I end up having to work on. So having this functionality built into the shell is a huge win.

Ed Responds:
When I quickly glanced at Hal's challenge initially, I thought... "Yeah, that's pretty easy... findstr supports regex, and I'll use the ren command to rename the files... No prob."

And then, I started to write the command, and it got horribly ugly really quickly. Hal squealed with delight when I told him how ugly it was... and believe me... you ain't seen nothing until you've seen Hal squeal with delight.

Anyway, to keep this article from getting unreasonably long, I'm going to address Hal's original command, which replaced the spaces in file names with underscores. Unfortunately, you see, the parsing, iteration, and recursion capabilities within a single command in cmd.exe are really limiting. For parsing strings, we've got FOR /F and a handful of substring operations I covered in Episode 12. For running a command multiple times, we've got for /L, as I mentioned in Episode 3. For recursion, well, that's just plain bad news in a single command unless we bend our rules to create a bat file that calls itself.

To start to address Hal's original challenge, we can use the following command to determine if there are any files that have at least one space in their names in our current directory:

C:\> dir /b "* *"

That's pretty straightforward to start, with the /b making dir show only the bare form of output, omitting cruft about volume names and sizes. Note that it will only show files that do not have the hidden attribute set. If you want, you can invoke dir with /a to make it show files regardless of their attributes, hidden or otherwise. Now, let's see what we can do with this building block.

Plan A: Every File Should Have Four Spaces in Its Name, Right?
My original plan was to wrap that command inside a FOR /F loop, iterating over each file using FOR /F functionality to parse it into its constituent elements. I was thinking something like this:

C:\> for /F "tokens=1-4" %i in ('dir /b "* *"') do ren "%i %j %k %l" "%i_%j_%k_%l"

Well, that's all very nice, but we've got a problem... let me show an example of what this beast creates when I run it in a directory with a file named "file 1.txt" and "file 2 is here.txt":

C:\> dir /b
file_1.txt__
file_2_is_here.txt

Ooops... the file1.txt name has two underscores after it. This option only works if files have exactly four spaces in their names. That's no good.

Plan B: Let's Just Change the First Space into an Underscore
Well, how about this... We could write a one-liner that assumes a file will have only one space in its name, and convert that one space into an underscore. That's not too bad:

C:\> for /f "tokens=1,*" %i in ('dir /b "* *"') do ren "%i %j" "%i_%j"

I'm parsing the output of the dir /b command using parsing logic of "tokens=1,*", which means use your default delimiters of space and break each line of the output of the dir command into the entity before the first space into %i, and everything afterward into the next iterator variable, %j.

Let's run that with our same file names as before, yielding:

C:\> dir /b
file_1.txt
file_2 is here.txt

Well, we got it right for file_1.txt, because there is only one space. But, we only fixed the first space in file 2 is here.txt. Hmmmm... How could we move closer to our result?

Hit the up arrow a couple times to go back to our Plan B FOR /F loop, and hit enter again. Now, running our dir, we get:

C:\> dir /b
file_1.txt
file_2_is here.txt

Ahh... we nailed our second space. Hit the up arrow again and re-run... and... well, you get the picture. We can take care of one space at a time. Not so bad.

But, who wants to hit the up arrow again and again and again until we get rid of all the spaces? You'd have to re-run my Plan B command N times, where N is the maximum number of spaces inside a file name in the current directory.

Plan C: Make the Shell Re-Run the Command Instead of Doing it Manually
Well, instead of re-running a command a bunch of times, let's make the shell do our work for us. We'll just wrap the whole thing in a FOR /L loop to count through integers 1 through 10 (1,1,10) and invoke the FOR /F loop at each iteration through our FOR /L loop:

C:\> for /L %a in (1,1,10) do @for /f "tokens=1,*" %i in ('dir /b "* *"')
  do ren "%i %j" "%i_%j"

That works, provided that none of the files have more than ten spaces in their name. Ummm... but what if they do? We could raise the number 10 to 20... but that's kind of a cheap hack, no?

Plan D: Violate the Rules -- Make a 3-Line Script
OK... if we had a while construct in cmd.exe, we could simply run my FOR /F loop of Plan B while the dir /b "* *" still returned valid output. But, we don't have a while command in cmd.exe. If we want to check a condition like that, we only have IF statements. And, if we want to jump around based on the results of IF statements, we need to use GOTOs. And, if we want to use IFs and GOTOs, we can't dump everything on a single one-line command, but will instead have to create a little bat file.

So, I'm going to have to bend our ground rules for this blog, which require a single command, and instead use a three-line bat file. Here's a bat file I wrote that converts all of the names of files in the current directory with spaces in them into underscores:

:begin
for /F "tokens=1,*" %%i in ('dir /b "* *"') do ren "%%i %%j" "%%i_%%j"
if exist "* *" goto begin

There you have it.... kind of an ugly little hack, but it works. Note that I had to change my iterator variables in my FOR loop from %i and %j into %%i and %%j. You have to do that to convert command-lines into bat files in Windows. Also, I'm using an IF statement to test for the existence of "* *", which would match any file with a space in its name.

A small script in cmd.exe can satisfy Hal's original challenge. To start addressing his other feats to convert other characters in file names, we could specify options for the FOR /F loop of everything we want to parse out with the syntax "tokens=1,* delims=~!@#$%^&*()+=" and whatever else you wanna take out.

I could drone on and on endlessly here, but I think you get the idea. It ain't pretty, but it is doable.... Now that should be the cmd.exe mantra.

PS: I too am a fan of the CommandLineFu site. It rocks.

Episode #25: My Shell Does Math

Ed says:

On Linux and Unix, I frequently rely on bc to do math at the command line. But, bc isn't built-in to Windows. A lot of people don't realize that the cmd.exe shell has some built-in capabilities for doing math... really lame math. But still, if you are caught in a pinch, don't wanna invoke the calc.exe GUI, and want to focus on integer math, cmd.exe can do your bidding. It's all invoked with the "set /a" command, as in:

C:\> set /a 2+2
4

Sexy, huh? Well, keep in mind that it is integer math only, as illustrated by:

C:\> set /a 3/2
1

That leads to some weirdness, as in:

C:\> set /a 3/2*2
2

Also, we're working only with signed 32-bit numbers here, so make sure your results stay under approximately two billion. Bigger than that will either push you into negative territory, or outright give you an error message:

C:\>set /a 1000000000*2
2000000000

c:\test>set /a 2000000000*2
-294967296

c:\test>set /a 3000000000*2
Invalid number.  Numbers are limited to 32-bits of precision.

We've also got some options for using hex (specified with 0x as a prefix):

C:\> set /a 0xA + 0x2
12

Or, you can do octal, by prefacing your numbers with a zero:

C:\> set /a 010 + 01
9

You can even mix and match:

C:\> set /a 0xA + 01 + 1
12

It gets exciting to think that you can do hex or octal math at the cmd.exe command line, until you realize that all of your outputs are in, well, decimal. What genius thought up adding hex and octal as input, without providing options for hex and octal in the output?

Also, we've got bitwise AND with &, bitwise OR with |, and XOR with ^. But, again, the output is in... ugh... decimal.

Who uses decimal anymore? I mean, when you are driving, those speed limit signs are all in hex, right? Cruising around at 0x65 mph or 0x100 km/hr can be very exciting. When pulled over, tell the officer that you thought the speed limit sign was hexadecimal. Of course, you'll have to say that you assumed your speedometer is in decimal... kinda like set /a. Inputs in hex, outputs in decimal. See if that'll get you out of a ticket. Good luck.

Anyway, for really simple integer math of smallish numbers, set /a can come in handy. I do sometimes use it if I've gotta do a quick integer calculation and I don't wanna take focus off of my cmd.exe on the screen.

Honestly, given the quite limited abilities for cmd.exe to do math this way, I'm kind of expecting my command line kung fu sparring partners working in bash to knock me out with this one. But, in leading with my jaw, I'm hoping in the process that we'll all learn some completely magical fu from them when doing math in bash. So, lay it on us, guys. What fu have you?

Mr. Bucket read the draft of this post, and kicked in some interesting fu from the dawn of time for doing some hex match using built-in Windows command line tools. I remembered this one, but barely. It was buried in the dark recesses of my mind... I haven't used this since I bought a really good hex calculator a long long time ago.

Mr. Bucket suggested running the built-in Windows debugger tool:

C:\> debug

Then, at the goofy little "-" prompt, you could type:

- H 4a44 90

The debugger will then print out the sum and difference of the two numbers you provided:

4AD4  49B4

So there.... next time you are stranded on a desert island with only a Windows box and need to do hex addition or subtraction to hack your way off, you'll be all set. Thanks for the interesting point, Mr. Bucket!

Paul Says:

On UNIX/Linux systems I've always found that the bc command was available to me if I need to perform simple math (or even that "new" math people talk about):

$ bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
3/2*2
2
quit

It can even do fancy things like square root:

$ bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
sqrt(3.14)
1.77
quit

Of course Bash itself can do math as well. This is handy in shell scripts. Below is a command line "one-liner" example:

$ let RESULT=2+2 ; echo $RESULT
4

As long as "2+2" continues to equal 4 the universe stays in balance and life is good :) Of course, if you leave out the "let" directive you get very different results:

$ RESULT=2+2 ; echo $RESULT
2+2

Here "2+2" is being treated as a string, and therefore no math will be performed. So remember, when using Bash always "Let there be math".

Hal Says:

Ummmm, Paul? How about just:

$ echo $((2+2))
4

Using "$((...))" to put arithmetic expressions in-line is a heck of a lot easier than that tedious "let ..." nonsense.

By the way, for those of you that aren't using bash or a more modern shell with built-in arithmetic, there's also the really old-school expr command in addition to bc:

$ expr 2 + 2
4

Episode #24: Copying and Synchronizing (Remote) Directories

Hal Says:

Copying files and directories around is a pretty common task, and there are all sorts of ways to accomplish this in Unix. But I have a strong preference for using the rsync command because (a) it allows me to copy files either within a single machine or between systems over the network, (b) it can keep directories in sync, preserve timestamps, ownerships, etc, and (c) it's smart enough to only copy files that have changed, which is a big savings on time and bandwidth.

Basic rsync usage is straightforward:

$ rsync -aH dir1/ dir2/                              #copy dir1 to dir2 on same machine
$ rsync -aH remotehost:/path/to/dir/ localdir/       #copy remote dir to local machine
$ rsync -aH dir/ remotehost:/path/to/dir/            #copy local dir to remote machine
$ rsync -aH dir/ someuser@remotehost:/path/to/dir/   #copy dir to remote host as someuser

The "-a" option combines the most common rsync options for copying and preserving directories: "-r" for recursive, "-ptog" to preserve permissions/timestamps/owner/group owner, "-l" to copy symlinks as symlinks, and "-D" to preserve device and special files. The only useful option not included in "-a" is the "-H" option to preserve hard links. Even though "-H" adds extra processing time, I tend to always use it, just to be on the safe side.

By the way, the trailing "/" characters on the source directory paths are significant. I could explain it in words, but it's easier to just show you an example:

$ ls source
bar.c  baz.c  foo.c
$ rsync -aH source dir1
$ ls dir1
source
$ rsync -aH source/ dir2
$ ls dir2
bar.c  baz.c  foo.c

Without the trailing "/", rsync copies the entire directory, including the top directory itself. With the trailing "/", the contents of the source directory are put into the top of the destination directory.

If you want to keep two directories in sync, you probably want to include the "--delete" option, which deletes files in the target directory that aren't present in the source:

$ rsync -aH source/ target
$ rm source/foo.c
$ rsync -aH source/ target
$ ls target
bar.c  baz.c  foo.c
$ rsync -aH --delete source/ target
$ ls target
bar.c  baz.c

Sometimes there are certain files that you don't want to copy:

$ ls source
bar.c  bar.o  baz.c  baz.o  foo.c  foo.o
$ rsync -aH --delete --exclude=\*.o source/ target
$ ls target
bar.c  baz.c  foo.c

Here I'm using "--exclude" to not copy the object files, just the source files. You can put multiple "--exclude" options on a single command line, but after a while it gets annoying to keep typing the same set of excludes over and over again. So there's also an "--exclude-from=<file>" option to read in a list of exclude patterns from the file "<file>".

rsync also has a "-n" option like the "make" command, which shows you what would happen without actually copying any files. You usually want to combine "-n" with "-v" (verbose) so you can actually see what rsync would be doing:

$ rsync -aHnv --delete --exclude=\*.o source/ newdir
sending incremental file list
created directory newdir
./
bar.c
baz.c
foo.c

sent 90 bytes  received 24 bytes  228.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

Ed Joyfully Responds:
You thought you had me with your fancy shmancy rsync, didn't ya, Hal? Well... ha! A couple of years ago, you'd have been right, because there was no good built-in Windows tool for synchronizing directories. The xcopy command is a decent file copy tool, but it doesn't really police file updates to maintain synchronicity (and yes, that oblique reference to "The Police" was intentional). A couple of years ago, you'd have to install a separate tool for synchronizing. A really nice tool for doing that is robocopy, available in various resource kits, such as the Windows 2003 kit here.

But, here's the really good news: robocopy is built-in to Vista and Windows 2008 server! It's a really nice synchronization tool, and its name is far cooler than the rather pedestrian "rsync". "Robocopy" sounds like a cool mechanized buddy, or perhaps even a robotized superhero law enforcement officer.

So, how can we use robocopy to achieve what Hal does with rsync above? Well, to mirror from one local directory to another local directory, you could run:

C:\> robocopy dir1 dir2 /s

The /s makes robocopy recurse subdirectories. You could put the /s up front, but, generally speaking, it's best to put the source and destination directories first with robocopy, especially when you start to define file and directory exclusions. Note that robocopy cannot copy hard links, so we lose them (i.e., there is no rsync -H equivalent). Note also that robocopy works the same way whether you specifiy dir1 or dir1/, unlike rsync. That's ok with me even though it is slightly less flexible, as there is less of a chance that I'll screw something up.

To use robocopy to replicate something to or from a remote directory, just refer to the directory as \\[machine]\[share]\[dir], as you might expect, as in:

C:\> robocopy plans_for_world_domination \\backup\rainbowsANDunicorns /s /z

Another nice feature associated with using robocopy across a network involves what happens when network connectivity is lost. If you invoke it the right way, robocopy maintains status so that it can pick up where it left off when doing a copy. When you invoke robocopy, use the /z option to run it in restartable mode. The /z makes it maintain the status information necessary to restart a copy that is interrupted.

If you want to keep directories in sync (removing files from the destination that have been deleted from the source), you can use the /MIR option (/MIR actually means /E plus /PURGE). As with rsync, robocopy will copy all files by default. To omit certain files, we have a huge number of exclusion options, such as /XF to exclude files and /XD to exclude directories, both of which support wildcards with *.

Thus, to mimic Hal's fu above, we could run:

C:\> robocopy source target /S /MIR /XF *.o 

Oh, and to do a dry run, just printing out information about what would be copied, instead of actually doing the copies, we can invoke robocopy with the /L option, as in:

C:\> robocopy source target /L /S /MIR /XF *.o 

If you'd like to copy all file attributes, ownership information, and file permissions, invoke robocopy with /COPYALL.

The output of robocopy is quite nice as well, showing detailed statistics about what was copied, what was skipped (because it was already there, or was excluded), detailed times, and a timestamp of invocation and completion:

c:\> robocopy source target /L /S /MIR /XF *.o

-------------------------------------------------------------------------------
ROBOCOPY     ::     Robust File Copy for Windows

-------------------------------------------------------------------------------

Started : Sun Apr 12 07:54:51 2009

Source : c:\test\source\
Dest : c:\test\target\

Files : *.*

Exc Files : *.o

Options : *.* /L /S /E /COPY:DAT /PURGE /MIR /R:1000000 /W:30

------------------------------------------------------------------------------

                     4    c:\test\source\
      New File                   4        bar.c
      New File                   4        baz.c
      New File                   4        foo.c
                     1    c:\test\source\hello\

------------------------------------------------------------------------------

          Total    Copied   Skipped  Mismatch    FAILED    Extras
Dirs :         2         0         2         0         0         0
Files :         5         3         2         0         0         0
Bytes :        24        12        12         0         0         0
Times :   0:00:00   0:00:00                       0:00:00   0:00:00

Ended : Sun Apr 12 07:54:51 2009

Yeah, robocopy!

Episode #23: Job Control

Hal Says:

Paul's last post left me hankering to talk a little bit more about job control in the Unix shell. Besides, it'll make Ed feel really bad about the Windows command shell, so what's not to like?

The simplest form of job control is to hit "^Z" to suspend a currently running process. Once the process is suspended, you get your command prompt back and can issue other commands. You can also type "bg" to force the suspended process into the background, where it will continue to run:

# tail -f /var/log/httpd/error_log
[... output not shown ...]
^Z
[1]+  Stopped                 tail -f /var/log/httpd/error_log
# bg
[1]+ tail -f /var/log/httpd/error_log &

Now you can fiddle with your Apache config file, restart the web server, etc. The "tail -f" process is still running in the background and displaying errors as they appear in the log file. I find this very useful if I'm working remotely to resolve problems on my web server. Actually, I could have simply started that "tail" command in the background by putting a "&" at the end of the command line:

# tail -f /var/log/httpd/error_log &
[1] 14050

I often use job control to flip in and out of a superuser shell:

$ /bin/su
Password:
# [... do some stuff as root ...]
# suspend

[1]+  Stopped                 /bin/su
$ [... do some stuff as a normal user ...]
$ fg
/bin/su
# 

Notice that you use "fg" to start a suspended process running again. This sure beats having to keep using "su" and re-enter the root password all the time. Of course, you probably should be using "sudo", but that's a topic for another day.

By the way, you can also use job control to blip in and out of remote SSH sessions too:

[hal@localhost ~]$ ssh remotehost
hal@remotehost's password:
Last login: Sun Apr  5 10:06:26 2009 from ...
[hal@remotehost ~]$
[hal@remotehost ~]$ ~^Z [suspend ssh]

[1]+  Stopped                 ssh remotehost
[hal@localhost ~]$ fg
ssh remotehost

[hal@remotehost ~]$ 

Note that the key sequence to suspend an SSH session is "\n~^Z"-- you have to enter the tilde character at the beginning of a line, not while you're in the middle of a command-line. By the way, if you're logged into multiple machines in series, you can use multiple tildes to pick which of the chained SSH sessions you actually want to suspend.

Sometimes you might have multiple jobs suspended at the same time. You can use the "jobs" command to display them, and use the job number displayed in the output to interact with the different jobs:

# jobs
[1]   Stopped                 vi /etc/httpd/conf/httpd.conf
[2]-  Stopped                 vi /etc/httpd/conf.d/ssl.conf
[3]+  Stopped                 tail -f /var/log/httpd/error_log
# bg %3
[3]+ tail -f /var/log/httpd/error_log &
# fg %1
vi /etc/httpd/conf/httpd.conf
^Z
[1]+  Stopped                 vi /etc/httpd/conf/httpd.conf
# %2
vi /etc/httpd/conf.d/ssl.conf
^Z
[2]+  Stopped                 vi /etc/httpd/conf.d/ssl.conf
# fg
vi /etc/httpd/conf.d/ssl.conf

In general you do "[fg|bg] %n" to foreground or background job number "n". However, you can leave off the "fg" if you want because foregrounding a job is the default. If you don't specify a job number after the "fg" or "bg", then the most recently manipulated job number is assumed-- you can see this job with a "+" next to it in the output of the "jobs" command.

Ed Wimpers:

Job control?!? What are you, Hal... just plain cruel? I long for true job control in my cmd.exe. If I want true job control, I just install Cygwin.

But, there are some job control-like things I can do in a pinch on a box without Cygwin.

For example, if I want to start a process in a separate cmd.exe window, I can use the start command. Here, I'll use it to start a separate shell window to search for wmic.exe on the C:\ partition, using the file search capabilities we described in Episode #21:

C:\> start dir /s /b c:\wmic.exe

That pops up a separate Window that will display my results. The window will linger displaying my results after it's done. If I want to start it up minimized, I can run:

C:\> start /MIN dir /s /b c:\wmic.exe

If I want it to start a process in a separate window and then have it disappear when it is done running, I use:

C:\> start cmd.exe /c dir /s /b c:\wmic.exe

Sadly, there's no easy way to get access to standard out of these separately invoked shells windows while they are running, other than to have them dump their output into a file, with "> filename" at the command line invocation.

But now for the tricky part... how can you mimic the & feature of most Linux shells to run something in the background of the current cmd.exe while still dumping their standard output into the current shell's display? We can use the /b option of start to put something in the background, as in:

C:\> start /b dir /s /b c:\wmic.exe

So, we've got &-like behavior! However, it's not really full job control, because I can't pull that job into the foreground, or select one job from a group of backgrounded tasks. But, I can kill my background task, by hitting CTRL-break in the cmd.exe window that spawned it. Also, each time you run "start /b", it will leave an extra cmd.exe process running until you either exit that process (by typing... you guessed it... "exit") or killing it.

To kill "jobs" (if I can call them that) more thoroughly, I usually rely on wmic or taskkill, as defined in Episode #22.

So, there you have it... not quite job control, but an ability to kick things into the background so you can keep your current shell active and get more work done. Oh, and install Cygwin. You'll be glad you did. :)

Paul Says:

I will never forget when I was first learning Linux/UNIX (I actually got my feet wet with Linux/UNIX by using Linux at home, and AIX at work). I was configuring a proxy server, carefully constructing each command line switch and running the command. I'd hit ^Z to put the command in the background and wonder why my proxy server wasn't listening. I had forgotten to put the & after the command, so it truly did "suspend" the command. I never made that mistake again :)

I would also like to quickly highlight the nohup command. I often use this when I run a command that will take a long time and want to go back and check its output:

$ nohup nmap -sS -T4 -iL edswindowsnetwork -oA myscanresults &

Sometimes processes you start while logged into a remote host will SIGHUP when you logoff, and nohup can prevent that.