Episode #73: Getting the perfect Perm(s)

Tim unwraps:

One of the things I find myself doing on a regular basis is creating a new directory structure and setting the permissions. The permissions are different for each folder and are based on who in the organization needs access to it. We could just write a script to create the directories and the permissions, but let's say we want to copy permissions from one directory structure to another. For this example let's assume we have a project folder structure that looks something this.

Prjs
+-Project1 (Managers - Full Access, Consultants - Full Access)
  |-Budget (Consultants - Deny, Finance - Full Access)
  |-Data
   +-Docs
      |-ForRelease (AdminStaff - Full Access)
      +-InProgress

Included above is the appropriate permissions on each folder. All permissions are inherited, so consultants and managers would have access to the Data directory.

We can verify these permissions by using Get-ChildItem (aliases gci, dir, ls) and piping the results into Get-Acl.

PS C:\> ls Prjs -recurse | Get-Acl | fl Path,AccessToString

Path           : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1
AccessToString : WINXP\Consultants Allow  FullControl
               WINXP\Managers Allow  FullControl

Path           : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Budget
AccessToString : WINXP\Consultants Deny  DeleteSubdirectoriesAndFiles, Modify,
               ChangePermissions, TakeOwnership
               WINXP\Consultants Allow  FullControl
               WINXP\Finance Allow  FullControl
               WINXP\Managers Allow  FullControl

Path           : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Data
AccessToString : WINXP\Consultants Allow  FullControl
               WINXP\Managers Allow  FullControl

Path           : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Docs
AccessToString : WINXP\Consultants Allow  FullControl
               WINXP\Managers Allow  FullControl

Path           : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Docs\ForRelease
AccessToString : WINXP\AdminStaff Allow  FullControl
               WINXP\Consultants Allow  FullControl
               WINXP\Managers Allow  FullControl

Path           : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project1\Docs\Working
AccessToString : WINXP\Consultants Allow  FullControl
               WINXP\Managers Allow  FullControl

So now we want to create a second project, Project2, and we want to make sure we have the same permissions. We could copy just the directories without files, but there may be more subdirectories further down that we don't want. So let's create the folders.

PS C:\> mkdir Prjs\Project2\Budget
PS C:\> mkdir Prjs\Project2\Data
PS C:\> mkdir Prjs\Project2\Docs\ForRelease
PS C:\> mkdir Prjs\Project2\Docs\Working

Note, when the Budget directory is created it also creates the Project2 directory since it doesn't exist.

What are the permissions on the new folder?

PS C:\Prjs> Get-Acl Project2 | fl Path,AccessToString

Path           : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project2
AccessToString : BUILTIN\Administrators Allow  FullControl
               NT AUTHORITY\SYSTEM Allow  FullControl
               WINXP\myuser Allow  FullControl
               CREATOR OWNER Allow  268435456
               BUILTIN\Users Allow  ReadAndExecute, Synchronize
               BUILTIN\Users Allow  AppendData
               BUILTIN\Users Allow  CreateFiles

Those are not the permissions we want. The permissions need to be copied from Project1 to Project2, but how? The Get-Acl and Set-Acl commands will do it.

PS C:\Prjs> Get-Acl Project1 | Set-Acl Project2

Let's verify.

PS C:\Prjs> Get-Acl Project2 | fl Path,AccessToString

Path           : Microsoft.PowerShell.Core\FileSystem::C:\Prjs\Project2
AccessToString : BUILTIN\Administrators Allow  FullControl
               WINXP\Managers Allow  FullControl
               WINXP\Consultants Allow  FullControl

Looks good. Now the subfolder permissions need to be copied as well.

PS C:\Prjs> ls Project1 -Recurse | Get-Acl |% {
Set-Acl $_ -Path ($_.Path -replace "Project1","Project2") }

First we do a recursive directory listing and get the Acl on each folder. We then take that Acl and apply it to a different folder. In our case all we need to do is replace Project1 for Project2 in the Path. Let's verify that the permissions match.

PS C:\Prjs> Compare-Object (ls Project1 -Recurse | Get-Acl)
(ls Project2 -Recurse | Get-Acl) -Property PSChildName, Access

No output, that's good, it means the permissions are identical. How did that work?

The Compare-Object cmdlet is used to find the differences between the collection of objects returned by these two commands:

ls Project1 -Recurse | Get-Acl
ls Project2 -Recurse | Get-Acl

The Property parameter specified in the original command allows us to select the properties to be checked for differences. PSChildName is the directory name and the Access property contains the permissions on the folder. We can't substitute the Path property for PSChildName since Path is the full path and it would always be different.

Copying permissions is pretty easy, I imagine it will be pretty easy for Hal since it isn't as granular. Finally, a bit of a leg up on Hal.

Hal just copies everything:

Do I detect a trace of jealousy and bitterness in my colleague's last comments? Better fix up that attitude Tim, or there will be nothing but coal in your stocking this year.

It's interesting that Tim brings up this subject, because it's another case where the differences in philosophy between Windows and Unix are apparent. In Windows, you need to fix up your directory permissions with an external tool after you copy the files. In Unix, it's just a natural part of the file copying operation-- particularly if you're doing the copy as the superuser.

This is also an area where we've seen some historical evolution in Unix-like operating systems. When I first got started with Unix in the 1980's, the "cp" command didn't have a "-p" option to preserve permissions, ownerships, and timestamps. The way you would copy directories when you wanted to preserve directory permissions was with the so-called "tar cp" idiom (actually, real old-timers will remember doing this with cpio):

# cd olddir
# tar cf - . | (cd /path/to/newdir; tar xfp -)

Here we're running the first tar command to create ("c") a new archive from the current working directory (".") and write it to the standard output ("f -"). We pipe that output to a subshell that first changes directories to our target dir and then runs another tar command to unpack the incoming archive on the standard input. The "p" option means preserve permissions, timestamps, and ownerships. Actually "p" is normally the default if you're running the tar command as root, so you can leave it off, but I prefer being explicit.

These days, however, there are a couple of simpler options. Obviously, you could just use "cp -p":

# cp -Rp olddir /path/to/newdir

I generally prefer rsync though:

# rsync -aH olddir /path/to/newdir

rsync not only allows you to copy directories within the same system, but also gives you the option of copying directories across the network. Also, if you just want to update the permissions on a directory, the rsync command will do that and not actually copy any file data that has previously been copied. For more information on rsync, see Episode #24.

One issue that Tim brought up was that sometimes you want to copy only part of a directory structure, but exclude certain files and/or subdirectories. This is another place where rsync beats cp. rsync has a couple of different ways of excluding content: the --exclude option for specifying patterns to exclude on the command line, and --exclude-from for reading in a list of patterns to exclude from a file. There's no way of excluding files built into the cp command at all. For those old fogies like me out there who still occasionally use "tar cp", the tar command typically has a switch like -X to exclude files and directories from the archive, and GNU tar has --exclude options very similar to rsync.

One thing you do need to be careful with for all of these copy options, however, is that they may not copy special permissions like extended attributes or file ACLs by default. Both cp and rsync have explicit options you can set to preserve these settings:

# cp -R --preserve=all olddir /path/to/newdir
# rsync -aHAX olddir /path/to/newdir

There's no way to do something similar with the "tar cp" idiom, because the tar archive format doesn't preserve extended attributes and ACLs.

Oh dear. Now it's Ed's turn. I hope Tim and I haven't spoiled his holiday cheer...

Ed Joyously Responds:

Ahhhh…. file permissions. They tend to be an absolute pain in the neck to deal with en masse in cmd.exe. Sure, we can use cacls or icacls to manipulate them on single files or directories just swell. But, synchronizing or applying changes to bunches of files using cacls or icacls is often dangerous and painful. When I first read Tim's challenge, I thought to myself, "This is gonna get ugly on us… as ugly as that feud between Snow Miser and Heat Miser." I immediately began to search my mind for a hook or trick to make this a lot easier, hoping to avoid a trip to visit Mother Nature.

Then, it hit me: we can use our little friend robocopy, the wonderfully named tool in Vista, Windows 7, and Windows 2008! Yeah, it's not built in to XP or Windows 2003, but it'll work for the latest version of Windows. We talked about robocopy in Episode #24.

To address Tim's challenge, I'm going to assume that the directory structure where we want to replicate our file permissions does not already exist, avoiding the mkdir commands Tim uses. Robocopy will make those for us, dutifully placing the proper access controls on them if we run:

C:\> robocopy [directory1] [directory2] /e /xf *

All of the subdirectories in directory1 will be created in directory2 with the same file permissions. The /e will make it recurse through those subdirectories, copying both directories with stuff in them and empty directories. The /xf means I want to exclude a certain set of files, which I've selected as *, meaning to exclude all files -- Only directories will be copied, including all of their luscious permissions.

Well, that's all fine and good, but what about Windows XP and 2003? Well, you can download and install robocopy on either of them, which is a pretty good idea. Alternatively, there is a way to trick Windows into applying the permissions from one directory structure to another, which applies to Windows 2003, Vista, 7, and 2008 Server. For this trick, we'll use the handy /save and /restore feature of icacls. Here, let's follow Tim's lead, and assume that we've got directory1 and directory2 already created, and we want to take the permissions from directory1 and its subdirectories and apply them to the already-existing directory2. Check out the following command:

C:\> icacls [directory1]\* /save aclfile /t /c

This command tells Windows to run icacls against directory1 and all of its contents (*), saving the results (/save) in a file called aclfile, recursing through the directory structure (/t), not stopping when it hits a problem (/c). Now, the resulting aclfile is not regular ASCII, but instead a unicode format that includes all of the permissions for the directories _and_ files inside of directory1.

Now, if there is a directory2 that already exists and has a similar overall structure to directory1, but perhaps without having any files in it, we can use icacls to restore the aclfile on a different directory! Wherever there is commonality in the directory structure, the permissions from directory1 will be used to overwrite the permissions on the given entity in directory2. The command to use is:

C:\> icacls [directory2] /restore aclfile /t /c

Voila! We've restored the ACLs from directory1 onto directory2! Now, that is a delicious holiday treat.

But, that leaves out poor little Windows XP, an operating system without robocopy and icacls built in. Sad, sad, sad little XP. Looks like it gets a lump of coal in its stocking this year, not only from Santa-Ed, but also from Microsoft, which has announced its impending withdrawal of support of this very near and dear friend.