rsync


Frequently I want to copy files or directories from within one tree and inject them into another tree. In such cases I want to preserve timestamps, ownerships, and permissions, including those of all parent directories on its path, and I don’t want unspecified children of directories nor siblings of files. Let’s look at doing that with the various common tools available: tar, rsync, cp, install, cpio, afio.

We’ll have a list of paths in /tmp/paths and the objective is to copy them from /tmp/source to /tmp/target. Most of these methods face the issue that they create ancestor directories fresh rather than copy them. The solution is to ensure /tmp/paths contains every directory in every path, always listing parents before children. For example, if aa/bb/cc/d.txt is a path, then aa, aa/bb, aa/bb/cc are also paths and are included in the file in that same order.

tar

tar -C /tmp/source -cf - --files-from /tmp/paths --no-recursion \
| \
tar -C /tmp/target -xf - --same-owner --atime-preserve --same-permissions

(If your tar doesn’t have a -C option, you can use the classic workaround as follows.)

(cd /tmp/source && tar -cf - --files-from /tmp/paths --no-recursion) \
| \
(cd /tmp/target && tar tar -xf - --same-owner --atime-preserve --same-permissions)

rsync

rsync -lptgod --files-from=/tmp/paths /tmp/source /tmp/target

This does exactly what we want, but the nice people designing the interface provide an even easier to remember set of options which is equivalent.

rsync -a --no-r --files-from=/tmp/paths /tmp/source /tmp/target

One big advantage of rsync is that you don’t need to include the prefix paths (ancestor dirs) in the paths file. Less prep and an easy-to-remember invocation. Thank you rsync.

cp

while read path; do
  echo /tmp/source/$path
done </tmp/paths \
| \
xargs cp -pd --parents -t /tmp/target

While an interesting invocation of cp it doesn’t meet the requirements; it copies everything to /tmp/target/tmp/source instead. In fact I failed to find a successful invocation of cp, even putting it in a loop (and so losing efficiency), because it declines to copy a directory by itself.

Others

install is very similar to cp, but even less useful for this task (because it doesn’t preserve ownership). cpio has a terrible manpage and afio seems to have fallen out of favour (superseded by cpio). (To be fair, there’s no point them competing with tar and rsync in this market.)

See also: Using rsync to copy just a directory

You want to copy a bundle of directories exactly, but without their contents. That is, you want to preserve owner, group, permissions, timestamps. cp -a and rsync -a would do it if you also want the contents. To do this just for top-level directories, you can use

rysnc -dogpt sourcecontainer targetcontainer

That’s very useful when sourcecontainer holds a bundle of directories with specific ownership and permissions that need to be preserved, eg when starting a new web app

rysnc -dogpt /srv/existing_web_app/ /srv/new_web_app

NB: The ‘/’ at the end of sourcecontainer copies its contents to the target. If you only want a specific directory, name that directory as the source but don’t have a ‘/’ at the end. I think things work safest when source and target are absolute paths (as in the example).

Possibly an easier invocation to remember is

rsync -ad --no-r sourcecontainer targetcontainer

the ‘d’ being for copying directories even without their contents.

See also: Using rsync to copy paths