COW Links

I didn't know this about emacs (I use vi).

I think that there's a good argument for making COW links appear to have Posix semantics by default, so that applications that don't know about COW links work correctly, but those that have been modified to know about them can do better.

The model to use is to think of COW links as an optimization of file copying, and then to use them all the time (e.g. tell cp to use COW links if the copy is to the same FS). The model, to a naive program, is that a COW link is a complete copy of the file. This means that a naive du program will show the COW links as taking up space that they don't, and diff and cmp's shortcuts, where they know that two files are identical if they have the same inode, won't work. But the key point is that all programs will continue to work correctly. Then a new interface can be added to detect the COW link, which will permit programs like du, cmp, diff etc. to work more efficiently. If we don't assure that COW links obey well-understood POSIX semantics, it is likely that new ways to attack security will be found (because some program that doesn't know about COW links might be made to malfunction if one is present), and if we depart from known semantics, that just opens up a can of worms (how do hard links to COW links to symbolic links behave, etc). If a COW link is treated by all of the existing POSIX operations as if it were a distinct, separate file, these questions answer themselves.

I hope that by the time COW links reach production, discussions take place with the BSD people so that portable free software can take advantage of them.

Translucent filesystems give us COW semantics and have been available on other forms of UNIX for many years (so we can learn from all the mistakes that they've made and be reasonably assured that most of the software that we'd be most likely to use on them has already been adapted to their quirks).

In some implementations you can view each of the layers separately from their union (via different paths/mountpoints).

I think that's a far more fruitful direction to go in this effort.

JimD

This is really quite an assault on the integrity of the file interface definition.

If I understand what's proposed, it means that when you write to a file, if you happened to originally find that file via a directory entry that has/had the "cow" attribute, then before your write goes through, the system creates a new file, copies the entire contents of that old file to the new one, and then magically changes some (which?) file pointers and some (which?) open instances to refer to the new file instead of the old one.

Or looked at another way, you have two different files all along, but most of the internal data structures and code think there's only one.

The implications of that are so messy, I don't even want to start going through all the scenarios.

The overlay filesystem mentioned above is a cleaner alternative. Another cleaner alternative is just having multiple files share the same blocks in copy-on-write fashion. Things like du and diff can be inconvenienced by these, but at least mainstream file access is still sane.

To answer some of the comments:

Inode numbers:
Contrary to Linus' comment, there are good arguments against using the same inode for cowlinks. People want to change the permissions, ownership and such independently for each copy. If you use my patch, be prepared for incompatible changes.

POSIX compliance:
It appears, as if complete POSIX compliance would be impossible anyway. The real copy is deferred until later, possible never. Whenever it does happen, -ENOSPC is possible, which is somewhat unexpected during open. Erik Biederman claims that no new problems arise that don't exist with sparse files already, but there may be more corner cases hidden somewhere.

Unionfs/Unionmount:
Unionmount can in principle do the same, unless you actually try do use it. Having to mount a new filesystem for each copy is a) clumsy, b) dangerous because you have to remember the exact order in which to remount everything after reboot and c) prevents anyone but root to use this feature. Bad idea.
On the other hand, it is possible to implement union mount on to of cowlinks, which may or may not be a sane design. I'll try to work closely with some of the people planning union mounts for 2.8.

"Messy" concept:
"Messy" is a description for both cowlinks and union mounts, until you start to think things through and define sane semantics. Think of the mess that can happen when you have hardlinks to symlinks. Obviously that is already forbidden and for the curious, this is how the cowlinks fit into the picture:
o Symlinks can point to hardlinks or cowlinks or regular files.
o Hardlinks can point to cowlinks or regular files.
o Cowlinks can point to invisible files.
o Invisible files cannot be accessed, except through a cowlink.

Security issues:
It appears, as if no new problems arise from the new design. Famous last words.
The current patch can definitely create a ton of new problems. Or rather, it can make existing problems exploitable. If all software was perfect, the existing patch was harmless.