Panasonic Youth rob sanheim writes about software, business, ruby, music, stuff and things

Git Clone vs cp -R –> WTF?

I knew git was fast, and I even knew it was faster than a lot of plain linux local file operations. Still, this still blew me away:

Ok, so a 26 meg repo takes almost three times as long to copy via a recursive cp than a local git clone. Thats a fairly small repo, lets try something bigger:

The rails repo at 75 megs is still ~ 3 times faster.

Obviously, this is not scientific at all, but the point is pretty clear. Git is doing some magic that lets it move files around locally 2 to 3 times faster than a plain copy. From looking at the man page, I would guess it has something to do with git using hardlinks for things in .git/objects when cloning locally. My linux fu falls down a bit here -- what are the ramifications of using hard links versus doing a "real" copy?

(This also makes me want to try out gitbak even more...)

‘gitbak’ link is broken. It should be: eigenclass.org/hiki/gibak-backup-system-introduction though that site seems to be down currently.

Thanks for the post though.

cp is not very fast. a faster way to copy is “tar cf - | (mkdir -p ; cd ; tar xf -)”

Pretty straightforward really. A hard link isn’t a copy:

$ touch foo; ln foo bar; echo ‘I am foo!’ >> foo; cat bar
I am foo!

So the cp -R is slinging data around, git clone is building a bunch of directories and pointing them at the data.

A “file” consists of a datablock on the filesystem and an inode that points to it. When you create a hard link, you just create a new inode that points to the same datablock. So it looks like you have the same file twice. Like in a directory, ‘.’ and ‘..’ are also hard links.

Note that if you change one file, the other will be changed too. (That’s why you should be carefull when messing with files in the .git directory ;-)

i hope you arent using cp to backup git repos. that makes no sense. cp isnt atomic.

what happens when you try to cp your git repo just as some one is committing? you get fucked.