Playing around with gitoxide - an implementation of git in Rust

2023-12-27 867 words 5 minutes

Contents

Background

There’s git which is a CLI tool implementing git SCM which we all know and love.

Additionally, there’s libgit2 which is a reference implementation of git “standard”/“protocol”. The focus of it, is primarily to be used within applications that wish to integrate with git repositories and provide programmatic access to git facilities.

Usually, languages like Python or Go use libgit2 under the hood to implement git bindings. This is the case with e.g. pygit2 or git2go and many more.

Recently, I stumbled upon gitoxide. Which, as I initially thought, would cultivate exactly the same approach, meaning it would use libgit2 as low level routines for git, but this doesn’t seem to be the case.

First thing that comes to mind is … why? Reading through project goals, the aspiration is to:

… be the go-to implementation for anyone who wants to solve problems around git, and become the alternative to GitPython and libgit2 in the process.

There’s a whole slew of questions I might have behind this motivation, but I’m gonna spare myself the effort. Regardless of all of that I decided to give it, at least, some sort of superficial valuation.

Benchmarks

gix is not yet feature complete so there’s a limited set of things that can be tested. Cloning a repository is supported though and its performance might be compared between git and gix so, this is what I’m going to do.

For the purpose of this exercise I’ve clone Linux kernel repo (to eliminate any networking related aspects) like so:

git clone --bare --progress https://github.com/torvalds/linux.git

Bare in mind as well that I’m using a relatively slow SATA SSD so the numbers might seem high.

git clone

Starting with git I get a bit of a weird results.

1
2
3
4
5
6
7


time git clone --bare linux linux_git.git
Cloning into bare repository 'linux_git.git'...
done.

real	0m0.079s
user	0m0.037s
sys	0m0.037s

This is too good to be true as it’s physically impossible to copy ~5G worth of data on my machine that quickly. So, what has actually happened?

Turns out that git implements an optimisation when cloning a local repository and instead of copying the data from .git/objects it just creates hard links.

Listing the contents of the new clone, sure enough, ls reports link count > 1 for majority of files.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


2056:hermod linux_git.git 0 (BARE:master) $ ls -l .
total 116
drwxr-xr-x 2 tomasz tomasz  4096 Dec 27 13:30 branches
-rw-r--r-- 1 tomasz tomasz   123 Dec 27 13:30 config
-rw-r--r-- 1 tomasz tomasz    73 Dec 27 13:30 description
-rw-r--r-- 1 tomasz tomasz    23 Dec 27 13:30 HEAD
drwxr-xr-x 2 tomasz tomasz  4096 Dec 27 13:30 hooks
drwxr-xr-x 2 tomasz tomasz  4096 Dec 27 13:30 info
drwxr-xr-x 4 tomasz tomasz  4096 Dec 27 13:30 objects
-rw-r--r-- 1 tomasz tomasz 84343 Dec 27 13:30 packed-refs
drwxr-xr-x 4 tomasz tomasz  4096 Dec 27 13:30 refs

This can be further confirmed by inspecting the inode number of an arbitrarily chosen object between two repositories:

1
2
3


stat linux{,_git}.git/objects/pack/pack-ff01ee2f2c6db77ba0e85e66b1fd277cf2545546.idx | grep Inode
Device: 254,2	Inode: 1058406     Links: 2
Device: 254,2	Inode: 1058406     Links: 2

As expected both point to the same inode. This behaviour can be disabled with --no-hardlinks flag. Let’s try that.

1
2
3
4
5
6
7


$ time git clone --no-hardlinks --bare linux.git linux_git2.git
Cloning into bare repository 'linux_git2.git'...
done.

real	0m11.134s
user	0m1.058s
sys	0m5.874s

This result is comparable with cp -r or rsync -a so, it seems to be more reliable and useful as a baseline for comparison.

gix clone

Installation of gitoxide is easy in my case as it is available in Arch so I need pacman to do it.

I will use version 0.32.0.

Let’s perform the same test as with git.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


$ time gix clone --bare linux.git/ linux_gix.git
 13:48:44 indexing done 9.9M objects in 142.39s (69.3k objects/s)                                                                      
 13:48:44 decompressing done 12.4GB in 142.39s (87.4MB/s)                                                                              
 13:51:47     Resolving done 9.9M objects in 183.17s (53.9k objects/s)                                                                 
 13:51:47      Decoding done 129.9GB in 183.17s (709.3MB/s)                                                                            
 13:51:55 writing index file done 334.4MB in 1.36s (245.6MB/s)                                                                         
 13:51:55  create index file done 9.9M objects in 333.32s (29.6k objects/s)                                                            
 13:51:55          read pack done 5.0GB in 614.66s (8.1MB/s)                                                                           
HEAD:refs/remotes/origin/HEAD (implicit)
        fbafc3e621c3f4ded43720fdb1d6ce1728ec664e HEAD symref-target:refs/heads/master -> refs/remotes/origin/HEAD [new]                
+refs/heads/*:refs/remotes/origin/*                                                                                                    
	d566ec94e63702a1bf83f43804b021a8f478a27b refs/heads/dependabot/pip/drivers/gpu/drm/ci/xfails/pip-23.3 -> refs/remotes/origin/dependabot/pip/drivers/gpu/drm/ci/xfails/pip-23.3 [new]
	fbafc3e621c3f4ded43720fdb1d6ce1728ec664e refs/heads/master -> refs/remotes/origin/master [new]
refs/tags/*:refs/tags/* (implicit, due to auto-tag)
	5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c refs/tags/v2.6.11 object:c39ae07f393806ccf406ef966e9a15afc43cc36a -> refs/tags/v2.6.11 [new]
	5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c refs/tags/v2.6.11-tree object:c39ae07f393806ccf406ef966e9a15afc43cc36a -> refs/tags/v2.6.11-tree [new]
    ...
	adab409b5eb1c5905c260f74c75725db3da46e38 refs/tags/v6.7-rc7 object:861deac3b092f37b2c5e6871732f3e11486f7082 -> refs/tags/v6.7-rc7 [new]

real	10m15.549s
user	14m13.188s
sys	0m10.899s

This is very disappointing. We’re talking about ~60x time difference. Judging by the log it seems like gix re-packaged all objects when cloning - this is something that git probably doesn’t do.

Conclusion

I’m not sure if gitoxide will ever become a viable rust based alternative to libgit2. There’s definitely a lot of effort and time required to bring the project to maturity. As of now, I’m happy to stick with libgit2 rust bindings like e.g. git2-rs. gix as a CLI tool utilising gitoxide is definitely not yet there to be a true competitor with git.