I wrote a nearly complete implementation of git file format parsers in Common Lisp over like a month of evenings and weekends. I’m sure there are hard parts between where I am and a full git implementation but you can get quite a bit of utility out of a relatively small amount of effort.

It's a case of Pareto. Parsing the git file format is relatively simple, but handling all the weird states a Git repo can be in and doing the correct things to those files in each state is a lot harder. And then adding the network protocol on top of that makes directly reproducing Git quite difficult.

I know JJ used to use Git2 for a lot of network operations like pushing and pulling, but ran into too many issues with SSH handling that they've since switched to directly invoking the Git binary for those operations.

There aren’t that many weird states a git repository can be in: the on-disk format of the repository is too simple for that. The hard part has to do with the various protocols for transferring objects around.

I think there's more corners out there than most people would give credit to? Just off the top of my head: files in the index (but maybe this isn't "weird enough"), rebasing but paused, rebasing with conflicts, merge with conflicts, cherry-picking but conflicts, middle of a bisect with all the state that implies, alternate objects dirs, alternate working dirs, submodules and all of their weirdness, and a "bare" repo.

Heck, had my PS1 return an error this week after I created a separate working dir for a repo and cd'd into it. Did you know .git can be a normal file? I didn't when I wrote my PS1.

I knew .git can be a normal file because of worktrees. But most of the weird states have to do with the working tree not the repository. Even rebasing isn’t weird as far as the file formats go: it just is replaying commits on top of a new base commit. Since my goal was basically to implement enough of git to serve files from a git repository as a website, the actual task was fairly small.

Yeah, I wrote mine in Haskell. It's a good exercise for understanding how Git works

Not sure what gitoxide is, but libgit already exists, and it seems to be an independent implementation - https://github.com/libgit2/libgit2

I think Github and most big Git hosts use it

libgit2 has a ton of compatibility issues, especially around authentication, that make it only useful in some circumstances.

(gitoxide is a similar project but in Rust, it's not ready for the big time either, though it keeps on getting better!)

If it's about auth, then I'd say that's more about the "edges" / porcelain than the core. The claim about git is that you can re-implement the core data structures pretty easily, and they are stable

While new functionality can be implemented on top of this stable base. It doesn't surprise me that it's hard to re-implement all of git, including the network protocols, but that's a different thing than the core

This thread has some good info about the core, including the tree encoding issue brought up by bradfitz, but it still seems to be approximately true ... e.g. perhaps compared to hg

I also remember that JGit is a very complete re-implementation of git, done by a git core dev, and used in Android infrastructure, etc. This was well over 10 years ago at this point, maybe 15

https://github.com/eclipse-jgit/jgit

Yes, I think that the core is much better, all I mean to say is that 100% compat is difficult.

Jujitsu threw in the towel and is shelling out to the git CLI because of minor variations in libgit vs the binary.

Failing to find a write-up, but there was this lobster thread[0] where someone from GitLab reported they had to do the same owing to some discrepancies vs the binary -where all of the real development happens.

[0] https://lobste.rs/s/vmdggh/jujutsu_v0_30_0_released

But nothing in that description of problems is tied to the repository format.

If I'm not mistaken, gitoxide is attempting to not only be feature complete, but also the fastest. Both of which reduce development velocity