[qub1n]

Hi there,

I spent some time with testing, fixing and optimizing ReFS support for git work tree implementation originally created with copilot and now I came to the point in which it works quite good.

Correctness was tested on artificial and real git repositories.

Overall the cloning is slower for small files, so I chosen threshold of files > 160KiB for which cloning happen on ReFS.

The last unaligned segment is not cloned, but rather copied due to performance.

I was comparing runs on two VHDX disk one with ReFS and second with NTFS on my laptop Acer Nitro V16. It is faster in almost all testing cases compare to original implementation with either NTFS or ReFS and in the same time it brings space savings around 30%, space saving was counted from occupied blocks on disk.

In common repositories threshold of 160Kb clone very small amount of files (in count), but good portion of size of repository. Lower threshold would mean slightly slower creating of work tree, but much bigger space saving.

Git-ReFS-Correctness-Report.pdf

[dscho]

Thank you for working on this.

In the current form, this PR is massive and not in a mergeable state, unfortunately. The commit history is in no way as clear and as succinct as we try to make it in the Git project. Git for Windows tries to follow the same rules as the Git project so that everything is upstreamable.

In the Git project, it is expected that a patch series tells a story, a very clear story, that the commit messages contain the context, the intent, the justification, and include explanations of the implementation as long as they are not also already clear from the diff. Further, the code is supposed to be DRY. 28 commits and 18 changed files do not speak that language to me. They tell me that a lot was added, and probably a lot of that is actually not even necessary to implement what we want to implement here.

Also, I fear that a lot of the aspects here are left unclear, starting with the PR title. In Git nomenclature, "cloning" has a very specific meaning. That means that every reader who is familiar with Git must be confused as to what you mean by block cloning. The term "block cloning" only makes sense for those who are familiar with ReFS, which is not the case for Git contributors who are, in large parts, Microsoft haters and would not touch it with a long pole. Coming along with terminology they are prone to dislike immediately isn't going to fly.

Since the changes are so massive, it is hard to come up with an exhaustive list of issues to fix. To name just two, after having a glimpse at the overall diff (because, as I said, the patch-for-patch story is totally unreviewable): git reset --hard will not benefit from ReFS block cloning because for those files it wants to write, there simply is no source to clone, that's the entire point of --hard; Everything outside of git worktree add won't benefit. I also spotted an inlined duplication of is_dir_empty(), and I suspect that the many changes outside of compat/mingw.c are indicative of a need for better structuring (I actually had intended to move the vast majority of the block-cloning stuff to compat/win32/, into a new file).

You said you built on #6424, and that's good, I just wanted it to be taken in a very different direction, focusing on surgical, precise fixes, re-structuring not only the commits into a clear and compelling story, but first and foremost the logic. Copilot did a really bad job at designing the structure of those code changes, so I wasn't even sure that I would start with the code as-is, but probably rather start from scratch, with a sound abstraction that stands a chance of surviving contact with the Git mailing list, and only pilfering the Win32-specific logic from the existing PR.

This is not a light undertaking, if we want to have a prayer at ending up with a PR that can be upstreamed with at most cosmetical fixes, we have to be very careful to be mindful of Git's conventions and expectations from the start.