← Writing

WorkBuddy Will Corrupt Your Git

6 min read#Git#WorkBuddy#Forensics

The same repo broke six times and I repaired it six times. On the seventh I opened the Recycle Bin: 12,934 entries belonged to .git.

Table of Contents

One repo broke six times: September 16, 17, 19, 20, 22, then October 1. The error was different every time — sometimes git status called HEAD a bad object, sometimes fetch said every object was present while checkout failed, and the worst one was a flat fatal: not a git repository, with .git down to 1.8 MB.

Five times I suspected the disk, then the antivirus, then the git version, ruling each out in turn. The sixth time I went and counted the Recycle Bin: 12,934 entries belonging to .git.

The cause is a safe-delete layer in the sandbox that rewrites deletion as moving to the Recycle Bin. It intercepts more than the rm I type myself — it also intercepts unlink from the git process. And a great deal of routine git work is unlink: removing lock files, temporary indexes, stale refs. Once intercepted, none of it errors. It just leaves half-finished state where it stood.

01How to confirm

Run a command that in theory writes nothing to disk, then go look at the Recycle Bin:

git update-index --refresh

A .git/index.lock shows up in the bin. Git creates the lock, deletes it when it is done, and that deletion became “move to Recycle Bin”. That single command settles it — no disk scan, no git downgrade required. I tried all three myself, and none of them were it.

There is a side effect worth its own paragraph. Git objects live in pack files with an index (.idx) beside each one. When the index is moved away and the pack stays, git concludes those objects are all local and stops refetching them, leaving orphan indexes: far more indexes than packs. To check whether you have them:

ls .git/objects/pack/*.pack | wc -l
ls .git/objects/pack/*.idx  | wc -l

If the two numbers differ a lot, that is this disease. It also explains why the error is never guessable: where the corruption lands is random, and so is the code path it lands on.

The most embarrassing one was self-inflicted. On the afternoon of October 1 I wanted a before-and-after diff, so I casually typed git stash push with a path attached — a trigger my own skill warns about explicitly, and I had not loaded that skill before running it. Starting at 14:56:43, 966 loose objects and four index.lock files went to the bin inside twelve seconds, 971 entries in total. My first reaction was that this was old damage unrelated to my change, and I reported exactly that. Wrong. The index.stash.17604 sitting in the bin carried my own process id.

That repair became a routine I ran several more times. Move everything back out of the bin first, but exclude index.lock: that lock was meant to be deleted, and restoring it simply locks the door again. Then git fsck — 21 missing blobs among 3,358 indexed files, rebuilt one at a time with git hash-object -w. Further along, the history turned out to be severed thirteen generations back with one commit’s parent gone entirely, leaving only git fetch --refetch: 20,471 objects, 37.67 MiB, the commit count climbing from 13 back to 776.

The most expensive failure was a different one. Trying to clean up redundant objects, I ran git prune-packed. The object count went from 2,717 to 0 and the commit HEAD pointed at became unreadable on the spot. The bin gave back 2,572; the remaining 145 were genuinely deleted and not in the bin, so another refetch — 20,524 objects, 37.74 MiB. The command is perfectly safe on a normal machine. The mistake was running routine maintenance without realizing that deletion here cannot be trusted.

Before that I tried three ways to prove it was not happening at the script layer: stripping every relevant environment variable, calling the real git binary directly, and performing the deletion through an interpreter that skips site config. All three were intercepted, and PowerShell’s delete command could not remove anything either. Don’t waste time hunting for a switch; this is a host capability and cannot be turned off from inside the repo.

One action is not hooked, though: rename. Move a file away with mv and the bin gains exactly zero entries. So anything cleaned out of .git now gets moved into a quarantine directory. Never deleted.

02A few more traps

Remote refs silently fail to write. When the nested directory .git/refs/remotes/<remote>/ does not exist, git update-ref returns 0, raises no error, and does not create the directory either. It shows up as [gone] in status, with every fetch reporting a new branch. Either mkdir -p that path yourself, or run git pack-refs --all once: packed-refs uses lock plus rename, which slips past the hook.

A hand-edited .git/packed-refs must use LF. On Windows, Python’s write_text turns \n into \r\n by default; git then reports a bad ref and rev-parse complains Needed a single revision. Run git show-ref the moment you finish editing.

And one I still don’t fully understand: cleaning an expired lock trips the guard, and the guard raises SystemExit, not a subclass of Exception, so none of the surrounding except Exception clauses caught it. The process simply died halfway with no output whatsoever. By the time it happened, pytest’s temp directory had grown to 187 MB across sixty-five thousand directories. Renaming that lock to .stale was what finally worked. Since then I no longer assume a cleanup succeeded, and I no longer treat except Exception as catching everything.

03Two things I still don’t know

The first: why prune-packed made the commit HEAD pointed at unreadable on the spot. I have two candidates — the bulk-deletion threshold branch, and the path that degrades to real deletion when the bin fails — and neither is confirmed. The second: how the orphan indexes are produced at all. Also just a hypothesis.

So my repo currently carries 2,572 loose objects, roughly ten times a normal count, and I don’t plan to clean them. The cleanup itself caused the last incident, and I still don’t know why it did.

What is left of the daily routine is short: use mv inside .git; run a read-only command and count the bin to see whether interception is live; judge sync state with git ls-remote rather than the status line, since local refs can be fake and swing between up to date and ahead 74; back up by copying the whole .git instead of stashing.

Written byShihao Qi