I think we need to define the desired outcome.
You could for example create a constraint that a branch is a pubkey, and only the private (or aggregate private, Schnorr, MuSig, etc) can merge to it (git already has signed commit hashes). That would allow replicating branch permissions.
^^ I think we should discuss and jack should settle on a design outcome like this.
Since git works with hashes, it's not hard to reference specific lines, commits, branches, etc and that's all that would be needed to add review notes.
Like you said, clients would then aggregate those notes into a concise view.