I have never used CalyxOS and it's been a few years since I've used GrapheneOS. Graphene is more granular though. I'd go that route.