This is really about Chaumian mints rather than federations in particular, right.
Something I find really interesting is, to the extent your point is true, it's also ~ the reason that these servers aren't auditable, which might be the thing that takes them from 'a bit dodgy' to 'really dubious trust model', and therefore, perhaps, why they will struggle to find usage.
That's bitcoin's special sauce, scantxoutset and so on.