The group state does change with each add/remove but those are handled via commit messages that only make delta changes and don't require the full group state to be sent around.