That is certainly an option.
But it comes with it's own set of problems. I can't follow you privately. You have to be able to determine who your followers are. You can cut somebody off from listening to you. It doesn't scale as well.
It seems to be to be more of the same authoritarian model that mastodon is known for: top-down push instead of bottom-up pull.
The outbox model is extremely successful in other protocols: HTTP, RSS, etc
