A proper framework to think about distributed processing for AI, is (Bayesian) predictive coding --- no need for backpropagation just message passing. Each dvm is trying to predict the output of oder dvms, some dvms are linked to data and decisions, they collectively form an information proccesing network, much like information processing in the brain. In ML this is also known as variational autoencoders but with distributed components.