Android system manages memory aggressively by putting background apps to sleep.

Sometimes you can manage it in android settings.

Or try this

By any chance are you on an old / limited memory phone?