I have a battery discharge protection circuit which should address that. When the power gets too low it just cuts off to avoid that wonkiness.
I hope it works as well as advertised when the rubber meets the road. Also, I just realized that I soldered the wires on my battery harness without that circuit in there. Oops. Guess I'll get to do it over.