I have a great deal of experience with this, and I'd highly recommend against it.
First of all, the endurance of USB drivers is very poor; most retail flash drives are designed for light weight storage of files with occasional updates and accesses, not the continuous hammering a drive gets during live OS operation. It's quite easy with continuous live OS operation to "burn out" a flash drive.
Secondly, typical OS configurations write to log files and update packages, etc., so the persistence space tends to fill up faster than one might expect.
Both of these are less of an issue with external hard drives connected over USB, either spinny disks or SSDs. But at that point you can do a full OS install onto the external disk and boot from that rather than using a live distribution. I've done this and it works well, though the USB speed becomes a big limitation.
There are other solutions that involve more work such as virtualization (running either linux as a VM on Windows, the reverse, or full virtual environments like #proxmox), but require more technical knowledge and administration.
