I'd like to make my own FIFO2 USB dongles and am thinking along the same lines.

Can't make my own chips for the time being (although https://www.bunniestudios.com/blog/page/9/ and https://www.crowdsupply.com/sutajio-kosagi/precursor) so it's about doing what's possible with off-the-shelf microcontrollers.