I used to play with DD-WRT, Tomato and OpenWRT in different routers. All have their pros and cons. You need to have a router the firmware works with, you have to be comfortable flashing the router to overwrite stock firmware and you also need to be comfortable doing all of your own config within the router. Mess that up and it won't work or worse you've created a huge security problem for your home network.
Discussion
I now run pfsense on a homemade firewall/VPN server. I repurposed an old Dell Optiplex microATX motherboard with Intel i7 and added an Intel 4-port ETH card; installed in 1U server chassis in my server rack. You could also do on something like a Protectli Vault device, which you can get for a few hundred $ on Amazon, depending on which one you want, number of ports, etc. I don't have one but have read good things.
Also, there are great tutorials on Youtube on how to set up pfsense. My favorite step-by-step guide was done a few years ago by another pleb EconoAlchemist; here is is post:
https://www.econoalchemist.com/post/bitcoin-home-mining-network-privacy
While this was written around bitcoin miners this is still perfectly relevant to those not mining. Even covers how to create and isolate various home networks from each other (e.g., keep your IP devices on a separate home network from your computers, etc). He did a fantastic job with this; highly recommend throwing him a few sats if you find value in this.