Finally figured out why Primal.net was causing my cpu to spike up to 100% at times. Relays. Played around with my relay list and found out relay.damus.io was the biggest culprit. No idea why but after removing it things are purring now.