you can also limit this further and enable extensions only on specific websites (though the extension software still needs to "know" that you visit these websites :) otherwise they can not provide the functionality.
https://guides.getalby.com/overall-guide/alby-browser-extension/faqs/how-to-enable-alby-only-on-specific-websites