If you really want to drop swings short iron is where you have to improve. Drive for show, putt for doe, and irons give you the opportunity to get that 1 putt. Dave Peltz Short Game Bible is a great read.

The TL/DR version of the book: replace your 4 iron with another short wedge. For every short iron do a full, 3/4 swing, 1/2 swing, and 1/4 swing and measure it print it and tape it to the shaft.