My podcast gripes:

#1 top gripe - music playing over speaking!!! Not just annoying but often winds me up and is very distracting, especially when it's usually on a repetitive loop, but also bad for hearing impaired people. It bad enough without the music. The music renders it completely inaccessible. Some podcasters play very loud music over the speaking, making it difficult to hear, so I just switch off.

#2 Too long - cut out the meandering. I know people want theirs to be chatty and friendly but don't have time for 1 to 2 hour long podcasts with lots of pointless chitchat.

#3 Guest interviews - let them talk (unless it's a mutual conversation style).

#4 Bad quality audio