A lot of channels will have 1 sat set up as a base fee and then a certain percentage, usually really small, on top of that. If you were to send, say, 500 sats, the fee might only be 3-4 sats. A much smaller percentage than the 20% you saw on the 5 sat invoice.