Not sure on history. Simon Singh's 'the Code Book' was decent on that, but it's popsci, so fairly light.

For learning fundamentals of modern crypto in practice I'd recommend Serious Cryptography by J P Aumasson.

Boneh and Shoup give you a lot of solid grounding for serious study (e.g. security arguments) in toc.cryptobook.us

Work through the cryptopals challenges if you like hands on learning.