Programming Bitcoin by Jimmy Song, while not being what you asked for, is probably the first book that really helped me understand the math behind elliptic curve cryptography.

Once you understand the finite fields, most of the rest is just multiplication.