Basic math related to computation and memory usage for transformers

Link: https://blog.eleuther.ai/transformer-math/

Discussion: https://news.ycombinator.com/item?id=35631546