It is an expensive business but crawling the entire web is how search engines work, a global view !== consistent global view.. theoretically it is what you can do client side, but practically cost prohibitive so you would rather get it precompiled for a fee or ads