I like it.

I don't see the problem. On average users update their contact lists during intervals way superior to ~10 seconds. And if the goal is to run this on a server, we can throw more money at the problem. A C++ impl and a beefy server can easily make it faster.