So by "everything" you mean any arbitrary spam sent to you by some random guy on the internet? And you're performing something like ecdh on it to see if you're being paid?
Discussion
So these "lite" wallets connect to a remote node and, query the transaction data from them through a rpc interface and scan through it on their side. They trust the remote node to speak the truth when scanning to not feed it fake transaction data. So in the worst case you might calculate a faked balance and transaction history.