If you know an address I own, you could watch my transactions, but would not know who I sent money too unless they too have their addresses known.