What relay are you using in amber for connecting with olas?
Olas don't support local relays like citrine yet and some relays are really slow depending on your network and where in the world you are.
Most relays are in US and even for people in the US it gives a high latency sometimes