No, that's why we have Coordinators. Coordinators are individuals running a LN node and a Robosats specific software, when there is a dispute on the trade they get involved and act as intermediary.
As far, you are doing things right and can prove it, there is nothing to worry about.