If you don't have the wallet's private key, the BTCs are not yours and it will depend on the person's willingness to return them. If they are in your wallet, then they were not lent. Person-to-person lending is always a problem and it doesn't matter if it is in Fiat money or BTC. I would avoid this.