It wasn't that easy.

Jafar had to find and combine those 2 unique bug pieces or whatever to even reveal the cave. Then on top of that he still couldn't enter but needed one specific person to do so.

It's a friggin' multi-sig level of security.