Have you ever used midnightcommander (as one excample)? You could simly use the mouse and not the keybindings. But if it annoys you, just disable moused.