I don’t know if I understand correctly what you want to achieve, but in case you didn’t know - you can use ESC to end searching in any moment and your cursor will be in place where you put it before that action. And you don’t need mouse in any moment - ctrl-f, type what you want, tab to results, arrows to go to specific place, esc to remove search form and be back in normal view.