General feedback:

I think the scrolling system is fine to begin with, but it would in my view be more efficient to navigate via pages, where you define let's say 20 - 50 notes per page. In this way way can easily go back and forth in the notes flow without scrolling, just skipping pages.

Another possibility would be to define one page as 30 minutes of notes, and let users go back and forth between pages that way.