Maybe this an old trick, but I just stumbled across it by accident:
Instead of using the left mouse key to drag PDF pages up and down, just left-click on any page and then you can use the mouse wheel to scroll through the document.
Interestingly, doing it this way disables the page counter unless you left-click the page, which then updates the counter.