You need to set eshell-scroll-show-maximum-output to nilnil.
Sadly the documentation doesn't really explain the difference to eshell-scroll-to-bottom-on-output, but its default value is nil anyway and therefore isn't responsible for the described behaviour. In contrast the default of eshell-scroll-show-maximum-output is t.
My understanding is that enabling eshell-scroll-to-bottom-on-output causes eshell to scroll to the bottom if you have scrolled upwards the buffer and you have a program running which produces some output. It will just scroll in a way though, that the last line will be visible in the buffer, and will not align the last line to the bottom of the window.
An enabled eshell-scroll-show-maximum-output on the other hand ignores output produced by a third party program when you scroll upwards, but will align the current prompt with the bottom of the window (hence "maximising" the last output) when you issue any kind of input with triggers a new prompt.