I'm not entirely sure what the problem is. I'm using the same version of Emacs on three different computers ( laptop, work, home ), and the problem only shows up on one machine ( the home machine ).
Basically, when I'm using Emacs ( 24.4, compiled from downloaded tar.gz ) Emacs seems to take quite a while to do any sort of vertical rendering. Scrolling, finding a file ( using a custom ido vertical mode ), helm... anything that requires Emacs to render vertically seems to slow down Emacs.
It doesn't seem to have to do anything with how many lines are in the current buffer, the slow-down happens even in the scratch buffer.
I've also tried starting with
emacs -Q, and it still happens.
Oddly, it's CPU/memory usage stays pretty low while this rendering is happening.
I took a video of the weirdness, just in case my description wasn't super clear.