Yes, it is most likely just the number of commits between the good and bad revisions. When I run the same bisect on my machine, it takes between five and ten seconds to finish.
The bulk of the time is probably being spent generating the log.
Aside from profiling, I can think of two quick ways to test this: 1)
stop the process with C-g and look at the state of the
status buffer (is the log partially highlighted?), and 2)
remove magit-insert-bisect-rest from magit-status-sections-hook to
see if it still hangs for you.