Like kevyleven007 said smaller is like gearing down, more torque less top end. Higher rpm, spins faster.
Smaller rear could also lower the back end, in effect reducing the head tube angle, making for slower steering.
Also, going from 26" to 24" you can stuff a larger tire in the back with more rubber for a more comfy ride. Doing that would not change the head angle much though as a fat 24"x3" is about the same diameter as a 26".