Somewhat, there is a difference though. While impact damage may be the same, the problem 2-fold from what I can think of right now: #1 (probably the main reason) noise. My bike is quieter than my roommates stock muffler but it's still fairly noisy. The ring-a-ting-ting of a 2-stroke isn't really appealing either, especially at close range (I ease off the throttle when someone's beside me with their window down). And #2 if you do crash into someone or something, that's a really hot motor you got there. 350 degrees F on the head under normal conditions (mine is 202F running though, plugs probably a couple ranges too cold and possibly running rich with low-ish compression), and your exhaust (which is substantially hotter when running an x-pipe and bigger) can typically hit 1100F, enough to set fire to alot of things with contact more than a couple seconds (again, mine typically hits around 232F running). #3 (I know I said 2) is pollution/smell, but with the tiny amount of gas our engines take, we produce less **overall** pollution than most cars (I asterisked overall because 2-strokes and bikes in general, due to their small size, lack catalytic converters. CC's are designed to reduce the especially harmful gases emitted from a running engine).
In the end, I wouldnt even run an electric motor on the sidewalk, so overall I agree with you.