Is your throttle cable adjusted a little bit too tight? there should be some free play in the twist grip before the throttle responds about 1/8" of free play is good.
I had the same issue with one of my bikes a while back & the remedy to it was exactly as Norman suggested, the cable did not have enough free play in it so every time I moved the handle bars left or right the engine statrted to rev higher & I couldnt get my revs down with the idle screw, gave the cable some slack & it fixed the problem instantly
So you would adjust that at the top of the carb I would guess?
I would just screw the cable adjuster IN a bit on top of the carb, or you could also adjust it at the throttle if it is not already screwed all the way in...