I had planned to ride it till the motor falls off but to tighten it every couple of rides in the meantime.

I have just now gotten the slack in the chain right I think. I'm still not positive but it rode better yesterday noise wise.