My first mistake was was not mounting my tensioner tight enough. The first time I tried to start the bike the tensioner flopped over. I got lucky though, the chain popped off the pulley and sprocket before the tensioner was pulled into the spokes.
The second mistake was made when I was fixing the tensioner. My chain stay was a little too small for the bracket, so I hammered the bracket a bit flatter for a good grip, then used some inner tube rubber to protect the paint and give it some grip. After riding for a while, I took the tensioner off to try and mount it better. There was just enough side to side play in it that the rubber had come apart, so the paint was messed up. And I had actually tightened the bracket down so much that I slightly flat spotted the chain stay on either side.
The funny part is that after wrapping the chain stay in electrical tape for grip, the flat spots keep the bracket in place