You need to take the arm off the hub first. It is very simple to do - remove the wheel from the bike, take off the nut holding the brake arm to the hub and remove the washer. The arm should then come off as it sits keyed to the brake cone.
The challenge you may have is the sprocket will not clear the dust cap (thin cap on the hub between the break arm and the brake cone that keeps dirt out of the bearings). You will need to either enlarge the hole on the sprocket (difficult) or trim down the outer edge of the dust cap (easy). I just grind down the dust cap on a bench grinder - takes about 30 seconds to do.