My experience is that you can be comfortable enough down to just a few degrees above zero F if you dress and protect yourself from wind properly. So cold isn't all that big of a deal.
Clear roads with occasional ice patches are not a problem if you have studded tires.
I did find myself feeling insecure on rough hard-packed snow/ice. As fall is approaching I've been working on ways of getting my center of gravity lower. And I expect to do better with that rough ice this year.
But it's not a slam dunk. That, to me, is that hardest part of winter.