In my case, I'd have to say the answer these days is, Yes.
Yes, it can be too cold to ride. Even when you dress for it, as I have done: laters of thin wool and wind-resistant riding pants, sweaters and a wind-proof shell, all extremities covered with hat and gloves and neckwarmer, shoes covered with an extra pair of old wool socks, my bicycle's toeclips with wind-covers installed for the winter. Yes, I am doing everything right. And it is still too cold ride some days.
For the last few weeks, Portland has been in the grip of some of the coldest temperatures we've ever seen in late November and early December in a long time. Lows have been in the teens. Highs have often been below freezing.
(My friends in the Midwestern US can go ahead and chuckle now. I know you deal with this stuff all the time; we'll compare notes when you guys start seeing 45 inches of rain a year, okay? It's all good.)
I missed some of it by being in Mendocino (along California's beautiful northern coastline) for Thanksgiving. There, we walked around in shirtsleeves along the rugged, windswept headlands. The high was near 70 some days! I definitely got some Vitamin D during that trip and, in spite of not having a bike to ride, I came home physically refreshed from walking outside every day.
Since returning home, I have resorted to multi-modal transportation most days. However, the last few days have just been Too Damned Cold for me to ride in, and when I've had to venture out I've taken the bus or gotten a ride in someone's car.
When does it get too cold to ride a bike? When I do so and the next morning my knees creak painfully from having been out in sub-freezing temps, that's when. I stubbornly rode errands last Thursday, to Portland State and then all the way up into NW Portland to deal with the album art and then to teach. By the time my class got out at 6 pm, the temp had dropped down to 21F and my knees let me know they were not up for a long ride over the Broadway Bridge. I hopped MAX over the river and all the way up to Overlook before riding the rest of the way home.
I love riding my bike, don't get me wrong. It is still the loveliest way to get from place to place. But when the thermometer tells me it's below 20F, well, that is just too cold for my knees. I still look younger than my age (I always have, since I was a kid, only now it's a source of amusement rater than frustration), but I can tell you that my joints really are fifty years old and riding when it's this cold out only confirms it.
Today it's going to reach a relatively balmy 32 degrees. Yessss! I'm going to ride over to Velo Cult and see what's happening. The sun may even peek out from behind the cloud cover now that the snow has melted. I'll be sure to leave before dark.