I'm not entirely sure I'm the only one with the problem, either. A couple of years ago I was hunting for a very specific button to replace one that had popped off my favourite pair of pants. I'd bought the pants from a tailor in the south of England, and the cost of sending them back for repair dwarfed the real cost of a button, some thread and a few minutes of my time, so I went shopping myself. After visiting three haberdashers around North Yorkshire I'd been engaged in conversation twice by the staff, who remarked that it was unusual to have a male customer. One of them asked if I was buying something for my mother.
In any case, these days I prefer to by anything I need from an online craft store. Rather than deal with the embarrassment of explaining that, no, as a male I'm capable of reattaching a button to an item of clothing I prefer to buy craft supplies online. Would I prefer to be able to shop in a brick and mortar store without feeling as if I don't belong? Sure. Until the day comes when I'm not treated as an oddity, though, I'll continue to get my craft supplies online.