I suppose I'm similar to you in this regard. For a long period of my life I refused to leave my house for anything other than school in fear of being seen by people that knew me. I was completely fine with passing random faceless, unimportant, forgettable strangers on the street, and even interacting with them became easier as I grew older, yet I absolutely couldn't deal with what I thought then was a fact; that the moment an acquaintance noticed me, they'd start silently judging me. Taking a stroll in an bustling foreign city, the few times that I got the chance, was a pretty relaxing experience, but unfortunately, I live in a small town, and peeking my head out at any time other than sunrise or late night terrified me. The rare occasions I didn't have a choice but to go out, along with general discomfort I'd always feel an unmistakable pang in my chest whenever I noticed an acquaintance in vicinity, and god help I actually interact with them. Interacting with distant family members was also a pretty painful experience.
As such, it felt disconcerting hearing people complain about how shy and anxious they were, always finding a way to avoid stuff like talking to the school administration (even when they had a fair reason for it) or paying the bill in a caffe, while for them starting conversations with the surrounding people, hanging out outside or going around clubbing with friends was… seemingly just fine, for whatever reason.
It's not like anxiety's exactly rational, though. Some people get fucking existential anxiety, I guess, sitting alone in their rooms.
To get back to OP's original topic, I've always had people mistake my behavior for arrogance and plain rudeness. If you don't look cute, I don't think you should expect it to be understood as anything else, either. >>5333