i just read that some people have no reaction to coffee at all, and for some it stimulates the flight or fight response, sounds like it maybe whats been happening and why others such as yourself are totally fine.
I don't react very strongly to coffee. For me to get reaction, the intake has to be several big cups, I'd said about 1 - 2 litres of coffee. Well, this in regular brewed coffee, not espresso for example. But even with espresso just one doesn't do the trick for me.
Personally I think it could be a somekind of build up / tolerance. Sometimes I do go days without coffee, but usually my intake is about two cups a day.
I don't go to coffee places or restaurants that often. I've tasted in several places and quite often felt displeased of the coffee.
There's one shop I might visit, but it's quite rarely I do so. I really rather brew it myself, trying different brands, roasts and beans from time to time.
Anyhow, sad to hear you had such a strong reaction! I've only got jitters once or twice, panic attacks are no joke.