If it takes 20 years for a smoker to be affected, and they're taking in 100% over seven days, then you taking in 10% over three days will take well over 200 years to affect you.
Like ANY chemical, the rate of exposure determines risk. Smoking Nazis want you to believe tobacco smoke defies physics, but it doesn't. Their logic states that a single beer gets you just as drunk as a gallon of vodka.
Personally, I don't like the smell of it but I don't worry about it bothering me any more than car exhaust or forest fires or cooking mistakes.