Maybe he was irritated that you didn't step back and let him press what ever button he wanted.
Kinda rude of you to
control the 'button space,' ask what floor, and screw w/ him like that.
and then with the couples... Maybe the guys didn't like that you were taking away their opportunity to be the "nice guy." ...the women, well, they just didn't know you're a Doc.