One of my neighbors popped in to borrow something and I was wearing shorts after having recently removed all hair and they did a bit of a double take, but no big deal.
Two days later I was talking outside by the pool with one of their best friends (big, older guy) and noticed he didn't have any hair on his legs at all. I'm not sure I personally would have noticed it unless I was already thinking about it.
Then again, some may be more apt to check and take notice if they suspect you might be up something interesting!