I think most people don't ask, figure it is just a style thing, or think its just gynecomastia which is pretty common anyway. After all, gynecomastia is like 100x more common than being Trans or gender variant, if you are trying to hide it you cannot get better social cover.
If you haven't read the forums 'gy ne comast ia dot com' it worth a read, cis-guys are REALLY distressed over it. So people just don't talk about it. If they notice at all.
- JJ