I am at a similar point. Things start being noticeable, my wife doesn't know and we have a 1 year old kid.
As other people suggested I should probably stop for peace of mind. Like you, I am at a level where heavy workout can help getting rid of most of the acquired shape.
I spent the last night thinking of what to do and I may go on just for a couple more weeks. The thrill is addictive indeed.
So: sensible solution is to stop. Remember the feeling of "having boobs" and maybe go back at it in the future.