Search to/from a user but exclude one domain

Hello. I need to conduct a search for all emails to/from one user in our system but need to exclude emails to/from internal users. For example, I need all emails to/from John.Doe@domain.com (that's easy) but I need to exclude anything that was sent or received from other users in @domain.com. Does that make sense? I've never done a search like this. Any guidance would be appreciated.