Personally, I would look into shielding. I see everything in terms of metaphysical energies when it comes to situations like this, and my solution is to focus and create a metaphysical shield around yourself that is meant to block out those specific energies.
If you don't know, shielding is done through a process of either grounding or centering, and then focusing your energy to create a shield around you. Sometimes people will make a general bubble around themselves, or they might create one to act like a second skin. The reason people ground or center beforehand is because it's like building. Building a house without a foundation leaves the building more susceptible to caving in, being torn down, etc.
Good luck in your endeavors!