If we take another example of distance into consideration. Distance is a concept which effects the physical world, as to get from one place to another physically one has to move.
What matters in spell work is where your thoughts, your conscious awareness is attending to. For example you may be st in a room but your conscious awareness, your focus can be at any part of that room, or indeed any area of the house. And once you can do projection, your consciousness can fully be in any room of the house.
Your awareness can be focused on any place in the world, and your body need not be there. I personally don't believe magic works on the physical plane, at all. It works on the astral and this causes manifestation on the physical plane. Whether you term the astral as the realm of imagination/sunconsciousness/collective unconsciousness/or an actual tangible concept is up for debate and I personally don't fully subscribe to any one.
Either way, short answer, I do not believe it matter one bit.