In addition to what has been stated, you'll have to remember that science is of physical domain while magic and related beliefs are of the subtle domain.
Physical change needs physical energy to occur. Magic isn't seen as a physical energy, but as a parallel to it that isn't as dense because it isn't affected by time and space like physical reality is; in this manner, it can be seen as an indirect influence.
Perhaps it would be better to look into magical practice without the lense of controlling physical reality and look into new practices that might interest you. Even without what is considered impossible, there are a lot of beliefs and practices to learn about and possibly get into.