In the Crusades, the magic users would carry a quarterstaff. a Quarterstaff is for attacking people, not shooting imaginary magical darts at them. it's still a magical weapon because of the physical properties of the wood. a chemical reaction takes place when the wood hits an opponent.
it's all a lot simpler than you're thinking. create a fireball, get a round wood, set it on fire, hurl it. leave a sword in a freezer and it's an ice sword. set fire to it, it's a fire sword. magnetize it, it's a lightning sword. poison it, it's a poison sword.
we live in a scientific universe. the laws of physics are obeyed.