A servitor doesn't absolutely have to take a permanent piece of you with it. I would personally recommend against creating such a servant however, as there are a great many potentially unforeseen circumstances.
The whole Frankenstein's monster thing comes to mind.
If you must, attempt to create minor servitors first, without significant traits of personality, form, etc.
A servitor, is, essentially programmed energy. Give it no personality, no name, no identity, and it will not act independently. This is both good and bad. It cannot think, and therefor it cannot react to circumstances outside of your instructions. It is also therefore entirely without the ability to resent its servitude, appose the tasks you put it to, etc.
If you give it personality, true form, and a real identity unto itself it will resent its servitude as any true slave will. And it will struggle against you, and possible do you lasting harm as any potent spirit can potentially due during a summoning or other such ritual.
But, so far as permanent loss of your "magic," all things return with rest, meditation, and study. It is entirely possible to leave yourself "blocked" by pushing your untrained self too far.