I believe that since familiars have such a close tie to us, they come back to us. My familiar followed my mother and I home when I was four years old, he was a stray kitten. He was a carrier for feline aids, but lived a long life free of the disease. He died when I was 17. A litter of kittens was born by a stray in my backyard when I was 19. I caught one of the kittens when they were 6 weeks old but unfortunately his mother had feline leukemia. He died when I was 21 of the disease his mother gave him, we had hoped it wouldn't become active but it did. Before he died, when he became ill, he began to display the same behavior of my old cat, and I recognized in his eyes that it was the same soul. I put him down to prevent him from suffering severely, cancer had developed in his stomach. A chihuahua showed up at my doorstep, 2 years later at 5mo old. He had no tags or chip. Like the others he quickly became attached to me. When he first saw me, he wagged his tail and came to me as if he recognized me. Thankfully he is very healthy and for that I'm grateful. He acts just like a cat. He chases string, laser pointers, and hates water/baths. He begs for milk and tries to rub his body on tuna fish like its an exotic perfume (I had to stop treating him to it because he would wear it instead of eat it). He even plays with toys by himself by tossing them and chasing after them. I don't think he realizes he isn't a cat.
So I am certain they come back ;-)