That's the ego speaking. You identify him as yours. When we speak of the ego we speak of for ourselves and only ourselves.
Your not thinking about him, your thinking of you. You are thinking of me, myself, and I.
I need HIM to make ME happy.
I need HIM to make MY feelings go away.
I need HIM to make ME feel complete.
When we truly love the other. We forget about our own feelings and our own self for the needs and desires of the other person. If you truly love him, you would have let him go, since letting go is probably what HE WANTED.