Why does a spouse having sexual relations with another immediately denote that the marriage is breaking or falling apart? Many marriages can be extraordinarily successful when one or both partners have occasional sexual relations with other people.
Just because you can not satisfy all of your husband's sexual needs does not mean he wants to end the marriage or leave you (or that you are doing anything wrong). In fact, he might be prolonging the marriage. If your husband was not getting his sexual needs met, then he might of decided that divorce was the best option to resolve his pent up frustration and had divorced you already. The fact that you haven't divorced yet means that your husband wants to maintain your family.
If it bothers you that he is having relations with another, try sitting down with him and talking it out. Ask him if there is anything that you can do to satisfy his sexual needs, whether that's trying new things, adjusting your appearance, being more flirty, etc. Trying to cut him off from his sexual partner will only cause a rift between the two of you, and will liken the chances of a divorce.
However, if the situation does not improve to your standards and you find yourself still upset, then perhaps the relationship wasn't meant to be. You and your husband might share different values, and have different ideas on what constitutes a marriage. If you think it will be helpful, bring up the topic of divorce and explore your options.