On this site you can leave a coven by going to your coven's page. At the top left where it says "accepting applications," that will change to "change status" once you are a member and you can use that to leave a coven as you wish.
In real life you can also typically leave a coven at just about any point, though you should check with whomever runs the coven for specific instructions. Some covens will want to perform a ritual or ceremony to recognize your leaving and/or to sever specific magical bonds.
While you can leave an initiatory order, also generally at any point, you cannot un-do the initiation and any initiatory oaths you took still apply. I've heard tales of a member of the O.T.O. who went back on the oath taken in the 1st degree, and while I cannot go into the specific nature of this oath, his death occurred after doing what he swore not to do, his death was painful and incredibly improbable, and it perfectly mirrored how the "on pain of (blank)" part of the oath is phrased.
Don't mess with magical oaths.