The Strawberry Letter: My Marriage or My Pleasure

Dear Steve and Shirley, I've been married for almost 20 years and my husband has put me in a difficult position. When we first got together, we were swingers for a few years. We only did it when we were on vacation and it was only with the couples we traveled with. We used to have an occasional threesome so my husband would be happy. He would not let me bring another man into the threesome, so he let me go have sex with an old friend of mine a few times over the years. But lately, my husband doesn't want to have sex at all. He said he's had enough wild sexual experiences to last a lifetime. It's not a medical thing because his stuff is still working properly. It's been 8 months since he's touched me, and on top of that, he's changed the rules of our marriage. He told me that if he suspects I'm hooking up with my guy on the side, he will leave me. I do not like ultimatums. So, I told him that I need to be pleasured, and since he's changed his outlook on intimacy, I have to handle business elsewhere. Two days ago after work, I went to meet my side guy for drinks. We planned to have sex after drinks, so we started fooling around at the bar. I couldn't wait to give him a nice long kiss. As we were kissing, my husband walked up and asked me to leave with him. I left but I did not talk to him all the way home. He has threatened to divorce me and he said he will say I cheated on him in the paperwork. How can he do that when we've we've cheated on each other and with each other? All of a sudden he is a model husband and he's forgotten that he introduced me to this lifestyle. I need to stay married because we have too many assets together, but I need the D too. What should I do here?