It's simple, really. The next time that he says something smartass to you, remind him that every cruel think he says makes you want to wait an additional week to have sex again. He'll quit soon enough.
You straight away ask why he infuriates you. You try to find out the reason. If you feel you are perfect, keep away from him temporarily. He will come closer.