If need be, you may be able to fire your ex-mistress. In one New York case, the guy decided to reconcile with his wife but she didn't want him to work with the other woman. He fired her, and a court threw out her sexual harassment case. (Mauro v. Orville, No. 84627, N.Y. App. Div. 3rd, 1999)
Taking an adverse employment action at the request of an individual motivated by a discriminatory purpose will get you in trouble. But this man won because the woman admitted the relationship had been consensual, and she wasn't fired because she refused to continue sexual relations. This court disagreed with a lower one, which had decided the action was, in a sense, based on sex.
Avoiding sexual liaisons in the workplace is your best bet.