Making a poor decision is not illegal. Courts have held time and time again that they do not sit as "super-personnel departments" and judge whether the decision to fire an employee was a good, right, or smart one. They really don't care if you make a bad, wrong, or dumb decision. What they do care about is whether the decision is discriminatory or retaliatory. If you make a questionable termination decision, you'd better have an honest and reasonable belief in the reason behind it.
Courts allow what is traditionally known as the honest-belief defense. In short, an employer can show that it did not discriminate or retaliate against an employee when it fired him/her by asserting that it had an honest, albeit mistaken, belief in the validity of the decision. That's all it usually takes to defeat a discrimination or retaliation claim: So long as you honestly believe in the reason, the employee cannot prove pretext, even if the reason is shown in the end to be "mistaken, foolish, trivial, or baseless" (according to the 7th Circuit Court of Appeals).
However, that might not be enough. At least one other circuit court (the 6th) has added that an employer's honest belief must be reasonably based on particularized facts that were available to it at the time the decision was made. In the case before the 6th Circuit, an employee was fired for being a no-call/no-show for three consecutive days after a doctor cleared him to return to work. The employee claimed he had missed only two days and that the company fired him in retaliation for filing a discrimination charge.
According to the court, the employer did not establish a reasonable reliance on the facts at hand. Fact was, the doctor had released the employee to return to work the afternoon of September 24, so the first full day he was available for work was September 25; the employer sent a termination letter dated September 26. While the employee has not won his claim, he did at least win his day in court.
Employment at-will allows you to fire an employee without having a good reason, but that doesn't mean you should press your legal luck. It's best to have reasonable evidence supporting your honest belief that terminating the employee was appropriate.
Finally, don't put a chink in your own legal armor by gathering the evidence and then failing to be honest with the employee. It may seem easier to sugarcoat the reason in order to spare the employee's feelings or to spare yourself from a nasty confrontation. Lying will raise suspicions of illegal intent and make it harder for you to prove that you acted legally.