The only person at your holiday lunch who demonstrated a lack of intelligence was the person who insulted you. In fact, by letting it go at the time, you demonstrated a high level of restraint, self-respect, and class. Had you lashed back at this individual, all you would have done is drop to his level.
Since the luncheon is over but his comments are still turning your stomach, you should say something to him. However, you need to know at the outset that a person who is foolish enough to make that kind of a comment is also foolish enough to refuse to listen to others. Nonetheless, if it makes you feel better, you can tell him how you feel about the comment, and ask him how he would feel if he had been on the receiving end.
More importantly, keep the big picture in mind: he made a fool out of himself in front of your manager, and you demonstrated to your manager that you are a quality person. If your manager is sharp, he or she will have already spoken to this employee. After all, if your co-worker acts like this during a holiday luncheon, how does he act on a day-to-day basis? If his behavior persists, he may be a candidate for a permanent holiday. | 
