Doesn't seem to have done too much damage, if any at all. Most Americans probably view this as a very human response in the midst of a heated debate. Certainly not very statesmanlike, but he's certainly not the only politician to use such language when referring or even speaking to a distinguished colleague. The only difference is that this one seems to be much more publicized.
