I believe it is fair to let people label themselves as they see fit. Of course he was a comedian, and actor, but the description of "song and dance man" fits as well. Andy Kaufman was an interesting character who rarely played by the rules, so it seems fitting that he would label himself different than others do.
Yes, I think Andy Kaufman's characterization of himself as a song and dance man, rather than a comedian, is fair and accurate, because he often said that he did not consider himself a comedian. Comedy was not the message that he wanted to convey. He thought he told his stories through his songs and physical comedy. It is appropriate to remember him that way.
Who are we to question what Kaufman thought of himself? If he wanted to be considered a song and dance man, as he clearly thought of himself at the time, than it makes sense for people to agree with him. Who better than the primary source to characterize him at the time?
Andy Kaufman was probably kidding when he characterized himself as a "song and dance man." A song and dance man was someone like a Fred Astaire or Bob Hope-- in other words, old style vaudevillians. Kaufman was the completely opposite of that, an experimental, surreal comedian and character actor who liked to push the envelope in comedy by pulling pranks, antagonizing the audience, and playing bizarre characters. Chances are that if he described himself as a song and dance man, he was being his usual ironic self.
Andy Kaufman was the consummate comedian, even if he could sing and dance and perform in other ways. And, like many comics, he laughed at himself which made his comedy so effective. But his tragic ending shows that, as with many other comics, he was hiding a deeper side of himself that wasn't so happy.