Many people see the least reference to their color as prejudice. Society has conditioned many people to be ultra thin skinned when it comes to race. Though your question may have been sincere, it may be taken entirely wrong and end in a very big mess.
Another thing you should try to avoid is asking about things that are stereotypes, like asking a Japanese person if they are all good at math. Obviously, even if your just wondering sincerely, and even though it would be something more flattering, they would still peg you as being racist.
It depends on the intentions. If you're doing it with a smug intention that's bad. But if you really thought that blacks or Hispanics had a specific culture, you're not really in the wrong. You just don't know much about it. Most who ask questions like that are not ill-intentioned and truly wanna know. So I don't think we should ban these kinda questions. It's good to have good-spirited discussion.