Yes, I think that the decision of planets and how they're classified should indeed be restricted to astronomers. After all, such things are in their domain of study, and therefore their authority. If we left it up to more official government bodies, the entire thing would get bogged down in process and committee.
Astronomers are clearly the experts in the field of planet classification. This is what they study and what they do. They classify planets based on scientific data, and whether or not a planet matches a certain set of criteria. Who better to judge whether or not a planet meets this set of criteria than people who dedicate their lives to researching the universe?
Astronomy is the subject which studies extraterrestrial phenomenon and classifies them accordingly. It is appropriate for astronomers to classify their terms just as much as it is appropriate in any other field of study to control definitions of their terms. The public at large may want to change definitions of planets and astronomers will not beat down their doors and hurt them for doing so, but there is no reason to take away field-specific terms from that field of study.
A simple way to look at this is by specialization of careers. You wouldn't want a dentist to diagnose a medical issue even though they are both doctors. The group that knows the most about a subject should have full rule on it. Some might argue that physicists should have a say, but what does a physicist know about a planet that an astronomer would not? Astronomers have the best idea as to what characteristics best describe planets.
Astronomers are the experts when it comes to planet classification, so I feel those classifications should be left up to them. I believe it is important to leave such decisions up to the professionals because they are the people who devoted their time to the subject and they are the ones who are more knowledgeable about the topic.