Growing up, I was actually a straight A student, something I was actually very proud of, especially when it came to PE.
I was an academic student, not a athletic one, and so whilst I didn't mind doing the actual lessons (although I would rather sit through a science lecture) I hated that I could try and try and there was no really way I could improve and bring up my grade. I had to work so hard to maintain my grade point average, had to do extra so that my PE grade would not affect my overalls grades too much.
Not every kid has athletic ability, and whilst it can be a fun subject, and does promote teamwork and fitness, it shouldn't have the ability to bring down a students grade so much, simply because they can't run as fast or can't kick a goal like the other kids.
PE should be a subject in schools, but not one that they are graded on, it should be more like an extra curricular activity.