It's awful that people who achieve something are often only remembered for who they are, rather than how hard they worked or what their achievements were, but Milk was definitely a victim of this. The first association people make with Milk is gay rights activist - even if people do know he was a politician.
Harvey Milk will always be known for his gay activism in San Francisco. That's what got him killed, unfortunately. Dan White was decidedly anti-gay in his stance. When Milk believed White's post shouldn't be reinstated, the conservative shot and killed Milk and the mayor in a single attack at City Hall. Milk was a martyr and will always be known for championing gay rights.
Harvey Milk will always be remembered first and foremost as a gay man and a gay activist more than a city politician because of his assassination, which as few may know, was actually largely unrelated to his homosexuality, but part of a jilted politician's nervous breakdown (whom Milk was not the only victim of, that day).
I would associate Harvey Milk's character with both politics and gay activism. By most accounts he was great politician and was in the business of politics to help out the little guy. I don't believe that he ignored his job in order to focus on gay activism, but I believe that it would be silly to say that he wasn't a great and early force for opening the eyes of people to the plight of the LGBT community .
I would associate Harvey Milk's character more with gay activism than with politics, because gay activism is what people know him for. He was openly gay, and would not have been, if he had not intended for his sexuality to be a political issue and a personal cause. In fact, his political activism resulted in a gay rights initiative passing in San Francisco.