Yes, Brown is ruined by the Recovery's failure, because the way British politics works, politicians rise and fall quickly. In England, if a politician does not show almost immediate results, they are replaced. The Prime Minister is particularly responsible to the people and to the House of Parliament for results. Voters will not give Brown enough time to show that he is a responsible and effective leader. He will be defined by the Recovery.
The failure of the economic recovery will doom Gordon Brown's Labour Party much like Republicans took a huge hit in the 2008 election because of the Great Recession. Europe's problems are manyfold and won't go away soon. The lumbering giant saw austerity issues, even in Britain with young people's protests, so Europe's problems are Britain's problems. Unfortunately, the United Kingdom's populace only has so much patience.
Did the recovery fail? The UK is doing better than other countries in the world, and austerity measures have been shown through study to have been effective, even as they are incredibly unpopular. History will vindicate Brown regardless of whether people do not reach that conclusion today, but it won't be his end.
It may be hard to say for sure if he is ruined but If I had to choose I would choose no because he might not have been effected as much as people say but then again I am not sure what could have happened if it did not fail.
While Brown committed many resources to a recovery, the recovery's failure doesn't doom him. Brown can still make contributions to the community and be a positive influence there. He isn't ruined by any means by the failure of the recovery. However, it does hurt his image to some degree now.