Yes, Andrew Jackson was a cold-hearted and brutal leader. One need not look further than the catastrophic "Indian Removal Act" for evidence. This act allowed for the forceful relocation of Native Americans who had already had to deal with a painful campaign of genocide, and lead to the creation of the infamous "Trail of Tears."
It is always easy for people, generations removed from a historical figure to judge that person through their worldview. Although some historical figures will probably stand the test of time as despicable that is probably not the case with Andrew Jackson. The fact that he was widely popular in his day and elected President is a good indicator that he was a reflection of his time and an exemplar of the worldview of his contemporaries. If he is just a reflection of his time, then the vast majority of his contemporaries have to be painted with the same brush.
He's known as "the people's president" so even if he was cold hearted, historically speaking, he's got a decent reputation. However, he was responsible for the Trail of Tears and for multiple campaigns to force Cherokees, Choctaws, Muscogee-Creeks, Chickasaws, and Seminoles off their lands, which resulted in many deaths, and is arguably brutal and cold-hearted.
While Andrew Jackson did things we would not approve of today, such as his treatment of Native Americans, we can only judge him through a historical lens. Any leader makes unpopular, and at times wrong, choices and for those we can judge him. But to say he was cold hearted is to not take into consideration of his good deeds and even his reasoning (and dated knowledge) in regards to some of his decisions.