This is actually like saying "There is still slavery and child labor in this world. But I guess I should not care about this at all because it's not my problem and it won't affect me." This is unfair because another person might really need help, and if you don't care, it's your fault.
"Not my problem" is an attitude that has it's place in society. Sometimes we have so much on our plate that we can't handle anymore. If someone is struggling to feed their own family and has lost their job, they shouldn't be required to give money away to the poor as they desperately need it.