The other day someone drop a pound, I left it, my friends didn't. I said someone could come back for it but they said its only a pound. Anyway 2 minutes later a boy came around our area and was saying has anyone seen a pound, and they said no. His face looked so dejected, it was horrible and you could see my friends felt guilty, but didn't give the pound to him still.
From a pound to £20 or whatever currency your from, money is important and I would leave any amount. A penny I wouldn't because its meant to be good luck "See a penny, pick it up and all day long you'll have good luck".
A $20 bill is quite a lot ( for me ) and it would be one of my first thoughts to return the money if I ever see something more then $5. In fact in Australia is is illegal to take money off the ground which is worth more then $5, it is not much of a strict rule as not much people no of it and no one would ever be able to punish them if you had dropped it and come back to it gone.