If evidence is found against a criminal it should not matter how the evidence was obtained, the crime was still committed and the person is still guilty of the crime. A criminal should be incarcerated using any means necessary as long as the evidence proves the criminal guilty. Evidence is still evidence even if it was illegally obtained, it still proves the criminal was doing something against the law.
If a criminal commits a crime, they should be convicted based on the evidence obtained legally, not illegally. This-in my view-is fundamental to the preservation of our 4th Amendment rights.
Keep in mind that our legal and justice system has alternative provisions for the use of illegally obtained evidence.
For example, in some states illegally obtained evidence is admissible in sentence hearings, and the aspect of it being discovered and dismissed may also have dire consequences for a defendant.
In Florida, the police illegally searched the house of a man who was accused of possessing child pornography. The evidence was dismissed however the story was published by the local news media. Although this suspect evaded prosecution, he eventually was fired by his employer and endured enormous harassment from the neighborhood.