If Mr. Potter is using his old grandaddy's wand from the attic, Then he wouldn't stand a chance but if he had the elder wand. Things could be different. He'd take out Spider man, Black Widow, And Bucky Barnes no problem and he could shrink the hulk to the size of Ant-Man but he'll have his work cut out when he faces Iron Man and Cap. Marvel. Plus Scarlet Witch, Vision, And Dr. Strange have their own version of bending time and reality.
Sure the wizards could beat captain America and Clint black widow and maybe Bucky.
However Spider man has his peter tingle aka spider sense and was able to beat the master of illusion. He would move to fast for them and could snatch thier wands
If quicksilver was alive he would snatch their wands.
Plus doctor strange is the most powerful magic user but you are also forgetting about Thor who could wield lightning
Point is that the wizards would beat some but they are all too dependent on their wands and the marvel heroes are varied and would solve the problem.