Yes, the federal government's ability to create new regulations has been spiraling out of control for decades. There are so many laws on the books currently, and more being written every day, that it is humanly impossible to know them all. Thus, everyone is exposed to possibly breaking laws they don't even know exist. Reducing the amount of regulation is a sensible way to stem this tide.
In part because Congressmen work on behalf of their home districts as well as the country as a whole, the number of regulations working in the law grows by leaps and bounds every year. Although many are enacted for only very short periods, many more remain in force long after the issue they address has been resolved, mostly because of poor oversight. Doing away with unnecessary regulations and repealing ones that were enacted wrongly or spuriously can help the government at all levels make better use of our tax dollars.
A country's economy can literally choke to death if there are too many regulations. In the United States, it has become hard to start a business and it has become hard to know what regulations are out there that you need to comply with. This is a set in the right direction.
Trump's executive orders are turning the country into a mockery of itself. He is tying the hands of people so they cannot do anything without his input and consent, making himself way more powerful than anyone else and eliminating many of the checks and balances we take for granted in the US.