He probably can, actually. It's more of a political problem than an administrative one.
Discussion
If given the authority, I don't doubt that Elon could cut a ton. I could cut a ton easily. There are some departments, like the Department of Education, that could be cut today and nobody would notice except the employees of that department and some local bureaucrats who paper-push to comply with them. The problem is will the House and Senate allow the cuts and get behind him.
This article has a section on Elon Musk and his motivations for all that he does which was very interesting and I think explains so many of the contradictions I see in him. The whole article is worth reading, but you can also skip to the subheading on Musk.