Why Have an XP Budget?
Jerry, also known as @DreadGazebo, has a nice article on his site about Abandoning the XP Budget. While I don't agree with everything in his article, I do think people shouldn't start off thinking in terms of the XP budget. Why? Well, as he mentions it often puts us down the path of thinking about the mechanics rather than the story and plot. The latter is what the players are likely to remember, not "wow, you did a really good job there finding monsters whose XP fit into your budget." That said, I'd like to defend the XP budget a bit.
Puts You in the Ballpark
So why use it? For one, it serves as a good 20,000 foot view of how difficult an encounter might be. So many other factors go into whether or not an encounter is difficult for the players: player experience, party composition, character optimization, NPC group composition, monster synergies, random chance, etc. Let's not forget that the DMG also has encounter templates to use to address part of these issues as well.
Helps New DMs
Also, it's good for DMs, especially new ones, in a group that would rather play a "heroic" campaign than a competitive delve. Neither of these are better than the other, it just comes down to group preference. I know I loved the guidance when I started creating my first encounters as a new DM. I was so afraid of killing the PCs, especially since they said they really didn't want that to happen. Coloring within the lines for a bit gave me the confidence to start hacking around with the encounters. What happens if I throw them a n + 1 encounter? an n + 2? now how about a ton of minions? what about a bunch of soldiers? These experiments were instrumental to me learning the game well enough to write about it and produce content for it. Without guidelines like the XP budget and encounter templates, I wouldn't know where to begin and I might not have met my players' expectations for the game. It also keeps us honest. If we don't keep track of the encounter levels, when things start heading in one direction, we won't know why. If we never calculate it, we'd never know that what we thought were n + 4 encounters are really n + 8.
Finally, abandoning XP budgets, building multi-layered encounters, and many of the other things Jerry mentioned in his article require DM skill, not to mention time. And that's hard for someone writing a book to know in advance. I think that's why it's called the Dungeon Master's Guide. As DMs grow in skill, I hope they feel empowered to use and change the rules and guidelines to create the experience their table wants.
Thoughts for the Future
While I understand why Jerry wants to do away with the XP budget, I have a different proposal. Let's think about what the XP table would look like for groups playing different modes of the game, normal, medium, hard, and nightmare. What might the min and max levels be for an individual monster? What might the max level encounter be? The min? When should we break these expectations? Should traps and some terrain elements count in the XP budget? We get a taste of this in the encounter templates and in the Player's Strategy Guide, but it would be awesome to get more.