Freedom lovers everywhere are biting their nails this election season, wondering how the damage can be limited. Depending on who gains control, we could have trade wars, nationalized health care, the pillaging of Wall Street, more war in the Middle East, a VAT tax, direct surveillance of your smartphone, internment camps, mass deportations, and so on.

But let’s take a step back and ask whether it has to be this way. What if the power of government were so limited that it didn’t matter who occupied the White House? This would be a vast improvement.

Let’s say that Rutherford B. Hayes, who was president from 1877-1881, was revealed to be a fascist demagogue and bearded would-be dictator. Maybe the same could be said of Presidents Chester A. Arthur (who ruled from 1881-1885), James Garfield (1881), or Benjamin Harrison (1889-1893). Let’s say they were all crazy authoritarians who longed to rule the country as their private fiefdoms.

Would it really matter? Probably not.

They had no large bureaucracy to control. There was no CIA, NSA, FBI, HUD, DHS, DOL, and so on. They didn’t exist and their functions didn’t exist. The Supreme Court didn’t do much. There was no IRS they could lean on to persecute their enemies. Surveillance of the population wasn’t yet possible. The government owned no weapons of mass destruction.

There was no central bank to bail out their wars and welfare. In fact, they had to balance the budget year to year (same as states now) because the country was on a strict gold standard. You couldn’t just print money without limit. The military was tiny. There were no migration controls, taxes, or even passports.

There was no federal government involvement in education, health care, or commerce generally. There was no antitrust regulation, no social security tax, no regulation of consumer products, no environmental land management, no price controls or labor laws that stand between workers and employers, no drug war, no decades-long process of pharmaceutical testing, no gun-free zones, no giant military contacts, and no ability to tax earnings.

Most of the power presidents had amounted to steering some infrastructure contracts to their friends. And here, their corruption was truly revealed, but the damage they could do was limited. Their play money came from some small tariffs. They were caretakers of a limited government that didn’t intrude into any intimate aspects of life. The governments they headed had strict limits on what they could do. They had no policy plans to speak of because policy as we know it barely existed.

Leviathan as we know it had not been invented yet. That all came later in the 20th century. Whatever the private ambitions of Gilded Age presidents, they couldn’t realized through their official capacity. Therefore, the stakes were extremely low for the country at large. This is why these men’s names are barely known, and, even then, hardly anyone paid much attention to the presidency as such. They were caretakers holding an honorary position of interest to only those directly affected.

As bad as the candidates are this year — as threatening as each of them are to someone’s rights and liberties — none of this would be a problem if the power to act on personal ambition were not a possibility. This is what limited government means: no matter how bad or dangerous a person is who holds office, he or she has no access to the tools necessary to spread damage around the population.

This is a much better system. It is how power over the population is limited, not at the discretion of a “good guy” who wins but rather because the institutions he or she controls cannot be used as tools of oppression.

There is a sense, then, that when we talk about how grim the policies or a Trump or Sanders or Rubio or Hilary or whomever are, we are not getting to the core of the problem. It should not be the case that anyone should have to worry so much about the character of the person we elect. A good system of government is one that is protected against control by wicked people. It should even be protected against good people who want to use state power to realize their ideals. Government should be a structure that is impervious to the personal ambitions of its temporary managers.

Under such a system, there is not much point to an active group of hysterics from the right and the left, people demanding that power be used for this group and against this group. You can scream all you want but it has no more effect than yelling at the paint on the wall to change color. This is what it means to live under rules rather than the arbitrary dictate of human ambition.

Blaming those who are currently demanding crazy, scary, destructive policies misses the core point. The real blame should go to the generation who 100 years ago overthrew a system of laissez faire and replaced it with a planning state with the power to run our lives, take our income, redistribute wealth, manage the industrial sector, enter into unlimited military conflicts, create giant financial bubbles and bail out failing industries.

Power once created will be used. That the special interests and then the masses clamor for it to be used on their behalf is the inevitable result. With power also comes a divided population, people seething with hatred against those who stand in their way and interest groups consumed by loathing of anyone who stands a chance of using power to their own advantage. The presence of power itself, not the people who seek to turn it to their advantage, is the source of conflict. And such a conflict threatens to destroy friendships and even the social fabric itself. The overweening government itself is the reason that we all can’t get along.

So think of it. Most of the people who created this mess are long dead but they still rule us. The bequeathed us a monster that the present generation must contend with. Given that, there is really only one responsible way forward: dismantle leviathan before it destroys us all.