"Giving every man a vote has no more made men wise and free than Christianity has made them good."
Taking it away doesn't make them any wiser or better.
People rail against the "lowest common denominator", but the fact is that evil people are just as intelligent as those who are good. Good people just want to be left alone, and live in peace. Evil people choose domination of their fellow man. And to do that, they'll organize.
So good people need to organize too.
That's why most of the founders considered government as a necessary evil.
People were born to compete. It's God's gift to us, and it is how we make ourselves better. Inspiration is the breath of God (which is where I belive the words comes from). People look to top the prior achievement of those who came before them. It's something that should be embraced.
There are those who seek to use that God-given ambition for their own, utterly selfish ends.
War is the ultimate form of competition. Few if any wars have been won where the winner never fought the war on the opposing side's territory. So it is that if you want to keep the good things you have - liberty and such - you need to confront those who would seek to take these things away from you on their own level.
Otherwise, you'll either get lucky, or be doomed.
Two-hundred-plus years have shown us that representitive democracies work, and work well.