The surprising truth behind why the experts can’t beat the stock market
In my last article, I talked a little bit about how bonds were a lot like dairy cows. And in today’s article, I’ll continue to on that cow theme by telling you about a man named Francis Galton who lived in England about 100 years ago. He was the kind of person who believed in experts because he figured they knew things that ordinary people just didn’t know. After all, that’s what experts do, right? Of course, that’s what makes them experts.
Well, one day Galton goes to a country fair and there's a contest going on where people are guessing the weight of an ox. Being a scientist and a statistician, Galton figures he could do a little experiment. He figures, I'm going to take everyone's guesses, calculate the average, and then compare that to the actual weight of the ox.
He thought what he was going to end up with was a really flawed guess because in his mind, what he was doing was taking guesses of a few smart people, a few mediocre people and then a lot of not so smart people - because he basically thought everyone was dumb. So he figured the group's guess was going to be way, way off the mark.
The contest organizers gave Galton the little slips of paper with everyone's guesses on them and he took them so he could calculate the average guess of the group. The average was 1,197 pounds.
And the ox?
According to James Surowiecki who wrote a story about this in his book The Wisdom Of Crowds, the ox actually weighed 1,198 pounds, just one pound off of the average people had guessed.
Much to Galton’s surprise, the crowd's judgment was essentially perfect! It was like there was some kind of collective unconscious magic going on – or was it?
We are greater together After observing this at the fair, Galton and other scientist began studying this phenomenon and found that it is actually everywhere in the world around us all the time. In fact, it's the underlying idea behind the stock market where thousands of random people are buying and selling shares of stocks, guessing what the actual price should be in their opinion. That average determines the actual prices. And Nobel prize winning economist Eugene Fama has argued that the collective price is actually very close to the correct price based on all of the information available in the market at any given time. In other words, when you hear that Apple's stock went up or that the Dow plunged, that's basically people guessing the weight of an ox.
A lot of things, from the price of oil to the price of orange juice - plus a lot of other things that are really important to the world, work exactly this same way. But why should it work? I mean why should a bunch of random people - a lot of whom have no idea what they're doing - somehow magically produce an answer that makes sense? Does this really work?
And if it does, why does it work?
Well, recently, a group of journalist repeated Galton’s experiment at a fair with a cow. They weighed the cow on a big scale and then had random people guess how much the cow weighed. They didn't want to just limit it to people at the fair, so they also took some pictures of the cow and posted them online asking the whole world to guess too. One of the journalist even stood by the cow so people could get some sense of scale in the picture, since they we’re actually at the fair to see the cow in person.
Kids guessed wild random numbers and some kids just repeated the number of other kids they had heard guess before them. Maybe it was just a “herd effect”… (Okay, sorry about that, I couldn’t resist.)
Back to the cow. According to their scale, Penelope their cow weighed 1,355 pounds.
As the guesses came in from online and from the people at the fair, the journalists added up all the guesses - over 17,200 in total (which is basically similar to the number of people in a small town). Everyone was wondering what the average was going to be and if the crowd was going to get it right.
So when the voting ended, they calculated the average of all the guesses. Now remember, the cow actually weighed 1,355 pounds.
And the crowd’s collective guess was …wait for it…1,287 pounds. The crowd was only 68 pounds off, which is a difference of only about 5%.
The expert (un)advantage And here's another amazing thing. This result was figured from a bunch of random people looking at this little cow picture in their Facebook feed on their iPhone, right? Well, to get a sense of who was responding to the cow’s weight question, the journalists also asked people another question. They asked, “Are you an expert?” or basically, “Have you ever worked with cows?”
Remember, Francis Galton thought experts might be better than ordinary people at things like this, and in this survey over 3,000 people answered “yes” to that question of whether they were cow experts or not. It turns out that a lot of them were farmers or ranchers, so by all measures they should have been experts when it came to knowing about cattle.
Okay then, how well did the experts do in their guesses? Surprisingly, their guesses were off the mark by 15 pounds more than the collective average. In other words, the experts were worse than everyone else!
And that’s why it’s hard to find an expert who can beat the stock market consistently. The wisdom of the crowds is a very real thing and chasing the expert expecting to get superior results is a mistake. Financial markets are usually very efficient, and while there is a slight margin of error, like the 5% difference I mentioned in the cow experience, the important thing to note is that the experts don’t usually perform any better than the collective masses in their guesses.
Because the idea is so counterintuitive to most folks, it's pretty extraordinary to digest the power a group has to arrive at really good decisions. There's something almost eerie about it. But, while this sort of thing can seem magical, it’s not magic. It's just math.
The Mastermind Surowiecki says the reason this phenomenon seems to work when guessing the weight of a cow and when guessing the price of an investment is that every person's guess is contributing some new little piece of information. Everybody is different. Everybody thinks slightly differently when they're trying to guess the cow's weight or the price of a particular company’s stock.
Maybe one person studies things from a particular angle while others are trying to compare the thing they are guessing about to something else they already know, as a point of reference. In that way, every person's guess in some way reflects their specific life experience of judging the size or value of things in the world from just decades of living. It's like every person's mind is a different scale for weighing the cow or evaluating the value of an investment option.
Naturally, everyone’s personal “scale” is likely to be a little off, which will make them all a little bit wrong in their calculations. But one’s person’s error will most likely cancel out another person’s error, leaving all those little bits of information and ultimately influencing the final result, which is usually amazingly good.
Of course, there are times when the wisdom of the crowds does not work, like when the stock market panics or bubbles. There's actually a technical term for this too. It's called “information cascade.” That like when your friend tells you he just bought a new stock position and you look at it and think, “Man, that seems like a lot of money to pay for a stock like that. But my friend has done fine with his picks in the past. Everybody else who’s buying it seems to be doing fine too. It must be a good stock. So, I guess I should buy it too.”
That’s a bad way to make decisions, my friend.
The stock market is not perfect, but what’s amazing about the stock market is that individual investors, some of which are very good at investing, are oftentimes irrational. They have tiny bits of information and they're making their decisions based on emotion, a hunch, or on some tip they got. And yet, collectively, we trust them to set the value of all of these companies.
It's kind of scary when you think about it, but the interesting thing is I don't think there's a better way to do it. And there is not a more effective way of doing it either. Their collective wisdom routinely outperforms the self-proclaimed expertise of most so-called financial experts, including those who are sometimes just plain lucky.
Like I mentioned back in a previous article where I talked about the story of Lester Moore, you can guess correctly from time to time and feel like you have a special gift for making the right calls. But, over time, mathematical probabilities take over and leave you in the dust – making you average at best, or at least you hope.
The stock market is not all that different from a contest asking people to guess the number of jelly beans in a jar. No matter how many times you see something like that or follow the market’s returns compared to most individual investor’s returns, you can think to yourself that it’s not going to work this time. Yet, time and time again, you’ll almost always find that the crowd’s average guess usually comes in within 3 to 5 percent of the actual value.
You can’t beat Buffett Try to beat the market if you want. Find the fanciest expert or most sensational financial strategy you can find. And then also put some of your money in the boring old averages by owning the market index too. Chances are very good that the crowd’s wisdom you’ll see reflected in the index will leave all the others behind.
Warren Buffett famously did this recently with a 10-year $1,000,000 bet placing a hedge fund manager against the S&P 500 index. The index outpaced the fund by such a wide margin that the fund manager threw in the towel years before the bet came due. Looks like the billionaire who knows a thing or two about the wisdom of the crowd proved his point and proved that even the smartest of experts is no match for the crowd’s collective insights over time.
Why make things harder than they have to be? Those Nobel Prizes don’t come easy, you know. Use the wisdom of those who have come before us to improve your financial results. And, while my crystal ball doesn’t work any better than anybody else’s when it comes to making investment forecasts, I can help you build a plan that incorporates sound principles of financial management, including building an investment portfolio that takes advantage of the principles I’ve shared with you today.
If you’d like some help bringing your financial goals within reach or even just a second opinion on whatever plan you have in place now, let me know. I’ll even take a guess at how much your cows weigh if you’d like. Just get in touch and let me know how I can help.