This is a five-part survival course designed to give you essential information you need to succeed as an investor in the stock market. None of it is mere opinion.

Everything stated in these articles is backed by research conducted by well-known economists and published in leading journals of economics and finance. Much of it is information you are unlikely to get from a stock broker.

What I Found Out as a Professional Investor

For several years, as a Registered Investment Advisor with my own small investment firm, I managed the stock accounts of private investors. Some were corporate executives, others were entrepreneurs and several were artists. All were smart and successful.

Because mine was a small firm (I’ve recently retired), I was in close, frequent contact with each of my clients. Several remain my personal friends.

What I gradually came to learn from conversations and email exchanges with them was that many, despite their intelligence and proven ability to succeed in their own fields, were really terrible investors. Most of my clients had earlier made the kinds of costly mistakes that account for the almost shocking degree to which individual investors underperform the stock market.

Some of my clients had to be regularly talked out of making more bad investments or ignoring fundamental investment precepts like diversification — not putting all of a half-million-dollar portfolio, for instance, into a single historically successful stock, like Apple.

I’ll tell you more about why that’s a bad idea in the following four articles. In this article, I’d first like to quantify how badly retail investors underperform the market and then to summarize the reasons why this is so.

How Bad Is It?

In 2011, Brad Barber and Terrance Odean, two well-known University of California economists, published a detailed study, “The Behavior of Individual Investors” in the “Handbook of the Economics of Finance.”

Their findings have since been widely cited by other economists and by business writers in The Wall Street Journal and elsewhere. Here is the briefest summary of what Barber and Odean found: Individual investors:

  • Sell winning investments, which increases their income taxes, but keep their losing investments in the hope they’ll recover what they’ve lost;
  • Invest to feel good and avoid pain rather than in accordance with any known investment principle, investing heavily in companies near where they live and most heavily in companies where they work;
  • Do little if any research in the companies they buy, trading actively (often with two or more complete portfolio turnovers every year), which drives up transaction costs;
  • Are heavily influenced by popular media, buying into brief upswings generated by hype.
  • Ignore portfolio diversification and allocations suggested by CAPM, the capital asset pricing model used to control systemic risk and generate expected returns.

The results are as bad as this behavior suggests. Much of the underperformance of individual investors is due to trading costs, but even before trading costs are included in the result, individual investors substantially underperform the market month after month, year after year.

The most active individual investors underperform other retail investors who buy and hold their investments by more than a third. Overall, retail investors underperform the market returns that are less than 40 percent of what they would have earned simply by parking the money in an index fund and leaving it there.

Index funds make no attempt to pick winners and losers. They simply buy proportionate amounts of every equity in a broad investment area, such as the S&P 500.

What Accounts for This Behavior?

There are several reasons for this underperformance, but a few are especially significant. First, as Barber and Odean point out, most Americans don’t know very much about investing in the stock market, but think they do — they don’t know what they don’t know. As Donald Rumsfeld famously remarked, it’s the “unknown unknowns” that cause the most damage.

Hearing about the stock market in the news every day makes it seem familiar, making investors overconfident. But almost all televised stock market coverage is useless as a guide to investment.

Further, a majority of U.S. students never receive any personal finance instruction. Those that do usually take a single course covering insurance, credit cards, loans, compound interest and budgeting. Investing in the stock market is covered briefly or not at all. Few of them get into anything as basic as 401(k) or IRA retirement plans.

Compounding this problem is the difficulty that investors, particularly beginning and middle-income investors, have in accessing objective, low-cost investment advice. One of the really disappointing aspects of financial regulation in the U.S. is that for these investors, no one is reliably in their corner, providing them with the best possible investment advice.

Although few beginning investors know it when they open a brokerage account, their personal broker (if they have one — many now invest online through a discount broker and have no personal advisor) has no obligation to give them what is called “fiduciary advice” stock market advice “in their best interests.”

Most broker representatives do try to mediate between the interests of their brokerage, which will often want in-house investment brands pushed, their own interests, which depend upon how many buy and sell transactions they execute each month and their customer’s interests. These interests are unlikely to fully align.

As a result, few representatives act in customers’ best interests at all times, even if they want to. And, as we shall see later in this series, even reputable brokerages employ (and are often slow to dismiss) representatives who are ruinously and sometimes illegally bad.

Some financial advisors, such as Registered Investment Advisors, do have a fiduciary duty to their clients, and invest on their behalf with generally good results, but many only accept clients with investment portfolios of $500,000 or more; few will take on clients with less than $250,000 to invest.

Summary

The miserable returns of most individual investors result from inadequate financial educations and a retail investment environment where the least affluent and knowledgeable investors get the least good advice. Without guidance, most retail investors trade too often, make bad buying and selling decisions and underperform the stock market substantially, on average by more than 60 percent.

In the second episode of this five-part stock market survival course, I’ll outline essential aspects of how the stock market works — knowledge you must have to do substantially better.