A growing number of institutional investors are trying to codify their processes, procedures and even beliefs and turn them into investment checklists. Why? In a world plagued by uncertainty, some savvy investors have begun prioritizing their investment decision-making inputs (i.e., the successful execution of internal investment processes and protocols) over and above the outputs from their investment decisions (i.e., investment returns against a benchmark). This begs the question: Why take an input-oriented approach to assessing investment competency? The answer is simple: because the outputs are noisy. Market uncertainty, especially over the short and medium term, means that investment returns are not a reliable indicator of capability. Highly complex systems, such as the global economy and financial markets specifically, will inevitably have unexpected and unintended interactions among variables, thus creating unpredictable outcomes — what some refer to as unknown unknowns. And, so, rather than making performance assessments based on noisy returns — i.e., on luck — some funds are looking for ways to add assessments based on the pre-investment operations and their execution. The latter is often more visible and distinguishable.
Personally, I like the idea of prioritizing inputs over outputs. It prioritizes investment governance and management, while minimizing misaligned incentives that are often created by market-based benchmarks. Specifically, I think an input focus is a useful way to push back against short-termism and drive investors towards real long-term value creation. So, in short, I think this is a concept that has real merit. The question, however, is how an investor should actually go about prioritizing inputs over outputs. Moreover, how can an investment organization strike the right balance between inputs and outputs, as an input-only focus is probably just as misguided as an output-only focus? One of the problems with medicine today is that doctors are paid for inputs (number of tests, procedures, etc.) and not on the outcome (whether someone heals quickly, stays healthy). Similarly in education, teachers are paid for having a master’s degree, years on the job, etc. and often not on how much their students actually learn. So it’s about striking a balance.
For inspiration, some investment organizations have looked to other complex industries in which inputs — such as aviation, nuclear energy and medicine — are prioritized. And this is what has led some investment organizations to using operational checklists, like the kind used by pilots in a cockpit, to verify that investment professionals are ticking off the necessary inputs prior to an investment being consummated. It’s all very interesting. So let’s take a step back for a second and consider checklists.
In 2009, the book The Checklist Manifesto by Atul Gawande, brought the notion of inputs back onto the agenda of many executives. The Manifesto offered a thought-provoking look at the power of simple checklists to minimize mistakes and improve performance (especially in the medical and aviation sectors). As a result, checklists have come to play increasingly prominent roles in many different industries, including finance. According to Gawande: “A pre-investment checklist is great because it forces you to be dispassionate and systematic rather than getting carried away in the heat and enthusiasm of the moment.” And he’s right. I’ve actually seen quite a few investment checklists over the past few years. (As an example, check out this recent post and list on Value Walk: very interesting.)
Personally, I think checklists offer a useful tool for the burgeoning input-oriented investor community. But I also think this tool can be dangerous. Here’s why.
4 Reasons Why You Shouldn’t Use Checklists
1. No Flexibility. Systems that are designed to build redundancy and risk mitigation into everything will inevitably hamper an organization’s ability to be creative and dynamic. While flexibility and creativity are not necessarily something you want in the nuclear, aviation or medical industries (where checklists are used with great success), it is something you need to be successful in finance. Research shows that too much redundancy can create additional complexity and rigidity. (Check out this paper by Scott D. Sagan entitled, “The Problem of Redundancy Problem: Why More Nuclear Security Forces May Produce Less Nuclear Security”.) Also, the environment that a financial investment checklist is aimed at is always evolving. This is very different from an aircraft checklist. The structure and capabilities of the aircraft are fixed for all time and thus once you have completed the checklist (provided the aircraft is continuously maintained), you are done. In the investment case, you always have the nagging question about what may have changed that the checklist may not cover.
2. New Risks: By adding to the complexity of an organization — through additional fail-safe systems and protocols — the same people that are trying to minimize failures sometimes sow the seeds of new failures; it’s what theorists call “hidden failure modes”. A 1993 paper by NASA scientist Asaf Degani and University of Miami professor Earl Wiener on what they term “cockpit checklists” offers some very useful insights into the power of checklists in helping to prevent disasters ... but also in their potential to fail when improperly executed. “Recently, within a 25-month period, there were three major airline accidents in which the misuse of a checklist was determined by the National Transportation Safety Board (NTSB) to be one of the probable causes of accident,” write Degani and Wiener. As our surroundings become safer and more familiar, we become much more prone to risk-taking. As such, as investors develop a sense of security associated with box-ticking, they may be prone to moral hazard and take more risks.
3. Lost Information. As you attempt to codify the investment process and investing more generally, you invariably strip out some of the local context: the very insights that often make investments successful. In addition, you can miss some fundamental risks by trying to codify too much. For example, how many banks used (and still use) flawed checklists when trying to assess individuals for mortgages? (Note: This is why most of my academic writing often speaks about principles and policies rather than practices. My hope is that the former, due to their high levels of abstraction, are more generalizable and useful than the latter, which, if applied universally, will often come across as overly prescriptive.) This leads to my next point.
4. Missed Opportunities. An overly rigid checklist can prevent an organization from doing something that would otherwise seem quite the obvious thing to do. As an anecdote, picture this scene that I had the pleasure of experiencing first-hand when I was trying to get a mortgage from a big American bank.
Banker: Do you have a car?
Me: Yes.
Banker: So why don’t you have a car loan?
Me: Because I own my car.
Banker: OK. I also see here you only have one credit card. Why is that?
Me: Because my credit card has a high enough limit that I only need one. And I like to get miles on that card.
Banker: Unfortunately, because you do not have two credit cards in good standing, I can’t give you a mortgage. I can see here that you’re an ideal candidate for a mortgage ... but I just can’t do it without your having a second credit card. My advice to you is that you go get a second credit card and buy some things and then pay it off. Then come back to us...
As you can imagine, I never came back. What I ended up doing was going to a local bank that had much more flexibility and agility, a bank that wasn’t limited by a checklist.
In short, all these checklists and redundant systems should come with a health warning: They don’t always work and, if executed poorly, can actually make organizations less reliable and less successful. Checklists need to be very well-thought-out.
Now I must confess something: I use checklists all the time. Moreover, I helped a sovereign fund develop checklists for direct investing in certain alternative investments. Here’s how checklists could benefit investors.
4 Reasons Why You Still Should Use Checklists
1. Fewer Errors. Checklists won’t provide investment organizations with a codified mechanism for becoming alpha-generating superstars (though, there are some VCs that have taken to trying to quantify as much of VC investing as possible). Rather, checklists offer investors a useful opportunity to avoid making silly mistakes. Research seems to show that checklists are very good at defeating human- and machine-based weaknesses before they can derail an operation. And no one can deny that investors are overwhelmingly irrational in their investment decision-making. So a checklist can be useful in countering individuals’ biases, and thus mistakes.
2. More Clarity. A well-crafted checklist can provide investors with quite a bit of clarity about their roles and responsibilities and communicate this information to all of the relevant team members. This creates accountability and also helps the team members to be thorough. (This is similar to the reason why Bridgewater films everybody in all meetings all the time ... but it’s less creepy). Clarity of mission and roles is one of the touchstones of good governance, so I’m all for tools that help to create this.
3. Deploying Knowledge: Gawande argues that, due to specialization, individuals today hold too much knowledge to make it possible for any single person to adequately deploy it all in a timely fashion. Checklists thus foster discipline around the fundamentals as well as ensure that the appropriate knowledge and expertise is being applied to specific circumstances in a pro-active manner.
4. Long-Termism. One of my over-arching objectives as an academic is to provide long-term investors with tools that can help them position capital for the long term. Far too many investment organizations that are ostensibly long-term revert back to short-termism due to benchmarks and short-term pressures. By using tools like checklists, an investor can revisit some of the fundamentals of long-term investing and make them a priority.
The Takeaway
My over-arching lesson is this: These are valuable tools that can, and often do, improve performance in complex environments. More to the point, I see in checklists a potentially powerful tool to help investment organizations take a step back from market-based metrics of performance evaluation and instead focus in on the factors that could truly add value over the long term (inputs over outputs). As such, long-term investors really should spend some time thinking about how checklists could be deployed. However, due to the potential dangers, my advice is to think of these systems as “cues for introspection” rather than binary yes-no checklists. I think it’s important to maintain the authority of the local agent in the investment process. For example, do we want people walking into an investment committee saying, This investment ticks all the boxes? I think the answer to that is no.
Let me end this long post with some random thoughts: Chaos has to do with the unpredictable change in outputs due to the minute changes in the inputs of a physical or simulated system; think of this as the wings of a single butterfly beating in the Amazon being the precursor to a hurricane in Florida. Sometimes the evolution of the chaos is so incremental and takes place over so long a period that observers get addled into believing the system is controllable when in fact they don’t truly understand it and just use long polynomials to predict where it will go next (think earthquakes prior to the discovery of plate tectonics). In my view, this phenomenon is ever-present in multi-layered financial systems for which no one person has the capacity to understand all the elements of, and yet many believe that they are personally capable of generating alpha. As such, pushing investors to think more about inputs (without forgetting the outputs altogether) is valuable and, in fact, I think it could help to extend the time horizon of investors. It’s in this respect that I’m interested in a set of thoughtful and smart checklists.