Putting Disclosure to the Test: User Comprehension Requirements

Given the limitations of Disclosure 2.0 and Disclosure 2.5 I described in my last posts, what is to be done? To answer this question, we might first ask what financial product disclosure is attempting to achieve. Although disclosure has several aims, one is consumer comprehension to the degree necessary to enable good decisions. Disclosure rules require particular information to be imparted, often in a specified format. What if the law instead allowed firms to disclose whatever truthful and nonmisleading content they choose in whatever format they choose, but required firms to demonstrate, through field-based testing, consumer comprehension of the key facts about the financial product needed to make a good fact-based decision? 

A version of this is required by the FDA for a firm to move a drug from prescription to over-the-counter (OTC). Firms must first do lab-based consumer label comprehension testing à la Disclosure 2.0, but then typically must do actual use studies and postmarketing surveillance to see if real consumers using the product comprehend the key facts about it sufficiently well to use it correctly. The FDA has found that a label that does well in the lab can bomb in the study of actual use. 

The substance, standards, and mechanics for applying comprehension requirements to consumer financial products would need to be worked out (a project the FDA is still working on in the OTC context). Specific questions that would need to be answered include: 

Substance: What should comprehension testing test? To answer this question we would need to determine what consumers need to comprehend to make good financial product decisions, which in turn would require us to define which financial decisions are good ones. 

Standards: How well do consumers need to comprehend these things? Here we would need to establish benchmarks for what proportion of each population subgroup that might buy the financial product would need to demonstrate comprehension, and how much comprehension each subgroup would need to demonstrate.

Mechanics: How should comprehension requirements be enforced? When would testing take place? Who would be tested? If a product failed the test, what would happen to existing consumers who had already bought the product?

Although each of these questions is extremely difficult to answer, most of the same questions also need to be answered for rational financial education and disclosure policymaking. Just as we would need to ask what comprehension testing should test, we need to ask what information education or disclosure should convey, and to answer that question we must decide what consumers need to comprehend to make good financial product decisions, which in turn requires us to define which financial decisions are good ones. Just as we would need benchmarks for what a firm must show about consumer financial product understanding in the field to meet comprehension requirements, we need benchmarks for what outcomes demonstrate that financial education is effective and how well disclosure needs to perform in the lab. As with comprehension requirements, establishing these benchmarks requires decisions about how much of the information taught in the course or contained in the disclosure consumers need to understand, and what proportion of each population subgroup must understand it. That we have not directly confronted these questions in the financial education and disclosure contexts just means we are to a large degree stumbling around in the dark with these policy approaches.

The potential benefits of regulation through comprehension requirements are multiple. First, because it is field-based during actual product purchase and use, it measures the total effect on consumer understanding from various sources - formal disclosures, marketing, social messages, etc. - giving firms an incentive to engage in informative rather than misleading marketing and sales tactics. Second, it moves some burden from regulators to firms, which are better able to keep up with changes in the marketplace that affect consumer comprehension. Third, in the same spirit as new governance proposals, it gives firms more flexibility because they can meet comprehension standards in whatever way they see fit (e.g., through labeling, advertising or social media), facilitating innovation. Fourth, comprehension requirements are more likely to lead to more informed citizens and consumers than education and disclosure have done. Fifth, regulation through comprehension standards would force us to directly face the questions we have dodged with our existing policies of education and disclosure.

Potential pitfalls are also plentiful. First, depending on who is tested, various unintended consequences could occur. If a sample of all consumers were subject to comprehension testing, then low comprehension by those who would be unlikely to buy the product could prevent the sale of more complex products to consumers who would benefit from them. But if only actual product purchasers were subject to comprehension testing, firms might avoid selling to low literacy subgroups even where those groups might benefit from the product. Second, the costs of testing could disadvantage small firms. Third, testing schedules could favor particular firms. Early-testing firms might be forced to provide education to all consumers who might be tested, allowing firms that test later to free ride. Or early-testing firms might take the opportunity to monkey with their competitors’ testing, waiting until their own testing was complete and then using advertising to confuse consumers subject to competitors’ testing. Fourth, regulators might have difficulty monitoring the integrity of the comprehension testing environment or keeping comprehension requirements up-to-date with the market.

More problematically, comprehension requirements address only the first of the four fundamental problems with disclosure identified in my last post. A consumer who understands a product does not necessarily know what alternatives are available in the market, how to compare alternatives, or what her own future product use experience is likely to be. Comprehension does not address the consumer’s decisionmaking resources or shape the entire frame that affects consumer decisions. That is, comprehension is not sufficient for good decisionmaking about financial products. I would even suggest that comprehension is also not necessary for the appropriate use of consumer financial products. People make good decisions for the “wrong” reasons all the time (My friend recommended this mortgage broker so I’m sure she’s getting me a good deal is probably the wrong reason, but maybe the mortgage is a good deal). 

Nonetheless, we should explore comprehension requirements. At the very least, it would force us to grapple with what a good financial decision is, a question we sidestep when we rely on financial education or disclosure, but which we should honestly confront in those contexts as well.