3. Identifying the Problem

Chapter authors: Prashant Narang and Aryan Pandey

This step can be thought of as answering the first of the above questions, i.e.:

What is the problem you are trying to address and why is government/regulator action or intervention needed to address the problem?

One of the first questions that regulators will often need to ask up-front while attempting to identify the problem is: “is there a market failure which is allocating resources inefficiently and restricting society from maximising the gains from trading in the market?”

Regulatory intervention is often undertaken when markets fail to function smoothly. When markets function well, scarce resources are allocated efficiently, allowing all the suppliers to compete effectively and providing consumers with plenty of choice and all the information that enables them to make well-informed buying decisions.

Determining the presence of market failure involves asking three basic questions:

• What is causing the market to produce a poor outcome?
• Who is affected, and how?
• How large or serious is the problem?

A clear answer to these questions helps show whether regulatory intervention is justified and what kind of response may be suitable.

The factors that cause markets to fail and the impacts from those failures are discussed in detail next.

3.1. Market failure

A market failure exists when the market does not allocate resources in a way that leads to the maximisation of social welfare. Social welfare refers to the total gains from the market, including gains to consumers and producers. When markets fail, these gains are not fully realised. Too much or too little of a good or service may be produced. Prices may not reflect the true costs or benefits of an activity. Some people may make decisions without the information they need. In such cases, regulation may help correct the problem and improve how resources are allocated.

Markets can fail for several reasons. In each case, the failure arises because one or more of the conditions needed for a well-functioning competitive market do not exist in practice. For example, there may be too little competition, unequal access to information, spillover effects such as pollution, or goods and services that the market has little incentive to provide. Identifying whether a market failure exists, and understanding its nature, is therefore an important early step in an RIA. It helps the regulator decide whether intervention is justified and what kind of response may be appropriate.

Markets are commonly seen to fail for a few well-known reasons. These include externalities, public goods, market power, and information asymmetry. Each of these involves a situation in which private decision-making does not lead to an efficient outcome for the market or the wider economy.

The key reasons why markets are seen to fail in practice are the following:

3.1.1. Externalities

Externalities arise when the production or consumption of a good or service affects others who are not directly part of the transaction. These effects may be negative or positive. Negative externalities impose costs on others. Common examples include pollution and traffic congestion. Positive externalities create benefits for others. For example, vaccination can reduce the spread of disease beyond the individual receiving it, and education can produce wider social benefits. Because these wider costs or benefits are not fully reflected in market prices, the market may produce too much of an activity in some cases and too little in others.

3.1.2. Public goods

Some goods and services have the characteristics of a public good. Public goods are usually described as non-rival, non-excludable, or both. A good is non-rival if one person’s use of it does not reduce its availability to others. A good is non-excludable if, once it is provided, it is difficult or impossible to prevent others from using it. For example, national defence, public parks (at least to an extent).

Markets often fail to provide public goods at an efficient level because individuals have little incentive to pay for them when they cannot easily be excluded from consuming them and enjoying the benefit. This is known as the free-rider problem. As a result, such goods may be under-provided unless the government steps in to support or arrange their provision.

3.1.3. Market power

Markets can also fail when one or a small number of suppliers have enough power to influence prices, output, quality, or other market conditions. This is more likely in concentrated markets, especially where there are high fixed costs, strong barriers to entry, network effects, or other features that limit effective competition. In such markets, firms may be able to charge prices above competitive levels, restrict output, or earn profits above what would be expected in a competitive market. Consumers may then pay higher prices, face less choice, or receive lower quality than they would in a well-functioning competitive market.

Examples can be found in sectors such as telecommunications, electricity distribution, and airports. An airport, for example, may operate as a local monopoly because users often have few practical alternatives. In such cases, regulation may be used to address the effects of market power. This may take the form of price regulation, revenue regulation, access regulation, or information disclosure requirements.

3.1.4. Information asymmetry

Information asymmetry arises when one side of the market has more or better information than the other, and that information matters for decision-making. This is common where suppliers know more than consumers about the quality, safety, or performance of a good or service, and consumers cannot easily verify that information.

A common example is the market for used cars. A seller may know the vehicle’s true condition, accident history, or maintenance record, while a potential buyer may not. If buyers cannot tell the difference between good quality and poor quality vehicles, they may be unwilling to pay a fair price for better vehicles. Over time, this can drive good quality vehicles out of the market and reduce the overall quality of what is offered. In such situations, disclosure requirements, certification, warranties, or quality standards may help improve market outcomes.

These market failures provide one of the main economic grounds for regulatory intervention. Once the type of market failure has been identified, the next step in the RIA is to examine how it affects different stakeholders, how serious the effects are, and whether regulation is likely to improve the situation.

3.2 How to identify market failure

The process of identifying market failure helps policymakers understand what kind of problem they are dealing with. It helps distinguish between problems caused by the structure of the market, problems that arise because of the nature of the good or service, and problems caused by gaps in information. This matters because the policy response should be shaped by the source of the problem.

3.2.1. Step 1: Define the market clearly

Defining market comprises of:

• Identifying the good or service involved.
• Identifying who the buyers and sellers are.
• Asking whether there are other people or groups affected by the transaction, even if they are not directly buying or selling.

This step is important because market failure cannot be identified in the abstract. It must be identified in relation to a particular market. A clear definition of the market helps policymakers understand where the problem is occurring, who is involved, and who may be affected by it.

3.2.2. Step 2: Identify the source of the problem.

Market failures can be broadly divided into three categories (NSW, 2017):

1. The nature of the market: These are problems that arise because of the way the market is structured. For example, there may be too few suppliers, high barriers to entry, or strong market power. In such cases, firms may be able to raise prices, restrict output, or reduce quality below otherwise competitive levels.

1. The nature of the good: These are problems that arise because of the characteristics of the good or service itself. For example, the good may create spillover effects on others, as in the case of pollution, or it may have public good characteristics, such as non-rivalry or non-excludability. In such cases, the market may produce too much, too little, or none at all relative to what is socially desirable.

1. The availability of information: These are problems that may arise when one side of the market has more or better information than the other, and that information is important for decision-making. For example, sellers may know more than buyers about the quality or safety of a product. It would be hard or impossible for the buyers to obtain such knowledge. This can lead to inefficient market outcomes due to the presence of ‘moral hazard’ and ‘adverse selection’.

Using these three categories can help policymakers move from a general concern that “the market is not working well” to a more precise diagnosis of why the problem has arisen. This diagnosis can then provide the basis for deciding whether regulatory intervention is justified and what form it should take.

3.3. Does the market failure fall within the regulator’s jurisdiction?

The next step in an RIA is to assess whether regulatory intervention is feasible, and justifiable, i.e. does the problem fall within the regulator’s jurisdiction and are there options for the regulator to intervene positively, i.e. intervene\ intervene in a way that improves the overall welfare of stakeholders involved.

This question is at times treated as self-evident i.e., the regulator who identifies the problem should be the one addressing it. But that assumption might not always be correct in regulatory practice.

The OECD’s recommendation on improving the quality of government regulation explicitly asks, what is the appropriate level of government for this action? And where multiple levels or bodies are involved, it requires that regulators design effective systems of coordination between them.

This matters especially because the type of market failure has direct implications for which institution is best placed to respond. A failure rooted in market power where a dominant firm uses its market power to restrict competition, sits most naturally within a competition authority’s mandate. A failure rooted in information asymmetry between a financial product provider and retail consumers sits more naturally with a financial conduct regulator. Similarly, a failure involving cross-border data misuse may engage the mandates of a data protection authority, a telecommunications regulator, and a financial regulator simultaneously.

3.3.1. Test for institutional appropriateness

Before a regulator can be confirmed as the right institution to act, an RIA should work through three questions.

The first is legal jurisdiction: Regulators are created by statute, and their contours are by that statute. Acting beyond those bounds, even in pursuit of a genuine public interest objective, undermines institutional legitimacy. Regulators or “decision-maker must understand correctly the law that regulates his decision-making power and must give effect to it.”. In multi-regulator environments, the question of which body has primacy over a given problem is not obvious, and must be worked through explicitly. For instance, the Supreme Court had set aside the RBI’s circular on resolution of stressed assets. ruling that the RBI had gone beyond its powers under the Banking Regulation Act.

The second is technical competence: even within its legal mandate, does the regulator have the expertise to understand the market it proposes to regulate and to design responses that will work? Technical proficiency is not only a necessity for tackling the sector’s problems and assessing alternatives, it is also a source of legitimacy for the regulator (Baldwin, Cave, and Lodge 2010).

The third is practical capacity: does the regulator have the operational resources to implement and enforce its interventions over time? Even where a regulator has clear statutory authority to act, it may lack the practical capacity to do so effectively.

When capacity gaps exist, the RIA should address them directly: whether the intervention should be phased, whether external technical partnerships are needed, or whether a less administratively demanding instrument would achieve comparable results at lower institutional cost.

3.3.2. The multi-regulator problem

Another issue arises when a single regulator clearly lacks authority, but when several regulators have a partial claim on the same problem. Modern markets, particularly in financial services, digital platforms, and telecommunications frequently produce harms that may cut across institutional boundaries.

Consider a case in which consumers are being defrauded through a digital payment platform. The fraud may involve the payment infrastructure itself (engaging a financial regulator's remit), the telecommunications network over which the transaction ran (engaging a telecoms regulator), and the personal data that was harvested or misused in the process (engaging a data protection authority). No single regulator has full sight of the problem. Each has an incentive to act within its own mandate, and little structural incentive to coordinate with the others. The result can be duplicated or incompatible enforcement action, contradictory requirements imposed on the same firms, or a gap between mandates through which the harm falls unaddressed.

Policy makers should aspire to identify cross-cutting regulatory issues across all levels of government and design explicit information sharing and coordination mechanisms to promote coherence and avoid duplication or conflict between regulatory approaches.