9  Two-way fixed effects: The old difference-in-differences

Published

December 27, 2025

Modified

September 26, 2026

Keywords

A/B Testing, Causal Inference, Causal Time Series Analysis, Data Science, Difference-in-Differences, Directed Acyclic Graphs, Econometrics, Impact Evaluation, Instrumental Variables, Heterogeneous Treatment Effects, Potential Outcomes, Power Analysis, Sample Size Calculation, Python and R Programming, Randomized Experiments, Regression Discontinuity, Treatment Effects

Get news about the book. Leave your address here.
Found an error or have a suggestion? Submit feedback here.
Follow the book’s page for new content and follow the author.

Listen to a podcast version of this chapter, generated by Google’s NotebookLM.1

9.1 Why simple comparisons fail — and how DiD fixes it

You might be asking about the chapter title: why “The old Difference-in-Differences”? The answer lies in a recent revolution in causal inference. In the last decade, a wave of research has exposed significant cracks in the traditional DiD approach, which was formally estimated through Two-Way Fixed Effects (TWFE). We now know this classic method can be fragile, especially when the treatment effect isn’t constant or when different units get treated at different times (a common scenario known as staggered adoption).

But for a long time, TWFE was the standard approach. For instance, it is the default method taught in textbooks like Angrist and Pischke (2008) and Angrist and Pischke (2015). The influence of this framework runs deep — Joshua Angrist, one of the authors of these books, received the 2021 Nobel Memorial Prize in Economic Sciences for his methodological contributions to the analysis of causal relationships.

I start with the ‘old’ TWFE for two reasons. It builds intuition for how time enters causal inference, which we haven’t covered yet. It also remains a valid estimator in simpler settings where all units adopt treatment at the same time and treatment effects are relatively homogeneous.2

Consider an e-commerce company that wants to measure how its paid membership program (e.g., Amazon Prime, Meli+, etc.) affects customer spending. Your team needs to know whether subscribing to the program causes customers to spend more on the platform.

Approach 1: Before vs. After (within-group)

A naive approach would be to simply compare customers’ spending before and after joining the program. However, this simple “pre-post” comparison (also called first difference or within-group comparison) is fraught with potential pitfalls. What if there was a general upward trend in e-commerce spending during that period? Or what if a major holiday, like Black Friday, coincided with when many customers joined the program? These external factors — what data scientists call confounders — make it impossible to see if the membership program actually caused the change.

Figure 9.1: Pre-post vs. Difference-in-Differences: Scenario with no real effect

The figure above illustrates a scenario where the treatment has no effect on the outcome. If we only calculate the pre-post comparison, we may have the wrong impression that the subscription to the membership program increases the amount spent by subscribed users, while the increase actually comes from a natural positive trend in consumption (panel A of Figure 9.1). This is a classic example of confounding by time trends — we mistakenly attribute a general market trend to our specific intervention.

With a comparable control group, panel B of Figure 9.1 shows that the treated and control groups follow parallel trends (sometimes referred to as “PT”) before and after treatment. This is what we expect under the parallel trends assumption when the treatment has no causal effect.

Approach 2: Treated vs. Control (between-groups)

Another tempting, but equally flawed, approach would be to compare the spending of customers who joined the membership program with those who didn’t. This is a classic example of a cross-sectional comparison already introduced in Chapter 2. The problem here is one of self-selection.

The people who chose to join the membership program are likely different from those who didn’t in ways we can’t observe. They might be more frequent online shoppers, have higher disposable incomes, or simply be more deal-savvy consumers. These unobserved characteristics, and not the membership program itself, could be the real drivers of the difference in spending patterns.

The DiD solution: Comparing changes over time

DiD combines the before-and-after and treated-and-control comparisons. As its name suggests, it calculates the difference in the differences rather than relying on either comparison alone.

We first look at the change in spending before and after the membership program for the group that was exposed to it (the “treated” group). Then, we do the same for a similar group that was not exposed to the program (the “control” group). Finally, we compare these two differences. The resulting “difference in differences” gives us a much more reliable estimate of the program’s causal effect.

By looking at the change within each group (the ‘before vs. after’ differences), we automatically control for characteristics that stay the same over time (e.g., customer’s registration date, which is ‘time-invariant’), effectively removing them from the equation. The difference within the control group specifically captures the general trends or external shocks that would have affected both groups had the treatment not occurred.

Then, by subtracting this ‘before vs. after’ difference of the control group from the ‘before vs. after’ difference of the treated group, we can isolate the effect of the treatment, assuming that the two groups would have followed similar trends in the absence of the treatment. This is the famous “parallel trends” assumption, which we will discuss later.

Figure 9.2: Pre-post vs. Difference-in-Differences: Scenario with a real positive effect

Figure 9.2 shows a scenario where the treatment does have a causal effect. Notice how both groups follow parallel trends before the treatment, but after the treatment, the treated group’s trajectory diverges from the control group’s trajectory. This divergence represents the causal effect of the treatment, and it’s precisely what DiD is designed to capture.

See that even when the treatment has a causal effect, Panel A’s first difference approach still misleads us. The observed growth in the treated group reflects both the underlying trend and the causal effect of the intervention, and we can’t tell how much comes from each. Without a comparison group, we’d overestimate the treatment’s impact by attributing the entire increase to the intervention.

Panel B uses the control group’s trajectory as the benchmark. Its growth represents the “natural” trend that would have occurred without treatment; the treated group’s additional growth — the gap between Diff 2 and Diff 1 — is the estimated treatment effect.

The time subscript enters the scene: Now we start working with data that has both individual units, denoted by subscript \(i\), and time periods, denoted by subscript \(t\). You are already familiar with \(i\) (e.g., in \(y_i\)), so we now add \(t\) to track changes over time: \(y_{i,t}\) represents the outcome for individual \(i\) at period \(t\).

How potential outcomes help us understand DiD

Recall the hypothetical exercise from Chapter 2, where we used João and Maria’s potential outcomes to calculate the true effect of showing the “Super Promo Banner”. The food delivery app displays special discounts to low-activity users to increase orders. Table 9.1 modifies that exercise for this chapter.

Table 9.1: Potential outcomes for João, who receives the banner (treatment), and Maria, who doesn’t (control).
João (Low Orders) Maria (High Orders)
Did this person see the banner? (\(D_i\)) Yes (\(D_{João}=1\)) No (\(D_{Maria}=0\))
Orders this month (observed) (\(Y_i\)) 3 (\(Y_{João}\)) 8 (\(Y_{Maria}\))
Orders if NOT shown banner (\(Y_{i0}\)) 2 (\(Y_{João,0}\)) 8 (\(Y_{Maria,0}\))
Orders if shown banner (\(Y_{i1}\)) 3 (\(Y_{João,1}\)) 8 (\(Y_{Maria,1}\))

In this hypothetical world, where we could see potential outcomes, the true effect of showing the banner would be:

  • Effect for João = \(Y_{João,1} - Y_{João,0}\) = 3 (with banner) - 2 (without banner) = 1
  • Effect for Maria = \(Y_{Maria,1} - Y_{Maria,0}\) = 8 (with banner) - 8 (without banner) = 0

But, in real life, we can never observe both potential outcomes for the same person. We don’t get to see what João’s orders would have been without the banner (\(Y_{João,0}\)), or what Maria’s orders would have been with the banner (\(Y_{Maria,1}\)). All we see is each user’s observed outcome.

This leads many analysts to make the mistake of comparing the observed results for João and Maria and calling it the “effect” of the banner. For instance, \(Y_{João} - Y_{Maria} = 3 - 8 = -5\). And the analyst might jump to the conclusion that “wow, the banner actually makes things worse!”

So far, I’ve recapped the cross-sectional example from Chapter 2: we observed João and Maria at a single point in time, saw their outcomes, and faced the fundamental problem of causal inference — we can’t observe both potential outcomes for the same person.

Time gives DiD another comparison. Although we cannot observe both potential outcomes, observing João and Maria before the banner gives each user a baseline and lets us compare changes rather than levels.

Let’s adapt the example to see how this would work had we followed João and Maria for two periods, before and after the banner was shown. In Table 9.2, I’ll adjust the numbers slightly, increasing orders for both users in the second period to reflect a general upward trend.

Table 9.2: Using DiD to approximate counterfactuals in the potential outcomes framework. Unlike Table 9.1, all orders here are observed, but in different periods.
João (treated in t=2) Maria (never treated)
Orders on month \(t = 1\) 2 (\(Y_{João,t=1}\)) 8 (\(Y_{Maria,t=1}\))
Orders on month \(t = 2\) 4 (\(Y_{João,t=2}\)) 9 (\(Y_{Maria,t=2}\))
Treatment \(D_{i,t=2}\) Yes (\(D_{João,t=2} = 1\)) No (\(D_{Maria,t=2} = 0\))

In DiD, we use what happened before the treatment to approximate what would have happened without it (i.e., to create counterfactuals). Here’s how this works:

For João (treated): Before the banner was shown, João placed 2 orders (\(Y_{João,t=1} = 2\)). After seeing the banner, he placed 4 orders (\(Y_{João,t=2} = 4\)).

For Maria (control): In month 1, Maria placed 8 orders (\(Y_{Maria,t=1} = 8\)). In month 2, she placed 9 orders (\(Y_{Maria,t=2} = 9\)). She never saw the banner.

The first difference for João, \(\Delta Y_{João} = Y_{João,t=2} - Y_{João,t=1} = 4 - 2 = 2\) , represents the change in João’s orders from month 1 to month 2. However, this change includes both the effect of the banner (treatment effect) and any general time trends or external factors affecting both groups.3

The first difference for Maria, \(\Delta Y_{Maria} = Y_{Maria,t=2} - Y_{Maria,t=1} = 9 - 8 = 1\), represents the change in Maria’s orders from month 1 to month 2. Since Maria never received the banner, this change captures only the general time trends or external factors that would have affected João in the absence of treatment.

The DiD estimator is computed as \(\text{DiD} = \textcolor{#1d66db}{\Delta Y_{João}} - \textcolor{#bd1c1c}{\Delta Y_{Maria}} = \textcolor{#1d66db}{2} - \textcolor{#bd1c1c}{1} = 1\). If the assumptions we discuss later hold, this \(\text{DiD} = 1\) means the banner caused João to place 1 additional order compared to what he would have placed without it.4

Figure 9.3: The DiD estimator illustrated with João and Maria’s example. Panel A: observed orders over time for both individuals. Panel B: DiD logic showing how Maria’s trend is used as a counterfactual for João, leading to a treatment effect estimate of 1 additional order.

Figure 9.3 visually illustrates the logic behind the DiD estimator. In Panel A, we observe the evolution of orders over time for both individuals: Maria (in red), who was never shown the banner, and João (in blue), who received the treatment in period \(t = 2\). Both exhibit a similar upward change from \(t = 1\) to \(t = 2\).

In Panel B, the DiD logic unfolds. João’s post-treatment outcome is 4 orders. If he had followed Maria’s trend (his counterfactual), we would expect his orders to rise by the same 1 unit Maria gained, putting him at 3 orders without the banner. The DiD estimate is João’s change (4 − 2 = 2) minus Maria’s change (9 − 8 = 1), yielding a treatment effect of 1.

DiD compares changes, not levels. Its causal interpretation depends on the parallel trends assumption: without treatment, João’s trajectory would have mirrored Maria’s. If João was already on a different trajectory, the estimate could capture that difference rather than the banner’s effect.

Next, I dig deeper into parallel trends: what the assumption really requires, how to check it, and what options you have when it appears to fail.

9.3 The Stable Unit Treatment Value Assumption (SUTVA)

Another key rule for DiD is that one person’s treatment shouldn’t affect another person’s outcome. This is formally known as the Stable Unit Treatment Value Assumption (SUTVA). The logic is the same as we discussed in Section 4.1.1.2 and Chapter 6:

  1. No Interference: The potential outcomes of one unit should not be affected by the treatment status of other units. In other words, there should be no “spillover” or “cannibalization” effects between the treated and control groups. For example, if a marketing campaign to promote some products in a marketplace (treatment group) leads to a significant increase in the demand for these products, it might depress the demand for substitute products in that same marketplace (the control group). This would violate the no-interference assumption, as the outcome of the control group is being affected by the treatment.

    SUTVA violations from geographic and network spillovers are particularly common in practice. When treatment is assigned by region, customers near the border may cross over to shop in treated areas, or treated customers may reduce their activity in control regions. Either way, the control group’s outcomes become entangled with the treatment. The same logic applies to digital businesses: a ride-hailing platform testing surge pricing in one city might see drivers relocate to neighboring cities; an e-commerce platform promoting sellers in one region might shift customer shopping patterns across borders; in social networks, treating one user with a new feature can influence their friends’ behavior even if those friends are in the control group.

    When you suspect geographic or network spillovers, consider using buffer zones (excluding units close to treated units from the control group) or explicitly modeling the spillover mechanism.

  2. No Hidden Variations in Treatment: The treatment should be the same for all treated units. If there are different versions or intensities of the treatment, and these variations are not accounted for, the DiD estimate will be an average of the effects of these different treatments, which might not be what we are interested in.

Violations of SUTVA can be just as problematic as violations of the parallel trends assumption. To illustrate the “no hidden variations” condition, imagine a paid membership program that offers different benefit tiers: some customers get free shipping only, others get free shipping plus exclusive discounts, and a third group also receives priority customer support. If we lump all these customers together as “treated,” our DiD estimate averages across very different treatment intensities. The estimated effect wouldn’t correspond to any specific version of the program — it would be a weighted blend of heterogeneous treatments, making it hard to draw actionable conclusions about what actually drove the results.

9.4 Important details before using DiD

The way we design a study should follow the question we’re trying to answer — and how the real world works. Instead of forcing a method onto the data, I try to understand how the treatment happens in practice and build the analysis around that. My fellow economists call it “design-based inference.”

Before using TWFE DiD, check whether the study satisfies these design requirements:

  1. Why are the treated units receiving the treatment? Is there a possibility of selection bias? Even if pre-treatment trends are parallel, there might be unobserved factors that are correlated with both the treatment and the outcome. Later in this chapter, after we introduce event studies, we will see how a symptom of selection bias called Ashenfelter’s dip (Section 9.5.5) can be detected by examining pre-treatment trends.

  2. Are there any other events happening at the same time as the treatment? This is the problem of “compound treatments.” If another event that affects the outcome occurs at the same time as the treatment, it will be impossible to disentangle the two effects without further information and variation in the treatments.

  3. Is there anticipation of treatment? Outcomes in the pre-treatment period must not be affected by people learning about or preparing for the treatment. If users can foresee the change and adjust before the official start, the “pre” period is already contaminated.6

  4. Are all units being treated at the same time? This classic DiD setup works better with a single treatment date. However, in many real-world scenarios, treatment is rolled out over time. This is known as a “staggered adoption” or “multiple time periods” setting. As we discuss in Chapter 10, this can introduce significant challenges and potential biases.

9.4.1 Adjusting for serial correlation

When working with data that tracks the same units over multiple time periods, we need to be aware of a common issue called serial correlation (also known as “autocorrelation” or “time correlation”). This happens when the “noise” or “unexplained variation” in our data is connected across time periods for the same unit.

Think of it this way: if you’re tracking a customer’s spending behavior over several months, their spending in January is likely related to their spending in February, March, and so on. This makes sense. People tend to have consistent spending patterns, habits, and financial situations that persist over time.

Why does this matter? Serial correlation doesn’t mess up our main estimate of the treatment effect, the “point estimate”. But it does affect how confident we can be in our results. Specifically, it makes the standard errors of our regression artificially small (Bertrand et al. 2004). Since standard errors are our measure of uncertainty about the point estimate, this can lead us to think we found a significant effect when we actually didn’t. In other words, it inflates our risk of a “false positive”.

The mathematical intuition: Classical regression assumes that errors (the unexplained part of our data) are independent across time and across individuals. In mathematical terms:

  • Time independence: What I do today shouldn’t predict the “random” part of what I do tomorrow. (Mathematical version: \(Cov(\varepsilon_t, \varepsilon_s) = 0\), for \(t \neq s\)). In other words, for a given unit, the unexplained variation in one time period should be unrelated to the unexplained variation in other time periods

  • Cross-person independence: What I do shouldn’t predict the “random” part of what others do. (Mathematical version: \(Cov(\varepsilon_i, \varepsilon_j) = 0\), for \(i \neq j\)). In other words, for a given time period, the unexplained variation for one unit should be unrelated to the unexplained variation for other units

But in real life, these assumptions often don’t hold:

Example 1 - time correlation: My LinkedIn posting behavior this month is correlated with my posting behavior last month. If I posted a lot in January, I’m likely to post a lot in February too. This means \(Cov(\varepsilon_t,\varepsilon_{t-1}) \neq 0\). Translation: “My unexplained posting behavior this month is related to my unexplained posting behavior last month”.

Example 2 - cross-person correlation: My TV watching hours this week are correlated with my spouse’s TV watching hours, since we live together and often watch shows together. This means \(Cov(\varepsilon_i,\varepsilon_j) \neq 0\). Translation: “My unexplained TV watching is related to my spouse’s unexplained TV watching”.

Example: after observing the sales of 100 stores across 10 months, you don’t have 1,000 independent observations. You have 100 independent store sales histories, so the effective sample size for hypothesis testing is closer to 100 than to 1,000. Clustering treats the unit of uncertainty as the cluster: “For measuring uncertainty, your \(N\) is the number of clusters, not the number of observations.”

The solution: clustered standard errors

To fix this, we “cluster” our standard errors. This tells the model to treat each unit’s history as a connected bundle rather than independent points, correcting our uncertainty estimates. Tools like R’s fixest and Python’s linearmodels handle this automatically, as we will see in the code below.

One caveat: this method assumes we have many clusters — typically at least 30 to 50 (Cameron and Miller 2015). When working with fewer clusters, like comparing just a handful of regions, standard clustered errors can be deceptively small, leading to false positives. In those cases, we should use specialized alternatives not covered in this book, such as the wild cluster bootstrap (MacKinnon and Webb 2018), aggregation to the cluster level (Ibragimov and Müller 2016), or randomization inference (MacKinnon and Webb 2020).

9.5 The structure of DiD models: from aggregated to event studies

With the conceptual foundation in place, we now translate intuition into estimable models.

To understand the difference between the models we are about to build, think of aggregated DiD like taking a photo and event studies like recording a video. The aggregated model takes a “snapshot” that shows us the overall impact of our intervention, averaging out all the effects after the treatment happened. It’s useful for high-level summaries, like saying “on average, our new feature increased user engagement by 10%.”

But sometimes we need the full video. Event studies show us how the effects play out over time. They tell us whether the impact was immediate or gradual, if it grew stronger or weaker, or if it eventually wore off. We will begin with the aggregated model to establish the basics, then expand to event studies to capture these dynamic effects.

9.5.1 The aggregated DiD: a single estimate of the overall impact

We can estimate one average effect with aggregated DiD or estimate period-specific effects with an event study.

The aggregated DiD model is the standard tool for estimating a single, overall effect. It captures the core DiD logic in the simplest possible way.

The formal specification of the aggregated DiD model is:

\[Y_{it} = \gamma_i + \lambda_t + {\color{#b93939ff}\delta} D_{it} + X^{'}_{it} \beta + \varepsilon_{it} \tag{9.1}\]

  • \(Y _{it}\) represents the outcome variable for unit \(i\) in period \(t\);
  • \(D_{it}\) is the treatment indicator that equals 1 for treated units in post-treatment periods, and 0 otherwise;
  • \(\delta\), the treatment effect: our parameter of interest, the average treatment effect of \(D_{it}\) on \(Y_{it}\).
  • \(\gamma_i\) captures unit-specific fixed effects - individual-level characteristics that don’t change over time;
  • \(\lambda_t\) captures time-specific fixed effects - period-specific shocks that affect all units equally;
  • \(X^{'}_{it}\) is a vector of covariates for unit \(i\) in period \(t\);
  • \(\beta\) is a vector of coefficients for the covariates;
  • \(\varepsilon_{it}\) is the error term, with \(E[\varepsilon_{it}] = 0\)

That’s a lot of notation, so let’s break it down:

Covariates in DiD models can improve the precision of the estimates and, more importantly, help address potential violations of the parallel trends assumption if they are time-varying and affect the outcome. However, care must be taken when including covariates, especially time-invariant ones. I leave this discussion for the Appendix 9.A

Fixed effects

Fixed effects are the building blocks of TWFE DiD, with two key components working together to isolate the treatment effect. Individual Fixed Effects (\(\gamma_i\)) capture time-invariant characteristics of each unit across time periods, controlling for both observed and unobserved factors that don’t change over time. Meanwhile, Time Fixed Effects (\(\lambda_t\)) account for period-specific shocks that affect all units equally, such as holidays or technical issues that would impact both treatment and control groups.

The two fixed effects work together: individual effects control for “who you are”, while time effects control for “when you are”. This leaves the variation used to identify the treatment effect. We discuss both in more detail in Appendix 9.A.

The causal structure of DiD

Figure 9.6 maps the confounding paths behind the failed comparisons and shows what DiD removes.

graph LR
    classDef unobserved stroke-dasharray: 5 5;
    classDef observed stroke-width: 1px;

    D["Treatment, D"] --> Y["Outcome, Y"]
    gamma["Unit fixed effects, γᵢ"] --> Y
    gamma -.-> D
    lambda["Time fixed effects, λₜ"] --> Y
    U["Unobserved confounders, U"] -.-> Y
    U -.-> D

    class U unobserved
    class D,Y,gamma,lambda observed

    linkStyle 0 stroke:#00BCD4,stroke-width:2px;
Figure 9.6: Causal structure for the DiD setting. Unobserved time-invariant confounders \(U\) (dashed lines) affect both treatment assignment and outcomes, creating selection bias in cross-sectional comparisons. Common time shocks \(\lambda_t\) affect all units equally, biasing pre-post comparisons. DiD’s identification strategy (cyan arrows): by comparing changes over time between treated and control groups, we difference out both \(\gamma_i\) and \(\lambda_t\), isolating the causal effect of \(D\) on \(Y\).

The diagram reveals the confounding problem. Unit fixed effects (\(\gamma_i\)) capture stable characteristics that differ across units — for example, some stores are simply busier than others due to location or local demographics. These factors influence both whether a store ends up in the treatment or control group and its sales levels. A naive cross-sectional comparison (treated vs. control at one point in time) would conflate the treatment effect with these baseline differences.

Time fixed effects (\(\lambda_t\)) represent shocks that affect everyone equally — seasonal patterns, macroeconomic conditions, or platform-wide promotions. A naive pre-post comparison (before vs. after for the treated group alone) would attribute these common trends to the treatment.

DiD solves both problems simultaneously. By comparing changes over time (differencing out \(\gamma_i\)) between groups (differencing out \(\lambda_t\)), we isolate the treatment effect \(\delta\). The cyan arrow in the diagram marks what remains after this double-differencing: the causal path from treatment to outcome. However, notice that this strategy relies on the assumption that unobserved confounders \(U\) are time-invariant. If there are unobserved factors that affect the treatment and the outcome that change over time differently for each group (i.e., time-varying confounders not captured by \(\lambda_t\)), DiD may fail to isolate the causal effect.

\(\delta\) as the treatment effect

To see why, let’s take the difference between treated and control units, before and after treatment, doing algebra based on Equation 9.1:

For treated units (\(D_{1,pre}=0\), \(D_{1,post}=1\)):7

  • Pre-treatment: \(Y_{1,pre} = \gamma_1 + \lambda_{pre} + {\color{#b93939ff}\delta} \times\) \(0\)
  • Post-treatment: \(Y_{1,post} = \gamma_1 + \lambda_{post} + {\color{#b93939ff}\delta} \times\) \(1\)

For control units (\(D_{0,pre}=0\), \(D_{0,post}=0\)):

  • Pre-treatment: \(Y_{0,pre} = \gamma_0 + \lambda_{pre} + {\color{#b93939ff}\delta} \times\) \(0\)
  • Post-treatment: \(Y_{0,post} = \gamma_0 + \lambda_{post} + {\color{#b93939ff}\delta} \times\) \(0\)

Taking the difference-in-differences:

  • \((Y_{1,post} - Y_{1,pre}) - (Y_{0,post} - Y_{0,pre}) = {\color{#b93939ff}\delta}\)

The fixed effects and covariates cancel out, leaving only \(\delta\) as our treatment effect.

Notice what the algebra just did: we computed the change for the treated units and subtracted the change for the control units. The control group’s trajectory serves as the counterfactual for what the treated group would have experienced without the intervention — not for the entire population. This means \(\delta\) identifies the Average Treatment Effect on the Treated (ATT, see Section 2.5 for a refresher), not the ATE.

The DiD coefficient is often reported as “the effect of the program” without specifying for whom. It tells us how the intervention affected the units that actually received it, not what would happen if we extended treatment to new, untreated units. The ATT and ATE coincide only when treatment effects are homogeneous across treated and untreated units, an assumption that is difficult to defend in many applications.

In our OOH example, \(\delta\) tells us how much the Paraíba stores benefited from the campaign — not what would happen if we also ran billboards in Pernambuco. Those could be different numbers. Pernambuco stores might respond differently to OOH advertising due to local market conditions, competition, or consumer demographics. The ATT is the right estimand here: we want to know whether the campaign worked where it was actually deployed.

9.5.2 The aggregated DiD: applying it to our OOH example

Data: All data we’ll use are available in the repository. You can download it there or load it directly by passing the raw URL to read.csv() in R or pd.read_csv() in Python.8

Packages: To run the code yourself, you’ll need a few tools. The block below loads them for you. Quick tip: you only need to install these packages once, but you must load them every time you start a new session in R or Python.

# If you haven't already, run this once to install the package
# install.packages("fixest")

# You must run the lines below at the start of every new R session.
library(tidyverse) # Data manipulation
library(fixest) # High-performance fixed-effects regression
# If you haven't already, run this in your terminal to install the packages:
# pip install pandas numpy linearmodels statsmodels (or use "pip3")

# You must run the lines below at the start of every new Python session.
import pandas as pd                        # Data manipulation
import numpy as np                         # Mathematical computing
from linearmodels.panel import PanelOLS    # Panel data models for DiD

A supermarket chain wants to know whether an Out-of-Home (OOH) marketing campaign increased sales in its Paraíba stores (PB). It compares them with untreated stores in neighboring Pernambuco (PE).

I simulated a dataset to represent the scenario described above. Here’s how the code variables map to the formal notation from Equation 9.1:

  • store: Store ID — this is our unit \(i\), and store fixed effects give us \(\gamma_i\)
  • period: Observation period — this is \(t\), and period fixed effects give us \(\lambda_t\)
  • sales: Sales in millions of R$ — this is our outcome \(Y_{it}\)
  • treatment: Dummy indicating whether the store is in the treated state (Paraíba = 1)
  • treated_period: Equals 1 for treated stores in post-campaign periods — this is \(D_{it}\)
  • relative_period: Event time \(k\), i.e., the number of periods from the campaign start (\(t = 7\)). We will use this variable in the event study section below, as we re-index time relative to the intervention start date.

Let’s first look at the raw data. Figure 9.7 shows the average sales trajectories for stores in Paraíba (treatment group, in blue) and Pernambuco (control group, in red) over time. The vertical dashed line marks the start of the OOH campaign (period 7).

Figure 9.7: Average sales in millions of R$ by group (Treatment [PB] vs Control [PE])

Notice that Paraíba stores start at a lower sales level than Pernambuco stores — this baseline gap doesn’t matter for DiD, because DiD compares changes over time, not levels, and the store fixed effects absorb these permanent differences. What matters is the trends: before the campaign, both states show relatively similar trends in sales, which is central to the parallel trends assumption. After the campaign begins, we can observe a noticeable increase in sales for stores in Paraíba compared to those in Pernambuco, suggesting a positive effect of the OOH campaign.

The code below implements Equation 9.1 with store fixed effects and period fixed effects. Terms containing cluster account for serial correlation within each store, which produces appropriate standard errors when stores are observed repeatedly over time.

#Load required library
library(fixest)
data <- read.csv("data/did-twfe-ooh.csv")

# Aggregate DiD model estimating the campaign's effect
agg_did_model <- feols(sales ~ treated_period | store + period,
                       cluster = ~store, data = data)
summary(agg_did_model)
# Import required libraries
import pandas as pd
import numpy as np
from linearmodels.panel import PanelOLS

# Load the dataset
data = pd.read_csv("data/did-twfe-ooh.csv")
data = data.set_index(['store', 'period'])

# Aggregated Diff-in-Diff
X = data[['treated_period']]
y = data['sales']

agg_did_model = PanelOLS(y, X, entity_effects=True, time_effects=True)
agg_did_results = agg_did_model.fit(cov_type='clustered', cluster_entity=True)

print(agg_did_results)

Take a look at the results in the tab:

Figure 9.8: Aggregated DiD regression output in R
Figure 9.9: Aggregated DiD regression output in Python

The stores in Paraíba experienced an average increase of R$2.99 million in monthly sales, and this increase is statistically significant (p-value less than 0.05). Since our outcome is measured in millions of Brazilian Reais per month, this estimated effect means that, on average, stores in Paraíba saw their monthly sales increase by R$2.99 million compared to stores in Pernambuco, which didn’t receive the intervention.

To put it another way: regardless of how much the Pernambuco stores were selling (which we can check in the data), the Paraíba stores sold R$2.99 million more per month on average. If all the assumptions discussed in previous sections are met, this represents the true causal effect of the intervention.

Note that this R$2.99 million is the average per-period effect, not the total cumulative impact of the campaign. If you want to know the total incremental revenue generated across all post-treatment months, you need to think more carefully about what this coefficient represents. We will return to this distinction in the section “What do the DiD coefficients actually measure?” (Section 9.5.6), after introducing the event study, where the answer becomes much clearer.

Treatment is assigned by state, so geographic spillovers threaten SUTVA. Customers from Pernambuco (the control state) might cross the border to shop at Paraíba stores after seeing the billboards, or Paraíba customers might reduce their shopping in Pernambuco.

Either pattern would contaminate the control group’s outcomes, biasing our estimate. When your treatment and control groups share a geographic border, it is worth considering buffer zones — for example, excluding stores near the state boundary — or at least checking whether control-group stores closest to the border behave differently from those farther away.

9.5.3 Event Study Designs: Looking beyond a single average effect

While the aggregated DiD model we just saw gives us one number - the average treatment effect - it does not show how the effect varies over time. Think of it like measuring the average temperature for an entire year: it’s useful, but it doesn’t tell you about the seasonal patterns or whether it was particularly cold in July or hot in December (I am in the southern hemisphere, so it’s the opposite of most books).

In that sense, the event study (also called dynamic DiD) design helps us see the complete picture by showing us:

  1. What happened before the treatment? - This helps us assess whether the parallel trends assumption is plausible, by checking whether the treated and control groups were following similar trends before the treatment.9
  2. How did the effect evolve over time? - Did it happen immediately, grow gradually, or fade away (such as in the novelty effects we discussed in the previous chapter)?

The notation for event studies

An optional but handy trick here is to think in terms of “event time” rather than calendar time. Instead of looking at the treated stores in period 9, we ask “what happened 2 periods before the treatment?”. Think of it like organizing your life around a major event, the treatment.

Here’s how it works: We define the treatment period as “event time zero” (\(k=0\)). Everything before that gets negative numbers:

  • \(k=-1\): One period before treatment
  • \(k=-2\): Two periods before treatment
  • And so on…

And everything after the treatment gets positive numbers:

  • \(k=1\): One period after treatment
  • \(k=2\): Two periods after treatment
  • And so on…

Including a dummy for every period creates the perfect multicollinearity discussed in Chapter 6: the columns add up to a constant, so the model cannot be estimated. We therefore omit one period as the “reference point” and compare every other period with it. We typically choose \(k=-1\), the period just before treatment. Its plotted value is a normalization choice, not a finding.

This means:

  • All other periods are compared to this baseline
  • We can’t estimate an effect for \(k=-1\) because it’s our comparison point (it’s set to zero by definition)

What the results tell us: leads and lags

The mathematical specification for those interested:

\[Y_{it} = \gamma_i + \lambda_t + \sum_{k=-l}^{m} {\color{#b93939ff}\delta_k} D_{ki} + X^{'}_{it} \beta + \varepsilon_{it} \tag{9.2}\]

Most terms match Equation 9.1. The new term is \(\sum_{k=-l}^{m} {\color{#b93939ff}\delta_k} D_{ki}\), where \(l\) and \(m\) count the pre- and post-treatment periods. Instead of one treatment-effect coefficient \(\delta\), the event study estimates a separate \(\delta_k\) for each period relative to treatment.

Think of it as breaking apart our single treatment dummy into multiple pieces, one for each period before and after treatment begins. To make this more concrete, let’s say we have data from 6 periods before treatment through 4 post-treatment periods (from \(k=0\) to \(k=3\)), and we’re using \(k=-1\) as our reference.

\[Y_{it} = \gamma_i + \lambda_t + {\color{#b93939ff}\delta_{-6}} D_{-6,i} + \ldots + {\color{#b93939ff}\delta_{0}} D_{0,i} + \ldots + {\color{#b93939ff}\delta_{3}} D_{3,i} + X^{'}_{it} \beta + \varepsilon_{it} \tag{9.3}\]

The equation would expand to Equation 9.3. Each \(\delta_k\) tells us the treatment effect for that specific period, allowing us to see how the impact changes over time rather than just getting one average effect.10

\(\delta_k\) for pre-treatment period — the “leads”, \(k \lt 0\): These show us the difference between treated and control groups before anything happened. If our parallel trends assumption is correct, these should all be close to zero and not statistically significant. Think of these as our “quality check”: if we see big, significant differences before treatment, it means our groups weren’t following similar trends to begin with.

\(\delta_k\) for post-treatment periods — the “lags”, \(k \geq 0\): These show us how the treatment effect evolved over time:

  • \(k=0\) tells us the immediate effect when treatment started
  • \(k=1\), \(k=2\), etc. tell us if the effect grew, stayed the same, or faded over time

Event-study graphs show whether treated and control groups followed similar trends before treatment (parallel lines before \(k=0\)) and whether suspicious pre-treatment patterns, such as anticipation effects, appear.

9.5.4 Event Study Designs: applying it to our OOH example

Let’s apply the event study approach to our OOH campaign. Remember, the campaign started in period 7, so periods 1-6 are our “pre-treatment” periods (\(k=-6\) to \(k=-1\)); and periods 7-10 are the treatment window (\(k=0\) to \(k=3\)). Since we’re using period 6 (\(k=-1\)) as our reference, we’ll estimate effects for all other periods relative to that baseline.

We now translate the group differences in Figure 9.7 into period-specific treatment-effect estimates. The R and Python tabs fit Equation 9.2 and report the regression output.

# Event Study Diff-in-Diff
event_model <- feols(sales ~ i(relative_period, treatment, ref = -1) |
                        store + period, cluster = ~store, data = data)
summary(event_model)

# Event Study Diff-in-Diff
# Create interaction terms for event study
# Exclude the reference period (-1)
event_study_terms = []
for p in sorted(data['relative_period'].unique()):
    if p != -1:
        data[f'rel_period_{p}_treated'] = (data['relative_period'] == p) * data['treatment']
        event_study_terms.append(f'rel_period_{p}_treated')


X_event_study = data[event_study_terms]
y_event_study = data['sales']

event_study_model = PanelOLS(y_event_study, X_event_study, entity_effects=True, time_effects=True)
event_study_results = event_study_model.fit(cov_type='clustered', cluster_entity=True)

print(event_study_results)
Figure 9.10: Event study regression output in R
Figure 9.11: Event study regression output in Python

The models use store and period fixed effects, cluster standard errors by store, and set -1 as the reference period. Each coefficient is an estimated average effect for one event-time period; the output marks leads in blue and lags in red. In the month the campaign began (relative period = 0), Paraíba stores had a statistically significant average sales increase of R$2.27 million.

The results reveal two key findings: no substantial pre-treatment differences and positive post-treatment effects.

Period -3 is individually significant, but one significant lead can occur by chance when we test several periods (the “multiple comparisons problem”). Individual t-tests are therefore insufficient. An F-test tests the joint null that all pre-treatment coefficients equal zero. Here, the joint test does not reject that null, which is consistent with parallel trends. The code below runs the test:

# Joint F-test for pre-treatment coefficients (leads)
# H0: All pre-treatment effects are jointly zero
wald(event_model, "relative_period::-[2-6]")
# Joint F-test for pre-treatment coefficients (leads)
# H0: All pre-treatment effects are jointly zero

# Identify pre-treatment terms (relative_period < -1)
pre_terms = [c for c in event_study_results.params.index
             if 'rel_period' in c and int(c.split('_')[2]) < -1]

# Build a restriction matrix: one row per pre-treatment term,
# with a 1 in the column matching that coefficient
R = np.zeros((len(pre_terms), len(event_study_results.params)))
for i, term in enumerate(pre_terms):
    j = list(event_study_results.params.index).index(term)
    R[i, j] = 1

# Wald test: H0 = all pre-treatment coefficients are jointly zero
event_study_results.wald_test(restriction=R)

Figure 9.12 shows how the effect unfolds over time and whether it persists:

Figure 9.12: Event Study: Sales lift by period in stores impacted by OOH campaign. Each dot represents the estimated DiD coefficient indicating the treatment effect in a given event time period, showing by how much the treated group sold more compared to the control group, with period -1 as the baseline reference.

Before the campaign (k=-6 to k=-2): The pre-treatment coefficients (“leads”) are small and mostly not statistically significant. That pattern is consistent with similar pre-treatment trends in Paraíba and Pernambuco, but it does not prove parallel trends.

During and after the campaign (k=0 to k=3): The post-treatment coefficients (“lags”) trace an immediate effect at k=0 and its path over subsequent periods. They show when the campaign effect emerged and how it changed, information the aggregated estimate cannot provide.

9.5.5 How selection bias breaks parallel trends: Ashenfelter’s dip

The OOH example showed no substantial pre-treatment differences. Ashenfelter’s dip, first described by Ashenfelter (1978), shows how selection can instead produce a visible pre-treatment failure.

This pattern emerges from selection bias: when individuals choose to participate (or are chosen) based on characteristics related to the outcome. The classic example is job training programs.

Suppose a job training program is more likely to enroll people whose earnings were already falling. Their pre-program earnings will show a deceptive “dip” relative to the control group, followed by an apparent recovery. Part of that recovery may reflect selection rather than the program, so the comparison can overstate the treatment effect.

Figure 9.13: Ashenfelter’s dip: when individuals with declining outcomes self-select into treatment, the pre-treatment “dip” violates parallel trends and inflates the apparent treatment effect.

Figure 9.13 illustrates this phenomenon with simulated earnings data. Imagine individuals who are already struggling the most (those with declining earnings) are more likely to enroll in a job training program. The treatment group shows a steep decline in earnings before the program starts — these are the individuals who self-selected into training precisely because they were struggling.

After the program, their earnings recover. But how much of that recovery is the true program effect, and how much is simply regression to the mean? The dashed line (the counterfactual trajectory) shows what parallel trends would have predicted: a world where the treatment group follows the same trajectory as the control. The gap between the observed post-treatment outcome and this counterfactual is the apparent effect — but it’s inflated because we’re measuring from an artificially depressed baseline.

The event study as a diagnostic tool

An event study can reveal Ashenfelter’s dip because its pre-treatment coefficients will not hover around zero. Their pattern exposes the selection that preceded treatment.

Figure 9.14: Event study reveals the parallel trends violation: pre-treatment coefficients significantly different from zero signal Ashenfelter’s dip.

Figure 9.14 shows an event study estimated on data with Ashenfelter’s dip. The pre-treatment coefficients (orange) are significantly positive, hovering around 3,700-4,000, before dropping to zero at the reference period (\(k=-1\)), the bottom of the dip. This pattern signals selection bias: the treatment group did relatively better in earlier periods and then experienced the “dip” that led its members to self-select into the program.

Remember what these coefficients represent: the difference between treatment and control groups relative to the reference period. If parallel trends held, these pre-treatment coefficients should cluster around zero — the two groups would be on similar trajectories. Instead, we see a clear pattern: the treatment group’s relative position deteriorates as we approach the treatment, revealing that something systematic (selection bias) is driving the divergence.

WarningThe event study test is necessary but not sufficient

An event study with pre-treatment coefficients around zero is consistent with parallel trends, but doesn’t prove it. The assumption is fundamentally about counterfactuals — what would have happened in the absence of treatment. Pre-trends can look parallel even when the post-treatment counterfactuals would diverge. Still, finding significant pre-trends (as in Figure 9.14) is strong evidence that something is wrong with your research design.

Does the reference period choice matter?

Standard practice uses \(k = -1\), the period just before treatment, as the reference, but the choice is somewhat arbitrary. Figure 9.15 holds the event-study regression and data fixed and changes only the reference period.

Figure 9.15: The same event study with different reference periods. Changing the reference period shifts all coefficients up or down, but the shape of the trajectory — and the evidence of pre-treatment divergence — remains the same.

Figure 9.15 shows three things:

  1. The reference period is set to zero by construction (circled points). This is a mechanical feature of how event studies work — we’re measuring everything relative to that period.

  2. Changing the reference shifts the entire curve vertically — the shape stays the same, but every coefficient moves up or down by the same constant. In this simulated earnings example, the left panel (\(k = -1\)) anchors at the bottom of the dip, so the post-treatment coefficients sit high on the y-axis. The middle panel (\(k = -3\)) anchors at a mid-pre-treatment point, shifting the whole curve down. The right panel (\(k = -5\)) anchors well before the dip, pushing the curve down further. Same data, same shape, but the numerical value you would report for any single coefficient changes with the reference.

  3. The diagnostic signal persists regardless of reference choice. In all three panels, we see significant coefficients in the pre-treatment period that shouldn’t be there if parallel trends held. Whether those coefficients are positive (left panel) or negative (right panel) depends on the reference, but their presence — and the clear pattern of divergence — reveals the selection bias.

When Ashenfelter’s dip is present, the estimated treatment-effect magnitude depends on the normalization period. Using \(k = -1\) anchors the coefficients at the bottom of the dip, a poor baseline for estimating the true effect. The diagnosis does not change: significant pre-treatment coefficients remain, although their signs differ across reference periods.

This is why experienced researchers don’t just look at whether pre-treatment coefficients are “positive” or “negative” — they look at the pattern. A clear trend in the pre-treatment coefficients (whether rising or falling toward the reference period) signals that the groups were on different trajectories before treatment, regardless of how you choose to anchor the comparison.

9.5.6 What do the DiD coefficients actually measure?

With the aggregated DiD and event study side by side, we can now answer: what exactly does the R$2.99 million from the aggregated model represent?

Remember the photo-versus-video analogy? The aggregated model gave us one snapshot — R$2.99 million (Figure 9.8). The event study gave us the full footage — four separate coefficients (Figure 9.10), each capturing a different post-treatment period. Table 9.3 puts them next to each other.

Table 9.3: Aggregated DiD vs. event study post-treatment coefficients
Model What it estimates Value (millions R$)
Aggregated DiD Single coefficient \(\delta\) 2.99
Event study, \(k=0\) \(\delta_0\) (campaign launch) 2.27
Event study, \(k=1\) \(\delta_1\) (peak effect) 5.78
Event study, \(k=2\) \(\delta_2\) 4.07
Event study, \(k=3\) \(\delta_3\) 2.09

The aggregated coefficient (R$2.99M) doesn’t match the sum of the event study coefficients (R$14.21M), nor does it match any single period’s effect. It lands in the neighborhood of their average: (2.27 + 5.78 + 4.07 + 2.09) / 4 \(\approx\) R$3.55M. In other words, the aggregated DiD tells you how much more a treated store sold in a typical post-treatment month — not across all months combined.

This matters the moment you walk into a stakeholder meeting. If the CMO asks “how much extra revenue did the campaign generate per month?”, the aggregated coefficient — roughly R$2.99 million — is your answer. But if they ask “what was the total incremental revenue across the entire post-treatment window?”, you need the sum of the event study coefficients: approximately R$14.21 million per store. Report the aggregated number as the total impact and you understate the campaign’s value by a factor of four. You can see this visually in Figure 9.12 — each post-treatment dot is a separate per-period effect, and the aggregated model compresses those four dots into a single number near their average. Useful for a quick summary, but it hides the dynamics: the peak at \(k=1\), the gradual decline, and the full cumulative impact.

TipAverage vs. cumulative: a rule of thumb

The aggregated DiD coefficient estimates the average per-period treatment effect on the treated (ATT). To get the cumulative impact over the entire post-treatment window, sum the event study post-treatment coefficients — or, as a rough shortcut, multiply the aggregated coefficient by the number of post-treatment periods.

The average event study coefficient (R$3.55M) does not exactly match the aggregated coefficient (R$2.99M) because the models use different baselines. The event study measures each coefficient relative to \(k = -1\), while the aggregated model implicitly averages over the entire pre-treatment window. Because the pre-treatment coefficients are not all zero — recall the positive estimate at \(k = -3\) and the smaller one at \(k = -6\) in Figure 9.10 — the baselines differ. As the pre-treatment coefficients approach zero, the two estimates converge.

Getting this distinction right is especially important when you translate results into business language — a topic we cover in Chapter 14 on business translations. Whether you feed the per-period average or the cumulative total into an ROI calculation changes the investment recommendation entirely. Confuse the two and you risk either underselling a successful campaign or overstating a marginal one.

9.6 When classic DiD breaks down: staggered treatment adoption

Classic DiD works well when all units begin treatment at the same time. In many applications, however, treatment rolls out gradually. A recommendation algorithm, for example, may reach users over several weeks. This staggered rollout creates treatment timing heterogeneity.

Traditional TWFE can fail under staggered adoption because it compares treated units not only with untreated or not-yet-treated units, but also with units treated earlier or later. Those treated-to-treated comparisons violate the clean contrast between treated and untreated units.

These “forbidden comparisons” can mislead and may even reverse the sign of the true treatment effect, especially when effects vary over time or across groups (heterogeneous treatment effects). Both forms of variation occur in practice.

Recent research addressed these problems. Methodological advances by Callaway and Sant’Anna (2021), Sun and Abraham (2021), and others offer alternatives that are robust to treatment timing and heterogeneous effects. These newer estimators work by isolating clean comparisons: they compare treated units to those not yet treated (when possible), avoiding bias from treated-to-treated comparisons. They also allow treatment effects to vary over time, rather than forcing a one-size-fits-all estimate.

These methods are more complex, but they produce credible comparisons when treatment timing or effects vary. In the next chapter, Chapter 10, I’ll apply modern DiD estimators to actual data using the same business scenario and code structure.

9.7 Wrapping up and next steps

We’ve covered a lot of ground. Here’s what we’ve learned:

  • Naive comparisons lie: Simple “before vs. after” or “treated vs. control” comparisons are almost always biased by time trends or selection effects.
  • DiD is the solution: By comparing the changes in the treated group against the changes in a control group, we isolate the treatment effect — assuming parallel trends hold.
  • TWFE is the workhorse: Two-Way Fixed Effects models use Unit Fixed Effects (“who you are”) and Time Fixed Effects (“when you are”) to implement this comparison regression-style.
  • Event studies show dynamic effects: Aggregated DiD gives a single number, but event studies visualize the dynamic effect and provide a key falsification test (pre-trends).
  • Clustering matters: Serial correlation in panel data is real. Always cluster your standard errors at the unit level to avoid false positives.

How to implement a TWFE DiD analysis:

  1. Plot the raw data: Before modeling, visualize the average outcomes for treated and control groups over time. Look for parallel movements in the pre-treatment period.
  2. Estimate the Aggregated Model: Run a TWFE regression (feols in R, PanelOLS in Python) including your treatment dummy, unit FEs, and time FEs. Cluster standard errors at the unit level.
  3. Run an Event Study: Replace the single treatment dummy with “leads” and “lags” relative to the treatment start (usually using \(k=-1\) as reference).
  4. Validate Assumptions: Check if the “leads” (pre-treatment coefficients) are statistically indistinguishable from zero (individually or via a joint F-test). If they aren’t, you likely have a violation of parallel trends or selection bias.
  5. Interpret the Result: If assumptions hold, the “lags” (post-treatment coefficients) represent the causal effect of your intervention in each period.

In this chapter, we assumed that everyone in the treated group started the treatment at the same time. But in the real world — especially in tech rollouts — adoption is often staggered: some users get the feature in January, others in March, others in June.

Using the traditional TWFE model on staggered data can produce biased estimates, including sign reversals. In Chapter 10, we’ll explore why this happens and introduce the modern DiD estimators designed to handle staggered adoption correctly.

Appendix 9.A: Understanding fixed effects: The building blocks of TWFE DiD

Fixed effects make the DiD comparison possible by controlling for two sources of variation. TWFE DiD uses individual fixed effects and time fixed effects.

Individual fixed effects (\(\gamma_i\)): Capturing time-invariant heterogeneity

Individual fixed effects control for all characteristics of each unit that remain constant over time. Think of them as each unit’s “baseline personality” or “inherent characteristics” that don’t change during our study period.

For example, in our Super Banner study, João might naturally be a high spender due to his income level, lifestyle preferences, or shopping habits. Maria might be a more conservative spender due to her different financial situation or spending philosophy. These individual characteristics don’t change from week to week, but they significantly influence spending behavior. Without controlling for them, we might mistakenly attribute differences in spending between João and Maria to the treatment, when they’re actually just reflecting their different baseline spending patterns.

The individual fixed effect \(\gamma_i\) creates a separate intercept for each unit. It absorbs observed and unobserved characteristics that do not change over time, whether or not they are measured.

Time fixed effects (\(\lambda_t\)): Capturing period-specific shocks

Time fixed effects control for factors that affect all units equally in a given time period but vary across periods. These represent “macro” or “environmental” factors that create period-specific shocks or trends.

In our Super Banner example, imagine that during week 3 of our study, there was a major holiday that increased everyone’s spending across the platform. Or perhaps there was a technical issue that temporarily reduced app performance for all users. These are time-specific shocks that would affect both João (treatment group) and Maria (control group) equally, regardless of whether they saw the Super Banner or not.

Without time fixed effects, we might incorrectly attribute the holiday-induced spending increase to the Super Banner treatment. The time fixed effect \(\lambda_t\) creates a separate intercept for each time period, ensuring that period-specific factors that affect everyone equally don’t bias our treatment effect estimate.

How Fixed Effects Work Together

The beauty of the DiD approach is how these two types of fixed effects work together. Individual fixed effects control for “who you are” (time-invariant characteristics), while time fixed effects control for “when you are” (period-specific factors). This leaves us with only the variation that comes from the interaction between individual characteristics and time-varying factors — precisely what we need to identify the treatment effect.

This is why the parallel trends assumption is so important: it ensures that in the absence of treatment, the remaining variation (after controlling for individual and time fixed effects) would be the same for both treatment and control groups. When we add the treatment, any difference in this remaining variation can be attributed to the treatment effect.

Important considerations when including covariates

Time-invariant covariates: If you include characteristics that don’t change over time (e.g., store size, location characteristics) in a model with unit fixed effects (\(\gamma_i\)), the model will drop them because they are redundant (i.e., the covariate and the fixed effect will be perfectly collinear). The “unit fixed effects” already handle everything that stays constant.

Time-varying covariates: Time-varying covariates are valuable because they can help control for confounding factors that change over time and might otherwise bias the DiD estimate. For example, if there’s a general economic downturn that affects sales differently in Paraíba and Pernambuco, including a covariate for local economic conditions could help account for this.

Endogeneity of covariates: Be cautious about including covariates that might themselves be affected by the treatment. If a covariate is an outcome of the treatment, including it in the model would lead to biased estimates of the treatment effect.

Avoiding “bad controls”: Never include covariates that could be affected by the treatment — these are sometimes called “post-treatment” or “endogenous” covariates. For example, if you’re studying how a paid membership program affects customer spending, don’t control for “number of purchases this month” — that’s likely a mediator (the program increases purchases, which increases spending). Controlling for it would block part of the very effect you’re trying to measure. Safe covariates are those determined before treatment or those that are structurally unaffected by it (e.g., demographic characteristics measured at baseline). When in doubt, draw a causal diagram and check whether the covariate lies on a path from treatment to outcome.

Covariates can improve precision and address time-varying confounding, but their inclusion depends on how they vary over time and relate to treatment and outcome.

  1. As Google advises, “NotebookLM can be inaccurate; please double-check its content”. I recommend reading the chapter first, then listening to the audio to reinforce what you’ve learned.↩︎

  2. Throughout this chapter, I use DiD to refer to the general strategy of comparing changes across groups, and TWFE to refer to the specific regression implementation. When the distinction matters, I say TWFE DiD explicitly.↩︎

  3. Don’t worry about the \(\Delta\) symbol, it is just a way to represent the change in a variable. It is not a new concept.↩︎

  4. In plain language, it reads “João’s orders increased by 2 units between months 1 and 2. Maria’s orders increased by 1 unit between months 1 and 2. If some assumptions hold, then the banner caused João to place 1 additional order compared to what he would have placed without the banner”.↩︎

  5. Another approach is to explicitly include group-specific time trends in your regression model (e.g., linear or quadratic time trends). However, this practice has fallen out of favor because it relies on strong assumptions: (1) that trends would have continued unchanged in the post-treatment period, and (2) that the functional form of the trend (linear, quadratic, etc.) is correctly specified. These assumptions are often difficult to justify empirically or theoretically.↩︎

  6. Example: A marketplace announces a free shipping badge starting next month. As a reaction, sellers adjust now to qualify to that program by lowering prices or changing inventory, pulling sales into the pre-period. In this case, measuring the effect of the free shipping badge starting next month would be biased, since the treatment effect is in fact a combination of the free shipping badge and the sellers’ anticipation of it.↩︎

  7. For simplicity, I’ve omitted the covariates and the error term in the equations below. In reality, we should use the expectation operator \(E[\cdot]\) on each side, which would yield 0 for the error term. The terms with covariates cancel out in the difference-in-differences calculation.↩︎

  8. To load it directly, use the example URL https://raw.githubusercontent.com/RobsonTigre/everyday-ci/main/data/did-twfe-ooh.csv.↩︎

  9. Some materials say “by ruling out effects before the treatment”.↩︎

  10. Notice that there’s no \(\delta_{-1} D_{-1,i}\) term - that’s because \(k=-1\) is our reference period (set to zero). Each \(D_{k,i}\) is a dummy variable that equals 1 when a treated unit is observed in event time \(k\), and 0 otherwise.↩︎