The R Package diseq:
Estimation Methods for Markets in Equilibrium and Disequilibrium
July 6th, 2021
useR! The R Conference
Pantelis Karapanagiotis
karapanagiotis[at]ebs[dot]edu
@pi_kappa_
EBS University Wiesbaden, Leibniz Institute for Financial Research SAFE
Markets in Equilibrium
- Equilibrium concepts are analytically convenient.
- Equilibrium models constitute reasonable econometric approximations on many occasions.
Markets in Disequilibrium
- On others… not so much
Market Clearing vs Short Side Rule
\begin{align*}
Q_{nt} &= D_{nt} = S_{nt}
\end{align*}
\begin{align*}
Q_{nt} &= \min\{D_{nt},S_{nt}\}
\end{align*}
Design goals
- Simple, approachable, common estimation interface for all market models.
- Fast estimation routines.
- Post estimation analytics.
Implementation
- Simple, approachable, common estimation interface for all market models.
- Object oriented design architecture. Polymorphic estimation (and other) calls.
- Fast estimation routines.
- Estimation using analytic gradient expressions by default.
- Performance gains are documented by extensive benchmarking estimations with simulated data.
- Post estimation analytics.
- Predicted demanded and supplied quantities. Aggregation.
- Analysis of shortages.
- Marginal effects.
Architecture overview
Basic model estimation benchmarks
Model specification
key_columns <- c("ID", "TREND")
quantity_column <- "HS"
price_column <- "RM"
time_column <- "TREND"
demand_specification <- "TREND + W + CSHS + L1RM + L2RM + MONTH"
supply_specification <- "TREND + W + L1RM + MA6DSF + MA3DHF + MONTH"
correlated_shocks <- FALSE
verbose <- 3
Model initialization
equilibrium_mdl <- new(
"equilibrium_model",
key_columns, quantity_column, price_column,
paste(price_column, demand_specification, sep = "+"),
paste(price_column, supply_specification, sep = "+"),
house_data,
correlated_shocks = correlated_shocks, verbose = verbose)
basic_mdl <- new(
"diseq_basic",
key_columns, quantity_column, price_column,
paste(price_column, demand_specification, sep = "+"),
paste(price_column, supply_specification, sep = "+"),
house_data,
correlated_shocks = correlated_shocks, verbose = verbose)
directional_mdl <- new(
"diseq_directional",
key_columns, time_column, quantity_column, price_column,
paste(price_column, demand_specification, sep = "+"),
supply_specification,
house_data,
correlated_shocks = correlated_shocks, verbose = verbose)
deterministic_adjustment_mdl <- new(
"diseq_deterministic_adjustment",
key_columns, time_column, quantity_column, price_column,
paste(price_column, demand_specification, sep = "+"),
paste(price_column, supply_specification, sep = "+"),
house_data,
correlated_shocks = correlated_shocks, verbose = verbose)
Model estimation
optimization_control <- list(maxit = 50000)
equilibrium_est <- estimate(equilibrium_mdl, control = optimization_control)
basic_est <- estimate(basic_mdl, control = optimization_control,
start = equilibrium_est@coef)
directional_est <- estimate(directional_mdl, method = "Nelder-Mead",
control = optimization_control)
deterministic_adjustment_est <- estimate(deterministic_adjustment_mdl,
control = optimization_control)
The R Package diseq:
Estimation Methods for Markets in Equilibrium and Disequilibrium
- Provides estimation solutions with unavailable alternatives.
- Simple market model estimation interface.
- Fast implementation of estimation routines.
- Post estimation analysis tools.
- Where to go from here?
- Additional models: Mayer, Madden, Jin & Tran (2015), Loberto & Zollino (2018).
- Alternative estimation methods: Quandt & Ramsey (1978), Zilinskas & Bogle (2006).
- Disequilibrium tests: Quandt (1978), Hwang (1980).
Pantelis Karapanagiotis
karapanagiotis[at]ebs[dot]edu
References
- [mayer2015] Mayer, Madden, Jin & Tran, Modelling OECD broadband subscriptions in disequilibrium, Technological Forecasting and Social Change, 90(PB), 476-486 (2015). doi.
- [loberto2016] Loberto & Zollino, Housing and Credit Markets in Italy in Times of Crisis, Bank of Italy, (2018).
- [quandt1978estimating] Quandt & Ramsey, Estimating mixtures of normal distributions and switching regressions, Journal of the American Statistical Association, 73(364), 730-738 (1978). doi.
- [zilinskas2006] Zilinskas & Bogle, Balanced random interval arithmetic in market model estimation, European Journal of Operational Research, 175(3), 1367-1378 (2006). doi.
- [quandt1978tests] Quandt, Tests of the Equilibrium vs. Disequilibrium Hypotheses, International Economic Review, 19(2), 435 (1978). doi.
- [hwang1980] Hwang, A test of a disequilibrium model, Journal of Econometrics, 12(3), 319-333 (1980). doi.
- [blpestimator2019] BLPestimatoR, BLP Demand Estimation, By Brunner D., Weiser C. & Romahn A. Distributed by CRAN, (2019).
- [miceconaids2017] MicEconAids, Demand analysis with the almost ideal demand system, By Henningsen A. Distributed by CRAN, (2017).
- [disequilibrium2020] Disequilibrium, Disequilibrium Models, By Latshaw N. & Guggisberg M. Distributed by CRAN, (2020).
- [henningsen2007] Henningsen & Hamann, systemfit : A Package for Estimating Systems of Simultaneous Equations in R, Journal of Statistical Software, 23(4), 1-40 (2007). link. doi.
- [bbmle2020] Bbmle, Tools for General Maximum Likelihood Estimation, By Bolker B, R Development Core Team & Giné-Vázquez I. Distributed by CRAN, (2020).
- [zellner1962] Zellner & Theil, Three-Stage Least Squares: Simultaneous Estimation of Simultaneous Equations, Econometrica, 30(1), 54 (1962). doi.
- [balestra1987] Balestra & Varadharajan-Krishnakumar, Full information estimations of a system of simultaneous equations with error component structure, Econometric Theory, 3(2), 223-246 (1987). doi.
- [fair1972] Fair & Jaffee, Methods of Estimation for Markets in Disequilibrium, Econometrica, 40(3), 497 (1972). doi.
- [amemiya1974] Amemiya, A Note on a Fair and Jaffee Model, Econometrica, 42(4), 759 (1974). doi.
- [maddala1974] Maddala & Nelson, Maximum Likelihood Methods for Models of Markets in Disequilibrium, Econometrica, 42(6), 1013 (1974). doi.
- [maddala1986] Maddala, Disequilibrium, self-selection, and switching models, , 3, in: Handbook of Econometrics (1986).
- [userp] Economic Report of the President, President and Council of Economic Advisers (U.S.), (1947).
- [fair1971] Fair, A short-run forecasting model of the United States economy, Heath Lexington Books (1971).
- [usfrb] Federal Reserve Bulletin, Board of Governors of the Federal Reserve System (U.S.), 1935- and Federal Reserve Board, (1914).
Appendix
Previous research and related software
- Similar and alternative software
- Demand estimation: BLPestimatoR (2019), MicEconAids (2017)
- Disequilibrium estimation: Disequilibrium (2020)
- Powered by
- R packages: Henningsen & Hamann (2007), Bbmle (2020)
- Native libraries: GNU Scientific Library
- Estimation theory:
- Equilibrium: Zellner & Theil (1962), Balestra & Varadharajan-Krishnakumar (1987)
- Disequilibrium: Fair & Jaffee (1972), Amemiya (1974), Maddala & Nelson (1974), Maddala (1986), Quandt & Ramsey (1978).
Market models
The equilibrium model
\begin{align}
\begin{aligned}
D_{nt} &= X_{d,nt}'\beta_{d} + P_{nt}\alpha_{d} + u_{d,nt}, \\
S_{nt} &= X_{s,nt}'\beta_{s} + P_{nt}\alpha_{s} + u_{s,nt}, \\
Q_{nt} &= D_{nt} = S_{nt} .
\end{aligned} \tag{E} \label{eq:model:equilibrium}
\end{align}
- Assumes that market clearing always holds.
- Estimated with 2SLS, 3SLS, and FIML. The last two methods are asymptotically equivalent.
The basic model
\begin{align}
\begin{aligned}
D_{nt} &= X_{d,nt}'\beta_{d} + u_{d,nt}, \\
S_{nt} &= X_{s,nt}'\beta_{s} + u_{s,nt}, \\
Q_{nt} &= \min\{D_{nt},S_{nt}\} .
\end{aligned} \tag{B} \label{eq:model:basic}
\end{align}
- Instead of market clearing, the short side rule governs the market. No price dynamics.
The directional model
\begin{align}
\begin{aligned}
D_{nt} &= X_{d,nt}'\beta_{d} + u_{d,nt}, \\
S_{nt} &= X_{s,nt}'\beta_{s} + u_{s,nt}, \\
Q_{nt} &= \min\{D_{nt},S_{nt}\}, \\
\Delta P_{nt} &\ge 0 \implies D_{nt} \ge S_{nt}.
\end{aligned} \tag{D} \label{eq:model:directional}
\end{align}
- Sample separation based on price changes (One directional assumption however).
The deterministic adjustment model
\begin{align}
\begin{aligned}
D_{nt} &= X_{d,nt}'\beta_{d} + P_{nt}\alpha_{d} + u_{d,nt}, \\
S_{nt} &= X_{s,nt}'\beta_{s} + P_{nt}\alpha_{s} + u_{s,nt}, \\
Q_{nt} &= \min\{D_{nt},S_{nt}\}, \\
\Delta P_{nt} &= \frac{1}{\gamma} \left( D_{nt} - S_{nt} \right).
\end{aligned} \tag{DA} \label{eq:model:deterministic_adjustment}
\end{align}
- Sample separation based on price movements, but this time in a quantitative nature. Price changes are analogous to shortages/surpluses.
The stochastic adjustment model
\begin{align}
\begin{aligned}
D_{nt} &= X_{d,nt}'\beta_{d} + P_{nt}\alpha_{d} + u_{d,nt}, \\
S_{nt} &= X_{s,nt}'\beta_{s} + P_{nt}\alpha_{s} + u_{s,nt}, \\
Q_{nt} &= \min\{D_{nt},S_{nt}\}, \\
\Delta P_{nt} &= \frac{1}{\gamma} \left( D_{nt} - S_{nt} \right) + X_{p,nt}'\beta_{p} + u_{p,nt}.
\end{aligned} \tag{SA} \label{eq:model:stochastic_adjustment}
\end{align}
- No sample separation. Price dynamics are stochastic.
Package design
Comparison of equilibrium estimation methods and tools
seed <- 25
parameters <- list(
nobs = 4000, tobs = 5,
alpha_d = -0.7, beta_d0 = 28.9, beta_d = c(0.3, -0.2), eta_d = c(-0.03, -0.01),
alpha_s = 0.6, beta_s0 = 10.2, beta_s = c(0.3), eta_s = c(0.5, 0.02),
sigma_d = 2.0, sigma_s = 3.0, rho_ds = -0.3)
equilibrium_mdl <- simulate_model("equilibrium_model", parameters, seed, verbose)
fiml_optim_est <- estimate(equilibrium_mdl, control = optimization_control)
fiml_gsl_est <- maximize_log_likelihood(
equilibrium_mdl, step = .01, objective_tolerance = .01, gradient_tolerance = .01)
ls_est <- estimate(equilibrium_mdl, method = "2SLS")
Market classes' design
Remaining models' estimation benchmarks
Equilibrium model
Directional model
Deterministic adjustment model
Stochastic adjustment model
Example
Houses data
data(houses)
| DATE | The date of the record. | |
| HS | Private non-farm housing starts in thousands of units (not seasonally adjusted). | Economic Report of the President (1947) |
| RM | FHA Mortgage Rate series on new homes in units of 100 (beginning-of-month Data). | Fair (1971) |
| DSLA | Savings capital (deposits) of savings and loan associations in millions of dollars. | Federal Reserve Bulletin (1914) |
| DMSB | Deposits of mutual savings banks in millions of dollars. | Federal Reserve Bulletin (1914) |
| DHLB | Advances of the federal home loan bank to savings and loan associations in million of dollars. | Federal Reserve Bulletin (1914) |
| W | Number of working days in month. | Manually collected |
Houses data
house_data <- fair_houses()
| DATE | The date of the record. | |
| HS | Private non-farm housing starts in thousands of units (not seasonally adjusted). | Economic Report of the President (1947) |
| RM | FHA Mortgage Rate series on new homes in units of 100 (beginning-of-month Data). | Fair (1971) |
| DSLA | Savings capital (deposits) of savings and loan associations in millions of dollars. | Federal Reserve Bulletin (1914) |
| DMSB | Deposits of mutual savings banks in millions of dollars. | Federal Reserve Bulletin (1914) |
| DHLB | Advances of the federal home loan bank to savings and loan associations in million of dollars. | Federal Reserve Bulletin (1914) |
| W | Number of working days in month. | Manually collected |
| ID | Dummy entity id | |
| DSF | \(DSLA_{t} + DMSB_t - (DSLA_{t-1} + DMSB_{t-1})\) | |
| DHF | \(DHLB_{t} - DHLB_{t-1}\) | |
| MONTH | Month of the date | |
| L2RM | \(RM_{t-2}\) | |
| L1RM | \(RM_{t-1}\) | |
| L1HS | \(HS_{t-1}\) | |
| CSHS | \(\sum_{s=1}^{t-1} HS_{s}\) | |
| MA6DSF | Moving average of order six of DSF | |
| MA3DHF | Moving average of order 3 of DHF |
Post estimation options
- Demanded, supplied quantities, and aggregation
- Analysis of shortages
- Marginal effects
Demanded, supplied quantities, and aggregation
demanded_quantities(model, est@coef)
supplied_quantities(model, est@coef)
aggregate_demand(model, est@coef)
aggregate_supply(model, est@coef)
Analysis of shortages
shortages(model, est@coef)
shortage_indicators(model, est@coef)
shortage_standard_deviation(model, est@coef)
normalized_shortages(model, est@coef)
relative_shortages(model, est@coef)
shortage_probabilities(model, est@coef)
Marginal effects
summary(model)
Stochastic Adjustment Model for Markets in Disequilibrium
Demand Equation : D_HS ~ D_RM + D_W + D_TREND + D_CSHS + D_MONTH
Supply Equation : S_HS ~ S_RM + S_W + S_TREND + S_MA6DSF + S_MA3DHF + S_MONTH
Price Equation : RM_DIFF ~ (D_HS - S_HS) + TREND + L2RM + L3RM
Shocks : Independent
Nobs : 128
Sample Separation : Not Separated
Quantity Var : HS
Price Var : RM
Key Var(s) : ID, TREND
Time Var : TREND
shortage_probability_marginal(model, est@coef, "RM")
shortage_probability_marginal(model, est@coef, "CSHS", aggregate = "at_the_mean")
shortage_marginal(model, est@coef, "MA3DHF")
More on functionality
Initialization
model <- new(
"diseq_stochastic_adjustment",
c("ID", "TREND"), "TREND", "HS", "RM",
"RM + TREND + W + CSHS + MONTH",
"RM + TREND + W + MA6DSF + MA3DHF + MONTH",
"TREND + L2RM + L3RM + L4RM",
fair_houses() %>% dplyr::mutate(
HS = log(HS),
L1HS = dplyr::lag(HS),
CSHS = cumsum(ifelse(is.na(L1HS), 0, L1HS)),
L3RM = dplyr::lag(RM, 3), L4RM = dplyr::lag(RM, 4)),
correlated_shocks = FALSE)
summary(model)
Stochastic Adjustment Model for Markets in Disequilibrium
Demand Equation : D_HS ~ D_RM + D_W + D_TREND + D_CSHS + D_MONTH
Supply Equation : S_HS ~ S_RM + S_W + S_TREND + S_MA6DSF + S_MA3DHF + S_MONTH
Price Equation : RM_DIFF ~ (D_HS - S_HS) + TREND + L2RM + L3RM
Shocks : Independent
Nobs : 128
Sample Separation : Not Separated
Quantity Var : HS
Price Var : RM
Key Var(s) : ID, TREND
Time Var : TREND
Estimation options
- Keyword options:
gradient %in% c("numerical", "calculated") # default: "calculated"
hessian %in% c("numerical", "calculated", "skip") # default: if not "calculated" then "numerical"
standard_errors %in% c("homoscedastic", "heteroscedastic") ||
standard_errors %in% names(object_to_be_estimated@model_tibble) # default: "homoscedastic"
- Example:
est <- estimate(model, control = list(maxit = 1e+5, reltol = 1e-4),
standard_errors = c("W"))