The summary()
and tidy()
functions compute the average marginal effect (or contrast) when they are applied to an object produced by marginaleffects()
(or comparisons()
). This is done in 3 steps:
As explained succinctly on Stack Exchange:
we want the variance of the Average Marginal Effect (AME) and hence our transformed function is: \(AME = \frac{1}{N} \sum_{i=1}^N g_i(x_i,\hat{\beta})\) Then using the delta method we have \(Var \left( g(\hat{\beta}) \right) = J_g' \Omega_{\hat{\beta}} J_g\) where \(\Omega_{\hat{\beta}} = Var(\hat{\beta})\) and \(J_g' = \frac{\partial\left[\frac{1}{N}\sum_{i=1}^N g (x_i,\hat{\beta})\right]}{\partial \hat\beta} = \frac{1}{N}{\left[\sum_{i=1}^N \frac{\partial \left (g (x_i,\hat{\beta})\right)}{\partial \hat\beta}\right]}\) Which justifies using the “average Jacobian” in the delta method to calculate variance of the AME.
References:
This section compares 4 quantities:
The ideas discussed in this section focus on contrasts, but they carry over directly to analogous types of marginal effects.
In models with interactions or non-linear components (e.g., link function), the value of a contrast or marginal effect can depend on the value of all the predictors in the model. As a result, contrasts and marginal effects are fundamentally unit-level quantities. The effect of a 1 unit increase in \(X\) can be different for Mary or John. Every row of a dataset has a different contrast and marginal effect.
The mtcars
dataset has 32 rows, so the comparisons()
function produces 32 contrast estimates:
By default, the marginaleffects()
and comparisons()
functions compute marginal effects and contrasts for every row of the original dataset. These unit-level estimates can be unwieldy and hard to interpret. To help interpretation, the summary()
function computes the “Average Marginal Effect” or “Average Contrast,” by taking the mean of all the unit-level estimates.
summary(cmp)
#> Average contrasts
#> Term Contrast Effect Std. Error z value Pr(>|z|) 2.5 % 97.5 %
#> 1 mpg +1 0.06081 0.01284 4.737 2.1714e-06 0.03565 0.08597
#>
#> Model type: glm
#> Prediction type: response
which is equivalent to:
We could also show the full distribution of contrasts across our dataset with a histogram:
library(ggplot2)
cmp <- comparisons(mod, variables = "gear")
ggplot(cmp, aes(comparison)) +
geom_histogram(bins = 30) +
facet_wrap(~contrast, scale = "free_x") +
labs(x = "Distribution of unit-level contrasts")
This graph display the effect of a change of 1 unit in the mpg
variable, for each individual in the observed data.
An alternative which used to be very common but has now fallen into a bit of disfavor is to compute “Contrasts at the mean.” The idea is to create a “synthetic” or “hypothetical” individual (row of the dataset) whose characteristics are completely average. Then, we compute and report the contrast for this specific hypothetical individual.
This can be achieved by calling using the newdata
argument and the datagrid()
function, which fixes variables to their mean by default:
comparisons(mod, variables = "mpg", newdata = datagrid())
#> rowid type term contrast comparison std.error statistic p.value
#> 1 1 response mpg +1 0.1664787 0.06245542 2.66556 0.007686022
#> conf.low conf.high gear mpg
#> 1 0.04406829 0.288889 3 20.09062
Contrasts at the mean can differ substantially from average contrasts.
The advantage of this approach is that it is very cheap and fast computationally. The disadvantage is that the interpretation is somewhat ambiguous. Often times, there simply does not exist an individual who is perfectly average across all dimensions of the dataset. It is also not clear why the analyst should be particularly interested in the contrast for this one, synthetic, perfectly average individual.
Yet another type of contrast is the “Contrast between marginal means.” This type of contrast is closely related to the “Contrast at the mean”, with a few wrinkles. It is the default approach used by the emmeans
package for R
.
Roughly speaking, the procedure is as follows:
btype
(focal variable) and resp
(group by
variable).btype
.The contrast obtained through this approach has two critical characteristics:
With respect to (a), the analyst should ask themselves: Is my quantity of interest the contrast for a perfectly average hypothetical individual? With respect to (b), the analyst should ask themselves: Is my quantity of interest the contrast in a model estimated using unbalanced data, but interpreted as if the data were perfectly balanced?
If the answer to both questions is “yes”, then it is easy to use the comparisons()
to estimate contrasts between marginal means. The emmeans
section of the Alternative Software vignette shows several examples with parallel code examples to highlight parallels between marginaleffects
and emmeans
.
The excellent vignette of the emmeans
package discuss the same issue in a slightly different way:
The point is that the marginal means of cell.means give equal weight to each cell. In many situations (especially with experimental data), that is a much fairer way to compute marginal means, in that they are not biased by imbalances in the data. We are, in a sense, estimating what the marginal means would be, had the experiment been balanced. Estimated marginal means (EMMs) serve that need.
All this said, there are certainly situations where equal weighting is not appropriate. Suppose, for example, we have data on sales of a product given different packaging and features. The data could be unbalanced because customers are more attracted to some combinations than others. If our goal is to understand scientifically what packaging and features are inherently more profitable, then equally weighted EMMs may be appropriate; but if our goal is to predict or maximize profit, the ordinary marginal means provide better estimates of what we can expect in the marketplace.