Solving Heterogeneous Estimating Equations with Gradient Forests

We propose a method for non-parametric statistical estimation, based on random forests (Breiman, 2001), that can be used to fit any heterogeneous parameter of interest identified as the solution to a set of local estimating equations. Following the classical literature on local maximum likelihood and generalized method of moments (GMM), our method estimates parameters at a particular point in covariate space by considering nearby training examples; but in contrast to these methods, which rely on kernel weighting functions and are prone to a strong curse of dimensionality, we use an adaptive weighting function derived from a forest. Our method, gradient forest, provides a practical and computationally efficient way of growing forest-based weighting functions that can meaningfully express heterogeneity in the parameters of interest. It proceeds by growing trees that recursively apply a pre-processing step in which we label each observation with the gradient of the estimating equations with respect to the parameters of interest, followed by a regression step that splits observations into leaves according to heterogeneity, as in a standard regression tree. We illustrate our method in two settings, non-parametric quantile regression and heterogeneous treatment effect estimation via instrumental variables. We also develop a large sample theory for gradient forests, show that our parameter estimates are consistent and asymptotically Gaussian, and propose an estimator for their asymptotic variance that enables valid confidence intervals.