Sensemakr#

Description#

Sensitivity analysis to unobserved confounders.

This class performs sensitivity analysis to omitted variables as discussed in Cinelli and Hazlett (2020). It returns an object of class Sensemakr with several pre-computed sensitivity statistics for reporting. After creating the object, you may directly use the plot and summary methods of the returned object.

Sensemakr is a convenience class. You may use the other sensitivity functions of the package directly, such as the functions for sensitivity plots (ovb_contour_plot, ovb_extreme_plot), the functions for computing bias-adjusted estimates and t-values (adjusted_estimate, adjusted_t), the functions for computing the robustness value and partial R2 (robustness_value, partial_r2), or the functions for bounding the strength of unobserved confounders (ovb_bounds), among others.

Parameters#

First argument should either be

  • a statsmodels OLSResults object (“fitted_model”); or

  • the numerical estimated value of the coefficient, along with the numeric values of standard errors and degrees of freedom (arguments estimate, se and df).

Return#

An object of class Sensemakr, containing:

  • sensitivity_stats : A Pandas DataFrame with the sensitivity statistics for the treatment variable, as computed by the function sensitivity_stats.

  • bounds : A pandas DataFrame with bounds on the strength of confounding according to some benchmark covariates, as computed by the function ovb_bounds.

Reference#

Cinelli, C. and Hazlett, C. (2020), “Making Sense of Sensitivity: Extending Omitted Variable Bias.” Journal of the Royal Statistical Society, Series B (Statistical Methodology).

Examples#

>>> # Load example dataset:
>>> import sensemakr as smkr
>>> darfur = smkr.load_darfur()
>>> # Fit a statsmodels OLSResults object ("fitted_model")
>>> import statsmodels.formula.api as smf
>>> model = smf.ols(formula='peacefactor ~ directlyharmed + age + farmer_dar + herder_dar + pastvoted + hhsize_darfur + female + village', data=darfur)
>>> fitted_model = model.fit()
>>> # Runs sensemakr for sensitivity analysis
>>> sensitivity = smkr.Sensemakr(fitted_model, treatment = "directlyharmed", benchmark_covariates = "female", kd = [1, 2, 3])
>>> # Manual bounds example:
>>> sensitivity_manual = smkr.Sensemakr(model = fitted_model, treatment = "directlyharmed", r2dz_x = [0.1, 0.2], r2yz_dx= [0.2, 0.3], bound_label= ["bound 1", "bound 2"])
>>> # List of Lists example:
>>> sensitivity_lists = smkr.Sensemakr(model = fitted_model, treatment = "directlyharmed", benchmark_covariates = [['female'], ['pastvoted','female']])
>>> # Dictionary Example:
>>> sensitivity_dict = smkr.Sensemakr(model = fitted_model, treatment = "directlyharmed", benchmark_covariates = {'Fem':['female'],"FemPast":["female","pastvoted"]})
>>> # Description of results
>>> sensitivity.summary()
class sensemakr.Sensemakr(model=None, treatment=None, estimate=None, se=None, dof=None, benchmark_covariates=None, kd=1, ky=None, q=1, alpha=0.05, r2dz_x=None, r2yz_dx=None, r2dxj_x=None, r2yxj_dx=None, bound_label='Manual Bound', reduce=True)[source]#

The constructor for a Sensemakr object.

Parameter descriptions are below. For usage and info, see the description of the class.

Required parameters: model and treatment, or estimate, se, and dof.

Parameters:

  • model (statsmodels OLSResults object) – a fitted statsmodels OLSResults object for the restricted regression model you have provided.

  • treatment (string) – a string with the name of the “treatment” variable, e.g. the independent variable of interest.

  • estimate (float) – a float with the estimate of the coefficient for the independent variable of interest.

  • se (float) – a float with the standard error of the regression.

  • dof (int) – an int with the degrees of freedom of the regression.

  • benchmark_covariates (string, list of strings or named dictionary) – a string, list of strings, or a named dictionary with the strings of the variables to use for benchmark bounding. If named dictionary, the names are used as labels.

  • kd (float or list of floats) – a float or list of floats with each being a multiple of the strength of association between a benchmark variable and the treatment variable to test with benchmark bounding.

  • ky (float or list of floats) – same as kd except measured in terms of strength of association with the outcome variable.

  • q (float) – a float with the percent to reduce the point estimate by for the robustness value RV_q.

  • alpha (float) – a float with the significance level for the robustness value RV_qa to render the estimate not significant.

  • r2dz_x (float or list of floats) – a float or list of floats with the partial R^2 of a putative unobserved confounder “z” with the treatment variable “d”, with observed covariates “x” partialed out. In this case, you are manually specifying a putative confounder’s strength rather than benchmarking.

  • r2yz_dx (float or list of floats) – a float or list of floats with the partial R^2 of a putative unobserved confounder “z” with the outcome variable “y”, with observed covariates “x” and treatment variable “d” partialed out. In this case, you are manually specifying a putative confounder’s strength rather than benchmarking.

  • r2dxj_x (float) – float with the partial R2 of covariate Xj with the treatment D (after partialling out the effect of the remaining covariates X, excluding Xj).

  • r2yxj_dx (float) – float with the partial R2 of covariate Xj with the outcome Y (after partialling out the effect of the remaining covariates X, excluding Xj).

  • bound_label (string or list of strings) – a string or list of strings with labels to the bounds provided manually via r2dz_x and r2yz_dx, for printing and plotting purposes.

  • reduce (boolean) – whether to reduce (True, default) or increase (False) the estimate due to putative confounding.

summary(digits=3)[source]#

Print a summary of the sensitivity results for a Sensemakr object, including robustness value, extreme confounding scenario, and benchmark bounding.

digits is the number of digits to round numbers to; default is 3. Following the example above, if you have a Sensemakr object called sensitivity, call this method using sensitivity.summary(). To round to 5 digits instead of 3, call sensitivity.summary(digits=5).

Parameters:

digits (int) – Rounding digit for summary (Default value = 3).

plot(plot_type='contour', sensitivity_of='estimate', **kwargs)[source]#

Provide the contour and extreme scenario sensitivity plots of the sensitivity analysis results obtained with the function Sensemakr.

They are basically dispatchers to the core plot functions ovb_contour_plot and ovb_extreme_plot.

This function takes as input a sensemakr object and one of the plot type “contour” or “extreme”. Optional arguments can be found in sensitivity_plots documentation including col_contour, col_thr_line etc.

Parameters:

  • plot_type (string) – Either “extreme” or “contour” (Default value = “contour”).

  • sensitivity_of (string) – Either “estimate” or “t-value” (Default value = ‘estimate’).

  • **kwargs – Arbitrary keyword arguments. See ovb_contour_plot and ovb_extreme_plot.

Examples

>>> # Load example dataset:
>>> import sensemakr as smkr
>>> darfur = smkr.load_darfur()
>>> # Fit a statsmodels OLSResults object ("fitted_model"):
>>> import statsmodels.formula.api as smf
>>> model = smf.ols(formula='peacefactor ~ directlyharmed + age + farmer_dar + herder_dar + pastvoted + hhsize_darfur + female + village', data=darfur)
>>> fitted_model = model.fit()
>>> # Runs sensemakr for sensitivity analysis
>>> sensitivity = smkr.Sensemakr(fitted_model, treatment = "directlyharmed", benchmark_covariates = "female", kd = [1, 2, 3])
>>> sensitivity.plot()
print(digits=3)[source]#

Print a short summary of the sensitivity results for a Sensemakr object, including formula, hypothesis, and sensitivity analysis.

digits is the number of digits to round numbers to; default is 3. Following the example above, if you have a Sensemakr object called sensitivity, call this method using sensitivity.print(). To round to 5 digits instead of 3, call sensitivity.print(digits=5).

Parameters:

digits (int) – Rounding digits for print (Default value = 3).

ovb_minimal_reporting(format='html', digits=3, display=True)[source]#

ovb_minimal_reporting returns the LaTeX/HTML code for a table summarizing the sensemakr object.

This function takes as input a sensemakr object, the digit to round number, one of the format type “latex” or “html”, and a boolean whether to display the output or not. The default is round 3 digits, ‘html’ format and display the table.

Parameters:

  • format (string) – Either “latex” or “html” (Default value = ‘html’).

  • display (boolean) – Default is True, to display the table.

  • digits (int) – Rounding digit for the table (Default value = 3).

Returns:

LaTex/HTML code for creating the table summarizing the sensemakr object.

Return type:

string

Examples

>>> # Load example dataset:
>>> import sensemakr as smkr
>>> darfur = smkr.load_darfur()
>>> # Fit a statsmodels OLSResults object ("fitted_model"):
>>> import statsmodels.formula.api as smf
>>> model = smf.ols(formula='peacefactor ~ directlyharmed + age + farmer_dar + herder_dar + pastvoted + hhsize_darfur + female + village', data=darfur)
>>> fitted_model = model.fit()
>>> # Runs sensemakr for sensitivity analysis
>>> sensitivity = smkr.Sensemakr(model=fitted_model, treatment = "directlyharmed", q=1.0, alpha=0.05, reduce=True)
>>> # Gets HTML code and table
>>> result=sensitivity.ovb_minimal_reporting() 
>>> # Prints raw html code
>>> print(result)