2020

Spatially-coupled hidden Markov models for short-term forecasting of wind speeds

A seminar by Vianey Leos Barajas, Assistant Professor at the University of Toronto, Dept. of Statistical Sciences and School of the Environment

November 18, 2020

Watch Recording of Vianey Leos Barajas seminar

Abstract

Hidden Markov models (HMMs) provide a flexible framework to model time series data where the observation process, Yt, is taken to be driven by an un-derlying latent state process, Zt. In this talk, we will focus on discrete-time, finite-state HMMs as they provide a flexible framework that facilitates extending the basic structure in many interesting ways.

HMMs can accommodate multivariate processes by (i) assuming that a single state governs the M observations at time t, (ii) assuming that each observation process is governed by its own HMM, irrespective of what occurs elsewhere, or (iii) a balance between the two, as in the coupled HMM framework. Coupled HMMs assume that a collection of M observation processes is governed by its respective M state processes. However, the mth state process at time t, Zm,t not only depends on Zm,t−1 but also on the collection of state processes Z−m,t−1. We introduce spatially-coupled hidden Markov models whereby the state processes interact according to an imposed neighborhood structure and the observations are collected across S spatial locations. We outline an application to short-term forecasting of wind speed using data collected across multiple wind turbines at a wind farm.

Digital Trace Data: Modes of Data Collection, Applications, and Errors

A seminar by Frauke Kreuter, Professor of Statistics and Data Science at the Ludwig-Maximilians-University of Munich

October 28, 2020

Watch Recording of Frauke Kreuter's PRIISM seminar

Abstract

Digital traces, left by individuals when they act or interact online provide researchers with new opportunities for studying social and behavioral phenomena. This talk covers digital trace data and their use in the computational social sciences. Key to a successful use of digital trace data is a clear vision of the research goals. Knowing how to match available data and research needs is just as important as the evaluation of data quality, and respecting respondents privacy. We will discuss inferential challenges and possible ways to deal with them, finding the right measures to ensure reproducibility and replicability, and how to create sufficient transparency when working with digital trace data. 

Bayesian Canonicalization of Voter Registration Files 

A seminar by Andee Kaplan, Assistant Professor, Colorado State University

October 14, 2020

Watch Recording of PRIISM Seminar with Andee Kaplan

Abstract

Entity resolution (record linkage or de-deduplication) is the process of merging noisy databases to remove duplicate entities in the absence of a unique identifier. One major challenge of utilizing linked data is identifying the canonical (or representative) records without duplicate information to pass to an inferential downstream task. The canonicalization step is particularly crucial after entity resolution, as a multi-stage approach allows for multiple analyses to be performed on the same linked data. While this approach can be scalable, the uncertainty from each stage of the entity resolution process is not naturally propagated throughout the pipeline and into the downstream task. In this talk, Dr. Kaplan presented five fully unsupervised methods to choose canonical records from linked data, including a fully Bayesian approach which propagates the error from linkage through to the downstream inference. This multi-stage approach is illustrated and evaluated on simulated entity resolution data sets as well as voter registration data available from the North Carolina State Board of Elections (NCSBE). The NCSBE has released a snapshot of their voter registration databases regularly since 2005, providing a changing view of the voter registration information over time as new voters register, voters are dropped from the register, and voter information is updated. Dr. Kaplan and her team compared the proposed canonicalization methods after performing entity resolution on five snapshots and examined the relationship between demographic information and party affiliation on the resulting canonical data sets.

A unified framework for the latent variable approach to statistical social network analysis

A seminar by Samrachana Adhikari, Assistant Professor, NYU School of Medicine

September 30, 2020

Watch Recording of PRIISM Seminar with Samrachana Adhikari

Abstract

While social network data provide new opportunities to understand complex relational mechanisms, they also present modeling challenges. Units of observation in social network are often not independent and identically distributed, as commonly assumed in many statistical models, and hence require new tools to analyze the data, to make inference and address issues of model selection and goodness of fit, while accounting for the complex dependence structures. Many recent developments have been made in statistical methodologies to account for such complications. In particular, latent variable network models that accommodate edge correlations implicitly, by assuming an underlying latent factor, are increasing in popularity. Although these models are examples of what is a growing body of research, much of the research is focused on proposing new models or extending others. There has been very little work on unifying the models in a single framework.

In this talk, Dr. Adhikari first reviewed different latent variable network models for analyzing social network data. She then introduced a complete framework that organizes existing latent variable network models within an integrative generalized additive model, called the Conditionally Independent Dyad (CID) models. The class of CID models includes existing network models that assume dyad (or edge) independence conditional on latent variables and other components in the model.  By presenting analysis of advice seeking network of teachers as an example, she illustrated the utility of the proposed framework. Dr. Adhikari ended with discussion of existing and future extensions of the proposed class of network models to incorporate multiple related networks.

Understanding reasons for differences in intervention effects across sites

A seminar by Kara Rudolph, Assistant Professor, Columbia University

September 16, 2020

Watch Recording of Kara Rudolph's seminar

Abstract

Multi-site interventions are common in public health, public policy, and economics. Do we expect an intervention effect in one site to be the same as the intervention effect in another site? In many cases, we would answer “no”. First, there could be differences in site-level variables related to intervention design/implementation or contextual variables, like the economy, that would modify intervention effectiveness. Such variables suggest that the intervention either is not the same or does not work the same in the two sites. Second, there could be differences in person-level variables—population composition—across sites that also modify intervention effectiveness. There could also be differences in the mechanisms producing intermediate variables on the pathway from the intervention to the outcome. The latter two reasons could cause intervention effects to differ across sites even if the interventions are structured and implemented in an identical fashion. An example of this, which we use to motivate this work, is from the Moving to Opportunity (MTO) trial. MTO is a five-site, encouragement-design intervention in which families in public housing were randomized to receive housing vouchers and logistical support to move to low-poverty neighborhoods. When we started this work, there had been no quantitative examination of the underlying reasons for differences in MTO’s effects across sites. We propose doubly robust and efficient estimators to predict total, direct, and indirect effects of treatment in a new site based on data in source sites. The extent to which these predicted estimates correspond with the observed estimates can shed light on reasons for site differences in intervention effects.

Inferential LASSO in Single Case Experimental Design to Estimate Effect Size

A seminar by Jay Verkulien, Associate Professor, CUNY

February 26, 2020

Abstract

Single case experimental design (SCED) is widely used in areas such as behavior modification, rehabilitation medicine, or training of special participants. The general goal in most analyses of SCED data is to assess the effect size of the intervention, often for subsequent processing in, for example, a meta-analysis. The design itself is a strong one for this purpose, particularly randomized multiple baseline designs (MBD). However, SCEDs tend to be fairly small N designs due to the nature of the target population and intervention. This means that regression analysis of SCED data is already running fairly lean in terms of error df even before one factors in covariates needed to model trend, external events, or other nuisance data features. These additional variables are not typically of substantive interest but need to be accounted for properly. Linear and generalized linear models do not cope with this situation well and fail completely when the number of predictors, P, exceeds N. Many machine learning methods have been devised to address this problem, which occurs in other areas such as genetic microarray studies, text mining, and neuroscience. These methods all incorporate some kind of regularization to provide enough structure to allow estimation to proceed. They are also useful even when N > P to avoid overfitting or to help determine if a visual object on a graph is "real" or not. However, they are fundamentally predictive in nature and do not work well when the goal involves interpreting the resulting regression components. Recent research has focused on adapting methods such as tree-based analyses or regularized regression to causal effect estimation. While the talk provides a conceptual overview to the problem of high dimension broadly, I focus on the inferential LASSO (e.g., Belloni, Chernozhukov, & Hansen, 2014) as a promising new method that is a straightforward extension of multiple regression. (Joint work with Mariola Moeyaert, SUNY Albany).

Measuring Poverty

A seminar by Chaitra Nagaraja, Associate Professor, Fordham University

February 12, 2020

Abstract

The development of any measure, particularly an official one, is a mixture of ideology, convenience, and chance. To illustrate this, I will use poverty measurement as an example. The primary choice in measuring poverty is between taking an absolute versus a relative approach. The former is favored by the U.S. whereas the latter is preferred by the European Union. In this talk, I will compare various measures (with a focus on the U.S.) using a historical perspective to understand the effects of those statistics on a country’s residents.

2018

Decision-driven sensitivity analyses via Bayesian optimization

A seminar by Russell Steele, Associate Professor, McGill

December 5, 2018

Abstract

Every statistical analysis requires at least some subjective or untestable assumptions. For example, in Bayesian modelling, the analysis requires specification of hyperparameters for prior distributions which are either intended to reflect subjective beliefs about the model or to reflect relative ignorance about the model under a certain notion of ignorance. Similarly, causal models require assumptions about parameters related to unmeasured confounding. Violations of these untestable or subjective assumptions can invalidate the conclusions of analyses or lead to conclusions that only hold for a narrow range of choices for those assumptions. Currently, researchers compute several estimates based on either multiple “reasonable” values or a wide range of “possible” values for these inestimable parameters. Even when the dimension of the inestimable parameter space is relatively small, the sensitivity analyses generally are not systematically conducted and may either waste valuable computational time on choices that lead to roughly the same inference or will miss examining values of those parameters that would change the conclusions of the analysis.

In this talk, I will propose the use of Bayesian optimization approaches for decision-driven sensitivity analyses. We assume that a decision will be made as a function of the model estimates or predictions from particular model which relies on inestimable parameters. We use a Bayesian optimization approach to identify partitions of the space of inestimable parameter values where the decision based on the observed data and assumed parameter values change, rather to rely on non-systematically chosen values for the sensitivity analysis. We will illustrate our proposed approach on a hierarchical Bayesian meta-analysis example from the literature.

The work that will be presented was done in collaboration with Louis Arsenault-Mahjoubi, an undergraduate mathematics and statistics student at McGill University.

Omitted and included variable bias in tests for disparate impact

A seminar by Ravi Shroff, Assistant Professor of Applied Statistics, New York University 

November 14, 2018

Abstract

Policymakers often seek to gauge discrimination against groups defined by race, gender, and other protected attributes. A common strategy is to estimate disparities after controlling for observed covariates in a regression model. However, not all relevant factors may be available to researchers, leading to omitted variable bias. Conversely, controlling for all available factors may also skew results, leading to so-called "included variable bias". We introduce a simple strategy, which we call risk-adjusted regression, that addresses both concerns in settings where decision makers have clear and measurable policy objectives. First, we use all available covariates to estimate the expected utility of possible decisions. Second, we measure disparities after controlling for these utility estimates alone, omitting other factors. Finally, we examine the sensitivity of results to unmeasured confounding. We demonstrate this method on a detailed dataset of 2.2 million police stops of pedestrians in New York City.

Structural Equation Modeling in Stata

A seminar by Chuck Huber, Director of Statistical Outreach, Stata Corp

October 31, 2018

Co-sponsored with CUNY Grad Center EPSY

Abstract

This talk introduces the concepts and jargon of structural equation modeling (SEM) including path diagrams, latent variables, endogenous and exogenous variables, and goodness of fit. I demonstrate how to fit many familiar models such as linear regression, multivariate regression, logistic regression, confirmatory factor analysis, and multilevel models using -sem-. I wrap up by demonstrating how to fit structural equation models that contain both structural and measurement components. *

Adaptive Designs in Clinical Trials: An Introduction and Example

A seminar by Leslie McClure, Professor, Chair of the Department of Epidemiology and Biostatistics, and Associate Dean for Faculty Affairs, Drexel

October 24, 2018

Abstract

Planning for randomized clinical trials relies on assumptions that are often incorrect, leading to inefficient designs that could spend resources unnecessarily. Recently, trialists have been advocating for implementation of adaptive designs, which allow researchers to modify some aspect of their trial part-way through the study based on accumulating data. In this talk, I will introduce the concept of adaptive designs and describe several different adaptations that can be made in clinical trials. I will then describe a real-life example of a sample size re-estimation from the Secondary Prevention of Small Subcortical Strokes (SPS3) study, describe the statistical impact of implementing this design change, and describe the effect of the adaptation on the practical aspects of the study.

Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India

A seminar by Alejandro Ganimian, Assistant Professor of Applied Psychology and Economics, New York University 

May 2, 2018

Abstract

We present experimental evidence on the impact of a personalized technology-aided after-school instruction program on learning outcomes. Our setting is middle-school grades in urban India, where a lottery provided winning students with a voucher to cover program costs. We find that lottery winners scored 0.36σ higher in math and 0.22σ higher in Hindi relative to lottery losers after just 4.5-months of access to the program. IV estimates suggest that attending the program for 90 days would increase math and Hindi test scores by 0.59σ and 0.36σ respectively. We find similar absolute test score gains for all students, but the relative gain was much greater for academically-weaker students because their rate of learning in the control group was close to zero. We show that the program was able to effectively cater to the very wide variation in student learning levels within a single grade by precisely targeting instruction to the level of student preparation. The program was cost effective, both in terms of productivity per dollar and unit of time. Our results suggest that well-designed technology-aided instruction programs can sharply improve productivity in delivering education.

BART for Causal Inference

A seminar by Jennifer Hill, Professor of Applied Statistics and Co-Director of PRIISM, New York University

April 28, 2018

Abstract

There has been increasing interest in the past decade in use of machine learning tools in causal inference to help reduce reliance on parametric assumptions and allow for more accurate estimation of heterogeneous effects. This talk reviews the work in this area that capitalizes on Bayesian Additive Regression Trees, an algorithm that embeds a tree-based machine learning technique within a Bayesian framework to allow for flexible estimation and valid assessments of uncertainty. It will further describe extensions of the original work to address common issues in causal inference: lack of common support, violations of the ignorability assumption, and generalizability of results to broader populations. It will also describe existing R packages for traditional BART implementation as well as debut a new R package for causal inference using BART, bartCause.

Simulating a Marginal Structural Model

A seminar by Keith Goldfeld, Associate Professor, New York Univerity Langone Health

February 28, 2018

Abstract

In so many ways, simulation is an extremely useful tool to learn, teach, and understand the theory and practice of statistics. A series of examples (interspersed with minimal theory) will hopefully illuminate the underbelly of confounding, colliding, and marginal structural models. Drawing on the potential outcomes framework, the examples will use the R simstudy package, a tool that is designed to make data simulation as painless as possible.

Graphs as Poetry

A seminar by Howard Wainer, Research Scientist,  National Board of Medical Examiners

February 7, 2018

Abstract

Visual displays of empirical information are too often thought to be just compact summaries that, at their best, can clarify a muddled situation. This is partially true, as far as it goes, but it omits the magic. We have long known that data visualization is an alchemist that can make good scientists great and transform great scientists into giants. In this talk we will see that sometimes, albeit too rarely, the combination of critical questions addressed by important data and illuminated by evocative displays can achieve a transcendent, and often wholly unexpected, result. At their best, visualizations can communicate emotions and feelings in addition to cold, hard facts.

2016

Large, Sparse Optimal Matching in an Observational Study of Surgical Outcomes

Abstract: How do health outcomes for newly-trained surgeons' patients compare with those for patients of experienced surgeons? To answer this question using data from Medicare, we introduce a new form of matching that pairs patients of 1252 new surgeons to patients of experienced surgeons, exactly balancing 176 surgical procedures and closely balancing 2.9 million finer patient categories. The new matching algorithm (which uses penalized network flows) exploits a sparse network to quickly optimize a match two orders of magnitude larger than usual in statistical matching, and allowing for extensive use of a new form of marginal balance constraint.

Generalized Ridge Regression Using an Iterative Solution

Speaker: Kathryn, postdoc at Columbia University's Earth Institute. Her PhD is in applied economics with interests in development economics, and applied statistics.

Abstract

An iterative method is introduced for solving noisy, ill-conditioned inverse problems, where the standard ridge regression is just the first iteration of the iterative method to be presented. In addition to the regularization parameter, lambda, we introduce an iteration parameter k, which generalizes the ridge regression. The derived noise damping filter is a generalization of the standard ridge regression filter (also known as Tikhonov). Application of the generalized solution performs better than the pseudo-inverse (the default solution to OLS in most statistical packages), and better than standard ridge regression (L-2 regularization), when the covariate matrix or design matrix is ill-conditioned, or highly collinear. A few examples are presented using both simulated and real data.

Latent Space Models for Affiliation Networks

Catherine (“Kate”) Calder, professor of statistics, The Ohio State University

Abstract

An affiliation network is a particular type of two-mode social network that consists of a set of `actors' and a set of `events' where ties indicate an actor's participation in an event. Methods for the analysis of affiliation networks are particularly useful for studying patterns of segregation and integration in social structures characterized by both people and potentially shared activities (e.g., parties, corporate board memberships, church attendance, etc.) One way to analyze affiliation networks is to consider one-mode network matrices that are derived from an affiliation network, but this approach may lead to the loss of important structural features of the data. The most comprehensive approach is to study both actors and events simultaneously. Statistical methods for studying affiliation networks, however, are less well developed than methods for studying one-mode, or actor-actor, networks. In this talk, I will describe a bilinear generalized mixed-effects model, which contains interacting random effects representing common activity pattern profiles and shared patterns of participation in these profiles. I will demonstrate how the proposed model is able to capture forth-order dependence, a common feature of affiliation networks, and describe a Markov chain Monte Carlo algorithm for Bayesian inference. I then will use the latent space interpretation of model components to explore patterns in extracurricular activity membership of students in a racially-diverse high school in a Midwestern metropolitan area. Using techniques from spatial point pattern analysis, I will show how our model can provide insight into patterns of racial segregation in the voluntary extracurricular activity participation profiles of adolescents. This talk is based on joint work with Yanan Jia and Chris Browning.

Why so many research hypotheses are mostly false and how to test

Paul De Boeck, professor of quantitative psychology, The Ohio State University

Abstract

From a recent Science article with a large number of replications of psychological studies the base rate of the null hypothesis of no effect can be estimated. It turns out to be extremely high, which implies that many research hypotheses are false. As I will explain they are perhaps not fully false but mostly false. A possible explanation for why unlikely hypotheses tend to be selected for empirical studies can be found in expected utility theory. It can be shown that for low to moderately high power rates, the expected utility of studies increases with the probability of the null hypothesis being true. A high probability of the null hypothesis being true can be understood as reflecting a contextual variation of effects that are in general not much different from zero. Increasing the power of studies has become a popular remedy to counter the replicability crisis but this strategy is highly misleading if effects vary. Meta-analysis is considered another remedy but it is a suboptimal and labor-intensive approach and it is only long-term method. Two more feasible methods will be discussed to deal with contextual variation.

Be the Data and More: Using interactive, analytic methods to enhance learning from data for students

Leanna House, Associate Professor of Statistics, Virginia Tech

The Ohio State University

Abstract

Datasets, no matter how big, are just tables of numbers without individuals to learn from the data, i.e., discover, process, assess, and communicate information in the data. Data visualizations are often used to present data to individuals, but most are created independently of human learning processes and lack transparency. To bridge the gap between people thinking critically about data and the utility of visualizations, we developed Bayesian Visual Analytics (BaVA) and its deterministic form, Visual to Parametric Interaction (V2PI). BaVA and V2PI transform static images of data to dynamic versions that respond to expert feedback. When applied iteratively, experts may explore data progressively in a sequence that parallels their personal sense-making processes. BaVA and V2PI have shown useful in both industry settings and the classroom. For example, we merged V2PI with motion detection software to create Be the Data. In Be the Data students physically move in a space to communicate their expert feedback about data projected overhead. The idea is that participants have an opportuntity to explore analytical relationships between data points by exploring relationships between themselves. This talk will focus on presenting the BaVA paradigm and its education applications.

Bayesian Inference and Stan Tutorial

Vincent Dorie, Postdoctoral Researcher, NYU PRIISM 

Abstract

This two hour session is focused on getting started with Stan and how to use it in your research. Stan is an open-source Bayesian probabilistic programming environment that takes a lot of the work out of model fitting so that researchers can focus on model building and interpretation. List of topics will include: overview of Bayesian statistics, overview of Stan and MCMC, writing models in Stan, and a tutorial session where participants can write a model on their own or develop models that they have been working on independently. Stan has interfaces to numerous programming languages, but the talk will focus on R.

Basing Causal Inferences about Policy Impacts on Non-Representative Samples of Sites – Risks, Consequences, and Fixes

Stephen Bell, Abt Associates Fellow 

Abstract

Randomized impact evaluations of social and educational interventions—while constituting the “gold standard” of internal validity due to the lack of selection bias between treated and untreated cases—usually lack external validity. Due to cost and convenience, or local resistance, they are almost always conducted in a set of sites that are not a probability sample of the desired inference population— the nation as a whole for social programs or a given state or school district for educational innovations. We use statistical theory and data from the Reading First evaluation to examine the risks and consequences for social experiments of non-representative site selection, asking when and to what degree policy decisions are led astray by tarnished “gold standard” evidence. We also explore possible ex ante design-based solutions to this problem and the performance of ex post methods in the literature for overcoming non-representative site selection through analytic adjustments after the fact.

Mediation: From Intuition to Data Analysis

Ilya Shpitser, Assistant Professor in the Department of Computer Science, Johns Hopkins University. 

Abstract

Modern causal inference links the "top-down" representation of causal intuitions and "bottom-up" data analysis with the aim of choosing policy. Two innovations that proved key for this synthesis were a formalization of Hume's counterfactual account of causation using potential outcomes (due to Jerzy Neyman), and viewing cause effect relationships via directed acyclic graphs (due to Sewall Wright). I will briefly review how a synthesis of these two ideas was instrumental in formally representing the notion of "causal effect" as a parameter in the language of potential outcomes, and discuss a complete identification theory linking these types of causal parameters and observed data, as well as approaches to estimation of the resulting statistical parameters. I will then describe, in more detail, how my collaborators and I are applying the same approach to mediation, the study of effects along particular causal pathways. I consider mediated effects at their most general: I allow arbitrary models, the presence of hidden variables, multiple outcomes, longitudinal treatments, and effects along arbitrary sets of causal pathways. As was the case with causal effects, there are three distinct but related problems to solve -- a representation problem (what sort of potential outcome does an effect along a set of pathways correspond to), an identification problem (can a causal parameter of interest be expressed as a functional of observed data), and an estimation problem (what are good ways of estimating the resulting statistical parameter). I report a complete solution to the first two problems, and progress on the third. In particular, my collaborators and I show that for some parameters that arise in mediation settings, triply robust estimators exist, which rely on an outcome model, a mediator model, and a treatment model, and which remain consistent if any two of these three models are correct. Some of the reported results are a joint work with Eric Tchetgen Tchetgen, Caleb Miles, Phyllis Kanki, and Seema Meloni.

Bayes vs Maximum Likelihood: The case of bivariate probit models

Adriana Crespo-Tenorio, PhD is on a mission to connect people’s online behavior to their offline lives. 

Abstract

Bivariate probit models are a common choice for scholars wishing to estimate causal effects in instrumental variable models where both the treatment and outcome are binary. However, standard maximum likelihood approaches for estimating bivariate probit models are problematic. Numerical routines in common software suites frequently generate inaccurate parameter estimates, and even estimated correctly, maximum likelihood routines provide no straightforward way to produce estimates of uncertainty for causal quantities of interest. In this article, we show that adopting a Bayesian approach provides more accurate estimates of key parameters and facilitates the direct calculation of causal quantities along with their attendant measures of uncertainty.

Scalable Bayesian Inference with Hamiltonian Monte Carlo

Michael Betancourt, Postdoctoral Research Associate, Warwick

Abstract

The modern preponderance of data has fueled a revolution in data science, but the complex nature of those data also limits naive inferences. To truly take advantage of these data we also need tools for building and fitting statistical models that capture those complexities. In this talk I’ll discuss some of the practical challenges of building and fitting such models in the context of real analyses. I will particularly emphasize the importance of Hamiltonian Monte Carlo and Stan, state-of-the-art computational tools that allow us to tackle these contemporary data without sacrificing the fidelity of our inferences.

Improving Human Learning with Unified Machine Learning Frameworks: Towards Faster, Better, and Less Expensive Education

José González-Brenes, Pearson

Abstract

Seminal results from cognitive science suggest that personalized education is effective to improve learners’ outcomes. However, the effort for instructors to create content for each of their students can sometimes be prohibitive. Recent progress in machine learning has enabled technology for teachers to deliver personalized education. Unfortunately, the statistical models used by these systems are often tailored for ad-hoc domains and do not generalize across applications. In this talk, I will discuss my work towards the goal of a unified statistical framework of human learning. This line of work is more flexible, more efficient, and more accurate than previous technology. Moreover, it generalizes previous popular models from the literature. Additionally, I will outline recent progress on novel methodology to evaluate statistical models for education with a learner-centric perspective. My findings suggest that prior work often uses evaluation methods that may misrepresent the educational value of educational systems. My work is a promising alternative that improves the evaluation of machine learning models in education.

Probabilistic Cause-of-death Assignment using Verbal Autopsies

Tyler McCormick, University of Washington, Seattle

Abstract

In regions without complete-coverage civil registration and vital statistics systems there is uncertainty about even the most basic demographic indicators. In such areas the majority of deaths occur outside hospitals and are not recorded. Worldwide, fewer than one-third of deaths are assigned a cause, with the least information available from the most impoverished nations. In populations like this, verbal autopsy (VA) is a commonly used tool to assess cause of death and estimate cause-specific mortality rates and the distribution of deaths by cause. VA uses an interview with caregivers of the decedent to elicit data describing the signs and symptoms leading up to the death. This paper develops a new statistical tool known as InSilicoVA to classify cause of death using information acquired through VA. InSilicoVA shares uncertainty between cause of death assignments for specific individuals and the distribution of deaths by cause across the population. Using side-by-side comparisons with both observed and simulated data, we demonstrate that InSilicoVA has distinct advantages compared to currently available methods.

Topic-adjusted visibility metric for scientific articles

Tian Zheng, Columbia University. 

Abstract

Measuring the impact of scientific articles is important for evaluating the research output of individual scientists, academic institutions and journals. While citations are raw data for constructing impact measures, there exist biases and potential issues if factors affecting citation patterns are not properly accounted for. In this talk, I present a new model that aims to address the problem of field variation and introduce an article level metric useful for evaluating individual articles’ topic-adjusted visibility. This measure derives from joint probabilistic modeling of the content in the articles and the citations amongst them using latent Dirichlet allocation (LDA) and the mixed membership stochastic blockmodel (MMSB). This proposed model provides a visibility metric for individual articles adjusted for field variation in citation rates, a structural understanding of citation behavior in different fields, and article recommendations which take into account article visibility and citation patterns. For this work, we also developed an efficient algorithm for model fitting using variational methods. To scale up to large networks, we developed an online variant using stochastic gradient methods and case-control likelihood approximation. Results from an application of our methods to the benchmark KDD Cup 2003 dataset with approximately 30,000 high energy physics papers will also be presented.

Small sample adjustments to F-tests for cluster robust standard errors

Elizabeth Tipton, Teachers College, Columbia University

Abstract

Data analysts commonly ‘cluster’ their standard errors to account for correlations arising from the sampling of aggregate units (e.g., states), each containing multiple observations. When the number of clusters is small to moderate, however, this approach can lead to biased standard errors and hypothesis tests with inflated Type I error. One solution that is receiving increased attention is the use of the bias-reduced linearization (BRL). In this paper, we extend the BRL approach to include an F-test that can be implemented in a wide range of applications. A simulation study reveals that that this test has Type I error close to nominal even with a very small number of clusters, and importantly, that it outperforms the usual estimator even when the number of clusters is moderate.

The Controversies over Null Hypothesis Testing and Replication

Barry Cohen, New York University

Abstract

The arguments against null hypothesis significance testing (NHST) have been greatly exaggerated, and do not apply equally to all types of psychological research. I will discuss the conditions under which NHST serves several useful purposes, which may outweigh its undeniable drawbacks. In brief, NHST works best when the null hypothesis is rarely true, the direction of the results is more important than the magnitude, extremely large samples are not used, and tiny effects have no serious consequences. Priming studies in social psychology will be used as an example of this type of research. Part of the controversy over failures to replicate notable psychological studies is related to misunderstandings and misuses of NHST. I will conclude by discussing the resistance to banning NHST and its p values in favor of reports of effects sizes and/or confidence intervals, and describing some of the possible solutions to the drawbacks of NHST.

The Curious Case of the Instrumental Variable Estimator for the Complier Average Causal Effect

Russell Steele, McGill University

Abstract

In randomized clinical trials, subjects often do not comply with their randomized treatment arm. Although one can still unbiasedly estimate the causal effect of being assigned to treatment using the common Intention-to-Treat (ITT) estimator, there is now potential confounding of the causal effect of actually *receiving* treatment. Basic alternative estimators such as the per protocol or as treated estimators have been used, but are generally biased for estimating the causal effect of interest. Balke and Pearl (1997) and Angrist, et al. (1996) independently proposed an instrumental variable (IV) estimator that would estimate the causal effect (the Complier Average Causal Effect — CACE) of receiving treatment in a subpopulation of people who would comply with treatment assignment (i.e. the compliers). In this talk, I will first review the CACE and the IV estimator. I will then dissect the instrumental variable estimator in order to compare it to the per protocol and as treated estimators. I will show that the basic IV estimator and its confidence interval can be computed from basic summary statistics that should be reported in any randomized trial. My formulation of the IV estimator will also allow for simple sensitivity analyses that can be done using a basic Excel spreadsheet. I will then describe future interesting directions for compliance research that I am currently working on. Most of this work appears in a recently published article at the American Journal of Epidemiology and is co-authored by Ian Shrier, Jay Kaufmann and Robert Platt.

Covariate Selection with Observational Data: Simulation Results and Discussion

Bryan Keller, Teachers College, Columbia University

Abstract

In an effort to protect against omitted variable bias, statisticians have traditionally favored an inclusive approach to covariate selection for causal inference, so long as covariates were measured before any treatment was administered. There are, however, three classes of variables, which, if conditioned upon, are known to degrade either the bias or efficiency of an estimate of a causal effect: non-informative variables (NVs), instrumental variables (IVs), and collider variables. The decision about whether to control for a potential collider variable must be based on theory about how the data were generated. In contrast, one need only establish a lack of association with the outcome variable in order to identify an NV or an IV. We investigate three empirical methods – forward stepwise selection, the lasso, and recursive feature elimination with random forests – for detection of NVs and IVs through simulation studies in which we judge their efficacy by (a) sensitivity and specificity in identifying true or near NVs and IVs and (b) the overall effect on bias and mean-squared error of the causal effect estimator, relative to inclusion of all pretreatment variables. Results and implications are discussed.

The End of Intelligence? What might Big Data, Learning Analytics and the Information Age Mean for how we Measure Education

Charles Lang, Postdoctoral Associate, ew York University  

Abstract

For over a century educational measurement has developed analytical tools designed to maximize the inferential power of limited samples: a biannual state test, a regular accreditation exam, a once in a lifetime SAT. But can this methodology adapt to a world in which previous limitations on data collection have been dramatically reduced? A world with a greater variety of data formats, representing a larger number of conditions, on a finer timescale, with a larger sample of students. Starting from a methodological basis, Charles will discuss the implications that changes in data collection may have on how education is measured and the impact that this might have on the disciplines, institutions, and practitioners that utilize educational measurement.

Studying Change with Difference Scores versus ANCOVA: Issues, Perspectives and Advances

Pat Shrout, New York University 

Abstract

Nearly 50 years ago, Lord (1967) described a so-called paradox in statistical analysis whereby two reasonable analyses of pre-treatment/post-treatment data lead to different results. I revisit the issues, review some of the historical discussion, and present an analysis of the alternate analyses with a causal model that distinguishes treatment effects from trait, state, and error variation. In addition to comparing numerical results from difference score and ANCOVA adjustment for pre-treatment group differences, I consider results based on propensity score adjustment.