2020

Spatially-coupled hidden Markov models for short-term forecasting of wind speeds

A seminar by Vianey Leos Barajas, Assistant Professor at the University of Toronto, Dept. of Statistical Sciences and School of the Environment

November 18, 2020

Watch Recording of Vianey Leos Barajas seminar

Abstract

Hidden Markov models (HMMs) provide a flexible framework to model time series data where the observation process, Yt, is taken to be driven by an un-derlying latent state process, Zt. In this talk, we will focus on discrete-time, finite-state HMMs as they provide a flexible framework that facilitates extending the basic structure in many interesting ways.

HMMs can accommodate multivariate processes by (i) assuming that a single state governs the M observations at time t, (ii) assuming that each observation process is governed by its own HMM, irrespective of what occurs elsewhere, or (iii) a balance between the two, as in the coupled HMM framework. Coupled HMMs assume that a collection of M observation processes is governed by its respective M state processes. However, the mth state process at time t, Zm,t not only depends on Zm,t−1 but also on the collection of state processes Z−m,t−1. We introduce spatially-coupled hidden Markov models whereby the state processes interact according to an imposed neighborhood structure and the observations are collected across S spatial locations. We outline an application to short-term forecasting of wind speed using data collected across multiple wind turbines at a wind farm.

Digital Trace Data: Modes of Data Collection, Applications, and Errors

A seminar by Frauke Kreuter, Professor of Statistics and Data Science at the Ludwig-Maximilians-University of Munich

October 28, 2020

Watch Recording of Frauke Kreuter's PRIISM seminar

Abstract

Digital traces, left by individuals when they act or interact online provide researchers with new opportunities for studying social and behavioral phenomena. This talk covers digital trace data and their use in the computational social sciences. Key to a successful use of digital trace data is a clear vision of the research goals. Knowing how to match available data and research needs is just as important as the evaluation of data quality, and respecting respondents privacy. We will discuss inferential challenges and possible ways to deal with them, finding the right measures to ensure reproducibility and replicability, and how to create sufficient transparency when working with digital trace data. 

Bayesian Canonicalization of Voter Registration Files 

A seminar by Andee Kaplan, Assistant Professor, Colorado State University

October 14, 2020

Watch Recording of PRIISM Seminar with Andee Kaplan

Abstract

Entity resolution (record linkage or de-deduplication) is the process of merging noisy databases to remove duplicate entities in the absence of a unique identifier. One major challenge of utilizing linked data is identifying the canonical (or representative) records without duplicate information to pass to an inferential downstream task. The canonicalization step is particularly crucial after entity resolution, as a multi-stage approach allows for multiple analyses to be performed on the same linked data. While this approach can be scalable, the uncertainty from each stage of the entity resolution process is not naturally propagated throughout the pipeline and into the downstream task. In this talk, Dr. Kaplan presented five fully unsupervised methods to choose canonical records from linked data, including a fully Bayesian approach which propagates the error from linkage through to the downstream inference. This multi-stage approach is illustrated and evaluated on simulated entity resolution data sets as well as voter registration data available from the North Carolina State Board of Elections (NCSBE). The NCSBE has released a snapshot of their voter registration databases regularly since 2005, providing a changing view of the voter registration information over time as new voters register, voters are dropped from the register, and voter information is updated. Dr. Kaplan and her team compared the proposed canonicalization methods after performing entity resolution on five snapshots and examined the relationship between demographic information and party affiliation on the resulting canonical data sets.

A unified framework for the latent variable approach to statistical social network analysis

A seminar by Samrachana Adhikari, Assistant Professor, NYU School of Medicine

September 30, 2020

Watch Recording of PRIISM Seminar with Samrachana Adhikari

Abstract

While social network data provide new opportunities to understand complex relational mechanisms, they also present modeling challenges. Units of observation in social network are often not independent and identically distributed, as commonly assumed in many statistical models, and hence require new tools to analyze the data, to make inference and address issues of model selection and goodness of fit, while accounting for the complex dependence structures. Many recent developments have been made in statistical methodologies to account for such complications. In particular, latent variable network models that accommodate edge correlations implicitly, by assuming an underlying latent factor, are increasing in popularity. Although these models are examples of what is a growing body of research, much of the research is focused on proposing new models or extending others. There has been very little work on unifying the models in a single framework.

In this talk, Dr. Adhikari first reviewed different latent variable network models for analyzing social network data. She then introduced a complete framework that organizes existing latent variable network models within an integrative generalized additive model, called the Conditionally Independent Dyad (CID) models. The class of CID models includes existing network models that assume dyad (or edge) independence conditional on latent variables and other components in the model.  By presenting analysis of advice seeking network of teachers as an example, she illustrated the utility of the proposed framework. Dr. Adhikari ended with discussion of existing and future extensions of the proposed class of network models to incorporate multiple related networks.

Understanding reasons for differences in intervention effects across sites

A seminar by Kara Rudolph, Assistant Professor, Columbia University

September 16, 2020

Watch Recording of Kara Rudolph's seminar

Abstract

Multi-site interventions are common in public health, public policy, and economics. Do we expect an intervention effect in one site to be the same as the intervention effect in another site? In many cases, we would answer “no”. First, there could be differences in site-level variables related to intervention design/implementation or contextual variables, like the economy, that would modify intervention effectiveness. Such variables suggest that the intervention either is not the same or does not work the same in the two sites. Second, there could be differences in person-level variables—population composition—across sites that also modify intervention effectiveness. There could also be differences in the mechanisms producing intermediate variables on the pathway from the intervention to the outcome. The latter two reasons could cause intervention effects to differ across sites even if the interventions are structured and implemented in an identical fashion. An example of this, which we use to motivate this work, is from the Moving to Opportunity (MTO) trial. MTO is a five-site, encouragement-design intervention in which families in public housing were randomized to receive housing vouchers and logistical support to move to low-poverty neighborhoods. When we started this work, there had been no quantitative examination of the underlying reasons for differences in MTO’s effects across sites. We propose doubly robust and efficient estimators to predict total, direct, and indirect effects of treatment in a new site based on data in source sites. The extent to which these predicted estimates correspond with the observed estimates can shed light on reasons for site differences in intervention effects.

Inferential LASSO in Single Case Experimental Design to Estimate Effect Size

A seminar by Jay Verkulien, Associate Professor, CUNY

February 26, 2020

Abstract

Single case experimental design (SCED) is widely used in areas such as behavior modification, rehabilitation medicine, or training of special participants. The general goal in most analyses of SCED data is to assess the effect size of the intervention, often for subsequent processing in, for example, a meta-analysis. The design itself is a strong one for this purpose, particularly randomized multiple baseline designs (MBD). However, SCEDs tend to be fairly small N designs due to the nature of the target population and intervention. This means that regression analysis of SCED data is already running fairly lean in terms of error df even before one factors in covariates needed to model trend, external events, or other nuisance data features. These additional variables are not typically of substantive interest but need to be accounted for properly. Linear and generalized linear models do not cope with this situation well and fail completely when the number of predictors, P, exceeds N. Many machine learning methods have been devised to address this problem, which occurs in other areas such as genetic microarray studies, text mining, and neuroscience. These methods all incorporate some kind of regularization to provide enough structure to allow estimation to proceed. They are also useful even when N > P to avoid overfitting or to help determine if a visual object on a graph is "real" or not. However, they are fundamentally predictive in nature and do not work well when the goal involves interpreting the resulting regression components. Recent research has focused on adapting methods such as tree-based analyses or regularized regression to causal effect estimation. While the talk provides a conceptual overview to the problem of high dimension broadly, I focus on the inferential LASSO (e.g., Belloni, Chernozhukov, & Hansen, 2014) as a promising new method that is a straightforward extension of multiple regression. (Joint work with Mariola Moeyaert, SUNY Albany).

Measuring Poverty

A seminar by Chaitra Nagaraja, Associate Professor, Fordham University

February 12, 2020

Abstract

The development of any measure, particularly an official one, is a mixture of ideology, convenience, and chance. To illustrate this, I will use poverty measurement as an example. The primary choice in measuring poverty is between taking an absolute versus a relative approach. The former is favored by the U.S. whereas the latter is preferred by the European Union. In this talk, I will compare various measures (with a focus on the U.S.) using a historical perspective to understand the effects of those statistics on a country’s residents.

2018

Decision-driven sensitivity analyses via Bayesian optimization

A seminar by Russell Steele, Associate Professor, McGill

December 5, 2018

Abstract

Every statistical analysis requires at least some subjective or untestable assumptions. For example, in Bayesian modelling, the analysis requires specification of hyperparameters for prior distributions which are either intended to reflect subjective beliefs about the model or to reflect relative ignorance about the model under a certain notion of ignorance. Similarly, causal models require assumptions about parameters related to unmeasured confounding. Violations of these untestable or subjective assumptions can invalidate the conclusions of analyses or lead to conclusions that only hold for a narrow range of choices for those assumptions. Currently, researchers compute several estimates based on either multiple “reasonable” values or a wide range of “possible” values for these inestimable parameters. Even when the dimension of the inestimable parameter space is relatively small, the sensitivity analyses generally are not systematically conducted and may either waste valuable computational time on choices that lead to roughly the same inference or will miss examining values of those parameters that would change the conclusions of the analysis.

In this talk, I will propose the use of Bayesian optimization approaches for decision-driven sensitivity analyses. We assume that a decision will be made as a function of the model estimates or predictions from particular model which relies on inestimable parameters. We use a Bayesian optimization approach to identify partitions of the space of inestimable parameter values where the decision based on the observed data and assumed parameter values change, rather to rely on non-systematically chosen values for the sensitivity analysis. We will illustrate our proposed approach on a hierarchical Bayesian meta-analysis example from the literature.

The work that will be presented was done in collaboration with Louis Arsenault-Mahjoubi, an undergraduate mathematics and statistics student at McGill University.

Omitted and included variable bias in tests for disparate impact

A seminar by Ravi Shroff, Assistant Professor of Applied Statistics, New York University 

November 14, 2018

Abstract

Policymakers often seek to gauge discrimination against groups defined by race, gender, and other protected attributes. A common strategy is to estimate disparities after controlling for observed covariates in a regression model. However, not all relevant factors may be available to researchers, leading to omitted variable bias. Conversely, controlling for all available factors may also skew results, leading to so-called "included variable bias". We introduce a simple strategy, which we call risk-adjusted regression, that addresses both concerns in settings where decision makers have clear and measurable policy objectives. First, we use all available covariates to estimate the expected utility of possible decisions. Second, we measure disparities after controlling for these utility estimates alone, omitting other factors. Finally, we examine the sensitivity of results to unmeasured confounding. We demonstrate this method on a detailed dataset of 2.2 million police stops of pedestrians in New York City.

Structural Equation Modeling in Stata

A seminar by Chuck Huber, Director of Statistical Outreach, Stata Corp

October 31, 2018

Co-sponsored with CUNY Grad Center EPSY

Abstract

This talk introduces the concepts and jargon of structural equation modeling (SEM) including path diagrams, latent variables, endogenous and exogenous variables, and goodness of fit. I demonstrate how to fit many familiar models such as linear regression, multivariate regression, logistic regression, confirmatory factor analysis, and multilevel models using -sem-. I wrap up by demonstrating how to fit structural equation models that contain both structural and measurement components. *

Adaptive Designs in Clinical Trials: An Introduction and Example

A seminar by Leslie McClure, Professor, Chair of the Department of Epidemiology and Biostatistics, and Associate Dean for Faculty Affairs, Drexel

October 24, 2018

Abstract

Planning for randomized clinical trials relies on assumptions that are often incorrect, leading to inefficient designs that could spend resources unnecessarily. Recently, trialists have been advocating for implementation of adaptive designs, which allow researchers to modify some aspect of their trial part-way through the study based on accumulating data. In this talk, I will introduce the concept of adaptive designs and describe several different adaptations that can be made in clinical trials. I will then describe a real-life example of a sample size re-estimation from the Secondary Prevention of Small Subcortical Strokes (SPS3) study, describe the statistical impact of implementing this design change, and describe the effect of the adaptation on the practical aspects of the study.

Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India

A seminar by Alejandro Ganimian, Assistant Professor of Applied Psychology and Economics, New York University 

May 2, 2018

Abstract

We present experimental evidence on the impact of a personalized technology-aided after-school instruction program on learning outcomes. Our setting is middle-school grades in urban India, where a lottery provided winning students with a voucher to cover program costs. We find that lottery winners scored 0.36σ higher in math and 0.22σ higher in Hindi relative to lottery losers after just 4.5-months of access to the program. IV estimates suggest that attending the program for 90 days would increase math and Hindi test scores by 0.59σ and 0.36σ respectively. We find similar absolute test score gains for all students, but the relative gain was much greater for academically-weaker students because their rate of learning in the control group was close to zero. We show that the program was able to effectively cater to the very wide variation in student learning levels within a single grade by precisely targeting instruction to the level of student preparation. The program was cost effective, both in terms of productivity per dollar and unit of time. Our results suggest that well-designed technology-aided instruction programs can sharply improve productivity in delivering education.

BART for Causal Inference

A seminar by Jennifer Hill, Professor of Applied Statistics and Co-Director of PRIISM, New York University

April 28, 2018

Abstract

There has been increasing interest in the past decade in use of machine learning tools in causal inference to help reduce reliance on parametric assumptions and allow for more accurate estimation of heterogeneous effects. This talk reviews the work in this area that capitalizes on Bayesian Additive Regression Trees, an algorithm that embeds a tree-based machine learning technique within a Bayesian framework to allow for flexible estimation and valid assessments of uncertainty. It will further describe extensions of the original work to address common issues in causal inference: lack of common support, violations of the ignorability assumption, and generalizability of results to broader populations. It will also describe existing R packages for traditional BART implementation as well as debut a new R package for causal inference using BART, bartCause.

Simulating a Marginal Structural Model

A seminar by Keith Goldfeld, Associate Professor, New York Univerity Langone Health

February 28, 2018

Abstract

In so many ways, simulation is an extremely useful tool to learn, teach, and understand the theory and practice of statistics. A series of examples (interspersed with minimal theory) will hopefully illuminate the underbelly of confounding, colliding, and marginal structural models. Drawing on the potential outcomes framework, the examples will use the R simstudy package, a tool that is designed to make data simulation as painless as possible.

Graphs as Poetry

A seminar by Howard Wainer, Research Scientist,  National Board of Medical Examiners

February 7, 2018

Abstract

Visual displays of empirical information are too often thought to be just compact summaries that, at their best, can clarify a muddled situation. This is partially true, as far as it goes, but it omits the magic. We have long known that data visualization is an alchemist that can make good scientists great and transform great scientists into giants. In this talk we will see that sometimes, albeit too rarely, the combination of critical questions addressed by important data and illuminated by evocative displays can achieve a transcendent, and often wholly unexpected, result. At their best, visualizations can communicate emotions and feelings in addition to cold, hard facts.

2016

Large, Sparse Optimal Matching in an Observational Study of Surgical Outcomes

Abstract: How do health outcomes for newly-trained surgeons' patients compare with those for patients of experienced surgeons? To answer this question using data from Medicare, we introduce a new form of matching that pairs patients of 1252 new surgeons to patients of experienced surgeons, exactly balancing 176 surgical procedures and closely balancing 2.9 million finer patient categories. The new matching algorithm (which uses penalized network flows) exploits a sparse network to quickly optimize a match two orders of magnitude larger than usual in statistical matching, and allowing for extensive use of a new form of marginal balance constraint.

Generalized Ridge Regression Using an Iterative Solution

Speaker: Kathryn, postdoc at Columbia University's Earth Institute. Her PhD is in applied economics with interests in development economics, and applied statistics.

Abstract

An iterative method is introduced for solving noisy, ill-conditioned inverse problems, where the standard ridge regression is just the first iteration of the iterative method to be presented. In addition to the regularization parameter, lambda, we introduce an iteration parameter k, which generalizes the ridge regression. The derived noise damping filter is a generalization of the standard ridge regression filter (also known as Tikhonov). Application of the generalized solution performs better than the pseudo-inverse (the default solution to OLS in most statistical packages), and better than standard ridge regression (L-2 regularization), when the covariate matrix or design matrix is ill-conditioned, or highly collinear. A few examples are presented using both simulated and real data.

Latent Space Models for Affiliation Networks

Catherine (“Kate”) Calder, professor of statistics, The Ohio State University

Abstract

An affiliation network is a particular type of two-mode social network that consists of a set of `actors' and a set of `events' where ties indicate an actor's participation in an event. Methods for the analysis of affiliation networks are particularly useful for studying patterns of segregation and integration in social structures characterized by both people and potentially shared activities (e.g., parties, corporate board memberships, church attendance, etc.) One way to analyze affiliation networks is to consider one-mode network matrices that are derived from an affiliation network, but this approach may lead to the loss of important structural features of the data. The most comprehensive approach is to study both actors and events simultaneously. Statistical methods for studying affiliation networks, however, are less well developed than methods for studying one-mode, or actor-actor, networks. In this talk, I will describe a bilinear generalized mixed-effects model, which contains interacting random effects representing common activity pattern profiles and shared patterns of participation in these profiles. I will demonstrate how the proposed model is able to capture forth-order dependence, a common feature of affiliation networks, and describe a Markov chain Monte Carlo algorithm for Bayesian inference. I then will use the latent space interpretation of model components to explore patterns in extracurricular activity membership of students in a racially-diverse high school in a Midwestern metropolitan area. Using techniques from spatial point pattern analysis, I will show how our model can provide insight into patterns of racial segregation in the voluntary extracurricular activity participation profiles of adolescents. This talk is based on joint work with Yanan Jia and Chris Browning.

Why so many research hypotheses are mostly false and how to test

Paul De Boeck, professor of quantitative psychology, The Ohio State University

Abstract

From a recent Science article with a large number of replications of psychological studies the base rate of the null hypothesis of no effect can be estimated. It turns out to be extremely high, which implies that many research hypotheses are false. As I will explain they are perhaps not fully false but mostly false. A possible explanation for why unlikely hypotheses tend to be selected for empirical studies can be found in expected utility theory. It can be shown that for low to moderately high power rates, the expected utility of studies increases with the probability of the null hypothesis being true. A high probability of the null hypothesis being true can be understood as reflecting a contextual variation of effects that are in general not much different from zero. Increasing the power of studies has become a popular remedy to counter the replicability crisis but this strategy is highly misleading if effects vary. Meta-analysis is considered another remedy but it is a suboptimal and labor-intensive approach and it is only long-term method. Two more feasible methods will be discussed to deal with contextual variation.

Be the Data and More: Using interactive, analytic methods to enhance learning from data for students

Leanna House, Associate Professor of Statistics, Virginia Tech

The Ohio State University

Abstract

Datasets, no matter how big, are just tables of numbers without individuals to learn from the data, i.e., discover, process, assess, and communicate information in the data. Data visualizations are often used to present data to individuals, but most are created independently of human learning processes and lack transparency. To bridge the gap between people thinking critically about data and the utility of visualizations, we developed Bayesian Visual Analytics (BaVA) and its deterministic form, Visual to Parametric Interaction (V2PI). BaVA and V2PI transform static images of data to dynamic versions that respond to expert feedback. When applied iteratively, experts may explore data progressively in a sequence that parallels their personal sense-making processes. BaVA and V2PI have shown useful in both industry settings and the classroom. For example, we merged V2PI with motion detection software to create Be the Data. In Be the Data students physically move in a space to communicate their expert feedback about data projected overhead. The idea is that participants have an opportuntity to explore analytical relationships between data points by exploring relationships between themselves. This talk will focus on presenting the BaVA paradigm and its education applications.

Bayesian Inference and Stan Tutorial

Vincent Dorie, Postdoctoral Researcher, NYU PRIISM 

Abstract

This two hour session is focused on getting started with Stan and how to use it in your research. Stan is an open-source Bayesian probabilistic programming environment that takes a lot of the work out of model fitting so that researchers can focus on model building and interpretation. List of topics will include: overview of Bayesian statistics, overview of Stan and MCMC, writing models in Stan, and a tutorial session where participants can write a model on their own or develop models that they have been working on independently. Stan has interfaces to numerous programming languages, but the talk will focus on R.

Basing Causal Inferences about Policy Impacts on Non-Representative Samples of Sites – Risks, Consequences, and Fixes

Stephen Bell, Abt Associates Fellow 

Abstract

Randomized impact evaluations of social and educational interventions—while constituting the “gold standard” of internal validity due to the lack of selection bias between treated and untreated cases—usually lack external validity. Due to cost and convenience, or local resistance, they are almost always conducted in a set of sites that are not a probability sample of the desired inference population— the nation as a whole for social programs or a given state or school district for educational innovations. We use statistical theory and data from the Reading First evaluation to examine the risks and consequences for social experiments of non-representative site selection, asking when and to what degree policy decisions are led astray by tarnished “gold standard” evidence. We also explore possible ex ante design-based solutions to this problem and the performance of ex post methods in the literature for overcoming non-representative site selection through analytic adjustments after the fact.

Mediation: From Intuition to Data Analysis

Ilya Shpitser, Assistant Professor in the Department of Computer Science, Johns Hopkins University. 

Abstract

Modern causal inference links the "top-down" representation of causal intuitions and "bottom-up" data analysis with the aim of choosing policy. Two innovations that proved key for this synthesis were a formalization of Hume's counterfactual account of causation using potential outcomes (due to Jerzy Neyman), and viewing cause effect relationships via directed acyclic graphs (due to Sewall Wright). I will briefly review how a synthesis of these two ideas was instrumental in formally representing the notion of "causal effect" as a parameter in the language of potential outcomes, and discuss a complete identification theory linking these types of causal parameters and observed data, as well as approaches to estimation of the resulting statistical parameters. I will then describe, in more detail, how my collaborators and I are applying the same approach to mediation, the study of effects along particular causal pathways. I consider mediated effects at their most general: I allow arbitrary models, the presence of hidden variables, multiple outcomes, longitudinal treatments, and effects along arbitrary sets of causal pathways. As was the case with causal effects, there are three distinct but related problems to solve -- a representation problem (what sort of potential outcome does an effect along a set of pathways correspond to), an identification problem (can a causal parameter of interest be expressed as a functional of observed data), and an estimation problem (what are good ways of estimating the resulting statistical parameter). I report a complete solution to the first two problems, and progress on the third. In particular, my collaborators and I show that for some parameters that arise in mediation settings, triply robust estimators exist, which rely on an outcome model, a mediator model, and a treatment model, and which remain consistent if any two of these three models are correct. Some of the reported results are a joint work with Eric Tchetgen Tchetgen, Caleb Miles, Phyllis Kanki, and Seema Meloni.

Bayes vs Maximum Likelihood: The case of bivariate probit models

Adriana Crespo-Tenorio, PhD is on a mission to connect people’s online behavior to their offline lives. 

Abstract

Bivariate probit models are a common choice for scholars wishing to estimate causal effects in instrumental variable models where both the treatment and outcome are binary. However, standard maximum likelihood approaches for estimating bivariate probit models are problematic. Numerical routines in common software suites frequently generate inaccurate parameter estimates, and even estimated correctly, maximum likelihood routines provide no straightforward way to produce estimates of uncertainty for causal quantities of interest. In this article, we show that adopting a Bayesian approach provides more accurate estimates of key parameters and facilitates the direct calculation of causal quantities along with their attendant measures of uncertainty.

Scalable Bayesian Inference with Hamiltonian Monte Carlo

Michael Betancourt, Postdoctoral Research Associate, Warwick

Abstract

The modern preponderance of data has fueled a revolution in data science, but the complex nature of those data also limits naive inferences. To truly take advantage of these data we also need tools for building and fitting statistical models that capture those complexities. In this talk I’ll discuss some of the practical challenges of building and fitting such models in the context of real analyses. I will particularly emphasize the importance of Hamiltonian Monte Carlo and Stan, state-of-the-art computational tools that allow us to tackle these contemporary data without sacrificing the fidelity of our inferences.

Improving Human Learning with Unified Machine Learning Frameworks: Towards Faster, Better, and Less Expensive Education

José González-Brenes, Pearson

Abstract

Seminal results from cognitive science suggest that personalized education is effective to improve learners’ outcomes. However, the effort for instructors to create content for each of their students can sometimes be prohibitive. Recent progress in machine learning has enabled technology for teachers to deliver personalized education. Unfortunately, the statistical models used by these systems are often tailored for ad-hoc domains and do not generalize across applications. In this talk, I will discuss my work towards the goal of a unified statistical framework of human learning. This line of work is more flexible, more efficient, and more accurate than previous technology. Moreover, it generalizes previous popular models from the literature. Additionally, I will outline recent progress on novel methodology to evaluate statistical models for education with a learner-centric perspective. My findings suggest that prior work often uses evaluation methods that may misrepresent the educational value of educational systems. My work is a promising alternative that improves the evaluation of machine learning models in education.

Probabilistic Cause-of-death Assignment using Verbal Autopsies

Tyler McCormick, University of Washington, Seattle

Abstract

In regions without complete-coverage civil registration and vital statistics systems there is uncertainty about even the most basic demographic indicators. In such areas the majority of deaths occur outside hospitals and are not recorded. Worldwide, fewer than one-third of deaths are assigned a cause, with the least information available from the most impoverished nations. In populations like this, verbal autopsy (VA) is a commonly used tool to assess cause of death and estimate cause-specific mortality rates and the distribution of deaths by cause. VA uses an interview with caregivers of the decedent to elicit data describing the signs and symptoms leading up to the death. This paper develops a new statistical tool known as InSilicoVA to classify cause of death using information acquired through VA. InSilicoVA shares uncertainty between cause of death assignments for specific individuals and the distribution of deaths by cause across the population. Using side-by-side comparisons with both observed and simulated data, we demonstrate that InSilicoVA has distinct advantages compared to currently available methods.

Topic-adjusted visibility metric for scientific articles

Tian Zheng, Columbia University. 

Abstract

Measuring the impact of scientific articles is important for evaluating the research output of individual scientists, academic institutions and journals. While citations are raw data for constructing impact measures, there exist biases and potential issues if factors affecting citation patterns are not properly accounted for. In this talk, I present a new model that aims to address the problem of field variation and introduce an article level metric useful for evaluating individual articles’ topic-adjusted visibility. This measure derives from joint probabilistic modeling of the content in the articles and the citations amongst them using latent Dirichlet allocation (LDA) and the mixed membership stochastic blockmodel (MMSB). This proposed model provides a visibility metric for individual articles adjusted for field variation in citation rates, a structural understanding of citation behavior in different fields, and article recommendations which take into account article visibility and citation patterns. For this work, we also developed an efficient algorithm for model fitting using variational methods. To scale up to large networks, we developed an online variant using stochastic gradient methods and case-control likelihood approximation. Results from an application of our methods to the benchmark KDD Cup 2003 dataset with approximately 30,000 high energy physics papers will also be presented.

Small sample adjustments to F-tests for cluster robust standard errors

Elizabeth Tipton, Teachers College, Columbia University

Abstract

Data analysts commonly ‘cluster’ their standard errors to account for correlations arising from the sampling of aggregate units (e.g., states), each containing multiple observations. When the number of clusters is small to moderate, however, this approach can lead to biased standard errors and hypothesis tests with inflated Type I error. One solution that is receiving increased attention is the use of the bias-reduced linearization (BRL). In this paper, we extend the BRL approach to include an F-test that can be implemented in a wide range of applications. A simulation study reveals that that this test has Type I error close to nominal even with a very small number of clusters, and importantly, that it outperforms the usual estimator even when the number of clusters is moderate.

The Controversies over Null Hypothesis Testing and Replication

Barry Cohen, New York University

Abstract

The arguments against null hypothesis significance testing (NHST) have been greatly exaggerated, and do not apply equally to all types of psychological research. I will discuss the conditions under which NHST serves several useful purposes, which may outweigh its undeniable drawbacks. In brief, NHST works best when the null hypothesis is rarely true, the direction of the results is more important than the magnitude, extremely large samples are not used, and tiny effects have no serious consequences. Priming studies in social psychology will be used as an example of this type of research. Part of the controversy over failures to replicate notable psychological studies is related to misunderstandings and misuses of NHST. I will conclude by discussing the resistance to banning NHST and its p values in favor of reports of effects sizes and/or confidence intervals, and describing some of the possible solutions to the drawbacks of NHST.

The Curious Case of the Instrumental Variable Estimator for the Complier Average Causal Effect

Russell Steele, McGill University

Abstract

In randomized clinical trials, subjects often do not comply with their randomized treatment arm. Although one can still unbiasedly estimate the causal effect of being assigned to treatment using the common Intention-to-Treat (ITT) estimator, there is now potential confounding of the causal effect of actually *receiving* treatment. Basic alternative estimators such as the per protocol or as treated estimators have been used, but are generally biased for estimating the causal effect of interest. Balke and Pearl (1997) and Angrist, et al. (1996) independently proposed an instrumental variable (IV) estimator that would estimate the causal effect (the Complier Average Causal Effect — CACE) of receiving treatment in a subpopulation of people who would comply with treatment assignment (i.e. the compliers). In this talk, I will first review the CACE and the IV estimator. I will then dissect the instrumental variable estimator in order to compare it to the per protocol and as treated estimators. I will show that the basic IV estimator and its confidence interval can be computed from basic summary statistics that should be reported in any randomized trial. My formulation of the IV estimator will also allow for simple sensitivity analyses that can be done using a basic Excel spreadsheet. I will then describe future interesting directions for compliance research that I am currently working on. Most of this work appears in a recently published article at the American Journal of Epidemiology and is co-authored by Ian Shrier, Jay Kaufmann and Robert Platt.

Covariate Selection with Observational Data: Simulation Results and Discussion

Bryan Keller, Teachers College, Columbia University

Abstract

In an effort to protect against omitted variable bias, statisticians have traditionally favored an inclusive approach to covariate selection for causal inference, so long as covariates were measured before any treatment was administered. There are, however, three classes of variables, which, if conditioned upon, are known to degrade either the bias or efficiency of an estimate of a causal effect: non-informative variables (NVs), instrumental variables (IVs), and collider variables. The decision about whether to control for a potential collider variable must be based on theory about how the data were generated. In contrast, one need only establish a lack of association with the outcome variable in order to identify an NV or an IV. We investigate three empirical methods – forward stepwise selection, the lasso, and recursive feature elimination with random forests – for detection of NVs and IVs through simulation studies in which we judge their efficacy by (a) sensitivity and specificity in identifying true or near NVs and IVs and (b) the overall effect on bias and mean-squared error of the causal effect estimator, relative to inclusion of all pretreatment variables. Results and implications are discussed.

The End of Intelligence? What might Big Data, Learning Analytics and the Information Age Mean for how we Measure Education

Charles Lang, Postdoctoral Associate, ew York University  

Abstract

For over a century educational measurement has developed analytical tools designed to maximize the inferential power of limited samples: a biannual state test, a regular accreditation exam, a once in a lifetime SAT. But can this methodology adapt to a world in which previous limitations on data collection have been dramatically reduced? A world with a greater variety of data formats, representing a larger number of conditions, on a finer timescale, with a larger sample of students. Starting from a methodological basis, Charles will discuss the implications that changes in data collection may have on how education is measured and the impact that this might have on the disciplines, institutions, and practitioners that utilize educational measurement.

Studying Change with Difference Scores versus ANCOVA: Issues, Perspectives and Advances

Pat Shrout, New York University 

Abstract

Nearly 50 years ago, Lord (1967) described a so-called paradox in statistical analysis whereby two reasonable analyses of pre-treatment/post-treatment data lead to different results. I revisit the issues, review some of the historical discussion, and present an analysis of the alternate analyses with a causal model that distinguishes treatment effects from trait, state, and error variation. In addition to comparing numerical results from difference score and ANCOVA adjustment for pre-treatment group differences, I consider results based on propensity score adjustment.

2015

Classroom Context and Observed Teacher Performance: What Do Teacher Observation Scores Really Measure?

Matthew Steinberg

March 25, 2015

Abstract

As federal, state, and local policy reforms mandate the implementation of more rigorous teacher evaluation systems, measures of teacher performance are increasingly being used to support improvements in teacher effectiveness and inform decisions related to teacher retention. Observations of teachers’ classroom instruction take a central role in these systems, accounting for the majority of a teacher’s summative evaluation rating upon which accountability decisions are based. This study explores the extent to which classroom context influences measures of teacher performance based on classroom observation scores. Using data from the Measures of Effective Teaching (MET) study, we find that the context in which teachers work—most notably, the incoming academic performance of their students—plays a critical role in determining teachers’ measured performance, even after accounting for teachers’ endowed instructional abilities. The influence of student achievement on measured teacher performance is particularly salient for English Language Arts (ELA) instruction; for aspects of classroom practice that depend on a teacher’s interactions with her students; and for subject-specific teachers compared with their generalist counterparts. Further, evidence suggests that the intentional sorting of teachers to students has a significant influence on measured ELA (though not math) instruction. Implications for high-stakes teacher-accountability policies are discussed.

2013

Income attraction: An online dating field experiment

David Ong, Assistant Professor of Economics at Peking University Business School

November 21, 2013

Abstract

Marriage rates have been decreasing in the US contemporaneously as women’s relative wages have been increasing. Dr. Ong found the opposite pattern in China. Prior empirical studies with US marriage data indicate that women marry up (and men marry down) economically. Furthermore, if the wife earns more, less happiness and greater strife are reported, the gender gap in housework increases, and they are more likely to divorce. However, these observational studies cannot identify whether these consequences were due to men’s preference for lower income women, or women’s preference for higher income men, or to other factors. Dr Ong complements this literature by measuring income based attraction in a field experiment. He randomly assigned income levels to 360 unique artificial profiles on a major online dating website and recorded the incomes of nearly 4000 visits. He found that men of all income levels visited women’s profiles with different income levels at roughly equal rates. In contrast, women at all income levels visited men with higher income at higher rates, and surprisingly, these higher rates increased with the women’s own income. Men with the highest level of income got ten times more visits than the lowest. He discussed how the gender difference in “income attraction” might shed light on marriage and gender wage patterns, the wage premium for married men, and other stylized facts, e.g., why the gender gap in housework is higher for women who earn more than their husbands. This is the first field experimental study of gender differences in preferences for mate income.

Front-door Difference-in-Differences Estimators: The Effects of Early In-person Voting on Turnout

Adam Glynn, Harvard University

November 7, 2013 

Abstract

In this talk, Dr. Glynn discussed front-door difference-in-differences estimators that utilize mechanistic information from post-treatment variables in addition to information from pre-treatment covariates. Even when the front-door criterion does not hold, these estimators allow the identification of causal effects by utilizing assumptions that are analogous to standard difference-in-differences assumptions. He also demonstrated that causal effects can be bounded by front-door and front-door difference-in-differences estimators under relaxed assumptions. He illustrated these points with an application to the effects of early in-person voting on turnout. Despite recent claims that early in-person voting had either an undetectable effect or a negative effect on turnout in 2008, he found evidence that early in-person voting had small positive effects on turnout in Florida in 2008. Moreover, he found evidence that early in-person voting disproportionately benefits African-American turnout.

Gaussian Processes for Causal Inference

Vincent Dorie, IES Postdoctoral Fellow, PRIISM Center

October 24, 2013

Abstract

This brown bag talk provided a mathematical and literature background for Gaussian Processes (GP) and discussed the use of GP in non-parametric modeling of the response surface for use in making straightforward causal comparisons. Additional topics included scalability, incorporating treatment levels as a spatial dimension, and the requirements for a fully-automated "black box" system for causal inference.

Linkage of viral sequences among HIV-infected village residents in Botswana: estimation of clustering rates in the presence of missing data

Nicole Carnegie, Harvard University

September 19, 2013

Abstract

Linkage analysis is useful in investigating disease transmission dynamics and the effect of interventions on them, but estimates of probabilities of linkage between infected people from observed data can be biased downward when missingness is informative. We investigate variation in the rates at which subjects' viral genotypes link by viral load (low/high) and ART status using blood samples from household surveys in the Northeast sector of Mochudi, Botswana. The probability of obtaining a sequence from a sample varies with viral load; samples with low viral load are harder to amplify. Pairwise genetic distances were estimated from aligned nucleotide sequences of HIV-1C env gp120. It is first shown that the probability that randomly selected sequences are linked can be estimated consistently from observed data. This is then used to develop maximum likelihood estimates of the probability that a sequence from one group links to at least one sequence from another group under the assumption of independence across pairs. Furthermore, a resampling approach is developed that adjusts for the presence of correlation within individuals, with diagnostics for assessing the reliability of the method.

Sequences were obtained for 65% of subjects with high viral load (HVL, n=117), 54% of subjects with low viral load but not on ART (LVL, n=180), and 45% of subjects on ART (ART, n=126). The probability of linkage between two individuals is highest if both have HVL, and lowest if one has LVL and the other has LVL or is on ART. Linkage across groups is high for HVL and lower for LVL and ART. Adjustment for missing data increases the group-wise linkage rates by 40-100%, and changes the relative rates between groups. Bias in inferences regarding HIV viral linkage that arise from differential ability to genotype samples can be reduced by appropriate methods for accommodating missing data.

The Inadequacy of the Summed Score (and How You Can Fix It!)

Daphna Harel, McGill University, Department of Mathematics and Statistics

October 17, 2013

Abstract

Health researchers often use patient and physician questionnaires to assess certain aspects of health status. Item Response Theory (IRT) provides a set of tools for examining the properties of the instrument and for estimation of the latent trait for each individual. In my research, I critically examine the usefulness of the summed score over items and an alternative weighted summed score (using weights computed from the IRT model) as an alternative to both the empirical Bayes estimator and maximum likelihood estimator for the Generalized Partial Credit Model. First, I will talk about two useful theoretical properties of the weighted summed score that I have proven as part of my work. Then I will relate the weighted summed score to other commonly used estimators of the latent trait. I will demonstrate the importance of these results in the context of both simulated and real data on the Center for Epidemiological Studies Depression Scale. 

2011

Dealing with Attrition in Randomized Experiments: Non-parametric and Semi-Parametric Approaches

Cyrus Samii, New York University

December 7, 2011

Abstract

Uncontrolled missingness in experimental data may undermine randomization as the basis for unbiased inference of average treatment effects. This paper reviews methods that attempt to address this problem for inference on average treatment effects. Dr. Samii reviewed inference with non-parametric bounds and inference with semi-parametric adjustment through inverse-probability weighting, imputation, and their combination. The analysis is rooted in the Neyman-Rubinpotential outcomes model, which helps to expose key assumptions necessary for identification and also for valid statistical inference(e.g., interval construction).

The Psychometrics of College Testing: Why Don't We Practice What We Teach?

Eric Loken, Pennsylvania State University

November 9, 2011

Abstract

Universities with large introductory classes are essentially operating like major testing organizations. The college assessment model, however, is many decades old, and almost no attention is given to evaluating the psychometric properties of classroom testing. This is surprising considering risks in accountability, and lost opportunities for innovation in pedagogy. As used in colleges, multiple choice tests are often guaranteed to provide unequal information across the ability spectrum, and almost nothing is known about the consistency of measurement properties across subgroups. Course management systems that encourage testing from item banks can expose students to dramatically unequal assessment. Aside from issues of fairness and validity, the neglect of research on testing in undergraduate classes represents a missed opportunity to take an empirical approach to pedagogy. Years of testing have generated vast amounts of data on student performance. These data can be leveraged to inform pedagogical approaches. They can also be leveraged to provide novel assessments and tools to better encourage and measure student learning.

An "Introduction" to Respondent Driven Sampling (RDS) methodology

Krista Gile, University of Massachusetts/Amherst

October 13, 2011

Abstract

Krista Gile (Department of Mathematics and Statistics University of Massachusetts/Amherst) is a statistician who works closely with social and behavioral scientists in the area of RDS. RDS is an innovative sampling technique for studying hidden and hard-to-reach populations for which no sampling frame can be obtained. RDS has been widely used to sample populations at high risk of HIV infection and has also been used to survey undocumented workers and migrants.

Subsample Ignorable Likelihood for Regression Analysis with Missing Data

Roderick J. Little, University of Michigan

April 15, 2011

Abstract

Two common approaches to regression with missing covariates are complete-case analysis (CC) and ignorable likelihood (IL) methods. Dr. Little reviewed these approaches, and proposed a hybrid class, subsample ignorable likelihood (SSIL) methods, which applies an IL method to the subsample of observations that are complete on one set of variables, but possibly incomplete on others. Conditions on the missing data mechanism are presented under which SSIL gives consistent estimates, but both CC and IL are inconsistent. He motivated and applied the proposed method to data from National Health and Nutrition Examination Survey, and illustrated properties of the methods by simulation. Extensions to non-likelihood analyses are also mentioned. (Joint Work with Nanhua Zhang)

Confronting selection into and out of social settings: Neighborhood change and children's economic outcomes

Pat Sharkey, NYU Sociology

March 23, 2011

Abstract

Selection bias continues to be a central methodological problem facing observational research estimating the effects of social settings on individuals. This article develops a method to estimate the impact of change in a particular social setting, the residential neighborhood, that is designed to address non-random selection into a neighborhood and non-random selection out of a neighborhood. Utilizing matching to confront selection into neighborhood environments and instrumental variables to confront selection out of changing neighborhoods, the method is applied to assess the effect of a decline in neighborhood concentrated disadvantage on the economic fortunes of African American children living within changing neighborhoods. Substantive findings indicate that a one standard deviation decline in concentrated disadvantage leads to increases in African American children's adult economic outcomes, but no effects on educational attainment or health.

Modelling Birthweight in the Presence of Gestational Age Measurement Error: A Semi-parametric Multiple Imputation Model

Russ Steel, McGill University

March 2, 2011

Abstract

Gestational age is an important variable in perinatal research, as it is a strong predictor of mortality and other adverse outcomes, and is also a component of measures of fetal growth. However, gestational ages measured using the date of the last menstrual period (LMP) are prone to substantial errors. These errors are apparent in most population-based data sources, which often show such implausible features as a bimodal distribution of birth weight at early preterm gestational ages (≤ 34 weeks) and constant or declining mean birth weight at postterm gestational ages (≥ 42 weeks). These features are likely consequences of errors in gestational age. Gestational age plays a critical role in measurement of outcome (preterm birth, small for gestational age) and is an important predictor of subsequent outcomes. It is important in the development of fetal growth standards. Therefore, accurate measurement of gestational age, or, failing that, a reasonable understanding of the structure of measurement error in the gestational age variable, is critical for perinatal research. In this talk, I will discuss the challenges in adjusting for gestational age measurement error via multiple imputation. In particular, Dr. Steel emphasizes the tension between flexibly modelling the distribution of birthweights within a gestational age and allowing for gestational age measurement error. He discusses strategies for incorporating prior information about the measurement error distribution and averaging over uncertainty in the distribution of the birthweights conditional on the true gestational age.

2010

Didactic Talk: Using Multilevel Data to Control for Unobserved Confounders: Fixed and Random Effects Approaches

Professor Jack Buckley, New York University 

November 3rd, 2010

Methods Lecture: An Empirical Model for Strategic Network Formation.

Guido Imbens, Harvard University

October 29, 2010

Co-sponsored with the NYU Department of Economics

Abstract

Dr. Guido Imbens and his team develop and analyze a tractable empirical model for strategic network formation that can be estimated with data from a single network at a single point in time. They model the network formation as a sequential process where in each period a single randomly selected pair of agents has the opportunity to form a link. Conditional on such an opportunity, a link will be formed if both agents view the link as beneficial to them. They base their decision on their own characteristics, the characteristics of the potential partner, and on features of the current state of the network, such as whether the two potential partners already have friends in common.  A key assumption is that agents do not take into account possible future changes to the network.  This assumption avoids complications with the presence of multiple equilibria, and also greatly simplifies the computational burden of analyzing these models.  They use Bayesian markov-chain-monte-carlo methods to obtain draws from the posterior distribution of interest.  The team applies their methods to a social network of 669 high school students, with, in average, 4.6 friends. They then use the model to evaluate the effect of an alternative assignment to classes on the topology of the network. This is joint work with Nicholas Christakis, James Fowler, and Karthik Kalyanaraman.

Brown Bag: Informal discussion of the methodology associated with a work in progress

Pat Shrout, New York University

October 27, 2010

Abstract

Pat Shrout presented a work-in-progress that examines lagged effects of conflict in intimate couples on same-day closeness. The data was derived from daily diaries, and as such is more intensive (dense) than traditional longitudinal data. Dr. Shrout discussed open issues arising in model selection, which highlight the tension between model choice, substantive questions, interpretation and causality.

Statistics in Society Lecture: Forecasting Large Panel Data with Penalized Least-Squares

Jianqing Fan, Professor of Finance and Professor of Statistics, Princeton University

September 17, 2010

Co-sponsored by the Stern IOMS-Statistics Group 

Abstract

Large Panel data arise from many diverse fields such as economics, finance, meteorology, energy demand management and ecology where spatial-temporal data are collected. Neighborhood correlations allow us to better forecast future outcomes, yet neighborhood selection becomes an important and challenging task. In this talk, Dr. Fan introduced the penalized least-squares to select the neighborhood variables that have an impact on the forecasting power. An iterative two-scale approach will be introduced. The inherent error (noise level) will also be estimated in the high-dimensional regression problems, which serves as the benchmark for forecasting errors. The techniques will be illustrated in forecasting the US house price indices at various Core Based Statistical Area (CBSA) levels.

An Introduction to Multiple Imputation: A More Principled Missing Data Solution

Jennifer Hill, Professor of Applied Statistics, New York University

May 5th, 2010

Brown Bag Talk - Variable Selection For Linear Mixed Effect Models

Ying Lu, Assistant Professor of Applied Statistics, New York University

March 24, 2010

Abstract

Mixed effect models are fundamental tools for the analysis of longitudinal data, panel data and cross-sectional data. They are widely used by various fields of social sciences, medical and biological sciences. However, the complex nature of these models has made variable selection and parameter estimation a challenging problem. In this paper, Dr. Lu proposed a simple iterative procedure that estimates and selects fixed and random effects for linear mixed models. In particular, she proposed to utilize the partial consistency property of the random effect coefficients and select groups of random effects simultaneously via a data-oriented penalty function (the smoothly clipped absolute deviation penalty function). She showed that the proposed method is a consistent variable selection procedure and possesses the Oracle properties. Simulation studies and a real data analysis are also conducted to empirically examine the performance of this procedure.

Statistical Methods for Sampling Hidden Networked Populations

Mark S. Handcock

February 12, 2010

Abstract

Part of the Stern IOMS-Statistics Seminar Series, this talk will provide an overview of probability models and inferential methods for the analysis of data collected using Respondent Driven Sampling (RDS). RDS is an innovative sampling technique for studying hidden and hard-to-reach populations for which no sampling frame can be obtained. RDS has been widely used to sample populations at high risk of HIV infection and has also been used to survey undocumented workers and migrants. RDS solves the problem of sampling from hidden populations by replacing independent random sampling from a sampling frame by a referral chain of dependent observations: starting with a small group of seed respondents chosen by the researcher, the study participants themselves recruit additional survey respondents by referring their friends into the study. As an alternative to frame-based sampling, the chain-referral approach employed by RDS can be extremely successful as a means of recruiting respondents. Current estimation relies on sampling weights estimated by treating the sampling process as a random walk on a graph, where the graph is the social network of relations among members of the target population. These estimates are based on strong assumptions allowing the sample to be treated as a probability sample. In particular, the current estimator assumes a with-replacement sample or small sample fraction, while in practice samples are without-replacement, and often include a large fraction of the population. A large sample fraction, combined with different mean nodal degrees for infected and uninfected population members, induces substantial bias in the estimates. Dr. Handcock introduces a new estimator which accounts for the without-replacement nature of the sampling process, and removes this bias. He then briefly introduce a further extension which uses a parametric model for the underlying social network to reduce the bias induced by the initial convenience sample.

The fifth PRIISM-organized Statistics in Society lecture

Mark S. Handcock, Department of Statistics, University of California - Los Angeles

February 11, 2010

Co-sponsored by the Stern IOMS-Statistics Group

Abstract

In many situations information from a sample of individuals can be supplemented by information from population level data on the relationship of the explanatory variable with the dependent variables. Sources of population level data include a census, vital events registration systems and other governmental administrative record systems. They contain too few variables, however, to estimate demographically interesting models. Thus in a typical situation, the estimation is done by using sample survey data alone, and the information from complete enumeration procedures is ignored. Sample survey data, however, are subjected to sampling error and bias due to non- response, whereas population level data are comparatively free of sampling error and typically less biased from the effects of non-response.

In this talk, Dr. Handcok reviewed statistical methods for the incorporation of population level information and showed it can lead to statistically more accurate estimates and better inference. Population level information can be incorporated via constraints on functions of the model parameters. In general the constraints are non-linear, making the task of maximum likelihood estimation more difficult. He presented an alternative approach exploiting the notion of an empirical likelihood. He gave an application to demographic hazard modeling by combining panel survey data with birth registration data to estimate annual birth probabilities by parity.

2008

Red State, Blue State, Rich State, Poor State: Why Americans Vote the Way They Do

Andrew Gelman, Professor in the Departments of Statistics and Political Science at Columbia University

October 14, 2008

Abstract

Andrew Gelman is a Professor in the Departments of Statistics and Political Science at Columbia University.  His new book, "Red State, Blue State, Rich State, Poor State: Why Americans Vote the Way They Do," is receiving tremendous critical praise. Gelman has recently been featured on several radio programs including WNYC's Leonard Lopate Show. Professor Gelman recently appeared on the Leonard Lopate show; his talk will draw from his book on the same topic.