Publications

A growing body of research has explored how to support humans in making better use of AI-based decision support, including via training and onboarding. Existing research has focused on decision-making tasks where it is possible to evaluate “appropriate reliance” by comparing each decision against a ground truth label that cleanly maps to both the AI’s predictive target and the human decision-maker’s goals. However, this assumption does not hold in many real-world settings where AI tools are deployed today (e.g., social work, criminal justice, and healthcare). In this paper, we introduce a process-oriented notion of appropriate reliance called critical use that centers the human’s ability to situate AI predictions against knowledge that is uniquely available to them but unavailable to the AI model. To explore how training can support critical use, we conduct a randomized online experiment in a complex social decision-making setting: child maltreatment screening. We find that, by providing participants with accelerated, low-stakes opportunities to practice AI-assisted decision-making in this setting, novices came to exhibit patterns of disagreement with AI that resemble those of experienced workers. A qualitative examination of participants’ explanations for their AI-assisted decisions revealed that they drew upon qualitative case narratives, to which the AI model did not have access, to learn when (not) to rely on AI predictions. Our findings open new questions for the study and design of training for real-world AI-assisted decision-making. READ MORE

Designers often struggle to sufficiently explore large design spaces, which can lead to design fixation and suboptimal outcomes. Here we introduce DesignAID, a generative AI tool that supports broader design space exploration by first using large language models to produce a range of diverse ideas expressed in words, and then using image generation software to create images from these words. This innovative combination of AI-based capabilities allows human-computer pairs to rapidly create a diverse set of visual concepts without time-consuming drawing. In a study with 87 crowd-sourced designers, we found that designers rated the automatic generation of images from words as significantly more inspirational, enjoyable, and useful than a conventional baseline condition of image search using Pinterest. Surprisingly, however, we found that automatically generating highly diverse ideas had less value. For image generation, the high diversity condition was somewhat better in inspiration but no better in the other dimensions, and for image search it was significantly worse in all dimensions. READ MORE

The electrochemical series is a useful tool in electrochemistry, but its effectiveness in materials chemistry is limited by the fact that the standard electrochemical series is based on a relatively small set of reactions, many of which are measured in aqueous solutions. We have used machine learning to create an electrochemical series for inorganic materials from tens of thousands of entries in the Inorganic Crystal Structure Database. We demonstrate that this approach enables the prediction of oxidation states directly from composition in a way that is physically justified, human-interpretable, and more accurate than a state-of-the-art transformer-based neural network model. We present applications of our model to structure prediction, materials discovery, and materials electrochemistry, and we discuss possible additional applications and areas for improvement. To facilitate the use of our approach, we introduce a freely available web site and API. READ MORE

This paper introduces the Chemical Environment Modeling Theory (CEMT), a novel, generalized framework designed to overcome the limitations inherent in traditional atom-centered Machine Learning Force Field (MLFF) models, widely used in atomistic simulations of chemical systems. CEMT demonstrated enhanced flexibility and adaptability by allowing reference points to exist anywhere within the modeled domain and thus, enabling the study of various model architectures. Utilizing Gaussian Multipole (GMP) featurization functions, several models with different reference point sets, including finite difference grid-centered and bond-centered models, were tested to analyze the variance in capabilities intrinsic to models built on distinct reference points. The results underscore the potential of non-atom-centered reference points in force training, revealing variations in prediction accuracy, inference speed and learning efficiency. Finally, a unique connection between CEMT and real-space orbital-free finite element Density Functional Theory (FE-DFT) is established, and the implications include the enhancement of data efficiency and robustness. It allows the leveraging of spatially-resolved energy densities and charge densities from FE-DFT calculations, as well as serving as a pivotal step towards integrating known quantum-mechanical laws into the architecture of ML models. READ MORE

As autonomous driving technology matures, end-to-end methodologies have emerged as a leading strategy, promising seamless integration from perception to control via deep learning. However, existing systems grapple with challenges such as unexpected open set environments and the complexity of black-box models. At the same time, the evolution of deep learning introduces larger, multimodal foundational models, offering multi-modal visual and textual understanding. In this paper, we harness these multimodal foundation models to enhance the robustness and adaptability of autonomous driving systems, enabling out-of-distribution, end-to-end, multimodal, and more explainable autonomy. Specifically, we present an approach to apply end-to-end open-set (any environment/scene) autonomous driving that is capable of providing driving decisions from representations queryable by image and text. To do so, we introduce a method to extract nuanced spatial (pixel/patch-aligned) features from transformers to enable the encapsulation of both spatial and semantic features. Our approach (i) demonstrates unparalleled results in diverse tests while achieving significantly greater robustness in out-of-distribution situations, and (ii) allows the incorporation of latent space simulation (via text) for improved training (data augmentation via text) and policy debugging. We encourage the reader to check our explainer video and to view the code and demos on our project webpage. READ MORE

Self-driving labs (SDLs) leverage combinations of artificial intelligence, automation, and advanced computing to accelerate scientific discovery. The promise of this field has given rise to a rich community of passionate scientists, engineers, and social scientists, as evidenced by the development of the Acceleration Consortium and recent Accelerate Conference. Despite its strengths, this rapidly developing field presents numerous opportunities for growth, challenges to overcome, and potential risks of which to remain aware. This community perspective builds on a discourse instantiated during the first Accelerate Conference, and looks to the future of self-driving labs with a tempered optimism. Incorporating input from academia, government, and industry, we briefly describe the current status of self-driving labs, then turn our attention to barriers, opportunities, and a vision for what is possible. Our field is delivering solutions in technology and infrastructure, artificial intelligence and knowledge generation, and education and workforce development. In the spirit of community, we intend for this work to foster discussion and drive best practices as our field grows. READ MORE

The effect of ionomer to carbon (I/C) weight ratio and relative humidity (RH) on cathode catalyst degradation was investigated by comprehensive in situ characterization. Membrane electrode assemblies (MEA) with I/C ratios of 0.5, 0.8 and 1.2 were subjected to an accelerated stress test performed at 40, 70 and 100% RH. The results show an increasing loss in electrochemical active surface area (ECSA) for both higher I/C ratios and RH during voltage cycling. To differentiate between ionomer and water connected ECSA, carbon monoxide stripping measurements were performed at varying RH. Before degradation, all MEAs show comparable total ECSA values, while higher I/C ratios lead to a larger fraction of ionomer connected ECSA. After degradation, ECSA measurements of the lowest I/C ratio showed a relatively higher loss of Pt in contact with ionomer than Pt in contact with water, while an opposite trend was observed for higher I/C ratios. H2/N2 impedance measurements showed drastically increasing protonic catalyst layer resistances for decreasing RH especially at low I/C ratios, which might hinder Pt2+ ion diffusion towards the membrane, hence decreasing the ECSA loss. Limiting current measurements show increasing molecular O2 diffusion resistances at end of test for samples with higher I/C ratios and higher ECSA loss. READ MORE

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train generalist X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. More details can be found on the project website. READ MORE

Object tracking is central to robot perception and scene understanding. Tracking-by-detection has long been a dominant paradigm for object tracking of specific object categories. Recently, large-scale pre-trained models have shown promising advances in detecting and segmenting objects and parts in 2D static images in the wild. This begs the question: can we re-purpose these large-scale pre-trained static image models for open-vocabulary video tracking? In this paper, we re-purpose an open-vocabulary detector, segmenter, and dense optical flow estimator, into a model that tracks and segments objects of any category in 2D videos. Our method predicts object and part tracks with associated language descriptions in monocular videos, rebuilding the pipeline of Tractor with modern large pre-trained models for static image detection and segmentation: we detect open-vocabulary object instances and propagate their boxes from frame to frame using a flow-based motion model, refine the propagated boxes with the box regression module of the visual detector, and prompt an open-world segmenter with the refined box to segment the objects. We decide the termination of an object track based on the objectness score of the propagated boxes, as well as forward-backward optical flow consistency. We re-identify objects across occlusions using deep feature matching. We show that our model achieves strong performance on multiple established video object segmentation and tracking benchmarks, and can produce reasonable tracks in manipulation data. In particular, our model outperforms previous state-of-the-art in UVO and BURST, benchmarks for open-world object tracking and segmentation, despite never being explicitly trained for tracking. We hope that our approach can serve as a simple and extensible framework for future research. READ MORE

Loneliness and social isolation reduce physical and mental wellbeing. Older adults are particularly prone to social isolation due to decreased connection with previous social networks such as at workplaces. Social technology can decrease loneliness and improve wellbeing. The COVID-19 pandemic prompted quarantine and social distancing for many people, creating a context of widespread social isolation. READ MORE