Publications

This paper studies the problem of object discovery -- separating objects from the background without manual labels. Existing approaches utilize appearance cues, such as color, texture, and location, to group pixels into object-like regions. However, by relying on appearance alone, these methods fail to separate objects from the background in cluttered scenes. This is a fundamental limitation since the definition of an object is inherently ambiguous and context-dependent. To resolve this ambiguity, we choose to focus on dynamic objects -- entities that can move independently in the world. We then scale the recent auto-encoder based frameworks for unsupervised object discovery from toy synthetic images to complex real-world scenes. To this end, we simplify their architecture, and augment the resulting model with a weak learning signal from general motion segmentation algorithms. Our experiments demonstrate that, despite only capturing a small subset of the objects that move, this signal is enough to generalize to segment both moving and static instances of dynamic objects. We show that our model scales to a newly collected, photo-realistic synthetic dataset with street driving scenarios. Additionally, we leverage ground truth segmentation and flow annotations in this dataset for thorough ablation and evaluation. Finally, our experiments on the real-world KITTI benchmark demonstrate that the proposed approach outperforms both heuristic- and learning-based methods by capitalizing on motion cues. READ MORE

Language allows humans to build mental models that interpret what is happening around them resulting in more accurate long-term predictions. We present a novel trajectory prediction model that uses linguistic intermediate representations to forecast trajectories, and is trained using trajectory samples with partially-annotated captions. The model learns the meaning of each of the words without direct per-word supervision. At inference time, it generates a linguistic description of trajectories which captures maneuvers and interactions over an extended time interval. This generated description is used to refine predictions of the trajectories of multiple agents. We train and validate our model on the Argoverse dataset, and demonstrate improved accuracy results in trajectory prediction. In addition, our model is more interpretable: it presents part of its reasoning in plain language as captions, which can aid model development and can aid in building confidence in the model before deploying it. READ MORE

We present a model of the cathode catalyst layer morphology before and after loading a porous catalyst support with Pt and ionomer. Support nanopores and catalyst particles within pores and on the support surface are described by size distributions, allowing for qualitative processes during the addition of a material phase to be dependent on the observed pore and particle size. A particular focus is put on the interplay of pore impregnation and blockage due to ionomer loading and the consequences for the Pt/ionomer interface, ionomer film thickness and protonic binding of particles within pores. We used the model to emulate six catalyst/support combinations from literature with different porosity, surface area and pore size distributions of the support as well as varying particle size distributions and ionomer/carbon ratios. Besides providing qualitatively and quantitatively accurate predictions, the model is able to explain why the protonically active catalyst surface area has been reported to not increase monotonically with ionomer addition for some supports, but rather decrease again when the optimum ionomer content is exceeded. The proposed model constitutes a fast translation from manufacturing parameters to catalyst layer morphology which can be incorporated into existing performance and degradation models in a straightforward way. READ MORE

Three-dimensional atomic-level models of polymers are the starting points for physics-based simulation studies. A capability to generate reasonable initial structural models is highly desired for this purpose. We have developed a python toolkit, namely, polymer structure predictor (PSP), to generate a hierarchy of polymer models, ranging from oligomers to infinite chains to crystals to amorphous models, using a simplified molecular-input line-entry system (SMILES) string of the polymer repeat unit as the primary input. This toolkit allows users to tune several parameters to manage the quality and scale of models and computational cost. The output structures and accompanying force field (GAFF2/OPLS-AA) parameter files can be used for downstream ab initio and molecular dynamics simulations. The PSP package includes a Colab notebook where users can go through several examples, building their own models, visualizing them, and downloading them for later use. The PSP toolkit, being a first of its kind, will facilitate automation in polymer property prediction and design. READ MORE

Reasoning about the future behavior of other agents is critical to safe robot navigation. The multiplicity of plausible futures is further amplified by the uncertainty inherent to agent state estimation from data, including positions, velocities, and semantic class. Forecasting methods, however, typically neglect class uncertainty, conditioning instead only on the agent's most likely class, even though perception models often return full class distributions. To exploit this information, we present HAICU, a method for heterogeneous-agent trajectory forecasting that explicitly incorporates agents' class probabilities. We additionally present PUP, a new challenging real-world autonomous driving dataset, to investigate the impact of Perceptual Uncertainty in Prediction. It contains challenging crowded scenes with unfiltered agent class probabilities that reflect the long-tail of current state-of-the-art perception systems. We demonstrate that incorporating class probabilities in trajectory forecasting significantly improves performance in the face of uncertainty, and enables new forecasting capabilities such as counterfactual predictions. READ MORE

Camera calibration is integral to robotics and computer vision algorithms that seek to infer geometric properties of the scene from visual input streams. In practice, calibration is a laborious procedure requiring specialized data collection and careful tuning. This process must be repeated whenever the parameters of the camera change, which can be a frequent occurrence for mobile robots and autonomous vehicles. In contrast, self-supervised depth and ego-motion estimation approaches can bypass explicit calibration by inferring per-frame projection models that optimize a view synthesis objective. In this paper, we extend this approach to explicitly calibrate a wide range of cameras from raw videos in the wild. We propose a learning algorithm to regress per-sequence calibration parameters using an efficient family of general camera models. Our procedure achieves self-calibration results with sub-pixel reprojection error, outperforming other learning-based methods. We validate our approach on a wide variety of camera geometries, including perspective, fisheye, and catadioptric. Finally, we show that our approach leads to improvements in the downstream task of depth estimation, achieving state-of-the-art results on the EuRoC dataset with greater computational efficiency than contemporary methods. READ MORE

Chapter 3

The materials discovery process naturally presents a slew of questions about the character of a material. These range from simply trying to learn basic facts about the atomic structure of the compound [1–3] all the way to producing a time-resolved profile of a functional process in operando from start to finish [4]. Spectros-copy is the process of measuring a materials’ response to external electromagnetic stimulus to deduce the properties of interest. Spectroscopy can help to solve problems in experimental design and decision-making (probing response of certain energy domains, measuring particular properties), inference (moving from raw data to the property of interest), and analysis (rationalizing and interpreting the data). These are all classes of problems which artificial intelligence (AI) is well-posed to address.As such, the interface between practitioners in both spectroscopy and AI has sparked a great deal of excitement and research activity. Because characterization is a crucial process of the materials discovery pipeline, insightful pairings of algorithms and experimental protocols can lead to gains in both experimental efficiency and accuracy.These gains may significantly compress the timescales of experimental procedures and help experimentalists learn more in less time or answer questions which were previously inaccessible with a given experimental apparatus. READ MORE

Lithium-ion batteries are one of the most commonly used energy storage device for electric vehicles. As battery chemistries continue to advance, an important question concerns how to efficiently determine charging protocols that best balance the desire for fast charging while limiting battery degradation mechanisms which shorten battery lifetime. Challenges in this optimization are the high dimensionality of the space of possible charging protocols, significant variability between batteries, and limited quantitative information on battery degradation mechanisms. Current approaches to addressing these challenges are model-based optimization and grid search. Optimization based on electrochemical models is limited by uncertainty in the underlying battery degradation mechanisms and grid search methods are expensive in terms of time, testing equipment, and cells. This article proposes a fast-charging Bayesian optimization strategy that explicitly includes constraints that limit degradation. The proposed BO-based charging approaches are sample-efficient and do not require first-principles models. Three different types of acquisition function (i.e., expected improvement, probability of improvement, and lower confidence bound) are evaluated. Their efficacies are compared for exploring and exploiting the parameter space of charging protocols for minimizing the charging time for lithium-ion batteries described by porous electrode theory. The probability-of-improvement acquisition function has lower mean and best minimum charging times than the lower-confidence-bound and expected-improvement acquisition functions. We quantify the decrease in the minimum charging time and increase in its uncertainty with increasing number of current steps used in charging protocols. Understanding ways to increase the convergence rate of Bayesian optimization, and how the convergence scales with the number of degrees of freedom in the optimization, serves as a baseline for extensions of the optimization to include battery design parameters. READ MORE

The ability to learn reward functions plays an important role in enabling the deployment of intelligent agents in the real world. However, comparing reward functions, for example as a means of evaluating reward learning methods, presents a challenge. Reward functions are typically compared by considering the behavior of optimized policies, but this approach conflates deficiencies in the reward function with those of the policy search algorithm used to optimize it. To address this challenge, Gleave et al. (2020) propose the Equivalent-Policy Invariant Comparison (EPIC) distance. EPIC avoids policy optimization, but in doing so requires computing reward values at transitions that may be impossible under the system dynamics. This is problematic for learned reward functions because it entails evaluating them outside of their training distribution, resulting in inaccurate reward values that we show can render EPIC ineffective at comparing rewards. To address this problem, we propose the Dynamics-Aware Reward Distance (DARD), a new reward pseudometric. DARD uses an approximate transition model of the environment to transform reward functions into a form that allows for comparisons that are invariant to reward shaping while only evaluating reward functions on transitions close to their training distribution. Experiments in simulated physical domains demonstrate that DARD enables reliable reward comparisons without policy optimization and is significantly more predictive than baseline methods of downstream policy performance when dealing with learned reward functions. READ MORE

Self-supervised learning (SSL) is a scalable way to learn general visual representations since it learns without labels. However, large-scale unlabeled datasets in the wild often have long-tailed label distributions, where we know little about the behavior of SSL. In this work, we systematically investigate self-supervised learning under dataset imbalance. First, we find out via extensive experiments that off-the-shelf self-supervised representations are already more robust to class imbalance than supervised representations. The performance gap between balanced and imbalanced pre-training with SSL is significantly smaller than the gap with supervised learning, across sample sizes, for both in-domain and, especially, out-of-domain evaluation. Second, towards understanding the robustness of SSL, we hypothesize that SSL learns richer features from frequent data: it may learn label-irrelevant-but-transferable features that help classify the rare classes and downstream tasks. In contrast, supervised learning has no incentive to learn features irrelevant to the labels from frequent examples. We validate this hypothesis with semi-synthetic experiments and theoretical analyses on a simplified setting. Third, inspired by the theoretical insights, we devise a re-weighted regularization technique that consistently improves the SSL representation quality on imbalanced datasets with several evaluation criteria, closing the small gap between balanced and imbalanced datasets with the same number of examples. READ MORE