Open Catalyst Experiments 2024 (OCx24): Bridging Experiments and Computational Models

J. Abed, J. Kim, M. Shuaibi, B. Wander, B. Duijf, S. Mahesh, . Lee, V. Gharakhanyan, S. Hoogland, E. Irtem, J. Lan, N. Schouten, A. Vijayakumar, J. Hattrick-Simpers, J. Kitchin, Z. Ulissi, A. van Vugt, E. H. Sargent, D. Sinton, C. L. Zitnick

arXiv:2411.11783, 2024

Open Materials 2024 (OMat24) Inorganic Materials Dataset and Models

L. Barroso-Luque, M. Shuaibi, X. Fu, B. Wood, M. Dzamba, M. Gao, A. Rizvi, C. L. Zitnick, Z. Ulissi

arXiv:2410.12771, 2024

CatTSunami: Accelerating Transition State Energy Calculations with Pre-trained Graph Neural Networks

B. Wander, M. Shuaibi, J. Kitchin, Z. Ulissi, C. L. Zitnick

arXiv:2405.02078, 2024

From molecules to materials: Pre-training large generalizable models for atomic property prediction

N. Shoghi, A. Kolluru, J. Kitchin, Z. Ulissi, C. L. Zitnick, B. Wood

ICLR, 2024 (arXiv:2310.16802)

Fine-Tuned Language Models Generate Stable Inorganic Materials as Text

N. Gruver, A. Sriram, A. Madotto, A. Wilson, C. L. Zitnick, Z. Ulissi

ICLR, 2024 (arXiv:2402.04379)

Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs

S. Passaro, C. L. Zitnick

ICML, 2023 (arXiv:2302.03655)

Deep Learning Reconstruction Enables Prospectively Accelerated Clinical Knee MRI

P. Johnson, D. Lin, J. Zbontar, C. L. Zitnick, A. Sriram, M. Muckley, J. Babb, M. Kline, G. Ciavarra, E. Alaia, M. Samim, W. Walter, L. Calderon, T. Pock, D. Sodickson, M. Recht, F. Knoll

Radiology, 2023

The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysis

R. Tran, J. Lan, M. Shuaibi, B. Wood, S. Goyal, A. Das, J. Heras-Domingo, A. Kolluru, A. Rizvi, N. Shoghi, A. Sriram, Z. Ulissi, C. L. Zitnick

ACS Catalysis, 2023 (arXiv:2206.08917)

AdsorbML: A Leap in Efficiency for Adsorption Energy Calculations Using Generalizable Machine Learning Potentials

J. Lan, A. Palizhati, M. Shuaibi, B. Wood, B. Wander, A. Das, M. Uyttendaele, C. L. Zitnick, Z. Ulissi

Nature Computational Materials, 2023 (arXiv:2211.16486)

Spherical Channels for Modeling Atomic Interactions

C. Lawrence Zitnick, A. Das, A. Kolluru, J. Lan, M. Shuaibi, A. Sriram, Z. Ulissi, B. Wood

NeurIPS, 2022 (arXiv:2206.14331)

Open Challenges in Developing Generalizable Large Scale Machine Learning Models for Catalyst Discovery

A. Kolluru, M. Shuaibi, A. Palizhati, N. Shoghi, A. Das, B. Wood, C. L. Zitnick, J. Kitchin, Z. Ulissi

ACS Catalysis, 2022 (arXiv:2206.02005)

GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular Simulation Datasets

J. Gasteiger, M. Shuaibi, A. Sriram, S. Günnemann, Z. Ulissi, C L. Zitnick, A. Das

TMLR, 2022 (arXiv:2204.02782)

Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations

A. Sriram, A. Das, B. Wood, S. Goyal, C. L. Zitnick

ICLR, 2022 (arXiv:2203.09697)

Transfer learning using attentions across atomic systems with graph neural networks (TAAG)

A. Kolluru, N. Shoghi, M. Shuaibi, S. Goyal, A. Das, C. L. Zitnick, Z. Ulissi

Journal of Chemical Physics, 2022

Rotation Invariant Graph Neural Networks using Spin Convolutions

M. Shuaibi, A. Kolluru, A. Das, A. Grover, A. Sriram, Z. Ulissi, C. L. Zitnick

arXiv:2106.09575, 2021

ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations

W. Hu, M. Shuaibi, A. Das, S. Goyal, A. Sriram, J. Leskovec, D. Parikh, C. L. Zitnick

arXiv:2103.01436, 2021

Compositional Transformers for Scene Generation

D. Hudson, C. L. Zitnick

NeurIPS, 2021 (arXiv:2111.08960)

Generative Adversarial Transformers

D. Hudson, C. L. Zitnick

ICML, 2021 (arXiv:2103.01209)

Creative Sketch Generation

S. Ge, V. Goswami, C. L. Zitnick, D. Parikh

ICLR, 2021 (arXiv:2011.10039)

The Open Catalyst 2020 (OC20) Dataset and Community Challenges

L. Chanussot, A. Das, S. Goyal, T. Lavril, M. Shuaibi, M. Riviere, K. Tran, J. Heras-Domingo, C. Ho, W. Hu, A. Palizhati, A. Sriram, B. Wood, J. Yoon, D. Parikh, C. L. Zitnick, Z. Ulissi

ACS Catalysis, 2021 (arXiv:2010.09990)

Biological Structure and Function Emerge from Scaling Unsupervised Learning to 250 Million Protein Sequences

A. Rives, S. Goyal, J. Meier, D. Guo, M. Ott, C. L. Zitnick, J. Ma, R. Fergus

PNAS, 2021 (bioRxiv:622803)

An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage

C. L. Zitnick, L. Chanussot, A. Das, S. Goyal, J. Heras-Domingo, C. Ho, W. Hu, T. Lavril, A. Palizhati, M. Riviere, M. Shuaibi, A. Sriram, K. Tran, B. Wood, J. Yoon, D. Parikh, Z. Ulissi

arXiv:2010.09435, 2020

Using Deep Learning to Accelerate Knee MRI at 3T: Results of an Interchangeability Study

M. Recht, J. Zbontar, D. Sodickson, F. Knoll, N. Yakubova, A. Sriram, T. Murrell, A. Defazio, M. Rabbat, L. Rybak, M. Kline, G. Ciavarra, E. Alaia, M. Samim, W. Walter, D. Lin, Y. Lui, M. Muckley, Z. Huang, P. Johnson, R. Stern, C. L. Zitnick

American Journal of Roentgenology (AJR), 2020 (arXiv paper describing the approach)

Exploring Crowd Co-creation Scenarios for Sketches

D. Parikh, C. L. Zitnick

International Conference on Computational Creativity (ICCC), 2020 (arXiv:2005.07328)

End-to-End Variational Networks for Accelerated MRI Reconstruction

A. Sriram, J. Zbontar, T. Murrell, A. Defazio, C. L. Zitnick, N. Yakubova, F. Knoll, P. Johnson

MICCAI, 2020 (arXiv:2004.06688)

Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge

F. Knoll, T. Murrell, A. Sriram, N. Yakubova, J. Zbontar, M. Rabbat, A. Defazio, M. Muckley, D. Sodickson, C. L. Zitnick, M. Recht

Magnetic Resonance in Medicine, 2020 (arXiv:2001.02518)

GrappaNet: Combining Parallel Imaging with Deep Learning for Multi-Coil MRI Reconstruction

A. Sriram, J. Zbontar, T. Murrell, C. L. Zitnick, A. Defazio, D. Sodickson

CVPR, 2020 (arXiv:1910.12325)

fastMRI: A Publicly Available Raw k-Space and DICOM Dataset of Knee Images for Accelerated MR Image Reconstruction Using Machine Learning

F. Knoll, J. Zbontar, A. Sriram, M. Muckley, M. Bruno, A. Defazio, M. Parente, K. Geras, J. Katsnelson, H. Chandarana, Z. Zhang, M. Drozdzal, A. Romero, M. Rabbat, P. Vincent and J. Pinkerton, D. Wang, N. Yakubova, E. Owens, C. L. Zitnick, M. Recht, D. Sodickson, Y. Lui

Radiology Artificial Intelligence, 2020

Order-Aware Generative Modeling Using the 3D-Craft Dataset

Z. Chen, D. Guo, T. Xiao, S. Xie, X. Chen, H. Yu, J. Gray, K. Srinet, H. Fan, J. Ma, C. R. Qi, S. Tulsiani, A. Szlam, C. L. Zitnick

ICCV, 2019

CraftAssist: A Framework for Dialogue-enabled Interactive Agents

J. Gray, K. Srinet, Y. Jernite, H. Yu, Z. Chen, D. Guo, S. Goyal, C. L. Zitnick, A. Szlam

arXiv:1907.08584, 2019

ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero

Y. Tian, J. Ma, Q. Gong, S. Sengupta, Z. Chen, J. Pinkerton, C. L. Zitnick

ICML, 2019 (arXiv:1902.04522)

fastMRI: An Open Dataset and Benchmarks for Accelerated MRI

J. Zbontar, F. Knoll, A. Sriram, M. Muckley, M. Bruno, A. Defazio, M. Parente, K. Geras, J. Katsnelson, H. Chandarana, Z. Zhang, M. Drozdzal, A. Romero, M. Rabbat, P. Vincent and J. Pinkerton, D. Wang, N. Yakubova, E. Owens, C. L. Zitnick, M. Recht, D. Sodickson, Y. Lui

arXiv:1811.08839, 2018

ELF: An Extensive, Lightweight and Flexible Research Platform for Real-time Strategy Games

Y. Tian, Q. Gong, W. Shang, Y. Wu, C. L. Zitnick

NIPS, 2017. (arXiv)

Learn2Smile: Learning Non-Verbal Interaction Through Observation

W. Feng, A. Kannan, G. Gkioxari, C. L. Zitnick

IROS, 2017.

Inferring and Executing Programs for Visual Reasoning

J. Johnson, B. Hariharan, L. van der Maaten, J. Hoffman, L. Fei-Fei, C. L. Zitnick, R. Girshick

ICCV, 2017. (arXiv)

CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning

J. Johnson, B. Hariharan, L. van der Maaten, L. Fei-Fei, C. L. Zitnick, R. Girshick

CVPR, 2017. (arXiv)

Measuring Machine Intelligence Through Visual Question Answering

C. L. Zitnick, A. Agrawal, S. Antol, M. Mitchell, D. Batra, D. Parikh

AI Magazine, 2016. (arXiv)

Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?

A. Das, H. Agrawal, C. L. Zitnick, D. Parikh, D. Batra

EMNLP, 2016. (arXiv)

Shuffle and Learn: Unsupervised Learning using Temporal Order Verification

I. Misra, C. L. Zitnick, M. Hebert

ECCV, 2016. (arXiv)

Visual Storytelling

T. H. Huang, F. Ferraro, N. Mostafazadeh, I. Misra, A. Agrawal, J. Devlin, R. Girshick, X. He, P. Kohli, D. Batra, C. L. Zitnick, D. Parikh, L. Vanderwende, M. Galley, M. Mitchell

NAACL, 2016. (arXiv)

Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels

I. Misra, C. L. Zitnick, M. Mitchell, R. Girshick

CVPR, 2016. (arXiv)

Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks

S. Bell, C. L. Zitnick, K. Bala, R. Girshick

CVPR, 2016. (arXiv)

We Are Humor Beings: Understanding and Predicting Visual Humor

A. Chandrasekaran, A. Kalyan, S. Antol, M. Bansal, D. Batra, C. L. Zitnick, D. Parikh

CVPR, 2016. (arXiv)

Reducing Overfitting in Deep Networks by Decorrelating Representations

M. Cogswell, F. Ahmed, R. Girshick, C. L. Zitnick, D. Batra

ICLR, 2016. (arXiv)

Adopting Abstract Images for Semantic Scene Understanding

C. L. Zitnick, R. Vedantam and D. Parikh

PAMI, 2016. (webpage)

VQA: Visual Question Answering

S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, D. Parikh

ICCV, 2015. (webpage, arXiv)

Learning Common Sense Through Visual Abstraction

R. Vedantam, X. Lin, T. Batra, C. L. Zitnick, D. Parikh

ICCV, 2015. (webpage)

VISALOGY: Answering Visual Analogy Questions

F. Sadeghi, C. L. Zitnick, A. Farhadi

NIPS, 2015. (arXiv)

Mind's Eye: A Recurrent Visual Representation for Image Caption Generation

X. Chen and C.L. Zitnick

CVPR, 2015. (arXiv)

From Captions to Visual Concepts and Back

H. Fang, S. Gupta, F. Iandola, R. Srivastava, L. Deng, P. Dollár, J. Gao, X. He, M. Mitchell, J. Platt, C.L. Zitnick, and G. Zweig

CVPR, 2015. (arXiv, code)

CIDEr: Consensus-based Image Description Evaluation

R. Vedantam, C. L. Zitnick, and D. Parikh

CVPR, 2015. (arXiv, webpage)

Fast Edge Detection Using Structured Forests

P. Dollar and C. L. Zitnick

PAMI, 2015. (code)

Exploring Nearest Neighbor Approaches for Image Captioning

J. Devlin, S. Gupta, R. Girshick, M. Mitchell, C. L. Zitnick

arXiv:1505.04467 , 2015.

Microsoft COCO Captions: Data Collection and Evaluation Server

X. Chen, H. Fang, T.Y. Lin, R. Vedantam, S. Gupta, P. Dollar, C. L. Zitnick

arXiv:1504.00325 , 2015.(MS COCO caption challenge)

Microsoft COCO: Common Objects in Context

T.Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick

ECCV, 2014. (webpage, arXiv)

Edge Boxes: Locating Object Proposals from Edges

C. L. Zitnick and Piotr Dollar

ECCV, 2014. (code)

Zero-Shot Learning via Visual Abstraction

S. Antol, C. L. Zitnick, and D. Parikh

ECCV, 2014. (webpage)

Predicting Object Dynamics in Scenes

D. Fouhey and C. L. Zitnick

CVPR, 2014. (webpage, supp. mat.)

Detecting Objects using Deformation Dictionaries

B. Hariharan, C. L. Zitnick, and P. Dollar

CVPR, 2014.

Structured Forests for Fast Edge Detection

P. Dollar and C. L. Zitnick

ICCV, 2013. (code, supp. mat.)

Learning the Visual Interpretation of Sentences

C. L. Zitnick, D. Parikh, and L. Vanderwende

ICCV, 2013.  (webpage, dataset, supp. mat.)

Handwriting Beautification Using Token Means

C. L. Zitnick

SIGGRAPH, 2013. (video, YouTube, talk slides, handwriting data)

Bringing Semantics Into Focus Using Visual Abstraction

C. L. Zitnick and D. Parikh

CVPR, 2013. (webpage, dataset, talk video, talk slides)

Sketch-Tokens: A Learned Mid-level Representation for Contour and Object Detection

J. Lim, C. L. Zitnick, and P. Dollar

CVPR, 2013. (code)

Exploring Weak Stabilization for Motion Feature Extraction

D. Park, C. L. Zitnick, D. Ramanan, and P. Dollar

CVPR, 2013.

Which Edges Matter?

A. Bansal, A. Kowdle, D. Parikh, A. C. Gallagher, and C. L. Zitnick

Workshop on 3D Representation and Recognition (3dRR), ICCV, 2013.

Exploring the Spatial Hierarchy of Mixture Models for Human Pose Estimation

Y. Tian, C. L. Zitnick, and S. Narasimhan

ECCV, 2012. (webpage)

The Role of Image Understanding in Contour Detection

C. L. Zitnick and D. Parikh

CVPR, 2012. (Additional thoughts)

Seeing through the Blur

H. Mobahi, C. L. Zitnick, and Y. Ma

CVPR, 2012. (webpage)

Exploring Tiny Images: The Roles of Appearance and Contextual Information for Machine and Human Object Recognition

D. Parikh, C. L. Zitnick, and T. Chen

PAMI, 34(10):1978-1991, 2012.

Image Restoration by Matching Gradient Distributions

T. Cho, C. L. Zitnick, N. Joshi, S. B. Kang, R. Szeliski, and W. Freeman

PAMI, 34(4):683-694, 2012.

A Memory Efficient Discriminative approach for Location aided Recognition

V. Hedau, S. Sinha, C. L. Zitnick, and R. Szeliski

Workshop on Visual Analysis and Geo-Localization of Large-Scale Imagery, ECCV, 2012.

Edge Foci Interest Points

C. L. Zitnick, and K. Ramnath

ICCV, 2011. (webpage, executable)

ShadowDraw: Real-Time User Guidance for Freehand Drawing

Y. J. Lee, C. L. Zitnick, and M. Cohen

SIGGRAPH, 2011. (video, YouTube, webpage)

Finding the Weakest Link in Person Detectors

D. Parikh and C. L. Zitnick

CVPR, 2011. (webpage)

A Viewer-Centric Editor for Stereoscopic Cinema

S. J. Koppal, C. L. Zitnick, M.F. Cohen, S.B. Kang, B. Ressler, and A. Colburn

IEEE Computer Graphics and Applications, 2011. (webpage)

Binary Coherent Edge Descriptors

Single Image Deblurring using Motion Density Functions

Image deblurring with inertial measurement sensors

The Role of Features, Algorithms and Data in Visual Recognition

A Content-Aware Image Prior

GradientShop: A Gradient-Domain Optimization Framework for Image and Video Filtering

Unsupervised Learning of Hierarchical Spatial Structures in Images 

Image Deblurring and Denoising using Color Priors

Clustering Videos by Location

Determining Patch Saliency Using Low-Level Context

Fourier Analysis of the 2D Screened Poisson Equation for Gradient Domain Problems

 Stereo Reconstruction with Mixed Pixels Using Adaptive Over-Segmentation

From Appearance to Context-Based Recognition: Dense Labeling in Small Images

Automatic estimation and removal of noise from a single image

Internet Video Category Recognition

Using Photographs to Enhance Videos of a Static Scene

Stereo for image-based rendering using image over-segmentation

Finding people in repeated shots of the same scene

Reconstructing occluded surfaces using synthetic apertures: Shape from focus vs.

shape from stereo

Escaping local minima through hierarchical model selection: Automatic object discovery, segmentation, and tracking in video

Stylizing 2.5-D video

Video and Image Bayesian Demosaicing With A Two Color Image Prior

Consistent segmentation for optical flow estimation

High-quality video view interpolation using a layered representation.

Maximum Entropy for Collaborative Filtering

Computing Conditional Probabilities in Large Domains by Maximizing Renyi's Quadratic Entropy

Gaze-awareness for Videoconferencing: A Software Approach

A Cooperative Algorithm for Stereo Matching and Occlusion Detection

An Active multibaseline stereo system with real-time image acquisition

Massive Arabesque