group 2024 banner

Research

The Vision and Learning group works on a variety of cutting-edge problems in computer vision and machine learning. The exact problems vary over time. The figure below summarizes the broad research directions in the group.

Scene Understanding

Scene graph example showing a street photo of a bus at a covered bus stop with a woman in a coat standing near it, alongside a corresponding scene graph diagram. Bounding boxes in the photo (color-coded) mark the bus in red, roof in blue, a lit sign in cyan, the woman/lady in green, and her coat in orange. The diagram shows these objects connected by labeled relationship arrows: Bus –under→ roof, Bus –across→ light, lady/woman –looking at→ Bus, lady/woman –near→ Bus, and lady/woman –wearing→ coat

Human Motion and Pose

A split-screen research graphic demonstrating 3D human motion capture from soccer footage. On the left, a close-up photo shows two soccer players, one in a yellow Brazil jersey labeled with the number 2, competing for the ball. On the right, the scene is decomposed into 3D wireframe models of the players, surrounded by multiple isolated camera perspectives capturing the action from different angles.

Distributed Decision Making

A grid of eight security camera frames showing indoor hallways and outdoor walkways. Pedestrians walking through the spaces are highlighted and enclosed in various colored bounding boxes, demonstrating a multi-camera pedestrian tracking system.

Vision and Security/Privacy

A diagram illustrating a facial swap or deepfake generation process. Two source images at the top—a news anchor wearing a traditional white ghutrah and igal, and a man in a dark blue sweater—are joined by a blue plus sign. An equals sign points down to a final synthesized image showing the news anchor's body with a altered facial expression matching the second man.

Learning for Robot Autonomy

Four simulated robot-learning environments, each showing a red robotic arm at a wooden table performing a manipulation task: top-left, the arm reaches toward a wall-mounted window frame with a green target dot; top-right, the arm reaches into an open dark cabinet or safe with a green target dot inside; bottom-left, the arm approaches a black and yellow box with a red button on top; bottom-right, the arm hovers over the table near small blue and brown objects with no other props present.

Image Analysis for Scientific Applications

A 3x3 grid of images displaying cellular structures segmented into multi-colored patches on a black background. Black arrows run along the top border from left to right and down the left border from top to bottom, indicating an analytical progression or variation in the image sequence.