Projects

Check out my Github to see all projects.

Selected Projects

Predictive Modeling for Robotic Perception

---

In the project, high-dimensional perception data was compressed using a variational autoencoder (VAE) pipeline, optimizing the efficiency of motion planning. The project’s ”Forward Model,” designed for task and motion planning applications, was constructed to predict the mobile robot’s future observations based on current sensory inputs and control actions. This predictive capability was achieved by integrating the VAE with a Long Short-Term Memory (LSTM) network.

Figure 1: Proposed Method for Predictive Modeling

Figure 2: Evolution of Observations and Predictions

Autonomous Structure Building with Graph Neural Networks

---

This project introduces a novel approach using graph networks for task and motion planning in robotic structure building. The system transforms an initial object configuration to a desired one by inferring spatial relations and representing them as a graph. A graph neural network (GNN) identifies movable objects and determines the necessary action sequence. Integrated with a Graph Search-based algorithm within the RAI framework, the system effectively classifies movable objects and operates a robot arm with a motion planner and solves challenges like tower and wall building fast.

Figure 1: Algorithm for Task and Motion Planning with GNNs

Figure 2: Object Feasibility Detection