Skip to content

Latest commit

 

History

History
154 lines (102 loc) · 14 KB

Transparent_Object_Manipulation.md

File metadata and controls

154 lines (102 loc) · 14 KB

Transparent Object Manipulation

[Challenges and Applications] 
- Challenges:
    - Visual Perception and Detection
    - Depth Perception
    - Refraction and Perspective Distortion
    - Grasping and Manipulation
    - Sensor Errors and Uncertainty
- Applications:
    - Medical Robotics
    - Robotic Surgery and Medical Assistance
    - Robotic Cleaning Systems

Survey

  • Robotic Perception of Transparent Objects: A Review, TAI 2024. [Paper]

Datasets

  • ClearPose: Large-scale Transparent Object Dataset and Benchmark, ECCV 2022. [Paper] [Code]

  • TransCG: A Large-Scale Real-World Dataset for Transparent Object Depth Completion and A Grasping Baseline, RAL 2022 & ICRA 2023. [Paper] [Website] [Code] [Cewu Lu]

  • TRansPose: Large-Scale Multispectral Dataset for Transparent Object, IJRR 2023. [Paper] [Website]

Robotic Grasping-Manipulation

  • ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation, ICRA 2020. [Paper] [Website] [Code]
  • Dex-NeRF: Using a Neural Radiance field to Grasp Transparent Objects, CoRL 2021. [Paper] [Website] [Datasets] [Code] [Unofficial Code]
  • Evo-NeRF: Evolving NeRF for Sequential Robot Grasping of Transparent Objects, CoRL 2022. [Paper] [Website]
  • GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF, ICRA 2023. [Paper] [Website] [Code]
  • NFL: Normal Field Learning for 6-DoF Grasping of Transparent Objects, R-AL 2024. [Paper] [Website] [Code]
  • Residual-NeRF: Learning Residual NeRFs for Transparent Object Manipulation, ICRA 2024. [Paper] [Website] [Code] [Data]
  • DREDS: Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects, ECCV 2020. [Paper] [Website] [Code] [He Wang, PKU-EPIC]
  • ASGrasp: Generalizable Transparent Object Reconstruction and 6-DoF Grasp Detection from RGB-D Active Stereo Camera, ICRA 2024. [Paper] [Website] [Code]
  • TransNet: Transparent Object Manipulation Through Category-Level Pose Estimation, ECCVW 2022. [Paper] [Website]
  • D^3RoMa: Disparity Diffusion-based Depth Sensing for Material-Agnostic Robotic Manipulation, CoRL 2024. [Paper] [Website] [Code]
  • 🔥 Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objects, ECCV 2024. [Paper] [Website] [Code] [Dataset]

Visual Tactile

  • Visual–Tactile Fusion for Transparent Object Grasping in Complex Backgrounds, T-RO 2023. [Paper] [Website] [Code] [Linqi Ye, THU-SHU]
  • TransTouch: Learning Transparent Objects Depth Sensing Through Sparse Touches, IROS 2023. [Paper] [Code]

Pose Estimation

  • KeyPose: Multi-View 3D Labeling and Keypoint Estimation for Transparent Objects, CVPR 2020. [Paper] [Website] [Code]
  • StereoPose: Category-Level 6D Transparent Object Pose Estimation from Stereo Images via Back-View NOCS, ICRA 2023. [Paper] [Webaite] [Code]

Simulation

  • Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation, T-RO 2023. [Paper] [Website] [Code] [University of California]
  • SuperCaustics: Real-time, open-source simulation of transparent objects for deep learning applications, arXiv 2021. [Paper] [Code]

Depth Completion

  • Implicit Depth: RGB-D Local Implicit Function for Depth Completion of Transparent Objects, CVPR 2021. [Paper] [Website] [Code]

  • FDCT: Fast Depth Completion for Transparent Objects, R-AL 2023. [Paper] [Code]

  • TDCNet: Transparent Objects Depth Completion with CNN-Transformer Dual-Branch Parallel Network, arXiv 2024. [Paper] [Code] [Website] [Cewu Lu]

  • Depth4ToM: Learning Depth Estimation for Transparent and Mirror Surfaces, ICCV 2023. [Paper] [Website] [Code]

  • Seeing Glass: Joint Point-Cloud and Depth Completion for Transparent Objects, CoRL 2021. [Paper] [Website] [Code]

  • ActiveZero: Mixed Domain Learning for Active Stereovision with Zero Annotation, CVPR 2022. [Paper] [Code]

  • ActiveZero++: Mixed Domain Learning Stereo and Confidence-based Depth Completion with Zero Annotation, TPAMI 2023. [Paper] [Code]

  • Consistent Depth Prediction for Transparent Object Reconstruction from RGB-D Camera, ICCV 2023. [Paper]

  • Monocular Depth Estimation for Glass Walls with Context: A New Dataset and Method, TPAMI 2023. [Paper] [Code]

  • TODE-Trans: Transparent Object Depth Estimation with Transformer, ICRA 2023. [Paper] [Code]

  • ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation, arXiv 2024. [Paper] [Website]

3D Reconstruction

  • DRT: Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects, SIGGRAPH Asia 2020. [Paper] [Website] [Code]
  • Eikonal Fields for Refractive Novel-View Synthesis, SIGGRAPH 2022. [Paper] [Website] [Code]
  • Deep Polarization Cues for Single-shot Shape and Subsurface Scattering Estimation, ECCV 2024. [Paper] [Code]
  • NeTO: Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing, ICCV 2023. [Paper] [Website] [Code]
  • NEMTO: Neural Environment Matting for Novel View and Relighting Synthesis of Transparent Objects, ICCV 2023. [Paper] [Website] [Code]
  • Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction, CVMJ 2023. [Paper] [Code]
  • TransPIR: Polarimetric Inverse Rendering for Transparent Shapes Reconstruction, TOM 2024. [Paper] [Code]
  • TransSfP: Transparent Shape from a Single View Polarization Image, ICCV 2023. [Paper] [Code]
  • Touch-GS: Visual-Tactile Supervised 3D Gaussian Splatting, IROS 2024. [Paper] [Website] [Code]
  • FusionSense: Bridging Common Sense, Vision, and Touch for Robust Sparse-View Reconstruction, arXiv 2024. [Paper] [Website] [Code]
  • Snap-it, Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces, arXiv 2024. [Paper]
  • Transparent Object Reconstruction in 3D Gaussian Splatting with RGB-to-TIR Translation, MLVU Projects (Spring 2024). [Paper] [Talk]
  • NU-NeRF: Neural Reconstruction of Nested Transparent Objects with Uncontrolled Capture Environment, SIGGRAPH Asia 2024. [Paper] [Website] [Code]
  • NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields, arXiv 2023. [Paper] [Code]

Segmentation

  • TransCut: Transparent Object Segmentation from a Light-Field Image, ICCV 2025. [Paper] [Website]
  • Segmenting Transparent Objects in the Wild, ECCV 2020. [Paper] [Code]
  • Deep Polarization Cues for Transparent Object Segmentation, CVPR 2020. [Paper]
  • Glass Segmentation with RGB-Thermal Image Pairs, TIP 2023. [Paper] [Code]
  • Leveraging RGB-D Data with Cross-Modal Context Mining for Glass Surface Detection, AAAI 2025. [Paper] [Code] [Website]
  • Self-supervised Transparent Liquid Segmentation for Robotic Pouring, ICRA 2022. [Paper] [Website] [Code]

Perception

  • MVTrans: Multi-view Perception to See Transparent Objects, ICRA 2023. [Paper] [Website] [Code]
  • Transparent Object Tracking Benchmark, ICCV 2021. [Paper] [Website] [Code]
  • A New Dataset and a Distractor-Aware Architecture for Transparent Object Tracking, IJCV 2024. [Paper] [Group]