Hochul Hwang

publications

1. 

   Lessons Learned from Developing a Human-Centered Guide Dog Robot for Mobility Assistance

   Hochul Hwang, Ken Suzuki, Nicholas A Giudice, and 3 more authors

   arXiv 2024

   Abs HTML

   While guide dogs offer essential mobility assistance, their high cost, limited availability, and care requirements make them inaccessible to most blind or low vision (BLV) individuals. Recent advances in quadruped robots provide a scalable solution for mobility assistance, but many current designs fail to meet real-world needs due to a lack of understanding of handler and guide dog interactions. In this paper, we share lessons learned from developing a human-centered guide dog robot, addressing challenges such as optimal hardware design, robust navigation, and informative scene description for user adoption. By conducting semi-structured interviews and hu- man experiments with BLV individuals, guide-dog handlers, and trainers, we identified key design principles to improve safety, trust, and usability in robotic mobility aids. Our findings lay the building blocks for future development of guide dog robots, ultimately enhancing independence and quality of life for BLV individuals.

2. 

   Synthetic data augmentation for robotic mobility aids to support blind and low vision people

   Hochul Hwang, Krisha Adhikari, Satya Shodhaka, and 1 more author

   arXiv 2024

   Abs HTML

   Robotic mobility aids for blind and low-vision (BLV) individuals rely heavily on deep learning-based vision models specialized for various navigational tasks. However, the performance of these models is often constrained by the availability and diversity of real-world datasets, which are challenging to collect in sufficient quantities for different tasks. In this study, we investigate the effectiveness of synthetic data, generated using Unreal Engine 4, for training robust vision models for this safety-critical application. Our findings demonstrate that synthetic data can enhance model performance across multiple tasks, showcasing both its potential and its limitations when compared to real-world data. We offer valuable insights into optimizing synthetic data generation for developing robotic mobility aids. Additionally, we publicly release our generated synthetic dataset to support ongoing research in assistive technologies for BLV individuals, available at https://hchlhwang.github.io/SToP.

3. 

   🏅 Is it safe to cross? Interpretable Risk Assessment with GPT-4V for Safety-Aware Street Crossing

   Hochul Hwang, Sunjae Kwon, Yekyung Kim, and 1 more author

   UR 2024

   Abs HTML

   Safely navigating street intersections is a complex challenge for blind and low-vision individuals, as it requires a nuanced understanding of the surrounding context – a task heavily reliant on visual cues. Traditional methods for assisting in this decision-making process often fall short, lacking the ability to provide a comprehensive scene analysis and safety level. This paper introduces an innovative approach that leverages large multimodal models (LMMs) to interpret complex street crossing scenes, offering a potential advancement over conventional traffic signal recognition techniques. By generating a safety score and scene description in natural language, our method supports safe decision-making for the blind and low-vision individuals. We collected crosswalk intersection data that contains multiview egocentric images captured by a quadruped robot and annotated the images with corresponding safety scores based on our predefined safety score categorization. Grounded on the visual knowledge, extracted from images, and text prompt, we evaluate a large multimodal model for safety score prediction and scene description. Our findings highlight the reasoning and safety score prediction capabilities of a LMM, activated by various prompts, as a pathway to developing a trustworthy system, crucial for applications requiring reliable decision-making support.

4. 

   🏆 Towards Robotic Companions: Understanding Handler-Guide Dog Interactions for Informed Guide Dog Robot Design

   Hochul Hwang, Hee-Tae Jung, Nicholas A Giudice, and 3 more authors

   CHI 2024

   Abs HTML Video

   Dog guides are favored by blind and low-vision (BLV) individuals for their ability to enhance independence and confidence by reducing safety concerns and increasing navigation efficiency compared to traditional mobility aids. However, only a relatively small proportion of BLV people work with dog guides due to their limited availability and associated maintenance responsibilities. There is considerable recent interest in addressing this challenge by developing legged guide dog robots. This study was designed to determine critical aspects of the handler-guide dog interaction and better understand handler needs to inform guide dog robot development. We conducted semi-structured interviews and observation sessions with 23 dog guide handlers and 5 trainers. Thematic analysis revealed critical limitations in guide dog work, desired personalization in handler-guide dog interaction, and important perspectives on future guide dog robots. Grounded on these findings, we discuss pivotal design insights for guide dog robots aimed for adoption within the BLV community.

5. 

   System Configuration and Navigation of a Guide Dog Robot: Toward Animal Guide Dog-Level Guiding Work

   Hochul Hwang, Tim Xia, Ibrahima Keita, and 4 more authors

   ICRA 2023

   Abs Bib HTML Video

   A robot guide dog has compelling advantages over animal guide dogs for its cost-effectiveness, potential for mass production, and low maintenance burden. However, despite the long history of guide dog robot research, previous studies were conducted with little or no consideration of how the guide dog handler and the guide dog work as a team for navigation. To develop a robotic guiding system that is genuinely beneficial to blind or visually impaired individuals, we performed qualitative research, including interviews with guide dog handlers and trainers and first-hand blindfold walking experiences with various guide dogs. Grounded on the facts learned from vivid experience and interviews, we build a collaborative indoor navigation scheme for a guide dog robot that includes preferred features such as speed and directional control. For collaborative navigation, we propose a semantic- aware local path planner that enables safe and efficient guiding work by utilizing semantic information about the environment and considering the handler’s position and directional cues to determine the collision-free path. We evaluate our integrated robotic system by testing guide blindfold walking in indoor settings and demonstrate guide dog-like navigation behavior by avoiding obstacles at typical gait speed (0.7 m/s).

   @article{hwang2023system,
     title = {System Configuration and Navigation of a Guide Dog Robot: Toward Animal Guide Dog-Level Guiding Work},
     author = {Hwang, Hochul and Xia, Tim and Keita, Ibrahima and Suzuki, Ken and Biswas, Joydeep and Lee, Sunghoon I and Kim, Donghyun},
     journal = {ICRA},
     year = {2023},
     video = {https://www.youtube.com/watch?v=9Y7Gvbw0qr4},
   }

6. 

   Dynamic Object Avoidance using Event-Data for a Quadruped Robot

   Shifan Zhu, Nisal Perera, Shangqun Yu, and 2 more authors

   IROS IPPC Workshop 2023

   Abs Bib HTML Video

   As robots increase in agility and encounter fast- moving objects, dynamic object detection and avoidance become notably challenging. Traditional RGB cameras, burdened by motion blur and high latency, often act as the bottleneck. Event cameras have recently emerged as a promising solution for the challenges related to rapid movement. In this paper, we introduce a dynamic object avoidance framework that integrates both event and RGBD cameras. Specifically, this framework first estimates and compensates for the event’s motion to detect dynamic objects. Subsequently, depth data is combined to derive a 3D trajectory. When initiating from a static state, the robot adjusts its height based on the predicted collision point to avoid the dynamic obstacle. Through real-world experiments with the Mini-Cheetah, our approach successfully circumvents dynamic objects at speeds up to 5 m/s, achieving an 83% success rate. Supplemental video:

   @article{zhu2023dynamic,
     title = {Dynamic Object Avoidance using Event-Data for a Quadruped Robot},
     author = {Zhu, Shifan and Perera, Nisal and Yu, Shangqun and Hwang, Hochul and Kim, Donghyun},
     journal = {IROS IPPC Workshop},
     year = {2023},
     video = {https://www.youtube.com/watch?v=wEPvynkVlLA},
   }

7. Adv.Materials

   Highly sensitive capacitive pressure sensors over a wide pressure range enabled by the hybrid responses of a highly porous nanocomposite

   Kyoung-Ho Ha, Weiyi Zhang, Hongwoo Jang, and 5 more authors

   Advanced Materials 2021

   Abs Bib

   Past research aimed at increasing the sensitivity of capacitive pressure sensors has mostly focused on developing dielectric layers with surface/porous structures or higher dielectric constants. However, such strategies have only been effective in improving sensitivities at low pressure ranges (e.g., up to 3 kPa). To overcome this well-known obstacle, herein, a flexible hybrid-response pressure sensor (HRPS) composed of an electrically conductive porous nanocomposite (PNC) laminated with an ultrathin dielectric layer is devised. Using a nickel foam template, the PNC is fabricated with carbon nanotubes (CNTs)-doped Ecoflex to be 86% porous and electrically conductive. The PNC exhibits hybrid piezoresistive and piezocapacitive responses, resulting in significantly enhanced sensitivities (i.e., more than 400%) over wide pressure ranges, from 3.13 kPa−1 within 0–1 kPa to 0.43 kPa−1 within 30–50 kPa. The effect of the hybrid responses is differentiated from the effect of porosity or high dielectric constants by comparing the HRPS with its purely piezocapacitive counterparts. Fundamental understanding of the HRPS and the prediction of optimal CNT doping are achieved through simplified analytical models. The HRPS is able to measure pressures from as subtle as the temporal arterial pulse to as large as footsteps.

   @article{ha2021highly,
     title = {Highly sensitive capacitive pressure sensors over a wide pressure range enabled by the hybrid responses of a highly porous nanocomposite},
     author = {Ha, Kyoung-Ho and Zhang, Weiyi and Jang, Hongwoo and Kang, Seungmin and Wang, Liu and Tan, Philip and Hwang, Hochul and Lu, Nanshu},
     journal = {Advanced Materials},
     year = {2021},
   }

8. 

   ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications

   Hochul Hwang, Cheongjae Jang, Geonwoo Park, and 2 more authors

   IEEE Access 2021

   Abs Bib HTML

   To train deep learning models for vision-based action recognition of elders’ daily activities, we need large-scale activity datasets acquired under various daily living environments and conditions. However, most public datasets used in human action recognition either differ from or have limited coverage of elders’ activities in many aspects, making it challenging to recognize elders’ daily activities well by only utilizing existing datasets. Recently, such limitations of available datasets have actively been compensated by generating synthetic data from realistic simulation environments and using those data to train deep learning models. In this paper, based on these ideas we develop ElderSim, an action simulation platform that can generate synthetic data on elders’ daily activities. For 55 kinds of frequent daily activities of the elders, ElderSim generates realistic motions of synthetic characters with various adjustable data-generating options and provides different output modalities including RGB videos, two- and three-dimensional skeleton trajectories. We then generate KIST SynADL, a large-scale synthetic dataset of elders’ activities of daily living, from ElderSim and use the data in addition to real datasets to train three state-of-the-art human action recognition models. From the experiments following several newly proposed scenarios that assume different real and synthetic dataset configurations for training, we observe a noticeable performance improvement by augmenting our synthetic data. We also offer guidance with insights for the effective utilization of synthetic data to help recognize elders’ daily activities.

   @article{9324837,
     author = {Hwang, Hochul and Jang, Cheongjae and Park, Geonwoo and Cho, Junghyun and Kim, Ig-Jae},
     journal = {IEEE Access},
     year = {2021},
     title = {ElderSim: A Synthetic Data Generation Platform for Human Action Recognition in Eldercare Applications},
     pages = {1-1},
     doi = {10.1109/ACCESS.2021.3051842},
     url = {https://ieeexplore.ieee.org/document/9324837},
   }

9. 

   Computationally-robust and efficient prioritized whole-body controller with contact constraints

   Donghyun Kim, Jaemin Lee, Junhyeok Ahn, and 3 more authors

   IROS 2018

   Abs Bib HTML Video

   In this paper, we devise methods for the multiobjective control of humanoid robots, a.k.a. prioritized whole-body controllers, that achieve efficiency and robustness in the algorithmic computations. We use a form of whole-body controllers that is very general via incorporating centroidal momentum dynamics, operational task priorities, contact reaction forces, and internal force constraints. First, we achieve efficiency by solving a quadratic program that only involves the floating base dynamics and the reaction forces. Second, we achieve computational robustness by relaxing task accelerations such that they comply with friction cone constraints. Finally, we incorporate methods for smooth contact transitions to enhance the control of dynamic locomotion behaviors. The proposed methods are demonstrated both in simulation and in real experiments using a passive-ankle bipedal robot.

   @article{kim2018computationally,
     title = {Computationally-robust and efficient prioritized whole-body controller with contact constraints},
     author = {Kim, Donghyun and Lee, Jaemin and Ahn, Junhyeok and Campbell, Orion and Hwang, Hochul and Sentis, Luis},
     journal = {IROS},
     year = {2018},
     video = {https://www.youtube.com/watch?v=3uc_p-6tzLg},
   }

10. Humanoids

    Control scheme and uncertainty considerations for dynamic balancing of passive-ankled bipeds and full humanoids

    Donghyun Kim, Steven Jens Jorgensen, Hochul Hwang, and 1 more author

    Humanoids 2018

    Abs Bib HTML

    We propose a methodology for dynamically balancing passive-ankled bipeds and full humanoids. As dynamic locomotion without ankle-actuation is more difficult than with actuated feet, our control scheme adopts an efficient whole-body controller that combines inverse kinematics, contact-consistent feed-forward torques, and low-level motor position controllers. To understand real-world sensing and controller requirements, we perform an uncertainty analysis on the linear-inverted-pendulum (LIP)-based footstep planner. This enables us to identify necessary hardware and control refinements to demonstrate that our controller can achieve long-term unsupported dynamic balancing on our series-elastic biped, Mercury. Through simulations, we also demonstrate that our control scheme for dynamic balancing with passive-ankles is applicable to full humanoid robots.

    @article{kim2018control,
      title = {Control scheme and uncertainty considerations for dynamic balancing of passive-ankled bipeds and full humanoids},
      author = {Kim, Donghyun and Jorgensen, Steven Jens and Hwang, Hochul and Sentis, Luis},
      journal = {Humanoids},
      year = {2018},
      organization = {IEEE},
    }

11. US Patent

    Human behavior recognition system and method using hierachical class learning considering safety

    Junghyun Cho, Ig Jae Kim, and Hochul Hwang

    US Patent 2024

    Abs Bib HTML

    Embodiments relate to a human behavior recognition system using hierarchical class learning considering safety, the human behavior recognition system including a behavior class definer configured to form a plurality of behavior classes by sub-setting a plurality of images each including a subject according to pre-designated behaviors and assign a behavior label to the plurality of images, a safety class definer configured to calculate a safety index for the plurality of images, form a plurality of safety classes by sub-setting the plurality of images based on the safety index, and additionally assign a safety label to the plurality of images, and a trainer configured to train a human recognition model by using the plurality of images defined as hierarchical classes by assigning the behavior label and the safety label as training images.

    @article{cho2024human,
      title = {Human behavior recognition system and method using hierachical class learning considering safety},
      author = {Cho, Junghyun and Kim, Ig Jae and Hwang, Hochul},
      journal = {US Patent},
      year = {2024},
      publisher = {Google Patents},
      note = {US Patent App. 17/565,453},
    }