Yang Zhang

Yang Zhang is a 5th (final) year Ph.D. candidate at Human-Computer Interaction Institute (HCII), School of Computer Science, Carnegie Mellon University, advised by Prof. Chris Harrison. He is also a Qualcomm Innovation Fellow.

He develops sensing techniques for next-generation human-computer interfaces that bring computing resources to users instead of forcing users to go to computing resources. Specifically, he builds sensing technologies to enable natural interactions on mobile devices beyond touchscreens and detect user activities for personal and environment informatics.

He publishes at CHI (ACM CHI Conference on Human Factors in Computing Systems) and UIST (ACM Symposium on User Interface Software and Technology), and has received 5 Best Paper (1%) and Honorable Mention Awards (5%).

Taxonomies of his completed research can be found below. More projects are on the way.

[Research focus diagram inspired by professor Bjoern Hartmann]

Sort by Date

Wide-Area Sensors for Ubiquitous Sensing

Smart environment relies on robust and accurate sensing techniques. Closest to this vision are cameras and general-purpose sensor tags, and yet several key challenges remain (e.g., high cost, demanding maintenance, low sensing versatility and resolution, visually unappealing, and privacy invasive). To tackle these challenges, I build wide-area sensors that can sense at distance and monitor activities at room-scale, enabling ubiquitous sensing with a sparse sensor deployment.
Sozu: Self-Powered Radio Tags for Building-Scale Activity Sensing

Y Zhang, Y Iravantchi, H Jin, S Kumar and C Harrison (to appear at UIST 2019) [PDF upon requests]

Sozu a low-cost sensing system that can detect a wide range of events wirelessly, through walls and without line of sight, at whole-building scale. Instead of using batteries, Sozu tags convert energy from activities that they sense into RF broadcasts, acting like miniature self-powered radio stations.

Vibrosight: Long-Range Vibrometry for Smart Environment Sensing

Y Zhang, G Laput and C Harrison (UIST 2018) [Video] [DOI] [PDF] [Code]

Honorable Mention Award

Vibrosight senses activities across entire rooms using long-range laser vibrometry. Unlike a microphone, our approach can sense physical vibrations at one specific point, making it robust to interference from other activities and noisy environments. This property enables detection of simultaneous activities, which has proven challenging in prior work.

Wall++: Room-Scale Interactive and Context-Aware Sensing

Y Zhang, C Yang, S E. Hudson, C Harrison and A Sample (CHI 2018) [Video] [DOI] [PDF]

Best Paper Award

Wall++ is a low-cost sensing approach that allows walls to become a smart infrastructure. Our wall treatment and sensing hardware can track users' touch and gestures, as well as estimate body pose if they are close. By capturing airborne electromagnetic noise, we can also detect what appliances are active and where they are located.

Synthetic Sensors: Towards General-Purpose Sensing

G Laput, Y Zhang and C Harrison (CHI 2017) [Video] [DOI] [PDF]

In this work, we explore the notion of general-purpose sensing, wherein a single, highly capable sensor can indirectly monitor a large context, without direct instrumentation of objects. Further, through what we call Synthetic Sensors, we can virtualize raw sensor data into actionable feeds, whilst simultaneously mitigating immediate privacy issues.

Electric Field Tomography for Interactivity

Since my first project with Electric Field Tomography (Tomo 2015), I have been improving this technique as well as broadening its use cases. Electric Field Tomography features many advantages of capacitive sensing but is of higher spatial resolution. The following projects demonstrate its applications in touch tracking and on-body sensing.
Electrick: Low-Cost Touch Sensing Using Electric Field Tomography

Y Zhang, G Laput and C Harrison (CHI 2017) [Video] [DOI] [PDF]

Electrick is a low-cost and versatile sensing technique that enables touch input on a wide variety of objects and surfaces, whether small or large, flat or irregular. This is achieved by using electric field tomography in concert with an electrically conductive material, which can be easily and cheaply added to objects and surfaces.

Tomo: Wearable, Low-cost, Electrical Impedance Tomography for Hand Gesture Recognition

Y Zhang and C Harrison (UIST 2015) [Video] [DOI] [PDF]

Tomo recovers the interior impedance geometry of a user's arm by measuring the cross-sectional impedances from surface electrodes resting on the skin. We integrated the technology into a prototype wristband, which can classify gestures in real-time. Our approach is sufficiently compact and low-powered that we envision this technique being integrated into future smartwatches to allow hand gestures to work together with touchscreens.

Advancing Hand Gesture Recognition with High Resolution Electrical Impedance Tomography

Y Zhang, R Xiao and C Harrison (UIST 2016) [Video] [DOI] [PDF]

We improved our prior work on wearable Electrical Impedance Tomography with higher sampling speed and resolution. In turn, this enables superior interior reconstruction and gesture recognition. More importantly, we use our new system as a vehicle for experimentation -- we compare two EIT sensing methods and three different electrode resolutions.

Pulp Nonfiction: Low-Cost Touch Tracking for Paper

Y Zhang and C Harrison (CHI 2018) [Video] [DOI] [PDF]

We developed a sensing technique for paper to track finger input and also drawn input with writing implements. Importantly, for paper to still be considered paper, our method had to be very low cost. This necessitated research into materials, fabrication methods and sensing techniques. We describe the outcome of our investigations and show that our method can be sufficiently low-cost and accurate to enable new interactive opportunities with this pervasive and venerable material.

Enhancing Smart Devices for Interactions beyond Touchscreens

Smart Devices can be only as smart as what they can sense. I build sensors to extend the sensing range of these devices to see user postures, hand proximity, finger touch around the device. These sensing capabilities serve as a secondary input channel besides touchscreens, which allows these smart devices to morph their UIs, fitting user context and being easier to use.
ActiTouch: Robust Touch Detection for On-Skin AR/VR Interfaces

Y Zhang, W Kienzle, Y Ma, S S. Ng, H Benko, C Harrison (to appear at UIST 2019) [PDF upon requests]

ActiTouch is a new electrical method that enables precise on-skin touch segmentation by using the body as an RF wave-guide. We combine this method with computer vision, enabling a system with both high tracking precision and robust touch detection. Our system can enable touchscreen-like interactions on the skin.

Interferi: Gesture Sensing using On-Body Acoustic Interferometry

Y Iravantchi, Y Zhang, E Bernitsas, M Goel, and C Harrison. (CHI 2019) [Video] [DOI] [PDF]

Honorable Mention Award

Interferi is an on-body gesture sensing technique using acoustic interferometry. We use ultrasonic transducers resting on the skin to create acoustic interference patterns inside the wearer’s body, which interact with anatomical features in complex, yet characteristic ways. We focus on two areas of the body with great expressive power: the hands and face.

Sensing Posture-Aware Pen+Touch Interaction on Tablets

Y Zhang, M Pahud, C Holz, H Xia, G Laput, M McGuffin, X Tu, A Mittereder, F Su, W Buxton and K Hinckley (CHI 2019) [Video] [DOI] [PDF]

Honorable Mention Award

The mobility of tablets affords interaction from various user-centric postures including shifting hand grips, varying screen angle and orientation, planting the palm while writing or sketching. We propose Posture-Aware Interface which morphs to a suitable frame of reference, at the right time, and for the right (or left) hand.

SkinTrack: Using the Body as an Electrical Waveguide for Continuous Finger Tracking on the Skin

Y Zhang, J Zhou, G Laput and C Harrison (CHI 2016) [Video] [DOI] [PDF]

Honorable Mention Award

SkinTrack is a wearable system that enables continuous touch tracking on the skin. It consists of a signal-emitting ring and a sensing wristband with multiple electrodes. Due to the phase delay inherent in a high-frequency AC signal propagating through the body, a phase difference can be observed between pairs of electrodes, which we use to compute a 2D finger touch coordinate.

AuraSense: Enabling Expressive Around-Smartwatch Interactions with Electric Field Sensing

J Zhou, Y Zhang, G Laput and C Harrison (UIST 2016) [Video] [DOI] [PDF]

AuraSense enhances smartwatches with Electric Field Sensing to support multiple interaction modalities. We identified four electrode configurations that can support six well-known modalities of particular interest and utility, including gestures above the watchface and touchscreen-like finger tracking on the skin.

Pyro: Thumb-Tip Gesture Recognition Using Pyroelectric Infrared Sensing

J Gong, Y Zhang, X Zhou, XD Yang (UIST 2017) [Video] [DOI] [PDF]

Pyro is a micro thumb-tip gesture recognition technique based on thermal infrared signals radiating from the fingers. Pyro uses a compact, low-power passive sensor, making it suitable for wearable and mobile applications. To demonstrate the feasibility of Pyro, we developed a self-contained prototype consisting of the infrared pyroelectric sensor, a custom sensing circuit, and software for signal processing and machine earning.

Deus EM Machina: On-Touch Contextual Functionality for Smart IoT Appliances

R Xiao, G Laput, Y Zhang and C Harrison (CHI 2017) [Video] [DOI] [PDF]

We propose an approach where users simply tap a smartphone to an appliance to discover and rapidly utilize contextual functionality. To achieve this, our prototype smartphone recognizes physical contact with uninstrumented appliances through EMI sensing, and summons appliance-specific interfaces.

LumiWatch: On-Arm Projected Graphics and Touch Input

R Xiao, T Cao, N Guo, J Zhuo, Y Zhang and C Harrison (CHI 2018) [Video] [DOI] [PDF]

LumiWatch is the first, fully-functional and self-contained projection smartwatch implementation, containing the requisite compute, power, projection and touch-sensing capabilities. Our watch offers more than five times that of a typical smartwatch display. We demonstrate continuous 2D finger tracking with interactive, rectified graphics, transforming the arm into a touchscreen.

Latest News

  • Sep 20 Two papers submitted to CHI 2020.
  • Aug 5 Working with Gregory Abowd to complete my thesis.
  • Jun 21 Two papers got conditionally accepted at UIST 2019.
  • May 21 Proposed my dissertation.
  • Mar 25 10 days before UIST deadline. Start writing Thesis proposal.
  • Mar 15 Two papers got CHI Honorable Mention Award!
  • 2019 - Feb 25 UIST 2019 projects in full swing.
  • Dec 15 Thesis committee finalized. Will propose May 2019.
  • Nov 25 Zhuoshu joined Google Pittsburgh!
  • Oct 15 Attend UIST at Berlin.
  • Jun 6 Start internship at MSR, Redmond.
  • Apr 20 Attend CHI at Montreal.
  • 2018 - Jan 19 New semester started.
  • Oct 22 Attended UIST 2017 at Quebec City.
  • Sep 23 One week off. Visiting Zhuoshu in New York.
  • Sep 10 Invited to give a talk at CCTV2.
  • Jul 11 Disney projects in full swing.
  • Jul 5 Kayak and swim at North Shore.
  • May 8 Present and demo Electrick at CHI 2017.
  • 2017 - Jan 2 Back in Pittsburgh.
  • Dec 22 Visit ASU and Ling.
  • Dec 17 Visit Hong Kong to see my wife.
  • Oct 20 Back in Pittsburgh.
  • Oct 16 Attend UIST 2016 @ Tokyo, give AuraSense presentation.
  • Oct 09 Talk about research and share experience living abroad with ICMLL lab.
  • Oct 06 Wonderful wedding ceremony with two families and friends at Beijing.
  • Sep 22 post-CHI party at Union Grill. Preparing for my wedding.
  • Aug 28 CHI 2017 projects final push.
  • Jun 26 Three lab papers got accepted at UIST 2016. Go FIGlab!
  • Jun 1 Summer projects for CHI 2017 are in full swing.
  • May 16 I got married!
  • Apr 13 UIST 2016 Paper submitted. Fly to St. Louis for weekends.
  • Apr 1 UIST 2016 in full swing.
  • Mar 23 Qualcomm Innovation Fellowship finalist presentation and demo.
  • Jan 26 Pittsburgh Penguins vs. New Jersey Devils. We won!
  • 2016 - Jan 9 Got back to Pittsburgh. New semester started!
  • Dec 16 Filming for CHI project done. Flying back to Beijing.
  • Nov 30 CHI rebuttals submitted.
  • Nov 26 Host friends from high school over thanksgiving.
  • Nov 11 Reunion dinner with CoDelab friends. Wonderful UIST2015.
  • Nov 7 Heading for UIST 2015, Charlotte, NC.
  • Oct 28 Demo at Engedget, NYC.
  • Oct 26 Received a happy birthday suprise from the lab.
  • Sep 12 CHI 2016 projects final push.
  • Aug 31 First day as a PhD student.
  • Aug 23 Summer project user study began.
  • Aug 20 Tomo and Quantifying Electrostatic Haptic Feedback got accepted by UIST and ITS 2015.
  • Jun 3 Summer projects in full swing.
  • May 20 Tour at DC with family.
  • Apr 15 Party after UIST submission at Butter Joint.
  • Apr 7 UIST 2015 in full swing.
  • Mar 24 Make food storage in the lab for UIST late night work.
  • Mar 21 Had a wonderful visit at Cornell Tech NYC and Ithaca.
  • Feb 14 Extreme cold weather in St Louis.
  • Jan 21 User test for the Fitts Law Project. 10 down, 10 to go!
  • Jan 13 Back at Pittsburgh.
  • 2015 - Jan 5 V1.0 bio-impedance meter board is sent for printing.
  • Dec 14 Went back home. Happy birthday mom!
  • Oct 9 ACM UIST conference Student Innovation Contest 1st Most Creative Award for our project!
  • Aug 22 Finished my internship at Kinoma, Marvell.
  • Jun 18 Won the first place in IoT Hackathon, evironment category.
  • May 29 The third day as intern. Developed an alarm using openweathermap and Google TTS API.
  • May 25 Arrive at Santa Clara for the summer intern.
  • May 07 Final exam of 15213, done with high score.
  • Apr 23 Final presentation of ZipperSense.
  • Apr 18 Travel to Phily, play basketball with friends.
  • Apr 04 V3.0 PCB board for the ZipperSense is sent for printing.
  • Mar 07 V2.0 PCB board for the ZipperSense is sent for printing.
  • Feb 21 V1.0 PCB board for the final project of gadget class is being printed.
  • Jan 22 Turned an old cushion and a box into a stray cat's nest.
  • Jan 20 30-minutes running, 1st day. The goal is to beats the number of days last semester.
  • Jan 13 New semester begins. I'm so excited.
  • Jan 11 Back to Pittsburgh.
  • Jan 05 St Louis snow storm. Store food and water.
  • 2014 - Jan 01 Went to St Louis to spend the first day of the new year with my girl friend.
  • Dec 27 Went to The Grand Canyon which is truly grand.
  • Dec 24 Went to Las Vegas.
  • Dec 02 Successful presentation of 24780 C++ class's final project--"Interactive Fish"
  • Nov 28 Have a big Thanks-giving Turkey dinner in Jake's parents' house.
  • Nov 14 Paper and video finished for TEI. Everyday 1000-yards-swimming over 66 days!
  • Nov 05 Prototype1 of Heart Pulse is completed. Busy finishing the work for TEI conference.
  • Oct 21 Three final projects proposal. I'm really excited to get start.
  • Oct 9 Top-5 program in C++ class. Reward is bowling bowl game tickets!
  • Oct 6 Everyday 1000-yards-swimming, over 28 days.
  • Oct 1 Perform well in C++ python midterm. Finally make our silicon mask work with solenoids.
  • Sep 21 Everyday 1000-yards-swimming, over 13 days.
  • Sep 9 Visit Prof. Dale's house. Having fun.
  • Aug 30 First week in CMU. Cool! Now going to St. Louis for weekend
  • Aug 13 Orientation as new graduated student. End with a hugh BBQ!
  • Aug 8 Arrive at Pittsburg.
  • Jul 28 Bought one pair of hiking shoes.
  • Jul 21 Pack up stuff for studing abroad.
  • Jul 20 Send our pet dog to school.
  • Jul 19 Change domain name to "bennyzhang.me"
  • Jul 7 Return to Beijing.
  • Jul 2 Arrive at Liaoyuan.
  • Jun 27 Best barbecue I ever had. Also had the local spicy soup and noodle. Super spicy!
  • Jun 25 Depart from my dorm. Head for Changchun, my roommate's hometown.
  • 2013 - Jun 15 Pack up my stuff. Only ten days to leave my Beihang University.
More >

Fun Projects