Haptics: motivations and challenges

an introduction with concrete examples

Jerome Pasquero March 11th, 2011

Presentation overview
1. Introduction Importance of touch Kinesthetic vs. tactile touch 2. Haptic devices and applications Research world Commercial world 3. Challenges in haptics High performance actuators Haptic rendering Roles of haptics in HCI 4. Summary

What is haptics?

hap!tics [hptiks], [noun] tactile sensation applied to computer applications:

the science of applying tactile sensation to computer applications in order to enable users to receive feedback in the form of felt sensations.
Microsoft Encarta

Why is touch so important?

sense of awareness only bidirectional sense unconscious exploration conscious and focused exploration strong channel of natural communication

Two haptics senses

1. kinesthetic (proprioceptive)
Sensations of motion and force arising in muscles and joints.

2. tactile (cutaneous)
Tactile sensing relates to sensations mediated by the skin.

 and two modes of interactions

1. kinesthetic:
force feedback

2. tactile:
heat mechanical pressure electricity friction vibration etc.

Touch sensors
Goldstein 1999

Kinesthetic Sensor

Tactile Sensors

Homunculus

Real vs. virtual environment

MIT Touchlab

Force feedback devices

MPBTs Freedom-6S If a real handle is

Iwatas Haptic Master

virtually attached to a virtual tool interacting with virtual objects, and if this handle is subject to the same perturbations that would be caused by a real tool interacting with real objects, then realism is achieved.

Tactile feedback devices

Refreshable Braille Display

20 mm

Miniature eccentric motor

Why use haptics in HCI?

Natural interface Bidirectional communication channel Overcrowded visual channel Critical manual operations

Why are haptic devices only starting to emerge now?

Touch is still a large mystery Special hardware requirements Too much focus on vision Potential roles for artificial haptic feedback are still unclear

Research and design in haptics require knowledge in

Control Psychophysics Physiology & neurophysiology Mathematics Electromechanical design Software HCI And more...

Recent growth

http://ngrams.googlelabs.com

Haptic devices
research

Pantograph McGill University

Tactile Display Forschungszentrum Karlsruhe

Scooter Northwestern U.

Haptic devices
commercial

Note taker Visuaide

Laparoscopic impulse Immersion

Freedom 6S MPB Technologies

Cybergrasp Virtual Technologies

Haptic Mice HP, Logitech, Saitek

iDrive controller BMW and Immersion

Applications
the entertainment industry the automotive industry the medical field rehabilitation e-commerce engineering design virtual reality sensory substitution systems research mobile devices

Potential roles for haptics in HCI

Substitute for another deficient or overloaded sense Make an experience more enjoyable Convey information Increase task performance

Challenges
1. Build high-performance actuators 2. Develop meaningful haptic rendering algorithms 3. Integrate haptics to HCI interactions

Challenge 1
high performance actuators
Requirements: high bandwidth high force stability power efficiency and safety small size real-time operation low friction ...

Challenge 1
high performance actuators

Single Motor

Shape Memory Alloys

Solenoid

Motors

Peltier Element

Electrostatic

Challenge 1
high performance actuators

Challenge 1
high performance actuators

Challenge 1
high performance actuators
e.g., Piezoelectric benders to represent Braille dots

Levesque & Pasquero

Challenge 2.a
force rendering
e.g., Phantom device from Sensable Technologies

Challenge 2.a
force rendering
e.g., Collision detection with the PHANTOM

PHANTOM Probe

Free Space

reaction force

Virtual Object
real avatar position F: Contact force f: Friction force virtual wall ideal avatar position

What happens if the probe penetrates the virtual object?

Challenge 2.b
tactile rendering
e.g., Bed-of-nails approach

Challenge 2.b
tactile rendering
e.g., Tactile rendering of graphics

How do we deal with 80 actuators in 1 cm2?

Levesque et al.

Challenge 2
summary
Differences with graphics: 1. Higher bandwidth 2. Local interaction Similarities with graphics: 1. Surface normals 2. Real-time requirements

Challenge 3
haptics for HCI

What role should haptics play in HCI? How to make haptics ubiquitous? How do you prove the added-value of haptics?

Challenge 3
haptics for HCI
e.g., The mobile space Main issues: High cognitive workload & competition for attention Limited screen real estate & awkward keypad space Unavailability of auditory and/or visual channel

Challenge 3
haptics for HCI
e.g., Reduce the number of glances at mobile device

Why would it be useful? Better awareness of the environment Less conspicuous gestures More appropriate in a social context

Challenge 3
haptics for HCI
e.g., Reduce the number of glances at mobile device

Pasquero & Luk

Challenge 3
haptics for HCI

scroll list on LCD screen

2 types of tactile icons

icons travel against scrolling motion

Challenge 3
haptics for HCI
e.g., Unconventional evaluation metrics

Task
scroll to a target minimize the nb. of (simulated) glances

Conditions
with (TACT) and without (CTRL) tactile feedback

Trials
24 trials per condition 10 participants: 3 x & 7x

Challenge 3
haptics for HCI
e.g., Unconventional evaluation metrics
Control Number of Glances With Tactile Feedback

28.5%

Wilcoxon Signed Ranked Test p<.01

Subject

Challenge 3
haptics for HCI
e.g., Eyes-free communications

capacitive lens rotating bezel

PCB with Bluetooth battery

piezoelectric actuator force sensing resistor plastic drum

Pasquero et al.

Summary
the big picture
1. Interface 2. Hardware Encoder resolution Motor & Power Electronics inertia friction torque ripple cogging max torque response time 3. Models x

Virtual Environment

surgical

procedure

x f
physical tools

Transmission backlash inertia friction

tissue modeling collision detection

Acknowledgements

@ McGill

@ UBC

@ RIM

Prof. Vincent Hayward Vincent Levesque Qi Wang Don Pavlasek

Prof. Karon MacLean Joseph Luk

Noel Stonehouse Scott Stobbe Steven Fyke

Questions?

visit laterotactile.com for more details