Brave NUI World: Designing Natural User Interfaces for Touch and Gesture
By Daniel Wigdor and Dennis Wixon
4/5
()
About this ebook
Brave NUI World is the first practical guide for designing touch- and gesture-based user interfaces. Written by the team from Microsoft that developed the multi-touch, multi-user Surface® tabletop product, it introduces the reader to natural user interfaces (NUI). It gives readers the necessary tools and information to integrate touch and gesture practices into daily work, presenting scenarios, problem solving, metaphors, and techniques intended to avoid making mistakes.
This book considers diverse user needs and context, real world successes and failures, and the future of NUI. It presents thirty scenarios, giving practitioners a multitude of considerations for making informed design decisions and helping to ensure that missteps are never made again.
The book will be of value to game designers as well as practitioners, researchers, and students interested in learning about user experience design, user interface design, interaction design, software design, human computer interaction, human factors, information design, and information architecture.
- Provides easy-to-apply design guidance for the unique challenge of creating touch- and gesture-based user interfaces
- Considers diverse user needs and context, real world successes and failures, and a look into the future of NUI
- Presents thirty scenarios, giving practitioners a multitude of considerations for making informed design decisions and helping to ensure that missteps are never made again
Daniel Wigdor
Daniel Wigdor is an Assistant Professor of computer science at the University of Toronto. Before joining U of T, he worked at Microsoft in nearly a dozen different roles, among them serving as the User Experience Architect of the Microsoft Surface product, and as a cross company expert in the creation of Natural User Interfaces. Before joining Microsoft, he previously conducted research in advanced user interfaces and devices at Mitsubishi Electric Research Labs, and at the Initiative in Innovative Computing at Harvard University. He is also co-founder of Iota Wireless, a company dedicated to the commercialization of NUI technologies for mobile phones. Daniel’s work has been described in dozens of publications in leading international conferences, journals, and books. His is the recipient of a Wolfond Fellowship and an ACM Best Paper Award.
Related to Brave NUI World
Related ebooks
Approachable Accessibility: Planning for Success Rating: 0 out of 5 stars0 ratingsAI and UX: Why Artificial Intelligence Needs User Experience Rating: 0 out of 5 stars0 ratingsDesign Beyond Devices: Creating Multimodal, Cross-Device Experiences Rating: 0 out of 5 stars0 ratingsDigital Experience Design: Ideas, Industries, Interaction Rating: 0 out of 5 stars0 ratingsSmart Things: Ubiquitous Computing User Experience Design Rating: 4 out of 5 stars4/5Contextual Design: Design for Life Rating: 4 out of 5 stars4/5The Visual Connection: Humans@WORK, #1 Rating: 0 out of 5 stars0 ratingsPersuasive Technology: Using Computers to Change What We Think and Do Rating: 4 out of 5 stars4/5Design Wise: A Guide for Evaluating the Interface Design of Information Resources Rating: 0 out of 5 stars0 ratingsStudying and Designing Technology for Domestic Life: Lessons from Home Rating: 0 out of 5 stars0 ratingsUniversal UX Design: Building Multicultural User Experience Rating: 0 out of 5 stars0 ratingsThere's Not an App for That: Mobile User Experience Design for Life Rating: 1 out of 5 stars1/5Writing for Interaction: Crafting the Information Experience for Web and Software Apps Rating: 3 out of 5 stars3/5Observing the User Experience: A Practitioner's Guide to User Research Rating: 5 out of 5 stars5/5Visual Usability: Principles and Practices for Designing Digital Applications Rating: 0 out of 5 stars0 ratingsFuture WI-FI Communication Technology Rating: 0 out of 5 stars0 ratingsDeveloping Accessible iOS Apps: Support VoiceOver, Dynamic Type, and More Rating: 0 out of 5 stars0 ratingsThe Business of UX Writing Rating: 0 out of 5 stars0 ratingsLiving in Information: Responsible Design for Digital Places Rating: 3 out of 5 stars3/5Pervasive Information Architecture: Designing Cross-Channel User Experiences Rating: 3 out of 5 stars3/5Design Thinking in Software and AI Projects: Proving Ideas Through Rapid Prototyping Rating: 0 out of 5 stars0 ratingsA Web for Everyone: Designing Accessible User Experiences Rating: 4 out of 5 stars4/5Design Transitions: Inspiring Stories. Global Viewpoints. How Design is Changing. Rating: 0 out of 5 stars0 ratingsDesign Research Through Practice: From the Lab, Field, and Showroom Rating: 3 out of 5 stars3/5Assistive Technology Specialist - The Comprehensive Guide: Vanguard Professionals Rating: 0 out of 5 stars0 ratingsDesign for Emotion Rating: 0 out of 5 stars0 ratingsPaper Prototyping: The Fast and Easy Way to Design and Refine User Interfaces Rating: 4 out of 5 stars4/5Personal Knowledge Graphs: Connected thinking to boost productivity, creativity and discovery Rating: 0 out of 5 stars0 ratingsComputer Vision Using Deep Learning: Neural Network Architectures with Python and Keras Rating: 0 out of 5 stars0 ratings
Computers For You
CompTIA Security+ Get Certified Get Ahead: SY0-701 Study Guide Rating: 5 out of 5 stars5/5Slenderman: Online Obsession, Mental Illness, and the Violent Crime of Two Midwestern Girls Rating: 4 out of 5 stars4/5The Invisible Rainbow: A History of Electricity and Life Rating: 4 out of 5 stars4/5101 Awesome Builds: Minecraft® Secrets from the World's Greatest Crafters Rating: 4 out of 5 stars4/5Procreate for Beginners: Introduction to Procreate for Drawing and Illustrating on the iPad Rating: 0 out of 5 stars0 ratingsStandard Deviations: Flawed Assumptions, Tortured Data, and Other Ways to Lie with Statistics Rating: 4 out of 5 stars4/5Elon Musk Rating: 4 out of 5 stars4/5Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are Rating: 4 out of 5 stars4/5CompTIA IT Fundamentals (ITF+) Study Guide: Exam FC0-U61 Rating: 0 out of 5 stars0 ratingsNetwork+ Study Guide & Practice Exams Rating: 4 out of 5 stars4/5SQL QuickStart Guide: The Simplified Beginner's Guide to Managing, Analyzing, and Manipulating Data With SQL Rating: 4 out of 5 stars4/5Alan Turing: The Enigma: The Book That Inspired the Film The Imitation Game - Updated Edition Rating: 4 out of 5 stars4/5The ChatGPT Millionaire Handbook: Make Money Online With the Power of AI Technology Rating: 0 out of 5 stars0 ratingsGrokking Algorithms: An illustrated guide for programmers and other curious people Rating: 4 out of 5 stars4/5Deep Search: How to Explore the Internet More Effectively Rating: 5 out of 5 stars5/5Mastering ChatGPT: 21 Prompts Templates for Effortless Writing Rating: 5 out of 5 stars5/5People Skills for Analytical Thinkers Rating: 5 out of 5 stars5/5Practical Lock Picking: A Physical Penetration Tester's Training Guide Rating: 5 out of 5 stars5/5Artificial Intelligence: The Complete Beginner’s Guide to the Future of A.I. Rating: 4 out of 5 stars4/5The Professional Voiceover Handbook: Voiceover training, #1 Rating: 5 out of 5 stars5/5Ultimate Guide to Mastering Command Blocks!: Minecraft Keys to Unlocking Secret Commands Rating: 5 out of 5 stars5/5CompTIA Security+ Practice Questions Rating: 2 out of 5 stars2/5Dawn of the New Everything: Encounters with Reality and Virtual Reality Rating: 4 out of 5 stars4/5How to Create Cpn Numbers the Right way: A Step by Step Guide to Creating cpn Numbers Legally Rating: 4 out of 5 stars4/5
Reviews for Brave NUI World
1 rating0 reviews
Book preview
Brave NUI World - Daniel Wigdor
Index
PART I
Introducing the NUI
1 Introduction
2 The Natural User Interface
3 Ecological Niche
Chapter 1
Introduction
Keywords
Moore’s law, GUI, WIMP, NUI, pitfalls
All things will be produced in superior quantity and quality, and with greater ease, when each man works at a single occupation, in accordance with his natural gifts, and at the right moment, without meddling with anything else.
—Plato
In the decades since the first digital computers were programmed using mechanical switches and plug boards, computing and the ways in which people interface with computers have evolved significantly. Some aspects of this evolution have both been anticipated and withstood the test of time. Moore’s law is an example. The law states that the number of transistors that can be placed inexpensively on an integrated circuit will double approximately every two years. The trend that this law describes has created opportunities for the growth of computing and its adoption into many aspects of our lives. As computers have increased in power and decreased in size and cost, new form factors have been created (e.g., smart phones, PDAs, and digital cameras), new platforms have evolved (e.g., the Internet), new infrastructures have become widely available (e.g., GPS), new industries have arisen (e.g., computer games), and new application families (e.g., spreadsheets, document processing, image creation, modification and sharing) have flourished. All of these trends have resulted in the democratization of computing as the number of people directly interacting with computers has steadily increased. This proliferation of computing has transcended national boundaries and permeated nearly all economic classes. It has changed the way people work, play, and interact with one another.
While the increase in computing power has been more or less continuous, the interfaces between human and computers have evolved more discontinuously. A widely held perspective is that interfaces have passed through phases. These phases are loosely defined but can be thought of as the phase of typing commands (the command line), followed by the graphical user interface (GUI). More specifically, most computers with which people interact regularly are based on the desktop metaphor (so called because windows are allowed to overlap, like paper atop a desk) and rely on a known set of user interface elements, commonly referred to as WIMP: windows, icons, menus, and pointers.
During that evolution some other contenders for the predominant interface, such as menu-only systems, did not attain dominance or widespread use. Instead, they were limited to niche applications, such as ATMs and televisions. A deeper analysis would show that many popular applications contain elements of each style. In effect, they are hybrids. For example, typical GUIs make use of menus (pull-down menus) and forms (dialog boxes and property sheets). In some cases, such as spreadsheets, the interaction style includes command-like elements, that is, complex formulas and sophisticated functions. Similarly, another class of successful applications, document processing, also introduces new concepts such as what you see is what you get,
(WYSIWYG) that is, what is on the screen is a reasonably faithful rendition of what will be printed. In both cases, the elements of the GUI are accompanied by particularly useful and appealing aspects that propelled these applications to wide adoption. It is important to be cognizant of the fact that these approaches are neither preordained nor task neutral. WYSIWYG, for example, fundamentally shapes the way people spend their time composing text—simultaneously focusing on both form and content.
While a thoughtful interpretation of the history of human-computer interaction is complex and nuanced, certain generalizations are evident. The way in which humans interact with computers has evolved. That evolution has enabled more people to do more things with computers. It has led to a vast and rapid increase of the volume, scope, and diversity of the computer business.
There are many perspectives from which we could view this evolution. Some are merely tautological. For example, a common view is that more people use more computation because the barriers have been reduced and the functionality of the machines has increased. While true, that characterization provides precious little insight. To provide it some intellectual weight, we need to be clearer with respect to what barriers have been reduced and which functions have been provided. It is also useful to examine closely the development of certain computing niches,
which thrive in limited but well-defined contexts.
Some of the early analysis of the GUI provided just this kind of deeper insight. Analysts pointed out that recognizing and choosing were easier than remembering then typing. In other words, with its menus, dialog boxes, icons, and familiar work spaces, the WIMP GUI represented a lower barrier for users than a command line interface. This difference becomes clearer if we consider specific applications. By and large, it is easier to learn and use a word processor to create simple documents than to edit in a markup language. In addition, the often-overlooked advantage of the computer, that is, that it produces revisable work products, was fully realized with a word processing system.
Functionality gains are also apparent for most users. Before the computer, a skilled typesetter could produce formatted documents, but the average citizen was confined to typing final work products in a mono-spaced font, with cumbersome correction tools. The combination of reduced thresholds for learning, easier recall, increased functionality for the average user, the reduced cost promised by Moore’s law, and the widespread capability to revise one’s work without redoing it signaled the explosive growth of a number of well-featured and -designed applications that adopted the desktop computer and the subsequent consignment of more traditional approaches to specialized niches.
When the desktop GUI was first created and made widely available, its ultimate fate was unclear. It was derided by many experts who coined its current name: the WIMP interface. Although WIMP stood for windows, icons, menus, and a pointer, it implied that the users of the GUI were not the manly men who had mastered the previous, more arcane ways of interacting with computers. Ultimately, those supposedly more manly folks were consigned to the social position of specialists or hobbyists. We can see this pattern in many other domains: personal transport, cooking, penmanship, and CB radio operators.
Now we stand at the brink of another potential evolution in computing. Natural user interfaces (NUIs) seem to be in a position similar to that occupied by the GUI in the early 1980s. Like the desktop GUIs, NUIs promise to reduce the barriers to computing still further, while simultaneously increasing the power of the user, and enabling computing to access still further niches of use. But just as GUIs did not simply make command systems easier, NUIs are not simply a natural veneer over a GUI. Instead, like GUIs, NUIs have a set of strengths based on what they make easier, how they make those things easier, how they shape the user’s interaction with technology, which niches they fit in, and whether or not these niches expand to dwarf the space occupied by traditional GUIs.
When examining this history and anticipating the future, we should not be distracted by single instances. Many of these will be failures and will not represent any overall trend. The failure of makes and models of some cars did not end the phase of personal transport. The failure of many GUI products and the inevitable consolidation of the marketplace did not impede the overall growth of personal computing or prevent GUIs from pre-eminence. It is as dangerous to generalize to the future based on a few examples as it is hard to anticipate the future when looking from our current perspective.
In this book we do not aim to provide an exhaustive overview of the NUI. We do not predict the future of human-computer interaction. We do not assume the predominance of NUI-based designs. We do not provide a complete set of rules for creating a successful NUI, because NUI is not yet at a state of evolution or standardi-zation to allow for such a definition.
We can make some relatively safe predictions. NUIs are here to stay. They either will find a successful niche, like menu system ATMs, or will come to dominate the computer landscape. If the latter comes to pass, we can still expect GUIs to persist in specialized environments. The ultimate evolution of the NUI will be determined not by the analysts and the critics, but by those who step forward and take the risk to build true NUI applications. Here we offer a way to think about NUIs that is counter to the predominant metaphor. This perspective on the NUI suggests that NUIs provide an enjoyable way for novices to move quickly and seemingly effortlessly to skilled practitioners. This approach involves more than being natural
or intuitive. It means that the domain of use and the requirements of context are carefully assessed. It also means that the conventions of the GUI should be studiously ignored. It requires careful design and rigorous research. We give some guidance on how to do those things and how to re-conceptualize the NUI.
We offer a number of essays on the NUI and on methods that can be utilized to achieve it, written from the perspective of two journeyman user experience experts who have had the opportunity to immerse themselves in the nitty-gritty of designing, engineering, building, testing, researching, and shipping multiple products that have come to define the category. Overall, the essays are intended to provide a nuanced set of perspectives on NUI systems. These perspectives range from specific descriptions of the syntax and semantics of the NUI to broad analyses of the NUI in terms of the history of computing. Each essay is composed of the same essential elements. Each concludes with specific, concrete design guidelines meant to help take words into action. Those guidelines are divided into three types: must, those that we believe are necessary conditions to achieving a natural-feeling experience; should, those elements that, while nonessential, have been found to add greatly to achieving such an experience; and could, those guidelines that may apply only to certain contexts or