Section: New Results

Fundamentals of Interaction

Participants : Michel Beaudouin-Lafon [correspondant] , Marianela Ciolfi Felice, Sarah Fdili Alaoui, Cédric Fleury, Carla Griggio, Wanyu Liu, Wendy Mackay, Nolwenn Maudet, Philip Tchernavskij, Theophanis Tsandilas.

In order to better understand fundamental aspects of interaction, ExSitu studies interaction under extreme situations. We conduct in-depth observational studies and controlled experiments which contribute to theories and frameworks that unify our findings and help us generate new, advanced interaction techniques.

On the theoretical side, in collaboration with Telecom ParisTech, we are bringing the tools and concepts from Information Theory to HCI. We conducted an information-theoretic analysis of human performance for command selection [21]. While a number of studies have focused on improving rapid command selection through novel interaction techniques, new interface design and innovative devices, user performance in this context has received little attention. We ran a controlled experiment to test the theory that the transmitted information from the user to the computer levels off as difficulty increases. Our reasoning is based on basic information-theoretic concepts such as entropy, mutual information and Fano's inequality. The important result is the bell-shaped behavior of the throughput as a function of command entropy, which shows that there is an optimal level of difficulty for a given input technique.

We also used the information-theoretic concept of mutual information, also known as information gain, in our BIG (Bayesian Information Gain) framework. We created BIGnav [20], a new multiscale navigation technique based on Bayesian Experimental Design where the criterion is to maximize the expected information gain from the next user input. In a controlled experiment, BIGnav was up to 40% faster than the standard pan-and-zoom technique. BIGnav creates a form of human-computer partnership (see below) where the computer challenges the user in order to maximize the amount of information extracted from the user's input. This work received a Best Paper Award at ACM CHI 2017, and the first prize for doctoral research from the Paris-Saclay doctoral school in computer science.

Finally, we continued our long-standing line of work on Fitts' law, with a novel analysis of minimal, as opposed to average, movement time in human aimed movement [18]. We showed that both metrics have a lot of support from theoretical and empirical perspectives and gave two examples, one in a controlled experiment and the other in a field study of pointing, where making the minimum versus average distinction is fruitful.

On the empirical side, we conducted two observational studies to better understand how people interact with technology. The first study [23] targeted expert graphic designers and their use of advanced computer tools. Traditional graphic design tools emphasize the grid for structuring layout. Interviews with professional graphic designers revealed that they use surprisingly sophisticated structures that go beyond the grid, which we call graphical substrates. These structures are not well supported by existing tools, so we developed two technology probes to explore how to embed graphical substrates into tools. Contextify lets designers tailor layouts according to each reader's intention and context, while Linkify lets designers create dynamic layouts based on relationships among content properties. We tested the probes with professional graphic designers, who all identified novel uses in their current projects. We incorporated their suggestions into StyleBlocks, a prototype that reifies CSS declarations into interactive graphical substrates. This work demonstrates that graphical substrates offer an untapped design space for tools that can help graphic designers generate personal layout structures.

The second study [30] targeted the operating system upgrade process that most users regularly have to go through to keep their system up to date. While current research has focused primarily on the security aspect of upgrades, we investigated the user’s perspective of upgrading software. We found that users delay major upgrades by an average of 80 days, and an extensive field study revealed that very few participants prepare for upgrades (e.g., by backing up files), and over half had negative reactions to the upgrade process and other changes (e.g., bugs, lost settings, unwanted features). During the upgrade process, waiting times were too long, feedback was confusing or misleading, and few had clear mental models of what was happening. Moreover, users almost never mentioned security as a concern or reason for upgrading, while interviews with technical staff responsible for one organization’s upgrades focused only on security and licensing, not user interface changes. This work shows that upgrades should be handled differently, offering users more control and decoupling security updates from the introduction of new features or the update of existing features.

These two sets of studies support our strong commitment to re-inventing interactive systems by identifying fundamental principles of interaction that unify, rather than separate, interaction styles in order to support the diversity of uses and users [33]. For example, most of our interactions with the digital world are mediated by apps: desktop, web, or mobile applications. Apps impose artificial limitations on collaboration among users, distribution across devices, and the changing procedures that constantly occur in real work. These limitations are partially due to the engineering principles of encapsulation and program-data separation, calling for new architectural principles [29]. Shareable dynamic media, which we have explored in our earlier work on Webstrates[5], provides an interesting approach as it blurs the limits between apps and documents and supports collaboration, distribution and flexibility as fundamental features [28]. In connection with these issues, we ran a workshop at the ACM CHI 2017 conference on HCI toolkits [37] where we discussed challenges and opportunities to develop new methods and approaches to design, evaluate, disseminate and share toolkits, as well as the technical, methodological and enabling role of toolkits for HCI research.