Section: Overall Objectives

Technology for Interaction

Studying the interactive phenomena described above is one of the pillars of HCI research, in order to understand, model and ultimately improve them. Yet, we have to make those phenomena happen, to make them possible and reproducible, whether it be for further research or for their diffusion  [37]. However, because of the high viscosity and the lack of openness of actual systems, this requires considerable efforts in designing, engineering, implementing and hacking hardware and software interactive artifacts. This is what we call “The Iceberg of HCI Research”, of which the hidden part supports the design and study of new artifacts, but also informs their creation process.

“Designeering Interaction”

Both parts of this iceberg are strongly influencing each other: The design of interaction techniques informs on the capabilities and limitations of the platform and the software being used, giving insights into what could be done to improve them. On the other hand, new architectures and software tools open the way to new designs, by giving the necessary bricks to build with  [39]. These bricks define the adjacent possible of interactive technology, the set of what could be designed by assembling the parts in new ways. Exploring ideas that lie outside of the adjacent possible require the necessary technological evolutions to be addressed first. This is a slow and gradual but uncertain process, which helps to explore and fill a number of gaps in our research field but can also lead to deadlocks. We want to better understand and master this process –i. e., analyzing the adjacent possible of HCI technology and methods– and introduce tools to support and extend it. This could help to make technology better suited to the exploration of fundamentals of interaction and to their integration into real systems, a way to ultimately improve interactive systems to be empowering tools.

Computers vs Interactive Systems

In fact, today's interactive systems –e. g., desktop computers, mobile devices– share very similar layered architectures inherited from the first personal computers of the 1970s. This abstraction of resources provides developers with standard components (UI widgets) and high-level input events (mouse and keyboard) that obviously ease the development of common user interfaces for predictable and well-defined tasks and users' behaviors. But it does not favor the implementation of non standard interaction techniques that could be better adapted to more particular contexts, to expressive and creative uses. It often requires to go deeper into the system layers and to hack them until getting access to the required functionalities and/or data, which implies switching between programming paradigms and/or languages.

And these limitations are even more pervading as interactive systems have changed deeply in the last 20 years. They are no longer limited to a simple desktop or laptop computer with a display, a keyboard and a mouse. They are becoming more and more distributed and pervasive (e. g., mobile devices, Internet of Things). They are changing dynamically with recombinations of hardware and software (e. g., transition between multiple devices, modular interactive platforms for collaborative use). Systems are moving “out of the box” with Augmented Reality, and users are going “ inside of the box” with Virtual Reality. This is obviously raising new challenges in terms of human factors, usability and design, but it also deeply questions actual architectures.

The Interaction Machine

We believe that promoting digital devices to empowering tools requires better fundamental knowledge about interaction phenomena AND to revisit the architecture of interactive systems in order to support this knowledge. By following a comprehensive systems approach –encompassing human factors, hardware elements and all software layers above– we want to define the founding principles of an Interaction Machine:

• a set of hardware and software requirements with associated specifications for interactive systems to be tailored to interaction by leveraging human skills;

• one or several implementations to demonstrate and validate the concept and the specifications in multiple contexts;

• guidelines and tools for designing and implementing interactive systems, based on these specifications and implementations.

To reach this goal, we will adopt an opportunistic and iterative strategy guided by the designeering approach, where the engineering part will be fueled by the interaction design and study part. We will address several fundamental problems of interaction related to our vision of “empowering tools”, which, in combination with state-of-the-art solutions, will instruct us on the requirements for the solutions to be supported in an interactive system. This consists in reifying the concept of the Interaction Machine in multiple contexts and for multiple problems, before to converge towards a more unified definition of “what is an interactive system”, the ultimate Interaction Machine, which makes the main scientific and engineering challenge of our project.