Section: New Results

Conceptual and Methodological Issues in Evaluating Multidimensional Visualizations for Decision Support

Participants : Evanthia Dimara [ISIR, Sorbonne Université, France] , Anastasia Bezerianos [ISIR, Sorbonne Université, France] , Pierre Dragicevic [correspondant] .

Figure 5. The three visualization techniques tested in our study.

We explored how to rigorously evaluate multidimensional visualizations for their ability to support decision making [22]. We first defined multi-attribute choice tasks, a type of decision task commonly performed with such visualizations. We then identified which of the existing multidimensional visualizations are compatible with such tasks, and evaluated three elementary visualizations: parallel coordinates, scatterplot matrices and tabular visualizations. Our method consisted in first giving participants low-level analytic tasks, in order to ensure that they properly understood the visualizations and their interactions. Participants were then given multi-attribute choice tasks consisting of choosing holiday packages. We assessed decision support through multiple objective and subjective metrics, including a decision accuracy metric based on the consistency between the choice made and self-reported preferences for attributes. We found the three visualizations to be comparable on most metrics, with a slight advantage for tabular visualizations. In particular, tabular visualizations allowed participants to reach decisions faster. Thus, although decision time is typically not central in assessing decision support, it can be used as a tie-breaker when visualizations achieve similar decision accuracy. Our results also suggest that indirect methods for assessing choice confidence may allow to better distinguish between visualizations than direct ones.

All supplemental material is on the project web page: aviz.fr/dm.