Designing responsive GUIs: An intelligent interface that tailors information presentation by responding to a user’s cognitive state

Submitted in partial fulfillment for the degree of Master of Graphic Design, 2018
Department of Graphic and Industrial Design / College of Design / North Carolina State University

A full document of this project is published in Then Finally, a digital publication from the Master of Graphic Design program at NC State University.

Abstract: A person faced with caring for a loved one with a serious health diagnosis has an immediate thirst for knowledge, even while their cognitive ability to find—let alone comprehend—useful information may be hindered due to their traumatized, high-stress mental state. The design of most online health information search platforms do not consider the cognitive state of this type of user, even though new technologies, such as machine learning and artificial intelligence, have the potential to offer personalized support for this particular information-seeking circumstance. The design of standard search tools and features encountered on ordinary health information websites typically take a one-size-fits-all approach. 

The objective of this study is to determine how intelligent human-computer interfaces can present information in meaningful and clearly comprehensive ways by responding to the health information-seeker’s cognitive state—in particular, for someone who is under duress from a recent medical diagnosis. In this study, cognitive state refers to the psychological and emotional state of the user. Methods included semi-structured qualitative interviews with 20 patient advocates from the NC Rare Disease Council and subsequent prototype testing. Data suggests that people under duress prefer that complex health information be presented in a minimal (i.e., simple content structure) fashion using assistive delivery strategies such as withholding, gathering, and prioritizing. This project suggests a useful framework for professionals involved in the design of medical information search tools, user-centric design methods, and intelligent interface design.

The conceptual framework for this study guided the design of prototypes.

The conceptual framework for this study guided the design of prototypes.

Project description: The concept of my master’s thesis comes from an activity ubiquitous in our everyday life - searching the web. Today’s web user relies heavily on search engines for information. I am particularly interested in how people use the web to search for health information. According to numbers from the Pew Research Center’s Internet and American Life Project, 72% of internet users say they have looked online for health information within the past year. And often, the content of web search result pages contributes to heightened anxiety in the user. This negative human experience of searching for health information online is an area where design intervention can significantly improve the experience for the end-user.

The design of most online health information search platforms does not consider the cognitive state of the user. The design of standard search tools and features encountered on ordinary health information websites typically takes a one-size-fits-all approach. 

New technologies, such as machine learning, augmented intelligence, and data fusion, have the potential to offer personalized support. As designers, this is exciting news! Computers are no longer dumb boxes that limit the types of interactions we can facilitate through designed interfaces. Computers can learn about us. We can begin to see past the technological limitations requiring one interface for all users and instead create one interface which will adapt to each individual.

For this project, I was interested in a very particular health information-seeking circumstance. I considered a parent faced with the devastating news of having a child diagnosed with a serious, life-threatening rare disease. A parent in this circumstance has an immediate thirst for knowledge concerning the medical condition and how to manage care moving forward. Having access to large quantities of information is an essential part of overseeing care. Receiving and interpreting this information and subsequently taking timely action will provide their child with opportunities that could potentially extend life and health. 

The number of factors that come into play in the care and support of the recently diagnosed child creates a tremendous need for understanding complex medical information. As the parent begins to seek this information, their cognitive ability to find — let alone comprehend — it may be hindered due to their traumatized, high-stress mental state. 

The National Institutes of Health (NIH) defines a rare disease as one that affects fewer than 200,000 people. The word “rare” infers these conditions are uncommon. However, there are over 7,000 classified rare diseases and more than 300 million people worldwide currently diagnosed. Rare diseases are chronic and often life-threatening. 

The research question for this study is How can the design of a responsive graphical user interface (GUI) for an online data repository adapt to different cognitive states of primary caregivers who are engaged in information-seeking and knowledge-acquisition search processes to learn about a dependent’s medical condition?

This study employed qualitative research methods by collecting data from 20 patient advocates from the NC Rare Disease Advisory Council through semi-structured interviews. I created scenarios, personas, and prototypes.

As I considered designing an intelligent interface that tailors information presentation by responding to a user’s cognitive state, I broke the problem down into several potential solution sets. I derived a collection of information presentation strategies aimed at presenting information to people under duress. I proposed a method for developing a looping series of user interactions that promotes discovery. I introduced seven sets of potential features for a responsive graphical user interface. Some of this work is displayed below.

As designers, we are living in exciting times. The landscape of design is ripe with new technologies. Machine learning, augmented intelligence, and data fusion create opportunities for designers to develop experiences for one individual rather than all users. It’s time we move beyond the one-size-fits-all solution. No longer restrained by old technological limitations, we can create one interface which will adapt to each individual. By creating adaptive interfaces, we can communicate with a user based on a meaningful characteristic, such as their cognitive state. 

The information presentation strategies defined below are influenced by both Gestalt principles (the top six figures above) and pictorial depth cues (the bottom three figures). The focus of Gestalt Theory is the idea that the grouping of characteristically similar stimuli causes people to structure or interpret a visual field or problem in a certain way (Werthmeimer, 1923).  Pictorial depth cues are often used in flat compositions to create an impression of depth and space for the viewer. The presence of these cues serves to trigger visual memories of mental constructs of three-dimensional space in the viewer (Rock, Shallo, & Shwartz, 1977).

The information presentation strategies defined below are influenced by both Gestalt principles (the top six figures above) and pictorial depth cues (the bottom three figures). The focus of Gestalt Theory is the idea that the grouping of characteristically similar stimuli causes people to structure or interpret a visual field or problem in a certain way (Werthmeimer, 1923). Pictorial depth cues are often used in flat compositions to create an impression of depth and space for the viewer. The presence of these cues serves to trigger visual memories of mental constructs of three-dimensional space in the viewer (Rock, Shallo, & Shwartz, 1977).

Early prototypes in my study explored how filtered information could be presented to the user in a layered manner. The user can self-direct into deeper layers of information. Depth is used as a way of prioritizing information.

Early prototypes in my study explored how filtered information could be presented to the user in a layered manner. The user can self-direct into deeper layers of information. Depth is used as a way of prioritizing information.

The bifocal display concept, adapted from Robert Spence and Mark Apperley (Spence, 1999), makes note of the fact that the information space is much larger than the screen containing the documents. Information is displayed in a “wrapped” fashion, where the view seen by the user is in focus with the suggestion that there is additional information that is easy to access through scrollability.

The bifocal display concept, adapted from Robert Spence and Mark Apperley (Spence, 1999), makes note of the fact that the information space is much larger than the screen containing the documents. Information is displayed in a “wrapped” fashion, where the view seen by the user is in focus with the suggestion that there is additional information that is easy to access through scrollability.

This study is showing the use of the bifocal display presentation technique with greyed out content.

This study is showing the use of the bifocal display presentation technique with greyed out content.

In this study, I explored the use of both the bifocal presentation technique and graying out content. Pictorial depth cues (interposition, shadow, relative size) are employed to signify “Specialists” as being the information the user is currently interacting with.

In this study, I explored the use of both the bifocal presentation technique and graying out content. Pictorial depth cues (interposition, shadow, relative size) are employed to signify “Specialists” as being the information the user is currently interacting with.

A variant of the bifocal display, in the fisheye technique, an information object responds to a user’s scroll-over by enlarging content dynamically. This iteration of the fisheye technique exploits the distortion of secondary content. The increased distortion lowers the likelihood that these focus spaces will distract the user from the “Specialists” content. 

A variant of the bifocal display, in the fisheye technique, an information object responds to a user’s scroll-over by enlarging content dynamically. This iteration of the fisheye technique exploits the distortion of secondary content. The increased distortion lowers the likelihood that these focus spaces will distract the user from the “Specialists” content. 

For this prototype, the system facilitates the novice user to behave like the expert. The system automatically highlights and saves keywords. Information is tagged and filed to accommodate the user’s impaired ability to remember accurately. The user observes these behaviors happening in real time. The system is bridging the gap to demonstrate expert research behaviors.

For this prototype, the system facilitates the novice user to behave like the expert. The system automatically highlights and saves keywords. Information is tagged and filed to accommodate the user’s impaired ability to remember accurately. The user observes these behaviors happening in real time. The system is bridging the gap to demonstrate expert research behaviors.

This study explored the possibility of allowing space for a continuous annotated side navigation. Here, an ongoing rolling history sidebar provides at-a-glance information about where the user has been and what information they have interacted with during their current search episode. Users can add annotations to any item appearing in the activity feed, making notes for future reference.

This study explored the possibility of allowing space for a continuous annotated side navigation. Here, an ongoing rolling history sidebar provides at-a-glance information about where the user has been and what information they have interacted with during their current search episode. Users can add annotations to any item appearing in the activity feed, making notes for future reference.

In this prototype, the system subtly draws attention to connecting information as the user browses data on one topic. The spheres grow over the names of the other focus spaces in the lower navigation cues for the information zones. The size of each sphere indicates the quantity of connecting information in the other topic areas, aiding in the user finding connecting information.

In this prototype, the system subtly draws attention to connecting information as the user browses data on one topic. The spheres grow over the names of the other focus spaces in the lower navigation cues for the information zones. The size of each sphere indicates the quantity of connecting information in the other topic areas, aiding in the user finding connecting information.

A close up detail of the visual indicators indicating connecting information.

A close up detail of the visual indicators indicating connecting information.

This study explored the concept of withholding information from the user for an on-demand content delivery system. Rather than swamp the user with all the information up front, the user can dig into deeper layers of information at a self-directed pace.

This study explored the concept of withholding information from the user for an on-demand content delivery system. Rather than swamp the user with all the information up front, the user can dig into deeper layers of information at a self-directed pace.

Rock, I., Shallo, J., & Schwartz, F., (1977). Pictorial depth and related constancy effects as a function of recognition. Perception, 7(1), 3-19.

Spence, R., (1999). A framework for navigation. International Journal of Human-Computer Studies, 15(5), 919-945.

Wertheimer, M. (1923). Laws of organization in perceptual forms. First published as Untersuchungenzur Lehre von der Gestalt II, in Psycologische Forschung, 4, 301-350. Translation published in Ellis, W. (1938). A source book of Gestalt psychology (pp. 71-88). London: Routledge & Kegan Paul. Retreived from http://psychclassics.yorku.ca/Wertheimer/Forms/forms.htm

Next
Next

distributed cognition: a useful theory in HCI and interface design