Low-fi Testing

First paper prototype


This prototype was the paper manifestation of the desired feature set, and an amalgamation of our sketches. The user first accessed a login dialog, and after authentication came to the main window. This consisted of an exploration panel in the center, for the chart display, with surrounding panels used to:
  • Select surveys and saved datasets (sets of surveys). Clicking the "Manage Datasets" button on this panel brought the user to the dialogs used to create and edit datasets.
  • Navigate through the categories in the selected survey or dataset. By clicking on a category, all the charts on that survey page would be displayed. Our intention was that by making this navigation mechanism so similar to the one in the Reporting Tool, users would find it familiar and easy to use.
  • Save charts. Checking a box on the chart placed a reference to the chart in a Library panel for later display or export. Clicking on the Export button showed the dialog used to save the charts and/or underlying data as specified by the user. The main workflow for comparing desired charts against one another was to first navigate to each of those charts through the categories, saving each one into the library. Then use the library to display just those charts in the exploration panel.
  • Make queries. Like the Filtering function in the reporting tool, a query could be created that when applied, would redraw the visible charts so they only showed the data that met the specified criteria.
  • Access advanced features. Below the chart display window was a toolbar used to determine the type of chart in the display, display statistics in a Stats panel, and display comments.

Second paper prototype


The first paper prototype was very flexible and feature-rich, but difficult for our users to navigate during testing. Our users gave us a lot of really valuable feedback, and we found the process to be so instructive that we immediately began work on creating and testing a second paper prototype.

It was too cumbersome for them to have to navigate through the categories, save the charts they wanted into a library, and then show the charts from the library in the exploration panel. It was clear that the library should not be a necessary intermediate step. We realized that we were needlessly duplicating too much of the existing reporting tool workflow, and that users really wanted to lay out charts for comparison in a crosstab format.

So we made headers for the rows and columns and put the controls directly on the headers. We considered the hierarchy of information in the database: Surveys contain Categories, which contain Questions, which contain Responses (at that time the terminology for "Responses" was "Answers"). We determined that only certain combinations of these levels were meaningful when creating a chart in one of the cells that represented an intersection between a column and row. Users would be prevented from making "illegal" combinations of the levels in the hierarchy.

To reflect this hierarchy, the column and row headers each had a tabbed panel containing three tabs: Dataset, Question, and Answer (there was no tab for Category, because although it is essential for navigating to the desired question, comparison against a Category is not meaningful). These tabs allowed the user to change levels on the column and row. Rather than having the Category, Question, and Answer controls on a side panel, we placed them directly onto the column and row headers so that they could serve as labels as well. To display charts, the user selected information from the column and row tabs, and charts appeared at the intersecting cells. A panel of radio buttons at the bottom determined whether the grid of charts displayed charts, data, or both.

We removed the advanced chart combination functionality in this version. We wanted to get the crosstab workflow hammered out first, and would add the combination charts later, time permitting. Also, we substantially reworked the dataset and export dialogs.

This time our users found the interface much easier to navigate, and were able to complete the tasks rapidly and with a much better success rate.