Goals and Constraints

As Graduate Student Researchers working on the occupant satisfaction survey project at CBE, we were fortunate that we had very few organizational constraints to contend with. The research group took great interest in the datamining tool, was receptive to our ideas, and numerous members of the organization took the time to work with us during the needs assessment and evaluation stages, giving us feedback and nurturing our design. The constraints we worked with, then, were largely technical in nature. We outline them here:

Single benchmarking solution
To CBE, the work we perform on the reporting tool and datamining tool is really all part of the same thing — the "benchmarking building quality" aspect of the project. The overall goal is to provide a tool or set of tools to enable researchers to analyze survey data, looking for correlations between building technologies and self-reported occupant satisfaction levels. Thus one of the major issues was whether to supersede the reporting tool, modify it, or work in tandem with it. At the very least we required consistency with the existing reporting tool, leading us to choose the same chart rendering software for a consistent look and feel, and also developing a plan for integrating with the reporting tool.

Survey data structure
We inherited the backend SQL Server database and while we worked with this existing structure, we also had approval to modify it if needed. So the existing database structure was a constraint, but a somewhat flexible constraint. Any modifications had to take into account the 100,000+ existing survey response records, as well as the other tools accessing that data. Those include the survey itself, and the reporting tool, either of which are in active use at any given time.

Frequently updated survey data accessed over the Internet
The organization currently runs 20-30 surveys per year, and this is increasing as the project gains momentum. So working from a local copy of the database is not feasible due to the size mentioned above, as well as the need to keep in sync with incoming survey data. The database resides on a dedicated server maintained by the survey project's Internet Service Provider, and must therefore be accessed over the Internet.

Evolving survey text and customized modules
Survey text is not the same across survey implementations. Rather, each survey is a snapshot in time of the text as it was implemented in a particular building. This is due in part to gradual evolution of the core survey over time (as modifications are made, or new needs arise), but especially because surveys may have customized survey “modules” attached. These are used to test occupant comfort with new building technologies, for example, or to poll respondents for their opinions about some other issue unrelated to building technology. This means that we cannot count on all of the categories, questions, and responses to be identical across different surveys — in fact, a perfect match is quite rare.

The situation reaches another level of complexity when considering that even when questions are identical, responses can differ due to wording changes, newly developed responses, and so on. Further, it is essential that we be able to match up non-identical question or response strings if the intent of the question or response is the same. Fortunately we were able to capitalize on the existing backend database architecture that enables us to keep track of which categories, questions and responses belong to which survey, and to match up the ones that are similar in intent.

Variable length of survey question and response text
Variable text length hampered our ability to predict how the information would be displayed. For example, the potential for long text on any given response makes it difficult to label some of the chart types as we would wish. For example, ideally pie charts would have the labels directly on the slice or pointing to the slice, rather than using the indirection of a legend [Tufte, 1983]. However, with the automated chart rendering application, the length of the labels affects the size of the chart itself. To make reliable comparisons, it is important that each of the charts (within a given chart type) be the same size. We believe that constant chart size is more crucial than direct slice labeling. Thus we do not label each slice with the text of a response.

Similarly, the text of questions and responses in the column and row headers is highly variable and frequently is either short or quite lengthy. Our design had to be flexible enough to be able to accommodate either extreme.

Security and permissions
The data collected by these surveys is of great value to the building industry partners who commission them, and confidential. The users who will have access to ArchMiner are at varying levels of permission. For example, in-house CBE researchers will be able to analyze data from all surveys in the database; CBE partners will be able to analyze most of the surveys; and non-partners will only have access to those surveys they commissioned.

Delivering a working application to a real organization
While this project serves as our capstone Master's project, it is in fact intended for use by CBE and its industry partners. In order to save time in the long run, we created our “prototype” in the final high-level language, Java — this is actually our real application. We will continue to work on it throughout the summer and perhaps beyond, depending on the needs and resources at CBE.