A Formative Evaluation of a Comprehensive Search System for Medical Professionals

Medical doctors need rapid and accurate answers, which they cannot easily find with current search systems. This paper describes a formative evaluation of a comprehensive search system for medical professionals. The study was designed to guide system development. The system features included search in text and 2D images, machine translated summaries of search results, as well as query disambiguation and suggestion features, and a comprehensive search user interface. The study design emphasizes qualitative user feedback, based on realistic simulated work tasks and data collection with spontaneous and prompted self-report, written and spoken feedback in response to questionnaires, was well as audio and video recordings, and log files. Results indicate that this is a fruitful approach to uncovering problems and eliciting requirements that would be harder to find in a component-based evaluation testing each feature separately.