188/4 E-Commerce Group
Institute of Software Technology and Interactive Systems
Vienna University of Technology
Favoritenstrasse 9-11/188, A-1040 Vienna, Austria

Measuring User Expertise in Online Communities

Doctoral Thesis
First name: 
Last name: 
Matr nr: 
Dorn, J.
The thesis at hand addresses the challenge to identify and measure expertise of individuals. This task is highly relevant since the location of individuals’ expertise is crucial to organizations in order to assign the most appropriate people to given tasks. Such effective assignments support organizations in sustaining competitive advantage as well as in fostering innovation. However, the elicitation of expertise is challenging since knowledge resides first and foremost in the heads of individuals and thus is inherently elusive. We iteratively develop a method to quantify users’ expertise based on their submissions to online communities. An online community offers a communication platform to its users that facilitates the informal exchange of knowledge. As a consequence, when people share their ex- periences in problem-solving contexts, they demonstrate expertise regarding certain topics. The proposed method aggregates data obtained from such an online community and automatically generates users’ expertise models containing expertise topics along with users’ expertise levels. Thereby, expertise levels correspond to numerical values on an absolute scale. Expertise levels mapped on an absolute scale allow to compare one’s expertise with others’ as well as to staff teams according to the expertise levels needed. To evaluate the proposed method we conduct a series of experiments with students at our uni- versity. Since the method constitutes a composite of various calculation steps, each experiment covers either a specific step or several steps of the proposed method. We set up hypotheses that are based on each other to systematically explore both the characteristics of the method and the value of users’ submissions to reliable expertise calculation. The method’s calculation accuracy is measured by comparing the calculated expertise levels with the participants’ self-assessments.
Oct 2009