Mária Bieliková > Teaching

Promoting Sustainability and Transferability of Community Question Answering

Ivan Srba

Doctoral thesis project supervised by prof. Mária Bieliková

Motivation and Goals

Standard information retrieval tools, such as Google Search, represent the most popular way how to search for required information on the Web. However, successfulness of these tools decrease when a user wants to find information, which is highly context-specific, subjective (e.g. a recommendation), scattered across many sources or which cannot be easily described by keywords. In these cases, Internet users have a possibility to utilize alternative tools that are based on knowledge sharing in great online communities of people. One of the most successful examples of these community-based knowledge sharing systems is Community Question Answering (CQA).

Some of the existing CQA systems provide a possibility to ask questions without any topic restriction, such as Yahoo! Answers or Wiki Answers. On the hand, there are CQA systems focused on specific topic areas, for example Stack Overflow, which concerns only with questions related to programming.

Following the state of the art in the area of CQA systems, we identified open problems, which resulted from constant development of these systems.

Absence of approaches addressing emerging problems of CQA ecosystems. In spite of overall popularity and successfulness, some of the most popular CQA systems have recently experienced negative development of their content and community. The most eminent problems, which significantly hinder the question answering process, are a rapidly increasing amount of low-quality content and a growing number of undesired groups of users who purposefully abuse CQA systems (e.g. in order to quickly solve their problems without returning the received help back to the community). In spite of the great effort in supporting collaboration of users, the existing state-of-the-art approaches do not sufficiently address these negative trends. Moreover, some of the approaches for providing users with collaboration support even indirectly support these undesired groups of users. The main reasons for this discrepancy is that collaboration support is primarily aimed at askers and their goals to receive answers in the shortest possible time. On the other hand, answerers and their expectations are not taken into consideration sufficiently. In addition, existing approaches involve in the question answering process only a small subset of highly active and expert users, while the rest of the community is usually left unutilized.
Undiscovered potential of educational CQA systems. In addition, in spite of many positive results of CQA systems on the open Web, their beneficial effects have not been fully discovered in other environments yet. Nowadays, we witness initial efforts on taking advance of their concepts in business context. Question answering in CQA systems can be perceived, however, not only as a process of knowledge sharing, but also as a specific kind of informal collaborative learning. Therefore, CQA systems incorporate also interesting and undiscovered learning potential that can be utilized especially in educational domain. This potential is obvious particularly at organizational level, as students are quite often struggling with various problems related to a learning process or learning materials that cannot be answered easily in CQA systems outside their educational organization.

We aim to address these open problems by exploring sustainability and transferability of community question answering. In particular, our thesis goals are:

Goal 1: Proposal and evaluation of new methods to preservation of CQA sustainability. In order to suppress the negative consequences of the current development in the most popular CQA systems and to maintain the long-term sustainability of their ecosystems, our first goal is to investigate the emerging problems in more details. Consequently, we aim to propose novel methods that can support the collaboration between users and at the same time contribute to the long-term sustainability of CQA systems. For example, the recommendation of questions to potential answerers represents a possibility to motivate and involve all kinds of answerers (not only active and expert ones) with respect to their interests in particular topics, question difficulty etc.
Goal 2: Adapting successful concepts of CQA systems for specifics of organization-wide and educational environment. Our second goal is to examine how verified CQA systems can be adapted in two transitions: (1) from a non-educational to an educational context; and (2) from the open Web to an organizational environment. Consequently, we introduce the novel organization-wide educational CQA system Askalot that is specifically designed to support the question answering process while taking organizational as well as educational specifics into consideration.

Results

Contributions achieved in the dissertation thesis can be divided into three main groups:

Overview of theories and state of the art in CQA systems. In spite of significant interest in this area in academy as well as in industry, the systematic overview of theories standing behind their success was missing. In our work, we described CQA systems from two perspectives - from the perspective of knowledge sharing and collaborative learning. In each of them we identified the most important theories, which provide an insight how the community-based question answering process works. At the same time, we did an analyses of 265 research publications, which served us as the basis for proposal of descriptive framework, complex three-level categorization hierarchy of approaches as well as for describing representative approaches from each category. This survey should help novice researchers to get better overview of the research domain and to identify optimal techniques in methods' proposal and evaluation.
Supporting long-term sustainability of CQA ecosystems. We identified the increasing negative trend in development of some CQA systems. In order to describe it in more details, we conducted the case study on CQA system Stack Overflow. In the study, we analysed community perception, which we supplemented with easily executable and reproducible quantitative analyses. These analyses allow any other researchers continue monitoring development of the negative trends not only on Stack Overflow but also on all other CQA systems built on the top of Stack Exchange platform. Following the achieved insight in the case study, we proposed several remedy solutions (e.g. new attitude to reputation calculation or systematic involvement of the whole community). These suggestions were illustrated and verified by means of innovative methods for (1) reputation calculation based on content quality and difficulty; and (2) question routing based on non-QA data.
Investigation of CQA transferability to educational domain. Last but not least, we identified a potential of CQA systems to be employed not only on the standard open Web, but also in organizational and educational environment. The proposed concept of university-wide educational CQA system has been evaluated by implementation of CQA system Askalot. Askalot has been deployed as a supplementary tool to formal educational process at our faculty. Its community consists of more than 1100 students and teacher. In addition, Askalot provides also the experimental infrastructure, which has been already used in evaluation of several research papers and bachelor or master theses.

Future work

The achieved results in the dissertation thesis provides good basis for additional research in the area of CQA systems. At first, it is possible to continue in proposal of additional methods for adaptive collaboration support, which will also contribute to sustainability of CQA community ecosystems. We perceive another potential in further development of educational CQA systems. We plan to deploy Askalot at Lugano University as a part of collaboration project in SCOPES programme. Moreover, we have started a collaboration with Harvard University with the aim to adjust implementation of Askalot so it can be used for question answering in MOOC system edX.

The thesis extended abstract is available in the Bulletin of the ACM Slovakia.

Selected publications

I. Srba, M. Bieliková: A Comprehensive Survey and Classification of Approaches for Community Question Answering. In ACM Transactions on the Web, ACM Press, to appear.