Design implications for task-specific search utilities for retrieval and re-engineering of code

Rahat Iqbal, Adam Grzywaczewski, John Halloran, Faiyaz Doctor, Kashif Iqbal

Abstract

The importance of information retrieval systems is unquestionable in the modern society and both individuals as well as enterprises recognise the benefits of being able to find information effectively. Current code-focused information retrieval systems such as Google Code Search, Codeplex or Koders produce results based on specific keywords. However, these systems do not take into account developers’ context such as development language, technology framework, goal of the project, project complexity and developer’s domain expertise. They also impose additional cognitive burden on users in switching between different interfaces and clicking through to find the relevant code. Hence, they are not used by software developers. In this paper, we discuss how software engineers interact with information and general-purpose information retrieval systems (e.g. Google, Yahoo!) and investigate to what extent domain-specific search and recommendation utilities can be developed in order to support their work-related activities. In order to investigate this, we conducted a user study and found that software engineers followed many identifiable and repeatable work tasks and behaviours. These behaviours can be used to develop implicit relevance feedback-based systems based on the observed retention actions. Moreover, we discuss the implications for the development of task-specific search and collaborative recommendation utilities embedded with the Google standard search engine and Microsoft IntelliSense for retrieval and re-engineering of code. Based on implicit relevance feedback, we have implemented a prototype of the proposed collaborative recommendation system, which was evaluated in a controlled environment simulating the real-world situation of professional software engineers. The evaluation has achieved promising initial results on the precision and recall performance of the system. Publisher statement: This is an Accepted Manuscript of an article published by Taylor & Francis in Enterprise Information Systems on 9th October 2015, available online: http://www.tandfonline.com/10.1080/17517575.2015.1086494
Original languageEnglish
Pages (from-to)738-757
Number of pages19
JournalEnterprise Information Systems
Volume11
Issue number5
Early online date9 Oct 2015
DOIs
StatePublished - 2017

Fingerprint

Information retrieval systems
Engineers
Feedback
Industry
Recommender systems
Search engines
Information systems

Keywords

  • Personalised information retrieval
  • opportunistic software development
  • development by example
  • retention actions
  • copy and paste
  • pseudo-relevance
  • software development
  • information-seeking behaviour
  • implicit relevance feedback

Cite this

Design implications for task-specific search utilities for retrieval and re-engineering of code. / Iqbal, Rahat; Grzywaczewski, Adam; Halloran, John; Doctor, Faiyaz; Iqbal, Kashif.

In: Enterprise Information Systems, Vol. 11, No. 5, 2017, p. 738-757.

Research output: Contribution to journalArticle

Iqbal, Rahat; Grzywaczewski, Adam; Halloran, John; Doctor, Faiyaz; Iqbal, Kashif / Design implications for task-specific search utilities for retrieval and re-engineering of code.

In: Enterprise Information Systems, Vol. 11, No. 5, 2017, p. 738-757.

Research output: Contribution to journalArticle

@article{d12a479200324c1789342122ba5900a7,
title = "Design implications for task-specific search utilities for retrieval and re-engineering of code",
abstract = "The importance of information retrieval systems is unquestionable in the modern society and both individuals as well as enterprises recognise the benefits of being able to find information effectively. Current code-focused information retrieval systems such as Google Code Search, Codeplex or Koders produce results based on specific keywords. However, these systems do not take into account developers’ context such as development language, technology framework, goal of the project, project complexity and developer’s domain expertise. They also impose additional cognitive burden on users in switching between different interfaces and clicking through to find the relevant code. Hence, they are not used by software developers. In this paper, we discuss how software engineers interact with information and general-purpose information retrieval systems (e.g. Google, Yahoo!) and investigate to what extent domain-specific search and recommendation utilities can be developed in order to support their work-related activities. In order to investigate this, we conducted a user study and found that software engineers followed many identifiable and repeatable work tasks and behaviours. These behaviours can be used to develop implicit relevance feedback-based systems based on the observed retention actions. Moreover, we discuss the implications for the development of task-specific search and collaborative recommendation utilities embedded with the Google standard search engine and Microsoft IntelliSense for retrieval and re-engineering of code. Based on implicit relevance feedback, we have implemented a prototype of the proposed collaborative recommendation system, which was evaluated in a controlled environment simulating the real-world situation of professional software engineers. The evaluation has achieved promising initial results on the precision and recall performance of the system. Publisher statement: This is an Accepted Manuscript of an article published by Taylor & Francis in Enterprise Information Systems on 9th October 2015, available online: http://www.tandfonline.com/10.1080/17517575.2015.1086494",
keywords = "Personalised information retrieval, opportunistic software development, development by example, retention actions, copy and paste, pseudo-relevance, software development, information-seeking behaviour, implicit relevance feedback",
author = "Rahat Iqbal and Adam Grzywaczewski and John Halloran and Faiyaz Doctor and Kashif Iqbal",
note = "This is an Accepted Manuscript of an article published by Taylor & Francis in Enterprise Information Systems on 9th October 2015, available online: http://www.tandfonline.com/10.1080/17517575.2015.1086494",
year = "2017",
doi = "10.1080/17517575.2015.1086494",
volume = "11",
pages = "738--757",
journal = "Enterprise Information Systems",
issn = "1751-7575",
publisher = "Taylor & Francis",
number = "5",

}

TY - JOUR

T1 - Design implications for task-specific search utilities for retrieval and re-engineering of code

AU - Iqbal,Rahat

AU - Grzywaczewski,Adam

AU - Halloran,John

AU - Doctor,Faiyaz

AU - Iqbal,Kashif

N1 - This is an Accepted Manuscript of an article published by Taylor & Francis in Enterprise Information Systems on 9th October 2015, available online: http://www.tandfonline.com/10.1080/17517575.2015.1086494

PY - 2017

Y1 - 2017

N2 - The importance of information retrieval systems is unquestionable in the modern society and both individuals as well as enterprises recognise the benefits of being able to find information effectively. Current code-focused information retrieval systems such as Google Code Search, Codeplex or Koders produce results based on specific keywords. However, these systems do not take into account developers’ context such as development language, technology framework, goal of the project, project complexity and developer’s domain expertise. They also impose additional cognitive burden on users in switching between different interfaces and clicking through to find the relevant code. Hence, they are not used by software developers. In this paper, we discuss how software engineers interact with information and general-purpose information retrieval systems (e.g. Google, Yahoo!) and investigate to what extent domain-specific search and recommendation utilities can be developed in order to support their work-related activities. In order to investigate this, we conducted a user study and found that software engineers followed many identifiable and repeatable work tasks and behaviours. These behaviours can be used to develop implicit relevance feedback-based systems based on the observed retention actions. Moreover, we discuss the implications for the development of task-specific search and collaborative recommendation utilities embedded with the Google standard search engine and Microsoft IntelliSense for retrieval and re-engineering of code. Based on implicit relevance feedback, we have implemented a prototype of the proposed collaborative recommendation system, which was evaluated in a controlled environment simulating the real-world situation of professional software engineers. The evaluation has achieved promising initial results on the precision and recall performance of the system. Publisher statement: This is an Accepted Manuscript of an article published by Taylor & Francis in Enterprise Information Systems on 9th October 2015, available online: http://www.tandfonline.com/10.1080/17517575.2015.1086494

AB - The importance of information retrieval systems is unquestionable in the modern society and both individuals as well as enterprises recognise the benefits of being able to find information effectively. Current code-focused information retrieval systems such as Google Code Search, Codeplex or Koders produce results based on specific keywords. However, these systems do not take into account developers’ context such as development language, technology framework, goal of the project, project complexity and developer’s domain expertise. They also impose additional cognitive burden on users in switching between different interfaces and clicking through to find the relevant code. Hence, they are not used by software developers. In this paper, we discuss how software engineers interact with information and general-purpose information retrieval systems (e.g. Google, Yahoo!) and investigate to what extent domain-specific search and recommendation utilities can be developed in order to support their work-related activities. In order to investigate this, we conducted a user study and found that software engineers followed many identifiable and repeatable work tasks and behaviours. These behaviours can be used to develop implicit relevance feedback-based systems based on the observed retention actions. Moreover, we discuss the implications for the development of task-specific search and collaborative recommendation utilities embedded with the Google standard search engine and Microsoft IntelliSense for retrieval and re-engineering of code. Based on implicit relevance feedback, we have implemented a prototype of the proposed collaborative recommendation system, which was evaluated in a controlled environment simulating the real-world situation of professional software engineers. The evaluation has achieved promising initial results on the precision and recall performance of the system. Publisher statement: This is an Accepted Manuscript of an article published by Taylor & Francis in Enterprise Information Systems on 9th October 2015, available online: http://www.tandfonline.com/10.1080/17517575.2015.1086494

KW - Personalised information retrieval

KW - opportunistic software development

KW - development by example

KW - retention actions

KW - copy and paste

KW - pseudo-relevance

KW - software development

KW - information-seeking behaviour

KW - implicit relevance feedback

U2 - 10.1080/17517575.2015.1086494

DO - 10.1080/17517575.2015.1086494

M3 - Article

VL - 11

SP - 738

EP - 757

JO - Enterprise Information Systems

T2 - Enterprise Information Systems

JF - Enterprise Information Systems

SN - 1751-7575

SN - 1751-7583

IS - 5

ER -