The importance of a human viewpoint on computer natural language capabilities: a Turing test perspective

Research output: Contribution to journalArticle

8 Citations (Scopus)
39 Downloads (Pure)

Abstract

When judging the capabilities of technology, different humans can have very different perspectives and come to quite diverse conclusions over the same data set. In this paper we consider the capabilities of humans when it comes to judging conversational abilities, as to whether they are conversing with a human or a machine. In particular the issue in question is the importance of human judges interrogating in practical Turing tests. As supportive evidence for this we make use of transcripts which originated from a series of practical Turing’s tests held 6–7 June 2014 at the Royal Society London. Each of the tests involved a 3-participant simultaneous comparison by a judge of two hidden entities, one being a human and the other a machine. Thirty different judges took part in total. Each of the transcripts considered in the paper resulted in a judge being unable to say for certain which was the machine and which was the human. The main point we consider here is the fallibility of humans in deciding whether they are conversing with a machine or a human; hence we are concerned specifically with the decision-making process.
Original languageEnglish
Pages (from-to)207-221
JournalAI and Society
Volume31
Issue number2
Early online date16 Jun 2015
DOIs
Publication statusPublished - Jun 2016

Fingerprint

Decision making
Turing Test
Natural Language

Keywords

  • Artificial Intelligence (incl. Robotics)
  • Computer Science
  • general
  • Engineering Economics
  • Organization
  • Logistics
  • Marketing
  • Control
  • Robotics
  • Mechatronics
  • Performing Arts
  • Methodology of the Social Sciences

Cite this

The importance of a human viewpoint on computer natural language capabilities: a Turing test perspective. / Warwick, Kevin; Shah, Huma.

In: AI and Society, Vol. 31, No. 2, 06.2016, p. 207-221.

Research output: Contribution to journalArticle

@article{c2b79b00c4db4f91bec50f02345320d7,
title = "The importance of a human viewpoint on computer natural language capabilities: a Turing test perspective",
abstract = "When judging the capabilities of technology, different humans can have very different perspectives and come to quite diverse conclusions over the same data set. In this paper we consider the capabilities of humans when it comes to judging conversational abilities, as to whether they are conversing with a human or a machine. In particular the issue in question is the importance of human judges interrogating in practical Turing tests. As supportive evidence for this we make use of transcripts which originated from a series of practical Turing’s tests held 6–7 June 2014 at the Royal Society London. Each of the tests involved a 3-participant simultaneous comparison by a judge of two hidden entities, one being a human and the other a machine. Thirty different judges took part in total. Each of the transcripts considered in the paper resulted in a judge being unable to say for certain which was the machine and which was the human. The main point we consider here is the fallibility of humans in deciding whether they are conversing with a machine or a human; hence we are concerned specifically with the decision-making process.",
keywords = "Artificial Intelligence (incl. Robotics), Computer Science, general, Engineering Economics, Organization, Logistics, Marketing, Control, Robotics, Mechatronics, Performing Arts, Methodology of the Social Sciences",
author = "Kevin Warwick and Huma Shah",
year = "2016",
month = "6",
doi = "10.1007/s00146-015-0588-5",
language = "English",
volume = "31",
pages = "207--221",
journal = "AI & Society",
issn = "0951-5666",
publisher = "Springer Verlag",
number = "2",

}

TY - JOUR

T1 - The importance of a human viewpoint on computer natural language capabilities: a Turing test perspective

AU - Warwick, Kevin

AU - Shah, Huma

PY - 2016/6

Y1 - 2016/6

N2 - When judging the capabilities of technology, different humans can have very different perspectives and come to quite diverse conclusions over the same data set. In this paper we consider the capabilities of humans when it comes to judging conversational abilities, as to whether they are conversing with a human or a machine. In particular the issue in question is the importance of human judges interrogating in practical Turing tests. As supportive evidence for this we make use of transcripts which originated from a series of practical Turing’s tests held 6–7 June 2014 at the Royal Society London. Each of the tests involved a 3-participant simultaneous comparison by a judge of two hidden entities, one being a human and the other a machine. Thirty different judges took part in total. Each of the transcripts considered in the paper resulted in a judge being unable to say for certain which was the machine and which was the human. The main point we consider here is the fallibility of humans in deciding whether they are conversing with a machine or a human; hence we are concerned specifically with the decision-making process.

AB - When judging the capabilities of technology, different humans can have very different perspectives and come to quite diverse conclusions over the same data set. In this paper we consider the capabilities of humans when it comes to judging conversational abilities, as to whether they are conversing with a human or a machine. In particular the issue in question is the importance of human judges interrogating in practical Turing tests. As supportive evidence for this we make use of transcripts which originated from a series of practical Turing’s tests held 6–7 June 2014 at the Royal Society London. Each of the tests involved a 3-participant simultaneous comparison by a judge of two hidden entities, one being a human and the other a machine. Thirty different judges took part in total. Each of the transcripts considered in the paper resulted in a judge being unable to say for certain which was the machine and which was the human. The main point we consider here is the fallibility of humans in deciding whether they are conversing with a machine or a human; hence we are concerned specifically with the decision-making process.

KW - Artificial Intelligence (incl. Robotics)

KW - Computer Science

KW - general

KW - Engineering Economics

KW - Organization

KW - Logistics

KW - Marketing

KW - Control

KW - Robotics

KW - Mechatronics

KW - Performing Arts

KW - Methodology of the Social Sciences

U2 - 10.1007/s00146-015-0588-5

DO - 10.1007/s00146-015-0588-5

M3 - Article

VL - 31

SP - 207

EP - 221

JO - AI & Society

JF - AI & Society

SN - 0951-5666

IS - 2

ER -