Can Machines Talk? Comparison of Eliza with Modern Dialogue Systems

Huma Shah, Kevin Warwick, Jordi Vallverdú, Defeng Wu

Research output: Contribution to journalArticle

29 Citations (Scopus)
190 Downloads (Pure)

Abstract

To find if current dialogue systems use the same, psychotherapist questioning technique as Joseph Weizenbaum's 1960 natural language understanding programme, Eliza, the authors carried out an original experiment comparing five successful artificial dialogue systems, Cleverbot, Elbot, Eugene Goostman, JFred and Ultra Hal with an online version of Eliza. More than one hundred male and female participants with 1st or non-1st English language, age range 13–64, interacted with the systems over the Internet scoring each for conversation ability. Developers of the modern conversation systems show they deploy a variety of techniques to initiate and maintain dialogue learning from interactions with humans over the Internet. Statistical significance shows these dialogue systems are an improvement on their predecessor. Embedded on the web affording round-the-clock interaction the nature of artificial dialogue systems is evolving as these systems learn from the way humans converse. The uses of modern Elizas are proven successful as virtual assistants in e-commerce; their conversational basis is already extending into education. What we can say is modern artificial dialogue systems do talk. They are able to participate in conversation in a way their predecessor Eliza could not: they are able to share personal opinions, relay experience of family dramas, be relevant, but also be vague, and mislead just as humans do.
Original languageEnglish
Pages (from-to)278–295
JournalComputers in Human Behavior
Volume58
Early online date19 Jan 2016
DOIs
Publication statusPublished - May 2016

Fingerprint

Internet
Language
Drama
Clocks
Aptitude
Education
Psychotherapy
Learning
Experiments
Dialogue Systems
World Wide Web
Artificial
Interaction

Bibliographical note

NOTICE: this is the author’s version of a work that was accepted for publication in Computers in Human Behavior. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Computers in Human Behavior, [58, (2016)]
DOI: 10.1016/j.chb.2016.01.004

© 2015, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/

Keywords

  • conversation ability
  • dialogue systems
  • Eliza
  • machine
  • talk
  • Turing test
  • Artificial Intelligence
  • Machine Intelligence
  • Human Computer Interaction
  • Chatbots

Cite this

Can Machines Talk? Comparison of Eliza with Modern Dialogue Systems. / Shah, Huma; Warwick, Kevin; Vallverdú, Jordi; Wu, Defeng.

In: Computers in Human Behavior, Vol. 58, 05.2016, p. 278–295.

Research output: Contribution to journalArticle

@article{0fd0808cd33242e5bf6adcd097911ee4,
title = "Can Machines Talk?: Comparison of Eliza with Modern Dialogue Systems",
abstract = "To find if current dialogue systems use the same, psychotherapist questioning technique as Joseph Weizenbaum's 1960 natural language understanding programme, Eliza, the authors carried out an original experiment comparing five successful artificial dialogue systems, Cleverbot, Elbot, Eugene Goostman, JFred and Ultra Hal with an online version of Eliza. More than one hundred male and female participants with 1st or non-1st English language, age range 13–64, interacted with the systems over the Internet scoring each for conversation ability. Developers of the modern conversation systems show they deploy a variety of techniques to initiate and maintain dialogue learning from interactions with humans over the Internet. Statistical significance shows these dialogue systems are an improvement on their predecessor. Embedded on the web affording round-the-clock interaction the nature of artificial dialogue systems is evolving as these systems learn from the way humans converse. The uses of modern Elizas are proven successful as virtual assistants in e-commerce; their conversational basis is already extending into education. What we can say is modern artificial dialogue systems do talk. They are able to participate in conversation in a way their predecessor Eliza could not: they are able to share personal opinions, relay experience of family dramas, be relevant, but also be vague, and mislead just as humans do.",
keywords = "conversation ability, dialogue systems, Eliza, machine, talk, Turing test, Artificial Intelligence, Machine Intelligence, Human Computer Interaction, Chatbots",
author = "Huma Shah and Kevin Warwick and Jordi Vallverd{\'u} and Defeng Wu",
note = "NOTICE: this is the author’s version of a work that was accepted for publication in Computers in Human Behavior. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Computers in Human Behavior, [58, (2016)] DOI: 10.1016/j.chb.2016.01.004 {\circledC} 2015, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/",
year = "2016",
month = "5",
doi = "10.1016/j.chb.2016.01.004",
language = "English",
volume = "58",
pages = "278–295",
journal = "Computers in Human Behavior",
issn = "0747-5632",
publisher = "Elsevier",

}

TY - JOUR

T1 - Can Machines Talk?

T2 - Comparison of Eliza with Modern Dialogue Systems

AU - Shah, Huma

AU - Warwick, Kevin

AU - Vallverdú, Jordi

AU - Wu, Defeng

N1 - NOTICE: this is the author’s version of a work that was accepted for publication in Computers in Human Behavior. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Computers in Human Behavior, [58, (2016)] DOI: 10.1016/j.chb.2016.01.004 © 2015, Elsevier. Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International http://creativecommons.org/licenses/by-nc-nd/4.0/

PY - 2016/5

Y1 - 2016/5

N2 - To find if current dialogue systems use the same, psychotherapist questioning technique as Joseph Weizenbaum's 1960 natural language understanding programme, Eliza, the authors carried out an original experiment comparing five successful artificial dialogue systems, Cleverbot, Elbot, Eugene Goostman, JFred and Ultra Hal with an online version of Eliza. More than one hundred male and female participants with 1st or non-1st English language, age range 13–64, interacted with the systems over the Internet scoring each for conversation ability. Developers of the modern conversation systems show they deploy a variety of techniques to initiate and maintain dialogue learning from interactions with humans over the Internet. Statistical significance shows these dialogue systems are an improvement on their predecessor. Embedded on the web affording round-the-clock interaction the nature of artificial dialogue systems is evolving as these systems learn from the way humans converse. The uses of modern Elizas are proven successful as virtual assistants in e-commerce; their conversational basis is already extending into education. What we can say is modern artificial dialogue systems do talk. They are able to participate in conversation in a way their predecessor Eliza could not: they are able to share personal opinions, relay experience of family dramas, be relevant, but also be vague, and mislead just as humans do.

AB - To find if current dialogue systems use the same, psychotherapist questioning technique as Joseph Weizenbaum's 1960 natural language understanding programme, Eliza, the authors carried out an original experiment comparing five successful artificial dialogue systems, Cleverbot, Elbot, Eugene Goostman, JFred and Ultra Hal with an online version of Eliza. More than one hundred male and female participants with 1st or non-1st English language, age range 13–64, interacted with the systems over the Internet scoring each for conversation ability. Developers of the modern conversation systems show they deploy a variety of techniques to initiate and maintain dialogue learning from interactions with humans over the Internet. Statistical significance shows these dialogue systems are an improvement on their predecessor. Embedded on the web affording round-the-clock interaction the nature of artificial dialogue systems is evolving as these systems learn from the way humans converse. The uses of modern Elizas are proven successful as virtual assistants in e-commerce; their conversational basis is already extending into education. What we can say is modern artificial dialogue systems do talk. They are able to participate in conversation in a way their predecessor Eliza could not: they are able to share personal opinions, relay experience of family dramas, be relevant, but also be vague, and mislead just as humans do.

KW - conversation ability

KW - dialogue systems

KW - Eliza

KW - machine

KW - talk

KW - Turing test

KW - Artificial Intelligence

KW - Machine Intelligence

KW - Human Computer Interaction

KW - Chatbots

U2 - 10.1016/j.chb.2016.01.004

DO - 10.1016/j.chb.2016.01.004

M3 - Article

VL - 58

SP - 278

EP - 295

JO - Computers in Human Behavior

JF - Computers in Human Behavior

SN - 0747-5632

ER -