Interdisciplinarity

Hoopla Aside, Turing’s Test Remains Unbeaten Interdisciplinarity
A depiction of Alan Turing at Bletchley Park, the World War II home of Britain's codebreakers.

Hoopla Aside, Turing’s Test Remains Unbeaten

June 9, 2014 1320

Alan Turing statue

A depiction of Alan Turing at Bletchley Park, the World War II home of Britain’s codebreakers.

In an event that is sure to capture the world’s imagination and that has been proclaimed as setting a new milestone in the study of artificial intelligence, a computer program has successfully passed the “Turing Test.” To do this, the computer engaged in a conversation with judges and convinced at least some of them, that they were in fact talking to a human. This success has come on the 60th anniversary of Alan Turing’s death and is expected to further fuel the controversy around the fundamental question of whether what has been demonstrated is actually a machine thinking like a human.

What is the Turing Test?

Alan Turing, the famous mathematician and computer programmer who worked on cracking the Enigma code during the Second World War proposed his test in a 1950 paper. In the paper he suggested a practical way in which you could test whether a machine could think. His original proposal involved taking three people, a man, a woman and an interrogator and had the interrogator ask unrestricted questions to one of the subjects chosen at random and try and determine whether they were talking to either the man or the woman. The interrogator was separated from the test subjects and conversed with them via text through a computer. The subject was then replaced with a computer and the interrogator went through the same process. The machine would win if the interrogator guessed the answer wrongly as many times as they had done when a human was answering.

The Conversation logo_AU

This article by David Glance originally appeared at The Conversation, a Social Science Space partner site, under the title “A computer beating humans and passing Turing’s test wasn’t a fair fight”

Since Turing’s original proposal, the Turing test has taken various forms but the form has settled on testing whether a human judge could successfully pick whether the entity answering questions was human or machine.

The current test organised by the University of Reading and run at the Royal Society in the UK, tested whether the judges would confuse the computer for a human at least 30% of the time. Thirty judges “conversed” with 5 “chatbots” and 25 humans and tried to tell them apart. The winning computer program posed as a 13 year old Ukrainian boy called Eugene Goostman. It was developed by Russian-born Vladimir Veselov, who lives in the United States, and Ukrainian Eugene Demchenko who lives in Russia.

Was it a fair fight?

Both Veselov and Demchenko have a history of being involved with the Turing test including the use of Goostman in previous attempts. They have publicly stated that their approach to passing the test was to think about the test itself and the way different judges tried to get computers to reveal their true selves. They noticed that judges for example will take several approaches to this and use questions that test a computer’s ability to deal with humor, logic, subtlety in language, word association, and common sense.

Their construction of the program Eugene Goostman took advantage of this knowledge. In the past, judges had tended to forgive nonsensical answers from chatterbots that were young. They also overlooked language issues if the chatterbot was foreign. It was no surprise then that Eugene Goostman was designed as a 13 year old Ukrainian.

Veselov and Demchenko were the first to admit that this was their intent when they themselves declared “Turing’s Test is, actually, no more than a joke of that genius British mathematician. This game has nothing (or very little) in common with the question, “Can machines think?”

The bet

Two people who also won’t benefit from the outcome of this competition are Mitch Kapor, founder of the computer company Lotus and Ray Kurtzweil, US author and scientist. They have a long standing bet of $20,000 as to whether the Turing test would be beaten by 2029. Kapor has said it won’t be and Kurzweil has claimed it will. The rules for their bet are different and more demanding than the test run by the Royal Society. For a computer to win in their version of the Turing Test, it will have to convince two of the three human judges that it is human and also to have outdone the human test subjects in an overall ordering of “humanness”.

Not all trickery

Although it is clear that Eugene Goostman was created with the sole purpose of passing a test, like any child at school of university today who approaches education in the same way, it doesn’t mean that it wasn’t displaying elements of true human thinking. In the 64 years since Turing initially posed the challenge, computer scientists have made enormous advances in the abilities of machines to do intelligent things that in many respects surpass what humans are capable of doing. Science fiction writer Charles Platt has suggested that by 2030 we may actually be running the “Gnirut Test” in which intelligent machines will be trying to determine ultimately whether “the human brain is capable of achieving machine intelligence”.The Conversation


David Glance is the director of innovation in the Faculty of Arts and Director of the University of Western Australia Centre for Software Practice, a UWA research and development center. Originally a physiologist working in the area of vascular control mechanisms in pregnancy, he subsequently worked in the software industry for over 20 years before spending the last 10 years at UWA; his research interests are in health informatics, public health and software engineering.

View all posts by David Glance

Related Articles

Civilisation – and Some Discontents
Public Engagement
April 30, 2024

Civilisation – and Some Discontents

Read Now
The Long Arm of Criminality
Opinion
April 29, 2024

The Long Arm of Criminality

Read Now
Exploring ‘Lost Person Behavior’ and the Science of Search and Rescue
Featured
April 24, 2024

Exploring ‘Lost Person Behavior’ and the Science of Search and Rescue

Read Now
Why Don’t Algorithms Agree With Each Other?
Innovation
February 21, 2024

Why Don’t Algorithms Agree With Each Other?

Read Now
A Black History Addendum to the American Music Industry

A Black History Addendum to the American Music Industry

The new editor of the case study series on the music industry discusses the history of Black Americans in the recording industry.

Read Now
When University Decolonization in Canada Mends Relationships with Indigenous Nations and Lands

When University Decolonization in Canada Mends Relationships with Indigenous Nations and Lands

Community-based work and building and maintaining relationships with nations whose land we live upon is at the heart of what Indigenizing is. It is not simply hiring more faculty, or putting the titles “decolonizing” and “Indigenizing” on anything that might connect to Indigenous peoples.

Read Now
Philip Rubin: FABBS’ Accidental Essential Man Linking Research and Policy

Philip Rubin: FABBS’ Accidental Essential Man Linking Research and Policy

As he stands down from a two-year stint as the president of the Federation of Associations in Behavioral & Brain Sciences, or FABBS, Social Science Space took the opportunity to download a fraction of the experiences of cognitive psychologist Philip Rubin, especially his experiences connecting science and policy.

Read Now
0 0 votes
Article Rating
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments