TED@IBM
Raphael Arar: How we can teach computers to make sense of our emotions
Filmed:
Readability: 4.3
1,255,602 views
How can we make AI that people actually want to interact with? Raphael Arar suggests we start by making art. He shares interactive projects that help AI explore complex ideas like nostalgia, intuition and conversation -- all working towards the goal of making our future technology just as much human as it is artificial.
Raphael Arar - Designer, researcher
IBM's Raphael Arar creates art and designs experiences that examine the complexities of our increasingly technocentric lives. Full bio
IBM's Raphael Arar creates art and designs experiences that examine the complexities of our increasingly technocentric lives. Full bio
Double-click the English transcript below to play the video.
00:13
I consider myself one part artist
and one part designer.
and one part designer.
0
1760
4680
00:18
And I work at an artificial
intelligence research lab.
intelligence research lab.
1
6480
3160
00:22
We're trying to create technology
2
10720
1696
00:24
that you'll want to interact with
in the far future.
in the far future.
3
12440
3296
00:27
Not just six months from now,
but try years and decades from now.
but try years and decades from now.
4
15760
4640
00:33
And we're taking a moonshot
5
21120
1616
00:34
that we'll want to be
interacting with computers
interacting with computers
6
22760
2456
00:37
in deeply emotional ways.
7
25240
2120
00:40
So in order to do that,
8
28280
1456
00:41
the technology has to be
just as much human as it is artificial.
just as much human as it is artificial.
9
29760
4480
00:46
It has to get you.
10
34920
2256
00:49
You know, like that inside joke
that'll have you and your best friend
that'll have you and your best friend
11
37200
3336
00:52
on the floor, cracking up.
12
40560
1936
00:54
Or that look of disappointment
that you can just smell from miles away.
that you can just smell from miles away.
13
42520
4560
01:00
I view art as the gateway to help us
bridge this gap between human and machine:
bridge this gap between human and machine:
14
48560
6040
01:07
to figure out what it means
to get each other
to get each other
15
55280
3136
01:10
so that we can train AI to get us.
16
58440
2760
01:13
See, to me, art is a way
to put tangible experiences
to put tangible experiences
17
61920
3816
01:17
to intangible ideas,
feelings and emotions.
feelings and emotions.
18
65760
3240
01:21
And I think it's one
of the most human things about us.
of the most human things about us.
19
69800
2600
01:25
See, we're a complicated
and complex bunch.
and complex bunch.
20
73480
2936
01:28
We have what feels like
an infinite range of emotions,
an infinite range of emotions,
21
76440
3136
01:31
and to top it off, we're all different.
22
79600
2496
01:34
We have different family backgrounds,
23
82120
2296
01:36
different experiences
and different psychologies.
and different psychologies.
24
84440
3080
01:40
And this is what makes life
really interesting.
really interesting.
25
88240
2720
01:43
But this is also what makes
working on intelligent technology
working on intelligent technology
26
91440
3496
01:46
extremely difficult.
27
94960
1600
01:49
And right now, AI research, well,
28
97640
3456
01:53
it's a bit lopsided on the tech side.
29
101120
2016
01:55
And that makes a lot of sense.
30
103160
2136
01:57
See, for every
qualitative thing about us --
qualitative thing about us --
31
105320
2456
01:59
you know, those parts of us that are
emotional, dynamic and subjective --
emotional, dynamic and subjective --
32
107800
4456
02:04
we have to convert it
to a quantitative metric:
to a quantitative metric:
33
112280
3136
02:07
something that can be represented
with facts, figures and computer code.
with facts, figures and computer code.
34
115440
4360
02:13
The issue is, there are
many qualitative things
many qualitative things
35
121000
3376
02:16
that we just can't put our finger on.
36
124400
1960
02:20
So, think about hearing
your favorite song for the first time.
your favorite song for the first time.
37
128400
3200
02:25
What were you doing?
38
133200
1200
02:28
How did you feel?
39
136000
1200
02:30
Did you get goosebumps?
40
138720
1360
02:33
Or did you get fired up?
41
141240
1640
02:36
Hard to describe, right?
42
144400
1200
02:38
See, parts of us feel so simple,
43
146800
2096
02:40
but under the surface,
there's really a ton of complexity.
there's really a ton of complexity.
44
148920
3656
02:44
And translating
that complexity to machines
that complexity to machines
45
152600
2936
02:47
is what makes them modern-day moonshots.
46
155560
2856
02:50
And I'm not convinced that we can
answer these deeper questions
answer these deeper questions
47
158440
4176
02:54
with just ones and zeros alone.
48
162640
1480
02:57
So, in the lab, I've been creating art
49
165120
1936
02:59
as a way to help me
design better experiences
design better experiences
50
167080
2456
03:01
for bleeding-edge technology.
51
169560
2096
03:03
And it's been serving as a catalyst
52
171680
1736
03:05
to beef up the more human ways
that computers can relate to us.
that computers can relate to us.
53
173440
3840
03:10
Through art, we're tacking
some of the hardest questions,
some of the hardest questions,
54
178000
2696
03:12
like what does it really mean to feel?
55
180720
2360
03:16
Or how do we engage and know
how to be present with each other?
how to be present with each other?
56
184120
4080
03:20
And how does intuition
affect the way that we interact?
affect the way that we interact?
57
188800
4000
03:26
So, take for example human emotion.
58
194440
2056
03:28
Right now, computers can make sense
of our most basic ones,
of our most basic ones,
59
196520
3256
03:31
like joy, sadness,
anger, fear and disgust,
anger, fear and disgust,
60
199800
3696
03:35
by converting those
characteristics to math.
characteristics to math.
61
203520
3000
03:39
But what about the more complex emotions?
62
207400
2536
03:41
You know, those emotions
63
209960
1216
03:43
that we have a hard time
describing to each other?
describing to each other?
64
211200
2376
03:45
Like nostalgia.
65
213600
1440
03:47
So, to explore this, I created
a piece of art, an experience,
a piece of art, an experience,
66
215640
3936
03:51
that asked people to share a memory,
67
219600
2096
03:53
and I teamed up with some data scientists
68
221720
2136
03:55
to figure out how to take
an emotion that's so highly subjective
an emotion that's so highly subjective
69
223880
3576
03:59
and convert it into something
mathematically precise.
mathematically precise.
70
227480
3200
04:03
So, we created what we call
a nostalgia score
a nostalgia score
71
231840
2136
04:06
and it's the heart of this installation.
72
234000
2216
04:08
To do that, the installation
asks you to share a story,
asks you to share a story,
73
236240
3056
04:11
the computer then analyzes it
for its simpler emotions,
for its simpler emotions,
74
239320
3256
04:14
it checks for your tendency
to use past-tense wording
to use past-tense wording
75
242600
2656
04:17
and also looks for words
that we tend to associate with nostalgia,
that we tend to associate with nostalgia,
76
245280
3336
04:20
like "home," "childhood" and "the past."
77
248640
3040
04:24
It then creates a nostalgia score
78
252760
2056
04:26
to indicate how nostalgic your story is.
79
254840
2736
04:29
And that score is the driving force
behind these light-based sculptures
behind these light-based sculptures
80
257600
4136
04:33
that serve as physical embodiments
of your contribution.
of your contribution.
81
261760
3896
04:37
And the higher the score,
the rosier the hue.
the rosier the hue.
82
265680
3216
04:40
You know, like looking at the world
through rose-colored glasses.
through rose-colored glasses.
83
268920
3936
04:44
So, when you see your score
84
272880
2616
04:47
and the physical representation of it,
85
275520
2656
04:50
sometimes you'd agree
and sometimes you wouldn't.
and sometimes you wouldn't.
86
278200
2936
04:53
It's as if it really understood
how that experience made you feel.
how that experience made you feel.
87
281160
3480
04:57
But other times it gets tripped up
88
285400
2216
04:59
and has you thinking
it doesn't understand you at all.
it doesn't understand you at all.
89
287640
2560
05:02
But the piece really serves to show
90
290680
1896
05:04
that if we have a hard time explaining
the emotions that we have to each other,
the emotions that we have to each other,
91
292600
4056
05:08
how can we teach a computer
to make sense of them?
to make sense of them?
92
296680
2360
05:12
So, even the more objective parts
about being human are hard to describe.
about being human are hard to describe.
93
300360
3576
05:15
Like, conversation.
94
303960
1240
05:17
Have you ever really tried
to break down the steps?
to break down the steps?
95
305880
2736
05:20
So think about sitting
with your friend at a coffee shop
with your friend at a coffee shop
96
308640
2656
05:23
and just having small talk.
97
311320
1320
05:25
How do you know when to take a turn?
98
313160
1720
05:27
How do you know when to shift topics?
99
315440
1840
05:29
And how do you even know
what topics to discuss?
what topics to discuss?
100
317960
2720
05:33
See, most of us
don't really think about it,
don't really think about it,
101
321560
2096
05:35
because it's almost second nature.
102
323680
1656
05:37
And when we get to know someone,
we learn more about what makes them tick,
we learn more about what makes them tick,
103
325360
3496
05:40
and then we learn
what topics we can discuss.
what topics we can discuss.
104
328880
2376
05:43
But when it comes to teaching
AI systems how to interact with people,
AI systems how to interact with people,
105
331280
3656
05:46
we have to teach them
step by step what to do.
step by step what to do.
106
334960
2856
05:49
And right now, it feels clunky.
107
337840
3496
05:53
If you've ever tried to talk
with Alexa, Siri or Google Assistant,
with Alexa, Siri or Google Assistant,
108
341360
4136
05:57
you can tell that it or they
can still sound cold.
can still sound cold.
109
345520
4200
06:02
And have you ever gotten annoyed
110
350440
1656
06:04
when they didn't understand
what you were saying
what you were saying
111
352120
2256
06:06
and you had to rephrase what you wanted
20 times just to play a song?
20 times just to play a song?
112
354400
3840
06:11
Alright, to the credit of the designers,
realistic communication is really hard.
realistic communication is really hard.
113
359440
4896
06:16
And there's a whole branch of sociology,
114
364360
2136
06:18
called conversation analysis,
115
366520
1936
06:20
that tries to make blueprints
for different types of conversation.
for different types of conversation.
116
368480
3136
06:23
Types like customer service
or counseling, teaching and others.
or counseling, teaching and others.
117
371640
4080
06:28
I've been collaborating
with a conversation analyst at the lab
with a conversation analyst at the lab
118
376880
2936
06:31
to try to help our AI systems
hold more human-sounding conversations.
hold more human-sounding conversations.
119
379840
4696
06:36
This way, when you have an interaction
with a chatbot on your phone
with a chatbot on your phone
120
384560
3176
06:39
or a voice-based system in the car,
121
387760
1856
06:41
it sounds a little more human
and less cold and disjointed.
and less cold and disjointed.
122
389640
3320
06:46
So I created a piece of art
123
394360
1336
06:47
that tries to highlight
the robotic, clunky interaction
the robotic, clunky interaction
124
395720
2816
06:50
to help us understand, as designers,
125
398560
1976
06:52
why it doesn't sound human yet
and, well, what we can do about it.
and, well, what we can do about it.
126
400560
4576
06:57
The piece is called Bot to Bot
127
405160
1456
06:58
and it puts one conversational
system against another
system against another
128
406640
2936
07:01
and then exposes it to the general public.
129
409600
2536
07:04
And what ends up happening
is that you get something
is that you get something
130
412160
2496
07:06
that tries to mimic human conversation,
131
414680
1896
07:08
but falls short.
132
416600
1896
07:10
Sometimes it works and sometimes
it gets into these, well,
it gets into these, well,
133
418520
2736
07:13
loops of misunderstanding.
134
421280
1536
07:14
So even though the machine-to-machine
conversation can make sense,
conversation can make sense,
135
422840
3096
07:17
grammatically and colloquially,
136
425960
2016
07:20
it can still end up
feeling cold and robotic.
feeling cold and robotic.
137
428000
3216
07:23
And despite checking all the boxes,
the dialogue lacks soul
the dialogue lacks soul
138
431240
4016
07:27
and those one-off quirks
that make each of us who we are.
that make each of us who we are.
139
435280
3136
07:30
So while it might be grammatically correct
140
438440
2056
07:32
and uses all the right
hashtags and emojis,
hashtags and emojis,
141
440520
2656
07:35
it can end up sounding mechanical
and, well, a little creepy.
and, well, a little creepy.
142
443200
4136
07:39
And we call this the uncanny valley.
143
447360
2336
07:41
You know, that creepiness factor of tech
144
449720
1936
07:43
where it's close to human
but just slightly off.
but just slightly off.
145
451680
2856
07:46
And the piece will start being
146
454560
1456
07:48
one way that we test
for the humanness of a conversation
for the humanness of a conversation
147
456040
3216
07:51
and the parts that get
lost in translation.
lost in translation.
148
459280
2160
07:54
So there are other things
that get lost in translation, too,
that get lost in translation, too,
149
462560
2856
07:57
like human intuition.
150
465440
1616
07:59
Right now, computers
are gaining more autonomy.
are gaining more autonomy.
151
467080
2776
08:01
They can take care of things for us,
152
469880
1736
08:03
like change the temperature
of our houses based on our preferences
of our houses based on our preferences
153
471640
3176
08:06
and even help us drive on the freeway.
154
474840
2160
08:09
But there are things
that you and I do in person
that you and I do in person
155
477560
2496
08:12
that are really difficult
to translate to AI.
to translate to AI.
156
480080
2760
08:15
So think about the last time
that you saw an old classmate or coworker.
that you saw an old classmate or coworker.
157
483440
4360
08:21
Did you give them a hug
or go in for a handshake?
or go in for a handshake?
158
489080
2480
08:24
You probably didn't think twice
159
492800
1496
08:26
because you've had so many
built up experiences
built up experiences
160
494320
2336
08:28
that had you do one or the other.
161
496680
2000
08:31
And as an artist, I feel
that access to one's intuition,
that access to one's intuition,
162
499440
3456
08:34
your unconscious knowing,
163
502920
1416
08:36
is what helps us create amazing things.
164
504360
3056
08:39
Big ideas, from that abstract,
nonlinear place in our consciousness
nonlinear place in our consciousness
165
507440
4056
08:43
that is the culmination
of all of our experiences.
of all of our experiences.
166
511520
2960
08:47
And if we want computers to relate to us
and help amplify our creative abilities,
and help amplify our creative abilities,
167
515840
4656
08:52
I feel that we'll need to start thinking
about how to make computers be intuitive.
about how to make computers be intuitive.
168
520520
3896
08:56
So I wanted to explore
how something like human intuition
how something like human intuition
169
524440
3096
08:59
could be directly translated
to artificial intelligence.
to artificial intelligence.
170
527560
3456
09:03
And I created a piece
that explores computer-based intuition
that explores computer-based intuition
171
531040
3216
09:06
in a physical space.
172
534280
1320
09:08
The piece is called Wayfinding,
173
536480
1696
09:10
and it's set up as a symbolic compass
that has four kinetic sculptures.
that has four kinetic sculptures.
174
538200
3936
09:14
Each one represents a direction,
175
542160
2056
09:16
north, east, south and west.
176
544240
2120
09:19
And there are sensors set up
on the top of each sculpture
on the top of each sculpture
177
547080
2696
09:21
that capture how far away
you are from them.
you are from them.
178
549800
2256
09:24
And the data that gets collected
179
552080
1816
09:25
ends up changing the way
that sculptures move
that sculptures move
180
553920
2136
09:28
and the direction of the compass.
181
556080
2040
09:31
The thing is, the piece doesn't work
like the automatic door sensor
like the automatic door sensor
182
559360
3656
09:35
that just opens
when you walk in front of it.
when you walk in front of it.
183
563040
2656
09:37
See, your contribution is only a part
of its collection of lived experiences.
of its collection of lived experiences.
184
565720
5056
09:42
And all of those experiences
affect the way that it moves.
affect the way that it moves.
185
570800
4056
09:46
So when you walk in front of it,
186
574880
1736
09:48
it starts to use all of the data
187
576640
1976
09:50
that it's captured
throughout its exhibition history --
throughout its exhibition history --
188
578640
2616
09:53
or its intuition --
189
581280
1816
09:55
to mechanically respond to you
based on what it's learned from others.
based on what it's learned from others.
190
583120
3560
09:59
And what ends up happening
is that as participants
is that as participants
191
587480
2536
10:02
we start to learn the level
of detail that we need
of detail that we need
192
590040
2816
10:04
in order to manage expectations
193
592880
2016
10:06
from both humans and machines.
194
594920
2776
10:09
We can almost see our intuition
being played out on the computer,
being played out on the computer,
195
597720
3616
10:13
picturing all of that data
being processed in our mind's eye.
being processed in our mind's eye.
196
601360
3600
10:17
My hope is that this type of art
197
605560
1656
10:19
will help us think differently
about intuition
about intuition
198
607240
2416
10:21
and how to apply that to AI in the future.
199
609680
2280
10:24
So these are just a few examples
of how I'm using art to feed into my work
of how I'm using art to feed into my work
200
612480
3936
10:28
as a designer and researcher
of artificial intelligence.
of artificial intelligence.
201
616440
3096
10:31
And I see it as a crucial way
to move innovation forward.
to move innovation forward.
202
619560
3496
10:35
Because right now, there are
a lot of extremes when it comes to AI.
a lot of extremes when it comes to AI.
203
623080
4376
10:39
Popular movies show it
as this destructive force
as this destructive force
204
627480
2816
10:42
while commercials
are showing it as a savior
are showing it as a savior
205
630320
3056
10:45
to solve some of the world's
most complex problems.
most complex problems.
206
633400
2936
10:48
But regardless of where you stand,
207
636360
2536
10:50
it's hard to deny
that we're living in a world
that we're living in a world
208
638920
2176
10:53
that's becoming more
and more digital by the second.
and more digital by the second.
209
641120
2576
10:55
Our lives revolve around our devices,
smart appliances and more.
smart appliances and more.
210
643720
4600
11:01
And I don't think
this will let up any time soon.
this will let up any time soon.
211
649400
2320
11:04
So, I'm trying to embed
more humanness from the start.
more humanness from the start.
212
652400
3736
11:08
And I have a hunch that bringing art
into an AI research process
into an AI research process
213
656160
5136
11:13
is a way to do just that.
214
661320
1896
11:15
Thank you.
215
663240
1216
11:16
(Applause)
216
664480
2280
ABOUT THE SPEAKER
Raphael Arar - Designer, researcherIBM's Raphael Arar creates art and designs experiences that examine the complexities of our increasingly technocentric lives.
Why you should listen
While his artwork raises questions about our relationship with modernity and technology, Raphael Arar’s design work revolves around answering those questions in human-centered ways.
Currently a designer and researcher at IBM Research, Arar has been exploring ways to translate aspects of emotional intelligence into machine-readable format. Through this, he is passionately tackling ethical platforms of artificial intelligence. Inc. Magazine says he "epitomizes the style and multi-disciplinary skill set of the modern designer," and in 2017, he was listed as one of Forbes's "30 under 30 in Enterprise Technology."
Raphael Arar | Speaker | TED.com