ABOUT THE SPEAKER
Poppy Crum - Neuroscientist, technologist
Poppy Crum builds technologies that best leverage human physiology to enhance our experiences and how we interact with the world.

Why you should listen

Poppy Crum is dedicated to the development of immersive technologies that leverage human physiology and perceptual realities to enhance our experiences and interactions in the world. She has advanced a mission to democratize the way people of all abilities benefit from sensory technologies -- and how effectively technology communicates back to each of us. She believes the power of intelligent technologies is only realized with dynamic optimization and learning of as much of our personal and contextual data as possible.

Crum is chief scientist at Dolby Laboratories, leading the company's integration of neuroscience and sensory data science into its entertainment, communication and future technologies. She is also adjunct professor at Stanford University, where her work focuses on the impact and feedback potential of gaming and immersive environments, such as augmented and virtual reality, on neuroplasticity and learning. She has been recognized with the Advanced Imaging Society's Distinguished Leadership Award and the Consumer Technology Association's Technology and Standards Achievement Award for work towards the introduction of affordable, over-the-counter hearing-aid devices, and she is a fellow of the Audio Engineering Society. She has also been named to Billboard Magazine's 100 most influential female executives in the music industry. Prior to joining Dolby Laboratories, Crum was Research Faculty in the Department of Biomedical Engineering at Johns Hopkins School of Medicine.

More profile about the speaker
Poppy Crum | Speaker | TED.com
TED2018

Poppy Crum: Technology that knows what you're feeling

Filmed:
1,640,972 views

What happens when technology knows more about us than we do? Poppy Crum studies how we express emotions -- and she suggests the end of the poker face is near, as new tech makes it easy to see the signals that give away how we're feeling. In a talk and demo, she shows how "empathetic technology" can read physical signals like body temperature and the chemical composition of our breath to inform on our emotional state. For better or for worse. "If we recognize the power of becoming technological empaths, we get this opportunity where technology can help us bridge the emotional and cognitive divide," Crum says.
- Neuroscientist, technologist
Poppy Crum builds technologies that best leverage human physiology to enhance our experiences and how we interact with the world. Full bio

Double-click the English transcript below to play the video.

00:12
What happens when technology
knows more about us than we do?
0
881
4456
00:17
A computer now can detect
our slightest facial microexpressions
1
5992
3672
00:21
and be able to tell the difference
between a real smile and a fake one.
2
9688
3611
00:25
That's only the beginning.
3
13323
1734
00:27
Technology has become
incredibly intelligent
4
15466
2865
00:30
and already knows a lot
about our internal states.
5
18355
3400
00:34
And whether we like it or not,
6
22085
2286
00:36
we already are sharing
parts of our inner lives
7
24395
3499
00:39
that's out of our control.
8
27918
1733
00:43
That seems like a problem,
9
31413
1421
00:44
because a lot of us like to keep
what's going on inside
10
32858
3246
00:48
from what people actually see.
11
36128
1647
00:50
We want to have agency
over what we share and what we don't.
12
38323
4420
00:55
We all like to have a poker face.
13
43473
2321
00:59
But I'm here to tell you
that I think that's a thing of the past.
14
47584
3346
01:03
And while that might sound scary,
it's not necessarily a bad thing.
15
51347
4770
01:09
I've spent a lot of time
studying the circuits in the brain
16
57030
2770
01:11
that create the unique perceptual
realities that we each have.
17
59824
3693
01:16
And now I bring that together
18
64110
1405
01:17
with the capabilities
of current technology
19
65539
2062
01:19
to create new technology
that does make us better,
20
67625
2537
01:22
feel more, connect more.
21
70186
1600
01:24
And I believe to do that,
22
72482
1786
01:26
we have to be OK
losing some of our agency.
23
74292
2749
01:30
With some animals, it's really amazing,
24
78149
2523
01:32
and we get to see into
their internal experiences.
25
80696
3474
01:36
We get this upfront look
at the mechanistic interaction
26
84649
3722
01:40
between how they respond
to the world around them
27
88395
2817
01:43
and the state of their biological systems.
28
91236
2008
01:45
This is where evolutionary pressures
like eating, mating
29
93268
3809
01:49
and making sure we don't get eaten
30
97101
1762
01:50
drive deterministic behavioral responses
to information in the world.
31
98887
4157
01:55
And we get to see into this window,
32
103806
2794
01:58
into their internal states
and their biological experiences.
33
106624
3636
02:02
It's really pretty cool.
34
110284
1642
02:03
Now, stay with me for a moment --
I'm a violinist, not a singer.
35
111950
4103
02:08
But the spider's already
given me a critical review.
36
116077
3590
02:16
(Video) (Singing in a low pitch)
37
124907
2060
02:19
(Singing in a middle pitch)
38
127868
2888
02:23
(Singing in a high pitch)
39
131800
2505
02:27
(Singing in a low pitch)
40
135069
1421
02:29
(Singing in a middle pitch)
41
137236
1600
02:31
(Singing in a high pitch)
42
139403
1777
02:33
(Laughter)
43
141204
1150
02:36
Poppy Crum: It turns out, some spiders
tune their webs like violins
44
144387
3198
02:39
to resonate with certain sounds.
45
147609
2158
02:41
And likely, the harmonics
of my voice as it went higher
46
149791
2771
02:44
coupled with how loud I was singing
47
152586
1730
02:46
recreated either the predatory call
of an echolocating bat or a bird,
48
154340
4467
02:50
and the spider did what it should.
49
158831
1881
02:53
It predictively told me to bug off.
50
161300
2817
02:56
I love this.
51
164824
1150
02:58
The spider's responding
to its external world
52
166546
3309
03:01
in a way that we get to see and know
what's happening to its internal world.
53
169879
4350
03:07
Biology is controlling
the spider's response;
54
175069
2206
03:09
it's wearing its internal
state on its sleeve.
55
177299
2776
03:13
But us, humans --
56
181768
1655
03:16
we're different.
57
184184
1150
03:17
We like to think we have cognitive control
over what people see, know and understand
58
185899
5735
03:23
about our internal states --
59
191658
1409
03:25
our emotions, our insecurities,
our bluffs, our trials and tribulations --
60
193091
4303
03:29
and how we respond.
61
197418
1267
03:31
We get to have our poker face.
62
199927
2282
03:35
Or maybe we don't.
63
203799
1200
03:37
Try this with me.
64
205728
1182
03:38
Your eye responds
to how hard your brain is working.
65
206934
2690
03:42
The response you're about to see
is driven entirely by mental effort
66
210363
3230
03:45
and has nothing to do
with changes in lighting.
67
213617
2635
03:48
We know this from neuroscience.
68
216276
1650
03:49
I promise, your eyes are doing
the same thing as the subject in our lab,
69
217950
4560
03:54
whether you want them to or not.
70
222534
1734
03:56
At first, you'll hear some voices.
71
224292
2173
03:58
Try and understand them
and keep watching the eye in front of you.
72
226489
3278
04:01
It's going to be hard at first,
73
229791
1498
04:03
one should drop out,
and it should get really easy.
74
231313
2391
04:05
You're going to see the change in effort
in the diameter of the pupil.
75
233728
3325
04:10
(Video) (Two overlapping voices talking)
76
238140
2567
04:12
(Single voice) Intelligent technology
depends on personal data.
77
240731
2963
04:15
(Two overlapping voices talking)
78
243718
2446
04:18
(Single voice) Intelligent technology
depends on personal data.
79
246188
2976
04:21
PC: Your pupil doesn't lie.
80
249680
1326
04:23
Your eye gives away your poker face.
81
251030
2400
04:25
When your brain's having to work harder,
82
253990
1913
04:27
your autonomic nervous system
drives your pupil to dilate.
83
255927
2785
04:30
When it's not, it contracts.
84
258736
1555
04:32
When I take away one of the voices,
85
260680
1691
04:34
the cognitive effort
to understand the talkers
86
262395
2262
04:36
gets a lot easier.
87
264681
1158
04:37
I could have put the two voices
in different spatial locations,
88
265863
3000
04:40
I could have made one louder.
89
268887
1666
04:42
You would have seen the same thing.
90
270577
1738
04:45
We might think we have more agency
over the reveal of our internal state
91
273006
4786
04:49
than that spider,
92
277816
1579
04:51
but maybe we don't.
93
279419
1266
04:53
Today's technology is starting
to make it really easy
94
281021
2969
04:56
to see the signals and tells
that give us away.
95
284014
2690
04:59
The amalgamation of sensors
paired with machine learning
96
287109
3294
05:02
on us, around us and in our environments,
97
290427
2413
05:04
is a lot more than cameras and microphones
tracking our external actions.
98
292864
4653
05:12
Our bodies radiate our stories
99
300529
2818
05:15
from changes in the temperature
of our physiology.
100
303371
2666
05:18
We can look at these
as infrared thermal images
101
306546
2261
05:20
showing up behind me,
102
308831
1160
05:22
where reds are hotter
and blues are cooler.
103
310015
2070
05:24
The dynamic signature
of our thermal response
104
312458
3183
05:27
gives away our changes in stress,
105
315665
3031
05:30
how hard our brain is working,
106
318720
2008
05:32
whether we're paying attention
107
320752
1936
05:34
and engaged in the conversation
we might be having
108
322712
2627
05:37
and even whether we're experiencing
a picture of fire as if it were real.
109
325363
4095
05:41
We can actually see
people give off heat on their cheeks
110
329482
2643
05:44
in response to an image of flame.
111
332149
2200
05:48
But aside from giving away
our poker bluffs,
112
336013
2929
05:50
what if dimensions of data
from someone's thermal response
113
338966
4746
05:55
gave away a glow
of interpersonal interest?
114
343736
2659
05:58
Tracking the honesty of feelings
in someone's thermal image
115
346966
3532
06:02
might be a new part of how
we fall in love and see attraction.
116
350522
3626
06:06
Our technology can listen,
develop insights and make predictions
117
354172
3693
06:09
about our mental and physical health
118
357889
2095
06:12
just by analyzing the timing dynamics
of our speech and language
119
360008
4000
06:16
picked up by microphones.
120
364032
1443
06:18
Groups have shown that changes
in the statistics of our language
121
366038
3880
06:21
paired with machine learning
122
369942
1420
06:23
can predict the likelihood
someone will develop psychosis.
123
371386
3161
06:27
I'm going to take it a step further
124
375442
1751
06:29
and look at linguistic changes
and changes in our voice
125
377217
2587
06:31
that show up with a lot
of different conditions.
126
379828
2239
06:34
Dementia, diabetes can alter
the spectral coloration of our voice.
127
382091
4367
06:39
Changes in our language
associated with Alzheimer's
128
387205
3119
06:42
can sometimes show up more
than 10 years before clinical diagnosis.
129
390348
4365
06:47
What we say and how we say it
tells a much richer story
130
395236
3960
06:51
than we used to think.
131
399220
1254
06:53
And devices we already have in our homes
could, if we let them,
132
401022
4047
06:57
give us invaluable insight back.
133
405093
2134
06:59
The chemical composition of our breath
134
407998
2978
07:03
gives away our feelings.
135
411959
1354
07:06
There's a dynamic mixture of acetone,
isoprene and carbon dioxide
136
414363
4478
07:10
that changes when our heart speeds up,
when our muscles tense,
137
418865
3384
07:14
and all without any obvious change
in our behaviors.
138
422809
2897
07:18
Alright, I want you to watch
this clip with me.
139
426268
2738
07:21
Some things might be going on
on the side screens,
140
429030
3119
07:24
but try and focus on
the image in the front
141
432173
3777
07:27
and the man at the window.
142
435974
1463
07:31
(Eerie music)
143
439633
2658
07:39
(Woman screams)
144
447767
1437
07:50
PC: Sorry about that.
I needed to get a reaction.
145
458692
2395
07:53
(Laughter)
146
461111
1785
07:55
I'm actually tracking the carbon dioxide
you exhale in the room right now.
147
463412
4972
08:01
We've installed tubes
throughout the theater,
148
469903
3293
08:05
lower to the ground,
because CO2 is heavier than air.
149
473220
2595
08:07
But they're connected
to a device in the back
150
475839
2667
08:10
that lets us measure, in real time,
with high precision,
151
478530
3287
08:13
the continuous differential
concentration of CO2.
152
481841
2922
08:17
The clouds on the sides are actually
the real-time data visualization
153
485246
5508
08:22
of the density of our CO2.
154
490778
1998
08:25
You might still see
a patch of red on the screen,
155
493374
3699
08:29
because we're showing increases
with larger colored clouds,
156
497097
3705
08:32
larger colored areas of red.
157
500826
2196
08:35
And that's the point
where a lot of us jumped.
158
503046
2559
08:38
It's our collective suspense
driving a change in carbon dioxide.
159
506173
4915
08:43
Alright, now, watch this
with me one more time.
160
511649
2722
08:46
(Cheerful music)
161
514395
2238
08:54
(Woman laughs)
162
522553
2137
09:05
PC: You knew it was coming.
163
533344
1349
09:06
But it's a lot different
when we changed the creator's intent.
164
534717
3363
09:10
Changing the music and the sound effects
165
538776
2769
09:13
completely alter the emotional
impact of that scene.
166
541569
3603
09:17
And we can see it in our breath.
167
545196
2134
09:20
Suspense, fear, joy
168
548196
2262
09:22
all show up as reproducible,
visually identifiable moments.
169
550482
4507
09:27
We broadcast a chemical signature
of our emotions.
170
555473
4151
09:35
It is the end of the poker face.
171
563249
2133
09:38
Our spaces, our technology
will know what we're feeling.
172
566582
3566
09:42
We will know more about each other
than we ever have.
173
570736
2785
09:45
We get a chance to reach in and connect
to the experience and sentiments
174
573911
4307
09:50
that are fundamental to us as humans
175
578242
1742
09:52
in our senses, emotionally and socially.
176
580008
2410
09:55
I believe it is the era of the empath.
177
583482
2540
09:58
And we are enabling the capabilities
that true technological partners can bring
178
586046
5222
10:03
to how we connect with each other
and with our technology.
179
591292
3047
10:06
If we recognize the power
of becoming technological empaths,
180
594363
3389
10:09
we get this opportunity
181
597776
1936
10:11
where technology can help us bridge
the emotional and cognitive divide.
182
599736
4424
10:16
And in that way, we get to change
how we tell our stories.
183
604680
2723
10:19
We can enable a better future
for technologies like augmented reality
184
607427
3580
10:23
to extend our own agency
and connect us at a much deeper level.
185
611031
4193
10:27
Imagine a high school counselor
being able to realize
186
615625
2547
10:30
that an outwardly cheery student
really was having a deeply hard time,
187
618196
3826
10:34
where reaching out can make
a crucial, positive difference.
188
622046
3180
10:37
Or authorities, being able
to know the difference
189
625766
3230
10:41
between someone having
a mental health crisis
190
629020
2325
10:43
and a different type of aggression,
191
631369
1826
10:45
and responding accordingly.
192
633219
1800
10:47
Or an artist, knowing
the direct impact of their work.
193
635609
3273
10:52
Leo Tolstoy defined his perspective of art
194
640173
2643
10:54
by whether what the creator intended
195
642840
1785
10:56
was experienced by the person
on the other end.
196
644649
2586
10:59
Today's artists can know
what we're feeling.
197
647259
2566
11:02
But regardless of whether
it's art or human connection,
198
650204
3005
11:06
today's technologies
will know and can know
199
654608
2802
11:09
what we're experiencing on the other side,
200
657434
2048
11:11
and this means we can be
closer and more authentic.
201
659506
2618
11:14
But I realize a lot of us
have a really hard time
202
662498
4293
11:18
with the idea of sharing our data,
203
666815
2267
11:21
and especially the idea
that people know things about us
204
669673
3111
11:24
that we didn't actively choose to share.
205
672808
2321
11:28
Anytime we talk to someone,
206
676728
2216
11:31
look at someone
207
679946
1555
11:33
or choose not to look,
208
681525
1468
11:35
data is exchanged, given away,
209
683017
2647
11:38
that people use to learn,
210
686533
2205
11:40
make decisions about
their lives and about ours.
211
688762
3267
11:45
I'm not looking to create a world
where our inner lives are ripped open
212
693469
3968
11:49
and our personal data
and our privacy given away
213
697461
2348
11:51
to people and entities
where we don't want to see it go.
214
699833
2713
11:55
But I am looking to create a world
215
703117
2762
11:57
where we can care about
each other more effectively,
216
705903
3408
12:01
we can know more about when
someone is feeling something
217
709335
3060
12:04
that we ought to pay attention to.
218
712419
1872
12:06
And we can have richer experiences
from our technology.
219
714800
3335
12:10
Any technology
can be used for good or bad.
220
718887
2357
12:13
Transparency to engagement
and effective regulation
221
721268
2412
12:15
are absolutely critical
to building the trust for any of this.
222
723704
3120
12:20
But the benefits that "empathetic
technology" can bring to our lives
223
728106
4834
12:24
are worth solving the problems
that make us uncomfortable.
224
732964
3891
12:29
And if we don't, there are
too many opportunities and feelings
225
737315
4025
12:33
we're going to be missing out on.
226
741364
1695
12:35
Thank you.
227
743083
1175
12:36
(Applause)
228
744282
2479

▲Back to top

ABOUT THE SPEAKER
Poppy Crum - Neuroscientist, technologist
Poppy Crum builds technologies that best leverage human physiology to enhance our experiences and how we interact with the world.

Why you should listen

Poppy Crum is dedicated to the development of immersive technologies that leverage human physiology and perceptual realities to enhance our experiences and interactions in the world. She has advanced a mission to democratize the way people of all abilities benefit from sensory technologies -- and how effectively technology communicates back to each of us. She believes the power of intelligent technologies is only realized with dynamic optimization and learning of as much of our personal and contextual data as possible.

Crum is chief scientist at Dolby Laboratories, leading the company's integration of neuroscience and sensory data science into its entertainment, communication and future technologies. She is also adjunct professor at Stanford University, where her work focuses on the impact and feedback potential of gaming and immersive environments, such as augmented and virtual reality, on neuroplasticity and learning. She has been recognized with the Advanced Imaging Society's Distinguished Leadership Award and the Consumer Technology Association's Technology and Standards Achievement Award for work towards the introduction of affordable, over-the-counter hearing-aid devices, and she is a fellow of the Audio Engineering Society. She has also been named to Billboard Magazine's 100 most influential female executives in the music industry. Prior to joining Dolby Laboratories, Crum was Research Faculty in the Department of Biomedical Engineering at Johns Hopkins School of Medicine.

More profile about the speaker
Poppy Crum | Speaker | TED.com