420 Maynard St.

Ann Arbor, MI 48109-1327

www.michigandaily.com

NISA KHAN

Editor in Chief

734-418-4115 ext. 1251

nisakhan@michigandaily.com

JUEUI HONG

Business Manager

734-418-4115 ext. 1241

jueuih@michigandaily.com

EDITORIAL STAFF

Dayton Hare 
 Managing Editor 

haredayt@michigandaily.com

Jennifer Meer Managing News Editor 

news@michigandaily.com 

SENIOR NEWS EDITORS: Andrew 
Hiyama, Kaela Theut

Sarah Khan Editorial Page Editor 

tothedaily@michigandaily.com

Jacob Shames Managing Sports Editor

sportseditors@michigandaily.com

Will Stewart Managing Arts Editor 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

arts@michigandaily.com

SENIOR ARTS EDITOR: Dom Polsinelli 

Alexis Rankin 

and Max Kuang Managing Photo Editors 

photo@michigandaily.com

Michelle Phillips Managing Design Editor 
 
 

design@michigandaily.com

Emily Miiller Managing Copy Editor

copydesk@michigandaily.com

SENIOR COPY EDITOR: Ashley Woonton

Christian Paneda Managing MiC Editor

michiganincolor@michigandaily.com

Abe Lofy Managing Video Editor

video@michigandaily.com

Ishan Vashinta Managing Web Editor

ivashi@michigandaily.com

Ellie Homant Managing Social Media Editor

CONTACT INFORMATION 

Newsroom Office hours:

Sun.-Thurs. 11 a.m. - 2 a.m.

 
 734-763-2459 opt.3

News Tips news@michigandaily.com

Corrections 
 corrections@michigandaily.com

Letters to the Editor tothedaily@michigandaily.com 

or visit michigandaily.com/letters

Photo Department photo@michigandaily.com
Arts Section arts@michigandaily.com

Editorial Page opinion@michigandaily.com
Sports Section sports@michigandaily.com
Magazine statement@michigandaily.com 
 

Advertising Phone: 734-418-4115 
Department dailydisplay@gmail.com

The Michigan Daily (ISSN 0745-967) is 
published every Thursday during the 
spring and summer terms by students 
at the University of Michigan. One copy 
is available free of charge to all readers. 
Additional copies may be picked up at the 
Daily’s office for $2. Subscriptions for fall 
term, starting in September, via U.S. mail 
are $110. Winter term (January through 
April) is $115, yearlong (September 
through April) is $195. University affiliates 
are subject to a reduced subscription rate. 
On-campus subscriptions for fall term 
are $35. Subscriptions must be prepaid. 
The Michigan Daily is a member of The 
Associated Press and The Associated 
Collegiate Press. 

2

Thursday, June 15, 2017
The Michigan Daily — michigandaily.com
NEWS

Rackham conference showcases findings
and innovations in decision-making

The four-day event 
brought together 
researchers, students 
and professionals 
across disciplines

By JENNIFER MEER

Summer Managing News Editor

The 
third 
Multidisciplinary 

Conference 
on 
Reinforcement 

Learning and Decision Making 
took place at the Rackham Graduate 
School 
between 
Sunday 
and 

Thursday this week.

Reinforcement learning refers to 

the relationship between the agent 
— like a person or robot, for example 
— and the environment — like a 
video game or puzzle the agent is 
trying to complete. The environment 
represents a certain state of being, 
while the agent acts upon that state 
and receives a reward, or response. 

The 
four-day 
event 
saw 

researchers, 
professionals 
and 

students from several disciplines, 
including psychology and computer 
science, come together to discuss 

discoveries 
and 
innovations 
in 

decision-making.

Among 
the 
event 
speakers 

were Ece Kamar, a researcher 
from Microsoft who specializes in 
artificial intelligence, Joelle Pineau, 
an assistant professor of computer 
science at McGill University and Kent 
Berridge, James Olds Distinguished 
University Professor of Psychology 
and Neuroscience at the University of 
Michigan.

The days were broken into sessions 

of several brief presentations from 
speakers describing their work.

On 
Tuesday 
afternoon, 
Max 

Kleiman-Weiner, a Ph.D. student in 
Computational Cognitive Science 
at MIT, presented “Learning to 
Cooperate and Compete.”

Prefacing his address with a 

quote from psychologist Nicholas 
Humphrey, Kleiman-Weiner related 
social interactions to games — 
incorporating elements of game 
theory in his presentation.

From a research standpoint, games 

like tic-tac-toe and checkers have 
effectively been solved and can be 
modeled by computers, he explained. 
There are other games where 
computers can act like humans, but 
cannot necessarily solve the game, 

such as in a game of chance.

“It’s these kinds of ad hoc 

interactions 
that 
involve 
lots 

of 
complexity, 
where 
you’re 

cooperatively playing a game, but 
there might be aspects of competition 
in that game that can explain a lot of 
the richness of human cooperative 
behavior in a lot of that real world 
contexts that we might want to have 
machines working with us and at 
least understanding us, so things 
like how to negotiate, playground 
games,” he said. “We want to study 
cooperation.”

There are models for studying 

cooperation, 
one 
of 
which 
is 

the prisoner’s dilemma, which 
highlights 
a 
struggle 
between 

what is good for the well-being of 
both players versus what is more 
advantageous to a single player. 
Mathematically, the dilemma points 
to tension between cooperation and 
competition but, experimentally, it 
doesn’t capture several important 
factors, according to Kleiman-
Weiner.

He 
explained 
his 
team 
of 

researchers designed a system to 
overcome obstacles presented in 
current models of cooperation.

“What we’ve done… to study 

these questions both in a way where 
we can build more models and look 
at human behavior is to take an 
approach that’s been well studied 
in multi-agent systems literature,” 
he said.

The 
team 
developed 
games 

with 
naturalistic 
environments 

that people can play intuitively, 
like video games. The games can 
represent different social situations.

“If we have these programs, we 

can sort of reconstruct the original 
social dilemma and think, ‘Well, 
I have this choice to cooperate or 
compete, and that corresponds 
to certain rewards,’ ” he said. “In 
reality, it’s a lot more challenging 
than that.”

Different ways of implementing 

potential plans yield different 
payoffs, 
Kleiman-Weiner 

explained. However, the team aims 
to build algorithms to play the 
games.

Difficulties 
arise 
when 
the 

researchers want to be able to 
plan at a high level of abstraction 
— 
determining 
cooperation 

or competition — but need to 
implement these goals through 
low-level actions, like moves in 
a game. They need to be able to 
determine if certain low-level 
actions confirm the higher-level 
goals. Furthermore, they also 
want to coordinate cooperation 
across different scenarios, so as 
to generalize and create coherent 
plans to tackle certain games.

Overall, 
the 
researchers 

aim to make plans for using 
generalizations and best-response 
scenarios for competition and 
coordination using reinforcement 
learning.

 
 
 
 
 COURTESY OF JENNIFER MEER

MIT graduate student Max Kleiman-Weiner presents at the third annual 
Multidiscipliary Conference on Reinforcement Learning and Decision Making at 
Rackham Graduate School.

Read more at MichiganDaily.com

