Badger Seminar 2020: MLMA Workshop

When and where?

The workshop was split into two parts:

• First part August 10- 14: one week for all, presentations, working groups, discussions
• Second part August 17 – September 4: extra 3 weeks of focused work (experiments), for people who can stay.

Full list of talks and videos

Day 1

• Workshop & Badger Introduction and Principia Badgerica (VIDEO) (Jan Feyereisl, GoodAI & Marek Rosa, GoodAI)
• Related Research 1 – End to end differentiable Self organising systems (Ettore Randazzo, Google Research & Alexander Mordvintsev, Google Research)
• Related Research 2 – Discovering Reinforcement Learning Algorithms  (Junhyuk Oh,  DeepMind)
• Related Research 3 – Compositional Control: Intelligence Without a Brain (VIDEO) (Deepak Pathak, CMU

Day 2

• A multi-agent perspective to AI (VIDEO) (Anuj Mahajan, University of Oxford)
• Multi-Agent Learning 2 (Jun Wang, UCL)
• Learned Communication (Angeliki Lazaridou, DeepMind)
• Big Picture Discussion: Benefits of Communication & Multi-Agentness
• Self-Play and Zero-Shot Human AI Coordination in Hanabi (VIDEO) (Jakob Foerster, The Vector Institute)

Day 3

• Relative Overgeneralization in Distributed Control (VIDEO) (Wendelin Boehmer, University of Oxford)
• Modular Meta-Learning: Learning to build up knowledge through modularity (VIDEO) (Ferran Alet, MIT)
• Big Picture Discussion: Benefits of Modularity & Internal Structure
• Modular & Compositional Computation (VIDEO) (Clemens Rosenbaum, ASAPP Inc)
• Complexity: Concepts, Abstraction, and Analogy in Natural and Artificial Intelligence (VIDEO) (Melanie Mitchell, The Santa Fe Institute)

Day 4

• Open-Endedness 1: Measuring growth of complexity (Tomas Mikolov, CIIRC)
• Why Think? (VIDEO) (Nicholas Guttenberg, GoodAI & Cross Compass)
• Minimum Viable Environments (VIDEO) (Julian Togelius, New York University)
• Open-Endedness 2: The Importance of Open-Endedness in AI and Machine Learning (VIDEO) (Kenneth Stanley, OpenAI)
• Big Picture Discussion: Open-Endedness, Deliberation, and Discovery of Algorithms

Day 5

• The Science of Deep Learning 1 (VIDEO) (Stanislav Fort, Stanford)
• Learned Learning Algorithms (VIDEO) (Luke Metz, Google Brain)
• The Science of Deep Learning 2, Understanding Neural Networks via Pruning (Jonathan Frankle, MIT)
• Big Picture Discussion: Generalization & Scalability

Code of conduct for the workshop

All attendees and speakers must abide by GoodAI’s Code of Conduct during the Workshop.

Recommended reading

The following is a list of recommended reading for the workshop. Items highlighted in bold demonstrate most representative related work within each topic. Items with an asterisk * denote articles whose author will be presenting or participating in the workshop. Items in italics represent articles that provide an informative summary of a particular topic or provide a comprehensive/unified view of the corresponding sub-area or its aspect. Some items are located in more than one section due to their relevance to more than one sub-topic.

Badger

• Badger paper*: BADGER: Learning to (Learn [Learning Algorithms] through Multi-Agent Communication)
• Recent badger blog posts*: here, here and here.
• Recent badger workshops*: here and here.

Meta-learning

• Discovering Reinforcement Learning Algorithms*: https://arxiv.org/abs/2007.08794
• Improving Generalization in Meta Reinforcement Learning using Learned Objectives*: https://arxiv.org/abs/1910.04098
• Meta-learning of Sequential Strategies: https://arxiv.org/abs/1905.03030
• Understanding and correcting pathologies in the training of learned optimizers*: https://arxiv.org/abs/1810.10180
• Meta-learners’ learning dynamics are unlike learners’: https://arxiv.org/abs/1905.01320
• Meta-Learning in Neural Networks: A Survey: https://arxiv.org/abs/2004.05439
• Learning to Learn with Feedback and Local Plasticity: https://arxiv.org/abs/2006.09549
• Finding online neural update rules by learning to remember: https://arxiv.org/abs/2003.03124

Multi-Agent Learning & Emergent Communication

• Learning Multiagent Communication with Backpropagation: https://arxiv.org/pdf/1605.07736.pdf
• Learning Attentional Communication for Multi-Agent Cooperation*: https://arxiv.org/abs/1805.07733
• Learning to communicate with deep multi-agent reinforcement learning*: http://papers.nips.cc/paper/6042-learning-to-communicate-with-deep-multi-agent-reinforcement-learning.pdf
• Emergent Multi-Agent Communication in the Deep Learning Era*: https://arxiv.org/abs/2006.02419
• Emergence of Grounded Compositional Language in Multi-Agent Populations: https://arxiv.org/abs/1703.04908
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments: https://arxiv.org/abs/1706.02275
• Decentralized Multi-Agent Actor-Critic with Generative Inference*: https://arxiv.org/abs/1910.03058
• AI-QMIX: Attention and Imagination for Dynamic Multi-Agent Reinforcement Learning*: https://arxiv.org/abs/2006.04222
• MAVEN: Multi-Agent Variational Exploration*: https://deepai.org/publication/maven-multi-agent-variational-exploration
• A Survey and Critique of Multiagent Deep Reinforcement Learning: https://arxiv.org/abs/1810.05587

Modularity & Generalized Deep Learning

• One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control*: https://wenlong.page/modular-rl/
• Learning to Control Self-Assembling Morphologies: A Study of Generalization via Modularity*: https://pathak22.github.io/modular-assemblies/
• MPLP: Learning a Message Passing Learning Protocol*: https://arxiv.org/abs/2007.00970
• Neural Relational Inference with Fast Modular Meta-learning*: https://papers.nips.cc/paper/9353-neural-relational-inference-with-fast-modular-meta-learning.pdf
• Modular meta-learning*: https://arxiv.org/abs/1806.10166
• Graph Element Networks: adaptive, structured computation and memory*: https://arxiv.org/abs/1904.09019
• Routing Networks and the Challenges of Modular and Compositional Computation*: https://arxiv.org/abs/1904.12774
• Are Neural Nets Modular? Inspecting Their Functionality Through Differentiable Weight Masks: http://people.idsia.ch/~csordas/are_neural_networks_modular.pdf
• Graph Structure of Neural Networks: https://arxiv.org/abs/2007.06559
• Recurrent Independent Mechanisms: https://arxiv.org/abs/1909.10893
• Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules: https://arxiv.org/abs/2006.16981
• S2RMs: Spatially Structured Recurrent Modules: https://arxiv.org/abs/2007.06533

Gradual and Continual Learning

• Learning to Continually Learn*: https://arxiv.org/abs/2002.09571
• Bilevel Continual Learning: https://arxiv.org/abs/2007.15553

Inductive Biases, Symmetries and Invariances

• Learning Invariance vs. Model Invariance: Implications for Meta-Learning*: tbd
• The Holy Grail of Deep Learning: Modelling Invariances: https://www.inference.vc/the-holy-gr/
• Natural Graph Networks: https://arxiv.org/abs/2007.08349
• Meta-Learning Symmetries by Reparameterization: https://arxiv.org/abs/2007.02933

Meta-control and Deliberation

• Automatically Composing Representation Transformations as a Means for Generalization: https://arxiv.org/abs/1807.04640
• Doing more with less: meta-reasoning and meta-learning in humans and machines: https://cocosci.princeton.edu/papers/doing-more-with-less.pdf
• Metacontrol for Adaptive Imagination-Based Optimization: https://arxiv.org/abs/1705.02670
• A Theory of Usable Information under Computational Constraints: https://openreview.net/forum?id=r1eBeyHFDH

Neural Message Passing & Graph Neural Networks

• Relational inductive biases, deep learning, and graph networks: https://arxiv.org/abs/1806.01261
• NerveNet: Learning Structured Policy with Graph Neural Networks: https://openreview.net/forum?id=S1sqHMZCb
• Pointer Graph Networks: https://arxiv.org/abs/2006.06380
• Natural Graph Networks: https://arxiv.org/abs/2007.08349
• Learning TSP Requires Rethinking Generalization: https://arxiv.org/abs/2006.07054
• Neural Message Passing for Quantum Chemistry: https://arxiv.org/abs/1704.01212
• Benchmarking Graph Neural Networks: https://arxiv.org/abs/2003.00982
• Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective: https://arxiv.org/abs/2003.00330

Neural Optimizers

• Learning to Learn without Gradient Descent by Gradient Descent: https://arxiv.org/abs/1611.03824
• Understanding and correcting pathologies in the training of learned optimizers*: https://arxiv.org/abs/1810.10180
• Learned Optimizers that Scale and Generalize: https://arxiv.org/abs/1703.04813

Optimization, Adaptation and Generalization

• Rapid Task-Solving in Novel Environments: https://arxiv.org/abs/2006.03662
• Generalized Planning With Deep Reinforcement Learning: https://arxiv.org/abs/2005.02305
• Learning TSP Requires Rethinking Generalization: https://arxiv.org/abs/2006.07054
• A Brief Look at Generalization in Visual Meta-Reinforcement Learning: https://arxiv.org/abs/2006.07262
• Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm: https://arxiv.org/abs/1710.11622
• The Early Phase of Neural Network Training*: https://arxiv.org/abs/2002.10365
• The Break-Even Point on the Optimization Trajectories of Deep Neural Networks*: https://openreview.net/pdf?id=r1g87C4KwB

Program Induction & Synthesis

• Strong Generalization and Efficiency in Neural Programs: https://arxiv.org/abs/2007.03629
• Neural Execution Engines: Learning to Execute Subroutines: https://arxiv.org/abs/2006.08084

Curriculum learning, open-endedness & Minimum Viable Environments

• Evolving Structures in Complex Systems*: https://arxiv.org/abs/1911.01086
• Enhanced POET: Open-Ended Reinforcement Learning through Unbounded Invention of Learning Challenges and their Solutions*: https://arxiv.org/abs/2003.08536
• Procedural Content Generation via Machine Learning (PCGML)*: https://arxiv.org/abs/1702.00539
• Object-Oriented Curriculum Generation for Reinforcement Learning*: https://www.researchgate.net/publication/323738119_Object-Oriented_Curriculum_Generation_for_Reinforcement_Learning
• Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data*: https://arxiv.org/abs/1912.07768
• CommAI: Evaluating the first steps towards a useful general AI*: https://arxiv.org/abs/1701.08954
• Pommerman: A multi-agent playground*: https://arxiv.org/abs/1809.07124

Further reading

• Growing Neural Cellular Automata*: https://distill.pub/2020/growing-ca/
• AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence: https://arxiv.org/abs/1905.10985
• Network of Evolvable Neural Units: Evolving to Learn at a Synaptic Level*: https://arxiv.org/abs/1912.07589
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games*: https://arxiv.org/abs/1703.10069
• Learning Structured Communication for Multi-agent Reinforcement Learning*: https://arxiv.org/pdf/2002.04235.pdf
• Graph Convolutional Reinforcement Learning: https://arxiv.org/pdf/1810.09202.pdf
• TarMAC: Targeted Multi-Agent Communication: https://arxiv.org/pdf/1810.11187.pdf