Reading groups should identify new research directions and expand understanding while maximizing convenience. But, many feature an unadventurous lectural format, monocultural participation and a discussion with many questions but few answers. We want to do better.
Output: “We don’t just discuss; we produce”
We recently retrospected on the group’s first year. We found our reading group had generated great discussions and novel formats: real-time exploration of hyperparameter optimization in ipython notebooks or reimplementation of Spatial Transformer Networks with REINFORCE. We now want to share this experience with the world.
So, our group now writes a blog post collaboratively during the session before implementing the paper and extensions over the next week. Experimental results are discussed in the following session. That structure ensures implementation receives the attention it deserves and participants are empowered to see their hypotheses tested (particularly appealing to experts in other fields yet new to DL). In this way, our reading group addresses implementation, knowledge, and insight – all core to DL excellence.
Diversity: “We cultivate orthogonal contribution”
Deep Learning has a rich tradition of interdisciplinary input. Neuroscience, statistical mechanics, and theoretical neuroscience – just to name a few – have each shaped the field. Inspired by that success we have sought to complement the DL core with biologists, astrophysicists, distributed systems programmers. Furthermore, we dramatically accelerate their comfort reading recent deep learning papers by “paper-pairing”: reading papers paragraph by paragraph proceeding only when both participants have complete understanding. Some of our best sessions have been co-presentations by a DL expert and leading scientist from another discipline. For those motivated by application, few offer a better opportunity to find collaboration.
Maximizers: “Learning rates, # of convs, initialization, etc… are not the only hyperparameters to be optimized; we are”
We believe most endeavors should be systematically optimized so we are committed to quantifying our work and experimenting with new approaches. While individual learning has been thoroughly quantified to great effect, social learning has not. We hope to address the latter and thereby converge towards the best reading group possible.
Our reading group is hosted at Topos which attracts leading scientists from around the Bay Area to live together and collaborate on interesting problems at intersections of fields that are historically not well-linked.