25
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Deep feedforward neural network models of vision dominate in both computational neuroscience and engineering. The primate visual system, by contrast, contains abundant recurrent connections. Recurrent signal flow enables recycling of limited computational resources over time, and so might boost the performance of a physically finite brain or model. Here we show: (1) Recurrent convolutional neural network models outperform feedforward convolutional models matched in their number of parameters in large-scale visual recognition tasks on natural images. (2) Setting a confidence threshold, at which recurrent computations terminate and a decision is made, enables flexible trading of speed for accuracy. At a given confidence threshold, the model expends more time and energy on images that are harder to recognise, without requiring additional parameters for deeper computations. (3) The recurrent model’s reaction time for an image predicts the human reaction time for the same image better than several parameter-matched and state-of-the-art feedforward models. (4) Across confidence thresholds, the recurrent model emulates the behaviour of feedforward control models in that it achieves the same accuracy at approximately the same computational cost (mean number of floating-point operations). However, the recurrent model can be run longer (higher confidence threshold) and then outperforms parameter-matched feedforward comparison models. These results suggest that recurrent connectivity, a hallmark of biological visual systems, may be essential for understanding the accuracy, flexibility, and dynamics of human visual recognition.

          Author summary

          Deep neural networks provide the best current models of biological vision and achieve the highest performance in computer vision. Inspired by the primate brain, these models transform the image signals through a sequence of stages, leading to recognition. Unlike brains in which outputs of a given computation are fed back into the same computation, these models do not process signals recurrently. The ability to recycle limited neural resources by processing information recurrently could explain the accuracy and flexibility of biological visual systems, which computer vision systems cannot yet match. Here we report that recurrent processing can improve recognition performance compared to similarly complex feedforward networks. Recurrent processing also enabled models to behave more flexibly and trade off speed for accuracy. Like humans, the recurrent network models can compute longer when an object is hard to recognise, which boosts their accuracy. The model’s recognition times predicted human recognition times for the same images. The performance and flexibility of recurrent neural network models illustrates that modeling biological vision can help us improve computer vision.

          Related collections

          Most cited references48

          • Record: found
          • Abstract: not found
          • Article: not found

          Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing

            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            ImageNet Large Scale Visual Recognition Challenge

              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              The Psychophysics Toolbox

                Bookmark

                Author and article information

                Contributors
                Role: ConceptualizationRole: Formal analysisRole: Funding acquisitionRole: InvestigationRole: MethodologyRole: VisualizationRole: Writing – original draftRole: Writing – review & editing
                Role: ConceptualizationRole: Funding acquisitionRole: InvestigationRole: MethodologyRole: SupervisionRole: Writing – original draftRole: Writing – review & editing
                Role: Data curationRole: Funding acquisitionRole: InvestigationRole: ResourcesRole: Writing – review & editing
                Role: Funding acquisitionRole: InvestigationRole: Writing – review & editing
                Role: ConceptualizationRole: Funding acquisitionRole: InvestigationRole: MethodologyRole: SupervisionRole: Writing – original draftRole: Writing – review & editing
                Role: Editor
                Journal
                PLoS Comput Biol
                PLoS Comput Biol
                plos
                ploscomp
                PLoS Computational Biology
                Public Library of Science (San Francisco, CA USA )
                1553-734X
                1553-7358
                October 2020
                2 October 2020
                : 16
                : 10
                : e1008215
                Affiliations
                [1 ] Medical Research Council Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom
                [2 ] Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
                [3 ] School of Psychology and Centre for Human Brain Health, University of Birmingham, United Kingdom
                [4 ] Department of Psychology, Department of Neuroscience, Department of Electrical Engineering, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA
                MIT, UNITED STATES
                Author notes

                The authors have declared that no competing interests exist.

                Author information
                http://orcid.org/0000-0003-3867-4900
                http://orcid.org/0000-0001-8076-6062
                http://orcid.org/0000-0002-3939-3003
                http://orcid.org/0000-0001-7433-9005
                Article
                PCOMPBIOL-D-19-01029
                10.1371/journal.pcbi.1008215
                7556458
                33006992
                cf0fb33c-6222-4410-a449-91a6932dba74
                © 2020 Spoerer et al

                This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

                History
                : 20 June 2019
                : 3 August 2020
                Page count
                Figures: 9, Tables: 4, Pages: 27
                Funding
                This project has received funding from the European Union’s Horizon 2020 Programme for Research and Innovation under the Specific Grant Agreement No. 720270 and 785907 (Human Brain Project SGA1 and SGA2), the NVIDIA GPU Grant Program, and the German Science Foundation (DFG grant ‘DynaVision’).
                Categories
                Research Article
                Biology and Life Sciences
                Neuroscience
                Cognitive Science
                Cognitive Neuroscience
                Reaction Time
                Biology and Life Sciences
                Neuroscience
                Cognitive Neuroscience
                Reaction Time
                Physical Sciences
                Physics
                Thermodynamics
                Entropy
                Computer and Information Sciences
                Neural Networks
                Recurrent Neural Networks
                Biology and Life Sciences
                Neuroscience
                Neural Networks
                Recurrent Neural Networks
                Biology and Life Sciences
                Neuroscience
                Cognitive Science
                Cognition
                Memory
                Visual Object Recognition
                Biology and Life Sciences
                Neuroscience
                Learning and Memory
                Memory
                Visual Object Recognition
                Biology and Life Sciences
                Neuroscience
                Cognitive Science
                Cognitive Psychology
                Perception
                Visual Object Recognition
                Biology and Life Sciences
                Psychology
                Cognitive Psychology
                Perception
                Visual Object Recognition
                Social Sciences
                Psychology
                Cognitive Psychology
                Perception
                Visual Object Recognition
                Computer and Information Sciences
                Data Management
                Data Visualization
                Infographics
                Graphs
                Computer and Information Sciences
                Neural Networks
                Feedforward Neural Networks
                Biology and Life Sciences
                Neuroscience
                Neural Networks
                Feedforward Neural Networks
                Biology and Life Sciences
                Physiology
                Sensory Physiology
                Visual System
                Biology and Life Sciences
                Neuroscience
                Sensory Systems
                Visual System
                Research and Analysis Methods
                Mathematical and Statistical Techniques
                Statistical Methods
                Multivariate Analysis
                Principal Component Analysis
                Physical Sciences
                Mathematics
                Statistics
                Statistical Methods
                Multivariate Analysis
                Principal Component Analysis
                Custom metadata
                vor-update-to-uncorrected-proof
                2020-10-14
                Code for neural network models is available at https://github.com/cjspoerer/rcnn-sat. Weights for trained neural network models and human behavioural data are available at https://osf.io/mz9hw/.

                Quantitative & Systems biology
                Quantitative & Systems biology

                Comments

                Comment on this article