Autonomous and Efficiently Scalable Deep Learning

• Applicant: Professor Dr. Jörg Lücke
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: from 2014 to 2020
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 260197604

• Final Report Year: 2019

Final Report Abstract

In summary, the project was not only successful in taking important steps towards more autonomous systems, that are able to learn from as little data as possible in a mathematically grounded fashion. But it also importantly contributed to spread light on a severe general issue in semi-supervised learning, that is the need to take validation data into account when comparing systems that learn on few labeled data. Furthermore, novel developments on efficient scaling enabled the developed NeSi networks to be applicable at realistic scales and with many parameters. Furthermore, the scalability methods applied gave rise to novel and more broadly applicable learning algorithms.

Publications

• Beyond manual tuning of hyperparameters. KI – Künstliche Intelligenz, 29(4):329–337, 2015
  Frank Hutter, Jörg Lücke, and Lars Schmidt-Thieme
  (See online at https://doi.org/10.1007/s13218-015-0381-0)
• Select-and-sample for spike-and-slab sparse coding. In Advances in Neural Information Processing Systems (NIPS), volume 29, pages 3927–3935, 2016
  Abdul-Saboor Sheikh and Jörg Lücke
  
• GP-select: Accelerating EM using adaptive subspace preselection. Neural Computation, 29(8): 2177–2202, 2017
  Jacquelyn A Shelton, Jan Gasthaus, Zhenwen Dai, Jörg Lücke, and Arthur Gretton
  (See online at https://doi.org/10.1162/neco_a_00982)
• Models of acetylcholine and dopamine signals differentially improve neural representations. Frontiers in Computational Neuroscience, 11:54, 2017
  Raphael Holca-Lamarre, Jörg Lücke, and Klaus Obermayer
  (See online at https://doi.org/10.3389/fncom.2017.00054)
• Truncated variational EM for semi-supervised Neural Simpletrons. In International Joint Conference on Neural Networks (IJCNN), pages 3769–3776, 2017
  Dennis Forster and Jörg Lücke
  (See online at https://doi.org/10.1109/IJCNN.2017.7966331)
• Can clustering scale sublinearly with its clusters? A variational EM acceleration of GMMs and k-means. In AISTATS, pages 124–132, 2018
  Dennis Forster and Jörg Lücke
  
• Neural simpletrons: Learning in the limit of few labels with directed generative networks. Neural Computation, (30):2113–2174, 2018
  Dennis Forster, Abdul-Saboor Sheikh, and Jörg Lücke
  (See online at https://doi.org/10.1162/neco_a_01100)
• Optimal neural inference of stimulus intensities. Scientific Reports, (8):10038, 2018
  Travis Monk, Cristina Savin, and Jörg Lücke
  (See online at https://doi.org/10.1038/s41598-018-28184-5)
• STRFs in primary auditory cortex emerge from masking-based statistics of natural sounds. PLOS Computational Biology, 15(1):e1006595, 2019
  Abdul-Saboor Sheikh, Nicol S. Harper, Jakob Drefs, Yosef Singer, Zhenwen Dai, Richard E. Turner, and Jörg Lücke
  (See online at https://doi.org/10.1371/journal.pcbi.1006595)