Autonomes und effizient skalierbares Deep Learning

• Antragsteller: Professor Dr. Jörg Lücke
• Fachliche Zuordnung: Bild- und Sprachverarbeitung, Computergraphik und Visualisierung, Human Computer Interaction, Ubiquitous und Wearable Computing
• Förderung: Förderung von 2014 bis 2020
• Projektkennung: Deutsche Forschungsgemeinschaft (DFG) - Projektnummer 260197604

• Erstellungsjahr: 2019

Zusammenfassung der Projektergebnisse

In summary, the project was not only successful in taking important steps towards more autonomous systems, that are able to learn from as little data as possible in a mathematically grounded fashion. But it also importantly contributed to spread light on a severe general issue in semi-supervised learning, that is the need to take validation data into account when comparing systems that learn on few labeled data. Furthermore, novel developments on efficient scaling enabled the developed NeSi networks to be applicable at realistic scales and with many parameters. Furthermore, the scalability methods applied gave rise to novel and more broadly applicable learning algorithms.

Projektbezogene Publikationen (Auswahl)

• Beyond manual tuning of hyperparameters. KI – Künstliche Intelligenz, 29(4):329–337, 2015
  Frank Hutter, Jörg Lücke, and Lars Schmidt-Thieme
  (Siehe online unter https://doi.org/10.1007/s13218-015-0381-0)
• Select-and-sample for spike-and-slab sparse coding. In Advances in Neural Information Processing Systems (NIPS), volume 29, pages 3927–3935, 2016
  Abdul-Saboor Sheikh and Jörg Lücke
  
• GP-select: Accelerating EM using adaptive subspace preselection. Neural Computation, 29(8): 2177–2202, 2017
  Jacquelyn A Shelton, Jan Gasthaus, Zhenwen Dai, Jörg Lücke, and Arthur Gretton
  (Siehe online unter https://doi.org/10.1162/neco_a_00982)
• Models of acetylcholine and dopamine signals differentially improve neural representations. Frontiers in Computational Neuroscience, 11:54, 2017
  Raphael Holca-Lamarre, Jörg Lücke, and Klaus Obermayer
  (Siehe online unter https://doi.org/10.3389/fncom.2017.00054)
• Truncated variational EM for semi-supervised Neural Simpletrons. In International Joint Conference on Neural Networks (IJCNN), pages 3769–3776, 2017
  Dennis Forster and Jörg Lücke
  (Siehe online unter https://doi.org/10.1109/IJCNN.2017.7966331)
• Can clustering scale sublinearly with its clusters? A variational EM acceleration of GMMs and k-means. In AISTATS, pages 124–132, 2018
  Dennis Forster and Jörg Lücke
  
• Neural simpletrons: Learning in the limit of few labels with directed generative networks. Neural Computation, (30):2113–2174, 2018
  Dennis Forster, Abdul-Saboor Sheikh, and Jörg Lücke
  (Siehe online unter https://doi.org/10.1162/neco_a_01100)
• Optimal neural inference of stimulus intensities. Scientific Reports, (8):10038, 2018
  Travis Monk, Cristina Savin, and Jörg Lücke
  (Siehe online unter https://doi.org/10.1038/s41598-018-28184-5)
• STRFs in primary auditory cortex emerge from masking-based statistics of natural sounds. PLOS Computational Biology, 15(1):e1006595, 2019
  Abdul-Saboor Sheikh, Nicol S. Harper, Jakob Drefs, Yosef Singer, Zhenwen Dai, Richard E. Turner, and Jörg Lücke
  (Siehe online unter https://doi.org/10.1371/journal.pcbi.1006595)