Aussie AI

NAS Research Papers

  • Book Excerpt from "Generative AI in C++"
  • by David Spuler, Ph.D.

This is not the full list of papers, I add with reasonable certainty, given that one survey paper stated there have been over 1,000 papers written on NAS since 2021. If this is your chosen dissertation topic, better start writing that lit review section early!

Survey papers on NAS include:

  1. Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Xiaojiang Chen, Xin Wang, 2022, A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions, ACM Computing Surveys 54(4):76:1–76:34, https://arxiv.org/abs/2006.02903
  2. Hadjer Benmeziane, Kaoutar El Maghraoui, Hamza Ouarnoughi, Smail Niar, Martin Wistuba, Naigang Wang, 2021, Hardware-Aware Neural Architecture Search: Survey and Taxonomy. In: International Joint Conference on Artificial Intelligence (IJCAI), https://arxiv.org/abs/2101.09336
  3. Dilyara Baymurzina, Eugene Golikov, Mikhail Burtsev, 2022, A review of neural architecture search, Neurocomputing, Volume 474, 14 February 2022, Pages 82-93, https://www.sciencedirect.com/science/article/abs/pii/S0925231221018439
  4. Thomas Elsken, Jan Hendrik Metzen, Frank Hutter, 2019, Neural architecture search: a survey, The Journal of Machine Learning Research, Volume 20Issue 1, pp. 1997–2017, https://dl.acm.org/doi/10.5555/3322706.3361996, https://arxiv.org/abs/1808.05377
  5. Martin Wistuba, Ambrish Rawat, Tejaswini Pedapati, 2019, A Survey on Neural Architecture Search, https://arxiv.org/abs/1905.01392
  6. Shiqing Liu, Haoyu Zhang, Yaochu Jin, Oct 2022, A Survey on Computationally Efficient Neural Architecture Search, https://arxiv.org/abs/2206.01520
  7. Colin White, Mahmoud Safari, Rhea Sukthanker, Binxin Ru, Thomas Elsken, Arber Zela, Debadeepta Dey, Frank Hutter, Jan 2023, Neural Architecture Search: Insights from 1000 Papers, https://arxiv.org/abs/2301.08727
  8. Bernd Bischl, Martin Binder, Michel Lang, Tobias Pielok, Jakob Richter, Stefan Coors, Janek Thomas, Theresa Ullmann, Marc Becker, Anne-Laure Boulesteix, Difan Deng, Marius Lindauer, Nov 2021, Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges, https://arxiv.org/abs/2107.05847

General research papers on NAS:

  1. Odema, M., Rashid, N., Demirel, B. U., and Faruque, M. A. A. (2021). Lens: Layer distribution enabled neural architecture search in edge-cloud hierarchies, In 2021 58th ACM/IEEE Design Automation Conference (DAC), pages 403–408, https://arxiv.org/abs/2107.09309
  2. A. Wong, M. Famuori, M. J. Shafiee, F. Li, B. Chwyl, and J. Chung, 2019, YOLO nano: A highly compact you only look once convolutional neural network for object detection, arXiv:1910.01271. https://arxiv.org/abs/1910.01271
  3. David R So, Chen Liang, and Quoc V Le. 2019. The evolved transformer, arXiv preprint arXiv:1901.11117. https://arxiv.org/abs/1901.11117
  4. Mingxing Tan and Quoc V Le. 2019, Efficientnet: Rethinking model scaling for convolutional neural networks, arXiv preprint arXiv:1905.1, https://arxiv.org/abs/1905.11946, Code: https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet
  5. Guihong Li, Duc Hoang, Kartikeya Bhardwaj, Ming Lin, Zhangyang Wang, Radu Marculescu, July 2023, Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunities, https://arxiv.org/abs/2307.01998
  6. C Fu, 2023, Machine Learning Algorithm and System Co-design for Hardware Efficiency, Ph.D. thesis, Computer Science, University of California San Diego, https://escholarship.org/content/qt52q368p3/qt52q368p3.pdf

For more research on NAS, see also https://www.aussieai.com/research/nas.

 

• Next:

• Up: Table of Contents

• Buy: Generative AI in C++: Coding Transformers and LLMs

Generative AI in C++ The new AI programming book by Aussie AI co-founders:
  • AI coding in C++
  • Transformer engine speedups
  • LLM models
  • Phone and desktop AI
  • Code examples
  • Research citations

Get your copy from Amazon: Generative AI in C++