AI Made A Lot Of "Hideous" Code But Found Major Bottlenecks For Faster Linux Compilation

An ARM Linux engineer leveraged AI/LLMs to help uncover some improvements to "significant speed up" the Linux kernel build times. In the process the LLM used generated "a lot of code, much of it hideous" but in the end was used to find some very significant time savings for faster kernel builds via more task parallelization in different areas of the kernel build process. ARM engineer and longtime Linux developer Lorenzo Stoakes posted the set of 23 patches today to "significantly speed up kernel builds." Fundamentally it's to address the significant time spent stuck in single-threaded bottlenecks throughout the kernel build process. In the end he found a kernel build with all modules enabled was around 36% faster, incremental kernel builds up to 70% faster, and noop builds up to ~90% faster. Lorenzo did note that an unnamed large language model (LLM) was used for uncovering these bottlenecks and addressing them, to which he then audited and evaluated the code personally. Lorenzo wrote: "An LLM was used to first determine where the bottlenecks were then to figure out how to improve them. It generated a lot of code, much of it hideous. I extensively audited and rewrote a lot of it, and heavily edited commit messages, the cover letter and comments.x The LLM has also orchestrated build runs, testing, debugging and analysis. I have manually checked for correctness in both build and running kernels generated with this series applied. Performance improvements were also verified manually. Since an LLM was used extensively, each commit carries an Assisted-by tag." The patches work to parallelize a lot of single-threaded tasks and improving the efficiency of the build process. Areas such as Kbuild, kallsyms, modpost, objtool, mksysmap, and the Rust build system were all affected by this work. In the end the build time improvements are very significant across a spectrum of hardware and different kernel configurations: More benchmark results and details can be found via the patch series. Once reviewed hopefully this work will make it to the mainline Linux kernel for sharply speeding up kernel build times. Linux kernel build times for reference across different CPUs can be found via this benchmark page for those interested in some current general data.