Compilers are the workhorse that bridge the gap between human-readable and machine-executable code. The diversity of modern programs, along with the advent of new and complex hardware architectures, has strained the capabilities of current compilers, making development and maintenance of automatic program optimizations in compilers exceedingly challenging. In spite of this, modern compiler optimizations are still hand-crafted using technology that existed decades ago and usually make optimization decisions considering an abstract machine model. It is high time that we modernize our compiler toolchains using more automated decision procedures to make better optimization decisions while reducing the expertise required to build and maintain compiler optimizations.
In this talk, I will show how we can leverage the changes in the computing environment to modernize compiler optimizations, using auto-vectorization (automatic conversion of scalar code into vector code) as an example.
First, I will demonstrate how we can take advantage of modern solvers and computing platforms to perform vectorization. Modern compilers perform vectorization using hand-crafted algorithms, which typically only find local solutions under linear performance models. I present goSLP, which uses integer linear programming to find a globally optimal instruction packing strategy to achieve superior vectorization performance.
Next, I will discuss how to modernize the construction of compiler optimizations by automatically learning the optimization algorithm. I present Vemal, the first end-to-end learned vectorizer which eliminates the need for hand-writing an algorithm. The key is to formulate the optimization problem as a sequential decision-making process in which all steps guarantee correctness of the resultant generated code. Not only does Vemal reduce the need for expert design and heuristics, but also it outperforms hand-crafted algorithms, reducing developer effort while increasing performance.
Finally, I will show how we can use data to learn better non-linear performance models, rather than the complex and incorrect hand-crafted models designed by experts, to enhance the decision procedure used in Vemal. I present Ithemal, the first learned cost model for predicting throughput of x86 code. Ithemal more than halves the error-rate of complex analytical models such as Intel’s IACA.
Both Vemal and Ithemal achieve state-of-the-art results and pave the way towards developing more automated and modern compiler optimizations with minimal human burden.
- Thesis Committee - Professors Saman Amarasinghe, Armando Solar-Lezama, Michael Carbin
To attend this defense, please contact the doctoral candidate, charithm at mit dot edu, for the details.