Hello. Now I am investigating PGO and PLO effects on different kinds of software - all my current results are available at https://github.com/zamazan4ik/awesome-pgo. According to these results, enabling PGO and PLO can help with achieving better overall performance in many cases. I think trying to optimize CPU usage for the Mageia packages would be a valuable change for the Mageia users. PGO is already a well-known compiler optimization technique. All currently known PGO effects on performance can be found at https://github.com/zamazan4ik/awesome-pgo#pgo-showcases . Several OS distros already enabled PGO for some packages like GCC, Rustc, Chromium, Firefox, and others (it depends on each OS distro, of course). I think we can try to expand PGO usage across Mageia packages. E.g. PGO can be enabled for Clang (https://svnweb.mageia.org/packages/updates/9/clang/current/SPECS/clang.spec?view=markup) since Clang already supports building with PGO + BOLT (https://github.com/llvm/llvm-project/blob/main/clang/cmake/caches/BOLT.cmake). Or GCC (https://svnweb.mageia.org/packages/updates/9/gcc/current/SPECS/gcc.spec?view=markup) since GCC also supports PGO build with `make profiledbootstrap`. Regarding Post Link Optimization (PLO), right now there are two main tools - LLVM BOLT and Google Propeller. According to the Facebook Research Paper (https://research.facebook.com/publications/bolt-a-practical-binary-optimizer-for-data-centers-and-beyond/), LLVM BOLT (https://github.com/llvm/llvm-project/blob/main/bolt/README.md) helps with achieving better performance for various packages like compilers and interpreters. I think it would be a good idea to enable LLVM BOLT for some packages to deliver faster binaries for users (since Propeller is less stable right now). Here I got some examples of how LLVM BOLT is already integrated into other projects: * Rustc: https://github.com/rust-lang/rust/pull/116352 * CPython: https://github.com/python/cpython/pull/95908 * Pyston: - https://github.com/pyston/pyston#building - https://github.com/pyston/pyston/blob/pyston_main/Makefile#L200 * Clang: https://github.com/llvm/llvm-project/blob/main/clang/cmake/caches/BOLT.cmake So at least for the projects above LLVM BOLT effects are tested and some preparations are already done in the upstream projects. In this case, it should be easier to enable BOLT for these packages. For some projects right now there is ongoing work on integrating LLVM BOLT into the build scripts: * Chromium: https://bugs.chromium.org/p/chromium/issues/detail?id=1163978 * Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1789087 - The same for Propeller (a LLVM BOLT alternative): https://bugzilla.mozilla.org/show_bug.cgi?id=1509314 * NodeJS: https://github.com/nodejs/node/issues/50379 * LDC: https://github.com/ldc-developers/ldc/issues/4228 * GCC: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=112492 More about LLVM BOLT performance results for other projects can be found in: * Rustc: - https://github.com/rust-lang/rust/pull/116352 - https://www.reddit.com/r/rust/comments/y4w2kr/llvm_used_by_rustc_is_now_optimized_with_bolt_on/ * CPython: https://github.com/python/cpython/pull/95908 * YDB: https://github.com/ydb-platform/ydb/issues/140 * Clang: - [Slides](https://llvm.org/devmtg/2022-11/slides/Lightning15-OptimizingClangWithBOLTUsingCMake.pdf) - [Results on building Clang](https://github.com/ptr1337/llvm-bolt-scripts/blob/master/results.md) - [Linaro results](https://android-review.linaro.org/plugins/gitiles/toolchain/llvm_android/+/f36c64eeddf531b7b1a144c40f61d6c9a78eee7a) - [on AMD 7950X3D](https://github.com/llvm/llvm-project/issues/65010#issuecomment-1701255347) * LDC: https://github.com/ldc-developers/ldc/issues/4228#issuecomment-1334499428 * NodeJS: https://aaupov.github.io/blog/2020/10/08/bolt-nodejs * Chromium: https://aaupov.github.io/blog/2022/11/12/bolt-chromium * MySQL, MongoDB, memcached, Verilator: https://people.ucsc.edu/~hlitz/papers/ocolos.pdf I don't create an issue per project (like "Enable BOLT for Clang", "Enable PGO for GCC", etc.) since I think first we need to discuss the approach. If we agree with enabling BOLT or expanding PGO usage for some packages, then we can create an additional issue (and use this issue as a meta issue).
Thank you for your interest, and thorough references to what you are talking about. This is all very technical, and detailed: almost per-package. The place to publicise this is the packager mailList, but this bug can be used to record the discussion. Small point: > Regarding Post Link Optimization (PLO), right now there are two main tools > - LLVM BOLT and Google Propeller We should be wary of using anything Google if we can help it... Pragmatically, we have more than enough on our hands to get involved with the refinements you propose. You seem up to dealing with it. Could you join the packagers to manage it? https://wiki.mageia.org/en/Becoming_a_Mageia_Packager Assigning to packagers generally, and suggest using this Bug 32511 for discussion in addition to your own dev mailList.
Assignee: bugsquad => pkg-bugsSummary: Increase using Profile-Guided Optimization (PGO) and Post Link Optimization (PLO) => Improving software performance using Profile-Guided Optimization (PGO) and Post Link Optimization (PLO)
See Also: (none) => https://bugs.mageia.org/show_bug.cgi?id=32519
> We should be wary of using anything Google if we can help it... Agree. LLVM BOLT now is a better option: is already an LLVM part, and works better (in my experience), originally from Facebook (worse or better - idk). > You seem up to dealing with it. Could you join the packagers to manage it? Hmm, it sounds like a good option to track it! I will try.