Bazel Build System Support for LLVM

  • (disclaimer: I'm one of the authors of this)

    A bunch of comments seem to be comparing and contrasting CMake+Ninja vs. Bazel. Just my two cents, but I think that's missing the real point of this work.

    It isn't about whether Bazel is a better build system for LLVM. At least, that isn't my motivation.

    There are users of LLVM's libraries that use Bazel. Whether for good or bad reasons, it is extremely useful to enable them to use LLVM's libraries with a fully native Bazel build. That is my motivation: enabling the users of LLVM libraries that need to use Bazel for whatever reason to have the best possible experience.

  • Ran some benchmarking with cmake+ninja vs these bazel BUILD files on a soyoustart GAME-2. Build from scratch...

    - cmake: ~2900 files @ 28:21.07 total (https://snapshot.raintank.io/dashboard/snapshot/ijkptI1dlteB...)

    - bazel: ~5k targets @ 23:55.13 total (https://snapshot.raintank.io/dashboard/snapshot/TbOP6L814R3q...)

    If anyone can recommend a file to modify I'd also be happy to test out incremental build times. Anyway, super glad to see google externalizing some of this stuff! I also have the metrics from this prometheus instance saved so if anyone wants to see any other metrics from node_exporter let me know!

    fwiw bazel "felt" slower because the number of targets is an estimate and that kept going up while cmake was constant progress to "finish". I'd love to see how the bazel time looks with remote build execution enabled to see how much faster that goes.

  • The world needs a Bazel package management system so the Bazel community can collaborate on projects like this. Buckaroo [0] is the closest I have seen but it’s Buck focused and they have not rolled out Bazel support.

    I think ultimately it needs a company like npm that can upstream the needed changes to Bazel itself so that it supports these workflows better. With the right focus you could hit cargo levels of ease of use, but for large multi-language projects.

    0 - https://github.com/LoopPerfect/buckaroo

  • While Bazel design seems not bad, could they have used a build system that doesn't require Java? It's especially important for bootstrapping. CMake requires only C++ at least.

  • LLVM has the most complicated CMake setup I’ve ever seen. I’m sure there’s a reason for that, but I always loathe hacking around in the source for that reason. Finding which cpp file belongs to which CMake file is an annoying and often frustrating task. Their CMake scripts also make some assumptions on Windows, making it impossible to build LLVM with LLVM without editing CMake files (at least, last time I checked). To me, the whole value of CMake is to allow you to swap in different toolchains with ease.

    So I wish someone smarter than me would go clean up the current CMake setup instead of using another build system altogether. But I guess any improvement to build usability is a good thing.

  • I was pleasantly surprised by Bazel. It seems to solve pretty much every concern I had, with not much downside. The biggest pain point was not being able to easily put the massive build files on an external hard drive, but you can do it with —disk_cache and —output_base. Unfortunately lots of scripts don’t expose a direct interface to the bazel command, making it necessary to modify those.

    Still, nice build system.

  • I have never seriously used bazel. Just looking at it, I found the the thought of manually mirroring all #includes to the build configuration in a big project pretty aggravating.

    Is that not really an issue in practice? Do people automate that away?

  • Would this allow people to host out-of-tree LLVM-based project easier? Last time I checked (shamefully, probably 2 or 3 years ago), LLVM still heavily prefer in-tree projects (creating a sub-directory and CMakeList.txt file) for source-code level distribution. If I can `deps = ["@llvm-project//:xxx"]` to start write new passes for llvm bitcode, it would be very exciting.

  • Byild systems are not equivalent! https://www.microsoft.com/en-us/research/publication/build-s... . Also yes, excel is akin to a build system.

  • undefined

  • no no no no no.

    Bazel has a some nice ideas, but has shat in my cornflakes too many times. It's really unusable for anything that isn't an all-inclusive monorepo.

    It's funny because we with Nixpkgs want to be all-inclusive too, except we don't think that requires vendoring the world / perfect synchronicity.