Building TRB on a 2022 Vintage Musl Gentoo

January 30th, 2022

I've been working on a new musl-based Gentoo build for my ThinkPad X1 Carbon (Gen 6) laptop. I recently popped in a new 2TB Samsung 970 EVO NVMe SSD since the original 512GB SSD was feeling a little cramped with the weight of Bitcoin's blockchain now approaching 400GB. Unlike my last Gentoo build on an FX-8350-based desktop, with this one I am documenting every step and pitfall along the way so that others new to TRB and V (and perhaps even Gentoo, or any flavor of Linux for that matter) can follow a set of instructions and have their own working vtron and TRB, on top of a reasonably sane Linux environment that is perfectly usable as a daily driver. As a prerequisite for publishing this guide I had to be sure that I could in fact build TRB on this system, otherwise the rest of the steps leading up to the TRB build would have been, for the most part, pointless. Well, I am now happy to report that (with only minimal patching) building TRB on a stock 2022 vintage Gentoo is indeed possible.

Building TRB with the System's Compiler and Libraries

To start, I grabbed all the patches up to and including jfw's system compiler patch. While I've had success building TRB via the 'Rotor' method on older Gentoo builds, namely asciilifeform's Dulap, my most recent attempt on a 2021 glibc-based Gentoo got stuck on this requirement for the m4 dependency. Additionally, jfw's system compiler is much faster, since there are a lot fewer steps, so I figured if I could get it to work it would be preferable.

Once I had all the patches and signatures I pressed the source and entered the new directory. From there I made my first attempt at make ONLINE=1.

Using the Right Version of Python

The first hiccup I encountered was with building the Boost dependency. It wasn't immediately apparent what the problem was but I remembered whaack had done a writeup of his experience using the system compiler build method so I checked his notes. Sure enough there was something in there about the Boost build failing, only he happened to also catch that it had something to do with Python. He was on 2.6 and the fix for him was to use 2.7. Out of the box my Gentoo came with Python 3.9.91 so I first installed 2.7 from Portage, along with virtualenv, and then set the virtual environment to target Python 2.7.


sudo emerge -av =dev-lang/python-2.7.18_p13
sudo emerge -av dev-python/virtualenv
virtualenv --python=python2.7 .env
source .env/bin/activate

With my Python virtual environment set I ran make again. This time Boost compiled successfully and the build continued until it hit the next snag with Berkley DB.

Patching Berkley DB to Build with Newer Versions of GCC

Compiling Berkley DB failed with an error that looked like:


../dist/../dbinc/atomic.h:179:19: error: definition of 'int __atomic_compare_exchange(db_atomic_t*, atomic_value_t, atomic_value_t)' ambiguates built-in declaration 'bool __atomic_compare_exchange(long unsigned int, volatile void*, void*, void*, int, int)'
  179 | static inline int __atomic_compare_exchange(
      |                   ^~~~~~~~~~~~~~~~~~~~~~~~~

I googled for some clues and found that the problem is fairly common, and the solution is straightforward. The issue is that some versions of GCC2 have their own __atomic_compare_exchange function that conflicts with that of BDB's. The solution is to rename BDB's to something else, so I added the patch and extra line to the build/Makefile and ran make again. This time there were no issues with BDB and I ran into the final issue preventing a successful TRB build.

A Small Adjustment to Boost's API

It's likely this won't even be an issue for you unless you've already installed a more recent version of Boost3. I got this error inside src/net.cpp:


../src/net.cpp:35:1: error: reference to 'array' is ambiguous
   35 | array vnThreadsRunning;
      | ^~~~~

Luckily this was also something that others had run into and tracked down to an API change in later versions of Boost. The fix is simply to update the offending line, like so:


-array vnThreadsRunning;
+boost::array vnThreadsRunning;

With this change in place the entire build finally succeeded and I was left with a working bitcoind inside of build/. I moved this to a more generic location on my machine, ran it, and now my new TRB is up and eating blocks.

Some Remaining Questions

There was one part of this process that didn't quite add up: if I had to go through the trouble of building the specified Boost 1.52.0 then why did the build at some point use my local 1.77.0 copy at /usr/lib? I decided to try the build again, this time removing all pointers to the Boost library in deps/Makefile and build/Makefile. This also worked for me4 and was the fastest I had ever seen TRB compile. I tested the resulting binary and it continued to eat blocks where the other left off just fine.

The two binaries were of different weights though, 31MB and 33MB. Running ldd showed some differences in their linked libraries:

Built with the specified Boost 1.52.0 present

$ ldd bitcoind
/lib/ld-musl-x86_64.so.1 (0x7ff899970000)
libstdc++.so.6 => /usr/lib/gcc/x86_64-gentoo-linux-musl/11.2.0/libstdc++.so.6 (0x7ff899344000)
libgcc_s.so.1 => /usr/lib/gcc/x86_64-gentoo-linux-musl/11.2.0/libgcc_s.so.1 (0x7ff89932b000)
libc.so => /lib/ld-musl-x86_64.so.1 (0x7ff899970000)

Built with only the local Boost 1.77.0

$ ldd bitcoind
/lib/ld-musl-x86_64.so.1 (0x7f43030db000)
libboost_system.so.1.77.0 => /usr/lib/libboost_system.so.1.77.0 (0x7f4302d0e000)
libboost_filesystem.so.1.77.0 => /usr/lib/libboost_filesystem.so.1.77.0 (0x7f4302cec000)
libboost_program_options.so.1.77.0 => /usr/lib/libboost_program_options.so.1.77.0 (0x7f4302c9c000)
libboost_thread.so.1.77.0 => /usr/lib/libboost_thread.so.1.77.0 (0x7f4302c7d000)
libstdc++.so.6 => /usr/lib/gcc/x86_64-gentoo-linux-musl/11.2.0/libstdc++.so.6 (0x7f4302a76000)
libgcc_s.so.1 => /usr/lib/gcc/x86_64-gentoo-linux-musl/11.2.0/libgcc_s.so.1 (0x7f4302a5d000)
libc.so => /lib/ld-musl-x86_64.so.1 (0x7f43030db000)
libboost_atomic.so.1.77.0 => /usr/lib/libboost_atomic.so.1.77.0 (0x7f4302a53000)

I don't understand why the two binaries have different library links when they appeared to use the same local copy of Boost at some point during the build,5 but I'm content for now with having found two new ways of buliding a working TRB on a 2022 Musl Gentoo.

The Unsigned Patch

I figured since I'd already done the work of patching the build to work on my heathen Gentoo configuration I'd share the complete set of changes in case anyone else wants to run TRB on a similar setup.

bitcoin_heathen_linux_build_fixes.vpatch

  1. Modern Gentoos now ship with Python 3, and eselect python no longer lets you select Python 2.7 as the system default Python. []
  2. I'm not sure why I only bumped into this now, considering the function was first added to GCC all the way back at version 4.7.0. If any GCC experts can write in and let me know what I'm missing I'd much appreciate it, I was using GCC 11.2.0 fwiw. []
  3. As I apparently did when I installed AbiWord. There are also a number of other packages that require Boost so it's possible you have it installed already. []
  4. And no longer required a specific version of Python. []
  5. At least that was my hunch, as indicated by them barfing on the same line in src/net.cpp. []
« New Logotoron Patches: Bug Fixes and Enhancements
Running MP-WP on a Modern Musl-Based Gentoo »

4 Comments

  1. Jacob Welsh says:

    Congrats on getting this far in the swamps.

    I had another couple patches out that may be of interest including one that resolves the python thing at the root (that it was touching python at all was a bug in my view). It should have pinged back to whaack's report too, if only *that* functionality weren't so thoroughly broken.

    > I'm not sure why I only bumped into this now, considering the function was first added to GCC all the way back at version 4.7.0.

    Yeah. Just from the context in the linked patch it looks like it was named deliberately to fill in the gap for *older* gccs where the builtin wasn't available. So I'm guessing it's some ifdef/autoconfiguration tangle that's failed for whatever reason on the verschlimmbessert system and that would be the level to look at, if it's worth looking at all. FWIW, in the build on Gales I do get a series of "warning: conflicting types for built-in function '__atomic_compare_exchange'" coming from dbinc/atomic.h:179.

    It's also mildly bothersome that an external Boost is getting mixed into things at all. Possibly it's getting headers from /usr/include and linking libraries from the shipped version, or vice versa - rather asking for trouble.

  2. billymg says:

    Thank you for the clarifications and for the links to the new patches. Cool stuff with the Boost build trimming, seems like a big win. I'll try them out as soon as I get a chance and publish my results.

    As I mentioned in the article, I'm working on a guide that even users new to linux can follow in order to build a working TRB machine. Any patches that make that process faster and/or easier would definitely be helpful so I appreciate the note.

  3. Jacob Welsh says:

    Cheers. Spending some time getting to know http://fixpoint.welshcomputing.com/2022/the-simplest-way-yet-to-fetch-bitcoin-code/ may be helpful for that guide of yours; at least in my view it's a game-changer. I suppose it would depend on your evaluation of the classical vtrons and their dependencies.

  4. [...] Building TRB on a 2022 Vintage Musl Gentoo [...]

Leave a Reply

*
*

You can use the following HTML tags in your comment: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>