billymg

Bitdash Crawler: Geolocation, Time Series Data Collection, and an IRC Bot

March 7th, 2022

The Bitdash Crawler has been running steadily since August of last year. The first release was minimal—only the basic crawling functionality, and only collecting data returned directly by the probed nodes. This patch adds additional data collection, as well as a new IRC interface for basic status reports.

Geolocation Data

When I first looked into adding this I was reluctant because I thought the only way was to connect to a third-party service via API keys. I didn't want the crawler to have to rely on any third-party service, as those come with the risk that they will change out from under you at any moment and without notice. Thankfully a user in #asciilifeform pointed me to Maxmind's excellent GeoLite2 geolocation database. I was almost surprised at how simple it was and that there was no catch. Yes, you have to provide an email and create an account with them, but after that they just give you their geolocation database¹, in CSV format that you can take wherever you like. They even provide clear documentation for creating the schema and importing the CSV data. Overall a very nice offering and I'd recommend it to anyone who needs to add this to their project.

Time Series Data

I'm working on a redesign of the www interface for the crawler² and it includes charts. Previously the crawler was only storing current snapshots of the data it collected, now it's also generating some aggregate statistics³, taken at set intervals, and storing them indefinitely. This job is handled separately from the main crawler script, via a set of SQL queries run from a cron job.

IRC Bot Interface

With the addition of geolocation and time series data collection the crawler can finally provide some reports its underlying library alone cannot. Soon this data will be available via the www interface but in the meantime I wanted to make it available in the chans via an IRC bot. The bot is based on asciilifeform's logotron bot, with which I've become familiar since using it to power the logger at logs.bitdash.io. It works roughly like this:


   billymg | !c help
crawlerbot | billymg: my valid commands are: src, uptime, help, net-summary, version, trb-status
   billymg | !c net-summary
crawlerbot | Bitcoin Network (IPv4 Nodes Active Within the Last 48 hours) Global: 8166; TRB-Compatible: 61; TRB: 13
crawlerbot | TRB-Compatible by Country: United States: 26; Canada: 4; Singapore: 4; Romania: 4; Russia: 3; France: 2; United Kingdom: 2; Italy: 1; Lithuania: 1; Norway: 1; Australia: 1; Germany: 1; Chile: 1; Belgium: 1; Spain: 1; Ukraine: 1; Netherlands: 1; Finland: 1; Sweden: 1; Switzerland: 1; Bulgaria: 1; Mexico: 1; South Africa: 1;
crawlerbot | TRB by Country: United States: 7; Canada: 1; Romania: 1; Singapore: 1; Lithuania: 1; France: 1; Norway: 1;

Patches and Signatures

I unfortunately had to regrind the genesis patch and the small bug fix patch that followed because there was a typo in the root directory's name. I'll leave the original two patches up for archival purposes but this patch is built on a new tree, with the genesis regrind also including the small fix from the original second patch.

bitdash_crawler_genesis_regrind.vpatch (sig)
bitdash_crawler_add_bot_geo_and_time_series.vpatch (sig)

Presumably a subset of the data included in their full commercial offering, but plenty good enough for my needs. [↩]
Which at the moment is nothing more than a few Sketch mockups. [↩]
Network breakdown by major user agent version, network breakdown by country, TRB breakdown by country, and TRB-compatible breakdown by country. All stats are collected both for recently active nodes (48hrs) and for recently active nodes returning at least one non-self peer (referred to as "participating"). [↩]

Posted in bitdash.io, crawler, patches | No Comments »

Straightening a Bent 1U Server Chassis

February 17th, 2022

I recently acquired a new (used) server which was damaged in shipping.¹ Luckily the damage wasn't anything serious, just a slightly bent frame.

In order to have any hope of fixing this I'd first have to remove the motherboard. As you can see, none of the internals appear damaged. It seems like the PCI riser's bracket may have provided some extra sturdiness to the location just at the edge of the motherboard, preventing the case from bending closer to the middle. In any case, buena suerte. Let's get started by first removing the plastic airflow cover and the PCI riser.

Now on to all of the fan and power cables. The fans were no problem but it was a bit tight around the mains, which required some extra finesse. The first thing that got in the way was the zip tie closest to the connectors. It made it difficult to get my fingers underneath the cables and from the other side it looked like it also might have been wrapped around some cables I did not want to disturb.

With a sharp knife I carefully sawed at it until it broke. I also removed one of the heat sinks, since that was getting in the way.² To help with some of the clips I used a flat head screwdriver to squeeze them open.

With all of the cables unplugged, all that was left was to unscrew and remove the motherboard. There were no surprises here and inspecting the motherboard upon removal further reassured me that it hadn't suffered any damage during transit.

Now with the case (mostly) empty I was ready to move it to the bodega³ for phase two. I scanned the walls and shelves until I found two vices that seemed perfect for the operation.⁴

I also noticed an old sheet which was being used as a rag, so I made some space on a workbench and laid it flat just to avoid adding any extra scratches to the chassis while working on it. I then attached the vices to either side and started gently tightening the one on the right side to straighten the frame.

As I was doing this I noticed the left side starting to arc up so I moved the left vice inward to prevent adding a new deformity while fixing the original.

I tightened the right side all the way but, as you may have noticed in the photo above, the flat surface isn't going to cut it. In order to fully straighten out the bend I'd need to add some leverage. I once again searched the bodega for something that would do the job. A piece of wood, some cardboard, or a folded piece of paper might do. Instead I found something even better, a small section of 4 AWG wire. I placed it right under the bend and proceeded to slowly tighten the right side again. I again noticed some unwanted bowing so I moved the right vice inward, closer to the wire, and continued tightening.

That was it. The case looked pretty straight to me now so I brought it back inside to take a closer look and reassemble the internals. As you can see from the before and after photos below, the operation seems to have been a success. Perhaps not perfect but hopefully good enough to slide into a rack somewhere.

Before

After

Here it is below with the motherboard back in place and the old thermal paste cleaned off of the CPUs.

And finally all put back together and loaded with 256GB of DDR3 ECC RAM.

I also loaded the SSDs (not visible in the photo) but unfortunately wasn't able to test anything past powering it on since I have no way to connect it to a monitor.⁵ The next step, after installing Dulap on it (or perhaps Pentacle), is to find a home for it here in Costa Rica. Oh, and I guess I'll also need a hardware RAID controller for it. If anyone has any recommendations I'd much appreciate it as this will be my first time buying one.

Getting things shipped to Costa Rica can be a pain. This item had to be smuggled in a suitcase, and although the impromptu importer went to great lengths to make sure it was packed properly, apparently the baggage handlers went to even greater lengths to ensure that that didn't matter. [↩]
It would've had to have been removed later anyway, as it was attached to the chassis. [↩]
Meaning in this case garage/workshop/storage area, not a convenience store in NYC. [↩]
The bodega in its current state is more like my personal junkyard than anything else. It needs to be thoroughly cleaned and organized, and it will be, just probably not this year. [↩]
I mistakenly ordered an HDMI to VGA cable, rather than the VGA to HDMI cable that I need. [↩]

Posted in photos | No Comments »

Running MP-WP on a Modern Musl-Based Gentoo

February 11th, 2022

A lot has changed since I last wrote a guide on installing and running MP-WP. For one, the previous article reads like pure cringe when I go back and look at it now. I was more concerned with exhibiting the right affectations than producing something truly useful. Whatever usefulness (if any) it served at the time, I can assure you, was purely accidental. I now have a much better grasp of what I'm doing, so it's easier to filter out noise and highlight parts of the process that deserve attention. There have also been changes to Gentoo—subtle differences in the current upstream versions of Apache, MySQL, and PHP that require small tweaks and patches. Lastly, the guide that served as my initial reference for setting up MP-WP back in 2018 has since been lost to bitrot. This guide addresses the shortcomings of my previous guide and aims to be the only guide you'll need for setting up MP-WP on Gentoo.

Collecting the Patches and Pressing with V

This will be the only handwavy portion of this guide.¹ First collect all the vpatches for MP-WP (along with all of the signatures) and press the tree using a working V. Now you should have a copy of the MP-WP source. Before you can use it you'll first have to install and configure some dependencies, so we'll leave the source alone for now and come back to it later.

Installing the Dependencies

MP-WP, derived from Wordpress 2.7, runs on top of Apache, MySQL, and PHP. So if you're on a fresh Gentoo install you'll need to first install and configure these before getting on to MP-WP.

MySQL 5.7

MP-WP is known to work with MySQL versions 5.5, 5.6, and 5.7. Since 5.7.36 was the version available upstream that's the one I selected. The process was mostly uneventful, except for a small incompatibility with musl² while building mysql-connector-c. The bug was already known and a user had submitted a patch to Oracle with the fix. I took the changes in his patch, added them to a copy of the 5.7.36 ebuild, and saved it in my local overlay. With the changes in place MySQL emerged without any other issues.

For your convenience I've uploaded a tarball of the patched mysql-connector-c ebuild: mysql-connector-c-8.0.27-r1.tgz.

Download the tarball and extract to wherever you have your local overlay.³ To make the ebuild available to portage, run the following from inside the new directory:


# from inside /usr/local/portage/dev-db/mysql-connector-c/
sudo ebuild mysql-connector-c-8.0.27-r1.ebuild digest

The ebuild should now be available to portage and you can emerge it by specifying the version, like so:


# first install the patched mysql-connector-c dependency (skip this if on glibc)
sudo emerge -av =dev-db/mysql-connector-c-8.0.27-r1

# now you can emerge mysql
sudo emerge -av =dev-db/mysql-5.7.36-r1

If you are on a glibc-based build then you didn't need to create the patched ebuild and you can just emerge MySQL 5.7 normally, it will automatically pull in mysql-connector-c.

PHP 5.6 and Apache 2.4

First let's grab PHP 5.6. MP-WP won't work with PHP versions > 5.6 (the minor version doesn't seem to matter) and the upstream Gentoo repo no longer includes any ebuilds for PHP 5.6 so you'll have to manually add one to your local overlay as we did in the MySQL step. If you look inside /var/db/repos/gentoo/dev-lang/php/ you'll see ebuilds for all the available versions. If a 5.6 version isn't among these then you'll have to add your own to /usr/local/portage. You can either go look for one buried in the packages commit history or use my copy: php-5.6.40-r7-ebuild.tgz.

Extract the tarball to /usr/local/portage/dev-lang/,⁴ then make the ebuild available to portage:


# from inside /usr/local/portage/dev-lang/php/
sudo ebuild php-5.6.40-r7.ebuild digest

Before installing you'll also have to enable the USE flags for apache2 and mysql so that PHP is built with support for these. I added mine to the global list in /etc/portage/make.conf.

You can now run:


sudo emerge -av =dev-lang/php-5.6.40-r7

It will likely complain about a missing required USE flag, gd. If this is the case simply add this to /etc/portage/package.use⁵:


# Either inside `/etc/portage/package.use` if you have it as a file or inside `/etc/portage/package.use/somefile` if you have it as a directory.
dev-lang/php gd

You can now emerge PHP again with the same command. This time it should build without any issues. If you set the apache2 USE flag as mentioned above then emerging PHP will also pull in Apache as a dependency, so after this step you should have all the dependencies you need to start configuring MP-WP.

Configuring Apache and MySQL

First a few basic knobs to turn in order to get the dependencies working properly with each other. Edit the file /etc/conf.d/apache2 and add "-D PHP" to the variable APACHE2_OPTS. Now Apache will load the PHP module when it starts. Next, configure MySQL by running the following command.⁶


sudo emerge -av --config =dev-db/mysql-5.7.36-r1

This step will create the root MySQL user and ask you to choose a password. Save the password because you'll need it to connect to MySQL for the first time and create your other user accounts.⁷ Before starting MySQL you'll likely have to set the SQL mode for compatibility with MP-WP.⁸ Edit /etc/mysql/my.cnf and add the following to the bottom of the file:


[mysqld]
sql-mode="REAL_AS_FLOAT,PIPES_AS_CONCAT,ANSI_QUOTES,IGNORE_SPACE"

Start mysql:


rc-service mysql start


mysql -u root -p

Verify that the sql_mode was set correctly


SELECT @@GLOBAL.sql_mode;

The output should match what you entered in /etc/mysql/my.cnf. If everything looks good you can create the MySQL user and database you will use for MP-WP:


-- change the placeholder values to your own, obviously
CREATE USER 'someusername'@'localhost' IDENTIFIED BY 'password';
CREATE DATABASE theblogdbname;
GRANT ALL ON theblogdbname.* TO 'someusername'@'localhost';

Edit mp-wp/wp-config.php and set the DB_NAME, DB_USER, and DB_PASSWORD variables to match your configuration. Also set the $table_prefix variable to whatever you like, e.g. "mpwp_". The tables autogenerated by MP-WP will now all be prefixed with this string, e.g. "mpwp_posts", "mpwp_users", etc.

Now that everything is configured the last step is just to point Apache to your new MP-WP direcory. Edit /etc/apache2/vhosts.d/default_vhost.include and change the lines DocumentRoot "/default/path" and <Directory "/default/path"> to point to wherever you have your blog code.

You can now start apache:


sudo apache2ctl start

Visit localhost in your browser, if everything is configured and working properly you should see the new blog setup screen. From here simply follow the instructions to create your admin account and start using MP-WP. If you encounter any errors double-check the steps above and make sure you have everything configured properly. If you're still stuck, leave a comment describing your issue in detail and perhaps I'll be able to help.

A future guide will walk through all of the steps involved in this but here I will assume you have a working vtron and know how to press vpatches. [↩]
If you're on a glibc rather than musl-based system you can skip ahead to emerging MySQL normally, it should work fine with the package in upstream. [↩]
Overlays are a concept in Gentoo's Portage. They allow you to "overlay" the main repository with additional repositories in order to make available additional packages—and versions of packages—that are not included in the default upstream repo. I have my local overlay at /usr/local/portage. The directory structure inside there matches the standard Portage repository structure, such as can be seen in /var/db/repos/gentoo. Packages are separated into category folders, within those are package folders, and within those are ebuilds and related files for available versions.

To create your own local overlay, simply create the new directory (e.g. mkdir /usr/local/portage) and designate it as your overlay in /etc/portage/make.conf. To do so, add this line inside make.conf: PORTDIR_OVERLAY="/usr/local/portage" and save the file. [↩]
Or wherever you have your local overlay. [↩]
In this case I didn't want the flag to be global, so I added it to package.use. This can be either a file or a directory. Read more in the wiki. [↩]
Note that the exact version of MySQL you installed may be different, just be consistent with what's on your machine. You can check which package you have installed by running eix -I mysql. [↩]
I like to use pass for these sorts of things. [↩]
If you're using MySQL 5.6 or earlier it's possible this step is unnecessary. In any case you'll know soon enough when you try to run MP-WP. If you see any MySQL errors in the setup flow it's likely due to some incompatibility with the SQL mode. [↩]

Posted in guides, mp-wp, v | 2 Comments »