Bug fix to preserve encoded html characters in post body between edits

November 2nd, 2019

A bug was introduced in the previous mp-wp patch that caused encoded ("escaped") html characters to be converted back to their symbols after being loaded into the post editor and re-saved.

#ossasepia 2019-10-31 18:33 whaack: lol til after hitting 'save draft' on a mpwp post with html characters escaped, the escaped characters are evaluated to their html characters, thus hitting 'save draft' twice in succession can alter the content of a post.

It was a bug I had run into myself, and apparently others as well. Diana Coman encouraged further investigation and helped to isolate the problem.

#ossasepia 2019-11-01 20:26 billymg: diana_coman: steps to reproduce: 1) create a new post, include some encoded html character, e.g. &lt; 2) save post, 3) reload post edit page, 4) the &lt; you previously saved will render as a < in the editor text area, so a subsequent save inserts the < into the db, not the &lt; you had before
#ossasepia 2019-11-01 20:26 diana_coman: billymg: did that and...nope?


#ossasepia 2019-11-01 20:32 diana_coman: billymg: I start suspecting it's something you broke with the latest /trimming patches really; iirc on ossasepia I have the original mpwp, ie not pressed to last vpatch

And indeed it was!

#trilema 2019-11-01 20:49 billymg: yeah, when originally ripping it out, my brain parsed a "if (! $richedit ) ..." as "if ( $richedit ) ..." and so removed the line

So let's see exactly the lines of the patch that caused this bug and also what may have caused my original confusion.

diff -uNr a/mp-wp/wp-includes/formatting.php b/mp-wp/wp-includes/formatting.php
--- a/mp-wp/wp-includes/formatting.php 10526d3d84bbe766511af25287f01bbe1125a06d18babb93e1b7c33488c1f1df15ea3e706848a67d36c30edac25d3a4dd360bc563ac21093f6bfe0183f9c4388
+++ b/mp-wp/wp-includes/formatting.php 96c9b9b0b8da6f255e10fd47fd4053958417127524a5b8be961e126cce7fde0476bd5c2b20c4580517a75b01e9751636c8dc0c6f250ab8f94dbbe8097057cf67
@@ -775,20 +775,16 @@
  * Acts on text which is about to be edited.
- * Unless $richedit is set, it is simply a holder for the 'format_to_edit'
- * filter. If $richedit is set true htmlspecialchars() will be run on the
- * content, converting special characters to HTMl entities.
+ * Holder for the 'format_to_edit'
+ * filter.
  * @since 0.71
  * @param string $content The text about to be edited.
- * @param bool $richedit Whether or not the $content should pass through htmlspecialchars(). Default false.
  * @return string The text after the filter (and possibly htmlspecialchars()) has been run.
-function format_to_edit($content, $richedit = false) {
+function format_to_edit($content) {
 	$content = apply_filters('format_to_edit', $content);
-	if (! $richedit )
-		$content = htmlspecialchars($content);
 	return $content;

Notice the contradiction in the comment:

If $richedit is set true htmlspecialchars() will be run on the content, converting special characters to HTMl entities.

And the code it describes:

if (! $richedit )
	$content = htmlspecialchars($content);

Which does precisely the opposite of what the comment describes. Now, the introduction of the bug was still 100% my fault, and I honestly can't say if I even bothered reading the comment at the time. Perhaps it wasn't the comment that misled me but the "!" wedged in there, which, after reading so many lines after however many days, simply slipped past me. Either way, having a test in place would've caught the bug before it went into production and so along with a patch to fix the bug I also added a test to assert the correct behavior.

MP-WP Patch and Signature


MP-WP-Tests Patch and Signature



Additional tests for mp-wp, now according to spec

September 2nd, 2019

After the first round of tests and some discussion in the logs, there is now a spec in place for a minimum viable blogging platform. The tests included in this patch evaluate against a portion of the requirements in the spec, with remaining requirements to be covered in future patches.

Together with the first round of tests, mp-wp-tests now provides coverage for most1 of the Write section of the spec.

The full tree can now be found here: MP-WP-Tests VTree.

Patch and Signature



  1. The tests to do not yet cover: multi-file upload, programmable tag presets, reviewable/diffable list of drafts, and time-deferred publishing []

MP-WP Automated Testing Proposal and VPatch

August 9th, 2019

The following is presented for the consideration of the lords and ladies of tmsr.

Automated End-To-End Testing

Since mp-wp is a web application consisting of a frontend and a backend, my proposal is to handle testing via end-to-end tests run in the browser itself. This will unfortunately require the operator to run the tests on a toilet box as the browser automation tool, Selenium WebDriver runs on Java and will not work with earlier vintages of Firefox (I initially tested with version 54 of Firefox and found myself having to install version 60 to get anything working). However, I believe this to be an acceptable temporary trade-off for the following reasons:

  • MP-WP is a gnarly mix of PHP, JavaScript, HTML, and CSS code. Unit tests can assert correctness of individual PHP and Javascript functions but one careless snip in a CSS file could result in a broken UI that no unit test would catch. End-to-end tests will not only insure against breakage from HTML/CSS changes, but will also allow for test coverage of critical code paths without first fully unwinding the codebase and fitting it into one's head.
  • These e2e tests are intended to be temporary, and only remain while mp-wp sweats away the pounds. They can be abandoned completely once the codebase is in its desired minimal form.
  • Only the tests need to be run on a polluted box, the requirements for running mp-wp are not affected.

I've put together a genesis patch that contains a small POC spec for the basic posting functionality in mp-wp. The spec is written as:

const { LoginPage, DashboardPage, PostsPage, EditPostPage } = require('../pages');

describe('Posts', () => {
  beforeAll(() => {
    const loginPage = new LoginPage();

  afterAll(() => {
    const dashboardPage = new DashboardPage();

  it('Can navigate to and display the Posts page', () => {
    const dashboardPage = new DashboardPage();
    dashboardPage.followNavLinkTo('Posts', 'Edit Posts');

    const postsPage = new PostsPage();


  it('Can create and publish a post', () => {
    const dashboardPage = new DashboardPage();
    dashboardPage.followNavSubMenuLinkTo('Posts', 'Add New', 'Add New Post');

    const postTitle = 'Bitcoin Declaration Of Sovereignty';
    const postBody = '<p>When in the course of human events, it becomes necessary for one person to dissolve the political bands which have connected them with the human herd, and to assume among the powers of the world the separate and equal station to which the laws of nature entitle them, a modicum of respect to their own intelligence requires that they should declare the causes which impel them to the separation.</p>';

    const editPostPage = new EditPostPage();
    editPostPage.editAndPublishPost({ postTitle, postBody });

    expect(editPostPage.page.$('#message*=Post published').isExisting()).toBe(true);



Run the tests from the root of the project with:


This will execute the runner which will look for test files in tests/specs/ and run any it finds. This will look something like this, along with output in your console on the status of the test.

After the initial setup, writing the test didn't take very long. If this seems like a worthwhile enough approach I'll follow up with another post outlining the full set of tests to be written. After receiving input/feedback on the list, I'll proceed to write and publish the remaining tests as additional patches on this tree.

Patch and Signature


Note: After pressing you will need to run npm install from the project directory in order to pull down just under 100mb of depshits in order for this to work. Comments welcome here or in #trilema.


The mp-wp weightloss program

March 30th, 2019

This is the first patch in a series of slimming patches for mp-wp. The biggest savings comes from the removal of TinyMCE, a JavaScript library for rendering "WYSIWYG"1 text editing boxes for the types of users who might fall and hurt themselves if, god forbid, they had to write some HTML tags by hand.

Additionally, all importers were removed with the exception of the importer that handles files generated by the native WordPress exporter2. Other items removed include the Google Gears plugin3, as well as any "auto-upgrade" "features" that any self-respecting citizen of the republic would neither need nor want to use.

This patch was first published and announced in #trilema last week but has since been reground to fit on top of hanbot's latest. Overall this patch proved to be an effective first step in mp-wp's healthy new lifestyle, resulting in a 14% reduction in total LOC and an 18% reduction in size-on-disk (21% when comparing gzipped directories).

Before After Reduction
Lines of Code 162,594 139,205 14.4%
Bytes on Disk 6,053,039 4,962,628 18%

Patch and Signature



  1. The acronym omits what you really get, which is typically HTML tag spew in the generated output. []
  2. This may be worth removing later but I chose to leave it in for now so that one can at least export and re-import their existing mp-wp blog—and to keep as a reference in case someone eventually decides to re-write this functionality. []
  3. Some local caching scheme that they in any case gave up on in 2011. []

Minor mp-wp patch updates

January 13th, 2019

A few small formatting issues came through in my first two mp-wp patches. Hanbot caught one in the .svg file extension patch. The other, in the footnotes and text-selection patch, I noticed myself after deploying the changes on this blog. Nothing was broken, just some stray text showing up in the footer of the rendered page that should have been parsed as a comment.

Current Patches

Get the latest patches for mp-wp here

Old Versions

For auditing purposes only. The patches and signatures have been updated in place on this blog, so the previous posts describing them point to the new versions. For reference only, copies of the old patches and signatures can be found here:



MP-WP Vpatch: Update internal image references to point to .svg extensions

January 7th, 2019

There was a small, albeit widespread, bug in the original mp-wp genesis vpatch that resulted in broken image references throughout the site (admin panel as well as blog themes). This vpatch simply updates any and all internal image references to point to the correct .svg file extension.

Here is the vpatch, signature, and my key:


Pressing mp-wp from this patch

If you don't already have a copy of the previous patches you can grab the files you need from this blog. I added the new mirror of the mp-wp vtree for convenience and redundancy, and because it was suggested to me in #trilema.

Import both hanbot's and my key to your gpg keyring and construct your working directory to contain the following:


If this looks right, you can now press mp-wp into the v_press directory (or whatever you want to call it):

./v.pl p v v_press mp-wp_update-image-references-to-svg.vpatch

Result of pressing from this patch vs. previous patches in the vtree



Additional work

It's been quite a pleasure thus far working on mp-wp using v. I have a few ideas for future mp-wp work, and I know there are others floating around, but outlining them will have to come in a later post, perhaps after some discussion.


MP-WP Vpatch: Add footnotes plugin and text-selection javascript

December 31st, 2018

As my first somewhat official task, I set out to create a vpatch for mp-wp to add the footnotes1 and shareable text-selection2 features seen on most republican blogs. After some false starts I realized I would first need to set up a proper local dev environment for working with vtools. Luckily I had some time available during the holidays for completing this particular prerequisite, after which I was able to create a working vpatch for mp-wp.

Without further ado, the vpatch, signature, and my key:



In order to press this patch you will need a keccak-powered v.pl. Diana Coman was kind and patient enough to help me with this and pointed me to her post on the subject. If you are starting out with V for the first time I highly recommend grabbing the starter pack from that post. It works out of the box and will save you a lot of headaches.

Pressing a fresh copy of mp-wp from this vpatch gets you everything included in the genesis vpatch plus:


As well as a small modification to wp-content/themes/default/single.php and wp-content/themes/default/page.php to wrap post and page content in a unique ID that the selection-magic javascript can reference.

Setup of the footnotes feature followed the directions from hanbot. After setting up mp-wp you'll just need to 'activate' the footnotes plugin in the Plugins section of your wordpress admin panel.

Next task

My next official task, from hanbot this time, is to put together a vpatch to fix a small image reference bug in the genesis vpatch. I've started working on this locally and after sufficient testing will write up another blog post with the new patch.


  1. This footnote was generated automatically by enclosing the footnote text in (( ... )) after the word 'footnotes' above. []
  2. Go ahead and try this link. []

The complete noob's guide to setting up mp-wp on a pizarro rock chip server

October 22nd, 2018

This guide outlines the steps required to go from not being a person to having your very own, uncensorable, blogging platform. Some of the prerequisite steps will be glossed over, the main focus will be on installing and running mp-wp on a Pizarro Rockchip server.

I. Get in the WoT

Like everything in the republic, Pizarro is wot-powered. This isn't Amazon EC2, and you won't be able to use your credit card(s) for anything, of any value, anywhere in the republic (you are, however, free to use them for other things, so knock yourself out).

Getting in the WoT is simple, any motivated noob can do it. 1) Create a PGP key for yourself (if you don't already have one), 2) register it with @deedbot in #trilema1, 3) get someone to give you your first rating.

II. Acquire PizarroISP hosting in #pizarro

Now that you're a person you will be able to conduct commerce with other civilized people (as opposed to heathens and other assorted subhumans). First, head on over to http://pizarroisp.net and peruse their offerings. I opted for a Rockchip mini-server at the quarterly rate (a steal if you ask me), but you'll have to decide on your own.

All Pizarro servers run a sanitary build of Gentoo, of which my favorite feature has to be the included asciilifeform crapolade™® mask file2. As someone new to Gentoo, having a pre-configured ban list that I could read and work with myself was a godsend.

Once you've read through the basics you can introduce yourself in #pizarro to ask any further questions or to complete your purchase. And once that's done, you can login to your shiny new box and start installing goodies.

III. Install the prerequisites for mp-wp (apache, php, mysql)

I was able to follow esthlos' guide almost verbatim so, please, start there. Steps 2.2 and 2.3 were the only ones that didn't "just work" in my environment. After completing 2.1, here is what worked for me:

emerge -av apache
emerge -av mysql

Easy enough, but notice the php omission? Apparently the shitgnomes in charge of the upstream Gentoo repos thought it would be cute to deprecate all php-5.6 versions with the exception of php-5.6.38. I updated the line in my mask file to:


Now let's try emerging again:

emerge -av =dev-lang/php-5.6.38

This resulted in some spew about dependencies that couldn't be merged because of mask configuration. Feeling stuck and afraid I may have to install php from source, outside of Portage, I described my situation in #trilema and learned about the --nodeps option.

emerge -av --nodeps =dev-lang/php-5.6.38

Almost. The install at least proceeded enough for me to discover which dependencies are actually required—in that, the actual compile step would fail in their absence. Turns out there were only two, which were uncovered one-by-one via compile failures. Let's install them:

emerge -av --nodeps dev-libs/oniguruma
emerge -av --nodeps dev-libs/libmcrypt

The first went fine, the second whined about being "masked by EAPI 7" and asked "would you like some new Portage?". Well, no, I wouldn't, but I guess so long as is doesn't violate any of my current USE or mask settings... And this where it started to become obvious why versions of anything are "deprecated" in the first place.

You see, php-5.6.38 requires libmcrypt, the only version of libmcrypt available upstream "requires" NewPortage with EAPI 7, which, in turn, requires gpg2. Standard wrecker playbook I suppose, let's see if we can get around it.

Once the concept of custom ebuilds was brought to my attention I started reading to see if this could solve my problem. The process seemed simple enough so I grabbed the current libmcrypt-2.5.8-r4.ebuild from upstream and modified the one known-to-be-offending line:

- EAPI=7
+ EAPI=6

To use a custom ebuild, copy the necessary files to /usr/local/portage/CATEGORY/PACKAGE_NAME So in this case:


And run the ebuild command:

ebuild /usr/local/portage/dev-libs/libmcrypt/libmcrypt-2.5.8-r4.ebuild manifest clean merge

Finally! Turns out that in this case "requires EAPI 7" means "requires the version string 7".

Now to install php with mysql support, first unmask mysql in /etc/portage/profile/use.mask:

# Unmask mysql

And then emerge php:

emerge -av =dev-lang/php-5.6.38

It took me a while but hopefully my stumblings outlined here make it easier for you.

IV. Obtain a working implementation of V and the mp-wp genesis, press and configure

V is tricky and I still haven't wrapped my head around all of it, but these steps worked for getting a working copy of mp-wp up and running (if they didn't, you wouldn't be reading this).

First, grab a working implementation of V. For this I turned to the build instructions for the Bitcoin reference client and downloaded the version mentioned there:

curl -O http://thebitcoin.foundation/v/V-20180222.tar.gz
curl -O http://thebitcoin.foundation/v/V-20180222.tar.gz.mod6.sig

Verify and extract the tarball and you'll see v.pl (the V implementation we'll be using) along with some readme files and signatures. V also depends on a certain directory structure so let's set that up:

mkdir patches .wot .seals

Now you can grab the mp-wp genesis vpatch from hanbot's site. Download both the vpatch and the signature. Move the vpatch to the patches directory you just created and move the signature file to .seals. Since V is built around the concept of verifying the authorship of any and all code in a given tree, you will also need to gather the public keys of any authors involved. In this case, hanbot. Ask @deedbot in #trilema for a copy of her key:

!!key hanbot

Import this key into your gpg keyring and add the pubkey file to the .wot directory with the user's name as the filename. In this case hanbot.asc. Your working directory should now have at least:


If this looks right, you can now press mp-wp:

./v.pl p v mp-wp mp-wp_genesis.vpatch

That's it! Now move the generated files to your server's site directory (/www/blog/ if you were following esthlos' guide), fill in the config, and load in your browser. I won't get into those details here but you can again reference esthlos' guide or Wordpress documentation.


  1. On IRC. If you are a noob here as well (as I was), you will have to do some further reading on proper IRC etiquette. At a bare minimum I suggest configuring a bouncer, such as ZNC, to avoid polluting the channel with join/part spam. This type of spam is frowned upon in civilized IRC channels.

    Furthermore, if you are just now discovering #trilema for the first time, it is advised that you read at least 6 months worth of logs before attempting to participate in any meaningful way.


  2. If you're new to Gentoo you'll want to read up on the basics of Portage, the native package management system. Unlike package managers in other Linux distributions, Portage deals with source rather than binaries. When you install something with Portage you are grabbing the source code, configuring build options, and compiling right there on your own machine. This of course provides for much better system hygiene but also requires knowing a lot more about what the hell it is you're doing. []