After some brief discussion in #ossasepia, which itself was prompted by a suggestion from MP on Diana's blog, I decided it would be worth compiling the thoughts into a proposal and including a few wireframes for illustration.
There are actually two proposals, the first is for the ability to embed vpatch code snippets into an mp-wp article by way of some custom markup, e.g. [[ code goes here ]]. The second proposal is for providing some level of vpatch management from within mp-wp, to facilitate the concept of a code shelf as well as a better format for full-length vpatch1 reviews. I'll discuss the former proposal first because it makes sense as a standalone whereas the second proposal requires work from the first. So from an implementation and release standpoint it makes sense to support embeddable snippets before full-length vpatch management.
Embeddable VPatch Snippets
 |
 |
This feature would:
- Allow users to embed vpatch content within an article via some new markup, e.g.
[[]], similar to(())for footnotes - Display patches with diff syntax highlighting2 and line numbers
- Allow linking to anchored line numbers. Clicking a line number navigates to its anchor, e.g. clicking line '22' appends '#L22' to the URL. useful for sharing in the logs or referencing in comments on the article
This is pretty straightforward. It would only require a new mp-wp plugin to parse the new markup and minor updates to a theme to support the diff formatting. This feature would work great for most quick vpatch reviews, and could even be used for larger vpatch reviews for the time being.
The only thing I could see making this feature more involved would be the notion of inline comments (wireframe) in an embedded vpatch. If we wanted to do that we might have to add a new comment type (to render it separately from the other post comments) and field (to indicate the line a comment is referencing). I do think this would be useful for full-length vpatch reviews but it's also something that can be added in a phase two of this feature.
Code Shelf, or MP-WP as a VPatch Repository
This feature is less clear in terms of scope and implementation. It is being proposed here to solicit feedback to help determine its viability and usefulness, as well as to shape the requirements. The name and inspiration comes from Diana Coman's excellent overview/map of the current V landscape.
The basic idea is to add a set of capabilities to mp-wp to make it work seamlessly as a vpatch repository. This might roughly include:
- The ability to upload vpatches and signatures within the mp-wp admin UI
- Automated storage and organization of uploaded vpatches, plus the ability to manage these
- Portability of uploaded vpatches (ability to embed in an article or include in one's Code Shelf)
- From within the Code Shelf: links back to the articles that introduced the patches, links to download VTree tarballs
Here's an example of what a Code Shelf might look like:
Having this feature could be very cool, and standardizing this could allow for aggregation of vpatches into a central (or federated) repository tool that pulls from a specified list of code shelves—something like http://btcbase.org/patches but auto-generated, and with links back to introductory blog posts.
Again, this second feature is still a little fuzzy in my head, and I haven't thought too much on how it would be implemented, so I'm sharing this mainly to get others thinking about it and collect feedback from anyone who is interested. There are also still a number of other items in the mp-wp queue3 so it might be worth getting an updated priority ranking as well.
- This is only intuition for now, but I currently think it would be easier to deal with full vpatches if they were their own entity, rather than text inside an article. For example, they could be stored in a new mysql table, which would allow for querying and referencing without having to parse article text.
It also seems like it would be a more pleasant experience when editing an article to update a reference to a vpatch rather than finding the opening '[[' and then scrolling however far down until you find the ']]' and making sure you paste exactly between them (for full patches of decent size, if that's what you wanted to include). [↩] - In theory this could be expanded later to include support for other syntaxes besides UNIX diff files. [↩]
- Completion of the test suite, slimming patches, a proper theme or two, the addition of the new server-side select mechanism as a standard feature (and removal of the old mechanism), and bulk upload support. [↩]
I would start with the minimal plugin for embeddable vpatch snippets, it looks quite good to me as you illustrate it there. I'd give it then some time in use to see if there really is a need for more since I can't quite tell upfront if it's even worth it to have inline and all of that, especially given the additional complexity it seems to add.
I quite like that illustration of "what a Code Shelf might look like". For this like for the above though, I'd start with a minimum too, basically the core functionality first and then - if and only if it turns out it makes sense - adding to it in a rather conservative way. From my current point of view, the core functionality for this part would be the update of the code shelf page to reflect current vpatches + sigs + links to introducing articles. Potential trouble that I see though is how do you order the vpatches if you aim to generate the page automatically. And I'm not that sure I like the idea of forcing click-interface on users just to have this otherwise.
> Allow linking to anchored line numbers. Clicking a line number navigates to its anchor, e.g. clicking line '22' appends '#L22'
It's probably a good idea to use per-snippet line numbers. Per-article line numbers would make sense if somehow you'd either a) have some way of importing global line numbers of the snippet in question (along with both a guarantee that no snippets from different trees are ever mixed as well as a way to unambiguously assign line numbers to lines in any tree -- which is by now veering into the absurd) or else b) rely heavily on by-hand typing of page anchors (except, obviously, almost every time someone will want to reference a line, it'll be because they had just read it, and the disadvantage of the more complex structure is minimal -- the machine can compose a #S3-L22 just as "easily" as a simpler L22 ; and of course taking the anchor out of the 2-3 character zone drastically decreases the likelihood of random collisions).
> ome new markup, e.g. [[]]
In retrospect [[]] is probably not the right choice seeing how it's actually used extensively in bash ; while {{}} is dubious because lotta bad C code will have trailing block closers like that. <<>> is of course taken by html itself... who knew ascii has a serious parens attrition problem.
> It would only require a new mp-wp plugin
Why the fuck would you make a new plug-in, in preference of just expanding the existing one ? Twice the maintenance or what's the idea ? Just take that one and own it.
> The ability to upload vpatches and signatures within the mp-wp admin UI
MP-WP has out of the box support for clearsigned comments, in the sense of displaying them correctly. Do you mean you wish to add signature verification ? I don't think that's such a good idea at all.
One thing that might be useful is introducing a special switch, such as the current b and e for selection, which if activated would simply display the article as plaintext (no html at all). Then vpatches could be first class article, just remember to add &patch=machine or somesuch whenever you wish to check them via a script.
I've been thinking about it, and I find it hard to repress the notion that detached sigs is not only a terrible fucking idea in the general, up there with the best examples of self-defeating nonsense beings an engineer so typically come up with ; but very much not what we want for patches. From time immemorial all the way to the fucking 80s people starting putting up a small comment header in files, but we don't because why ?
Imo the top priority re V-tree support is the patching of "whatever we use" (ie, koch's gpg, after all these many years STILL koch's gpg) to alter the headers from the current -----BEGIN / comment / -----END format into something more useful. Like say //BEGIN or ##BEGIN such that the lines are naturally seen as comments by all compilers ; and then add some useful verbiage (imo at least tree comment lines following, to explain TMSR in general, V-trees in particular and the current piece's relation, in that order) would be immensely useful.
Detached signatures permit people to pretend they don't know whom they're copying.
@Diana Coman
> I'd give it then some time in use to see if there really is a need for more since I can't quite tell upfront if it's even worth it to have inline and all of that
I agree, that makes sense
> Potential trouble that I see though is how do you order the vpatches if you aim to generate the page automatically.
I was thinking just in chronological order, by upload timestamp. Potentially we could parse the vpatch for the added block height stamp in the manifest
> And I'm not that sure I like the idea of forcing click-interface on users just to have this otherwise.
Hmm, I definitely haven't gotten that far yet in terms of how it would be implemented, but potentially it could read from a directory specified in a config. Then once files are SCP'd the page content will be updated. Need to think about this more...
@Mircea Popescu
> the machine can compose a #S3-L22 just as "easily" as a simpler L22
Fair enough. Although I don't see how there'd be a higher risk of collision since the anchors are per article (unique URL) anyway. Maybe I'm missing something? A format of #S3-L22 sounds good regardless though, I'll go with that.
> In retrospect [[]] is probably not the right choice seeing how it's actually used extensively in bash
Perhaps [[vpatch]] ... [[/vpatch]] ? (sub "vpatch" for e.g. "code", "patch" if desired, go with single brackets instead of double, etc.)
> Why the fuck would you make a new plug-in, in preference of just expanding the existing one ? Twice the maintenance or what's the idea ?
I don't know that there would be much effort saved in combining this with the footnotes plugin. The functionality is completely different (except for the parsing of an article for given open/close characters, which is only a regex one-liner) and there isn't really any boilerplate/overhead in implementing a new plugin that could be saved by piggybacking on an existing one—at least not that I can see. If anything, I think it would complicate maintenance by mixing two unrelated features into a single plugin.
> Do you mean you wish to add signature verification ?
No, no verification. This was in reference to the .sig files accompanying vpatchs, simply letting a user select a vpatch and the associated .sig file in the file browser and uploading both at once.
> One thing that might be useful is introducing a special switch, such as the current b and e for selection, which if activated would simply display the article as plaintext (no html at all).
This is really interesting, I personally like it. It would also answer the question of whether vpatches in the code shelf are stored on disk or in the db. I will keep this in mind when I get to spec'ing the code shelf feature.
> From time immemorial all the way to the fucking 80s people starting putting up a small comment header in files, but we don't because why ?
Heh, this is probably way above my level at this point. I hadn't thought of it before but it certainly does seem cleaner to have a signature embedded in a vpatch. How are multiple signatures from different signers handled then?
> since the anchors are per article
you could conceivably want to use that anchor for something else in the article. dunno, but short strings are often surprising in that way.
> Perhaps [[vpatch]] ... [[/vpatch]]
meh.
I suppose you could just hijack the current <code> html tag, though.
> I don't know that there would be much effort saved in combining this with the footnotes plugin.
On whose part ? I have a total of two plugins on trilema : one that handles recent comments and one that handles html extension to produce footnotes and such, because html was written by retarded monkeys and it's useless half mile out of base, in spite of weighing more than any adult whale. you want me to now have TWO html-extension plugins, because you don't see what effort it saves ~YOU~ ?
html extension can be at most one plugin, though if our fathers & grandfathers weren't complete idiots, it'd be zero.
> The functionality is completely different
Now I want to hear out this "complete". Hit me.
> How are multiple signatures from different signers handled then?
Consider something like this :
Would it be THAT bad ?
The snippet got massively uglified by the css, here's the original intent : http://paste.deedbot.org/?id=lwOR
In typing my response I started to make the argument to myself that the features would indeed be best housed within the same html-mutating plugin. My main arguments for separating them were 1) to be able modify one plugin with less risk of introducing bugs in the other, and 2) so that the operator could choose to enable one or the other and not always both.
The first is actually an argument in favor of having them in the same plugin. Since they're both operating, sequentially, on the same content, a change in one could easily screw up the second. At least with the code in the same file the maintainer is forced to be aware of the other formatting being applied.
The second could be remedied simply with a knob inside the plugin.
Regarding hijacking <code>, perhaps the <pre> tag instead? <code> is used inline sometimes, e.g. when mentioning a variable. <pre class="vpatch">, while more verbose, would also avoid applying the formatting when not wanted. Though this may be overthinking it, not sure when one would use <pre> for something other than a code block.
Also apologies for the comment formatting, this theme is in desperate need of an overhaul. I'll try and fiddle with it a little later so at least the example in your comment displays properly.
> The basic idea is to add a set of capabilities to mp-wp to make it work seamlessly as a vpatch repository.
What's wrong with e.g. http://lucian.mogosanu.ro/src/? It looks like a more than decent V patch repository to me -- and I've had no one bitch about it so far, but perhaps now's a good time for that.
That said, I do see the need for annotating/discussing code, but it's entirely unclear to me how said annotations would work. Inline comments look cool, IMHO.
@Mircea Popescu:
> Consider something like this :
From what I see, this would change the semantics of "signatures attached to V patches" to "signatures attached to source code files"? I.e. the signatory is not signing the change anymore, but the actual source file. Moreover, given that V patches contain the hashes of both the "old" and the "new" version of a file, it could be said that currently signatures are attached to (or well, detached from) all these three elements (old file, new file and code changes), whereas in your example I don't see this.
Am I missing something here?
> Detached signatures permit people to pretend they don't know whom they're copying.
Certainly. On the other hand I wouldn't understate the ability to choose the signatories of a V patch, because I want it to reflect my own WoT. Maybe I don't know who Dillion-Harper is and I've rated AlexisTexas -10 for some reason, so say I only trust AsaAkira and madison_ivy: what am I going to do with the couple of signatures I don't give two shits about then? Just import them in my tree because, well... they're there? Why would I be forced to do that?
And sure, if I'm stupid enough then I'll include zero signatures in my repository and I'll (mis)use it the way heathens use GitHub. Let the WoTless folk shoot themselves in the foot, why not.
That said, I do see the point in mentioning the author(s) of some piece of code in the source file itself. Though as things stand, I think manifests also work fine for that purpose, as far as V patches stand.
> From what I see, this would change the semantics of "signatures attached to V patches" to "signatures attached to source code files"?
Well, terrible fucking example for this purpose.
This is the problem with examples, they exemplify one thing not all the things, under pain of never being done constructing one otherwise. Should I have put it through a differ and everything first, to produce a proper patch looking thing, rather than just a simple textfile ? Should it have had a proper genesis first ? Should I maybe first go write an actual tree and then ?
I just meant, "this is how a clearsigned file'd look", I didn't mean "let's ditch patches and regress to signing the actual code" anymore than I meant "all programs must be hello world from now on".
> I wouldn't understate the ability to choose the signatories of a V patch, because I want it to reflect my own WoT
Definitely. However, your "choice" is very limited : you may only choose from among what's already there. You don't get to choose in any other sense than this eliminatory approach.
> what am I going to do with the couple of signatures I don't give two shits about then?
Nothing.
Is this so bad ?
> Why would I be forced to do that?
Your "that" is two orthogonal things. This is unhealthy. Proceeding on the resolution of the mix, on one hand how could you have rated someone -10 without having their signature ? If you've rated them you're stuck with them, that's what ratings mean. And on the other hand, who the fuck is forcing you for your machinery to be broken ? If your terminal isn't flowing lines the way you want it to, fix it, and if your whatever else, signature parser, doesn't parse signatures the way you want it to -- notably, by complaining of missing signatures from people you don't want it to complain about -- FIX IT.
> That said, I do see the point in mentioning the author(s) of some piece of code in the source file itself.
None of the foregoing had anything to do with a (necessarily doomed from the onset) attempt to rescue the moronworld notion of "author" into republican primitives. It's a great way to waste one's time, granted, but as it happens we're not at all in need of more of those.
Also, do me the favour and say something in #trilema if you do one of these ? As I say, I do check blogs periodically, more or less ; but the only way to make sure I actually will is either ping trilema or else ding me in #trilema -- and obviously you can't send pingbacks from the comment section.
@spyked: Sorry for the long delay in responding, the weekend was spent with some unwinding after having jumped ship.
There's nothing wrong with hosting patches in a web accessible directory and letting users browse that way. The advantage of a formatted page, when visited via a graphical www client, is that the information can be annotated. In the wireframe example, one can see at a glance for each patch (without navigating one level deeper) the signatures as well as a link to the announcing article for more context. The patches for each included project are also laid out in a single page, separated by secttions, with a tarball link for each vtree. None of this precludes having the underlying file structure on the server be a simple nested directory, navigable via graphical www client or command line.
I mainly put together these wireframes as a way of getting my interpretation of these ideas out of my head in a way that others could review/discuss. Based on the comments so far it sounds like code-snippets-via-some-new-markup is ready for work, while features like inline comments or the code shelf need some more consideration as to how they would be implemented and as to their utility in general.
@Mircea Popescu:
> I just meant, "this is how a clearsigned file'd look"
Oh I see, that makes sense.
> you may only choose from among what's already there
Is that a feature though? The side-question arising here being, how does a signature line "get" there in the first place? Then the other problem implied being, what if I trust none of the signatories there? Sure, then that's probably a problem with me, not the signatories, I'm just trying to figure out why do it this way and not the other, and what "this [technical] way" entails from a political point of view.
> how could you have rated someone -10 without having their signature ?
I don't understand. Maybe I rated them before they signed that, for some unrelated reason? Maybe I don't trust the hypothetical AlexisTexas to sign code at all on some previously established basis. I don't know, which begs the question: maybe the presence of that signature there gives me a reason might give me reason to not use said code in this hypothetical scenario I'm pulling out of my ass? If anything, this'd be an argument for including the signatures in the V patch, sure.
> FIX IT.
No argument here. I'm not complaining about the machinery, I'm just not convinced that this patch+signature organization is the right way to go and so I'm poking at it, is all.
@billymg: So if I understand correctly, this would provide a bit of automation to explore and grab V patches? Maybe it's worth detailing your reasoning here. FWIW, my biggest issue at the moment is the lack of a (at least partially automated) graphical means to look at the V trees, similarly to phf's thing. I don't know if including this in MP-WP is desirable by the maintainers, but it'd definitely get me to use the plugin.
> Is that a feature though?
> I don't understand. Maybe I rated them before they signed that, for some unrelated reason?
It's a feature in the sense light and heat are welded, non-optional call-it-what-you-will basic property of the universe.
I think perhaps our difficulties here come from an unresolved ambiguity, whereby "signature" denotes indistinctly two things : both the number which allows one to verify signatures issued by a certain key ("the signature" as in, the pubkey) and the number which allows one to verify a certain piece of text was "signed" in this manner by a certain pubkey ("the signature" as in, the respective rsa digest). Now, I meant the former, while I suspect you read the latter ?
> how does a signature line "get" there in the first place?
In my mental model, you can just copy/paste it in. Or I guess out, for the same money.
> Then the other problem implied being, what if I trust none of the signatories there?
This isn't the right node to attach this problem. If you don't know anyone, get some friends, go out more, all that.
> maybe the presence of that signature there gives me a reason might give me reason to not use said code in this hypothetical scenario I'm pulling out of my ass?
I think this'd be very inept misuse of the tools, or to quote
@spyked: Yeah, the idea was to add some automation to the process of organizing vpatches + introductory articles in a code shelf, as well as a more at-a-glance layout for easier browsing.
I like phf's patch visualizer too and was thinking potentially one could be built that pulls from code shelves hosted on individual blogs via a standard API. So for example, let's say you stand up a new "V Repository" and configure it to include patches from billymg.com, thetarpit.org, ossasepia.com, bvt-trace.net, etc. etc. and then the software reads data from /code-shelf/vpatches.xml from each site to construct something similar to phf's thing but automatically updated when any included blog publishes a new patch to its code shelf. It could generate graphs for the v trees, automatically create backups of the patches, generate archive.is links for the introductory articles, etc.
So the "V Repository" (mapping to what's currently btcbase.org/patches) would be its own thing, that anyone could choose to stand up and configure (like the new logotrons), but would be able to read from a standard code shelf API hosted by mp-wp instances.
@Mircea Popescu:
> Now, I meant the former, while I suspect you read the latter ?
Indeed...
> I think this'd be very inept misuse of the tools
... and agreed! :D
@billymg: I'm going to have to ruminate on your "V Repository" idea for now, at least until I've revisited Diana's "V-Tree Nursery", because for some reason it looks to me like it's quite related to the subject.
[...] postings from the L1. I've still Lobbes Portage exploration, Bvt's Gales exploration and Billymg's MP-WP patch viewer discussion to (re)read, but getting back to stuff I've done so far, well... reviewing 2019 took more than a [...]
[...] billymg Do you read the logs? You should probably read the logs. « MP-WP Patch Viewer and Code Shelf [...]
[...] these lists by hand. Fortunately for me, there's some preliminary discussion on something called "V repositories" which would be an immense help in replicating some of Phf's patch viewer functionality... which [...]