Before I start: given what I am going to say in the next paragraph, I should like to make it clear that I was in no way responsible for the several hours of downtime on March the first. Or if I was, then it was a really strange bug that should have been cleared up years ago, and it wasn't my fault. Or if it was my fault, then I was sleepwalking and I didn't mean it. The most likely reason is just that there was too much work for too little computer. Having cleared up which:
Those of you who were paying attention will have noticed a pause in the trickle of fabulous new features that had so enhanced your enjoyment of the site over the past months. One or two of the rest of you may have noticed that the area at the bottom of the page where you enter or edit your writeups got a little tidier. Some of the unfortunately eagle-eyed might have noticed that for a couple of hours on February 14th it was not possible to edit existing writeups, while for about twenty minutes on February 26th it was not possible to post new ones. These were the tiny visible signs of the hidden herioc and Herculean endeavour under way beneath the surface of the page, as I wrestled with code, nodes, demons and death (yea, even the death of the development server) to bring you D R A F T S!
Having said which, I would like to make it quite clear that I didn't break the server last Tuesday. Or indeed, even bring you D R A F T S! So to distract you from this failure, let me tell you a story:
How Everything2 Works
You may have heard that on this site 'everything is a node'. If you hadn't, you have now. This is a useful generalisation, even if it is not in fact true. A node is a bunch of related pieces of information in the everything2 database. The page you are looking at now is a representation of a node sent to you by the everything2 web server. But what happens between the database and the web to produce that representation?
When a page request arrives at the server, it is taken apart to put two essential pieces of information into a form the e2 engine can digest: what node does the user want to see, and how do they want to look at it? Both pieces of information are essential: the node is simply a bunch of bytes. To know what to do with it, the server has to know whether you want to edit it, look at it on a mobile device, get it in a machine-readable form for further processing, or just read it embedded in a normal everything2 web page. (And those aren't the only options.) And to know that it looks at the 'displaytype' parameter, which is either sent in full view in the url or passed in surreptitiously as a post parameter. (I personally would have chosen a shorter name, like 'view', but I wasn't there and they didn't ask me.) If there is no displaytype given the server assumes it should be 'display'.
What a node is
After working out who you are (a logged-in user or Guest User) via arcane rituals that shall not concern us today, the server calls up the node from the database. A database is essentially a set of tables of information, with rows and columns. The rows contain various pieces of data that belong together, the columns each contain one particular kind of information. The E2 database has over 100 tables. A node is made up of information from one or more of those tables. Every node has an entry (row) in the 'node' table. One of the pieces of information stored there is nodetype, which is the id of a row in the 'nodetype' table. This table stores information defining a whole class of nodes, including the name of the nodetype, who is allowed to view, edit, create and destroy nodes of this type, and so on. Two very important pieces of information are which other nodetype a nodetype 'inherits' from, and which database tables contain information for nodes of this type.
By way of example: the everything2 front page is a node of type 'superdoc'. This type has no entry for database tables, but it inherits from the nodetype 'document', which uses the 'document' database table. The nodetype 'document' inherits from nodetype 'node'. So when someone goes to the front page, the server calls up the appropriate entry from the 'node' table, looks at its nodetype, calls up the entry for the 'superdoc' nodetype from the database, looks at which other database tables are needed for this nodeteype (none), looks to see which nodetype it inherits from ('document'), and which database table that needs ('document'), looks to see what nodetype 'document' inherits from (node, so it looks no further) and calls up the appropriate entry from the 'document' table. All the information from the database is stuck together into a data structure whose technical details need not concern us here, which is what is generally referred to as a 'node' in these parts. The software that deals with the information from this point on does not need to worry about where or how the information is stored in the database.
What an htmlpage (sic) is
What it does need to worry about is what it should do with this node. The node for the front page is relatively simple: it only contains 13 different pieces of information. To put the right selection of this information into the right form for you to read, the server runs a piece of code, which is stored in another node, of type 'htmlpage'. (Yes, another node. Everything is a node, you know.) The htmlpage required is selected on the basis of the displaytype and the nodetype. First, ecore looks for an htmlpage for this displaytype for this nodetype. If it can't find one, it looks for one for the nodetype this type inherits from, and so on. If it can't find one for any nodetype along the inheritance chain it gives up and tries again for displaytype 'display'. Since there is an htmlpage for displaytype 'display' and nodetype 'node', the search is always successful in the end.
The htmlpage has a data field imaginatively entitled 'code'. This code is run, and the resulting output is sent to your computer. Usually, this isn't all of what is sent to your computer, because a field in the htmlpage node will specify that it is associated with a 'container'. A container is a node (of course) containing (among other things) code. When run, this code produces two additional chunks of output to be put before and after the text produced by the htmlpage. The container may itself specify a 'parent' container, which is used for further wrapping. In the case of the front page, the htmlpage 'classic superdoc display page' produces the html code for 'Logs' through to 'News for Noders', which is wrapped in more code by 'zen container', providing the header with the search form, the nodelets, and the footer with the credits and copyright information, and that is all then embedded in Invisible but Important html code provided by 'zen stdcontainer', which includes useful things like references to the stylesheets that make the page pretty and the scripts that make it jump when you tell it how high. This last container has no parent, so that is what then gets shot along the string to your tin can.
So what?
All of this arcane knowledge is of little use except to those who wish to code for the site and don't feel inclined to have to work out everything for themselves (hur hur), and to those who wish to understand the rest of this root log.
Drafts and Writeups: a tale of Two Nodetypes.
At present, a writeup is a writeup unless it hasn't been written yet or has already been nuked. A Scratch pad is a strange thing for Everything2, because (brace yourself, son, the world is a hard place) it is not a node. This makes it rather difficult to do anything much useful with it, since all of the tools available in the E2 engine for manipulating nodes are not available. Still, at least you can edit it or delete it. A writeup that has been nuked is in an even worse state: all you can do is look at it in Node Heaven. So for some time, the idea had been floating around that it would be a good idea to be able to combine scratch pads, writeups and Node Heaven into a single flexible, robust, intuitive, seamless, [insert further positive adjectives here], and generally rather cool system. The idea being that a writeup would have extra information attached to it to indicate at what stage of its life-cycle it was, and who was allowed to see it if its current state was not 'published and visible to everyone'. This idea is known among the coding 'staff' here as 'the drafts project,' since it would involve writeups being able to start life as 'drafts' and being able to revert to that status instead of being nuked.
Unfortunately, there are several hundred nodes containing code that does something or other with a writeup or writeups or the writeup database table. So this was not going to be an easy job. Nonetheless, after some discussion, and after nothing had happened for a while, I foolishly volunteered to get on with the job. At that point I didn't know how about the 'several hundred' I mentioned above. Once I had found them using the useful tool provided for this purpose by OldMiner on the devel server, I found my mind wandering in search of an easier solution. There is little that concentrates the mind like the prospect of avoiding work, and I am moderately pleased with the solution I came up with: a draft is not a writeup with particular special features, but a writeup is a kind of draft.
Specifically, and technically: the nodetype 'writeup', which previously inherited from nodetype 'document' will inherit from the new nodetype 'draft', which will inherit from 'document'. Thus drafts will not be writeups. Information on the status and history of drafts will be kept in the 'drafts' database table, and an entry will only be added to the 'writeup' table when a draft's nodetype is changed when is published (and removed if it is 'unpublished'). Thus, drafts are invisible to all the existing code that deals with writeups. Several hundred patches are no longer required, and DonJaime is faced with a much more manageable task.
You can run and you can hide
There is still a lot of code that just shows you any kind of node. The only general access restrictions existing at present restrict access to a nodetype to a particular usergroup. There is no way to specify that a draft is only to be accessible to the author, let alone to allow the author to specify particular users or groups who may be allowed to see or edit it. But there would not be much point having drafts at all if they were as public as anything else on the site, just not attached to e2nodes. So having created drafts, the first thing I had to do was hide them. OldMiner has a plan to produce a generalised system of access control at individual node level, but it could take some time, so for the moment there is an ugly but effective system in operation, based on the use and abuse of htmlpages: the first thing that happens on any htmlpage that could be used to output a draft is that a check is made to see if the node is a draft and if the user can see it. If it is and they can't, the node is replaced for the further processing of the page by the 'Nothing Found' node, which is then further processed with the appropriate htmlpage for a superdoc. Thus there is no indication given that an inaccessible draft even exists.
Ooh ooh ooh: I wanna be like you-hoo-hoo
Drafts and writeups are essentially the same kind of thing and should therefore be displayed in the same kinds of way. As you will recall, if an htmlpage does not exist for a particular nodetype, ecore looks for an htmlpage for any nodetype it inherits from. So it might seem that the easiest way to get drafts displayed properly would be to change the htmlpages for writeups into htmlpages for drafts. And that would be true, but I lied to you when I told you about that: I missed out the bit about themes. Themes were the mechanism used in the Bad Old Days to customise the look of the site by serving variants of the html code. These days, we do that with stylesheets, which tell your browser how to style html code, which is the same for everyone. This is a better way of doing it (even if CSS is horribly badly designed and difficult to use. Which it is). Only the zen theme is supported at present, but the others are still there. And ecore gives up on trying to find an htmlpage for your particular theme at a fairly early stage. So if you entered an url for a draft with ?displaytype=bunga-bunga at the end, you would be treated to the draft in all its glory in a putrid purple colour-scheme. The easiest way to get around this turned out to be to create an htmlpage for drafts for every displaytype available for writeups or documents, and have it carry out an access check and then grab and run the code from the htmlpage for writeups or documents as appropriate. I could tell you the details, but I kept my notes on the dev server, and it's dead.
Drafts should also be edited like writeups. And writeups and drafts should be created in an environment that is similar to the one in which they are edited. So there is now an editwriteup htmlcode that takes care of all of that.
Where's the beef?
At present there is a drafts nodetype, which is generally handled very much like writeups but with access control. An author can delete their own drafts, whereupon they are gone forever (there is an undo function, but only until you close the page). There is a converter to turn scratch pads into drafts, code to manage the status and accessibility of drafts, choose an e2node to publish a draft to, and to publish drafts into writeups. The code for insertion of a writeup into an e2node has been taken out of the code that manages new writeups so that it can be used when publishing a draft, and all the different checks on a user's ability to publish a writeup to a particular e2node have been brought together in one place to avoid duplication of code. And you are wondering why you can't get drafting right away. And the reasons are:
- There are a lot of places where the order of writeups in lists would be wrong, because they would be sorted by the time of creation of the draft, not of publication of the writeup. For example, you could publish a draft and never have it appear in New Writeups. I have a list of all of them on the devel server. Had.
- Drafts should be accessible using the same url structure as writeups, so the url doesn't change. This requires a patch to ecore. This is easy to do on the dev server.
- For the urls to work we need to make sure that an author's writeups and drafts all have unique titles. I'll be doing this soon.
- nodetype 'writeup' on the production server does not inherit from 'draft', and there are a few hundred thousand entries needing to be made in the 'drafts' table when it does. I have a record of exactly how I did this on the dev server without trashing it. I'm not going to rush ahead with this until the rest has been done.
So here it is, Merry Christmas
I have a plan, and it's a good plan. It comes in phases:
- Fairly soon it will be possible to create manipulate, delete and publish drafts. After that:
- Attach a draft to an e2node. If a user can read such a draft, they will see a link to it at the bottom of the node.
- Ask for editor review of a draft automatically by changing its status to 'review'. 'Review' drafts will be shown to editors in the e2node they are attached to.
- Don't nuke writeups, but turn them back into drafts. Authors will be able to do this themselves.
- Convert the contents of Node Heaven into drafts for appropriate treatment.
I've consulted the Management, but there may still be slight changes of points of detail.
Oh, and by the way:
- The draggable menus on the Nodelet Settings page have uppy-downy handles to make it clearer how you can move them.
- Everything2 Ajax will not die with an error if an ajax update specification refers to a non-existent form element.
- Settings nodes and user vars with multi-line settings can now be edited without removing all the linebreaks.
- Buttons as well as links can have an automatic javascript confirmation dialog.
- If Guest User happens across one of the settings pages they will only see an invitation to log in, and not a pile of half-digested html rubble.
- The suspension type 'writeup', intended to suspend a user's ability to post writeups in the same way as other privileges are revoked, which previously did not work, now does. Node Forbiddance is still better, because it provides a reminder of the reason for the restriction every time it operates.