It's December, and the days are getting colder. Huddle up by the fire, and let me tell you a story about the season
These root logs are a lot of technical jargon, and sometimes it's hard to go through and pick them apart for the non-tech staff here. The near-term goal here is to get a new feature out the door, and to not put us in a position where I'm going to have to rewrite it as we move to the new architecture. It may seem like I'm working on somewhat random items, but there's a method to my madness, so let me take you behind the curtain.
The first major thing I got done last month was the transition of the configuration object so that I can add some logic to the configuration object and This is going to help greatly in mocking things out for QA, as that is something that nags at me. The front line of problem discovery right now is that someone (politely) yells at me. This message is two fold: this sucks, and I want to do better at it, but that if you see something weird, don't assume that I know about it.
The second major thing I got done last month was a new nodetype datastash, which works like a JSON-backed key value store. The JSON parser is 7x faster than the $VARS parser we use now to look at everybody's settings. If I had super extensive QA on this type of engine thing and a good editing page, I'd convert all of the settings stuff to json right now. In fact, the notifications code already jams JSON into the $VARS hash, so that'd be an optimization there as well. Eventually all of these items are going to be out of the database, including settings, which means that they can be edited with an operation on the source code tree. The push for this is to fix the display snarliess that is nodelets. Nodelets are the boxes on the side of your page like Epicenter, Other Users, Messages, Random Nodes, etc that render every page. They are an archaic design, but are so built into the way the page is loaded that it has made it hard to modernize. Where they get problematic is there are certain types where they update every X seconds on a script that gets run by the bastion server in an odd emulation mode. The code to generate them assembles the data and then spits out HTML. This means we have a lot of code basically copy-and-pasted all over the codebase to perform different functions. The place where we do it semi-correctly is New Writeups Feeder, which breaks the paradigm a bit and stashes JSON in the HTML piece, and is simply called by the New Writeups code that you know and love.
I think of the work here in terms of the "stack"; what needs to be accomplished in what order to move the site forward. I'm driven by the eventual goal to get a new layout and start really pushing traffic to the site. We've got over a decade worth of content here and I really want to put it on display. The other consideration is the long-term technical survivability of the site. The business side is stable, the growth side is more stagnant. I have little faith in mod_perl's continued existence as a well-maintained stack, so we need to get to Plack and a new middleware as quickly as we can.
So, what's in the stack?
Clean up the nodelet architecture
Nodelets present a challenge in that they need to be pushed into the existing Mason 2-based templating system, and the underlying data generators there need to move into Everything::DataStash::* objects. Once that is done and the actual featureset pared down to the features that are actually being used and the data grabbed from the DataStash.
Templates already work. In fact, the entire containing page is templatized with the exception of nodelets. Epicenter is templatized, but once the data generation is out of them, then nodelets will work more like standard view objects. Once nodelets are templatized, we can turn the new engine on.
Finish non-controllerified emulation layer
Another way to think about this is "retire zen stdcontainer/zen container". I have to make sure the fullpage and node ticker stuff works, and add test suite support for Controller/Router mockups. Superdocs are going to live in here for a while since about half of our side code lives inside of superdocs, like Message Inbox, Login, Create a node, etc. Eventually the whole concept of different types of superdocs will go away, and they will simply be documents that all handle their own permissions the same way.
Templatize the document nodetype
The document display type is the easiest to nail, and thus the basis for a lot of the other work going on. It's the most straight-ahead, but still requires all of the other work to be there.
Turn on templatized layout
Actually cut the code over. Again, this is one of those things where the bar of technical triumph for me is you folks noticing nothing
Migrate e2node/writeup layout
Once we have the ability to start untangling things, then I'll push it the main writeup pages on over to the new system.
Turn on writeup commentary feature
Once we have the clean layout all set, we can turn on writeup commentary works.
Migrate over the non-superdoc/superdocnolinks/oppressor_superdoc/plaindoc/restricted_superdoc types
Once each of the page types exist in the system as controller/view combinations, we can retire htmlpages.
New layout based on Twitter Bootstrap
Bootstrap is awesome. It's going to help us actually have a good mobile strategy. It's modern.
That's the roadmap. It's a clear series of work from here to there and I'm making progress on it. In January, I'll do a State of the Site report with what the current plans are.
Dec 4: Updated the Random Nodes data driver with randomnodes.
Dec 5: Replaced the Random Nodes nodelet with the new code.
Dec 6: Nuked Test NW Feeder. It wasn't used.
Changed the way that the Random Nodes XML Ticker to use randomnodes. Nuked Random Nodes XML Feed.
Cleaned up New Writeups Atom Feed. Moved it over to the newwriteups datastash. Nuked New Writeups Atom Source. Doing some validation tests on it show that usernames in spaces cause it to fail validation.
Moved New Writeups over to the datastash architecture. Looks good. This should also fix a bug where writeups would flicker on and off of it.
Cleaned up Cream of the Cool to use the datastash drivers.
Dropped a bunch of unneeded columns from nodelet because it was bugging me.
Changed the way that we wrapped nodelets. It's temporarily an APP function for speed (to counteract some of the other expensive stuff I'm doing), but it'll be a part of the template stuff shortly
Moved the non-guest frontpage to use frontpage_creamofthecool instead of the HTML stuff inside of Cream of the Cool. Nuked Cream of the Cool
Nuked ascorbic's front page, even though it's super similar to the direction in which we are going.
Same thing, but with Cool User Picks.
Staff picks as been mainlined using the datastash system. The corresponding nodelet is gone.
Cool Staff Picks was only used by 144 users, and isn't that important in the grand scheme of things. We can recreate it in the new layout if it is desired.
Dec 7: Nuked Everything2 - Oolong's Front Page, so that we can continue to get rid of old blocks and the things that use them. It was a bet anyways.
frontpagenews now grabs items as appropriate. I'll need to expand it so that it actually contains the correct data eventually, but that's an optimization for a later date
Dec 8: New Cool Stuff is the next nodelet to get decoupled. It powers Guest Front Page, and has been moved to the Everything::DataStash::altfrontpagecontent module. Nuked New Cool Stuff.
Really getting to demonstrate the power of the new nodelet stuff. Recommended Reading now just gets generated from the existing feeds. Less code, same features
Nuked Other Users - Zen, as it is an unused beta
Dec 15: Fixed a UTF-8 encoding problem with the new data stash stuff.
Dec 27: So finals and Christmas. Anyways, fixed a noisy bug in choose theme display page that filled up our logs with errors, making it harder to debug other things. For Review now uses the datastash system, and properly displays the number of notes on a particular node for editors