Priorities for September, in order of execution:
- Canonical URL investigations. In10se's bug listing and canonical URLs on the printable page we need one to collapse the URL duplication
Migrate the cron items to better cookbooks, as in done via APIs
- Simplify Everything::Experience
Move over to JSON configuration instead of Config.pm to allow to really smart data structures Done!
- License support for writeups (more on that shortly)
- Extract out the static.everything2.com stuff so that I can set the imagehost elsewhere
- Update the javascript stuff to upload to S3.
- Remove HRLF stuff entirely to simplify
- Hunt down and remove unused maintenances
- Broke the HTML edit (inside of WYSIWYG editing) button when I moved to jscss.everything2, so will need to fix it
- Install an MTA on the bastion hosts to capture cron output
September 3
- Nuked fttask. That was more full-text indexing stuff that I just don't need.
September 4
- Nuked Grandfathering removal impact, which I am surprised that I wrote some time ago.
- Migrated to the larger AWS AMIs, so that we won't run out of space. (Man that was dumb)
- Reintegrated long-standing experimental mysqlproc support
Added the get_recent_softlink mysqlproc to the codebase, now to track what was otherwise an invisible piece of code
September 5th
- Added update_time, the mysqlproc to the operationlized list of mysql procedures
- Added fixup_blacklistref to the list of mysqlprocs
- Nuked node maintenance update because it has no point
- Fixed bootstrap so that I can login off of cold github bootstrap, which is a huge accomplishment.
- Changed some text in Donation Box to reflect the new nature of how we're run
- Deleted fdsafdas, the only meta-user
- Deleted meta-user, the type. Deleting unused nodetypes is a bit of a performance enchancement.
September 12th
- Been working on a lot of stuff that doesn't direct affect the content of the site: largely hammering out the chef recipes into a state where I can release them publicly, and then hook them up to the chef-solo provisioner inside of vagrant. It's taking some time, and some production work, but I'm pretty happy with the results. This is exposing some other work, which will need to get done, specifically some interim work on the S3 syncing front. More info on that in a few.
September 15th
- The chef recipes for the vagrant development/debugging environment are now consistent with what we use in production. This means that we are using exactly what production is using, minus production secrets
September 18th
- Added in a JSON configuration entry in /etc/everything/everything.conf.json so that I can have deeper configuration data structures, and can remove the dependency on Config.pm. This means I can also write utility scrips that don't need to include Everything.pm
- Removed an apache memory usage monitoring script which was replaced by the internal apache memory monitoring directives
- Going to have to replace %Everything::CONFIG with $Everything::CONF, but both exist for now as a stopgap
September 19th
- Deleted: E2 FAQ: How to use full text search
- Fixed the Node Bucket section in Personal Links because I use it like a task list, and deleted nodes were staying in there. So annoying.
- After pretty extensive testing, dropped the {core} and {package} attributes on node, basically causing pretty large memory savings, just on the number of times we had to have a hash key to store unused junk.
- Turned DKIM back on for the accounthelp@everything2 email address. No idea why it got turned off, but I'd imagine it was the DNS swapover.
- Moved over the core libs to using the JSON non-compatibility structure for CONF ($Everything::CONF) instead of ::CONFIG
- Port over zen stdcontainer to CONF vs. CONFIG
- Ported showuserimage
September 21st
- Created guestuserbanner and added it to zen container. This is going to help us try and explain the site to people as a stopgap measure until I can get actual real interface stuff in place. It's currently an inline style (I know, I know), because it's only showing for logged out users, and I don't have a great way to update the stylesheets just yet, but I felt it was important to get that into place.
- Removed the everything.conf (not .json) stuff from the chef recipes to fully move everything over with
- Nuked E2 Zeitgeist, and I'm filing a github issue to replace it in the future.
- Updated All The Code to remove the CacheQueue reference and add in a few other items that were missing.
- Fixed a typo in guestuserbanner. It said in the nodeshell message "add something something here". Which was, as Halspal pointed out, a tad racy.
- On an SEO kick today. Updated zen stdcontainer to always use the canonical href.
- Removed Cachestore broken, as it was you know, broken.
- Removed displayPageTest nuked, as it isn't referenced anywhere, and it used cacheStore
- Removed cache_stats, as it referenced cacheStore
- Rewrote cache dump some to get some execution stats. Super interesting counts.
September 23rd
- Moved the e2cron entries over to a totally saner API-driven usage, instead of just dumping a file into cron.d
September 24th
- Fixed uploaduserimage to use the new s3 credential layout as we make way for Everything::S3 stuff, which enables a slew of new and completely useful features
September 25th
- Added a meta name="description" bit to every page for SEO
September 26th
- More infrastructure work to test out automated backups.
September 27th
- Added noindex to the meta information tag of the printable page so it won't show up as being a little spammy
- Added configurable nodecache sizes so that the development environment has a bit of an easier time in the VM (600 production, 200 dev)
September 28th
- Tracked down and destroyed our last remaining memory leak, associated with the {_ORIGINAL_VALUES} update hack. Clone.pm was causing a few problems, namely that handles were sticking around when they fell off of the CacheQueue. Also, it was using three times the memory by deep copying the $$NODE{type} and $$NODE{type}{type} hashes, causing huge amounts of extra memory usage. I implemented a nocached selectop for getNode, and we simply fetch a copy of the node from the database if we need to perform an update. This is going to allow us to do greater work with sitemaps, because it was a long running job, and being OOMkilled by the cron server, because the leak was killing all available memory
- Also, whacked all traces of the internal sqlLog. If I need to see what we are doing, we'll log to a file, or check the mysql log. This was also causing a slower, but still painful memory leak in NodeBase.
- Updated and extracted all of the API stuff for recaptcha, and now it lives as a part of Everything::CONF
- Older aws_e2media conf keys are integrated into new S3 setup
September 29th
- Wired up the cron jobs to properly capture their output to a file. Some cronjobs are incredibly quiet, so I'll need to add some verbosity to them
- Wired up the sitemap stuff. I've put in a rewrite rule for everything2.com/sitemaps to point to sitemaps.everything2 on an internal proxy
- Reduced some old transition-to-AWS area DNS records
- Removed Clone from Everything::HTML
- Created nodeparam, to start coming up with how we're going to do arbitrary metadata
September 30th
- Did a scan of the database tables to create dbtables for ones that were created raw. Which means I created the deletedhits,e2temp,nodebak,nodenote_restore,podcastLinks,softlinks,useractionlog, and xmlschema dbtables, even though I'm not sure what all of them do. softlinks is a view in the database however, so I might end up needing to track that as a mysqlview type eventually if we still need it.
- Changed the way titles look inside of the zen stdcontainer. We now have unique titles for writeups, and the author's name is in the titlebar.
- Investigated the lay of the land re: VARS contents: E2 VARS site sweep, September 30, 2012
- Installed the sitemap so google/bing/yahoo/ask etc can find us easier. Added robots.txt back again so that crawlers will hit that first. Nodeshells and users that have logged in are excluded from the sitemap, and we include writeups, e2nodes, and users
- Created the s3content dbtable, so that I can squeeze it inside of jscript and stylesheet nodes, so that updates automatically minify + compress + upload to s3. This is going to remove the practice of checking the versions table, which is broken in the development environment (as we don't export it), so that I can do more localized stylesheet editing and testing (without uploading it)