Mapping it all

Hello guys, hope you’re well. In case you missed it, my previous post touts the debut of Discord integration with Patreon. If you like you can link your account and get the equivalent of a shiny badge and another place to chat.  

Moving on, earlier in the week I push some of the things I had working onto the site and these changes are now live. I wanted to offer a quick recap of what they are and give you some behind-the-scenes perspective.  

The new sitemap

This is one of the things that was sorely deficient about THP. I mentioned it in one of my previous posts but to recap a sitemap is a supplementary tool to all the crawlers from search sites (Google, Bing, Yandex etc) that lists relevant pages on the site that they should take into consideration. Not every search site treats it equally and they sort their results based off any number of parameters but it doesn’t hurt to have an up-to-date sitemap.

The key thing about the new sitemap is that it lists the thousands of pages belonging to archived stories. In order to do that, I had to rewrite the code that generates the sitemaps from the ground up, Some of the previous changes and reforms made to THP’s code base made this a lot easier than it would have been just a year ago!  

My wizard powers keep evil at bay. I modified that sucker for the sitemap.

Sitemaps were something handled by the configuration file for whatever reason instead of generated whenever a thread was actually updated. I moved code into its own function and added a bunch of checks, mainly for the archived threads. Speaking of archived threads—I first had to write a one-time snippet of code to index all of the old threads into JSON format so that the new sitemap code would be able to read it and add those entries to the output file. Whenever a thread is archived by the board software it also gets added to the JSON file. In the very long term it might be desirable to actually add a list to the database of all threads.

Optimization

When it comes to optimizing things, there’s two broad categories that apply to THP: output and processes. The former is easier to grasp as it mostly refers to content that users interact with. In other words, site pages and loadable content. I’ve long since implemented work to make things like CSS and scripts leaner and set appropriately long caching for images (so that they’re not requested too often leading to longer load times). Now, I think I found a solution for the actual HTML files that I’m satisfied with.  

To make a long story short, pages should now have a smaller file size universally. This is achieved by minifying output, which mostly means getting rid of spaces and unecessary characters. The benefits are modest, accounting for about an overall 5% decrease in file size (larger files that are over 600KB have less of a % but higher absolute figures like 20-30KB reduction whereas a smaller 90KB file might be reduced to 80KB). When you account for the fact that THP is actually thousands of HTML pages (every board page and thread) then it makes sense to make sure that your content is as small as possible. The trade off is more processing time when pages are generated but it’s a relatively small rise. Making sure users on slower connections can access more pages faster is worth it to me.

There had been some attempts at minification before but some were too aggressive and ended up modifying how pages looked. I didn’t write the code from scratch but instead ended up modifying an existing library to suit our needs. There were quite a few quirks that needed to be accounted for that are unique to THP.

The other type of optimization is something that I do whenever possible and there’s no one thing that’s special this time around. Some of the javascript was made to reuse codes from other parts and some of the PHP code was also simplified or otherwise more generic. There’s a lot more that can be done here but it’s not exactly high priority, particularly the PHP as that’s server-side.  

Misc  

The usual spate of bug fixes also made it in. The thread auto updater should work slightly better, for one, and the timer has also had a few edge cases dealt with. More importantly, threads should have a button on the bottom to return to the top of the thread. This had been in the code for years but had inadvertently disabled because the conditions that made it appear weren’t being fulfilled anymore.  

Some other features have been deprecated. Most notably, the first 100 and last 50 posts that appeared on board pages. The usage stats I had showed that the only people that clicked on that were crawler bots looking for links. Normal users didn’t really use it. It may have made sense to have leaner HTML files ten years ago when devices were less capable and speeds much slower but there’s not a very strong case for it now. There were a few bugs related to the page generation there as well so, with all of the above considered, I decided to axe the feature completely.  

I almost decided to scrap the catalog view for boards as well. For the time being they’ll remain and have gotten a few basic improvements such as specifying the number of images and accepting other file formats as thumbnails for the thread. This may change in the near future and it’s something that I’ll revisit as THP continues to be restructured.  

There’s a few other minor things such as making sure that spaces between characters are consistent across threads and such but they’re too insignificant to warrant in-depth explanation. Suffice it to say that about a dozen small issues like that were dealt with in the last round of site updates.

What’s next

I’ve a mix of about 30 fixes and features that I plan to work on in the coming weeks. Most of these are quality of life type things for me but some of them affect the user-facing portions of the site as well. Like, there will be a standardized way to add a separator to your posts.  Changes to the settings menu and enhancements to existing features will be added in along with a pair or so of new user settings.  

I’ll make sure to detail all that in a future post.

For now, the biggest news is that I’m not going to be working on the mobile overhaul this month. There’s dual reasons for that: 1) I don’t have the free time to schedule work on it nor can I budget a working day or two to it (there’s a lot of QA and testing that needs to happen)  and 2) I feel that a bigger difference can be made for THP by prioritizing work on another project that I’ve wanted to do for a while.

In practical terms, I’ll spend the next week or two added a few minor features and fixes and then move onto the big thing that I’ve had in mind. I’ll beta test it with patrons first and get feedback so look forward to that! There’s not much that I want to share right now but rest assure that it’ll make sense why I’m prioritizing this over the mobile overhaul.  

As always, let me know what you think and what you’d like to see more of. I’m always interested in hearing more from the community. Until next time, take it easy!