The wonders of optimization

Hello everyone,

I've been busy the last few days going into full coding mode.

I'll give you a quick TLDR, since the explanation will be quite technical:

The game is now much smaller in size, uses less RAM, but actually increased in image quality. (Full quality at 1.3GB zipped)

(Very) long Explanation for anyone interested in the technical side:

I've been having some problems in the last few releases with building taking longer and longer, and file size scaling up a bit too much. With the current build size being almost 3.5 GB zipped, adding new sessions for the other girls as well as just regular updates would soon end up in a ridiculously huge size.

Additionally, the building time for each platform was taking insanely long. For the last few updates I had to spend 1-2 days purely building, zipping and uploading. Just the public versions alone are 15 GB in total to upload including the compressed versions, doubled to 30GB since I'm uploading to two file-providers. The new versions are an additional 10-15 GB.

Switching between platform builds also took a crazy amount of time. Switching between Windows, Linux, Mac and Android build modes can take 2 hours each, as Unity likes to re-import all textures for the platform, and there are now thousands of textures. Additionally, building the assets bundles for each platform does the same.

All of these processes would have to be reset and done all over if I spotted a bug in the build. My reaction to game breaking bugs would be slow because of all of the building and uploads, like it was the case with the shop bug in 0.17.1.

This leads to all the copy-cat pages that parsing from f95, to copying the bugged version, which leads to a lot of people downloading the bugged version instead of the fixed one, since the copy-cat pages don't usually update once they spotted a new update on f95's "latest updates" page. If I can get a fix up before they parse it, there won't be an issue.

But there is great news: all of these issues are now fixed. The full quality game is now even smaller than the compressed version was before, and it has better quality than the full 3.5GB version had.

To explain how that is possible I will have to first explain a bit how textures and compression works.

An image as you see it in your browser is usually compressed using common formats (jpeg, png, etc). These formats are perfect to store images as they significantly reduce the file-size over uncompressed bitmaps. However, the compression in those formats makes use of the file only having to be read only once while loading. Their compression is sequential, meaning that data in the middle of the file is worthless if you didn't start reading everything from the start.

There are different rules for textures used in most common games. Textures have the restriction that they have to be randomly accessible. This means that if the GPU says "Give me the color of the pixel at x=123 y=345", the texture has to return a color without reading the whole file. This would be impossible to do for a png, since it would have to read the whole thing to find out what that pixel is.

Because of the random accessibility requirement, texture compression is much less efficient than regular image compression. It's larger and it looks uglier (artifacts).
Unity by default is using texture compression, since it's a standard for most games. Each platform has its own compression standards etc, which causes the huge waiting time.
The size is also much bigger than regular image compression, although still a lot smaller than storing it uncompressed.

The main advantage of texture compression is that the texture can be loaded into video memory compressed, while still be accessible. This solves a few bottlenecks and actually speeds up drawing (even though this is pretty irrelevant in a game like Hail Dicktator).

Now to reduce file size, I am a storing all renders with regular image compression. I'm using the more modern "WebP" format, which is essentially the best of both world of jpeg and png. File size is much smaller than png, even in lossless compression, and lossy compression is virtually undetectable in a high enough quality setting.

The disadvantage of doing this is that I don't have the luxury of storing compressed images in video memory. When I load a render I have to store it uncompressed, which for a 1080p render takes about 15MB, which is huge compared to the 1MB a compressed texture would take.
Additionally, it takes a bit of time to actually load the webp image into a texture, so loading more than 10 images will be noticeable.

That's why I'm now using a mixture of textures and webp on-the-fly textures: For all locations and composite pictures I'm using compressed textures, which ensures that everything loads as fast as before. For all scenes, I'm now using webP, since only one picture has to be loaded. The loading time is small enough that it doesn't make a difference between this and using a compressed texture, since it's just one image at a time.

Since all webp images are loaded while the game is running and not "baked" into the game, they don't require any re-import. This cuts down the amount of images needing to be reimported from thousands to around a hundred.

It's also smaller in memory, since only one texture is loaded at a time. Scenes load faster and are smaller in memory, since the asset bundles are smaller. The image quality is better since the game now loads a better image compression into an uncompressed texture. The file size is much smaller since webp pictures only take a fraction of size compared to storing compressed textures.

It is possible to make the game even smaller by adjusting webp compression, but for now it won't be necessary, which significantly reduces the amount of data I have to upload per release. (~14GB vs 45+GB)

Builds are much faster as well as asset building, since Unity doesn't handle the textures anymore. It also has become a lot more responsive.

All images are automatically imported and converted accordingly, so no extra work is necessary from my side. It also has the advantage that I can automate the full building process and not just per platform. This will significantly reduce the probability of errors during the build process.

It's really a big relief for me, since the whole release process was getting extremely tedious. The amount of time, bandwidth and headaches saved will be worth the invested time almost immediately.

Sorry for the lengthy rambling. Thank you for reading, and thank you all for your support! Have a wonderful week,

- hachi