Today's Downtime
Added 2021-01-01 23:25:32 +0000 UTCHi all -
Today we had some extended downtime on the All Bad Cards server. Unfortunately, I was not around to fix it immediately as I was enjoying some time with my wife, who finally had a day off after working on Christmas (she's a nurse). We're a very small operation, and I'm the only developer, so unfortunately things like this are very dependent on my free time to fix them.
I was able to determine the cause of the problem, which was a failure on our database server that caused it to refuse connections from the rest of the servers. This prevented the site from loading packs, user data, and game data. Up through now, I've never seen an error like this happen. In the meantime, I spun up a second database server that we will use instead, and I'm going to be trying to diagnose the problem that happened with the other server in the coming days.
It definitely sucks to deal with downtime, and I don't want any of you to have that happen, especially during the holidays. Of course, if you became a subscriber during that time for a party today, only to have that plan foiled, we're happy to refund this month's subscription for you. We want to treat you all well! <3
I'll continue to do my best to keep the site running all the time, of course. Last night we had MASSIVELY more traffic than any other time (nearly 4x what we hit during Christmas) and it revealed some cracks in our setup that we'll have to try to fix up. Back when I started this site, I really had no idea how to deal with the traffic I was getting at that time. The server used to die every weekend! Since then, I was able to build a system that reacted to user counts pretty well. Looks like it's time to start that process again :)
Much love to you all, and thanks for the continued support. You're the best! Happy New Year to you all.
- Jake
