We have completed the bulk of the system and infrastructure upgrades that started on December 16th.
What was done?
There were three parts to the upgrade, completed in two phases:
- We upgraded almost half of our cloud servers to the latest and greatest machines with the latest stable versions of all security, application, and server software.
- We upgraded databases to a much faster, more scalable database engine.
- We moved core servers to the newest available facility within our cloud environment at IBM, which provides additional speed, fault tolerance, and manageability.
Why was this done?
When newer technology becomes available, we evaluate it to assess the benefit of upgrading, compared to the cost to you (primarily in terms of potential service disruption). We have some hard-and-fast rules on how old we will allow our technology to get before we replace it proactively, but usually there is a compelling case to upgrade sooner. In this case, we had the chance to take advantage of a number of benefits, all in a single upgrade window (minimum impact to our members). This was the biggest single upgrade we’ve undertaken since 2012.
Additionally, we are scaling up dramatically for the upcoming public release of HERO Harmony. Since Harmony includes accounting, CRM, and all of the other features of HERO PM, it requires a boatload of capacity to securely handle all of the transactions and messages that you deal with daily. This step allows us to smoothly jump a few rungs up the ladder, ensuring that your business can grow with us, or continue reliably and securely.
How did it all go?
All things considered, pretty good. We did have a couple of problems that deserve an apology:
- We were a little too ambitious on the configuration of the new database engine. This left some users with the inability to add new leases, reservations, workorders, and listings for a few hours until we could adjust the settings to exactly where they needed to be.
- Distributing the data over to new database/facility took longer than we planned for. Although the new database engine is blazing fast, the old one is not quite as amazing. We ran multiple tests and determined that we could get all of the data from the old to the new in plenty of time to ensure no disruption, but in real life, it didn’t work out as planned. This meant that it was still in process when users showed up for work Tuesday morning. The result was that we had to keep some systems on the old database engine for a part of the business day Tuesday. The impact to you was that things were slow – not what you want to see when returning to a wonderful new upgrade that is supposed to improve speed!
What still needs to happen?
We’re still doing some fine-tuning. There is nothing that we expect to impact users, and it will get even faster as we make a few minor adjustments.