You may have noticed a few site slowdowns over the last couple of months. This is due to the really unexpectedly high growth of the site.
Last month we upgraded our server to double the amount of memory, which we thought would solve the problem, but it rapidly returned. This Friday we found that the cause was our mailserver. We now have over 6,000 subscribers to our smallest genre, and the system was taking too long to process the emails. On Saturday we discovered we still had 12,000 queued emails as the system simply could not send them fast enough and was having to stagger them over two days. This causes a huge overhead and slows the entire site down.
After a chat, we have bitten the bullet and are moving from our self-hosted subscriber management entirely onto Sendgrid. This will remove the load from our servers, and at this level will actually be cheaper to run than our current list management.
This does give us a few advantages: lower bandwidth costs, mailing costs, and better subscriber management of unsubscribe and bounces/blocked reports. The other thing we plan to do is add an extra mailing list that users can opt in to when submitting books to get updates about the site, new opportunities, and any issues that may occur. These won’t be sent often, but it means users get some warning before we perform upgrades.
Sendgrid is a little more complicated to set up, so please bear with us while we get the details running. The weekly reviews email, our largest list, has already been moved, as has the standard Free Books List. Personalised by Genre is currently in transition.
The first emails from the new system will be going out later today, so if you have any problems during this period, please let us know, and be patient with us. Its a brand new system so we are learning as we go, and hopefully it will make the site a lot better.
