Cassandra Advanced Database Migration

Description

A leading international dating site with a massive PHP codebase had a messaging system used for personal communication between its members, implemented in MySQL. For the first ten years of its existence, this was sufficient, but changes in the company's business model resulted in members sending and receiving thousands of messages per day. This quickly became a pain point in searching, writing, and storing the messages, all the waydown to how primary keys were used by the messaging system. A new solution was clearly needed. Any solution would not only need to address the pain points of the load, but also would need to take account of the fact that the site could not be allowed to experience downtime at any time during the rollout.

Parthenon developed a messaging system for the site based on Apache Cassandra paired with Elasticsearch, in a radical departure from conventional web-based handling of data. Cassandra has the property of initial insertion of data being extremely cheap, though updates and deletion are expensive and lower a table's read performance; reads are extremely cheap, with special restrictions to sorting and filtering. Parthenon used this ability of Cassandra to manage the message load, which consisted mostly of insertions, combined with a denormalized set of tables presorted and sharded in a manner that would allow messages to expire out after a period of time (a site requirement).

The project was developed in early 2017, inside of the existing codebase, running in parallel with the existing messaging system. We also created a set of scripts that performed the migrations of the 3.2TB of messaging data into Cassandra in the background. The new messaging system was deployed successfully and is in use today. As a side benefit of the new design, it is also fully scalable across multiple data centers.