Join for free and connect with our local tech scene

Stay on top of the latest companies and upcoming events with our weekly newsletter, and be counted among the people building the future of your local tech community.

Boston, MA /

Scaling a high traffic database: Moving tables across clusters

Bryana Knight from GitHub will present "Scaling a high traffic database: Moving tables across clusters".

While GitHub isn't the biggest database around in terms of the amount of data we hold in MySQL, it is among the top 50 busiest sites on the internet. Facing an immediate need to distribute load, we came up with creative ways to move significant amount of traffic off of our main MySQL cluster, with no user impact. Moving 5 of our hottest tables required collaboration between engineers, DBAs and SRE. This talk will describe when and how to do it, and prove it to be an efficient database scalability solution.

Moving tables required changes to our database infrastructure as well as our application. I'll explain the impetus for this work and why we did it. We'll walk through the application-level changes that allowed us to change connections while still serving data. Then, I'll discuss the ways we moved tables to different clusters, using MySQL replication, or in some cases, temporary sharding and copying billions of rows. Finally, I'll outline the orchestration of the actual cutovers.

------------- Logistics -------------

MIT's map with the building (E51) highlighted: http://whereis.mit.edu/map-jpg?selection=E5... (http://whereis.mit.edu/map-jpg?selection=E51&Buildings=go)

MIT's map with the Kendall square T stop highlighted: http://whereis.mit.edu/map-jpg?selection=L5... (http://whereis.mit.edu/map-jpg?selection=L5&Landmarks=go) (the stop is in red on that map, and you can see E51 in the bottom right)

MIT's maps with parking lot information: http://whereis.mit.edu/map-jpg?selection=P4... (http://whereis.mit.edu/map-jpg?selection=P4&Parking=go) and http://whereis.mit.edu/map-jpg?selection=P5... (http://whereis.mit.edu/map-jpg?selection=P5&Parking=go)

Note that MIT lots are free to park in after 3 pm or so, even if it says "reserved for faculty"

Free pizza and soda will be served, so please RSVP accurately.

To RSVP anonymously, please login to the Meetup site with the e-mail address "admin at sheeri dot com" and the password "guest". For more information, see: http://www.meetup.com/mysqlbos/calendar/

Comments

Sign in to comment.