You started out as a proof of concept that exploded and now have a product that is used by millions.

Let’s talk about scaling your data due to this explosion.

What is horizontal scaling?

You started out with a few users, and that small instance on your cloud vendor was more than enough to ensure performance your 200 megabytes of data.

But then you were listed on Reddit and over the next several months you were scrambling to keep up.

You were scaling up by increasing instance sizes to add more resources every other week.

Now you have reached a point where its hard to make changes without downtime, and this reduces your velocity.

And the performance of the application is degrading as well.

Without new features to keep your users happy, and with degrading performance, you risk losing them.

What do you do?

When you were adding resources by increasing instance sizes, you were vertically scaling.

But as you found out, there’s a practical limit to how far you can scale up.

So now you need to consider scaling out. This is known as horizontal scaling.

How do I scale horizontally?

Horizontally scaling is simply splitting up your data to multiple servers. At this point, your data doesn’t completely live on a single server.

Another term for horizontal scaling your data is sharding. Some options for horizontal scaling might be:

• If you are a Software as a Service (SaaS) company, you might be able to move certain accounts to one server. And another set to a different server. This is an example of multi-tenant sharding.

• Or you might be able to put your user data on one server, and their interaction data on another. This is an example of functional sharding.

• Another option is to split the data that is closely related by time or location.

As you can see, there are many options to scaling horizontally.

Complexities of horizontal scaling databases

Scaling out gives you the flexibility to grow as you need to.

But as is often the case, that flexibility comes at the cost of complexity.

This complexity can be in:

• operations for managing schema changes, patching and upgrading, and even user management.

• making sure your availability objectives are met across your data landscape.

• the applications for knowing how and where to look for specific data, and the software in between that helps with this.

Conclusion

At some point, it is likely you will have to consider scaling out your data.

The way you horizontally scale will depend on your requirements, but you will need to plan for the additional complexity.

The Data Guardian will address the complexities with automation and sound principles of data availability, security and performance.