What types of systems have to “scale up” rather than “scale out”?

Question

I have been wondering for a long time if there are systems that have to "scale up" (onto a more powerful, more expensive server) rather than "scale out" by being split across many smaller servers.

Do such systems exist, and if so, is there anything in particular that tends to cause systems to need to be scaled up, rather than scaled out? (For example, maybe ACID-complaint database transactions, or other strong data integrity requirements create this need.)

Since scaling up seems like it would bring much higher hardware costs than scaling out, it seems like something you would want to avoid, if possible, but I'm not sure if it is always avoidable or not.

So, are there systems that can't be scaled out, and have to be scaled up instead? What can cause this, and how would you identify such a system? (Do they generally share some characteristics in common that might make them more easily identifiable?)

Answer 1

I primarily work with an application that has zero horizontal scaling potential. Even though it runs on Linux, the application, data structures and I/O requirements force me to "scale up" onto progressively larger systems in order to accommodate increased user workloads.

Many legacy line-of-business and transactional applications have these types of constraints. It's one reason I stress that the industry focus on cloud solutions and DevOps-driven web-scale architectures ignores a good percentage of the computing world.

Unfortunately, the scale-up systems I describe are really unsexy, so the industry tends to ignore their value or deemphasize the skills needed to address large, critical systems (e.g. cattle versus pets).

Answer 2

From a developer perspective I can say that nearly every traditional mainstream database engine out there can only scale up and scaling out is very much an after thought.

In recent years with the need for greater scalability and highly available systems there have been efforts to make existing databases scale out. But because the designs are hindered by legacy code, it's very much just bolted on rather than fundamental to the design. You'll encounter this if try to scale most of the well known database engines. Adding slave servers can be quite difficult to set up and you'll notice that it comes with significant limitations, some of which may require re-jigging your database tables.

For example, most of them are master/(multi-)slave rather than multi-master designs. In other words, you might just have an entire server just sitting there and not able to process queries. Some do, but with limitations... e.g. read only multi-slave design. So you might have one server that takes writes and all the others provide read-only data. You'll notice when you set these systems up it's not always a straight forward process and difficult to get working well. It feels very much a bolt on addition in many cases.

On the other hand, there are some newer database engines being developed with concurrency and multi-master design from the beginning. NOSQL and NewSQL are the new design class.

So it would seem the best way to get better performance from a traditional SQL server is scale up! While with NOSQL & NewSQL it's both scale up & scale out.

The reason traditional RDBMS systems are tightly coupled is because they all need a consistent view of the same data. When you have multiple servers accepting updates to the same data from different clients, which one do you trust? Any method that attempts to ensure that the data is consistent through some sort of locking mechanism requires cooperation from other servers that either hurts performance or affects data quality in that any data read from a client might be out of date. And the servers need to decide among themselves which data is most recent when writing to the same record. As you can see it's a complex problem made more complex by the fact that the workload is spread across servers and not just among processes or threads where access to the data is still quite fast.

Answer 3

In my opinion the scale up/out demarcation is determined on how parallel a workflow can be, and how tightly the parallel threads need to coordinate with each other.

Single Threaded
For whatever reason, this workflow can only work in a single thread.

One thread means one system means scale up to make it go faster.

Tightly coupled parallelism
This is a multi-threaded system where the threads need to be tightly coupled with each other. Perhaps inter-process-communication needs to be very fast, or it all needs to be managed through a single memory manager. Most RDBMS systems are this kind of parallel computing.

For the most part, these systems are ones that scale up better than out though there are exceptions. For instance, workflows that would work on a Single System Image style cluster, single memory space but high latency between threads, may make scaling out easier. But such SSI systems are very tricky to work with so most engineers just make a bigger box.

Loosely coupled parallelism
This is a multi-threaded/process system where the threads are OK with high latencies between each other. Or don't need to talk to each other at all. Scaled out web-serving and render-farms are classic examples of this kind of system. Systems like these are a lot easier to make bigger than tightly-coupled parallelism, which is why there is a lot of excitement about this style of system.

This is the style where scale out is a lot easier.