System Design: Building Blocks - Load Balancer

What is Load Balancing

When tons of requests arrive to our server, how the incoming request can be divided among all the available servers?

Load balancing is the answer, the job of load balancer is fairly divide the requests among the available servers.

Algorithm of Load Balancing

The algorithm of LB can be static or dynamic, depending on the machines’ state.

static algorithm

in static algorithm, LB don’t consider the status changing from servers, like their work load, resource using etc.

dynamic algorithm

the current and recent status of server are considered. LB maintain the state by communicating with server.

in practice, dynamic algorithm provides far better results because they maintain the state of serving hosts, which worth the effort and complexity.

stateful algorithm

there is another division way which is considering whether to maintain the information of session established between client and hosting servers.

Stateful LB maintain the data structure that maps incoming client to hosting servers.

while this function could increase the complexity, and limit the scalability because session information of all clients is maintained across all the load balancers.

Types of Load Balancers

Based on the requirement, load balancing can be performed on network/transport level, or application level.

Layer 4 LB

it performs the load balancing on layer 4 of OSI, which is transport protocol like TCP or UDP.

These kind of LB maintain the connection/session with the client, and ensures the same communication will be forwarded to same hosting server.

For example, LVS is a Layer-4 LB

Layer 7 LB

it’s based on the data of application layer protocol. it’s based on HTTP headers, URL or other application specific data, for example user id.

Nginx is an example of Layer 7 LB

Evaluation

scalability

when we introduce the load balancer, we can seamlessly increase or decrease the number of server.

availability

with the load balancer, we can also increase the availability of overall system, because load balancer can forward the request to healthy servers.

performance

Load balancer can also increase the performance of overall system, because it can forward the request to the servers with less load.