Chengze Li

Code Once, Think Twice

2 minute read

Problem

A typical system consists of following components:

  • a client that request the services
  • service hosts that entertain client request
  • database used by service for data storage

Database usually can become the bottleneck when the number of users and request increase.

What Is Cache

Cache is the solution of above problem.

It’s a nonpersistent storage area use to keep the repeatedly read and written data.

Sounds familiar, right? The functionality cache is somehow like the CDN, both of them are trying to increase the system performance, and reduce the bandwidth usage of system.

So the performance is one of the most important property cache should provide. That’s why cache store the data in RAM.

Design

Requirements

the distributed cache need to fulfill following requirements

  • insert data, the distributed cache should be able to accept the write operation from client, or from origin server.
  • respond request: distributed cache should be able to return the result for client request.

Challenging

  • client need to realize the addition or failure of a cache server.
  • we should avoid SPOF, and imbalanced distribution

Components

  • cache client: act as the entry point of overall cache system
  • cache server: where we actually store the data
  • configuration service: used to update the cache client about the situations of all servers
  • monitoring service: used to monitor overall system healthiness

Integration

  • clients’ request reach the service hosts through the load balancer where the cache client reside
  • each cache client uses consistent hashing to identify the cache server.
  • cache client forward the request to the cache server maintaining the specific shard.
  • each cache server has primary and replica servers.
  • configuration service ensure that all client see the updated and consistent view of the cache servers
  • monitoring service can be used to log and report the metrics of caching service

Evaluation

remember, the main goal we introduce the cache is reduce the perceived latency, and reduce the burden of origin servers.

Performance

the considerations to increase the performance

  1. the data is stored in RAM, which is much faster than DB which stored in hard drive
  2. the communication between cache client and cache server should be fast, which can be achieved through RPC or UDP

Availability

  • Redundancy: since we build multiple copies of original data, say, cache itself has the copy of data, we also perform replication for each cache server.
  • Configuration: configuration service can also make sure to provide the available data as much as possible.

Scalability

besides the extensibility of system, another important factor is “hot key”

we can introduce dynamic replication for specific keys to resolve the hot key.

Retrospect

Cache server is very similar to CDN, the difference is origin server don’t need to publish the content into cache.

Besides, all the learnings and properties of K-V store is also applicable to cache, say, the consistent hashing, etc.

You May Also Enjoy