Chengze Li

Code Once, Think Twice

2 minute read

For the opt-in users who grants the permission to access their location, the mobile client present a list of friends who are geographically nearby.

Step1: Scope the problem

Let’s assume the core features to design are:

  1. users should be able to see nearby friends on their mobile app
  2. friends list should be updated every few seconds

Some back of the envelop estimation:

  1. 100M DAU
  2. 10% concurrent user
  3. user reports their location every 30s
  4. location update QPS ~= 334,000

Step2: High level design

The communication protocol between client and server might not be straightforward HTTP protocol.

Conceptually, each user would like to receive location updates from every active friends nearby. This can be a peer to peer connection, while due to the flaky connection of mobile, we should design a more stable backend for the location communication.

Components

  • web socket server: this is a cluster of stateful server that handles the near real-time of friends’ location, by maintaining a persistent web socket connection with each client.
  • location cache: used to store the recent location data for active user, we set TTL on it, as long as we don’t receive the update in this period, we will invalid this cache.
  • user DB: used to store user data and user friendship data.
  • pub-sub: used to notify the friends when user’s location has been updated

Integration

  1. mobile client send location update to load balancer
  2. load balancer forward the request to WebSocket server for that client
  3. WebSocket server save the location data to location history DB
  4. WebSocket server update the new location in the location cache and update its TTL.
  5. WebSocket server publish a new location to user’s channel
  6. Pub-sub system broadcast the update to all the subscribers (WebSocket connection handler), for each subscriber (user’s friends), its WebSocket connection handler will receive the user location update.
  7. On receiving the message, WebSocket server will compute the distance and send to client
  8. Based on the distance, we either update the distance on client, or drop the friend from the list

Step3: Design deep dive

How well each component scale?

  • Web Server: stateless server, so should be very easy to scale up
  • WebSocket Server: not difficult to scale up as well, while these servers are stateful, so care must be taken to remove existing nodes.
  • Pub-sub Server: the good news is that the channels are independent of each other, this makes it relatively easy to spread the channel among multiple pub-sub servers by sharding.
    • besides, we should treat pub/sub cluster more like a stateful cluster. because each channels stores its subscriber list.

Client Initialization

  1. WebSocket connection are built between client and server
  2. updates the user’s location in location cache
  3. loads all users’ friends from user DB
  4. make a batched request to the location cache to fetch the location for all the friends
  5. for each returned location, compute the distance, return those friends within the search radius
  6. for each friend, server subscribes to the friends channel in pub-sub server.
  7. send user’s current location to user’s channel in pub-sub server

Step4: Wrap Up

This article introduces how to design a nearby friends app, in my opinion, the core part is how we update the location of user, and how to notify their user regarding the location update.

Besides, there is an interesting topic:

Nearby random persons

One way is we can add a pool of pub-sub channels by geo-hash.

Anyone will subscribe the channel of their geo-hash, one user’s location update will notify all users who subscribe this channel

You May Also Enjoy