diff --git a/doc/images/load-balancing.png b/doc/images/load-balancing.png index 18b68bfdad83d..7c70465e653dc 100644 Binary files a/doc/images/load-balancing.png and b/doc/images/load-balancing.png differ diff --git a/doc/images/load-balancing.svg b/doc/images/load-balancing.svg index 425a9d33aa71c..18d836d345bbb 100644 --- a/doc/images/load-balancing.svg +++ b/doc/images/load-balancing.svg @@ -1,4 +1,4 @@ - + diff --git a/doc/load-balancing.md b/doc/load-balancing.md index c5e59331c2da4..1071961b1166b 100644 --- a/doc/load-balancing.md +++ b/doc/load-balancing.md @@ -1,17 +1,25 @@ Load Balancing in gRPC -======================= +====================== -# Objective +# Scope -To design a load balancing API between a gRPC client and a Load Balancer to -instruct the client how to send load to multiple backend servers. +This document explains the design for load balancing within gRPC. # Background +## Per-Call Load Balancing + +It is worth noting that load-balancing within gRPC happens on a per-call +basis, not a per-connection basis. In other words, even if all requests +come from a single client, we still want them to be load-balanced across +all servers. + +## Approaches to Load Balancing + Prior to any gRPC specifics, we explore some usual ways to approach load balancing. -## Proxy Model +### Proxy Model Using a proxy provides a solid trustable client that can report load to the load balancing system. Proxies typically require more resources to operate since they @@ -21,7 +29,7 @@ latency to the RPCs. The proxy model was deemed inefficient when considering request heavy services like storage. -## Balancing-aware Client +### Balancing-aware Client This thicker client places more of the load balancing logic in the client. For example, the client could contain many load balancing policies (Round Robin, @@ -41,7 +49,7 @@ It would also significantly complicate the client's code: the new design hides the load balancing complexity of multiple layers and presents it as a simple list of servers to the client. -## External Load Balancing Service +### External Load Balancing Service The client load balancing code is kept simple and portable, implementing well-known algorithms (e.g., Round Robin) for server selection. @@ -104,9 +112,7 @@ works: a load balancer address, and a [service config](service_config.md) that indicates which client-side load-balancing policy to use (e.g., `round_robin` or `grpclb`). -2. The client instantiates the load balancing policy, which is then - responsible for deciding which requests will be sent to which - addresses. +2. The client instantiates the load balancing policy. - Note: If all addresses returned by the resolver are balancer addresses, then the client will use the `grpclb` policy, regardless of what load-balancing policy was requested by the service config. @@ -114,19 +120,28 @@ works: by the service config. If no load-balancing policy is requested by the service config, then the client will default to a policy that picks the first available server address. -3. In the case of the `grpclb` policy, it opens a stream to one of the - balancer addresses returned by the resolver. It asks the balancer for - the server addresses to use for the server name originally requested by - the client (i.e., the same one originally passed to the name resolver). - - Note: Currently, the `grpclb` policy ignores any non-balancer - addresses returned by the resolver. However, in the future, it may - be changed to use these addresses as a fallback in case no balancers - can be contacted. -4. The gRPC servers to which the load balancer is directing the client - may report load to the load balancers, if that information is needed - by the load balancer's configuration. -5. The load balancer returns a server list to the gRPC client. If the - server list is empty, the call will block until a non-empty one is - received. -6. The gRPC client will send RPCs to the gRPC servers contained in - the server list from the Load Balancer. +3. The load balancing policy creates a subchannel to each server address. + - For all policies *except* `grpclb`, this means one subchannel for each + address returned by the resolver. Note that these policies + ignore any balancer addresses returned by the resolver. + - In the case of the `grpclb` policy, the workflow is as follows: + a. The policy opens a stream to one of the balancer addresses returned + by the resolver. It asks the balancer for the server addresses to + use for the server name originally requested by the client (i.e., + the same one originally passed to the name resolver). + - Note: The `grpclb` policy currently ignores any non-balancer + addresses returned by the resolver. However, in the future, it + may be changed to use these addresses as a fallback in case no + balancers can be contacted. + b. The gRPC servers to which the load balancer is directing the client + may report load to the load balancers, if that information is needed + by the load balancer's configuration. + c. The load balancer returns a server list to the gRPC client's `grpclb` + policy. The `grpclb` policy will then create a subchannel to each of + server in the list. +4. For each RPC sent, the load balancing policy decides which + subchannel (i.e., which server) the RPC should be sent to. + - In the case of the `grpclb` policy, the client will send requests + to the servers in the order in which they were returned by the load + balancer. If the server list is empty, the call will block until a + non-empty one is received. diff --git a/doc/naming.md b/doc/naming.md index 588f611f1633d..676aa9f298068 100644 --- a/doc/naming.md +++ b/doc/naming.md @@ -20,12 +20,18 @@ uses the syntax: scheme://authority/endpoint_name ``` -Here, `scheme` indicates the name-system to be used. Example schemes to -be supported include: +Here, `scheme` indicates the name-system to be used. Currently, we +support the following schemes: -* `dns` +- `dns` -* `etcd` +- `ipv4` (IPv4 address) + +- `ipv6` (IPv6 address) + +- `unix` (path to unix domain socket -- unix systems only) + +In the future, additional schemes such as `etcd` could be added. The `authority` indicates some scheme-specific bootstrap information, e.g., for DNS, the authority may include the IP[:port] of the DNS server to @@ -38,7 +44,7 @@ syntax of the endpoint name is dictated by the scheme in use. ### Resolver Plugins -The gRPC client library will switch on the scheme to pick the right +The gRPC client library will use the specified scheme to pick the right resolver plugin and pass it the fully qualified name string. Resolvers should be able to contact the authority and get a resolution @@ -46,7 +52,7 @@ that they return back to the gRPC client library. The returned contents include: - A list of resolved addresses, each of which has three attributes: - - The address itself, in IP:port form. + - The address itself, including both IP address and port. - A boolean indicating whether the address is a backend address (i.e., the address to use to contact the server directly) or a balancer address (for cases where [external load balancing](load-balancing.md)