By default, web browsers limit the number of active connections for each domain. When the number of resources to download exceeds that limit, users experience slow page load times as downloads are queued.
To work around this limitation, developers may split content across multiple subdomains. Since browsers reset the connection limit for each domain, each additional domain allows an additional number of active connections. This lets the user retrieve files from the same source with greater throughput.
When a user connects to a web page, his or her browser scans the resulting HTML for resources to download. Normally these resources are supplied by a single domain – the domain providing the web page or a domain created specifically for resources. With domain sharding, the user’s browser connects to two or more different domains to simultaneously download the resources needed to render the web page.
The number of shards used by a web service depends on the number of resources the user is required to download. Modern web browsers support on average 6 concurrent downloads per domain. This number is divided across the total number of resources to give the total number of requests. For example, a website with 30 resources has to do 6 sequential requests for a single user.
Adding multiple domains can, however, introduce performance losses. Web browsers have to perform a DNS lookup on each additional domain and maintain connections to each domain, resulting in slower initial load times. A study by Yahoo! shows 2 to 4 domains as the optimal ratio, with additional domains resulting in degraded performance.
Google’s SPDY protocol introduced support for unlimited concurrent requests, solving the same issues as domain sharding with no additional setup. However, in 2016, SPDY was deprecated and HTTP/2 was released in its place. While HTTP/2’s increased adoption will eventually make domain sharding obsolete, older browsers will continue to rely on domain sharding for increased performance.
YouTube splits images and script resources across two domains: i.ytimg.com and s.ytimg.com. i.ytimg.com contains icons, logos, thumbnails, avatars, and other visual elements while s.ytimg.com contains JavaScript, CSS, favicons, sprite sheets, and objects related to website optimization and analytics. Users visiting youtube.com will simultaneously download the scripts and media needed to render the page.
Because browsers distinguish domains by name rather than by IP address, domain sharding can be performed by a single web server. As long as the server has the bandwidth to support multiple concurrent connections, it can take advantage of domain sharding to quickly distribute content to visitors.
An average of 50 requests are made on a single domain for the web’s top 300,00 URLs. That’s a lot of time spent waiting on resources to download, especially when some downloads can only start once others finish. Domain sharding can boost a website’s performance without adding a substantial amount of complexity.
