A modern HTTP server running on somewhat recent hardware is capable of servicing a huge number of requests with very low latency. Here’s a plot showing requests per second vs. number of concurrent connections for the default index.html page included with nginx 1.0.14.
With this particular hardware & software combination the server quickly reaches over 500,000 requests/sec and sustains that with gradually increasing latency. Even at 1,000 concurrent connections, each requesting the page as quickly as possible, latency is only around 1.5ms.
The plot shows the average requests/sec and per-request latency of 3 runs of wrk -t 10 -c N -r 10m http://localhost:8080/index.html where N = number of connections. The load generator is wrk, a scalable HTTP benchmarking tool.
Software
The OS is Ubuntu 11.10 running Linux 3.0.0-16-generic #29-Ubuntu SMP Tue Feb 14 12:48:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux. The following kernel parameters were changed to increase the number of ephemeral ports, reduce TIME_WAIT, increase the allowed listen backlog, and the number of connections Netfilter can track:
echo "2048 64512" > /proc/sys/net/ipv4/ip_local_port_range echo "1" > /proc/sys/net/ipv4/tcp_tw_recycle echo "1" > /proc/sys/net/ipv4/tcp_tw_reuse echo "10" > /proc/sys/net/ipv4/tcp_fin_timeout echo "65536" > /proc/sys/net/core/somaxconn echo "65536" > /proc/sys/net/ipv4/tcp_max_syn_backlog echo "262144" > /proc/sys/net/netfilter/nf_conntrack_max
The HTTP server is nginx 1.0.14 built with ./configure && make, and run in-place with objs/nginx -p . -c nginx.conf.
nginx.conf
worker_processes 16;
worker_rlimit_nofile 262144;
daemon off;
events {
use epoll;
worker_connections 16384;
}
error_log error.log;
pid /dev/null;
http {
sendfile on;
tcp_nopush on;
keepalive_requests 100;
open_file_cache max=100;
gzip off;
gzip_min_length 1024;
access_log off;
server {
listen *:8080 backlog=16384;
location / {
root html;
index index.html;
}
}
}
Hardware
A dual Intel Xeon X5670 with 24GB of RAM from SoftLayer. The X5670 has 6 cores @ 2.93 GHz, 2 threads per core, /proc/cpuinfo shows 24 CPUs.
