Explorar o código

Add keep-alive parking (Server::set_keep_alive_parking())

A kept-alive connection holds a worker thread while it waits for its
next request, so connections beyond the thread count wait for one to be
closed, and some clients get nothing until another connection ends.

With keep-alive parking, listen() runs an event loop on the
detail::Poller instead of the blocking accept loop. A connection waiting
for its next request is parked there and goes back to a worker through
the TaskQueue when the request arrives. The handlers, content providers
and WebSocket keep their blocking API and still hold a thread while they
run. SSLServer supports it too: a worker runs the TLS handshake once the
client starts it, and the session stays with the parked connection.

It is opt-in and off by default. With it on:
- consecutive requests on a connection may run on different threads;
- when the queue is full, a connection that has not been served (or has
  not finished its TLS handshake) is closed as before, while an
  established one waits in a backlog instead of losing its request;
- a parked connection frees its read buffer, and with OpenSSL 3.0.14,
  3.1.6, 3.2.2, 3.3.1 or later also its TLS record buffers;
- handing a connection over adds roughly 10-20 us to a request;
- a subclass that overrides the private process_and_close_socket() is
  bypassed.

On two CPUs with two workers, 50 keep-alive clients of a 2.7 ms handler
were all served evenly (the slowest response took about 70 ms instead of
5 s, with 10 clients getting nothing), light requests from 50 clients
went from 79k to about 240k per second, and with 10000 idle connections
parked, 10 active clients got 95k requests per second where the old loop
served none. An idle connection costs under 1 KB, or about 15 KB with
TLS.

The old accept loop moved unchanged into accept_loop(). KeepAliveConnection
holds a parked connection, PlainConnection and SSLServer::TlsConnection
add the transport, and SSLServer::start_session() holds the session setup
and handshake that both paths use.
yhirose hai 1 día
pai
achega
b2a81c0019
Modificáronse 3 ficheiros con 1135 adicións e 126 borrados
  1. 18 1
      README.md
  2. 654 125
      httplib.h
  3. 463 0
      test/test.cc

+ 18 - 1
README.md

@@ -919,6 +919,23 @@ svr.set_keep_alive_max_count(2); // Default is 100, 0 for no limit
 svr.set_keep_alive_timeout(10);  // Default is 5
 ```
 
+#### Keep-alive parking
+
+```cpp
+svr.set_keep_alive_parking(true); // Call it before listen()
+```
+
+By default, a kept-alive connection holds a worker thread while it waits for its next request, so connections beyond the thread count wait for one to be closed. With keep-alive parking, a connection waiting for its next request is handed to an event loop instead, and goes back to a worker when the request arrives. An idle connection then costs a socket and a few kilobytes instead of a thread, and connections that outnumber the threads take turns.
+
+With it:
+
+- Consecutive requests on a connection may run on different threads.
+- When the [max queued requests](#max-queued-requests) limit is reached, only a connection that has not been served yet (or, with `SSLServer`, has not finished its TLS handshake) is closed. An established one waits for room.
+- Each request pays for handing the connection over, so with no more connections than threads and very light handlers, it can serve fewer requests per second.
+- A handler, a content provider and a WebSocket still hold their thread while they run.
+- A subclass that overrides the private `process_and_close_socket()` is bypassed.
+- `SSLServer` runs the TLS handshake on a worker when the client starts it. Until then the connection waits in the event loop and is closed at the keep-alive timeout, not the read timeout, and `ssl_last_error()` is not set for it.
+
 ### Client address behind a reverse proxy
 
 ```cpp
@@ -962,7 +979,7 @@ Please see [Server example](https://github.com/yhirose/cpp-httplib/blob/master/e
 
 When a new task arrives and the queued tasks outnumber the idle threads, a temporary thread is spawned (up to the maximum). A dynamic thread exits automatically once it has waited for the idle timeout without receiving a task. The idle timeout defaults to 3 seconds, configurable via `CPPHTTPLIB_THREAD_POOL_IDLE_TIMEOUT`.
 
-A connection holds its thread while it is served: between requests up to the keep-alive timeout, through a request body sent slowly but within the read timeout, and for the whole life of a WebSocket. Enough of them leave no thread for anyone else, so in front of untrusted clients, put a reverse proxy that limits connections per client, and reject unwanted WebSocket upgrades in the pre-request handler.
+A connection holds its thread while it is served: between requests up to the keep-alive timeout (unless [keep-alive parking](#keep-alive-parking) is on), through a request body sent slowly but within the read timeout, and for the whole life of a WebSocket. Enough of them leave no thread for anyone else, so in front of untrusted clients, put a reverse proxy that limits connections per client, and reject unwanted WebSocket upgrades in the pre-request handler.
 
 If you want to set the thread counts at runtime:
 

+ 654 - 125
httplib.h

@@ -2391,6 +2391,10 @@ public:
   template <class Rep, class Period>
   Server &set_idle_interval(const std::chrono::duration<Rep, Period> &duration);
 
+  // Keeps idle keep-alive connections in an event loop instead of a worker
+  // thread each. Call it before listen(). See README for what changes.
+  Server &set_keep_alive_parking(bool on);
+
   Server &set_payload_max_length(size_t length);
 
   Server &set_compress(bool on);
@@ -2461,6 +2465,13 @@ protected:
   // stop() takes the listening socket away; listen() then waits for handlers
   bool is_stopping() const { return svr_sock_ == INVALID_SOCKET; }
 
+  // See set_keep_alive_parking(). Defined in the implementation part.
+  struct KeepAliveConnection;
+  struct ParkingState;
+  std::unique_ptr<ParkingState> parking_;
+  bool keep_alive_parking_ = false;
+  bool idle_interval_set_ = false;
+
   std::vector<std::string> trusted_proxies_;
 
   size_t keep_alive_max_count_ = CPPHTTPLIB_KEEPALIVE_MAX_COUNT;
@@ -2520,6 +2531,12 @@ private:
                                 SocketOptions socket_options) const;
   int bind_internal(const std::string &host, int port, int socket_flags);
   bool listen_internal();
+  bool accept_loop(TaskQueue &task_queue);
+  bool parking_loop(TaskQueue &task_queue);
+  void prepare_accepted_socket(socket_t sock) const;
+  struct PlainConnection;
+  void resume_connection(std::shared_ptr<KeepAliveConnection> conn);
+  bool abandon_listen_socket();
 
   bool routing(Request &req, Response &res, Stream &strm,
                bool skip_pre_routing);
@@ -2567,6 +2584,9 @@ private:
                          ContentReceiver multipart_receiver) const;
 
   virtual bool process_and_close_socket(socket_t sock);
+  // The connection that parking_loop() keeps for a socket it has accepted.
+  virtual std::shared_ptr<KeepAliveConnection>
+  new_keep_alive_connection(socket_t sock);
 
   void output_log(const Request &req, const Response &res) const;
   void output_pre_compression_log(const Request &req,
@@ -3527,6 +3547,12 @@ public:
 
 private:
   bool process_and_close_socket(socket_t sock) override;
+  std::shared_ptr<KeepAliveConnection>
+  new_keep_alive_connection(socket_t sock) override;
+  struct TlsConnection;
+  // Creates a session and runs the handshake. Null on failure, which sets
+  // last_ssl_error_.
+  tls::session_t start_session(socket_t sock);
 
   tls::ctx_t ctx_ = nullptr;
   // update_certs_pem() replaces the certificates and CAs that handshakes read
@@ -7193,6 +7219,10 @@ public:
   // question that was answered a moment ago. Consumed by that read.
   void set_readable_hint() { readable_hint_ = true; }
 
+  // Frees the read buffer of an idle stream until its next read. Nothing may
+  // be buffered.
+  void release_buffer();
+
 private:
   bool ensure_readable();
 
@@ -7249,30 +7279,27 @@ inline bool keep_alive(const std::atomic<socket_t> &svr_sock, socket_t sock,
   return false;
 }
 
-// `has_buffered_request` reports whether the connection's stream already holds
-// bytes of the next request. A client may pipeline its requests (RFC 9112
-// 9.3.2), so reading one request can pull the start of the next one into the
-// stream's buffer; that request must be served without waiting for the socket
-// to become readable again, since its bytes are no longer on the socket.
-// Otherwise `wait_for_next_request` waits for the socket, and returning false
-// ends the loop. `callback` is told whether the socket has just been seen
-// readable. `served` counts the requests served on this connection so far.
-template <typename W, typename P, typename T>
-inline bool process_server_socket_core(size_t &served,
+// Serves the requests of one connection while they come. A client may
+// pipeline its requests (RFC 9112 9.3.2), so reading one request can pull the
+// start of the next one into the stream's buffer; that request must be served
+// without waiting for the socket to become readable again, since its bytes are
+// no longer on the socket. Otherwise `wait_for_next_request` waits for the
+// socket, and returning false ends the loop. `served` counts the requests
+// served on this connection so far.
+template <typename S, typename W, typename T>
+inline bool process_server_socket_core(S &strm, size_t &served,
                                        size_t keep_alive_max_count,
-                                       W wait_for_next_request,
-                                       P has_buffered_request, T callback) {
+                                       W wait_for_next_request, T callback) {
   auto ret = false;
   // A keep_alive_max_count of 0 sets no limit.
   for (; keep_alive_max_count == 0 || served < keep_alive_max_count; served++) {
-    auto socket_readable = false;
-    if (!has_buffered_request()) {
+    if (!strm.is_readable()) {
       if (!wait_for_next_request()) { break; }
-      socket_readable = true;
+      strm.set_readable_hint();
     }
     auto close_connection = served + 1 == keep_alive_max_count;
     auto connection_closed = false;
-    ret = callback(socket_readable, close_connection, connection_closed);
+    ret = callback(strm, close_connection, connection_closed);
     if (!ret || connection_closed) { break; }
   }
   return ret;
@@ -7291,14 +7318,9 @@ process_server_socket(const std::atomic<socket_t> &svr_sock, socket_t sock,
                     write_timeout_sec, write_timeout_usec);
   size_t served = 0;
   return process_server_socket_core(
-      served, keep_alive_max_count,
+      strm, served, keep_alive_max_count,
       [&]() { return keep_alive(svr_sock, sock, keep_alive_timeout_sec); },
-      [&]() { return strm.is_readable(); },
-      [&](bool socket_readable, bool close_connection,
-          bool &connection_closed) {
-        if (socket_readable) { strm.set_readable_hint(); }
-        return callback(strm, close_connection, connection_closed);
-      });
+      callback);
 }
 
 inline bool process_client_socket(
@@ -7325,8 +7347,10 @@ inline int shutdown_socket(socket_t sock) noexcept {
 // the final shutdown+close. Closing with unread data in the receive queue
 // (or bytes arriving after the receive side is closed) makes the stack send
 // an abortive RST instead of a graceful FIN, which can make the peer see the
-// response as a failed read even though it was fully written.
-inline void drain_and_close_socket(socket_t sock) noexcept {
+// response as a failed read even though it was fully written. With a
+// `wait_msec` of 0, it reads once, without waiting.
+inline void drain_and_close_socket(socket_t sock,
+                                   time_t wait_msec = 100) noexcept {
 #ifdef _WIN32
   shutdown(sock, SD_SEND);
 #else
@@ -7336,19 +7360,21 @@ inline void drain_and_close_socket(socket_t sock) noexcept {
   char buf[CPPHTTPLIB_RECV_BUFSIZ];
   size_t total = 0;
   const auto deadline = std::chrono::steady_clock::now() +
-                        std::chrono::milliseconds(100); // bound #1
-
-  while (total < size_t(1024u * 1024u)) { // bound #2
-    const auto remaining =
-        std::chrono::duration_cast<std::chrono::microseconds>(
-            deadline - std::chrono::steady_clock::now())
-            .count();
-    if (remaining <= 0) { break; }
-    if (select_read(sock, 0, static_cast<time_t>(remaining)) <= 0) { break; }
+                        std::chrono::milliseconds(wait_msec); // bound #1
+  auto usec_left = [&]() {
+    const auto left = std::chrono::duration_cast<std::chrono::microseconds>(
+                          deadline - std::chrono::steady_clock::now())
+                          .count();
+    return static_cast<time_t>(
+        (std::max)(left, std::chrono::microseconds::rep(0)));
+  };
+
+  do {
+    if (select_read(sock, 0, usec_left()) <= 0) { break; }
     const auto n = read_socket(sock, buf, sizeof(buf), CPPHTTPLIB_RECV_FLAGS);
     if (n <= 0) { break; }
     total += static_cast<size_t>(n);
-  }
+  } while (usec_left() > 0 && total < size_t(1024u * 1024u)); // bound #2
 
   shutdown_socket(sock);
   close_socket(sock);
@@ -7899,6 +7925,26 @@ inline bool is_accept_transient_error() {
 #endif
 }
 
+inline socket_t accept_socket(socket_t svr_sock) {
+#if defined _WIN32
+  // sockets connected via WASAccept inherit flags NO_HANDLE_INHERIT,
+  // OVERLAPPED
+  return WSAAccept(svr_sock, nullptr, nullptr, nullptr, 0);
+#elif defined SOCK_CLOEXEC
+  return accept4(svr_sock, nullptr, nullptr, SOCK_CLOEXEC);
+#else
+  return accept(svr_sock, nullptr, nullptr);
+#endif
+}
+
+inline bool is_would_block_error() {
+#ifdef _WIN32
+  return WSAGetLastError() == WSAEWOULDBLOCK;
+#else
+  return errno == EAGAIN || errno == EWOULDBLOCK;
+#endif
+}
+
 inline bool create_socket_pair(socket_t sv[2]) {
 #ifdef _WIN32
   auto listener = create_socket(
@@ -11632,14 +11678,9 @@ inline bool process_server_socket_ssl(
                        write_timeout_sec, write_timeout_usec);
   size_t served = 0;
   return process_server_socket_core(
-      served, keep_alive_max_count,
+      strm, served, keep_alive_max_count,
       [&]() { return keep_alive(svr_sock, sock, keep_alive_timeout_sec); },
-      [&]() { return strm.is_readable(); },
-      [&](bool socket_readable, bool close_connection,
-          bool &connection_closed) {
-        if (socket_readable) { strm.set_readable_hint(); }
-        return callback(strm, close_connection, connection_closed);
-      });
+      callback);
 }
 
 template <typename T>
@@ -13369,8 +13410,7 @@ inline SocketStream::SocketStream(
       read_timeout_usec_(read_timeout_usec),
       write_timeout_sec_(write_timeout_sec),
       write_timeout_usec_(write_timeout_usec),
-      max_timeout_msec_(max_timeout_msec), start_time_(start_time),
-      read_buff_(read_buff_size_, 0) {}
+      max_timeout_msec_(max_timeout_msec), start_time_(start_time) {}
 
 inline SocketStream::~SocketStream() = default;
 
@@ -13410,6 +13450,11 @@ inline const char *SocketStream::buffered_data(size_t &size) const {
   return size ? read_buff_.data() + read_buff_off_ : nullptr;
 }
 
+inline void SocketStream::release_buffer() {
+  assert(read_buff_off_ == read_buff_content_size_);
+  std::vector<char>().swap(read_buff_);
+}
+
 inline void SocketStream::consume_buffered(size_t size) {
   assert(size <= read_buff_content_size_ - read_buff_off_);
   read_buff_off_ += size;
@@ -13450,6 +13495,8 @@ inline ssize_t SocketStream::read(char *ptr, size_t size) {
   read_buff_content_size_ = 0;
 
   if (size < read_buff_size_) {
+    // Allocated on the first use, which an idle connection may never make.
+    if (read_buff_.empty()) { read_buff_.resize(read_buff_size_); }
     auto n = read_socket(sock_, read_buff_.data(), read_buff_size_,
                          CPPHTTPLIB_RECV_FLAGS);
     if (n <= 0) {
@@ -14274,6 +14321,228 @@ WebSocketSSLStream::set_read_interrupt(std::function<bool()> interrupted) {
  * Group 4: Server implementation
  */
 
+// A kept-alive connection that the event loop of set_keep_alive_parking()
+// owns while it is idle, and a worker owns while it serves a request. A
+// derived class adds the transport: the stream, and for TLS the session.
+struct Server::KeepAliveConnection {
+  KeepAliveConnection(socket_t s, Server &server) : svr(server), sock(s) {}
+  // Every owner closes it before letting go, which the transport hooks need;
+  // this is only a net, which cannot reach them any more.
+  virtual ~KeepAliveConnection() {
+    if (sock != INVALID_SOCKET) { detail::drain_and_close_socket(sock, 0); }
+  }
+
+  // On a worker before the first request, e.g. for a TLS handshake. False
+  // closes the connection. One that reads from the socket clears turn.woken.
+  virtual bool open() { return true; }
+  // Serves requests while they come: serve_on() with the transport's stream.
+  virtual bool serve() = 0;
+  // Before it is parked. Nothing is buffered then.
+  virtual void release_buffers() {}
+  // Before the socket of an open connection is closed, e.g. to send TLS
+  // close_notify.
+  virtual void shut_down(bool /*graceful*/) {}
+
+  // On a worker, once the event loop has seen the socket readable. Afterwards
+  // turn.park tells whether it may be parked again.
+  bool resume() {
+    turn = Turn();
+    return svr.serve_guarded([this]() {
+      if (!opened) {
+        detail::get_remote_ip_and_port(sock, remote_addr, remote_port);
+        detail::get_local_ip_and_port(sock, local_addr, local_port);
+        if (!open()) { return false; }
+        opened = true;
+      }
+      return serve();
+    });
+  }
+
+  // From the event loop, which must not wait for the peer.
+  void close_idle() {
+    assert(sock != INVALID_SOCKET);
+    if (opened) {
+      detail::set_nonblocking(sock, true);
+      shut_down(true);
+    }
+    detail::drain_and_close_socket(sock, 0);
+    sock = INVALID_SOCKET;
+  }
+
+  // From the worker that served it.
+  void close_served(bool graceful) {
+    assert(sock != INVALID_SOCKET);
+    if (opened) {
+      shut_down(graceful);
+      detail::drain_and_close_socket(sock);
+    } else {
+      // It never got as far as a request, e.g. a failed TLS handshake.
+      detail::shutdown_socket(sock);
+      detail::close_socket(sock);
+    }
+    sock = INVALID_SOCKET;
+  }
+
+  template <typename S>
+  bool serve_on(S &strm,
+                const std::function<void(Request &)> &setup_request = nullptr) {
+    return detail::process_server_socket_core(
+        strm, served, svr.keep_alive_max_count_,
+        [this]() { return wait_for_next_request(); },
+        [&](Stream &s, bool close_connection, bool &connection_closed) {
+          return svr.process_request(s, remote_addr, remote_port, local_addr,
+                                     local_port, close_connection,
+                                     connection_closed, setup_request,
+                                     &turn.websocket_upgraded);
+        });
+  }
+
+  bool wait_for_next_request() {
+    // The event loop saw the socket readable just before this ran.
+    if (turn.woken) {
+      turn.woken = false;
+      return true;
+    }
+    if (svr.is_stopping()) { return false; }
+    // Waiting here for a slow client would keep the worker from others.
+    turn.park = detail::select_read(sock, 0, 0) <= 0;
+    return !turn.park;
+  }
+
+  // A worker's time with it, from resume() until it is parked or closed.
+  struct Turn {
+    bool woken = true;
+    bool park = false;
+    bool websocket_upgraded = false;
+  };
+
+  Server &svr;
+  socket_t sock;
+  int remote_port = 0;
+  int local_port = 0;
+  bool opened = false; // by resume(), once open() has succeeded
+  Turn turn;
+  size_t served = 0;
+  std::string remote_addr;
+  std::string local_addr;
+
+  // While parked
+  std::list<std::shared_ptr<KeepAliveConnection>>::iterator parked_at;
+  std::chrono::steady_clock::time_point deadline;
+};
+
+struct Server::PlainConnection final : Server::KeepAliveConnection {
+  PlainConnection(socket_t s, Server &server)
+      : KeepAliveConnection(s, server),
+        strm(s, server.read_timeout_sec_, server.read_timeout_usec_,
+             server.write_timeout_sec_, server.write_timeout_usec_) {}
+
+  bool serve() override { return serve_on(strm); }
+  void release_buffers() override { strm.release_buffer(); }
+
+  // Kept across requests: it may hold the start of a pipelined request.
+  detail::SocketStream strm;
+};
+
+struct Server::ParkingState {
+  using Clock = std::chrono::steady_clock;
+  using Token = detail::Poller::Token;
+  using Connection = std::shared_ptr<KeepAliveConnection>;
+
+  // Kept for the Server's life, so that stop() can wake() it without a lock.
+  std::unique_ptr<detail::Poller> poller;
+
+  // The listening socket while parking_loop() watches it. Whichever of stop()
+  // and parking_loop() takes it out of here decides who closes it, so that it
+  // is never closed while the poller still watches it.
+  std::atomic<socket_t> listen_sock{INVALID_SOCKET};
+
+  // From stop(): whether parking_loop() takes over closing `sock`.
+  bool release_listen(socket_t sock) {
+    if (!listen_sock.compare_exchange_strong(sock, INVALID_SOCKET)) {
+      return false;
+    }
+    poller->wake();
+    return true;
+  }
+
+  // From a worker: hands an idle connection back, unless the loop has ended.
+  bool give_back(Connection conn) {
+    {
+      std::lock_guard<std::mutex> guard(mutex);
+      if (closed) { return false; }
+      returned.push_back(std::move(conn));
+    }
+    poller->wake();
+    return true;
+  }
+
+  // The rest belongs to parking_loop(). A parked connection's token is its
+  // address, and `parked` lists them in the order they expire.
+
+  // Closes it and returns false when the poller cannot watch it: unwatched,
+  // it would only wait for its deadline.
+  bool park(Connection conn, Clock::time_point deadline) {
+    auto &c = *conn;
+    c.deadline = deadline;
+    c.parked_at = parked.insert(parked.end(), std::move(conn));
+    if (poller->arm(c.sock, reinterpret_cast<Token>(&c))) { return true; }
+    unpark(reinterpret_cast<Token>(&c))->close_idle();
+    return false;
+  }
+
+  Connection unpark(Token token) {
+    auto &c = *reinterpret_cast<KeepAliveConnection *>(token);
+    auto conn = std::move(*c.parked_at);
+    parked.erase(c.parked_at);
+    return conn;
+  }
+
+  Clock::time_point next_deadline() const {
+    return parked.empty() ? (Clock::time_point::max)()
+                          : parked.front()->deadline;
+  }
+
+  void expire(Clock::time_point now) {
+    while (!parked.empty() && parked.front()->deadline <= now) {
+      poller->disarm(parked.front()->sock);
+      parked.front()->close_idle();
+      parked.pop_front();
+    }
+  }
+
+  // Closes them here rather than wherever the last reference goes, which may
+  // be a worker that has just given one back, or the destructor, which cannot
+  // reach the transport.
+  void close_all() {
+    std::vector<Connection> left;
+    {
+      std::lock_guard<std::mutex> guard(mutex);
+      closed = true;
+      left.swap(returned);
+    }
+    for (auto &conn : parked) {
+      poller->disarm(conn->sock);
+      conn->close_idle();
+    }
+    for (auto &conn : left) {
+      conn->close_idle();
+    }
+    for (auto &conn : backlog) {
+      conn->close_idle();
+    }
+    parked.clear();
+    backlog.clear();
+  }
+
+  std::mutex mutex;
+  std::vector<Connection> returned;
+  bool closed = false;
+
+  std::list<Connection> parked;
+  std::list<Connection> backlog;
+};
+
 // HTTP server implementation
 inline Server::Server()
     : new_task_queue([] {
@@ -14618,6 +14887,20 @@ inline Server &Server::set_write_timeout(time_t sec, time_t usec) {
 inline Server &Server::set_idle_interval(time_t sec, time_t usec) {
   idle_interval_sec_ = sec;
   idle_interval_usec_ = usec;
+  idle_interval_set_ = true;
+  return *this;
+}
+
+inline Server &Server::set_keep_alive_parking(bool on) {
+  // Never destroyed once made: stop() may use it from another thread.
+  if (on && !parking_) {
+    auto poller = detail::make_poller();
+    if (poller) {
+      parking_.reset(new ParkingState());
+      parking_->poller = std::move(poller);
+    }
+  }
+  keep_alive_parking_ = on;
   return *this;
 }
 
@@ -14697,7 +14980,7 @@ inline void Server::stop() noexcept {
   // bind_to_port() without listen_after_bind() still owns the descriptor. The
   // exchange is what makes this safe to call concurrently with the accept loop.
   socket_t sock = svr_sock_.exchange(INVALID_SOCKET);
-  if (sock != INVALID_SOCKET) {
+  if (sock != INVALID_SOCKET && !(parking_ && parking_->release_listen(sock))) {
     detail::shutdown_socket(sock);
     detail::close_socket(sock);
   }
@@ -15319,78 +15602,69 @@ inline bool Server::listen_internal() {
   {
     std::unique_ptr<TaskQueue> task_queue(new_task_queue());
 
-    while (svr_sock_ != INVALID_SOCKET) {
+    ret = keep_alive_parking_ && parking_ ? parking_loop(*task_queue)
+                                          : accept_loop(*task_queue);
+
+    task_queue->shutdown();
+  }
+
+  is_decommissioned = !ret;
+  return ret;
+}
+
+inline void Server::prepare_accepted_socket(socket_t sock) const {
+  detail::set_socket_opt_time(sock, SOL_SOCKET, SO_RCVTIMEO, read_timeout_sec_,
+                              read_timeout_usec_);
+  detail::set_socket_opt_time(sock, SOL_SOCKET, SO_SNDTIMEO, write_timeout_sec_,
+                              write_timeout_usec_);
+  if (tcp_nodelay_) { set_socket_opt(sock, IPPROTO_TCP, TCP_NODELAY, 1); }
+}
+
+// Serves each connection on one worker thread for its whole life.
+inline bool Server::accept_loop(TaskQueue &task_queue) {
+  auto ret = true;
+  while (svr_sock_ != INVALID_SOCKET) {
 #ifndef _WIN32
-      if (idle_interval_sec_ > 0 || idle_interval_usec_ > 0) {
+    if (idle_interval_sec_ > 0 || idle_interval_usec_ > 0) {
 #endif
-        auto val = detail::select_read(svr_sock_, idle_interval_sec_,
-                                       idle_interval_usec_);
-        if (val == 0) { // Timeout
-          task_queue->on_idle();
-          continue;
-        }
-#ifndef _WIN32
+      auto val = detail::select_read(svr_sock_, idle_interval_sec_,
+                                     idle_interval_usec_);
+      if (val == 0) { // Timeout
+        task_queue.on_idle();
+        continue;
       }
+#ifndef _WIN32
+    }
 #endif
 
-#if defined _WIN32
-      // sockets connected via WASAccept inherit flags NO_HANDLE_INHERIT,
-      // OVERLAPPED
-      socket_t sock = WSAAccept(svr_sock_, nullptr, nullptr, nullptr, 0);
-#elif defined SOCK_CLOEXEC
-      socket_t sock = accept4(svr_sock_, nullptr, nullptr, SOCK_CLOEXEC);
-#else
-      socket_t sock = accept(svr_sock_, nullptr, nullptr);
-#endif
-
-      if (sock == INVALID_SOCKET) {
-        // NOTE: Winsock reports failures through WSAGetLastError() and never
-        // touches the CRT errno, so the two have to be asked platform by
-        // platform rather than by testing errno here.
-        if (detail::is_accept_resource_error()) {
-          // The per-process descriptor limit or the network stack's buffer
-          // space has been reached. Try to accept new connections after a
-          // short sleep.
-          std::this_thread::sleep_for(std::chrono::microseconds{1});
-          continue;
-        } else if (detail::is_accept_transient_error()) {
-          continue;
-        }
-        // Take the descriptor out of svr_sock_ before closing it: a later
-        // stop() would otherwise shutdown()/close() a value the OS may have
-        // reused, and keep_alive() watches svr_sock_ to notice the server is
-        // gone. The exchange also settles the race with a concurrent stop(),
-        // since whichever side takes the descriptor closes it exactly once.
-        auto listen_sock = svr_sock_.exchange(INVALID_SOCKET);
-        if (listen_sock != INVALID_SOCKET) {
-          detail::close_socket(listen_sock);
-          ret = false;
-          output_error_log(Error::Connection, nullptr);
-        } else {
-          ; // The server socket was closed by user.
-        }
-        break;
-      }
-
-      detail::set_socket_opt_time(sock, SOL_SOCKET, SO_RCVTIMEO,
-                                  read_timeout_sec_, read_timeout_usec_);
-      detail::set_socket_opt_time(sock, SOL_SOCKET, SO_SNDTIMEO,
-                                  write_timeout_sec_, write_timeout_usec_);
-
-      if (tcp_nodelay_) { set_socket_opt(sock, IPPROTO_TCP, TCP_NODELAY, 1); }
+    auto sock = detail::accept_socket(svr_sock_);
 
-      if (!task_queue->enqueue(
-              [this, sock]() { process_and_close_socket(sock); })) {
-        output_error_log(Error::ResourceExhaustion, nullptr);
-        detail::shutdown_socket(sock);
-        detail::close_socket(sock);
+    if (sock == INVALID_SOCKET) {
+      // NOTE: Winsock reports failures through WSAGetLastError() and never
+      // touches the CRT errno, so the two have to be asked platform by
+      // platform rather than by testing errno here.
+      if (detail::is_accept_resource_error()) {
+        // The per-process descriptor limit or the network stack's buffer
+        // space has been reached. Try to accept new connections after a
+        // short sleep.
+        std::this_thread::sleep_for(std::chrono::microseconds{1});
+        continue;
+      } else if (detail::is_accept_transient_error()) {
+        continue;
       }
+      if (abandon_listen_socket()) { ret = false; }
+      break;
     }
 
-    task_queue->shutdown();
-  }
+    prepare_accepted_socket(sock);
 
-  is_decommissioned = !ret;
+    if (!task_queue.enqueue(
+            [this, sock]() { process_and_close_socket(sock); })) {
+      output_error_log(Error::ResourceExhaustion, nullptr);
+      detail::shutdown_socket(sock);
+      detail::close_socket(sock);
+    }
+  }
   return ret;
 }
 
@@ -16208,6 +16482,190 @@ inline bool Server::process_and_close_socket(socket_t sock) {
   return ret;
 }
 
+// After accept() failed for good. Takes the descriptor out of svr_sock_
+// before closing it: a later stop() would otherwise shutdown()/close() a value
+// the OS may have reused, and keep_alive() watches svr_sock_ to notice the
+// server is gone. The exchange also settles the race with a concurrent stop(),
+// since whichever side takes the descriptor closes it exactly once. Returns
+// false when stop() took it first.
+inline bool Server::abandon_listen_socket() {
+  auto sock = svr_sock_.exchange(INVALID_SOCKET);
+  if (sock == INVALID_SOCKET) { return false; }
+  detail::close_socket(sock);
+  output_error_log(Error::Connection, nullptr);
+  return true;
+}
+
+// Like accept_loop(), but a connection waiting for its next request sits in
+// the poller instead of keeping a worker. A worker serves a connection while it
+// has requests to read, then gives it back to this loop.
+inline bool Server::parking_loop(TaskQueue &task_queue) {
+  using Clock = ParkingState::Clock;
+  using Connection = ParkingState::Connection;
+  const ParkingState::Token listen_token = 0;
+  const auto never = (Clock::time_point::max)();
+
+  auto &st = *parking_;
+  auto &poller = *st.poller;
+
+  std::vector<ParkingState::Token> ready;
+  poller.wait(0, ready); // drops a wake() left from an earlier stop()
+  {
+    std::lock_guard<std::mutex> guard(st.mutex);
+    st.closed = false;
+  }
+
+  // Publish the listening socket before checking that stop() has not taken it,
+  // so that a stop() from now on leaves closing it to this loop.
+  auto listen_sock = svr_sock_.load();
+  st.listen_sock = listen_sock;
+  auto listening = listen_sock != INVALID_SOCKET && svr_sock_ == listen_sock;
+  if (listening) { detail::set_nonblocking(listen_sock, true); }
+  auto listen_rearm_at = Clock::now(); // `never` while it is armed
+  auto fatal = false;
+
+  const auto check_interval =
+      std::chrono::duration_cast<Clock::duration>(std::chrono::microseconds(
+          CPPHTTPLIB_KEEPALIVE_TIMEOUT_CHECK_INTERVAL_USECOND));
+  // Never shorter than the interval keep_alive() first waits, so that a
+  // request sent right after connecting is served even with a timeout of 0.
+  const auto keep_alive =
+      (std::max)(Clock::duration(std::chrono::seconds(keep_alive_timeout_sec_)),
+                 check_interval);
+  // Only an interval the user asked for: Windows' default exists for the sake
+  // of stop(), which wake() already serves here.
+  const auto idle = std::chrono::microseconds(
+      idle_interval_set_ ? idle_interval_sec_ * 1000000 + idle_interval_usec_
+                         : 0);
+  auto next_idle = idle.count() > 0 ? Clock::now() + idle : never;
+
+  auto park = [&](Connection conn, Clock::time_point now) {
+    if (!st.park(std::move(conn), now + keep_alive)) {
+      output_error_log(Error::ResourceExhaustion, nullptr);
+    }
+  };
+  auto dispatch = [&](const Connection &conn) {
+    return task_queue.enqueue([this, conn]() { resume_connection(conn); });
+  };
+  // The only way into the queue besides the backlog, which nothing overtakes.
+  // When the queue is full, a new connection is turned away, as accept_loop()
+  // does, and an established one waits in the backlog, since closing it would
+  // lose its request. A TaskQueue gives no sign when it has room again, so the
+  // backlog is retried every check_interval.
+  auto submit = [&](Connection conn) {
+    if (st.backlog.empty() && dispatch(conn)) { return; }
+    if (conn->opened) {
+      st.backlog.push_back(std::move(conn));
+      return;
+    }
+    conn->close_idle();
+    output_error_log(Error::ResourceExhaustion, nullptr);
+  };
+
+  std::vector<Connection> returned;
+  while (listening && svr_sock_ == listen_sock) {
+    auto now = Clock::now();
+    if (now >= listen_rearm_at) {
+      listen_rearm_at =
+          poller.arm(listen_sock, listen_token) ? never : now + check_interval;
+    }
+
+    // At most a minute, which keeps the timeout below within an int.
+    auto until = (std::min)({st.next_deadline(), listen_rearm_at, next_idle,
+                             st.backlog.empty() ? never : now + check_interval,
+                             now + std::chrono::seconds(60)});
+    // Rounded up, so that the wait does not end just short of the deadline.
+    auto timeout = std::chrono::duration_cast<std::chrono::milliseconds>(
+        until - now + std::chrono::microseconds(999));
+    poller.wait(static_cast<int>((std::max)(timeout.count(),
+                                            std::chrono::milliseconds::rep(0))),
+                ready);
+    if (svr_sock_ != listen_sock) { break; }
+    now = Clock::now();
+
+    {
+      std::lock_guard<std::mutex> guard(st.mutex);
+      returned.swap(st.returned);
+    }
+    for (auto &conn : returned) {
+      park(std::move(conn), now);
+    }
+    returned.clear();
+
+    // Connections that found the queue full go first, in their order.
+    while (!st.backlog.empty() && dispatch(st.backlog.front())) {
+      st.backlog.pop_front();
+    }
+
+    for (auto token : ready) {
+      if (token == listen_token) {
+        listen_rearm_at = now;
+        // A batch per round, so that a burst of new connections does not
+        // hold up the ones that are ready.
+        for (auto i = 0; i < 64; i++) {
+          auto sock = detail::accept_socket(listen_sock);
+          if (sock == INVALID_SOCKET) {
+            if (detail::is_would_block_error()) { break; }
+            if (detail::is_accept_resource_error()) {
+              // Out of descriptors: let the deadlines free some first.
+              listen_rearm_at = now + check_interval;
+              break;
+            }
+            if (detail::is_accept_transient_error()) { continue; }
+            fatal = true;
+            break;
+          }
+          // Some platforms make it non-blocking like the listening socket.
+          detail::set_nonblocking(sock, false);
+          prepare_accepted_socket(sock);
+          park(new_keep_alive_connection(sock), now);
+          if (idle.count() > 0) { next_idle = now + idle; }
+        }
+        continue;
+      }
+
+      submit(st.unpark(token));
+    }
+    if (fatal) { break; }
+
+    st.expire(now);
+    if (now >= next_idle) {
+      task_queue.on_idle();
+      next_idle = now + idle;
+    }
+  }
+
+  // A worker still serving a request closes its connection itself.
+  st.close_all();
+  poller.disarm(listen_sock);
+
+  // See ParkingState::listen_sock.
+  if (st.listen_sock.exchange(INVALID_SOCKET) == INVALID_SOCKET) {
+    // stop() handed it over.
+    detail::shutdown_socket(listen_sock);
+    detail::close_socket(listen_sock);
+    return true;
+  }
+  return !(fatal && abandon_listen_socket());
+}
+
+inline std::shared_ptr<Server::KeepAliveConnection>
+Server::new_keep_alive_connection(socket_t sock) {
+  return std::make_shared<PlainConnection>(sock, *this);
+}
+
+inline void
+Server::resume_connection(std::shared_ptr<KeepAliveConnection> conn) {
+  auto &st = *parking_;
+  auto &c = *conn;
+  auto ret = c.resume();
+  if (c.turn.park) {
+    c.release_buffers(); // it may stay idle for long
+    if (st.give_back(conn)) { return; }
+  }
+  c.close_served(!c.turn.websocket_upgraded && ret);
+}
+
 inline void Server::output_log(const Request &req, const Response &res) const {
   if (logger_) {
     std::lock_guard<std::mutex> guard(logger_mutex_);
@@ -19719,7 +20177,73 @@ inline bool SSLServer::is_valid() const {
   return ctx_ != nullptr && Server::is_valid();
 }
 
-inline bool SSLServer::process_and_close_socket(socket_t sock) {
+#ifdef CPPHTTPLIB_OPENSSL_SUPPORT
+namespace detail {
+
+// Before these, SSL_free_buffers() could free a buffer that a record whose
+// body has not arrived yet still uses (CVE-2024-4741). `version` is
+// OpenSSL_version_num().
+inline bool ssl_free_buffers_is_safe(unsigned long version) {
+  if (version >= 0x30400000UL) { return true; }
+  if (version >= 0x30300000UL) { return version >= 0x30300010UL; } // 3.3.1
+  if (version >= 0x30200000UL) { return version >= 0x30200020UL; } // 3.2.2
+  if (version >= 0x30100000UL) { return version >= 0x30100060UL; } // 3.1.6
+  return version >= 0x300000e0UL;                                  // 3.0.14
+}
+
+} // namespace detail
+#endif
+
+// A connection that keep-alive parking keeps for SSLServer. The worker that
+// first resumes it runs the handshake, so one that has not sent its
+// ClientHello yet waits in the event loop like an idle connection.
+struct SSLServer::TlsConnection final : Server::KeepAliveConnection {
+  TlsConnection(socket_t s, SSLServer &server)
+      : KeepAliveConnection(s, server), ssl_svr(server) {}
+  ~TlsConnection() override { tls::free_session(session); }
+
+  bool open() override {
+    // A handshake would hold up the stop() that is under way.
+    if (ssl_svr.is_stopping()) { return false; }
+    session = ssl_svr.start_session(sock);
+    if (!session) { return false; }
+    // The handshake has read what the event loop saw, so a client that sends
+    // nothing after it is parked instead of being waited for.
+    turn.woken = false;
+    return true;
+  }
+
+  // The session, not the stream, holds what a pipelined request has left.
+  bool serve() override {
+    detail::SSLSocketStream strm(
+        sock, session, ssl_svr.read_timeout_sec_, ssl_svr.read_timeout_usec_,
+        ssl_svr.write_timeout_sec_, ssl_svr.write_timeout_usec_);
+    return serve_on(strm, [this](Request &req) { req.ssl = session; });
+  }
+
+  void release_buffers() override {
+#if defined(CPPHTTPLIB_OPENSSL_SUPPORT) && !defined(OPENSSL_IS_BORINGSSL) &&   \
+    !defined(LIBRESSL_VERSION_NUMBER)
+    // The record buffers are most of an idle session's memory, and the next
+    // read makes new ones. The library that runs decides whether that is safe.
+    static const auto safe =
+        detail::ssl_free_buffers_is_safe(OpenSSL_version_num());
+    if (safe) { SSL_free_buffers(static_cast<SSL *>(session)); }
+#endif
+  }
+
+  void shut_down(bool graceful) override { tls::shutdown(session, graceful); }
+
+  SSLServer &ssl_svr;
+  tls::session_t session = nullptr;
+};
+
+inline std::shared_ptr<Server::KeepAliveConnection>
+SSLServer::new_keep_alive_connection(socket_t sock) {
+  return std::make_shared<TlsConnection>(sock, *this);
+}
+
+inline tls::session_t SSLServer::start_session(socket_t sock) {
   using namespace tls;
 
   session_t session = nullptr;
@@ -19732,29 +20256,17 @@ inline bool SSLServer::process_and_close_socket(socket_t sock) {
 
   if (!session) {
     last_ssl_error_ = static_cast<int>(get_error());
-    detail::shutdown_socket(sock);
-    detail::close_socket(sock);
-    return false;
+    return nullptr;
   }
 
-  // Use scope_exit to ensure cleanup on all paths (including exceptions)
-  bool handshake_done = false;
-  bool ret = false;
-  bool websocket_upgraded = false;
+  // Frees the session on all paths but success (including exceptions)
+  auto accepted = false;
   auto cleanup = detail::scope_exit([&] {
-    if (handshake_done) { shutdown(session, !websocket_upgraded && ret); }
-    free_session(session);
-    if (handshake_done) {
-      detail::drain_and_close_socket(sock);
-    } else {
-      detail::shutdown_socket(sock);
-      detail::close_socket(sock);
-    }
+    if (!accepted) { free_session(session); }
   });
 
   // Perform TLS accept handshake with timeout
   TlsError tls_err;
-  auto accepted = false;
   {
     auto handshake_left = detail::scope_exit([&] {
       std::lock_guard<std::mutex> guard(ctx_mutex_);
@@ -19776,10 +20288,27 @@ inline bool SSLServer::process_and_close_socket(socket_t sock) {
 #else
     last_ssl_error_ = static_cast<int>(get_error());
 #endif
+    return nullptr;
+  }
+  return session;
+}
+
+inline bool SSLServer::process_and_close_socket(socket_t sock) {
+  auto session = start_session(sock);
+  if (!session) {
+    detail::shutdown_socket(sock);
+    detail::close_socket(sock);
     return false;
   }
 
-  handshake_done = true;
+  // Use scope_exit to ensure cleanup on all paths (including exceptions)
+  bool ret = false;
+  bool websocket_upgraded = false;
+  auto cleanup = detail::scope_exit([&] {
+    tls::shutdown(session, !websocket_upgraded && ret);
+    tls::free_session(session);
+    detail::drain_and_close_socket(sock);
+  });
 
   std::string remote_addr;
   int remote_port = 0;

+ 463 - 0
test/test.cc

@@ -76,8 +76,13 @@ ctx_t create_server_context();
 void free_context(ctx_t ctx);
 bool set_server_cert_file(ctx_t ctx, const char *cert_path,
                           const char *key_path, const char *password);
+ctx_t create_client_context();
+void set_verify_client(ctx_t ctx, bool require);
 session_t create_session(ctx_t ctx, socket_t sock);
 void free_session(session_t session);
+bool connect_nonblocking(session_t session, socket_t sock, time_t timeout_sec,
+                         time_t timeout_usec, TlsError *err,
+                         const std::function<bool()> &interrupted);
 bool accept_nonblocking(session_t session, socket_t sock, time_t timeout_sec,
                         time_t timeout_usec, TlsError *err);
 ssize_t read(session_t session, void *buf, size_t len, TlsError &err);
@@ -90,6 +95,11 @@ macos_ssl_trust_settings_result(CFArrayRef trust_settings);
 } // namespace impl
 #endif
 } // namespace tls
+#ifdef CPPHTTPLIB_OPENSSL_SUPPORT
+namespace detail {
+bool ssl_free_buffers_is_safe(unsigned long version);
+} // namespace detail
+#endif
 } // namespace httplib
 #endif
 
@@ -34152,3 +34162,456 @@ TEST(PollerFactoryTest, MakePollerWorks) {
   std::vector<detail::Poller::Token> ready;
   EXPECT_TRUE(poller->wait(1000, ready));
 }
+
+// ---------------------------------------------------------------------------
+// Keep-alive parking
+// ---------------------------------------------------------------------------
+
+namespace parking_test {
+
+// Runs `svr` on a free port and stops it when it goes away. `before_listen`
+// runs once the port is bound and listening, before the server accepts.
+struct RunningServer {
+  explicit RunningServer(
+      Server &svr, const std::function<void(int)> &before_listen = nullptr)
+      : svr_(svr) {
+    port = svr_.bind_to_any_port("127.0.0.1");
+    if (before_listen) { before_listen(port); }
+    thread = std::thread([this] { svr_.listen_after_bind(); });
+    svr_.wait_until_ready();
+  }
+  ~RunningServer() {
+    svr_.stop();
+    if (thread.joinable()) { thread.join(); }
+  }
+
+  Server &svr_;
+  int port = 0;
+  std::thread thread;
+};
+
+struct Sockets {
+  std::vector<socket_t> socks;
+  ~Sockets() {
+    for (auto s : socks) {
+      detail::close_socket(s);
+    }
+  }
+  socket_t connect(int port) {
+    auto error = Error::Success;
+    auto s = detail::create_client_socket("127.0.0.1", "", port, AF_UNSPEC,
+                                          false, false, nullptr, 2, 0, 2, 0, 2,
+                                          0, std::string(), error);
+    if (s != INVALID_SOCKET) { socks.push_back(s); }
+    return s;
+  }
+};
+
+std::chrono::steady_clock::time_point in_ms(int ms) {
+  return std::chrono::steady_clock::now() + std::chrono::milliseconds(ms);
+}
+
+bool readable_by(socket_t sock,
+                 std::chrono::steady_clock::time_point deadline) {
+  auto left = std::chrono::duration_cast<std::chrono::microseconds>(
+                  deadline - std::chrono::steady_clock::now())
+                  .count();
+  return left > 0 && detail::select_read(sock, 0, left) > 0;
+}
+
+bool send_get(socket_t sock, const std::string &path = "/") {
+  auto req = "GET " + path + " HTTP/1.1\r\nHost: localhost\r\n\r\n";
+  return detail::send_socket(sock, req.data(), req.size(), 0) ==
+         static_cast<ssize_t>(req.size());
+}
+
+// Reads one response whose body is "ok".
+bool receives_ok(socket_t sock, int timeout_ms) {
+  auto deadline = in_ms(timeout_ms);
+  std::string buf;
+  while (readable_by(sock, deadline)) {
+    char tmp[1024];
+    auto n = detail::read_socket(sock, tmp, sizeof(tmp), 0);
+    if (n <= 0) { return false; }
+    buf.append(tmp, static_cast<size_t>(n));
+    auto end = buf.find("\r\n\r\n");
+    if (end != std::string::npos && buf.size() >= end + 4 + 2) {
+      return buf.compare(0, 12, "HTTP/1.1 200") == 0 &&
+             buf.compare(end + 4, 2, "ok") == 0;
+    }
+  }
+  return false;
+}
+
+bool get_ok(socket_t sock, int timeout_ms) {
+  return send_get(sock) && receives_ok(sock, timeout_ms);
+}
+
+// Whether the server closes `sock` by `deadline`.
+bool sees_eof(socket_t sock, std::chrono::steady_clock::time_point deadline) {
+  char c;
+  return readable_by(sock, deadline) &&
+         detail::read_socket(sock, &c, 1, 0) <= 0;
+}
+
+bool wait_until(const std::function<bool()> &cond, int timeout_ms) {
+  auto deadline = in_ms(timeout_ms);
+  while (!cond()) {
+    if (std::chrono::steady_clock::now() > deadline) { return false; }
+    std::this_thread::sleep_for(std::chrono::milliseconds(1));
+  }
+  return true;
+}
+
+void serve_ok(Server &svr, std::function<TaskQueue *(void)> new_task_queue) {
+  svr.new_task_queue = std::move(new_task_queue);
+  svr.set_keep_alive_parking(true);
+  svr.Get("/", [](const Request &, Response &res) {
+    res.set_content("ok", "text/plain");
+  });
+}
+
+std::function<TaskQueue *(void)> threads(size_t n) {
+  return [n] { return new ThreadPool(n); };
+}
+
+// Counts what the event loop managed to queue, so that a test can wait for it.
+class CountingTaskQueue final : public TaskQueue {
+public:
+  CountingTaskQueue(size_t threads, size_t max_queued, std::atomic<int> &ok,
+                    std::atomic<int> &full)
+      : pool_(threads, threads, max_queued), ok_(ok), full_(full) {}
+
+  bool enqueue(std::function<void()> fn) override {
+    auto queued = pool_.enqueue([this, fn]() {
+      fn();
+      // Holds the worker back from taking the next job.
+      std::this_thread::sleep_for(std::chrono::milliseconds(pause_after_ms));
+    });
+    (queued ? ok_ : full_)++;
+    return queued;
+  }
+  void shutdown() override { pool_.shutdown(); }
+
+  std::atomic<int> pause_after_ms{0};
+
+private:
+  ThreadPool pool_;
+  std::atomic<int> &ok_;
+  std::atomic<int> &full_;
+};
+
+// Holds handlers until it opens, or for 3 seconds at most.
+class Gate {
+public:
+  void wait() {
+    std::unique_lock<std::mutex> lock(m_);
+    entered_++;
+    cv_.notify_all();
+    cv_.wait_for(lock, std::chrono::seconds(3), [&] { return open_; });
+  }
+  void open() {
+    std::lock_guard<std::mutex> guard(m_);
+    open_ = true;
+    cv_.notify_all();
+  }
+  int entered() {
+    std::lock_guard<std::mutex> guard(m_);
+    return entered_;
+  }
+
+private:
+  std::mutex m_;
+  std::condition_variable cv_;
+  bool open_ = false;
+  int entered_ = 0;
+};
+
+} // namespace parking_test
+
+TEST(KeepAliveParkingTest, IdleConnectionsDoNotHoldTheWorkers) {
+  using namespace parking_test;
+  Server svr;
+  serve_ok(svr, threads(1));
+  RunningServer running(svr);
+  Sockets socks;
+
+  // One kept-alive connection that has been served, and many that have not
+  // sent anything yet. None of them may keep the only worker.
+  auto served = socks.connect(running.port);
+  ASSERT_TRUE(get_ok(served, 2000));
+  for (int i = 0; i < 49; i++) {
+    ASSERT_NE(INVALID_SOCKET, socks.connect(running.port));
+  }
+
+  EXPECT_TRUE(get_ok(socks.connect(running.port), 1000));
+  EXPECT_TRUE(get_ok(served, 1000));
+}
+
+TEST(KeepAliveParkingTest, IdleConnectionsTimeOutTogether) {
+  using namespace parking_test;
+  Server svr;
+  serve_ok(svr, threads(1));
+  svr.set_keep_alive_timeout(1);
+  RunningServer running(svr);
+  Sockets socks;
+
+  for (int i = 0; i < 3; i++) {
+    ASSERT_NE(INVALID_SOCKET, socks.connect(running.port));
+  }
+  // One worker waiting on each in turn would take about 3 seconds.
+  auto deadline = in_ms(2000);
+  for (auto s : socks.socks) {
+    EXPECT_TRUE(sees_eof(s, deadline));
+  }
+}
+
+TEST(KeepAliveParkingTest, EstablishedConnectionWaitsForAFullQueue) {
+  using namespace parking_test;
+  std::atomic<int> queued{0}, full{0};
+  Server svr;
+  // One worker and room for one queued task.
+  serve_ok(svr, [&] { return new CountingTaskQueue(1, 1, queued, full); });
+
+  Gate gate;
+  svr.Get("/block", [&](const Request &, Response &res) {
+    gate.wait();
+    res.set_content("ok", "text/plain");
+  });
+
+  RunningServer running(svr);
+  // Before the server stops, so that a failed run does not wait for the
+  // handler's timeout.
+  auto open_gate = detail::scope_exit([&] { gate.open(); });
+  Sockets socks;
+
+  auto a = socks.connect(running.port);
+  ASSERT_TRUE(get_ok(a, 2000));
+
+  // B keeps the only worker busy.
+  auto b = socks.connect(running.port);
+  ASSERT_TRUE(send_get(b, "/block"));
+  ASSERT_TRUE(wait_until([&] { return gate.entered() == 1; }, 2000));
+
+  // A new connection C fills the queue.
+  auto c = socks.connect(running.port);
+  ASSERT_TRUE(send_get(c));
+  ASSERT_TRUE(wait_until([&] { return queued == 3; }, 2000));
+
+  // A's next request finds the queue full, but A has been served before, so it
+  // waits instead of being cut off.
+  ASSERT_TRUE(send_get(a));
+  ASSERT_TRUE(wait_until([&] { return full >= 1; }, 2000));
+
+  gate.open();
+  EXPECT_TRUE(receives_ok(b, 2000));
+  EXPECT_TRUE(receives_ok(c, 2000));
+  EXPECT_TRUE(receives_ok(a, 2000));
+}
+
+TEST(KeepAliveParkingTest, BacklogIsRetriedOnceTheQueueHasRoom) {
+  using namespace parking_test;
+  std::atomic<int> queued{0}, full{0};
+  CountingTaskQueue *queue = nullptr;
+  Server svr;
+  // Two workers and room for one queued task.
+  serve_ok(svr,
+           [&] { return queue = new CountingTaskQueue(2, 1, queued, full); });
+  Gate first, second;
+  svr.Get("/first", [&](const Request &, Response &res) {
+    first.wait();
+    res.set_content("ok", "text/plain");
+  });
+  svr.Get("/second", [&](const Request &, Response &res) {
+    second.wait();
+    res.set_content("ok", "text/plain");
+  });
+
+  RunningServer running(svr);
+  auto open_gates = detail::scope_exit([&] {
+    first.open();
+    second.open();
+  });
+  Sockets socks;
+
+  auto a = socks.connect(running.port);
+  ASSERT_TRUE(get_ok(a, 2000));
+
+  // B and D take both workers, and C fills the queue. One at a time: the
+  // queue has room for one job only until a worker takes it.
+  ASSERT_TRUE(send_get(socks.connect(running.port), "/first"));
+  ASSERT_TRUE(wait_until([&] { return first.entered() == 1; }, 2000));
+  ASSERT_TRUE(send_get(socks.connect(running.port), "/second"));
+  ASSERT_TRUE(wait_until([&] { return second.entered() == 1; }, 2000));
+  ASSERT_TRUE(send_get(socks.connect(running.port), "/second"));
+  ASSERT_TRUE(wait_until([&] { return queued == 4; }, 2000));
+
+  // A's next request waits in the backlog.
+  ASSERT_TRUE(send_get(a));
+  ASSERT_TRUE(wait_until([&] { return full >= 1; }, 2000));
+
+  // B's worker finishes, then takes C off the queue only later, which makes
+  // room for A.
+  queue->pause_after_ms = 200;
+  first.open();
+  EXPECT_TRUE(wait_until([&] { return queued == 5; }, 1000));
+  queue->pause_after_ms = 0;
+
+  second.open();
+  EXPECT_TRUE(receives_ok(a, 2000));
+}
+
+TEST(KeepAliveParkingTest, ZeroKeepAliveTimeoutServesTheFirstRequest) {
+  using namespace parking_test;
+  Server svr;
+  serve_ok(svr, threads(1));
+  svr.set_keep_alive_timeout(0);
+
+  // The request is on the socket before the server accepts the connection,
+  // so it does not race the 10 ms floor. Without the floor the connection is
+  // closed unread.
+  Sockets socks;
+  auto sock = INVALID_SOCKET;
+  RunningServer running(svr, [&](int port) {
+    sock = socks.connect(port);
+    send_get(sock);
+  });
+  EXPECT_TRUE(receives_ok(sock, 1000));
+}
+
+TEST(KeepAliveParkingTest, StopClosesParkedConnectionsPromptly) {
+  using namespace parking_test;
+  for (int round = 0; round < 5; round++) {
+    SCOPED_TRACE(round);
+    Server svr;
+    serve_ok(svr, threads(2));
+    RunningServer running(svr);
+    Sockets socks;
+
+    for (int i = 0; i < 20; i++) {
+      ASSERT_TRUE(get_ok(socks.connect(running.port), 1000));
+    }
+
+    auto start = std::chrono::steady_clock::now();
+    svr.stop();
+    running.thread.join();
+    EXPECT_LT(std::chrono::steady_clock::now() - start,
+              std::chrono::milliseconds(500));
+
+    auto deadline = in_ms(1000);
+    for (auto s : socks.socks) {
+      EXPECT_TRUE(sees_eof(s, deadline));
+    }
+  }
+}
+
+#ifdef CPPHTTPLIB_SSL_ENABLED
+namespace parking_test {
+
+// Completes a TLS handshake on `sock` and sends nothing more, like a browser
+// that has opened a connection ahead of time. Freeing the session sends no
+// close_notify, so the connection stays quiet.
+bool handshake_only(socket_t sock) {
+  auto ctx = tls::create_client_context();
+  tls::set_verify_client(ctx, false);
+  auto session = tls::create_session(ctx, sock);
+  auto se = detail::scope_exit([&] {
+    tls::free_session(session);
+    tls::free_context(ctx);
+  });
+  tls::TlsError err;
+  return session &&
+         tls::connect_nonblocking(session, sock, 2, 0, &err, nullptr);
+}
+
+Client tls_client(int port) {
+  Client cli("https://127.0.0.1:" + std::to_string(port));
+  cli.enable_server_certificate_verification(false);
+  cli.set_keep_alive(true);
+  cli.set_connection_timeout(1);
+  cli.set_read_timeout(1);
+  return cli;
+}
+
+bool tls_get_ok(Client &cli) {
+  auto res = cli.Get("/");
+  return res && res->body == "ok";
+}
+
+} // namespace parking_test
+
+#ifdef CPPHTTPLIB_OPENSSL_SUPPORT
+TEST(KeepAliveParkingTest, SslFreeBuffersOnlyWhereTheCveIsFixed) {
+  struct {
+    unsigned long version;
+    bool safe;
+  } cases[] = {
+      {0x30000020UL, false}, {0x300000d0UL, false}, {0x300000e0UL, true},
+      {0x300000f0UL, true},  {0x30100050UL, false}, {0x30100060UL, true},
+      {0x30200010UL, false}, {0x30200020UL, true},  {0x30300000UL, false},
+      {0x30300010UL, true},  {0x30400000UL, true},  {0x40000000UL, true},
+  };
+  for (const auto &c : cases) {
+    EXPECT_EQ(c.safe, detail::ssl_free_buffers_is_safe(c.version))
+        << std::hex << c.version;
+  }
+}
+#endif
+
+TEST(KeepAliveParkingTest, IdleTlsConnectionsDoNotHoldTheWorkers) {
+  using namespace parking_test;
+  SSLServer svr(SERVER_CERT_FILE, SERVER_PRIVATE_KEY_FILE);
+  ASSERT_TRUE(svr.is_valid());
+  serve_ok(svr, threads(1));
+  svr.set_read_timeout(1); // so that a failing run does not take long
+  RunningServer running(svr);
+  Sockets socks;
+
+  // One served connection, some that have not started their handshake, and
+  // one that has finished it and gone quiet. None may keep the only worker.
+  auto served = tls_client(running.port);
+  ASSERT_TRUE(tls_get_ok(served));
+  for (int i = 0; i < 10; i++) {
+    ASSERT_NE(INVALID_SOCKET, socks.connect(running.port));
+  }
+  EXPECT_TRUE(handshake_only(socks.connect(running.port)));
+
+  auto other = tls_client(running.port);
+  EXPECT_TRUE(tls_get_ok(other));
+  EXPECT_TRUE(tls_get_ok(served));
+}
+
+TEST(KeepAliveParkingTest, FailedTlsHandshakeClosesTheConnection) {
+  using namespace parking_test;
+  SSLServer svr(SERVER_CERT_FILE, SERVER_PRIVATE_KEY_FILE);
+  ASSERT_TRUE(svr.is_valid());
+  serve_ok(svr, threads(1));
+  RunningServer running(svr);
+  Sockets socks;
+
+  auto plain = socks.connect(running.port);
+  ASSERT_TRUE(send_get(plain));
+  EXPECT_TRUE(sees_eof(plain, in_ms(1000)));
+  EXPECT_NE(0, svr.ssl_last_error());
+}
+
+TEST(KeepAliveParkingTest, StopClosesParkedTlsConnections) {
+  using namespace parking_test;
+  SSLServer svr(SERVER_CERT_FILE, SERVER_PRIVATE_KEY_FILE);
+  ASSERT_TRUE(svr.is_valid());
+  serve_ok(svr, threads(2));
+  RunningServer running(svr);
+
+  std::vector<Client> clients;
+  for (int i = 0; i < 5; i++) {
+    clients.push_back(tls_client(running.port));
+    ASSERT_TRUE(tls_get_ok(clients.back()));
+  }
+
+  auto start = std::chrono::steady_clock::now();
+  svr.stop();
+  running.thread.join();
+  EXPECT_LT(std::chrono::steady_clock::now() - start,
+            std::chrono::milliseconds(500));
+}
+#endif