Browse Source

Read Content-Encoding as the combined field value

RFC 9110 Section 5.3 makes a Content-Encoding spread over several field lines
the same message as the comma-joined one, so the two have to be read the same
way. Reading occurrence zero did not: a response carrying "gzip" on two field
lines was decoded as a single gzip coding, so a body the sender says was
encoded twice came back after one pass -- still compressed, but presented to
the caller as decoded. The same value written as "gzip, gzip" on one line took
the pass-through path instead.

Read the combined value at both sites. A value naming several codings matches
none of the ones cpp-httplib implements, so both representations now take the
pass-through path that prepare_content_receiver() already documents for an
unrecognized coding.

This does mean a sender that repeats "Content-Encoding: gzip" on two lines for
a body it gzipped once no longer has that body decoded. There is no way to tell
that sender apart from one that really did encode twice, and the conservative
reading is the one the field value states.
yhirose 1 week ago
parent
commit
3e3e4863b0
2 changed files with 44 additions and 2 deletions
  1. 3 2
      httplib.h
  2. 41 0
      test/test.cc

+ 3 - 2
httplib.h

@@ -8087,7 +8087,7 @@ bool prepare_content_receiver(T &x, int &status,
                               bool decompress, size_t payload_max_length,
                               bool &exceed_payload_max_length, U callback) {
   if (decompress) {
-    std::string encoding = x.get_header_value("Content-Encoding");
+    auto encoding = get_combined_header_value(x.headers, "Content-Encoding");
     std::unique_ptr<decompressor> decompressor;
 
     if (!encoding.empty()) {
@@ -14253,7 +14253,8 @@ ClientImpl::open_stream(const std::string &method, const std::string &path,
   handle.body_reader_.chunked =
       detail::is_chunked_transfer_encoding(handle.response->headers);
 
-  auto content_encoding = handle.response->get_header_value("Content-Encoding");
+  auto content_encoding = detail::get_combined_header_value(
+      handle.response->headers, "Content-Encoding");
   if (!content_encoding.empty()) {
     // Same policy as prepare_content_receiver(): reject a coding we know about
     // but were not built with, pass an unrecognized one through as-is.

+ 41 - 0
test/test.cc

@@ -11054,6 +11054,47 @@ static const char GZIPPED_HELLO_WORLD[] = {
 // as "fibre" look like Brotli, and turned a multi-coding value like
 // "gzip, br" into a Brotli-labeled body, so a decompressor was run over data
 // it was never meant to see.
+// RFC 9110 Section 5.3: a Content-Encoding split over several field lines is
+// the same message as the comma-joined one, so both have to be read the same
+// way. Reading only the first line made "gzip" followed by "gzip" look like a
+// single gzip coding, and a body the sender says was encoded twice was handed
+// back after one pass, still compressed but presented as decoded.
+TEST(ContentEncodingTest, DuplicateFieldLinesAreTheSameAsTheJoinedValue) {
+  const std::string body = "\xff\xd8\xff\xe0 not really a jpeg";
+
+  Server svr;
+  svr.Get("/split", [&](const Request & /*req*/, Response &res) {
+    res.set_content(body, "image/jpeg");
+    res.headers.emplace("Content-Encoding", "gzip");
+    res.headers.emplace("Content-Encoding", "gzip");
+  });
+  svr.Get("/joined", [&](const Request & /*req*/, Response &res) {
+    res.set_content(body, "image/jpeg");
+    res.set_header("Content-Encoding", "gzip, gzip");
+  });
+
+  auto port = svr.bind_to_any_port(HOST);
+  thread t = thread([&]() { svr.listen_after_bind(); });
+  auto se = detail::scope_exit([&] {
+    svr.stop();
+    t.join();
+    ASSERT_FALSE(svr.is_running());
+  });
+
+  svr.wait_until_ready();
+
+  Client cli(HOST, port);
+
+  // Two codings is not something cpp-httplib decodes, so both representations
+  // take the pass-through path rather than one of them being gunzipped once.
+  for (const char *path : {"/split", "/joined"}) {
+    auto res = cli.Get(path);
+    ASSERT_TRUE(res) << path << " -> " << to_string(res.error());
+    EXPECT_EQ(StatusCode::OK_200, res->status) << path;
+    EXPECT_EQ(body, res->body) << path;
+  }
+}
+
 TEST(ContentEncodingTest, SubstringOfACodingIsNotTheCoding) {
   const std::string body = "\xff\xd8\xff\xe0 not really a jpeg";