bcb09b3f4a qa: Verify HTTP listen port exclusivity (Hodlinator)
af65069fd1 windows: Use SO_EXCLUSIVEADDRUSE over SO_REUSEADDR (Hodlinator)
Pull request description:
#### Problem
`HTTPServer::BindAndStartListening()` unconditionally enables `SO_REUSEADDR` before binding the RPC listener. On Windows, a reuse-enabled listener does not reserve the port exclusively: another local process can request `SO_REUSEADDR` and bind to the same port (see https://learn.microsoft.com/en-us/windows/win32/winsock/using-so-reuseaddr-and-so-exclusiveaddruse).
If the competing socket receives a new connection, it can capture the HTTP Basic `Authorization` header (including the cookie credential) and proxy or issue privileged RPC calls as the victim. This crosses a local-user boundary and can expose wallet-controlling RPC credentials.
#### Fix
Have Windows use `SO_EXCLUSIVEADDRUSE` instead which makes the port exclusive to the process which first requests it, while retaining the restart-friendly behavior which `SO_REUSEADDR` enabled. Abort if another process is already bound to the port.
#### Further context & rationale
This issue is new in our homegrown HTTP server implementation, since libevent had a guard against setting `SO_REUSEADDR` on Windows, see `evutil_make_listen_socket_reuseable()` d82464a277/evutil.c (L483). libevent does not reference `SO_EXCLUSIVEADDRUSE`.
Why should we not just avoid `SO_REUSEADDR` on Windows and skip `SO_EXCLUSIVEADDRUSE` like the libevent approach?
Because setting either option makes the process less prone to failing to bind to a port after having been restarted. Not sure why this wasn't an issue before, maybe the node startup was usually slow enough to time out the port before we tried to re-bind it on Windows.
---
Discovered by Project Loupe.
ACKs for top commit:
pinheadmz:
ACK bcb09b3f4a
sedited:
utACK bcb09b3f4a
jeanpablojp:
tACK bcb09b3f4a
Tree-SHA512: 7f2362cc8399e8c4e95b27b39066d3e591b5aebfc2b562aba10786609456f526f562394818a3d1042d64dabf497548ffabf0757322cfb201454a134321108cf5
3d1004cb9b http: throttle per-connection reads while a request is in flight (Matthew Zipkin)
Pull request description:
This patches a memory exhaustion scenario found while auditing the new http server with kimi-k3. A shallow version of this scenario was addressed in #35735 (See https://github.com/bitcoin/bitcoin/pull/35735#discussion_r3720177656 and https://github.com/bitcoin/bitcoin/pull/35735#issuecomment-5217000202) but a OOM vector still remained.
On master when the sever is busy handling a request from a client, it will still read data from that client and "queue up" the next request. In #35735 we handled the scenario where that additional incoming data was an invalid HTTP request by not attempting to parse the data. However, we didn't add a size limit.
A misbehaving client could block its request queue with something like `waitforblock` and then flood the server with nonsense data without any limit.
The solution in this patch is to not even read from the socket at all if we are busy with a request. Similar to the intent of #35735, the kernel will buffer incoming data until backpressure kicks in and the TCP window drops to 0.
If unaddressed, the attack vector is still limited to authenticated clients: unauthenticated REST requests don't block for very long, so the server *should* be able to drain the receive buffer.
ACKs for top commit:
jeanpablojp:
tACK 3d1004cb9b
frankomosh:
ACK 3d1004cb9b
hodlinator:
ACK 3d1004cb9b
winterrdog:
tACK 3d1004cb9b
sedited:
ACK 3d1004cb9b
Tree-SHA512: 56f7678a9ab6789aa542c1f252df0b6ccf9137cb426ff915a0a3fe8285200fdb62b7a47c476ed8617c3592e7a7eac18158cd8c0dac309cdcf4e5fd887e016209
Fixes#35632 by allowing both outcomes of a race condition.
The server behavior is unchanged: in response to a malformed request
we send an error code and disconnect. The issue is that sometimes
on Windows the RST is caught by the platform and the receive buffer
is discarded before the Python client can process it with recv().
We can also be much more polite to misbehaving clients by
implementing SO_LINGER as suggested in #35780 but that will require
more review.
A client streaming pipelined requests into a busy connection
(or any connection whose replies are slower than the sender) could grow
server memory without limit, up to remote OOM.
Stop selecting RecvEvent for clients whose request is being processed;
pipelined data then backs up in the kernel socket buffer, applying TCP
backpressure to the sender. One request per connection is in flight
at a time.
Functional test streams pipelined submitblock requests into a connection
blocked on waitforblockheight. Unpatched builds continue draining the
socket buffer indefinitely, patched builds will stall.
On some systems, such as NetBSD, the non-default
`-rpcmaxconnections=128` is too high, so bitcoind refuses to start:
```
Error: Not enough file descriptors available. 256 available, 290 required.
```
The test only needs a value above the default of 16. Use 64 and lower
`-maxconnections` in that case so the total fits in 256.
Instead of sending 403 Forbidden, disconnect as soon as possible.
To facilitate unit testing, this commit includes a refactor
that moves the subnet allow list and relevant methods
into the HTTPServer class instead of file-scope static scope.
- Only one node needed for test
- Use ascii encoding instead of utf-8
- Make tests independent of each other
- Expect HTTP error code 413 for too-large request
- Clarify python client race condition in comment
Covers:
- http pipelining
- rpcservertimeout
Testing this requires adding an option to TestNode to force
the test framework to establish a new HTTP connection for
every RPC. Otherwise, attempting to reuse a persistent connection
would cause framework RPCs during startup and shutdown to fail.
- "chunked" Transfer-Encoding
The previous diff touched most files in ./test/, so bump the headers to
avoid having to touch them again for a bump later.
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./test/
-END VERIFY SCRIPT-
