Add MySQL client protocol (text protocol, transactions, prepared statements) on clean-room auth (#2093)

[rajvarun77]

Implements a MySQL client protocol for brpc built on a clean-room authentication codec (no GPL lineage), addressing #2093.

Scope

• Clean-room connection-phase auth — mysql_native_password and caching_sha2_password (fast-auth + full-auth RSA / secure-transport cleartext), derived from the public MySQL protocol documentation.
• COM_QUERY text protocol — OK / ERR / EOF / text result-set parsing.
• Interactive transactions via connection affinity (reserve/use a pooled connection; SINGLE rejected, auto-rollback on drop).
• Prepared statements (binary protocol) with per-connection stmt_id caching and transparent re-prepare when a statement lands on a fresh connection.

Shared-core footprint (kept redis-like / minimal)

The only generic Controller/Socket additions are: a socket reserve/use hook and fd_version (ABA guard) for transaction/prepared-statement affinity, an auth empty-write guard for MySQL's server-greets-first handshake, and protocol registration. No protocol-specific type lives on the shared Controller (the prepared-statement pointer rides a generic opaque per-RPC slot).

Tests

Clean-room integration tests (transactions, prepared statements, pooled-connection concurrency, connection-type boundary) run against a self-spawned mysqld; 19/19 passing locally. Concurrency coverage includes per-transaction connection pinning under overlap, abort/auto-rollback under contention, and prepared re-prepare when a connection is stolen by a transaction.

🤖 Generated with Claude Code

[rajvarun77]

@chenBright @wwbmmm — marking this ready for review.

Full MySQL client protocol for brpc — COM_QUERY text protocol, interactive transactions, and prepared statements — built on a clean-room authentication codec (no GPL lineage), addressing #2093.

• Auth (clean-room): mysql_native_password + caching_sha2_password (fast-auth + full-auth RSA / secure-transport cleartext), derived from the public MySQL protocol docs.
• Minimal shared-core footprint (redis-like): the only generic Controller/Socket additions are a socket reserve/use hook + fd_version (ABA guard) for transaction/prepared-statement affinity, an auth empty-write guard for the server-greets-first handshake, and protocol registration. No protocol-specific type lives on the shared Controller.
• Tests: clean-room integration tests (transactions, prepared statements, pooled-connection concurrency, connection-type, binary TIME/DATETIME) run against a self-spawned mysqld — 20/20 passing locally; a standalone 30-case end-to-end run was independently verified.

The clean-room auth codec is also up as a smaller, focused PR (#3310, Stage 1a) if you'd prefer to review that foundation first.

[Copilot]

Pull request overview

This PR adds a first-class MySQL client protocol implementation to brpc, including clean-room authentication, text protocol querying, transaction connection-affinity, and prepared statements support, along with new integration/unit tests and examples.

Changes:

• Introduce PROTOCOL_MYSQL and register a MySQL client protocol stack (pack/parse/process) with connection-phase authentication handling.
• Add MySQL client APIs (MysqlRequest, MysqlResponse, prepared statements, transactions) and supporting wire/protocol utilities.
• Add MySQL-focused tests, docs, and examples; extend build/test integration (CMake/Bazel/Makefile).

Reviewed changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
test/mysql/README.md	Documents auth-handshake integration test modes and setup.
test/mysql/brpc_mysql_connection_type_unittest.cpp	Adds integration test for SHORT connection behavior with prepared statements and plain queries.
test/mysql/brpc_mysql_auth_packet_unittest.cpp	Adds unit tests for MySQL auth packet wire helpers (lenenc ints/strings, headers).
test/CMakeLists.txt	Ensures MySQL unit tests are included in the CMake test glob.
test/BUILD.bazel	Adds a Bazel cc_test target covering MySQL unit tests.
src/brpc/socket.h	Exposes fd_version() for ABA-guarding per-connection statement caching.
src/brpc/socket.cpp	Increments fd version on fd reset; allows empty writes for auth-handshake flows.
src/brpc/policy/mysql/mysql.h	Introduces MySQL request/response message types and statement stub plumbing.
src/brpc/policy/mysql/mysql.cpp	Implements MysqlRequest/MysqlResponse serialization and parsing entrypoints.
src/brpc/policy/mysql/mysql_transaction.h	Adds MySQL transaction API and options for connection-affinity usage.
src/brpc/policy/mysql/mysql_transaction.cpp	Implements transaction start/commit/rollback with socket reserve/use semantics.
src/brpc/policy/mysql/mysql_statement.h	Adds prepared statement API and statement-id caching interface.
src/brpc/policy/mysql/mysql_statement.cpp	Implements stmt-id cache keyed by socket id + fd version; placeholder counting.
src/brpc/policy/mysql/mysql_statement_inl.h	Provides DBD map helpers for stmt-id caching and related gflag.
src/brpc/policy/mysql/mysql_protocol.h	Declares MySQL protocol functions (parse/process/serialize/pack).
src/brpc/policy/mysql/mysql_protocol.cpp	Implements client-side MySQL protocol codec, including auth handshake and prepare/execute flows.
src/brpc/policy/mysql/mysql_common.h	Adds MySQL constants/types and collation mapping used by the client implementation.
src/brpc/policy/mysql/mysql_common.cpp	Defines common MySQL defaults and type-to-string helper.
src/brpc/policy/mysql/mysql_command.h	Declares MySQL command packet builders for query/prepare/execute/long-data.
src/brpc/policy/mysql/mysql_command.cpp	Implements MySQL packetization and prepared statement wire builders.
src/brpc/policy/mysql/mysql_authenticator.h	Adds MysqlAuthenticator carrying user/password/schema/params/collation.
src/brpc/policy/mysql/mysql_authenticator.cpp	Implements credential serialization and clean-room scrambles (native + caching_sha2 fast path).
src/brpc/policy/mysql/mysql_auth_scramble.h	Declares clean-room auth scramble and RSA/cleartext helpers.
src/brpc/policy/mysql/mysql_auth_scramble.cpp	Implements native and caching_sha2 scrambles + RSA-OAEP encryption with OpenSSL EVP.
src/brpc/policy/mysql/mysql_auth_packet.h	Adds wire helpers for lenenc ints/strings, packet headers, NUL strings.
src/brpc/policy/mysql/mysql_auth_packet.cpp	Implements the MySQL wire-format helpers used by handshake parsing/building.
src/brpc/policy/mysql/mysql_auth_handshake.h	Declares clean-room handshake packet codecs (greeting/response/switch/more-data).
src/brpc/policy/mysql/mysql_auth_handshake.cpp	Implements parsing/building for connection-phase handshake payloads.
src/brpc/options.proto	Adds PROTOCOL_MYSQL enum value.
src/brpc/global.cpp	Registers the MySQL protocol with the global protocol registry.
src/brpc/details/controller_private_accessor.h	Adds private controller hooks for socket reserve/use and opaque session data.
src/brpc/controller.h	Introduces bind-socket actions/state to support MySQL transaction affinity and session data.
src/brpc/controller.cpp	Implements socket reserve/use behavior across RPCs for pooled/short connection types.
Makefile	Includes src/brpc/policy/mysql in the build directories list.
example/mysql_c++/mysqlclient_press.cpp	Adds libmysqlclient-based press tool (comparison baseline).
example/mysql_c++/mysql_tx.cpp	Adds brpc MySQL transaction example.
example/mysql_c++/mysql_stmt.cpp	Adds brpc MySQL prepared statement example.
example/mysql_c++/mysql_press.cpp	Adds brpc MySQL press example for CRUD.
example/mysql_c++/mysql_go_press.go	Adds Go MySQL press example (comparison baseline).
example/mysql_c++/mysql_cli.cpp	Adds brpc-based interactive MySQL CLI example.
example/mysql_c++/CMakeLists.txt	Adds CMake build for MySQL examples and dependency discovery.
docs/cn/mysql_client.md	Adds/updates Chinese documentation describing brpc MySQL client usage.

Comments suppressed due to low confidence (1)

src/brpc/policy/mysql/mysql_authenticator.h:92

• The trailing #endif comment refers to COUCHBASE, which doesn’t match this header guard and is confusing when grepping/include-debugging. Update it to BRPC_POLICY_MYSQL_AUTHENTICATOR_H.

}  // namespace policy
}  // namespace brpc

#endif  // BRPC_POLICY_COUCHBASE_AUTHENTICATOR_H

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[wwbmmm]

It seems that this _fd_version doesn't need to be atomic, since the static fd_version is atomic.

[rajvarun77]

Thanks — you're right that the uniqueness of the value comes from the static fd_version atomic. But the per-Socket _fd_version still needs to be atomic for a different reason: it is written and read from different threads with no lock between them.

• Writer: ResetFileDescriptor() stores _fd_version on the reconnect path (CheckConnectedAndKeepWrite -> ResetFileDescriptor), with no socket mutex held.
• Reader: the mysql prepared-statement path reads socket->fd_version() on the RPC bthread (MysqlStatement::StatementId). It obtains the socket via Socket::Address(), which only gates on the SocketId version — and ResetFileDescriptor does not bump that version — so Address() succeeds even while the fd is being reset. The read can therefore overlap the write on the same Socket.

A concurrent non-atomic read/write on the same object is a data race (UB), and a 64-bit load is not tear-free on 32-bit targets, so the member has to be atomic. This mirrors _fd itself, which is a butil::atomic<int> written in the same ResetFileDescriptor and read lock-free (memory_order_relaxed) all over the IO path. I kept _fd_version on the same relaxed-atomic pattern. Happy to add a brief comment to that effect if it helps.

[rajvarun77]

@wwbmmm @chenBright — pushed a revision addressing the review feedback and fixing the integration-test setup. Summary:

Review comments

• Stored the mysql-transaction BindSockAction in two Controller::_flags bits instead of a dedicated member.
• Made MysqlResponse explicitly non-copyable and turned the previously no-op MergeFrom into a hard CHECK-failure (so an accidental copy is caught instead of silently dropping replies).
• Renamed get_tx()/get_stmt() to tx()/stmt().
• Added a clearly named ControllerPrivateAccessor::set_mysql_statement_type() alias over the pipelined_count slot the mysql protocol reuses.
• The MysqlCollations ODR note, the gflag spelling, and the two doc/comment mismatches were already addressed in the current revision.
• On _fd_version: I left a reply explaining why it needs to stay atomic — it is written in ResetFileDescriptor (reconnect path, no lock) and read lock-free from another thread via fd_version() in the prepared-statement path, so a non-atomic field would be a data race. It mirrors how _fd itself is handled. Happy to discuss further.

Integration tests now run against a real MySQL server

• The suites previously did not exercise a live server in CI: a legacy test pointed at an external host and hard-failed, and the make/bazel wiring built it as a single globbed target. I removed the legacy test, switched the test/mysql/* suites to the redis-style self-spawning + skip-if-absent pattern, build one cc_test per file, and merged master so the CI image installs mysql-server. The suites now spawn a local throwaway mysqld and run.

Local verification
I built and ran all seven mysql suites locally against a live MySQL 9.6 server: 110 tests, 0 skipped, 0 failed — covering caching_sha2_password auth, the text and binary (prepared-statement) protocols, transactions, and connection pooling. (MySQL 9.x removed mysql_native_password, so this also exercises the caching_sha2 path end-to-end.)

CI is running on the latest revision. Could you take another look when you have a chance? Thanks for the reviews.

[rajvarun77]

@wwbmmm @chenBright — heads-up on CI: the two *-unittest-with-bazel lanes show one failure, and it is an unrelated flake, not a regression from this PR.

• The failing target is //test:brpc_channel_unittest, and within it a single timing-sensitive case, ChannelTest.backup_request, hung and blew the 300s test timeout (the timeout stack trace points at ChannelTest::TestBackupRequest()). It is a timeout, not an assertion failure.
• The same commit passes that test in the make lanes (clang-unittest and clang-unittest-asan are green), and all seven //test:mysql/* targets pass in the bazel lanes.
• It is independent of this change: the new flags use _flags bits 14/15 while backup-request uses FLAGS_BACKUP_REQUEST (bit 5) — no collision — and the backup-request path never touches bind_sock_action().

A re-run of the failed jobs should clear it (I cannot trigger re-runs as an outside contributor).

This PR is ready for another look whenever you have time: all review threads are addressed (I left one reply on _fd_version explaining why it stays atomic), the mysql integration suites now run against a real CI-provisioned mysqld, and I verified all seven suites locally against a live MySQL 9.6 (110 tests, 0 skipped, 0 failed). Thanks!