diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index 179ae93..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,43 +0,0 @@ - - - -# 代码库指南 (Repository Guidelines) - -## 项目结构与模块组织 (Project Structure & Module Organization) -`OmniSocketGo` 是一个基于 Go 1.22 的小型模块，在 `cmd/` 目录下有三个命令行（CLI）入口点：`cmd/server`、`cmd/peer` 和 `cmd/latencysummary`。共享代码存放于 `cmd/internal/` 目录下： - -- `protocol` 用于消息的编码/解码 -- `server` 用于连接中心（hub）逻辑 -- `peer` 用于客户端的发送/接收与持久化 -- `transport` 用于 TCP 传输和 Linux 时间戳处理 -- `latencylog` 用于 JSONL 日志记录和摘要生成 - -请将测试文件与它们覆盖的代码放在一起（即 `*_test.go`）。使用 `bin/` 目录存放本地构建输出，使用 `inbox/` 目录存放接收到的负载数据；这两个目录均已被忽略（不在版本控制内），不应提交到代码库中。 - -## 构建、测试与开发命令 (Build, Test, and Development Commands) -显式构建主要的二进制文件： - -- `go build -o bin/server ./cmd/server` -- `go build -o bin/peer ./cmd/peer` -- `go build -o bin/latencysummary ./cmd/latencysummary` - -在 Linux 上运行完整的测试套件： - -- `go test ./...` - -需要部署时进行交叉编译： - -- `CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o bin/server-linux-amd64 ./cmd/server` - -在代码审查前格式化已编辑的文件： - -- `gofmt -w cmd/peer/main.go` - -## 编码风格与命名规范 (Coding Style & Naming Conventions) -遵循标准的 Go 代码风格，让 `gofmt` 控制代码格式；请勿手动对齐代码。保持包名小写，导出的标识符使用大驼峰命名法（`CamelCase`），未导出的辅助函数/变量使用小驼峰命名法（`mixedCase`）。与现有的 CLI 标志（flag）命名保持一致，使用小写且带连字符的选项，例如 `-bind-device` 和 `-latency-log`。优先编写功能聚焦的包和简短的函数，而不是随意添加新的顶层二进制文件或包含过多杂项的通用工具文件。 - -## 测试指南 (Testing Guidelines) -使用 Go 内置的 `testing` 包。优先编写结合 `t.Run` 的表格驱动测试（table-driven tests），参考类似 `cmd/peer/interactive_test.go` 的文件。测试的命名应基于可观察到的行为，而不是内部实现细节。Linux 特有的行为测试应放在 `*_linux_test.go` 文件中。本项目未配置代码覆盖率门禁限制，但新增的协议、传输或持久化逻辑应当包含单元测试，并在相关的地方提供错误路径（error-path）的覆盖测试。 - -## 平台与配置说明 (Platform & Configuration Notes) -本项目以 Linux 为目标平台。传输层依赖于 Linux 特有的时间戳代码，因此完整构建和 `go test ./...` 应视为仅限 Linux 平台的验证操作。 \ No newline at end of file diff --git a/c/Makefile b/Makefile similarity index 100% rename from c/Makefile rename to Makefile diff --git a/README.md b/README.md index 770f75f..3e98a38 100644 --- a/README.md +++ b/README.md @@ -1,94 +1,88 @@ -# OmniSocketGo +# OmniSocketC -Linux only. Go 1.22. +Linux-only C11 implementation of the UDP/KCP transport stack from `OmniSocketGo`. -如果目标机器只运行 `server`，只需要编译并拷贝 `server` 二进制。 -如果目标机器只运行 `peer`，只需要编译并拷贝 `peer` 二进制。 - -`go build ./cmd/server` 和 `go build ./cmd/peer` 会把各自依赖到的功能一起编译进最终二进制，不需要再单独编译 `cmd/internal/...` 包。 - -- `server` 二进制会包含它依赖到的转发、协议、传输等代码 -- `peer` 二进制会包含它依赖到的注册、交互发送、接收落盘、协议、传输等代码 -- 只有没有被这个可执行程序引用的其他命令，才不在该二进制里，比如 `cmd/latencysummary` +This subtree is intentionally standalone. The Go code stays in place as the behavior reference, while the C implementation builds its own binaries under `c/bin/`. ## Build -按目标架构分别编译。 - mkdir -p bin - go build -o bin/server ./cmd/server - go build -o bin/peer ./cmd/peer - go build -o bin/latencysummary ./cmd/latencysummary - -### Linux amd64 - ```bash -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o bin/server-linux-amd64 ./cmd/server -``` - -### Linux arm64 - -```bash -CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o bin/peer-linux-arm64 ./cmd/peer -``` - -### Linux armv7 - -```bash -CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -o bin/server-linux-armv7 ./cmd/server -CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -o bin/peer-linux-armv7 ./cmd/peer +cd c +make ``` +Build outputs: +- `c/bin/udpserver` +- `c/bin/udppeer` +- `c/bin/udpping` +- `c/bin/udprelay` +- `c/bin/kcpserver` +- `c/bin/kcppeer` +- `c/bin/kcpping` ## Run On Different Machines -`server D` 所在机器监听 `0.0.0.0:10909`。 +Server `D` runs the KCP hub on `0.0.0.0:10909`: ```bash -go run cmd/kcpserver/ -listen 0.0.0.0:10909 --kcp-ts-debug-log logs/d-kcp-ts.jsonl -kcp-session-stats-log logs/d-kcp-stats.jsonl +./c/bin/kcpserver -listen 0.0.0.0:10909 \ + -kcp-ts-debug-log logs/d-kcp-ts.jsonl \ + -kcp-session-stats-log logs/d-kcp-stats.jsonl ``` -`relay server C` 所在机器 +Relay `C` runs a raw UDP forwarder to `D`: ```bash -go run ./cmd/kcpserver/ -mode=relay -listen 0.0.0.0:10909 -relay-remote 172.21.32.15:10909 - -2>&1 | tee logs/c.stdout.log +./c/bin/kcpserver -mode=relay -listen 0.0.0.0:10909 -relay-remote 172.21.32.15:10909 ``` -### peer-a (A) +Peer `A` dials `D` through relay `C`: ```bash -go run ./cmd/kcppeer/ -id peer-a -server 172.21.32.15:10909 -relay-via 106.55.173.235:10909 -inbox-dir inbox/a - --latency-log logs/a-latency.jsonl -kcp-ts-debug-log logs/a-kcp-ts.jsonl -kcp-session-stats-log logs/a-kcp-stats.jsonl - -go run ./cmd/kcpping/ -id peer-a -server 106.55.173.235:10909 -echo +./c/bin/kcppeer -id peer-a -server 172.21.32.15:10909 -relay-via 106.55.173.235:10909 \ + -inbox-dir inbox/a \ + -latency-log logs/a-latency.jsonl \ + -kcp-ts-debug-log logs/a-kcp-ts.jsonl \ + -kcp-session-stats-log logs/a-kcp-stats.jsonl ``` -### peer-b (B) +Peer `B` dials `D` directly: ```bash -go run ./cmd/kcppeer/ -id peer-b -server 81.70.156.140:10909 -inbox-dir inbox/b +./c/bin/kcppeer -id peer-b -server 81.70.156.140:10909 \ + -inbox-dir inbox/b \ + -latency-log logs/b-latency.jsonl \ + -kcp-ts-debug-log logs/b-kcp-ts.jsonl \ + -kcp-session-stats-log logs/b-kcp-stats.jsonl +``` --latency-log logs/b-latency.jsonl -kcp-ts-debug-log logs/b-kcp-ts.jsonl -kcp-session-stats-log logs/b-kcp-stats.jsonl +Optional ping / echo tools: -go run ./cmd/kcpping -id peer-b -server 81.70.156.140:10909 -to peer-a -count 20 -interval 100ms +```bash +./c/bin/kcpping -id peer-a -server 106.55.173.235:10909 -echo +./c/bin/kcpping -id peer-b -server 81.70.156.140:10909 -to peer-a -count 20 -interval 100ms +./c/bin/udpserver -listen 0.0.0.0:9001 +./c/bin/udppeer -id peer-a -server 127.0.0.1:9001 +./c/bin/udpping -id pinger -server 127.0.0.1:9001 -to peer-a -count 20 ``` ## Interactive Commands -`peer` 启动后可以在终端里持续使用同一条长连接发送多次消息。 +`udppeer` and `kcppeer` support the same interactive shell: ```text help text peer-b hello text peer-a hi file peer-a /tmp/test125.bin -file peer-a /tmp/test5.bin quit ``` -### 自动化拉取更新汇总数据 -cd /home/limingjie/LMJ_Work/OmniSocketGo -./scripts/refresh-latency-summary.sh \ No newline at end of file + +## Notes + +- The C project targets Linux only. +- It preserves the Go wire format for UDP datagrams and KCP stream frames. +- It keeps runtime JSONL logging, UDP TX timestamp debug, KCP packet debug, and KCP session stats. +- Offline `latencysummary` and HTML chart generation are intentionally not migrated. +- No automated C tests are included in this subtree; validation is expected to happen on Linux via `make` and manual smoke tests. diff --git a/c/README.md b/c/README.md deleted file mode 100644 index 3e98a38..0000000 --- a/c/README.md +++ /dev/null @@ -1,88 +0,0 @@ -# OmniSocketC - -Linux-only C11 implementation of the UDP/KCP transport stack from `OmniSocketGo`. - -This subtree is intentionally standalone. The Go code stays in place as the behavior reference, while the C implementation builds its own binaries under `c/bin/`. - -## Build - -```bash -cd c -make -``` - -Build outputs: - -- `c/bin/udpserver` -- `c/bin/udppeer` -- `c/bin/udpping` -- `c/bin/udprelay` -- `c/bin/kcpserver` -- `c/bin/kcppeer` -- `c/bin/kcpping` - -## Run On Different Machines - -Server `D` runs the KCP hub on `0.0.0.0:10909`: - -```bash -./c/bin/kcpserver -listen 0.0.0.0:10909 \ - -kcp-ts-debug-log logs/d-kcp-ts.jsonl \ - -kcp-session-stats-log logs/d-kcp-stats.jsonl -``` - -Relay `C` runs a raw UDP forwarder to `D`: - -```bash -./c/bin/kcpserver -mode=relay -listen 0.0.0.0:10909 -relay-remote 172.21.32.15:10909 -``` - -Peer `A` dials `D` through relay `C`: - -```bash -./c/bin/kcppeer -id peer-a -server 172.21.32.15:10909 -relay-via 106.55.173.235:10909 \ - -inbox-dir inbox/a \ - -latency-log logs/a-latency.jsonl \ - -kcp-ts-debug-log logs/a-kcp-ts.jsonl \ - -kcp-session-stats-log logs/a-kcp-stats.jsonl -``` - -Peer `B` dials `D` directly: - -```bash -./c/bin/kcppeer -id peer-b -server 81.70.156.140:10909 \ - -inbox-dir inbox/b \ - -latency-log logs/b-latency.jsonl \ - -kcp-ts-debug-log logs/b-kcp-ts.jsonl \ - -kcp-session-stats-log logs/b-kcp-stats.jsonl -``` - -Optional ping / echo tools: - -```bash -./c/bin/kcpping -id peer-a -server 106.55.173.235:10909 -echo -./c/bin/kcpping -id peer-b -server 81.70.156.140:10909 -to peer-a -count 20 -interval 100ms -./c/bin/udpserver -listen 0.0.0.0:9001 -./c/bin/udppeer -id peer-a -server 127.0.0.1:9001 -./c/bin/udpping -id pinger -server 127.0.0.1:9001 -to peer-a -count 20 -``` - -## Interactive Commands - -`udppeer` and `kcppeer` support the same interactive shell: - -```text -help -text peer-b hello -text peer-a hi -file peer-a /tmp/test125.bin -quit -``` - -## Notes - -- The C project targets Linux only. -- It preserves the Go wire format for UDP datagrams and KCP stream frames. -- It keeps runtime JSONL logging, UDP TX timestamp debug, KCP packet debug, and KCP session stats. -- Offline `latencysummary` and HTML chart generation are intentionally not migrated. -- No automated C tests are included in this subtree; validation is expected to happen on Linux via `make` and manual smoke tests. diff --git a/cmd/internal/peer/client.go b/cmd/internal/peer/client.go deleted file mode 100644 index 108e970..0000000 --- a/cmd/internal/peer/client.go +++ /dev/null @@ -1,281 +0,0 @@ -package peer - -import ( - "fmt" - "net" - "os" - "path/filepath" - "sync/atomic" - "syscall" - "time" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -var dialServer = dialServerWithOptions - -type clientOptions struct { - logger latencylog.Logger - txTimestampDebugLogger transport.TXTimestampDebugLogger - kcpPacketDebugLogger transport.KCPPacketDebugLogger - kcpSessionStatsLogger transport.KCPSessionStatsLogger - kcpSessionStatsInterval time.Duration - udpLinuxTimestamping bool - bindIP string - bindDevice string - kcpDialAddress string -} - -// Option 用于配置 Client 的可选行为，例如时延日志。 -type Option func(*clientOptions) - -// WithLogger 为 client 注入时延日志记录器。 -func WithLogger(logger latencylog.Logger) Option { - return func(options *clientOptions) { - options.logger = logger - } -} - -// WithTXTimestampDebugLogger 为 client 注入 TX errqueue 调试日志器。 -func WithTXTimestampDebugLogger(logger transport.TXTimestampDebugLogger) Option { - return func(options *clientOptions) { - options.txTimestampDebugLogger = logger - } -} - -// WithKCPPacketDebugLogger 为 KCP UDP packet timestamp 调试日志注入记录器。 -func WithKCPPacketDebugLogger(logger transport.KCPPacketDebugLogger) Option { - return func(options *clientOptions) { - options.kcpPacketDebugLogger = logger - } -} - -// WithKCPSessionStatsLogger 为 KCP 会话统计日志注入记录器与采样间隔。 -func WithKCPSessionStatsLogger(logger transport.KCPSessionStatsLogger, interval time.Duration) Option { - return func(options *clientOptions) { - options.kcpSessionStatsLogger = logger - options.kcpSessionStatsInterval = interval - } -} - -// WithBindIP 指定拨号时使用的本地源 IP。 -func WithBindIP(ip string) Option { - return func(options *clientOptions) { - options.bindIP = ip - } -} - -// WithBindDevice 指定拨号时绑定的 Linux 网卡名，例如 eth0 或 wwan0。 -func WithBindDevice(device string) Option { - return func(options *clientOptions) { - options.bindDevice = device - } -} - -// WithKCPDialAddress 指定 KCP 实际拨号使用的 UDP 地址，可用于通过 relay 连接逻辑上的 server。 -func WithKCPDialAddress(addr string) Option { - return func(options *clientOptions) { - options.kcpDialAddress = addr - } -} - -// WithUDPLinuxTimestamping controls whether UDP clients enable Linux timestamping. -func WithUDPLinuxTimestamping(enabled bool) Option { - return func(options *clientOptions) { - options.udpLinuxTimestamping = enabled - } -} - -// Client 表示一个已经连接到 server 的 peer。 -type Client struct { - id string - conn *transport.TCPConn - logger latencylog.Logger - - nextID uint64 -} - -// Dial 连接到 server，并立即发送 register 消息完成身份注册。 -func Dial(serverAddr, peerID string, opts ...Option) (*Client, error) { - options := clientOptions{ - logger: latencylog.NoopLogger{}, - udpLinuxTimestamping: true, - } - for _, opt := range opts { - opt(&options) - } - if options.logger == nil { - options.logger = latencylog.NoopLogger{} - } - - rawConn, err := dialServer(serverAddr, options) - if err != nil { - return nil, fmt.Errorf("peer: dial server: %w", err) - } - - conn, err := transport.NewTCPConn( - rawConn, - transport.WithLogger(options.logger, latencylog.NodeRolePeer, peerID), - transport.WithTXTimestampDebugLogger(options.txTimestampDebugLogger), - ) - if err != nil { - _ = rawConn.Close() - return nil, fmt.Errorf("peer: create transport conn: %w", err) - } - client := &Client{ - id: peerID, - conn: conn, - logger: options.logger, - } - - if err := conn.Send(protocol.Message{ //向 server 发送一条 register 消息，完成身份注册。 - Type: protocol.MessageTypeRegister, - From: peerID, - To: protocol.ServerPeerID, - }); err != nil { - _ = conn.Close() - return nil, fmt.Errorf("peer: register with server: %w", err) - } - - return client, nil -} - -// ID 返回当前 client 的 peer 标识。 -func (c *Client) ID() string { - return c.id -} - -// SendText 向目标 peer 发送一条文本消息。 -func (c *Client) SendText(to, body string) error { - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: c.nextMessageID(), - From: c.id, - To: to, - } - // 记录 A 端应用开始准备消息的时间点。 - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventAAppPrepBegin, msg) - - msg.Body = []byte(body) - - return c.conn.Send(msg) -} - -// SendFile 向目标 peer 发送一条文件消息。 -func (c *Client) SendFile(to, fileName string, body []byte) error { - msg := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: c.nextMessageID(), - From: c.id, - To: to, - FileName: fileName, - } - // 记录 A 端应用开始准备消息的时间点。 - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventAAppPrepBegin, msg) - - bodyCopy := make([]byte, len(body)) - copy(bodyCopy, body) - - msg.Body = bodyCopy - - return c.conn.Send(msg) -} - -// SendFilePath 从本地文件读取内容并发送给目标 peer。 -func (c *Client) SendFilePath(to, path string) error { - msg := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: c.nextMessageID(), - From: c.id, - To: to, - FileName: filepath.Base(path), - } - // 记录 A 端应用开始准备消息的时间点。 - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventAAppPrepBegin, msg) - - body, err := os.ReadFile(path) - if err != nil { - return fmt.Errorf("peer: read file %s: %w", path, err) - } - - msg.Body = body - - return c.conn.Send(msg) -} - -// Receive 读取一条来自 server 的消息。 -func (c *Client) Receive() (protocol.Message, error) { - msg, err := c.conn.Receive() //从底层 TCP 连接读取一条消息，返回一个 protocol.Message 结构体。 - if err != nil { - return protocol.Message{}, fmt.Errorf("peer: receive from server: %w", err) - } - - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBAppRecv, msg) - - return msg, nil -} - -// ReceiveLoop 持续接收 server 消息并交给 handler 处理。 -func (c *Client) ReceiveLoop(handler func(protocol.Message) error) error { - return c.conn.ReceiveLoop(func(msg protocol.Message) error { - switch msg.Type { - case protocol.MessageTypeText, protocol.MessageTypeFile, protocol.MessageTypeError: - // 记录 B 端应用真正读到完整消息的时间点。 - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBAppRecv, msg) - return handler(msg) - default: - return fmt.Errorf("peer: unexpected message type from server: %s", msg.Type) - } - }) -} - -// Close 关闭与 server 的连接。 -func (c *Client) Close() error { - return c.conn.Close() -} - -func (c *Client) nextMessageID() uint64 { - return atomic.AddUint64(&c.nextID, 1) -} - -// 根据提供的选项创建 TCP 连接，并进行必要的配置，例如绑定本地 IP 或网卡。成功建立连接后，返回一个封装了该连接的 TCPConn 实例。 -func dialServerWithOptions(serverAddr string, options clientOptions) (net.Conn, error) { - dialer, err := buildDialer(options) - if err != nil { - return nil, err - } - - return dialer.Dial("tcp", serverAddr) -} - -func buildDialer(options clientOptions) (*net.Dialer, error) { - dialer := &net.Dialer{} - - if options.bindIP != "" { - ip := net.ParseIP(options.bindIP) - if ip == nil { - return nil, fmt.Errorf("peer: invalid bind ip %q", options.bindIP) - } - dialer.LocalAddr = &net.TCPAddr{IP: ip} - } - - if options.bindDevice != "" { - device := options.bindDevice - dialer.Control = func(_, _ string, rawConn syscall.RawConn) error { - var bindErr error - if err := rawConn.Control(func(fd uintptr) { - bindErr = syscall.BindToDevice(int(fd), device) - }); err != nil { - return err - } - if bindErr != nil { - return fmt.Errorf("peer: bind device %s: %w", device, bindErr) - } - return nil - } - } - - return dialer, nil -} diff --git a/cmd/internal/peer/client_linux_test.go b/cmd/internal/peer/client_linux_test.go deleted file mode 100644 index f5432af..0000000 --- a/cmd/internal/peer/client_linux_test.go +++ /dev/null @@ -1,188 +0,0 @@ -//go:build linux - -package peer - -import ( - "net" - "strings" - "sync" - "testing" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/server" -) - -func TestClientsExchangeMessagesWithLinuxTimestamps(t *testing.T) { - hub := server.NewHub() - serverAddr, cleanup := startRealHubServer(t, hub) - defer cleanup() - - peerALogger := &recordingLogger{} - peerA, err := Dial(serverAddr, "peer-a", WithLogger(peerALogger)) - if err != nil { - t.Fatalf("Dial(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerBLogger := &recordingLogger{} - peerB, err := Dial(serverAddr, "peer-b", WithLogger(peerBLogger)) - if err != nil { - t.Fatalf("Dial(peer-b) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - inboxDir := t.TempDir() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered") - - if err := peerA.SendText("peer-b", "hello"); err != nil { - t.Fatalf("SendText() error = %v", err) - } - textMsg, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive(text) error = %v", err) - } - if _, err := peerB.PersistMessage(textMsg, inboxDir); err != nil { - t.Fatalf("peerB.PersistMessage(text) error = %v", err) - } - - if err := peerA.SendFile("peer-b", "payload.bin", []byte{0x01, 0x02, 0x03}); err != nil { - t.Fatalf("SendFile() error = %v", err) - } - fileMsg, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive(file) error = %v", err) - } - if _, err := peerB.PersistMessage(fileMsg, inboxDir); err != nil { - t.Fatalf("peerB.PersistMessage(file) error = %v", err) - } - - waitFor(t, func() bool { return hasMessageEvents(peerALogger.Events(), 1, latencylog.EventAAppPrepBegin, latencylog.EventATXSched, latencylog.EventATXSoftware) }, "peer-a text kernel timestamps") - waitFor(t, func() bool { return hasMessageEvents(peerALogger.Events(), 2, latencylog.EventAAppPrepBegin, latencylog.EventATXSched, latencylog.EventATXSoftware) }, "peer-a file kernel timestamps") - waitFor(t, func() bool { return hasMessageEvents(peerBLogger.Events(), 1, latencylog.EventBRXSoftware, latencylog.EventBAppRecv, latencylog.EventBPersistBegin, latencylog.EventBPersistEnd) }, "peer-b text receive timestamps") - waitFor(t, func() bool { return hasMessageEvents(peerBLogger.Events(), 2, latencylog.EventBRXSoftware, latencylog.EventBAppRecv, latencylog.EventBPersistBegin, latencylog.EventBPersistEnd) }, "peer-b file receive timestamps") -} - -func startRealHubServer(t *testing.T, hub *server.Hub) (string, func()) { - t.Helper() - - listener, err := net.Listen("tcp", "127.0.0.1:0") - if err != nil { - t.Fatalf("net.Listen() error = %v", err) - } - - var ( - wg sync.WaitGroup - stop = make(chan struct{}) - errOnce sync.Once - ) - - wg.Add(1) - go func() { - defer wg.Done() - for { - conn, acceptErr := listener.Accept() - if acceptErr != nil { - select { - case <-stop: - return - default: - } - if strings.Contains(acceptErr.Error(), "closed") { - return - } - t.Errorf("listener.Accept() error = %v", acceptErr) - return - } - - wg.Add(1) - go func(rawConn net.Conn) { - defer wg.Done() - if serveErr := hub.ServeConn(rawConn); serveErr != nil && !isExpectedHubServeExit(serveErr) { - errOnce.Do(func() { - t.Logf("hub.ServeConn() ended with %v", serveErr) - }) - } - }(conn) - } - }() - - cleanup := func() { - close(stop) - _ = listener.Close() - wg.Wait() - } - - return listener.Addr().String(), cleanup -} - -func hasMessageEvents(events []latencylog.Event, messageID uint64, wantEvents ...string) bool { - seen := make(map[string]bool, len(wantEvents)) - for _, event := range events { - if event.MessageID != messageID { - continue - } - if event.TsUnixNano <= 0 { - return false - } - seen[event.Event] = true - } - - for _, wantEvent := range wantEvents { - if !seen[wantEvent] { - return false - } - } - - return true -} - -func isExpectedHubServeExit(err error) bool { - if err == nil { - return true - } - - message := err.Error() - return strings.Contains(message, "closed") || strings.Contains(message, "protocol: read frame: EOF") -} - -func TestLinuxTimestampedReceivePreservesBusinessMessageShape(t *testing.T) { - hub := server.NewHub() - serverAddr, cleanup := startRealHubServer(t, hub) - defer cleanup() - - peerA, err := Dial(serverAddr, "peer-a") - if err != nil { - t.Fatalf("Dial(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerB, err := Dial(serverAddr, "peer-b") - if err != nil { - t.Fatalf("Dial(peer-b) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered") - - want := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 1, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0xde, 0xad, 0xbe, 0xef}, - } - if err := peerA.SendFile(want.To, want.FileName, want.Body); err != nil { - t.Fatalf("SendFile() error = %v", err) - } - - got, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive() error = %v", err) - } - if got.Type != want.Type || got.ID != want.ID || got.From != want.From || got.To != want.To || got.FileName != want.FileName || string(got.Body) != string(want.Body) { - t.Fatalf("received message mismatch: got %+v want %+v", got, want) - } -} diff --git a/cmd/internal/peer/client_test.go b/cmd/internal/peer/client_test.go deleted file mode 100644 index 0a34a7f..0000000 --- a/cmd/internal/peer/client_test.go +++ /dev/null @@ -1,819 +0,0 @@ -package peer - -import ( - "bytes" - "encoding/json" - "net" - "os" - "path/filepath" - "reflect" - "strings" - "sync" - "testing" - "time" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/server" - "omnisocketgo/cmd/internal/transport" -) - -type recordingLogger struct { - mu sync.Mutex - events []latencylog.Event -} - -func (l *recordingLogger) LogEvent(event latencylog.Event) error { - l.mu.Lock() - defer l.mu.Unlock() - - l.events = append(l.events, event) - return nil -} - -func (l *recordingLogger) Events() []latencylog.Event { - l.mu.Lock() - defer l.mu.Unlock() - - return append([]latencylog.Event(nil), l.events...) -} - -type failingLogger struct{} - -func (failingLogger) LogEvent(latencylog.Event) error { - return net.ErrClosed -} - -func TestDialRegistersPeer(t *testing.T) { - hub := server.NewHub() - cleanup := stubDialToHub(t, hub) - defer cleanup() - - client, err := Dial("ignored", "peer-a") - if err != nil { - t.Fatalf("Dial() error = %v", err) - } - defer func() { _ = client.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") -} - -func TestDialRegistersPeerWithBindIP(t *testing.T) { - hub := server.NewHub() - cleanup := stubDialToHub(t, hub) - defer cleanup() - - client, err := Dial("ignored", "peer-a", WithBindIP("127.0.0.1")) - if err != nil { - t.Fatalf("Dial() with bind ip error = %v", err) - } - defer func() { _ = client.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") -} - -func TestDialRejectsInvalidBindIP(t *testing.T) { - _, err := Dial("ignored", "peer-a", WithBindIP("not-an-ip")) - if err == nil { - t.Fatal("Dial() error = nil, want invalid bind ip error") - } - if !strings.Contains(err.Error(), `invalid bind ip "not-an-ip"`) { - t.Fatalf("Dial() error = %v, want invalid bind ip error", err) - } -} - -func TestDialPassesBindDeviceOptionToDialer(t *testing.T) { - originalDial := dialServer - defer func() { - dialServer = originalDial - }() - - gotDevice := "" - dialServer = func(_ string, options clientOptions) (net.Conn, error) { - gotDevice = options.bindDevice - return nil, net.ErrClosed - } - - _, err := Dial("ignored", "peer-a", WithBindDevice("wwan0")) - if err == nil { - t.Fatal("Dial() error = nil, want dial error") - } - if gotDevice != "wwan0" { - t.Fatalf("bind device = %q, want %q", gotDevice, "wwan0") - } -} - -func TestClientsExchangeTextAndFileMessages(t *testing.T) { - hub := server.NewHub() - cleanup := stubDialToHub(t, hub) - defer cleanup() - - peerA, err := Dial("ignored", "peer-a") - if err != nil { - t.Fatalf("Dial(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerB, err := Dial("ignored", "peer-b") - if err != nil { - t.Fatalf("Dial(peer-b) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered") - - received := make(chan protocol.Message, 2) - receiveErr := make(chan error, 1) - go func() { - for i := 0; i < 2; i++ { - msg, err := peerB.Receive() - if err != nil { - receiveErr <- err - return - } - received <- msg - } - receiveErr <- nil - }() - - if err := peerA.SendText("peer-b", "hello"); err != nil { - t.Fatalf("SendText() error = %v", err) - } - fileBody := []byte{0x01, 0x02, 0x03} - if err := peerA.SendFile("peer-b", "payload.bin", fileBody); err != nil { - t.Fatalf("SendFile() error = %v", err) - } - - if err := <-receiveErr; err != nil { - t.Fatalf("peerB.Receive() error = %v", err) - } - - gotFirst := <-received - wantFirst := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - if !reflect.DeepEqual(gotFirst, wantFirst) { - t.Fatalf("first message mismatch: got %+v want %+v", gotFirst, wantFirst) - } - - gotSecond := <-received - wantSecond := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 2, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: fileBody, - } - if !reflect.DeepEqual(gotSecond, wantSecond) { - t.Fatalf("second message mismatch: got %+v want %+v", gotSecond, wantSecond) - } -} - -func TestClientReceivesServerErrorForUnknownTarget(t *testing.T) { - hub := server.NewHub() - cleanup := stubDialToHub(t, hub) - defer cleanup() - - client, err := Dial("ignored", "peer-a") - if err != nil { - t.Fatalf("Dial() error = %v", err) - } - defer func() { _ = client.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") - - if err := client.SendText("missing-peer", "hello"); err != nil { - t.Fatalf("SendText() error = %v", err) - } - - got, err := client.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if got.Type != protocol.MessageTypeError { - t.Fatalf("got type %s, want %s", got.Type, protocol.MessageTypeError) - } - if string(got.Body) != "unknown target: missing-peer" { - t.Fatalf("error body = %q, want unknown target message", got.Body) - } -} - -func TestClientReceiveLoopHandlesForwardedMessages(t *testing.T) { - hub := server.NewHub() - cleanup := stubDialToHub(t, hub) - defer cleanup() - - peerA, err := Dial("ignored", "peer-a") - if err != nil { - t.Fatalf("Dial(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerB, err := Dial("ignored", "peer-b") - if err != nil { - t.Fatalf("Dial(peer-b) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered") - - var ( - mu sync.Mutex - got []protocol.Message - ) - loopErr := make(chan error, 1) - go func() { - loopErr <- peerB.ReceiveLoop(func(msg protocol.Message) error { - mu.Lock() - defer mu.Unlock() - got = append(got, msg) - if len(got) == 1 { - return peerB.Close() - } - return nil - }) - }() - - if err := peerA.SendText("peer-b", "hello"); err != nil { - t.Fatalf("SendText() error = %v", err) - } - - err = <-loopErr - if err == nil || (!strings.Contains(err.Error(), "closed") && !strings.Contains(err.Error(), "use of closed network connection")) { - t.Fatalf("ReceiveLoop() error = %v, want close-related error", err) - } - - mu.Lock() - defer mu.Unlock() - want := []protocol.Message{ - { - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - }, - } - if !reflect.DeepEqual(got, want) { - t.Fatalf("received messages mismatch: got %+v want %+v", got, want) - } -} - -func TestClientSendLogsLatencyEvents(t *testing.T) { - tests := []struct { - name string - setup func(*testing.T) string - send func(*Client, string) error - wantMsg protocol.Message - wantEvents []string - }{ - { - name: "text", - send: func(client *Client, _ string) error { - return client.SendText("peer-b", "hello") - }, - wantMsg: protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - }, - wantEvents: []string{ - latencylog.EventAAppPrepBegin, - latencylog.EventSendHandoffBegin, - latencylog.EventATXSched, - latencylog.EventATXSoftware, - latencylog.EventSendHandoffEnd, - }, - }, - { - name: "file-bytes", - send: func(client *Client, _ string) error { - return client.SendFile("peer-b", "payload.bin", []byte{0x01, 0x02, 0x03}) - }, - wantMsg: protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 1, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0x01, 0x02, 0x03}, - }, - wantEvents: []string{ - latencylog.EventAAppPrepBegin, - latencylog.EventSendHandoffBegin, - latencylog.EventATXSched, - latencylog.EventATXSoftware, - latencylog.EventSendHandoffEnd, - }, - }, - { - name: "file-path", - setup: func(t *testing.T) string { - t.Helper() - - path := filepath.Join(t.TempDir(), "payload.bin") - if err := os.WriteFile(path, []byte{0x01, 0x02, 0x03}, 0o644); err != nil { - t.Fatalf("os.WriteFile() error = %v", err) - } - - return path - }, - send: func(client *Client, path string) error { - return client.SendFilePath("peer-b", path) - }, - wantMsg: protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 1, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0x01, 0x02, 0x03}, - }, - wantEvents: []string{ - latencylog.EventAAppPrepBegin, - latencylog.EventSendHandoffBegin, - latencylog.EventATXSched, - latencylog.EventATXSoftware, - latencylog.EventSendHandoffEnd, - }, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - inputPath := "" - if tt.setup != nil { - inputPath = tt.setup(t) - } - - logger := &recordingLogger{} - clientConn, receiver := newClientTransportPair( - t, - []transport.Option{transport.WithLogger(logger, latencylog.NodeRolePeer, "peer-a")}, - nil, - ) - client := &Client{ - id: "peer-a", - conn: clientConn, - logger: logger, - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- tt.send(client, inputPath) - }() - - got, err := receiver.Receive() - if err != nil { - t.Fatalf("receiver.Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("send() error = %v", err) - } - if !reflect.DeepEqual(got, tt.wantMsg) { - t.Fatalf("message mismatch: got %+v want %+v", got, tt.wantMsg) - } - - events := logger.Events() - if len(events) != len(tt.wantEvents) { - t.Fatalf("event count = %d, want %d", len(events), len(tt.wantEvents)) - } - for i, wantEvent := range tt.wantEvents { - if events[i].Event != wantEvent { - t.Fatalf("event[%d] = %q, want %q", i, events[i].Event, wantEvent) - } - if events[i].MessageID != tt.wantMsg.ID || events[i].From != tt.wantMsg.From || events[i].To != tt.wantMsg.To { - t.Fatalf("event[%d] metadata mismatch: %+v", i, events[i]) - } - } - }) - } -} - -func TestClientReceiveLogsOnlyBusinessMessages(t *testing.T) { - logger := &recordingLogger{} - clientConn, sender := newClientTransportPair( - t, - []transport.Option{transport.WithLogger(logger, latencylog.NodeRolePeer, "peer-b")}, - nil, - ) - client := &Client{ - id: "peer-b", - conn: clientConn, - logger: logger, - } - - textMsg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 21, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(textMsg) - }() - if _, err := client.Receive(); err != nil { - t.Fatalf("client.Receive(text) error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("sender.Send(text) error = %v", err) - } - - errorMsg := protocol.Message{ - Type: protocol.MessageTypeError, - ID: 22, - From: protocol.ServerPeerID, - To: "peer-b", - Body: []byte("failure"), - } - sendErr = make(chan error, 1) - go func() { - sendErr <- sender.Send(errorMsg) - }() - if _, err := client.Receive(); err != nil { - t.Fatalf("client.Receive(error) error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("sender.Send(error) error = %v", err) - } - - events := logger.Events() - if len(events) != 2 { - t.Fatalf("event count = %d, want 2", len(events)) - } - if events[0].Event != latencylog.EventBRXSoftware { - t.Fatalf("first event = %q, want %q", events[0].Event, latencylog.EventBRXSoftware) - } - if events[1].Event != latencylog.EventBAppRecv { - t.Fatalf("second event = %q, want %q", events[1].Event, latencylog.EventBAppRecv) - } - if events[0].MessageID != textMsg.ID || events[1].MessageID != textMsg.ID { - t.Fatalf("message IDs = %d,%d, want %d", events[0].MessageID, events[1].MessageID, textMsg.ID) - } -} - -func TestClientPersistTextMessageWritesInboxFileAndLogs(t *testing.T) { - inboxDir := t.TempDir() - logger := &recordingLogger{} - client := &Client{ - id: "peer-b", - logger: logger, - } - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 31, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - - path, err := client.PersistMessage(msg, inboxDir) - if err != nil { - t.Fatalf("PersistMessage() error = %v", err) - } - - if path != filepath.Join(inboxDir, textInboxFileName) { - t.Fatalf("path = %q, want %q", path, filepath.Join(inboxDir, textInboxFileName)) - } - - data, err := os.ReadFile(path) - if err != nil { - t.Fatalf("os.ReadFile() error = %v", err) - } - - var record textInboxRecord - if err := json.Unmarshal(bytes.TrimSpace(data), &record); err != nil { - t.Fatalf("json.Unmarshal() error = %v", err) - } - if record.MessageID != msg.ID || record.From != msg.From || record.To != msg.To || record.Body != "hello" { - t.Fatalf("record mismatch: got %+v want message %+v", record, msg) - } - - events := logger.Events() - if len(events) != 2 { - t.Fatalf("event count = %d, want 2", len(events)) - } - if events[0].Event != latencylog.EventBPersistBegin { - t.Fatalf("first event = %q, want %q", events[0].Event, latencylog.EventBPersistBegin) - } - if events[1].Event != latencylog.EventBPersistEnd { - t.Fatalf("second event = %q, want %q", events[1].Event, latencylog.EventBPersistEnd) - } -} - -func TestClientPersistFileMessageWritesInboxFileAndLogs(t *testing.T) { - inboxDir := t.TempDir() - logger := &recordingLogger{} - client := &Client{ - id: "peer-b", - logger: logger, - } - - msg := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 32, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0x01, 0x02, 0x03}, - } - - path, err := client.PersistMessage(msg, inboxDir) - if err != nil { - t.Fatalf("PersistMessage() error = %v", err) - } - - wantPath := filepath.Join(inboxDir, "peer-a-32-payload.bin") - if path != wantPath { - t.Fatalf("path = %q, want %q", path, wantPath) - } - - data, err := os.ReadFile(path) - if err != nil { - t.Fatalf("os.ReadFile() error = %v", err) - } - if !reflect.DeepEqual(data, msg.Body) { - t.Fatalf("file body mismatch: got %v want %v", data, msg.Body) - } - - events := logger.Events() - if len(events) != 2 { - t.Fatalf("event count = %d, want 2", len(events)) - } - if events[0].Event != latencylog.EventBPersistBegin { - t.Fatalf("first event = %q, want %q", events[0].Event, latencylog.EventBPersistBegin) - } - if events[1].Event != latencylog.EventBPersistEnd { - t.Fatalf("second event = %q, want %q", events[1].Event, latencylog.EventBPersistEnd) - } -} - -func TestClientPersistMessageReturnsErrorOnWriteFailure(t *testing.T) { - blocker := filepath.Join(t.TempDir(), "blocker") - if err := os.WriteFile(blocker, []byte("not a directory"), 0o644); err != nil { - t.Fatalf("os.WriteFile() error = %v", err) - } - - logger := &recordingLogger{} - client := &Client{ - id: "peer-b", - logger: logger, - } - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 33, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - - if _, err := client.PersistMessage(msg, blocker); err == nil { - t.Fatal("PersistMessage() error = nil, want non-nil") - } - - events := logger.Events() - if len(events) != 1 { - t.Fatalf("event count = %d, want 1", len(events)) - } - if events[0].Event != latencylog.EventBPersistBegin { - t.Fatalf("event = %q, want %q", events[0].Event, latencylog.EventBPersistBegin) - } -} - -func TestClientIgnoresLoggerFailure(t *testing.T) { - clientConn, receiver := newClientTransportPair( - t, - []transport.Option{transport.WithLogger(failingLogger{}, latencylog.NodeRolePeer, "peer-a")}, - nil, - ) - client := &Client{ - id: "peer-a", - conn: clientConn, - logger: failingLogger{}, - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- client.SendText("peer-b", "hello") - }() - - got, err := receiver.Receive() - if err != nil { - t.Fatalf("receiver.Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("SendText() error = %v, want nil even if logger fails", err) - } - if string(got.Body) != "hello" { - t.Fatalf("body = %q, want hello", got.Body) - } -} - -func TestClientPersistIgnoresLoggerFailure(t *testing.T) { - client := &Client{ - id: "peer-b", - logger: failingLogger{}, - } - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 34, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - - path, err := client.PersistMessage(msg, t.TempDir()) - if err != nil { - t.Fatalf("PersistMessage() error = %v, want nil even if logger fails", err) - } - if path == "" { - t.Fatal("PersistMessage() path = empty, want non-empty") - } -} - -func TestClientsExchangeMessagesWithLatencyLogs(t *testing.T) { - hub := server.NewHub() - cleanup := stubDialToHub(t, hub) - defer cleanup() - - peerALogger := &recordingLogger{} - peerA, err := Dial("ignored", "peer-a", WithLogger(peerALogger)) - if err != nil { - t.Fatalf("Dial(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerBLogger := &recordingLogger{} - peerB, err := Dial("ignored", "peer-b", WithLogger(peerBLogger)) - if err != nil { - t.Fatalf("Dial(peer-b) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - inboxDir := t.TempDir() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered") - - if err := peerA.SendText("peer-b", "hello"); err != nil { - t.Fatalf("SendText() error = %v", err) - } - textMsg, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive(text) error = %v", err) - } - if _, err := peerB.PersistMessage(textMsg, inboxDir); err != nil { - t.Fatalf("peerB.PersistMessage(text) error = %v", err) - } - - if err := peerA.SendFile("peer-b", "payload.bin", []byte{0x01, 0x02, 0x03}); err != nil { - t.Fatalf("SendFile() error = %v", err) - } - fileMsg, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive(file) error = %v", err) - } - if _, err := peerB.PersistMessage(fileMsg, inboxDir); err != nil { - t.Fatalf("peerB.PersistMessage(file) error = %v", err) - } - - waitFor(t, func() bool { return len(peerALogger.Events()) == 10 }, "peer-a latency events") - waitFor(t, func() bool { return len(peerBLogger.Events()) == 8 }, "peer-b latency events") - - assertEventSequencesByMessage(t, peerALogger.Events(), map[uint64][]string{ - 1: {latencylog.EventAAppPrepBegin, latencylog.EventSendHandoffBegin, latencylog.EventATXSched, latencylog.EventATXSoftware, latencylog.EventSendHandoffEnd}, - 2: {latencylog.EventAAppPrepBegin, latencylog.EventSendHandoffBegin, latencylog.EventATXSched, latencylog.EventATXSoftware, latencylog.EventSendHandoffEnd}, - }) - assertEventSequencesByMessage(t, peerBLogger.Events(), map[uint64][]string{ - 1: {latencylog.EventBRXSoftware, latencylog.EventBAppRecv, latencylog.EventBPersistBegin, latencylog.EventBPersistEnd}, - 2: {latencylog.EventBRXSoftware, latencylog.EventBAppRecv, latencylog.EventBPersistBegin, latencylog.EventBPersistEnd}, - }) -} - -func stubDialToHub(t *testing.T, hub *server.Hub) func() { - t.Helper() - - originalDial := dialServer - serverAddr, cleanup := startRealHubServer(t, hub) - - dialServer = func(_ string, options clientOptions) (net.Conn, error) { - dialer, err := buildDialer(options) - if err != nil { - return nil, err - } - return dialer.Dial("tcp", serverAddr) - } - - return func() { - dialServer = originalDial - cleanup() - } -} - -func waitFor(t *testing.T, condition func() bool, description string) { - t.Helper() - - deadline := time.Now().Add(500 * time.Millisecond) - for time.Now().Before(deadline) { - if condition() { - return - } - time.Sleep(10 * time.Millisecond) - } - - t.Fatalf("timed out waiting for %s", description) -} - -func assertEventSequencesByMessage(t *testing.T, events []latencylog.Event, want map[uint64][]string) { - t.Helper() - - grouped := make(map[uint64][]latencylog.Event) - for _, event := range events { - grouped[event.MessageID] = append(grouped[event.MessageID], event) - if event.TsUnixNano <= 0 { - t.Fatalf("event timestamp must be positive: %+v", event) - } - } - - if len(grouped) != len(want) { - t.Fatalf("message group count = %d, want %d", len(grouped), len(want)) - } - - for messageID, wantEvents := range want { - gotEvents := grouped[messageID] - if len(gotEvents) != len(wantEvents) { - t.Fatalf("message %d event count = %d, want %d", messageID, len(gotEvents), len(wantEvents)) - } - for i, wantEvent := range wantEvents { - if gotEvents[i].Event != wantEvent { - t.Fatalf("message %d event[%d] = %q, want %q", messageID, i, gotEvents[i].Event, wantEvent) - } - } - } -} - -func newClientTransportPair(t *testing.T, clientOpts []transport.Option, peerOpts []transport.Option) (*transport.TCPConn, *transport.TCPConn) { - t.Helper() - - listener, err := net.Listen("tcp", "127.0.0.1:0") - if err != nil { - t.Fatalf("net.Listen() error = %v", err) - } - - type acceptResult struct { - conn net.Conn - err error - } - - accepted := make(chan acceptResult, 1) - go func() { - conn, acceptErr := listener.Accept() - accepted <- acceptResult{conn: conn, err: acceptErr} - }() - - clientSide, err := net.Dial("tcp", listener.Addr().String()) - if err != nil { - _ = listener.Close() - t.Fatalf("net.Dial() error = %v", err) - } - - result := <-accepted - if err := listener.Close(); err != nil { - t.Fatalf("listener.Close() error = %v", err) - } - if result.err != nil { - _ = clientSide.Close() - t.Fatalf("listener.Accept() error = %v", result.err) - } - - clientConn, err := transport.NewTCPConn(clientSide, clientOpts...) - if err != nil { - _ = clientSide.Close() - _ = result.conn.Close() - t.Fatalf("transport.NewTCPConn(client) error = %v", err) - } - peerConn, err := transport.NewTCPConn(result.conn, peerOpts...) - if err != nil { - _ = clientConn.Close() - _ = result.conn.Close() - t.Fatalf("transport.NewTCPConn(peer) error = %v", err) - } - - t.Cleanup(func() { - _ = clientConn.Close() - _ = peerConn.Close() - }) - - return clientConn, peerConn -} diff --git a/cmd/internal/peer/kcp_client.go b/cmd/internal/peer/kcp_client.go deleted file mode 100644 index 246d8e3..0000000 --- a/cmd/internal/peer/kcp_client.go +++ /dev/null @@ -1,190 +0,0 @@ -package peer - -import ( - "fmt" - "os" - "path/filepath" - "sync/atomic" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -// KCPClient 表示一个通过 KCP 连接到 server 的 peer。 -type KCPClient struct { - id string - conn *transport.KCPConn - logger latencylog.Logger - - nextID uint64 -} - -// DialKCP 通过 KCP 连接到 server，并发送 register 消息完成身份注册。 -func DialKCP(serverAddr, peerID string, opts ...Option) (*KCPClient, error) { - options := clientOptions{ - logger: latencylog.NoopLogger{}, - } - for _, opt := range opts { - opt(&options) - } - if options.logger == nil { - options.logger = latencylog.NoopLogger{} - } - - dialAddr := serverAddr - if options.kcpDialAddress != "" { - dialAddr = options.kcpDialAddress - } - - session, err := transport.DialKCPSession( - dialAddr, - options.bindIP, - options.bindDevice, - options.kcpPacketDebugLogger, - latencylog.NodeRolePeer, - peerID, - ) - if err != nil { - return nil, fmt.Errorf("peer: dial kcp server: %w", err) - } - - conn, err := transport.NewKCPConn( - session, - transport.WithKCPLogger(options.logger, latencylog.NodeRolePeer, peerID), - transport.WithKCPSessionStatsLogger(options.kcpSessionStatsLogger, options.kcpSessionStatsInterval), - ) - if err != nil { - _ = session.Close() - return nil, fmt.Errorf("peer: create kcp transport conn: %w", err) - } - - client := &KCPClient{ - id: peerID, - conn: conn, - logger: options.logger, - } - - if err := conn.Send(protocol.Message{ - Type: protocol.MessageTypeRegister, - From: peerID, - To: protocol.ServerPeerID, - }); err != nil { - _ = conn.Close() - return nil, fmt.Errorf("peer: register with kcp server: %w", err) - } - - return client, nil -} - -// ID 返回当前 client 的 peer 标识。 -func (c *KCPClient) ID() string { - return c.id -} - -// SendText 向目标 peer 发送一条文本消息。 -func (c *KCPClient) SendText(to, body string) error { - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: c.nextMessageID(), - From: c.id, - To: to, - } - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventAAppPrepBegin, msg) - msg.Body = []byte(body) - return c.conn.Send(msg) -} - -// SendFile 向目标 peer 发送一条文件消息。 -func (c *KCPClient) SendFile(to, fileName string, body []byte) error { - msg := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: c.nextMessageID(), - From: c.id, - To: to, - FileName: fileName, - } - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventAAppPrepBegin, msg) - - bodyCopy := make([]byte, len(body)) - copy(bodyCopy, body) - msg.Body = bodyCopy - - return c.conn.Send(msg) -} - -// SendFilePath 从本地文件读取内容并发送给目标 peer。 -func (c *KCPClient) SendFilePath(to, path string) error { - msg := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: c.nextMessageID(), - From: c.id, - To: to, - FileName: filepath.Base(path), - } - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventAAppPrepBegin, msg) - - body, err := os.ReadFile(path) - if err != nil { - return fmt.Errorf("peer: read file %s: %w", path, err) - } - msg.Body = body - - return c.conn.Send(msg) -} - -// Receive 读取一条来自 server 的消息。 -func (c *KCPClient) Receive() (protocol.Message, error) { - msg, err := c.conn.Receive() - if err != nil { - return protocol.Message{}, fmt.Errorf("peer: receive from kcp server: %w", err) - } - - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBAppRecv, msg) - return msg, nil -} - -// ReceiveLoop 持续接收 server 消息并交给 handler 处理。 -func (c *KCPClient) ReceiveLoop(handler func(protocol.Message) error) error { - return c.conn.ReceiveLoop(func(msg protocol.Message) error { - switch msg.Type { - case protocol.MessageTypeText, protocol.MessageTypeFile, protocol.MessageTypeError: - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBAppRecv, msg) - return handler(msg) - default: - return fmt.Errorf("peer: unexpected message type from kcp server: %s", msg.Type) - } - }) -} - -// PersistMessage 将收到的业务消息写入本地磁盘。 -func (c *KCPClient) PersistMessage(msg protocol.Message, inboxDir string) (string, error) { - if !latencylog.IsBusinessMessage(msg) { - return "", fmt.Errorf("peer: cannot persist message type %s", msg.Type) - } - if inboxDir == "" { - return "", fmt.Errorf("peer: inbox directory is required") - } - - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBPersistBegin, msg) - if err := os.MkdirAll(inboxDir, 0o755); err != nil { - return "", fmt.Errorf("peer: create inbox dir %s: %w", inboxDir, err) - } - - path, err := persistMessageToDisk(msg, inboxDir) - if err != nil { - return "", err - } - - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBPersistEnd, msg) - return path, nil -} - -// Close 关闭与 server 的 KCP 会话。 -func (c *KCPClient) Close() error { - return c.conn.Close() -} - -func (c *KCPClient) nextMessageID() uint64 { - return atomic.AddUint64(&c.nextID, 1) -} diff --git a/cmd/internal/peer/kcp_client_test.go b/cmd/internal/peer/kcp_client_test.go deleted file mode 100644 index f6ffcf2..0000000 --- a/cmd/internal/peer/kcp_client_test.go +++ /dev/null @@ -1,636 +0,0 @@ -package peer - -import ( - "bytes" - "net" - "os" - "path/filepath" - "reflect" - "strings" - "sync" - "sync/atomic" - "testing" - - kcp "github.com/xtaci/kcp-go/v5" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/server" - "omnisocketgo/cmd/internal/transport" -) - -func TestKCPDialRegistersPeer(t *testing.T) { - hub := server.NewKCPHub() - serverAddr, cleanup := startRealKCPHubServer(t, hub) - defer cleanup() - - client, err := DialKCP(serverAddr, "peer-a") - if err != nil { - t.Fatalf("DialKCP() error = %v", err) - } - defer func() { _ = client.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") -} - -func TestKCPDialRejectsInvalidBindIP(t *testing.T) { - _, err := DialKCP("127.0.0.1:9002", "peer-a", WithBindIP("not-an-ip")) - if err == nil { - t.Fatal("DialKCP() error = nil, want invalid bind ip error") - } - if !strings.Contains(err.Error(), `invalid bind ip "not-an-ip"`) { - t.Fatalf("DialKCP() error = %v, want invalid bind ip error", err) - } -} - -func TestKCPClientsExchangeTextAndFileMessages(t *testing.T) { - hub := server.NewKCPHub() - serverAddr, cleanup := startRealKCPHubServer(t, hub) - defer cleanup() - - peerA, err := DialKCP(serverAddr, "peer-a") - if err != nil { - t.Fatalf("DialKCP(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerB, err := DialKCP(serverAddr, "peer-b") - if err != nil { - t.Fatalf("DialKCP(peer-b) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered") - - received := make(chan protocol.Message, 2) - receiveErr := make(chan error, 1) - go func() { - for i := 0; i < 2; i++ { - msg, err := peerB.Receive() - if err != nil { - receiveErr <- err - return - } - received <- msg - } - receiveErr <- nil - }() - - if err := peerA.SendText("peer-b", "hello over kcp"); err != nil { - t.Fatalf("SendText() error = %v", err) - } - fileBody := []byte{0x01, 0x02, 0x03} - if err := peerA.SendFile("peer-b", "payload.bin", fileBody); err != nil { - t.Fatalf("SendFile() error = %v", err) - } - - if err := <-receiveErr; err != nil { - t.Fatalf("peerB.Receive() error = %v", err) - } - - gotFirst := <-received - wantFirst := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello over kcp"), - } - if !reflect.DeepEqual(gotFirst, wantFirst) { - t.Fatalf("first message mismatch: got %+v want %+v", gotFirst, wantFirst) - } - - gotSecond := <-received - wantSecond := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 2, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: fileBody, - } - if !reflect.DeepEqual(gotSecond, wantSecond) { - t.Fatalf("second message mismatch: got %+v want %+v", gotSecond, wantSecond) - } -} - -func TestKCPClientReceivesServerErrorForUnknownTarget(t *testing.T) { - hub := server.NewKCPHub() - serverAddr, cleanup := startRealKCPHubServer(t, hub) - defer cleanup() - - client, err := DialKCP(serverAddr, "peer-a") - if err != nil { - t.Fatalf("DialKCP() error = %v", err) - } - defer func() { _ = client.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") - - if err := client.SendText("missing-peer", "hello"); err != nil { - t.Fatalf("SendText() error = %v", err) - } - - got, err := client.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if got.Type != protocol.MessageTypeError { - t.Fatalf("got type %s, want %s", got.Type, protocol.MessageTypeError) - } - if string(got.Body) != "unknown target: missing-peer" { - t.Fatalf("error body = %q, want unknown target message", got.Body) - } -} - -func TestKCPClientsExchangeMessagesWithLatencyLogs(t *testing.T) { - hub := server.NewKCPHub() - serverAddr, cleanup := startRealKCPHubServer(t, hub) - defer cleanup() - - peerALogger := &recordingLogger{} - peerA, err := DialKCP(serverAddr, "peer-a", WithLogger(peerALogger)) - if err != nil { - t.Fatalf("DialKCP(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerBLogger := &recordingLogger{} - peerB, err := DialKCP(serverAddr, "peer-b", WithLogger(peerBLogger)) - if err != nil { - t.Fatalf("DialKCP(peer-b) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - inboxDir := t.TempDir() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered") - - if err := peerA.SendText("peer-b", "hello"); err != nil { - t.Fatalf("SendText() error = %v", err) - } - textMsg, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive(text) error = %v", err) - } - if _, err := peerB.PersistMessage(textMsg, inboxDir); err != nil { - t.Fatalf("peerB.PersistMessage(text) error = %v", err) - } - - filePath := filepath.Join(t.TempDir(), "payload.bin") - if err := os.WriteFile(filePath, []byte{0x01, 0x02, 0x03}, 0o644); err != nil { - t.Fatalf("os.WriteFile() error = %v", err) - } - if err := peerA.SendFilePath("peer-b", filePath); err != nil { - t.Fatalf("SendFilePath() error = %v", err) - } - fileMsg, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive(file) error = %v", err) - } - if _, err := peerB.PersistMessage(fileMsg, inboxDir); err != nil { - t.Fatalf("peerB.PersistMessage(file) error = %v", err) - } - - waitFor(t, func() bool { return len(peerALogger.Events()) == 6 }, "peer-a latency events") - waitFor(t, func() bool { return len(peerBLogger.Events()) == 6 }, "peer-b latency events") - - assertEventSequencesByMessage(t, peerALogger.Events(), map[uint64][]string{ - 1: {latencylog.EventAAppPrepBegin, latencylog.EventSendHandoffBegin, latencylog.EventSendHandoffEnd}, - 2: {latencylog.EventAAppPrepBegin, latencylog.EventSendHandoffBegin, latencylog.EventSendHandoffEnd}, - }) - assertEventSequencesByMessage(t, peerBLogger.Events(), map[uint64][]string{ - 1: {latencylog.EventBAppRecv, latencylog.EventBPersistBegin, latencylog.EventBPersistEnd}, - 2: {latencylog.EventBAppRecv, latencylog.EventBPersistBegin, latencylog.EventBPersistEnd}, - }) -} - -func TestKCPClientsExchangeMessagesAcrossRelayedServers(t *testing.T) { - fixture := startRelayedKCPHubs(t) - defer fixture.cleanup() - - peerA, err := DialKCP(fixture.serverCAddr, "peer-a") - if err != nil { - t.Fatalf("DialKCP(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerB, err := DialKCP(fixture.serverDAddr, "peer-b") - if err != nil { - t.Fatalf("DialKCP(peer-b) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - waitFor(t, func() bool { return fixture.hubC.HasPeer("peer-a") && fixture.hubD.HasPeer("peer-b") }, "both relayed peers to be registered") - - if err := peerA.SendText("peer-b", "hello via relay"); err != nil { - t.Fatalf("peerA.SendText() error = %v", err) - } - gotAtB, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive() error = %v", err) - } - wantAtB := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello via relay"), - } - if !reflect.DeepEqual(gotAtB, wantAtB) { - t.Fatalf("peerB received %+v, want %+v", gotAtB, wantAtB) - } - - if err := peerB.SendText("peer-a", "hello back"); err != nil { - t.Fatalf("peerB.SendText() error = %v", err) - } - gotAtA, err := peerA.Receive() - if err != nil { - t.Fatalf("peerA.Receive() error = %v", err) - } - wantAtA := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-b", - To: "peer-a", - Body: []byte("hello back"), - } - if !reflect.DeepEqual(gotAtA, wantAtA) { - t.Fatalf("peerA received %+v, want %+v", gotAtA, wantAtA) - } - - if got := fixture.relayC.WriteCount(); got != 1 { - t.Fatalf("relayC write count = %d, want 1", got) - } - if got := fixture.relayD.WriteCount(); got != 1 { - t.Fatalf("relayD write count = %d, want 1", got) - } -} - -func TestKCPClientsExchangeMessagesViaUDPRelayToSingleHub(t *testing.T) { - hub := server.NewKCPHub() - serverAddr, cleanupHub := startRealKCPHubServer(t, hub) - defer cleanupHub() - - remoteAddr, err := net.ResolveUDPAddr("udp", serverAddr) - if err != nil { - t.Fatalf("ResolveUDPAddr(server) error = %v", err) - } - - baseRelayConn, err := net.ListenPacket("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ListenPacket(relay) error = %v", err) - } - relayConn := &countingPacketConn{PacketConn: baseRelayConn} - - relay, err := server.NewUDPRelay(relayConn, remoteAddr) - if err != nil { - _ = relayConn.Close() - t.Fatalf("NewUDPRelay() error = %v", err) - } - - var relayWG sync.WaitGroup - relayWG.Add(1) - go func() { - defer relayWG.Done() - if serveErr := relay.Serve(); serveErr != nil && !isExpectedKCPRelayServeExit(serveErr) { - t.Errorf("relay.Serve() error = %v", serveErr) - } - }() - defer func() { - _ = relay.Close() - relayWG.Wait() - }() - - peerA, err := DialKCP(serverAddr, "peer-a", WithKCPDialAddress(relayConn.LocalAddr().String())) - if err != nil { - t.Fatalf("DialKCP(peer-a via relay) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerB, err := DialKCP(serverAddr, "peer-b") - if err != nil { - t.Fatalf("DialKCP(peer-b direct) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered on the single hub") - - if err := peerB.SendText("peer-a", "hello via udp relay"); err != nil { - t.Fatalf("peerB.SendText() error = %v", err) - } - gotAtA, err := peerA.Receive() - if err != nil { - t.Fatalf("peerA.Receive() error = %v", err) - } - wantAtA := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-b", - To: "peer-a", - Body: []byte("hello via udp relay"), - } - if !reflect.DeepEqual(gotAtA, wantAtA) { - t.Fatalf("peerA received %+v, want %+v", gotAtA, wantAtA) - } - - if err := peerA.SendText("peer-b", "hello back through relay"); err != nil { - t.Fatalf("peerA.SendText() error = %v", err) - } - gotAtB, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive() error = %v", err) - } - wantAtB := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello back through relay"), - } - if !reflect.DeepEqual(gotAtB, wantAtB) { - t.Fatalf("peerB received %+v, want %+v", gotAtB, wantAtB) - } - - if got := relayConn.WriteCount(); got == 0 { - t.Fatal("relay should have forwarded packets for peer-a session") - } -} - -func TestKCPHubPrefersLocalPeerBeforeRelay(t *testing.T) { - fixture := startRelayedKCPHubs(t) - defer fixture.cleanup() - - peerA, err := DialKCP(fixture.serverCAddr, "peer-a") - if err != nil { - t.Fatalf("DialKCP(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - peerB, err := DialKCP(fixture.serverCAddr, "peer-b") - if err != nil { - t.Fatalf("DialKCP(peer-b) error = %v", err) - } - defer func() { _ = peerB.Close() }() - - waitFor(t, func() bool { return fixture.hubC.HasPeer("peer-a") && fixture.hubC.HasPeer("peer-b") }, "local peers on hubC to be registered") - - if err := peerA.SendText("peer-b", "local delivery"); err != nil { - t.Fatalf("peerA.SendText() error = %v", err) - } - got, err := peerB.Receive() - if err != nil { - t.Fatalf("peerB.Receive() error = %v", err) - } - want := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("local delivery"), - } - if !reflect.DeepEqual(got, want) { - t.Fatalf("peerB received %+v, want %+v", got, want) - } - - if got := fixture.relayC.WriteCount(); got != 0 { - t.Fatalf("relayC write count = %d, want 0 for local delivery", got) - } - if got := fixture.relayD.WriteCount(); got != 0 { - t.Fatalf("relayD write count = %d, want 0 for local delivery", got) - } -} - -func TestKCPRelayedUnknownTargetReturnsErrorToOriginalSender(t *testing.T) { - fixture := startRelayedKCPHubs(t) - defer fixture.cleanup() - - peerA, err := DialKCP(fixture.serverCAddr, "peer-a") - if err != nil { - t.Fatalf("DialKCP(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - waitFor(t, func() bool { return fixture.hubC.HasPeer("peer-a") }, "peer-a to be registered on hubC") - - if err := peerA.SendText("remote-missing", "hello"); err != nil { - t.Fatalf("peerA.SendText() error = %v", err) - } - - got, err := peerA.Receive() - if err != nil { - t.Fatalf("peerA.Receive() error = %v", err) - } - if got.Type != protocol.MessageTypeError { - t.Fatalf("got type %s, want %s", got.Type, protocol.MessageTypeError) - } - if got.From != protocol.ServerPeerID { - t.Fatalf("error from = %s, want %s", got.From, protocol.ServerPeerID) - } - if got.To != "peer-a" { - t.Fatalf("error to = %s, want peer-a", got.To) - } - if string(got.Body) != "unknown target: remote-missing" { - t.Fatalf("error body = %q, want unknown target from relayed hub", got.Body) - } - - if got := fixture.relayC.WriteCount(); got != 1 { - t.Fatalf("relayC write count = %d, want 1 for outbound relay", got) - } - if got := fixture.relayD.WriteCount(); got != 1 { - t.Fatalf("relayD write count = %d, want 1 for return error relay", got) - } -} - -func TestKCPHubRejectsOversizeRelayedMessage(t *testing.T) { - fixture := startRelayedKCPHubs(t) - defer fixture.cleanup() - - peerA, err := DialKCP(fixture.serverCAddr, "peer-a") - if err != nil { - t.Fatalf("DialKCP(peer-a) error = %v", err) - } - defer func() { _ = peerA.Close() }() - - waitFor(t, func() bool { return fixture.hubC.HasPeer("peer-a") }, "peer-a to be registered on hubC") - - body := bytes.Repeat([]byte("a"), 70*1024) - if err := peerA.SendFile("remote-peer", "payload.bin", body); err != nil { - t.Fatalf("peerA.SendFile() error = %v", err) - } - - got, err := peerA.Receive() - if err != nil { - t.Fatalf("peerA.Receive() error = %v", err) - } - if got.Type != protocol.MessageTypeError { - t.Fatalf("got type %s, want %s", got.Type, protocol.MessageTypeError) - } - if string(got.Body) != "message too large for relay udp" { - t.Fatalf("error body = %q, want oversize relay error", got.Body) - } - if got := fixture.relayC.WriteCount(); got != 0 { - t.Fatalf("relayC write count = %d, want 0 when relay rejects oversize payload", got) - } -} - -func startRealKCPHubServer(t *testing.T, hub *server.KCPHub) (string, func()) { - t.Helper() - - listener, packetConn, err := transport.ListenKCPSessions("127.0.0.1:0", "", nil, latencylog.NodeRoleServer, "hub") - if err != nil { - t.Fatalf("ListenKCPSessions() error = %v", err) - } - - var ( - wg sync.WaitGroup - stop = make(chan struct{}) - ) - - wg.Add(1) - go func() { - defer wg.Done() - for { - session, acceptErr := listener.AcceptKCP() - if acceptErr != nil { - select { - case <-stop: - return - default: - } - if strings.Contains(acceptErr.Error(), "closed") { - return - } - t.Errorf("AcceptKCP() error = %v", acceptErr) - return - } - - wg.Add(1) - go func(sess *kcp.UDPSession) { - defer wg.Done() - if serveErr := hub.ServeSession(sess); serveErr != nil && !isExpectedKCPHubServeExit(serveErr) { - t.Logf("hub.ServeSession() ended with %v", serveErr) - } - }(session) - } - }() - - cleanup := func() { - close(stop) - _ = listener.Close() - _ = packetConn.Close() - wg.Wait() - } - - return listener.Addr().String(), cleanup -} - -type relayedKCPHubFixture struct { - hubC *server.KCPHub - hubD *server.KCPHub - serverCAddr string - serverDAddr string - relayC *countingPacketConn - relayD *countingPacketConn - cleanup func() -} - -func startRelayedKCPHubs(t *testing.T) relayedKCPHubFixture { - t.Helper() - - hubC := server.NewKCPHub() - serverCAddr, cleanupC := startRealKCPHubServer(t, hubC) - - hubD := server.NewKCPHub() - serverDAddr, cleanupD := startRealKCPHubServer(t, hubD) - - baseRelayC, err := net.ListenPacket("udp", "127.0.0.1:0") - if err != nil { - cleanupD() - cleanupC() - t.Fatalf("ListenPacket(relayC) error = %v", err) - } - relayC := &countingPacketConn{PacketConn: baseRelayC} - - baseRelayD, err := net.ListenPacket("udp", "127.0.0.1:0") - if err != nil { - _ = relayC.Close() - cleanupD() - cleanupC() - t.Fatalf("ListenPacket(relayD) error = %v", err) - } - relayD := &countingPacketConn{PacketConn: baseRelayD} - - hubC.SetRelaySocket(relayC, relayD.LocalAddr(), false) - hubD.SetRelaySocket(relayD, relayC.LocalAddr(), false) - - stopRelayC := startRelayLoop(t, hubC, relayC) - stopRelayD := startRelayLoop(t, hubD, relayD) - - cleanup := func() { - stopRelayC() - stopRelayD() - cleanupD() - cleanupC() - } - - return relayedKCPHubFixture{ - hubC: hubC, - hubD: hubD, - serverCAddr: serverCAddr, - serverDAddr: serverDAddr, - relayC: relayC, - relayD: relayD, - cleanup: cleanup, - } -} - -func startRelayLoop(t *testing.T, hub *server.KCPHub, conn net.PacketConn) func() { - t.Helper() - - var wg sync.WaitGroup - wg.Add(1) - go func() { - defer wg.Done() - if err := hub.ServeRelay(); err != nil && !isExpectedKCPRelayServeExit(err) { - t.Errorf("hub.ServeRelay() error = %v", err) - } - }() - - return func() { - _ = conn.Close() - wg.Wait() - } -} - -type countingPacketConn struct { - net.PacketConn - writeCount int32 -} - -func (c *countingPacketConn) WriteTo(p []byte, addr net.Addr) (int, error) { - atomic.AddInt32(&c.writeCount, 1) - return c.PacketConn.WriteTo(p, addr) -} - -func (c *countingPacketConn) WriteCount() int { - return int(atomic.LoadInt32(&c.writeCount)) -} - -func isExpectedKCPHubServeExit(err error) bool { - if err == nil { - return true - } - - message := err.Error() - return strings.Contains(message, "closed") || strings.Contains(message, "broken pipe") || strings.Contains(message, "io: read/write on closed pipe") -} - -func isExpectedKCPRelayServeExit(err error) bool { - if err == nil { - return true - } - - message := err.Error() - return strings.Contains(message, "closed") || strings.Contains(message, "use of closed network connection") -} diff --git a/cmd/internal/peer/persist.go b/cmd/internal/peer/persist.go deleted file mode 100644 index cdd982c..0000000 --- a/cmd/internal/peer/persist.go +++ /dev/null @@ -1,99 +0,0 @@ -package peer - -import ( - "encoding/json" - "fmt" - "os" - "path/filepath" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -const textInboxFileName = "messages.log" - -type textInboxRecord struct { - MessageType protocol.MessageType `json:"message_type"` - MessageID uint64 `json:"message_id"` - From string `json:"from"` - To string `json:"to"` - Body string `json:"body"` -} - -// PersistMessage 将收到的业务消息写入本地磁盘，并记录处理完成节点。 -func (c *Client) PersistMessage(msg protocol.Message, inboxDir string) (string, error) { - if !latencylog.IsBusinessMessage(msg) { - return "", fmt.Errorf("peer: cannot persist message type %s", msg.Type) - } - if inboxDir == "" { - return "", fmt.Errorf("peer: inbox directory is required") - } - - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBPersistBegin, msg) - - if err := os.MkdirAll(inboxDir, 0o755); err != nil { - return "", fmt.Errorf("peer: create inbox dir %s: %w", inboxDir, err) - } - - path, err := persistMessageToDisk(msg, inboxDir) - if err != nil { - return "", err - } - - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBPersistEnd, msg) - - return path, nil -} - -// persistMessageToDisk 根据消息类型将消息内容写入磁盘，文本消息追加到文本日志文件，文件消息写成独立文件。 -func persistMessageToDisk(msg protocol.Message, inboxDir string) (string, error) { - switch msg.Type { - case protocol.MessageTypeText: - return persistTextMessage(msg, inboxDir) - case protocol.MessageTypeFile: - return persistFileMessage(msg, inboxDir) - default: - return "", fmt.Errorf("peer: cannot persist unsupported message type %s", msg.Type) - } -} - -// registerPeer 验证 peer ID 的合法性和唯一性，并将其与连接关联起来。 -func persistTextMessage(msg protocol.Message, inboxDir string) (string, error) { - record := textInboxRecord{ - MessageType: msg.Type, - MessageID: msg.ID, - From: msg.From, - To: msg.To, - Body: string(msg.Body), - } - - line, err := json.Marshal(record) - if err != nil { - return "", fmt.Errorf("peer: encode text inbox record: %w", err) - } - - path := filepath.Join(inboxDir, textInboxFileName) - file, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644) - if err != nil { - return "", fmt.Errorf("peer: open text inbox %s: %w", path, err) - } - defer file.Close() - - if _, err := file.Write(append(line, '\n')); err != nil { - return "", fmt.Errorf("peer: append text inbox %s: %w", path, err) - } - - return path, nil -} - -// persistFileMessage 将文件消息的内容写成独立文件，文件名包含发送方、消息 ID 和原始文件名，保证唯一性和可读性。 -func persistFileMessage(msg protocol.Message, inboxDir string) (string, error) { - fileName := filepath.Base(msg.FileName) - path := filepath.Join(inboxDir, fmt.Sprintf("%s-%d-%s", msg.From, msg.ID, fileName)) - - if err := os.WriteFile(path, msg.Body, 0o644); err != nil { - return "", fmt.Errorf("peer: write received file %s: %w", path, err) - } - - return path, nil -} diff --git a/cmd/internal/peer/udp_client.go b/cmd/internal/peer/udp_client.go deleted file mode 100644 index 70840f3..0000000 --- a/cmd/internal/peer/udp_client.go +++ /dev/null @@ -1,204 +0,0 @@ -package peer - -import ( - "fmt" - "net" - "os" - "path/filepath" - "sync/atomic" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -// UDPClient 表示一个通过 UDP 连接到 server 的 peer。 -type UDPClient struct { - id string - conn *transport.UDPConn - logger latencylog.Logger - - nextID uint64 -} - -// DialUDP 通过 UDP 连接到 server，并发送 register 消息完成身份注册。 -func DialUDP(serverAddr, peerID string, opts ...Option) (*UDPClient, error) { - options := clientOptions{ - logger: latencylog.NoopLogger{}, - udpLinuxTimestamping: true, - } - for _, opt := range opts { - opt(&options) - } - if options.logger == nil { - options.logger = latencylog.NoopLogger{} - } - - udpServerAddr, err := net.ResolveUDPAddr("udp", serverAddr) - if err != nil { - return nil, fmt.Errorf("peer: resolve udp server addr %s: %w", serverAddr, err) - } - - var localAddr *net.UDPAddr - if options.bindIP != "" { - ip := net.ParseIP(options.bindIP) - if ip == nil { - return nil, fmt.Errorf("peer: invalid bind ip %q", options.bindIP) - } - localAddr = &net.UDPAddr{IP: ip} - } - - rawConn, err := net.DialUDP("udp", localAddr, udpServerAddr) - if err != nil { - return nil, fmt.Errorf("peer: dial udp server %s: %w", serverAddr, err) - } - - conn, err := transport.NewUDPConn( - rawConn, - nil, // peer 侧已连接模式，不需要指定 peerAddr - transport.WithUDPLogger(options.logger, latencylog.NodeRolePeer, peerID), - transport.WithUDPLinuxTimestamping(options.udpLinuxTimestamping), - transport.WithUDPTXTimestampDebugLogger(options.txTimestampDebugLogger), - ) - if err != nil { - _ = rawConn.Close() - return nil, fmt.Errorf("peer: create udp transport conn: %w", err) - } - - client := &UDPClient{ - id: peerID, - conn: conn, - logger: options.logger, - } - - // 发送 register 消息完成身份注册 - if err := conn.Send(protocol.Message{ - Type: protocol.MessageTypeRegister, - From: peerID, - To: protocol.ServerPeerID, - }); err != nil { - _ = conn.Close() - return nil, fmt.Errorf("peer: udp register with server: %w", err) - } - - return client, nil -} - -// ID 返回当前 client 的 peer 标识。 -func (c *UDPClient) ID() string { - return c.id -} - -// SendText 向目标 peer 发送一条文本消息。 -func (c *UDPClient) SendText(to, body string) error { - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: c.nextMessageID(), - From: c.id, - To: to, - } - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventAAppPrepBegin, msg) - - msg.Body = []byte(body) - - return c.conn.Send(msg) -} - -// SendFile 向目标 peer 发送一条文件消息。 -func (c *UDPClient) SendFile(to, fileName string, body []byte) error { - msg := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: c.nextMessageID(), - From: c.id, - To: to, - FileName: fileName, - } - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventAAppPrepBegin, msg) - - bodyCopy := make([]byte, len(body)) - copy(bodyCopy, body) - - msg.Body = bodyCopy - - return c.conn.Send(msg) -} - -// SendFilePath 从本地文件读取内容并发送给目标 peer。 -func (c *UDPClient) SendFilePath(to, path string) error { - msg := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: c.nextMessageID(), - From: c.id, - To: to, - FileName: filepath.Base(path), - } - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventAAppPrepBegin, msg) - - body, err := os.ReadFile(path) - if err != nil { - return fmt.Errorf("peer: read file %s: %w", path, err) - } - - msg.Body = body - - return c.conn.Send(msg) -} - -// Receive 读取一条来自 server 的消息。 -func (c *UDPClient) Receive() (protocol.Message, error) { - msg, _, err := c.conn.Receive() - if err != nil { - return protocol.Message{}, fmt.Errorf("peer: udp receive from server: %w", err) - } - - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBAppRecv, msg) - - return msg, nil -} - -// ReceiveLoop 持续接收 server 消息并交给 handler 处理。 -func (c *UDPClient) ReceiveLoop(handler func(protocol.Message) error) error { - return c.conn.ReceiveLoop(func(msg protocol.Message, _ *net.UDPAddr) error { - switch msg.Type { - case protocol.MessageTypeText, protocol.MessageTypeFile, protocol.MessageTypeError: - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBAppRecv, msg) - return handler(msg) - default: - return fmt.Errorf("peer: unexpected message type from server: %s", msg.Type) - } - }) -} - -// PersistMessage 将收到的业务消息写入本地磁盘。 -func (c *UDPClient) PersistMessage(msg protocol.Message, inboxDir string) (string, error) { - if !latencylog.IsBusinessMessage(msg) { - return "", fmt.Errorf("peer: cannot persist message type %s", msg.Type) - } - if inboxDir == "" { - return "", fmt.Errorf("peer: inbox directory is required") - } - - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBPersistBegin, msg) - - if err := os.MkdirAll(inboxDir, 0o755); err != nil { - return "", fmt.Errorf("peer: create inbox dir %s: %w", inboxDir, err) - } - - path, err := persistMessageToDisk(msg, inboxDir) - if err != nil { - return "", err - } - - latencylog.LogMessageEvent(c.logger, latencylog.NodeRolePeer, c.id, latencylog.EventBPersistEnd, msg) - - return path, nil -} - -// Close 关闭与 server 的 UDP 连接。 -func (c *UDPClient) Close() error { - return c.conn.Close() -} - -func (c *UDPClient) nextMessageID() uint64 { - return atomic.AddUint64(&c.nextID, 1) -} diff --git a/cmd/internal/peer/udp_client_test.go b/cmd/internal/peer/udp_client_test.go deleted file mode 100644 index d23750c..0000000 --- a/cmd/internal/peer/udp_client_test.go +++ /dev/null @@ -1,337 +0,0 @@ -package peer - -import ( - "net" - "os" - "path/filepath" - "sync" - "testing" - "time" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/server" - "omnisocketgo/cmd/internal/transport" -) - -type recordingLatencyLogger struct { - mu sync.Mutex - events []latencylog.Event -} - -func (l *recordingLatencyLogger) LogEvent(event latencylog.Event) error { - l.mu.Lock() - defer l.mu.Unlock() - - l.events = append(l.events, event) - return nil -} - -func (l *recordingLatencyLogger) Events() []latencylog.Event { - l.mu.Lock() - defer l.mu.Unlock() - - return append([]latencylog.Event(nil), l.events...) -} - -type recordingUDPTXTimestampDebugLogger struct { - mu sync.Mutex - records []transport.TXTimestampDebugRecord -} - -func (l *recordingUDPTXTimestampDebugLogger) LogTXTimestampDebugRecord(record transport.TXTimestampDebugRecord) error { - l.mu.Lock() - defer l.mu.Unlock() - - l.records = append(l.records, record) - return nil -} - -func (l *recordingUDPTXTimestampDebugLogger) Records() []transport.TXTimestampDebugRecord { - l.mu.Lock() - defer l.mu.Unlock() - - return append([]transport.TXTimestampDebugRecord(nil), l.records...) -} - -func TestUDPDialAndSendText(t *testing.T) { - hubAddr := startUDPTestHub(t) - - clientA, err := DialUDP(hubAddr.String(), "peer-a") - if err != nil { - t.Fatalf("DialUDP(peer-a) error = %v", err) - } - defer clientA.Close() - - clientB, err := DialUDP(hubAddr.String(), "peer-b") - if err != nil { - t.Fatalf("DialUDP(peer-b) error = %v", err) - } - defer clientB.Close() - - time.Sleep(50 * time.Millisecond) - - if err := clientA.SendText("peer-b", "hello from udp"); err != nil { - t.Fatalf("SendText() error = %v", err) - } - - msg := receiveUDPClientMessage(t, clientB) - if msg.Type != protocol.MessageTypeText { - t.Fatalf("message type = %s, want text", msg.Type) - } - if string(msg.Body) != "hello from udp" { - t.Fatalf("message body = %q, want %q", string(msg.Body), "hello from udp") - } -} - -func TestUDPClientID(t *testing.T) { - hubAddr := startUDPTestHub(t) - - client, err := DialUDP(hubAddr.String(), "my-peer-id") - if err != nil { - t.Fatalf("DialUDP() error = %v", err) - } - defer client.Close() - - if got := client.ID(); got != "my-peer-id" { - t.Fatalf("ID() = %q, want %q", got, "my-peer-id") - } -} - -func TestUDPClientPersistMessage(t *testing.T) { - hubAddr := startUDPTestHub(t) - - client, err := DialUDP(hubAddr.String(), "peer-persist") - if err != nil { - t.Fatalf("DialUDP() error = %v", err) - } - defer client.Close() - - inboxDir := t.TempDir() - - textMsg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "sender", - To: "peer-persist", - Body: []byte("persisted text"), - } - - path, err := client.PersistMessage(textMsg, inboxDir) - if err != nil { - t.Fatalf("PersistMessage(text) error = %v", err) - } - if !filepath.IsAbs(path) && path == "" { - t.Fatalf("PersistMessage(text) returned empty path") - } - - fileMsg := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 2, - From: "sender", - To: "peer-persist", - FileName: "test.bin", - Body: []byte{0x01, 0x02, 0x03}, - } - - filePath, err := client.PersistMessage(fileMsg, inboxDir) - if err != nil { - t.Fatalf("PersistMessage(file) error = %v", err) - } - - content, err := os.ReadFile(filePath) - if err != nil { - t.Fatalf("ReadFile(%s) error = %v", filePath, err) - } - if len(content) != 3 || content[0] != 0x01 { - t.Fatalf("file content mismatch: got %v", content) - } -} - -func TestUDPClientSendFile(t *testing.T) { - hubAddr := startUDPTestHub(t) - - clientA, err := DialUDP(hubAddr.String(), "peer-a") - if err != nil { - t.Fatalf("DialUDP(peer-a) error = %v", err) - } - defer clientA.Close() - - clientB, err := DialUDP(hubAddr.String(), "peer-b") - if err != nil { - t.Fatalf("DialUDP(peer-b) error = %v", err) - } - defer clientB.Close() - - time.Sleep(50 * time.Millisecond) - - fileBody := []byte{0xDE, 0xAD, 0xBE, 0xEF} - if err := clientA.SendFile("peer-b", "test.bin", fileBody); err != nil { - t.Fatalf("SendFile() error = %v", err) - } - - msg := receiveUDPClientMessage(t, clientB) - if msg.Type != protocol.MessageTypeFile { - t.Fatalf("message type = %s, want file", msg.Type) - } - if msg.FileName != "test.bin" { - t.Fatalf("file name = %q, want %q", msg.FileName, "test.bin") - } - if len(msg.Body) != 4 { - t.Fatalf("body length = %d, want 4", len(msg.Body)) - } -} - -func TestUDPClientBidirectionalLogsAndServerDiagnostics(t *testing.T) { - serverLogger := &recordingLatencyLogger{} - serverDebugLogger := &recordingUDPTXTimestampDebugLogger{} - hubAddr := startUDPTestHubWithOptions( - t, - server.WithUDPLogger(serverLogger), - server.WithUDPTXTimestampDebugLogger(serverDebugLogger), - ) - - clientALogger := &recordingLatencyLogger{} - clientADebugLogger := &recordingUDPTXTimestampDebugLogger{} - clientA, err := DialUDP( - hubAddr.String(), - "peer-a", - WithLogger(clientALogger), - WithTXTimestampDebugLogger(clientADebugLogger), - ) - if err != nil { - t.Fatalf("DialUDP(peer-a) error = %v", err) - } - defer clientA.Close() - - clientBLogger := &recordingLatencyLogger{} - clientBDebugLogger := &recordingUDPTXTimestampDebugLogger{} - clientB, err := DialUDP( - hubAddr.String(), - "peer-b", - WithLogger(clientBLogger), - WithTXTimestampDebugLogger(clientBDebugLogger), - ) - if err != nil { - t.Fatalf("DialUDP(peer-b) error = %v", err) - } - defer clientB.Close() - - time.Sleep(50 * time.Millisecond) - - if err := clientA.SendText("peer-b", "hello from peer-a"); err != nil { - t.Fatalf("clientA.SendText() error = %v", err) - } - msgFromA := receiveUDPClientMessage(t, clientB) - if string(msgFromA.Body) != "hello from peer-a" { - t.Fatalf("message body = %q, want %q", string(msgFromA.Body), "hello from peer-a") - } - - if err := clientB.SendText("peer-a", "hello from peer-b"); err != nil { - t.Fatalf("clientB.SendText() error = %v", err) - } - msgFromB := receiveUDPClientMessage(t, clientA) - if string(msgFromB.Body) != "hello from peer-b" { - t.Fatalf("message body = %q, want %q", string(msgFromB.Body), "hello from peer-b") - } - - assertPeerEvent(t, clientBLogger.Events(), latencylog.EventBRXSoftware, "peer-a", "peer-b", msgFromA.ID) - assertPeerEvent(t, clientBLogger.Events(), latencylog.EventBAppRecv, "peer-a", "peer-b", msgFromA.ID) - assertPeerEvent(t, clientALogger.Events(), latencylog.EventBRXSoftware, "peer-b", "peer-a", msgFromB.ID) - assertPeerEvent(t, clientALogger.Events(), latencylog.EventBAppRecv, "peer-b", "peer-a", msgFromB.ID) - - if len(clientADebugLogger.Records()) == 0 { - t.Fatal("clientA debug records = 0, want at least 1") - } - if len(clientBDebugLogger.Records()) == 0 { - t.Fatal("clientB debug records = 0, want at least 1") - } - if len(serverLogger.Events()) == 0 { - t.Fatal("server latency events = 0, want at least 1") - } - if len(serverDebugLogger.Records()) == 0 { - t.Fatal("server debug records = 0, want at least 1") - } -} - -func startUDPTestHub(t *testing.T) *net.UDPAddr { - return startUDPTestHubWithOptions(t) -} - -func startUDPTestHubWithOptions(t *testing.T, opts ...server.UDPOption) *net.UDPAddr { - t.Helper() - - addr, err := net.ResolveUDPAddr("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ResolveUDPAddr() error = %v", err) - } - - conn, err := net.ListenUDP("udp", addr) - if err != nil { - t.Fatalf("ListenUDP() error = %v", err) - } - - hub, err := server.NewUDPHub(conn, opts...) - if err != nil { - _ = conn.Close() - t.Fatalf("NewUDPHub() error = %v", err) - } - - go func() { - _ = hub.Serve() - }() - - t.Cleanup(func() { - _ = hub.Close() - }) - - return conn.LocalAddr().(*net.UDPAddr) -} - -func receiveUDPClientMessage(t *testing.T, client *UDPClient) protocol.Message { - t.Helper() - - type result struct { - msg protocol.Message - err error - } - - ch := make(chan result, 1) - go func() { - msg, err := client.Receive() - ch <- result{msg: msg, err: err} - }() - - select { - case r := <-ch: - if r.err != nil { - t.Fatalf("Receive() error = %v", r.err) - } - return r.msg - case <-time.After(2 * time.Second): - t.Fatal("Receive() timed out after 2s") - return protocol.Message{} - } -} - -func assertPeerEvent(t *testing.T, events []latencylog.Event, wantEvent, wantFrom, wantTo string, wantMessageID uint64) { - t.Helper() - - for _, event := range events { - if event.Event != wantEvent { - continue - } - if event.From != wantFrom || event.To != wantTo || event.MessageID != wantMessageID { - continue - } - if event.TsUnixNano <= 0 { - t.Fatalf("event %s timestamp must be positive: %+v", wantEvent, event) - } - return - } - - t.Fatalf("missing event %s from %s to %s for message %d in %+v", wantEvent, wantFrom, wantTo, wantMessageID, events) -} - -// Ensure transport package is used (needed for WithTXTimestampDebugLogger option). -var _ transport.TXTimestampDebugLogger = nil diff --git a/cmd/internal/server/hub.go b/cmd/internal/server/hub.go deleted file mode 100644 index ed11bc3..0000000 --- a/cmd/internal/server/hub.go +++ /dev/null @@ -1,192 +0,0 @@ -package server - -import ( - "fmt" - "net" - "sync" - "time" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -const gracefulRejectCloseTimeout = 100 * time.Millisecond - -// Hub 管理已注册 peer 的连接，并负责在它们之间转发消息。 -type Hub struct { - mu sync.RWMutex - peers map[string]*transport.TCPConn - logger latencylog.Logger -} - -// Option 用于配置 Hub 的可选行为，例如时延日志。 -type Option func(*Hub) - -// WithLogger 为 hub 注入时延日志记录器。 -func WithLogger(logger latencylog.Logger) Option { - return func(hub *Hub) { - hub.logger = logger - } -} - -// NewHub 创建一个空的连接中心。 -func NewHub(opts ...Option) *Hub { - hub := &Hub{ - peers: make(map[string]*transport.TCPConn), - logger: latencylog.NoopLogger{}, - } - - for _, opt := range opts { - opt(hub) - } - - if hub.logger == nil { - hub.logger = latencylog.NoopLogger{} - } - - return hub -} - -// HasPeer 返回给定 ID 是否已经注册到 hub。 -func (h *Hub) HasPeer(peerID string) bool { - h.mu.RLock() - defer h.mu.RUnlock() - - _, ok := h.peers[peerID] - return ok -} - -// ServeConn 处理一条新接入的底层 TCP 连接。 -// 连接上的第一条消息必须是 register，之后才允许转发 text/file。 -func (h *Hub) ServeConn(rawConn net.Conn) error { - conn, err := transport.NewTCPConn(rawConn) - if err != nil { - _ = rawConn.Close() - return fmt.Errorf("server: create transport conn: %w", err) - } - - peerID, gracefulClose, err := h.registerConn(conn) - if err != nil { - h.closeConn(conn, gracefulClose) - return err - } - defer h.unregister(peerID, conn) - - if err := h.receivePeerLoop(peerID, conn); err != nil { - return err - } - - return nil -} - -// registerConn 从新连接上读取第一条消息，验证它是 register 消息，并把连接注册到 hub。 -func (h *Hub) registerConn(conn *transport.TCPConn) (string, bool, error) { - msg, err := conn.Receive() - if err != nil { - return "", false, fmt.Errorf("server: receive register: %w", err) - } - - if msg.Type != protocol.MessageTypeRegister { - if sendErr := sendServerError(conn, msg.From, "first message must be register"); sendErr != nil { - return "", false, fmt.Errorf("server: reject unregistered peer: %w", sendErr) - } - return "", true, fmt.Errorf("server: first message must be register, got %s", msg.Type) - } - - h.mu.Lock() - defer h.mu.Unlock() - - if _, exists := h.peers[msg.From]; exists { - if sendErr := sendServerError(conn, msg.From, fmt.Sprintf("duplicate peer id: %s", msg.From)); sendErr != nil { - return "", false, fmt.Errorf("server: duplicate peer id %s: %w", msg.From, sendErr) - } - return "", true, fmt.Errorf("server: duplicate peer id: %s", msg.From) - } - - h.peers[msg.From] = conn - return msg.From, false, nil -} - -// handlePeerMessage 验证消息类型并执行相应的转发或错误响应。 -func (h *Hub) handlePeerMessage(peerID string, conn *transport.TCPConn, msg protocol.Message) (bool, error) { - switch msg.Type { - case protocol.MessageTypeText, protocol.MessageTypeFile: //只允许已注册的 peer 发送文本或文件消息，其他类型都视为协议错误。 - msg.From = peerID - targetConn, ok := h.lookup(msg.To) - if !ok { - return false, sendServerError(conn, peerID, fmt.Sprintf("unknown target: %s", msg.To)) - } - if err := targetConn.Send(msg); err != nil { //转发消息，如果发送失败，说明目标连接可能已经不可用，此时从 hub 中注销该连接并关闭它，并向发送方返回错误响应。 - h.unregister(msg.To, targetConn) - _ = targetConn.Close() - return false, sendServerError(conn, peerID, fmt.Sprintf("failed to forward to %s", msg.To)) - } - return false, nil - case protocol.MessageTypeRegister, protocol.MessageTypeError: //已注册的 peer 不允许再发送 register 或 error 消息，这些都视为协议错误。 - if err := sendServerError(conn, peerID, "registered peers can only send text or file messages"); err != nil { - return false, fmt.Errorf("server: send protocol error: %w", err) - } - return true, fmt.Errorf("server: unexpected message type from peer %s: %s", peerID, msg.Type) - default: // 其他任何消息类型都视为协议错误。 - if err := sendServerError(conn, peerID, fmt.Sprintf("unsupported message type: %s", msg.Type)); err != nil { - return false, fmt.Errorf("server: send unsupported type error: %w", err) - } - return true, fmt.Errorf("server: unsupported message type: %s", msg.Type) - } -} - -func (h *Hub) receivePeerLoop(peerID string, conn *transport.TCPConn) error { - for { - msg, err := conn.Receive() - if err != nil { - _ = conn.Close() - return fmt.Errorf("transport: receive loop read: %w", err) - } - - gracefulClose, err := h.handlePeerMessage(peerID, conn, msg) - if err != nil { - h.closeConn(conn, gracefulClose) - return fmt.Errorf("transport: receive loop handler: %w", err) - } - } -} - -// lookup 在 hub 中查找目标 peer 的连接。 -func (h *Hub) lookup(peerID string) (*transport.TCPConn, bool) { - h.mu.RLock() - defer h.mu.RUnlock() - - conn, ok := h.peers[peerID] - return conn, ok -} - -// unregister 从 hub 中移除指定 peer 的连接，通常在连接关闭或发生错误时调用。 -func (h *Hub) unregister(peerID string, conn *transport.TCPConn) { - h.mu.Lock() - defer h.mu.Unlock() - - current, ok := h.peers[peerID] - if ok && current == conn { - delete(h.peers, peerID) - } -} - -func (h *Hub) closeConn(conn *transport.TCPConn, graceful bool) { - if graceful { - _ = conn.CloseGracefully(gracefulRejectCloseTimeout) - return - } - - _ = conn.Close() -} - -// sendServerError 是一个辅助函数，用于向指定 peer 发送错误消息。 -func sendServerError(conn *transport.TCPConn, to, message string) error { - return conn.Send(protocol.Message{ - Type: protocol.MessageTypeError, - From: protocol.ServerPeerID, - To: to, - Body: []byte(message), - }) -} diff --git a/cmd/internal/server/hub_test.go b/cmd/internal/server/hub_test.go deleted file mode 100644 index 2cee305..0000000 --- a/cmd/internal/server/hub_test.go +++ /dev/null @@ -1,398 +0,0 @@ -package server - -import ( - "net" - "reflect" - "strings" - "sync" - "testing" - "time" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -type recordingLogger struct { - mu sync.Mutex - events []latencylog.Event -} - -func (l *recordingLogger) LogEvent(event latencylog.Event) error { - l.mu.Lock() - defer l.mu.Unlock() - - l.events = append(l.events, event) - return nil -} - -func (l *recordingLogger) Events() []latencylog.Event { - l.mu.Lock() - defer l.mu.Unlock() - - return append([]latencylog.Event(nil), l.events...) -} - -func TestServeConnRegistersPeer(t *testing.T) { - hub := NewHub() - client, done := startHubConn(t, hub) - - if err := client.Send(protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-a", - To: protocol.ServerPeerID, - }); err != nil { - t.Fatalf("Send(register) error = %v", err) - } - - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") - - if err := client.Close(); err != nil { - t.Fatalf("client.Close() error = %v", err) - } - - err := <-done - if err == nil || !strings.Contains(err.Error(), "receive loop read") { - t.Fatalf("ServeConn() error = %v, want read-loop shutdown error", err) - } -} - -func TestServeConnRejectsDuplicatePeerID(t *testing.T) { - hub := NewHub() - - first, firstDone := startHubConn(t, hub) - registerPeer(t, first, "peer-a") - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") - - second, secondDone := startHubConn(t, hub) - registerPeer(t, second, "peer-a") - - got, err := second.Receive() - if err != nil { - t.Fatalf("second.Receive() error = %v", err) - } - if got.Type != protocol.MessageTypeError { - t.Fatalf("got message type %s, want %s", got.Type, protocol.MessageTypeError) - } - if string(got.Body) != "duplicate peer id: peer-a" { - t.Fatalf("error body = %q, want duplicate peer message", got.Body) - } - - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "original peer-a to remain registered") - - if err := first.Close(); err != nil { - t.Fatalf("first.Close() error = %v", err) - } - - if err := <-secondDone; err == nil || !strings.Contains(err.Error(), "duplicate peer id") { - t.Fatalf("second ServeConn() error = %v, want duplicate peer id error", err) - } - if err := <-firstDone; err == nil || !strings.Contains(err.Error(), "receive loop read") { - t.Fatalf("first ServeConn() error = %v, want read-loop shutdown error", err) - } -} - -func TestServeConnForwardsMessages(t *testing.T) { - tests := []struct { - name string - msg protocol.Message - }{ - { - name: "text", - msg: protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "spoofed", - To: "peer-b", - Body: []byte("hello"), - }, - }, - { - name: "file", - msg: protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 2, - From: "spoofed", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0x01, 0x02, 0x03}, - }, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - hub := NewHub() - - sender, senderDone := startHubConn(t, hub) - receiver, receiverDone := startHubConn(t, hub) - registerPeer(t, sender, "peer-a") - registerPeer(t, receiver, "peer-b") - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered") - - if err := sender.Send(tt.msg); err != nil { - t.Fatalf("sender.Send() error = %v", err) - } - - got, err := receiver.Receive() - if err != nil { - t.Fatalf("receiver.Receive() error = %v", err) - } - - want := tt.msg - want.From = "peer-a" - if !reflect.DeepEqual(got, want) { - t.Fatalf("forwarded message mismatch: got %+v want %+v", got, want) - } - - _ = sender.Close() - _ = receiver.Close() - <-senderDone - <-receiverDone - }) - } -} - -func TestServeConnReturnsErrorForUnknownTarget(t *testing.T) { - hub := NewHub() - - client, done := startHubConn(t, hub) - registerPeer(t, client, "peer-a") - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") - - if err := client.Send(protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "missing-peer", - Body: []byte("hello"), - }); err != nil { - t.Fatalf("Send(text) error = %v", err) - } - - got, err := client.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if got.Type != protocol.MessageTypeError { - t.Fatalf("got message type %s, want %s", got.Type, protocol.MessageTypeError) - } - if string(got.Body) != "unknown target: missing-peer" { - t.Fatalf("error body = %q, want unknown target message", got.Body) - } - if !hub.HasPeer("peer-a") { - t.Fatal("peer-a should remain registered after unknown target error") - } - - _ = client.Close() - <-done -} - -func TestServeConnRejectsRegisterAfterRegistration(t *testing.T) { - hub := NewHub() - - client, done := startHubConn(t, hub) - registerPeer(t, client, "peer-a") - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") - - if err := client.Send(protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-a", - To: protocol.ServerPeerID, - }); err != nil { - t.Fatalf("Send(register again) error = %v", err) - } - - got, err := client.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if got.Type != protocol.MessageTypeError { - t.Fatalf("got message type %s, want %s", got.Type, protocol.MessageTypeError) - } - if string(got.Body) != "registered peers can only send text or file messages" { - t.Fatalf("error body = %q, want registered-peer protocol error", got.Body) - } - - if err := <-done; err == nil || !strings.Contains(err.Error(), "unexpected message type from peer peer-a: register") { - t.Fatalf("ServeConn() error = %v, want unexpected register message error", err) - } -} - -func TestServeConnUnregistersPeerOnClose(t *testing.T) { - hub := NewHub() - - client, done := startHubConn(t, hub) - registerPeer(t, client, "peer-a") - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") - - if err := client.Close(); err != nil { - t.Fatalf("client.Close() error = %v", err) - } - <-done - - waitFor(t, func() bool { return !hub.HasPeer("peer-a") }, "peer-a to be unregistered") -} - -func TestServeConnDoesNotEmitEndpointLatencyEventsOnForward(t *testing.T) { - logger := &recordingLogger{} - hub := NewHub(WithLogger(logger)) - - sender, senderDone := startHubConn(t, hub) - receiver, receiverDone := startHubConn(t, hub) - registerPeer(t, sender, "peer-a") - registerPeer(t, receiver, "peer-b") - waitFor(t, func() bool { return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") }, "both peers to be registered") - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 11, - From: "spoofed", - To: "peer-b", - Body: []byte("hello"), - } - if err := sender.Send(msg); err != nil { - t.Fatalf("sender.Send() error = %v", err) - } - - got, err := receiver.Receive() - if err != nil { - t.Fatalf("receiver.Receive() error = %v", err) - } - msg.From = "peer-a" - if !reflect.DeepEqual(got, msg) { - t.Fatalf("forwarded message mismatch: got %+v want %+v", got, msg) - } - - events := logger.Events() - if len(events) != 0 { - t.Fatalf("event count = %d, want 0 because server is a black-box relay", len(events)) - } - - _ = sender.Close() - _ = receiver.Close() - <-senderDone - <-receiverDone -} - -func TestServeConnDoesNotLogLatencyEventsForUnknownTarget(t *testing.T) { - logger := &recordingLogger{} - hub := NewHub(WithLogger(logger)) - - client, done := startHubConn(t, hub) - registerPeer(t, client, "peer-a") - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") - - if err := client.Send(protocol.Message{ - Type: protocol.MessageTypeText, - ID: 15, - From: "peer-a", - To: "missing-peer", - Body: []byte("hello"), - }); err != nil { - t.Fatalf("Send(text) error = %v", err) - } - - got, err := client.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if got.Type != protocol.MessageTypeError { - t.Fatalf("got message type %s, want %s", got.Type, protocol.MessageTypeError) - } - if events := logger.Events(); len(events) != 0 { - t.Fatalf("event count = %d, want 0 for unknown target path", len(events)) - } - - _ = client.Close() - <-done -} - -func TestServeConnDoesNotLogLatencyEventsForDuplicateRegister(t *testing.T) { - logger := &recordingLogger{} - hub := NewHub(WithLogger(logger)) - - first, firstDone := startHubConn(t, hub) - registerPeer(t, first, "peer-a") - waitFor(t, func() bool { return hub.HasPeer("peer-a") }, "peer-a to be registered") - - second, secondDone := startHubConn(t, hub) - registerPeer(t, second, "peer-a") - - got, err := second.Receive() - if err != nil { - t.Fatalf("second.Receive() error = %v", err) - } - if got.Type != protocol.MessageTypeError { - t.Fatalf("got type %s, want %s", got.Type, protocol.MessageTypeError) - } - if events := logger.Events(); len(events) != 0 { - t.Fatalf("event count = %d, want 0 for duplicate register path", len(events)) - } - - _ = first.Close() - <-secondDone - <-firstDone -} - -func startHubConn(t *testing.T, hub *Hub) (*transport.TCPConn, <-chan error) { - t.Helper() - - listener, err := net.Listen("tcp", "127.0.0.1:0") - if err != nil { - t.Fatalf("net.Listen() error = %v", err) - } - done := make(chan error, 1) - - go func() { - serverSide, acceptErr := listener.Accept() - if acceptErr != nil { - done <- acceptErr - return - } - done <- hub.ServeConn(serverSide) - }() - - clientSide, err := net.Dial("tcp", listener.Addr().String()) - if err != nil { - _ = listener.Close() - t.Fatalf("net.Dial() error = %v", err) - } - if err := listener.Close(); err != nil { - t.Fatalf("listener.Close() error = %v", err) - } - - conn, err := transport.NewTCPConn(clientSide) - if err != nil { - _ = clientSide.Close() - t.Fatalf("transport.NewTCPConn() error = %v", err) - } - - return conn, done -} - -func registerPeer(t *testing.T, conn *transport.TCPConn, peerID string) { - t.Helper() - - if err := conn.Send(protocol.Message{ - Type: protocol.MessageTypeRegister, - From: peerID, - To: protocol.ServerPeerID, - }); err != nil { - t.Fatalf("Send(register %s) error = %v", peerID, err) - } -} - -func waitFor(t *testing.T, condition func() bool, description string) { - t.Helper() - - deadline := time.Now().Add(500 * time.Millisecond) - for time.Now().Before(deadline) { - if condition() { - return - } - time.Sleep(10 * time.Millisecond) - } - - t.Fatalf("timed out waiting for %s", description) -} diff --git a/cmd/internal/server/kcp_hub.go b/cmd/internal/server/kcp_hub.go deleted file mode 100644 index 717eeaa..0000000 --- a/cmd/internal/server/kcp_hub.go +++ /dev/null @@ -1,414 +0,0 @@ -package server - -import ( - "errors" - "fmt" - "log" - "net" - "strings" - "sync" - "time" - - kcp "github.com/xtaci/kcp-go/v5" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -const kcpRelayMaxDatagramSize = 60 * 1024 - -var ( - errKCPRelayUnavailable = errors.New("server: kcp relay socket is not configured") - errKCPRelayPeerUnknown = errors.New("server: kcp relay peer address is unknown") - errKCPRelayTooLarge = errors.New("server: kcp relay message too large") - errKCPUnknownLocalTarget = errors.New("server: unknown local kcp target") -) - -// KCPOption 用于配置 KCPHub 的可选行为。 -type KCPOption func(*KCPHub) - -// WithKCPLogger 为 KCP hub 注入时延日志记录器。 -func WithKCPLogger(logger latencylog.Logger) KCPOption { - return func(hub *KCPHub) { - hub.logger = logger - } -} - -// WithKCPSessionStatsLogger 为 KCP hub 注入会话统计日志器。 -func WithKCPSessionStatsLogger(logger transport.KCPSessionStatsLogger, interval time.Duration) KCPOption { - return func(hub *KCPHub) { - hub.sessionStatsLogger = logger - hub.sessionStatsInterval = interval - } -} - -// KCPHub 管理已注册 peer 的 KCP 会话，并负责在它们之间转发消息。 -type KCPHub struct { - mu sync.RWMutex - peers map[string]*transport.KCPConn - logger latencylog.Logger - sessionStatsLogger transport.KCPSessionStatsLogger - sessionStatsInterval time.Duration - relaySocket net.PacketConn - relayPeerAddr net.Addr - relayLearnPeer bool -} - -// NewKCPHub 创建一个空的 KCP 连接中心。 -func NewKCPHub(opts ...KCPOption) *KCPHub { - hub := &KCPHub{ - peers: make(map[string]*transport.KCPConn), - logger: latencylog.NoopLogger{}, - } - for _, opt := range opts { - opt(hub) - } - if hub.logger == nil { - hub.logger = latencylog.NoopLogger{} - } - return hub -} - -// SetRelaySocket 配置 KCPHub 的原始 UDP relay 信道。 -func (h *KCPHub) SetRelaySocket(conn net.PacketConn, peerAddr net.Addr, learnPeer bool) { - h.mu.Lock() - defer h.mu.Unlock() - - h.relaySocket = conn - h.relayPeerAddr = cloneRelayAddr(peerAddr) - h.relayLearnPeer = learnPeer -} - -// HasPeer 返回给定 ID 是否已经注册到 hub。 -func (h *KCPHub) HasPeer(peerID string) bool { - h.mu.RLock() - defer h.mu.RUnlock() - - _, ok := h.peers[peerID] - return ok -} - -// ServeRelay 持续从 relay UDP socket 读取消息，并尝试本地投递。 -func (h *KCPHub) ServeRelay() error { - h.mu.RLock() - conn := h.relaySocket - h.mu.RUnlock() - - if conn == nil { - return errKCPRelayUnavailable - } - - buffer := make([]byte, kcpRelayMaxDatagramSize) - for { - n, addr, err := conn.ReadFrom(buffer) - if err != nil { - if isExpectedRelayServeExit(err) { - return nil - } - return fmt.Errorf("server: relay receive packet: %w", err) - } - - if !h.acceptRelayPeer(addr) { - log.Printf("kcp relay dropped packet from unexpected peer %s", addr) - continue - } - - msg, err := protocol.DecodeMessage(buffer[:n]) - if err != nil { - log.Printf("kcp relay dropped invalid packet from %s: %v", addr, err) - continue - } - - if !isRelayBusinessOrErrorMessage(msg.Type) { - log.Printf("kcp relay dropped unsupported message type %s from %s", msg.Type, addr) - continue - } - - if err := h.deliverRelayedMessage(msg); err != nil { - log.Printf("kcp relay delivery for %s -> %s failed: %v", msg.From, msg.To, err) - } - } -} - -// ServeSession 处理一条新接入的 KCP 会话。 -func (h *KCPHub) ServeSession(session *kcp.UDPSession) error { - sessionDesc := describeKCPSession(session) - log.Printf("kcp hub accepted session %s", sessionDesc) - - conn, err := transport.NewKCPConn( - session, - transport.WithKCPLogger(h.logger, latencylog.NodeRoleServer, "hub"), - transport.WithKCPSessionStatsLogger(h.sessionStatsLogger, h.sessionStatsInterval), - ) - if err != nil { - _ = session.Close() - return fmt.Errorf("server: create kcp transport conn: %w", err) - } - - peerID, err := h.registerConn(conn, sessionDesc) - if err != nil { - _ = conn.Close() - return err - } - defer h.unregister(peerID, conn, sessionDesc) - - return h.receivePeerLoop(peerID, conn, sessionDesc) -} - -// 注册新连接时，KCPHub 期望第一条消息是一个 register 消息，包含 peer 的 ID -func (h *KCPHub) registerConn(conn *transport.KCPConn, sessionDesc string) (string, error) { - msg, err := conn.Receive() - if err != nil { - log.Printf("kcp hub session %s failed before register: %v", sessionDesc, err) - return "", fmt.Errorf("server: receive kcp register: %w", err) - } - - if msg.Type != protocol.MessageTypeRegister { - log.Printf("kcp hub rejecting session %s: first message type=%s from=%s", sessionDesc, msg.Type, msg.From) - if sendErr := sendKCPServerError(conn, msg.From, "first message must be register"); sendErr != nil { - return "", fmt.Errorf("server: reject unregistered kcp peer: %w", sendErr) - } - return "", fmt.Errorf("server: first kcp message must be register, got %s", msg.Type) - } - - h.mu.Lock() - defer h.mu.Unlock() - - if _, exists := h.peers[msg.From]; exists { - log.Printf("kcp hub rejecting duplicate peer %q on session %s", msg.From, sessionDesc) - if sendErr := sendKCPServerError(conn, msg.From, fmt.Sprintf("duplicate peer id: %s", msg.From)); sendErr != nil { - return "", fmt.Errorf("server: duplicate kcp peer id %s: %w", msg.From, sendErr) - } - return "", fmt.Errorf("server: duplicate kcp peer id: %s", msg.From) - } - - h.peers[msg.From] = conn - log.Printf("kcp hub registered peer %q on session %s (peers=%d)", msg.From, sessionDesc, len(h.peers)) - return msg.From, nil -} - -// handlePeerMessage 处理已注册 peer 发来的消息，并将其转发给目标 peer。 -func (h *KCPHub) handlePeerMessage(peerID string, conn *transport.KCPConn, msg protocol.Message) error { - switch msg.Type { - case protocol.MessageTypeText, protocol.MessageTypeFile: - msg.From = peerID - - if err := h.deliverToLocalPeer(msg); err == nil { - return nil - } else if !errors.Is(err, errKCPUnknownLocalTarget) { - log.Printf("kcp hub local delivery failed for %s -> %s: %v", peerID, msg.To, err) - return sendKCPServerError(conn, peerID, fmt.Sprintf("failed to forward to %s", msg.To)) - } - - log.Printf("kcp hub local target miss for %s -> %s; attempting relay", peerID, msg.To) - err := h.forwardToRelay(msg) - switch { - case err == nil: - return nil - case errors.Is(err, errKCPRelayUnavailable): - log.Printf("kcp hub target %s unavailable for %s: no relay configured", msg.To, peerID) - return sendKCPServerError(conn, peerID, fmt.Sprintf("unknown target: %s", msg.To)) - case errors.Is(err, errKCPRelayPeerUnknown): - log.Printf("kcp hub relay peer address is unknown for %s -> %s", peerID, msg.To) - return sendKCPServerError(conn, peerID, "failed to relay to remote peer") - case errors.Is(err, errKCPRelayTooLarge): - log.Printf("kcp hub relay rejected oversize message %s -> %s (%d bytes)", peerID, msg.To, len(msg.Body)) - return sendKCPServerError(conn, peerID, "message too large for relay udp") - default: - log.Printf("kcp hub relay forward failed for %s -> %s: %v", peerID, msg.To, err) - return sendKCPServerError(conn, peerID, "failed to relay to remote peer") - } - case protocol.MessageTypeRegister, protocol.MessageTypeError: - if err := sendKCPServerError(conn, peerID, "registered peers can only send text or file messages"); err != nil { - return fmt.Errorf("server: send kcp protocol error: %w", err) - } - return fmt.Errorf("server: unexpected kcp message type from peer %s: %s", peerID, msg.Type) - default: - if err := sendKCPServerError(conn, peerID, fmt.Sprintf("unsupported message type: %s", msg.Type)); err != nil { - return fmt.Errorf("server: send unsupported kcp type error: %w", err) - } - return fmt.Errorf("server: unsupported kcp message type: %s", msg.Type) - } -} - -// receivePeerLoop 持续读取 peer 发来的消息，并交给 handlePeerMessage 处理，直到连接出错。 -func (h *KCPHub) receivePeerLoop(peerID string, conn *transport.KCPConn, sessionDesc string) error { - for { - msg, err := conn.Receive() - if err != nil { - _ = conn.Close() - log.Printf("kcp hub receive loop ending for peer %q on session %s: %v", peerID, sessionDesc, err) - return fmt.Errorf("transport: kcp receive loop read: %w", err) - } - - if err := h.handlePeerMessage(peerID, conn, msg); err != nil { - _ = conn.Close() - log.Printf("kcp hub handler ending for peer %q on session %s: %v", peerID, sessionDesc, err) - return fmt.Errorf("transport: kcp receive loop handler: %w", err) - } - } -} - -func (h *KCPHub) deliverRelayedMessage(msg protocol.Message) error { - if err := h.deliverToLocalPeer(msg); err == nil { - return nil - } else if !errors.Is(err, errKCPUnknownLocalTarget) { - if msg.Type == protocol.MessageTypeError { - log.Printf("kcp relay dropped undeliverable server error to %s: %v", msg.To, err) - return nil - } - return h.forwardRelayServerError(msg.From, fmt.Sprintf("failed to forward to %s", msg.To)) - } - - if msg.Type == protocol.MessageTypeError { - log.Printf("kcp relay dropped server error for unknown local peer %s", msg.To) - return nil - } - - log.Printf("kcp hub relayed target miss for %s -> %s; sending error back", msg.From, msg.To) - return h.forwardRelayServerError(msg.From, fmt.Sprintf("unknown target: %s", msg.To)) -} - -func (h *KCPHub) deliverToLocalPeer(msg protocol.Message) error { - targetConn, ok := h.lookup(msg.To) - if !ok { - return fmt.Errorf("%w: %s", errKCPUnknownLocalTarget, msg.To) - } - if err := targetConn.Send(msg); err != nil { - h.unregister(msg.To, targetConn, "local-forward-failure") - _ = targetConn.Close() - return fmt.Errorf("server: forward to local peer %s: %w", msg.To, err) - } - return nil -} - -func (h *KCPHub) forwardToRelay(msg protocol.Message) error { - payload, err := protocol.EncodeMessage(msg) - if err != nil { - return fmt.Errorf("server: encode relay message: %w", err) - } - if len(payload) > kcpRelayMaxDatagramSize { - return errKCPRelayTooLarge - } - - h.mu.RLock() - conn := h.relaySocket - peerAddr := cloneRelayAddr(h.relayPeerAddr) - h.mu.RUnlock() - - if conn == nil { - return errKCPRelayUnavailable - } - if peerAddr == nil { - return errKCPRelayPeerUnknown - } - - if _, err := conn.WriteTo(payload, peerAddr); err != nil { - return fmt.Errorf("server: relay write to %s: %w", peerAddr, err) - } - return nil -} - -func (h *KCPHub) forwardRelayServerError(to, message string) error { - return h.forwardToRelay(protocol.Message{ - Type: protocol.MessageTypeError, - From: protocol.ServerPeerID, - To: to, - Body: []byte(message), - }) -} - -func (h *KCPHub) acceptRelayPeer(addr net.Addr) bool { - h.mu.Lock() - defer h.mu.Unlock() - - if h.relayPeerAddr == nil && h.relayLearnPeer { - h.relayPeerAddr = cloneRelayAddr(addr) - log.Printf("kcp hub learned relay peer %s", addr) - return true - } - if h.relayPeerAddr == nil { - return true - } - return sameRelayAddr(h.relayPeerAddr, addr) -} - -func (h *KCPHub) lookup(peerID string) (*transport.KCPConn, bool) { - h.mu.RLock() - defer h.mu.RUnlock() - - conn, ok := h.peers[peerID] - return conn, ok -} - -func (h *KCPHub) unregister(peerID string, conn *transport.KCPConn, sessionDesc string) { - h.mu.Lock() - defer h.mu.Unlock() - - current, ok := h.peers[peerID] - if ok && current == conn { - delete(h.peers, peerID) - log.Printf("kcp hub unregistered peer %q from session %s (peers=%d)", peerID, sessionDesc, len(h.peers)) - } -} - -func sendKCPServerError(conn *transport.KCPConn, to, message string) error { - return conn.Send(protocol.Message{ - Type: protocol.MessageTypeError, - From: protocol.ServerPeerID, - To: to, - Body: []byte(message), - }) -} - -func isRelayBusinessOrErrorMessage(messageType protocol.MessageType) bool { - switch messageType { - case protocol.MessageTypeText, protocol.MessageTypeFile, protocol.MessageTypeError: - return true - default: - return false - } -} - -func isExpectedRelayServeExit(err error) bool { - return errors.Is(err, net.ErrClosed) || strings.Contains(err.Error(), "use of closed network connection") -} - -func cloneRelayAddr(addr net.Addr) net.Addr { - if addr == nil { - return nil - } - udpAddr, ok := addr.(*net.UDPAddr) - if !ok { - return addr - } - ipCopy := make([]byte, len(udpAddr.IP)) - copy(ipCopy, udpAddr.IP) - return &net.UDPAddr{ - IP: ipCopy, - Port: udpAddr.Port, - Zone: udpAddr.Zone, - } -} - -func sameRelayAddr(left, right net.Addr) bool { - if left == nil || right == nil { - return left == right - } - return left.String() == right.String() -} - -func describeKCPSession(session *kcp.UDPSession) string { - if session == nil { - return "conv= remote= local=" - } - return fmt.Sprintf("conv=%d remote=%s local=%s", session.GetConv(), addrString(session.RemoteAddr()), addrString(session.LocalAddr())) -} - -func addrString(addr net.Addr) string { - if addr == nil { - return "" - } - return addr.String() -} diff --git a/cmd/internal/server/udp_hub.go b/cmd/internal/server/udp_hub.go deleted file mode 100644 index 515e32c..0000000 --- a/cmd/internal/server/udp_hub.go +++ /dev/null @@ -1,189 +0,0 @@ -package server - -import ( - "fmt" - "log" - "net" - "sync" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -// UDPOption 用于配置 UDPHub 的可选行为。 -type UDPOption func(*UDPHub) - -// WithUDPLogger 为 UDP hub 注入时延日志记录器。 -func WithUDPLogger(logger latencylog.Logger) UDPOption { - return func(hub *UDPHub) { - hub.logger = logger - } -} - -// WithUDPTXTimestampDebugLogger 为 UDP hub 注入 TX errqueue 调试日志器。 -func WithUDPTXTimestampDebugLogger(logger transport.TXTimestampDebugLogger) UDPOption { - return func(hub *UDPHub) { - hub.txTimestampDebugLogger = logger - } -} - -// WithUDPLinuxTimestamping controls whether the UDP hub enables Linux timestamping. -func WithUDPLinuxTimestamping(enabled bool) UDPOption { - return func(hub *UDPHub) { - hub.linuxTimestampingEnabled = enabled - } -} - -// UDPHub 管理通过 UDP 注册的 peer，并负责在它们之间转发消息。 -type UDPHub struct { - mu sync.RWMutex - peers map[string]*net.UDPAddr - addrs map[string]string - - conn *transport.UDPConn - logger latencylog.Logger - txTimestampDebugLogger transport.TXTimestampDebugLogger - linuxTimestampingEnabled bool -} - -// NewUDPHub 创建一个新的 UDP 连接中心。 -func NewUDPHub(conn *net.UDPConn, opts ...UDPOption) (*UDPHub, error) { - hub := &UDPHub{ - peers: make(map[string]*net.UDPAddr), - addrs: make(map[string]string), - logger: latencylog.NoopLogger{}, - linuxTimestampingEnabled: true, - } - - for _, opt := range opts { - opt(hub) - } - - if hub.logger == nil { - hub.logger = latencylog.NoopLogger{} - } - - udpConn, err := transport.NewUDPConn( - conn, - nil, - transport.WithUDPLogger(hub.logger, latencylog.NodeRoleServer, "hub"), - transport.WithUDPLinuxTimestamping(hub.linuxTimestampingEnabled), - transport.WithUDPTXTimestampDebugLogger(hub.txTimestampDebugLogger), - ) - if err != nil { - return nil, fmt.Errorf("server: create udp transport conn: %w", err) - } - - hub.conn = udpConn - return hub, nil -} - -// Serve 启动 UDP 接收主循环，持续读取消息并处理注册/转发。 -func (h *UDPHub) Serve() error { - return h.conn.ReceiveLoop(func(msg protocol.Message, addr *net.UDPAddr) error { - if err := h.handleMessage(msg, addr); err != nil { - log.Printf("udp hub: handle message from %s: %v", addr, err) - } - return nil - }) -} - -// HasPeer 返回给定 ID 是否已经注册到 hub。 -func (h *UDPHub) HasPeer(peerID string) bool { - h.mu.RLock() - defer h.mu.RUnlock() - - _, ok := h.peers[peerID] - return ok -} - -func (h *UDPHub) handleMessage(msg protocol.Message, addr *net.UDPAddr) error { - switch msg.Type { - case protocol.MessageTypeRegister: - return h.registerPeer(msg, addr) - case protocol.MessageTypeText, protocol.MessageTypeFile: - return h.forwardMessage(msg, addr) - case protocol.MessageTypeError: - return h.sendErrorTo(addr, msg.From, "peers cannot send error messages") - default: - peerID := h.lookupPeerID(addr) - if peerID == "" { - peerID = msg.From - } - return h.sendErrorTo(addr, peerID, fmt.Sprintf("unsupported message type: %s", msg.Type)) - } -} - -func (h *UDPHub) registerPeer(msg protocol.Message, addr *net.UDPAddr) error { - peerID := msg.From - if peerID == "" { - return h.sendErrorTo(addr, "", "register: missing peer id") - } - - h.mu.Lock() - defer h.mu.Unlock() - - if existingAddr, exists := h.peers[peerID]; exists { - delete(h.addrs, existingAddr.String()) - } - - h.peers[peerID] = addr - h.addrs[addr.String()] = peerID - log.Printf("udp hub: registered peer %s from %s", peerID, addr) - return nil -} - -func (h *UDPHub) forwardMessage(msg protocol.Message, senderAddr *net.UDPAddr) error { - senderID := h.lookupPeerID(senderAddr) - if senderID == "" { - return h.sendErrorTo(senderAddr, msg.From, "not registered; send register first") - } - - msg.From = senderID - - targetAddr := h.lookupAddr(msg.To) - if targetAddr == nil { - return h.sendErrorTo(senderAddr, senderID, fmt.Sprintf("unknown target: %s", msg.To)) - } - - if err := h.conn.SendTo(msg, targetAddr); err != nil { - _ = h.sendErrorTo(senderAddr, senderID, fmt.Sprintf("failed to forward to %s", msg.To)) - return fmt.Errorf("forward to %s at %s: %w", msg.To, targetAddr, err) - } - - return nil -} - -// lookupPeerID 通过 UDP 地址反查 peerID。 -func (h *UDPHub) lookupPeerID(addr *net.UDPAddr) string { - h.mu.RLock() - defer h.mu.RUnlock() - - return h.addrs[addr.String()] -} - -// lookupAddr 通过 peerID 查找 UDP 地址。 -func (h *UDPHub) lookupAddr(peerID string) *net.UDPAddr { - h.mu.RLock() - defer h.mu.RUnlock() - - return h.peers[peerID] -} - -func (h *UDPHub) sendErrorTo(addr *net.UDPAddr, to, message string) error { - if to == "" { - to = "unknown" - } - return h.conn.SendTo(protocol.Message{ - Type: protocol.MessageTypeError, - From: protocol.ServerPeerID, - To: to, - Body: []byte(message), - }, addr) -} - -// Close 关闭底层 UDP 连接。 -func (h *UDPHub) Close() error { - return h.conn.Close() -} diff --git a/cmd/internal/server/udp_hub_test.go b/cmd/internal/server/udp_hub_test.go deleted file mode 100644 index f3db724..0000000 --- a/cmd/internal/server/udp_hub_test.go +++ /dev/null @@ -1,238 +0,0 @@ -package server - -import ( - "net" - "testing" - "time" - - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -// TestUDPHubRegisterAndForward 验证 peer 注册后可以互相转发消息。 -func TestUDPHubRegisterAndForward(t *testing.T) { - hub, hubAddr := startUDPHub(t) - _ = hub - - peerA := dialUDPPeer(t, hubAddr) - peerB := dialUDPPeer(t, hubAddr) - - // 注册 peer-a - sendUDPMessage(t, peerA, protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-a", - To: protocol.ServerPeerID, - }) - - // 注册 peer-b - sendUDPMessage(t, peerB, protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-b", - To: protocol.ServerPeerID, - }) - - // 等待注册被处理 - time.Sleep(50 * time.Millisecond) - - // peer-a 发送消息给 peer-b - sendUDPMessage(t, peerA, protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello from peer-a"), - }) - - // peer-b 应该收到消息 - msg := receiveUDPMessage(t, peerB) - if msg.Type != protocol.MessageTypeText { - t.Fatalf("message type = %s, want text", msg.Type) - } - if msg.From != "peer-a" { - t.Fatalf("message from = %s, want peer-a", msg.From) - } - if msg.To != "peer-b" { - t.Fatalf("message to = %s, want peer-b", msg.To) - } - if string(msg.Body) != "hello from peer-a" { - t.Fatalf("message body = %q, want %q", string(msg.Body), "hello from peer-a") - } -} - -// TestUDPHubRejectsUnregistered 验证未注册的 peer 发送业务消息会收到错误。 -func TestUDPHubRejectsUnregistered(t *testing.T) { - _, hubAddr := startUDPHub(t) - - peer := dialUDPPeer(t, hubAddr) - - // 直接发送业务消息而不注册 - sendUDPMessage(t, peer, protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("should fail"), - }) - - // 应该收到错误响应 - msg := receiveUDPMessage(t, peer) - if msg.Type != protocol.MessageTypeError { - t.Fatalf("message type = %s, want error", msg.Type) - } -} - -// TestUDPHubRejectsUnknownTarget 验证发送到不存在的目标会返回错误。 -func TestUDPHubRejectsUnknownTarget(t *testing.T) { - _, hubAddr := startUDPHub(t) - - peer := dialUDPPeer(t, hubAddr) - - // 注册 - sendUDPMessage(t, peer, protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-a", - To: protocol.ServerPeerID, - }) - - time.Sleep(50 * time.Millisecond) - - // 发送到不存在的目标 - sendUDPMessage(t, peer, protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-nonexistent", - Body: []byte("should fail"), - }) - - // 应该收到错误响应 - msg := receiveUDPMessage(t, peer) - if msg.Type != protocol.MessageTypeError { - t.Fatalf("message type = %s, want error", msg.Type) - } -} - -// TestUDPHubOverridesFromField 验证 server 会覆盖消息的 From 字段。 -func TestUDPHubOverridesFromField(t *testing.T) { - _, hubAddr := startUDPHub(t) - - peerA := dialUDPPeer(t, hubAddr) - peerB := dialUDPPeer(t, hubAddr) - - sendUDPMessage(t, peerA, protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-a", - To: protocol.ServerPeerID, - }) - sendUDPMessage(t, peerB, protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-b", - To: protocol.ServerPeerID, - }) - - time.Sleep(50 * time.Millisecond) - - // peer-a 伪造 From 为 "fake-id" - sendUDPMessage(t, peerA, protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "fake-id", - To: "peer-b", - Body: []byte("spoofed"), - }) - - msg := receiveUDPMessage(t, peerB) - // server 应该用实际注册的 "peer-a" 覆盖 From - if msg.From != "peer-a" { - t.Fatalf("message from = %s, want peer-a (server should override)", msg.From) - } -} - -// startUDPHub 创建并启动一个 UDPHub，返回 hub 和监听地址。 -func startUDPHub(t *testing.T) (*UDPHub, *net.UDPAddr) { - t.Helper() - - addr, err := net.ResolveUDPAddr("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ResolveUDPAddr() error = %v", err) - } - - conn, err := net.ListenUDP("udp", addr) - if err != nil { - t.Fatalf("ListenUDP() error = %v", err) - } - - hub, err := NewUDPHub(conn) - if err != nil { - _ = conn.Close() - t.Fatalf("NewUDPHub() error = %v", err) - } - - go func() { - _ = hub.Serve() - }() - - t.Cleanup(func() { - _ = hub.Close() - }) - - return hub, conn.LocalAddr().(*net.UDPAddr) -} - -// dialUDPPeer 创建一个连接到指定地址的 UDP transport 连接。 -func dialUDPPeer(t *testing.T, serverAddr *net.UDPAddr) *transport.UDPConn { - t.Helper() - - raw, err := net.DialUDP("udp", nil, serverAddr) - if err != nil { - t.Fatalf("DialUDP() error = %v", err) - } - - conn, err := transport.NewUDPConn(raw, nil) - if err != nil { - _ = raw.Close() - t.Fatalf("NewUDPConn() error = %v", err) - } - - t.Cleanup(func() { - _ = conn.Close() - }) - - return conn -} - -// sendUDPMessage 发送一条 UDP 消息。 -func sendUDPMessage(t *testing.T, conn *transport.UDPConn, msg protocol.Message) { - t.Helper() - - if err := conn.Send(msg); err != nil { - t.Fatalf("Send() error = %v", err) - } -} - -// receiveUDPMessage 接收一条 UDP 消息，带超时。 -func receiveUDPMessage(t *testing.T, conn *transport.UDPConn) protocol.Message { - t.Helper() - - type result struct { - msg protocol.Message - err error - } - - ch := make(chan result, 1) - go func() { - msg, _, err := conn.Receive() - ch <- result{msg: msg, err: err} - }() - - select { - case r := <-ch: - if r.err != nil { - t.Fatalf("Receive() error = %v", r.err) - } - return r.msg - case <-time.After(2 * time.Second): - t.Fatal("Receive() timed out after 2s") - return protocol.Message{} - } -} diff --git a/cmd/internal/server/udp_relay.go b/cmd/internal/server/udp_relay.go deleted file mode 100644 index 044be15..0000000 --- a/cmd/internal/server/udp_relay.go +++ /dev/null @@ -1,135 +0,0 @@ -package server - -import ( - "fmt" - "log" - "net" - "sync" - - "omnisocketgo/cmd/internal/protocol" -) - -const udpRelayBufSize = protocol.MaxFrameSize + 1024 - -// UDPRelay transparently forwards UDP datagrams between one downstream client -// and a fixed upstream server. -type UDPRelay struct { - downstream net.PacketConn - upstream *net.UDPConn - - mu sync.RWMutex - clientAddr net.Addr -} - -// NewUDPRelay creates a relay that listens on listenConn and forwards all -// traffic to upstreamAddr. -func NewUDPRelay(listenConn net.PacketConn, upstreamAddr *net.UDPAddr) (*UDPRelay, error) { - if listenConn == nil { - return nil, fmt.Errorf("relay: listen conn is required") - } - if upstreamAddr == nil { - return nil, fmt.Errorf("relay: upstream addr is required") - } - - upstreamConn, err := net.DialUDP(relayUDPNetwork(upstreamAddr), nil, upstreamAddr) - if err != nil { - return nil, fmt.Errorf("relay: dial upstream %s: %w", upstreamAddr, err) - } - - return &UDPRelay{ - downstream: listenConn, - upstream: upstreamConn, - }, nil -} - -// Serve starts bidirectional forwarding and blocks until either direction -// exits with an error. -func (r *UDPRelay) Serve() error { - errCh := make(chan error, 2) - - go func() { - errCh <- r.forwardDownstreamToUpstream() - }() - go func() { - errCh <- r.forwardUpstreamToDownstream() - }() - - err := <-errCh - _ = r.downstream.Close() - _ = r.upstream.Close() - return err -} - -func (r *UDPRelay) forwardDownstreamToUpstream() error { - buf := make([]byte, udpRelayBufSize) - for { - n, addr, err := r.downstream.ReadFrom(buf) - if err != nil { - return fmt.Errorf("relay: read downstream: %w", err) - } - - clientAddr := cloneRelayAddr(addr) - - r.mu.Lock() - previousAddr := cloneRelayAddr(r.clientAddr) - r.clientAddr = clientAddr - r.mu.Unlock() - - switch { - case previousAddr == nil: - log.Printf("relay: learned downstream client %s", clientAddr) - case !sameRelayAddr(previousAddr, clientAddr): - log.Printf("relay: downstream client changed from %s to %s", previousAddr, clientAddr) - } - - if _, err := r.upstream.Write(buf[:n]); err != nil { - return fmt.Errorf("relay: write upstream: %w", err) - } - - log.Printf("relay: forwarded %d bytes downstream(%s) -> upstream", n, addr) - } -} - -func (r *UDPRelay) forwardUpstreamToDownstream() error { - buf := make([]byte, udpRelayBufSize) - for { - n, err := r.upstream.Read(buf) - if err != nil { - return fmt.Errorf("relay: read upstream: %w", err) - } - - r.mu.RLock() - addr := cloneRelayAddr(r.clientAddr) - r.mu.RUnlock() - - if addr == nil { - log.Printf("relay: dropping %d bytes from upstream (no downstream client yet)", n) - continue - } - - if _, err := r.downstream.WriteTo(buf[:n], addr); err != nil { - return fmt.Errorf("relay: write downstream to %s: %w", addr, err) - } - - log.Printf("relay: forwarded %d bytes upstream -> downstream(%s)", n, addr) - } -} - -func (r *UDPRelay) Close() error { - err1 := r.downstream.Close() - err2 := r.upstream.Close() - if err1 != nil { - return err1 - } - return err2 -} - -func relayUDPNetwork(addr *net.UDPAddr) string { - if addr == nil || addr.IP == nil { - return "udp" - } - if addr.IP.To4() != nil { - return "udp4" - } - return "udp6" -} diff --git a/cmd/internal/server/udp_relay_test.go b/cmd/internal/server/udp_relay_test.go deleted file mode 100644 index 3c1de41..0000000 --- a/cmd/internal/server/udp_relay_test.go +++ /dev/null @@ -1,283 +0,0 @@ -package server - -import ( - "net" - "strings" - "sync" - "testing" - "time" - - kcp "github.com/xtaci/kcp-go/v5" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -func TestUDPRelayKCPForwardAndReturn(t *testing.T) { - hub, hubAddr, hubCleanup := startKCPHubForRelay(t) - defer hubCleanup() - - relayAddr, relay := startUDPRelay(t, hubAddr) - - peerBConn := dialKCPPeer(t, hubAddr) - peerAConn := dialKCPPeer(t, relayAddr) - - if err := peerBConn.Send(protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-b", - To: protocol.ServerPeerID, - }); err != nil { - t.Fatalf("peerB register: %v", err) - } - if err := peerAConn.Send(protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-a", - To: protocol.ServerPeerID, - }); err != nil { - t.Fatalf("peerA register: %v", err) - } - - waitForRelay(t, func() bool { - return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") - }, "both peers to be registered") - waitForRelay(t, func() bool { - relay.mu.RLock() - defer relay.mu.RUnlock() - return relay.clientAddr != nil - }, "relay to learn the downstream peer") - - if err := peerBConn.Send(protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-b", - To: "peer-a", - Body: []byte("hello from peer-b"), - }); err != nil { - t.Fatalf("peerB send text: %v", err) - } - - msg, err := peerAConn.Receive() - if err != nil { - t.Fatalf("peerA receive: %v", err) - } - if msg.Type != protocol.MessageTypeText { - t.Fatalf("message type = %s, want text", msg.Type) - } - if msg.From != "peer-b" { - t.Fatalf("message from = %s, want peer-b", msg.From) - } - if string(msg.Body) != "hello from peer-b" { - t.Fatalf("message body = %q, want %q", string(msg.Body), "hello from peer-b") - } - - if err := peerAConn.Send(protocol.Message{ - Type: protocol.MessageTypeText, - ID: 2, - From: "peer-a", - To: "peer-b", - Body: []byte("reply from peer-a"), - }); err != nil { - t.Fatalf("peerA send text: %v", err) - } - - msg2, err := peerBConn.Receive() - if err != nil { - t.Fatalf("peerB receive: %v", err) - } - if msg2.Type != protocol.MessageTypeText { - t.Fatalf("message type = %s, want text", msg2.Type) - } - if msg2.From != "peer-a" { - t.Fatalf("message from = %s, want peer-a", msg2.From) - } - if string(msg2.Body) != "reply from peer-a" { - t.Fatalf("message body = %q, want %q", string(msg2.Body), "reply from peer-a") - } -} - -func TestUDPRelayKCPFileMessage(t *testing.T) { - hub, hubAddr, hubCleanup := startKCPHubForRelay(t) - defer hubCleanup() - - relayAddr, relay := startUDPRelay(t, hubAddr) - - peerBConn := dialKCPPeer(t, hubAddr) - peerAConn := dialKCPPeer(t, relayAddr) - - if err := peerBConn.Send(protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-b", - To: protocol.ServerPeerID, - }); err != nil { - t.Fatalf("peerB register: %v", err) - } - if err := peerAConn.Send(protocol.Message{ - Type: protocol.MessageTypeRegister, - From: "peer-a", - To: protocol.ServerPeerID, - }); err != nil { - t.Fatalf("peerA register: %v", err) - } - - waitForRelay(t, func() bool { - return hub.HasPeer("peer-a") && hub.HasPeer("peer-b") - }, "both peers to be registered") - waitForRelay(t, func() bool { - relay.mu.RLock() - defer relay.mu.RUnlock() - return relay.clientAddr != nil - }, "relay to learn the downstream peer") - - if err := peerBConn.Send(protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 1, - From: "peer-b", - To: "peer-a", - FileName: "test.bin", - Body: []byte{0xDE, 0xAD, 0xBE, 0xEF}, - }); err != nil { - t.Fatalf("peerB send file: %v", err) - } - - msg, err := peerAConn.Receive() - if err != nil { - t.Fatalf("peerA receive: %v", err) - } - if msg.Type != protocol.MessageTypeFile { - t.Fatalf("message type = %s, want file", msg.Type) - } - if msg.FileName != "test.bin" { - t.Fatalf("file name = %q, want %q", msg.FileName, "test.bin") - } - if string(msg.Body) != string([]byte{0xDE, 0xAD, 0xBE, 0xEF}) { - t.Fatalf("file body = %v, want %v", msg.Body, []byte{0xDE, 0xAD, 0xBE, 0xEF}) - } -} - -func startKCPHubForRelay(t *testing.T) (*KCPHub, string, func()) { - t.Helper() - - hub := NewKCPHub() - - listener, packetConn, err := transport.ListenKCPSessions("127.0.0.1:0", "", nil, latencylog.NodeRoleServer, "hub") - if err != nil { - t.Fatalf("ListenKCPSessions() error = %v", err) - } - - var ( - wg sync.WaitGroup - stop = make(chan struct{}) - ) - - wg.Add(1) - go func() { - defer wg.Done() - for { - session, acceptErr := listener.AcceptKCP() - if acceptErr != nil { - select { - case <-stop: - return - default: - } - if strings.Contains(acceptErr.Error(), "closed") { - return - } - t.Errorf("AcceptKCP() error = %v", acceptErr) - return - } - - wg.Add(1) - go func(sess *kcp.UDPSession) { - defer wg.Done() - if serveErr := hub.ServeSession(sess); serveErr != nil { - msg := serveErr.Error() - if !strings.Contains(msg, "closed") && !strings.Contains(msg, "broken pipe") { - t.Logf("hub.ServeSession() ended with %v", serveErr) - } - } - }(session) - } - }() - - cleanup := func() { - close(stop) - _ = listener.Close() - _ = packetConn.Close() - wg.Wait() - } - - return hub, listener.Addr().String(), cleanup -} - -func dialKCPPeer(t *testing.T, serverAddr string) *transport.KCPConn { - t.Helper() - - session, err := transport.DialKCPSession(serverAddr, "", "", nil, latencylog.NodeRolePeer, "test") - if err != nil { - t.Fatalf("DialKCPSession(%s) error = %v", serverAddr, err) - } - - conn, err := transport.NewKCPConn(session) - if err != nil { - _ = session.Close() - t.Fatalf("NewKCPConn() error = %v", err) - } - - t.Cleanup(func() { - _ = conn.Close() - }) - - return conn -} - -func startUDPRelay(t *testing.T, upstreamAddr string) (string, *UDPRelay) { - t.Helper() - - remoteAddr, err := net.ResolveUDPAddr("udp", upstreamAddr) - if err != nil { - t.Fatalf("ResolveUDPAddr(%s) error = %v", upstreamAddr, err) - } - - conn, err := net.ListenPacket("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ListenPacket() error = %v", err) - } - - relay, err := NewUDPRelay(conn, remoteAddr) - if err != nil { - _ = conn.Close() - t.Fatalf("NewUDPRelay() error = %v", err) - } - - var wg sync.WaitGroup - wg.Add(1) - go func() { - defer wg.Done() - if serveErr := relay.Serve(); serveErr != nil && !isExpectedRelayServeExit(serveErr) { - t.Errorf("relay.Serve() error = %v", serveErr) - } - }() - - t.Cleanup(func() { - _ = relay.Close() - wg.Wait() - }) - - return conn.LocalAddr().String(), relay -} - -func waitForRelay(t *testing.T, condition func() bool, description string) { - t.Helper() - - deadline := time.Now().Add(2 * time.Second) - for time.Now().Before(deadline) { - if condition() { - return - } - time.Sleep(10 * time.Millisecond) - } - - t.Fatalf("timed out waiting for %s", description) -} diff --git a/cmd/internal/transport/kcp.go b/cmd/internal/transport/kcp.go deleted file mode 100644 index 8e7dcc0..0000000 --- a/cmd/internal/transport/kcp.go +++ /dev/null @@ -1,303 +0,0 @@ -package transport - -import ( - "context" - "crypto/rand" - "encoding/binary" - "fmt" - "net" - "sync" - "time" - - kcp "github.com/xtaci/kcp-go/v5" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -const ( - kcpNoDelayNodelay = 1 - kcpNoDelayInterval = 10 - kcpNoDelayResend = 2 - kcpNoDelayNC = 1 - kcpWindowSize = 256 - kcpMTU = 1400 -) - -// KCPConn 是对单条活跃 KCP 会话的轻量封装。 -type KCPConn struct { - session *kcp.UDPSession - - logger latencylog.Logger - nodeRole string - nodeID string - - sessionStatsLogger KCPSessionStatsLogger - sessionStatsInterval time.Duration - sessionStatsSampler *kcpSessionStatsSampler - - writeMu sync.Mutex - closeOnce sync.Once - closeErr error -} - -// KCPOption 用于为 KCPConn 注入可选行为。 -type KCPOption func(*KCPConn) - -// WithKCPLogger 为 KCP 连接发送路径注入业务消息日志上下文。 -func WithKCPLogger(logger latencylog.Logger, nodeRole, nodeID string) KCPOption { - return func(conn *KCPConn) { - conn.logger = logger - conn.nodeRole = nodeRole - conn.nodeID = nodeID - } -} - -// WithKCPSessionStatsLogger 为 KCP 连接注入会话级与进程级统计日志器。 -func WithKCPSessionStatsLogger(logger KCPSessionStatsLogger, interval time.Duration) KCPOption { - return func(conn *KCPConn) { - conn.sessionStatsLogger = logger - conn.sessionStatsInterval = interval - } -} - -// NewKCPConn 用已有的 KCP 会话创建 transport 连接封装。 -func NewKCPConn(session *kcp.UDPSession, opts ...KCPOption) (*KCPConn, error) { - if session == nil { - return nil, fmt.Errorf("transport: nil kcp session") - } - - conn := &KCPConn{ - session: session, - logger: latencylog.NoopLogger{}, - } - for _, opt := range opts { - opt(conn) - } - if conn.logger == nil { - conn.logger = latencylog.NoopLogger{} - } - - configureKCPSession(session) - conn.sessionStatsSampler = newKCPSessionStatsSampler(session, conn.sessionStatsLogger, conn.nodeRole, conn.nodeID, conn.sessionStatsInterval) - return conn, nil -} - -// Send 将一条协议消息完整写入底层 KCP 会话。 -func (c *KCPConn) Send(msg protocol.Message) error { - c.writeMu.Lock() - defer c.writeMu.Unlock() - - latencylog.LogMessageEvent(c.logger, c.nodeRole, c.nodeID, latencylog.EventSendHandoffBegin, msg) - if c.sessionStatsSampler != nil { - c.sessionStatsSampler.SampleEvent(kcpStatsSampleReasonSendHandoffBegin) - } - if err := protocol.WriteMessage(c.session, msg); err != nil { - return fmt.Errorf("transport: kcp send message: %w", err) - } - latencylog.LogMessageEvent(c.logger, c.nodeRole, c.nodeID, latencylog.EventSendHandoffEnd, msg) - if c.sessionStatsSampler != nil { - c.sessionStatsSampler.SampleEvent(kcpStatsSampleReasonSendHandoffEnd) - } - return nil -} - -// Receive 从底层 KCP 会话读取一条完整协议消息。 -func (c *KCPConn) Receive() (protocol.Message, error) { - msg, err := protocol.ReadMessage(c.session) - if err != nil { - return protocol.Message{}, fmt.Errorf("transport: kcp receive message: %w", err) - } - if c.sessionStatsSampler != nil { - c.sessionStatsSampler.SampleEvent(kcpStatsSampleReasonReceive) - } - return msg, nil -} - -// ReceiveLoop 持续读取消息并交给 handler 处理。 -func (c *KCPConn) ReceiveLoop(handler func(protocol.Message) error) error { - for { - msg, err := c.Receive() - if err != nil { - _ = c.Close() - return fmt.Errorf("transport: kcp receive loop read: %w", err) - } - - if err := handler(msg); err != nil { - _ = c.Close() - return fmt.Errorf("transport: kcp receive loop handler: %w", err) - } - } -} - -// Close 关闭底层 KCP 会话，并保证重复调用是安全的。 -func (c *KCPConn) Close() error { - c.closeOnce.Do(func() { - if c.sessionStatsSampler != nil { - c.sessionStatsSampler.Close() - } - c.closeErr = c.session.Close() - }) - return c.closeErr -} - -// DialKCPSession 创建一条主动发起的 KCP 会话，并按项目默认参数配置底层 UDP socket。 -func DialKCPSession(serverAddr, bindIP, bindDevice string, logger KCPPacketDebugLogger, nodeRole, nodeID string) (*kcp.UDPSession, error) { - packetConn, remoteAddr, err := dialKCPPacketConn(serverAddr, bindIP, bindDevice, logger, nodeRole, nodeID) - if err != nil { - return nil, err - } - - convID, err := generateKCPConversationID() - if err != nil { - _ = packetConn.Close() - return nil, fmt.Errorf("transport: generate kcp conversation id: %w", err) - } - - session, err := kcp.NewConn4(convID, remoteAddr, nil, 0, 0, true, packetConn) - if err != nil { - _ = packetConn.Close() - return nil, fmt.Errorf("transport: create kcp session: %w", err) - } - - return session, nil -} - -// ListenKCPSessions 在给定地址上启动 KCP listener，并返回 listener 与底层 packetConn。 -func ListenKCPSessions(listenAddr, bindDevice string, logger KCPPacketDebugLogger, nodeRole, nodeID string) (*kcp.Listener, net.PacketConn, error) { - packetConn, err := listenKCPPacketConn(listenAddr, bindDevice, logger, nodeRole, nodeID) - if err != nil { - return nil, nil, err - } - - listener, err := kcp.ServeConn(nil, 0, 0, packetConn) - if err != nil { - _ = packetConn.Close() - return nil, nil, fmt.Errorf("transport: serve kcp listener: %w", err) - } - - return listener, packetConn, nil -} - -func configureKCPSession(session *kcp.UDPSession) { - session.SetStreamMode(true) - session.SetNoDelay(kcpNoDelayNodelay, kcpNoDelayInterval, kcpNoDelayResend, kcpNoDelayNC) - session.SetWindowSize(kcpWindowSize, kcpWindowSize) - session.SetACKNoDelay(true) - session.SetWriteDelay(false) - session.SetMtu(kcpMTU) -} - -func generateKCPConversationID() (uint32, error) { - var convID uint32 - if err := binary.Read(rand.Reader, binary.LittleEndian, &convID); err != nil { - return 0, err - } - return convID, nil -} - -// ResolveUDPListenConfig parses a UDP listen address and returns the socket -// family that should be used for binding it. -func ResolveUDPListenConfig(listenAddr string) (string, *net.UDPAddr, error) { - udpAddr, err := net.ResolveUDPAddr("udp", listenAddr) - if err != nil { - return "", nil, fmt.Errorf("transport: resolve udp listen addr %s: %w", listenAddr, err) - } - - return udpListenNetwork(udpAddr), udpAddr, nil -} - -func listenKCPPacketConn(listenAddr, bindDevice string, logger KCPPacketDebugLogger, nodeRole, nodeID string) (net.PacketConn, error) { - network, udpAddr, err := ResolveUDPListenConfig(listenAddr) - if err != nil { - return nil, fmt.Errorf("transport: resolve kcp listen addr %s: %w", listenAddr, err) - } - - rawConn, err := listenUDPConn(network, udpAddr, bindDevice) - if err != nil { - return nil, fmt.Errorf("transport: listen %s for kcp on %s: %w", network, udpListenAddr(udpAddr), err) - } - - packetConn, err := newKCPPacketConn(rawConn, logger, nodeRole, nodeID) - if err != nil { - _ = rawConn.Close() - return nil, err - } - - return packetConn, nil -} - -func dialKCPPacketConn(serverAddr, bindIP, bindDevice string, logger KCPPacketDebugLogger, nodeRole, nodeID string) (net.PacketConn, *net.UDPAddr, error) { - remoteAddr, err := net.ResolveUDPAddr("udp", serverAddr) - if err != nil { - return nil, nil, fmt.Errorf("transport: resolve kcp server addr %s: %w", serverAddr, err) - } - - localAddr := &net.UDPAddr{Port: 0} - if bindIP != "" { - ip := net.ParseIP(bindIP) - if ip == nil { - return nil, nil, fmt.Errorf("transport: invalid bind ip %q", bindIP) - } - localAddr.IP = ip - } - - network := "udp" - if remoteAddr.IP.To4() != nil { - network = "udp4" - } - - rawConn, err := listenUDPConn(network, localAddr, bindDevice) - if err != nil { - return nil, nil, fmt.Errorf("transport: listen udp for kcp dial to %s: %w", serverAddr, err) - } - - packetConn, err := newKCPPacketConn(rawConn, logger, nodeRole, nodeID) - if err != nil { - _ = rawConn.Close() - return nil, nil, err - } - - return packetConn, remoteAddr, nil -} - -func listenUDPConn(network string, localAddr *net.UDPAddr, bindDevice string) (*net.UDPConn, error) { - listenConfig := net.ListenConfig{} - if bindDevice != "" { - control, err := udpBindDeviceControl(bindDevice) - if err != nil { - return nil, err - } - listenConfig.Control = control - } - - packetConn, err := listenConfig.ListenPacket(context.Background(), network, udpListenAddr(localAddr)) - if err != nil { - return nil, err - } - - udpConn, ok := packetConn.(*net.UDPConn) - if !ok { - _ = packetConn.Close() - return nil, fmt.Errorf("transport: expected *net.UDPConn, got %T", packetConn) - } - - return udpConn, nil -} - -func udpListenAddr(addr *net.UDPAddr) string { - if addr == nil { - return ":0" - } - return addr.String() -} - -func udpListenNetwork(addr *net.UDPAddr) string { - if addr == nil || addr.IP == nil { - return "udp" - } - if addr.IP.To4() != nil { - return "udp4" - } - return "udp6" -} diff --git a/cmd/internal/transport/kcp_linux_test.go b/cmd/internal/transport/kcp_linux_test.go deleted file mode 100644 index ec1efdd..0000000 --- a/cmd/internal/transport/kcp_linux_test.go +++ /dev/null @@ -1,93 +0,0 @@ -//go:build linux - -package transport - -import ( - "testing" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -func TestKCPLinuxPacketDebugLogsKernelEvents(t *testing.T) { - senderPacketLogger := &recordingKCPPacketDebugLogger{} - receiverPacketLogger := &recordingKCPPacketDebugLogger{} - - sender, accepted, cleanup := newKCPConnPair(t, nil, nil, senderPacketLogger, receiverPacketLogger) - defer cleanup() - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello kcp linux"), - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - receiver := awaitAcceptedKCPConn(t, accepted) - if _, err := receiver.Receive(); err != nil { - t.Fatalf("receiver.Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("sender.Send() error = %v", err) - } - - waitForKCPPacketRecords(t, senderPacketLogger, func(records []KCPPacketDebugRecord) bool { - return hasKCPPacketEvent(records, latencylog.EventATXSched) && hasKCPPacketEvent(records, latencylog.EventATXSoftware) - }, "sender tx kernel timestamp records") - waitForKCPPacketRecords(t, receiverPacketLogger, func(records []KCPPacketDebugRecord) bool { - return hasKCPPacketEvent(records, latencylog.EventBRXSoftware) - }, "receiver rx kernel timestamp records") - - senderRecords := senderPacketLogger.Records() - receiverRecords := receiverPacketLogger.Records() - - assertKCPPacketRecord(t, senderRecords, latencylog.EventATXSched, true) - assertKCPPacketRecord(t, senderRecords, latencylog.EventATXSoftware, true) - assertKCPPacketRecord(t, receiverRecords, latencylog.EventBRXSoftware, false) -} - -func hasKCPPacketEvent(records []KCPPacketDebugRecord, wantEvent string) bool { - for _, record := range records { - if record.Event == wantEvent { - return true - } - } - return false -} - -func assertKCPPacketRecord(t *testing.T, records []KCPPacketDebugRecord, wantEvent string, wantUDPTXID bool) { - t.Helper() - - for _, record := range records { - if record.Event != wantEvent { - continue - } - if record.TSUnixNano <= 0 { - t.Fatalf("record %s timestamp must be positive: %+v", wantEvent, record) - } - if record.PacketBytes <= 0 { - t.Fatalf("record %s packet bytes must be positive: %+v", wantEvent, record) - } - if record.KCPConv == nil { - t.Fatalf("record %s missing kcp_conv: %+v", wantEvent, record) - } - if len(record.Segments) == 0 { - t.Fatalf("record %s missing parsed segments: %+v", wantEvent, record) - } - if wantUDPTXID && record.UDPTXID == nil { - t.Fatalf("record %s missing udp_tx_id: %+v", wantEvent, record) - } - if !wantUDPTXID && record.UDPTXID != nil { - t.Fatalf("record %s unexpected udp_tx_id: %+v", wantEvent, record) - } - return - } - - t.Fatalf("missing KCP packet debug event %s in %+v", wantEvent, records) -} diff --git a/cmd/internal/transport/kcp_packet_conn.go b/cmd/internal/transport/kcp_packet_conn.go deleted file mode 100644 index 68908f0..0000000 --- a/cmd/internal/transport/kcp_packet_conn.go +++ /dev/null @@ -1,83 +0,0 @@ -package transport - -import ( - "net" - "sync" - "time" -) - -func newKCPPacketConn(conn *net.UDPConn, logger KCPPacketDebugLogger, nodeRole, nodeID string) (net.PacketConn, error) { - return newPlatformKCPPacketConn(conn, logger, nodeRole, nodeID) -} - -type kcpPacketConnBase struct { - conn *net.UDPConn - logger KCPPacketDebugLogger - nodeRole string - nodeID string - - closeOnce sync.Once - closeErr error - closed chan struct{} -} - -func (c *kcpPacketConnBase) LocalAddr() net.Addr { - return c.conn.LocalAddr() -} - -func (c *kcpPacketConnBase) Close() error { - c.closeOnce.Do(func() { - close(c.closed) - c.closeErr = c.conn.Close() - }) - return c.closeErr -} - -func (c *kcpPacketConnBase) SetDeadline(t time.Time) error { - return c.conn.SetDeadline(t) -} - -func (c *kcpPacketConnBase) SetReadDeadline(t time.Time) error { - return c.conn.SetReadDeadline(t) -} - -func (c *kcpPacketConnBase) SetWriteDeadline(t time.Time) error { - return c.conn.SetWriteDeadline(t) -} - -func (c *kcpPacketConnBase) SetReadBuffer(bytes int) error { - return c.conn.SetReadBuffer(bytes) -} - -func (c *kcpPacketConnBase) SetWriteBuffer(bytes int) error { - return c.conn.SetWriteBuffer(bytes) -} - -func (c *kcpPacketConnBase) logKCPPacketDebugRecord(record KCPPacketDebugRecord) { - if c.logger == nil { - return - } - _ = c.logger.LogKCPPacketDebugRecord(record) -} - -func (c *kcpPacketConnBase) newKCPPacketDebugRecord(event string, remoteAddr net.Addr, packetBytes int, tsUnixNano int64, udpTxID *uint32, kcpConv *uint32, segments []KCPPacketDebugSegment) KCPPacketDebugRecord { - record := KCPPacketDebugRecord{ - Event: event, - NodeRole: c.nodeRole, - NodeID: c.nodeID, - LocalAddr: "", - RemoteAddr: "", - PacketBytes: packetBytes, - UDPTXID: udpTxID, - KCPConv: kcpConv, - Segments: append([]KCPPacketDebugSegment(nil), segments...), - TSUnixNano: tsUnixNano, - } - if localAddr := c.conn.LocalAddr(); localAddr != nil { - record.LocalAddr = localAddr.String() - } - if remoteAddr != nil { - record.RemoteAddr = remoteAddr.String() - } - return record -} diff --git a/cmd/internal/transport/kcp_packet_conn_linux.go b/cmd/internal/transport/kcp_packet_conn_linux.go deleted file mode 100644 index 91b8a75..0000000 --- a/cmd/internal/transport/kcp_packet_conn_linux.go +++ /dev/null @@ -1,344 +0,0 @@ -//go:build linux - -package transport - -import ( - "errors" - "fmt" - "net" - "sync" - "syscall" - "time" - - "omnisocketgo/cmd/internal/latencylog" -) - -type kcpPendingPacketDebug struct { - remoteAddr net.Addr - packetBytes int - kcpConv *uint32 - segments []KCPPacketDebugSegment - timestamps map[string]int64 -} - -type platformKCPPacketConn struct { - *kcpPacketConnBase - - raw syscall.RawConn - - writeMu sync.Mutex - pendingMu sync.Mutex - pendingTX map[uint32]kcpPendingPacketDebug - nextTXID uint32 -} - -func newPlatformKCPPacketConn(conn *net.UDPConn, logger KCPPacketDebugLogger, nodeRole, nodeID string) (net.PacketConn, error) { - packetConn := &platformKCPPacketConn{ - kcpPacketConnBase: &kcpPacketConnBase{ - conn: conn, - logger: logger, - nodeRole: nodeRole, - nodeID: nodeID, - closed: make(chan struct{}), - }, - pendingTX: make(map[uint32]kcpPendingPacketDebug), - } - - if logger == nil { - return packetConn, nil - } - - if err := packetConn.initLinuxTimestamping(); err != nil { - return nil, err - } - - go packetConn.collectTXErrqueueLoop() - return packetConn, nil -} - -func (c *platformKCPPacketConn) Close() error { - return c.kcpPacketConnBase.Close() -} - -func (c *platformKCPPacketConn) ReadFrom(p []byte) (int, net.Addr, error) { - if c.raw == nil { - return c.conn.ReadFrom(p) - } - - for { - n, addr, rxTimestamp, err := c.recvmsgRaw(p) - if err != nil { - if isWouldBlock(err) { - time.Sleep(linuxDataPollInterval) - continue - } - return 0, nil, err - } - - if rxTimestamp > 0 { - kcpConv, segments := parseKCPPacketMetadata(p[:n]) - c.logKCPPacketDebugRecord(c.newKCPPacketDebugRecord( - latencylog.EventBRXSoftware, - addr, - n, - rxTimestamp, - nil, - kcpConv, - segments, - )) - } - - return n, addr, nil - } -} - -func (c *platformKCPPacketConn) WriteTo(p []byte, addr net.Addr) (int, error) { - c.writeMu.Lock() - defer c.writeMu.Unlock() - - if c.raw == nil { - return c.conn.WriteTo(p, addr) - } - - udpAddr, ok := addr.(*net.UDPAddr) - if !ok { - return 0, fmt.Errorf("transport: kcp packet write target must be UDPAddr, got %T", addr) - } - - // Reserve the local txID before the send so an immediately-arriving errqueue - // event can still find its pending record. If the send never succeeds, roll - // the reservation back to keep the local txID mirror aligned with the kernel. - kcpConv, segments := parseKCPPacketMetadata(p) - expectedTXID := c.reservePendingTX(udpAddr, len(p), kcpConv, segments) - for { - err := c.sendmsgRaw(p, udpAddr) - if err != nil { - if isWouldBlock(err) { - time.Sleep(linuxDataPollInterval) - continue - } - c.rollbackPendingTX(expectedTXID) - return 0, err - } - - return len(p), nil - } -} - -func (c *platformKCPPacketConn) initLinuxTimestamping() error { - rawConn, err := c.conn.SyscallConn() - if err != nil || rawConn == nil { - if err != nil { - return fmt.Errorf("transport: kcp get syscall conn: %w", err) - } - return fmt.Errorf("transport: kcp missing syscall conn") - } - - if err := configureLinuxSocketWriteBuffer(rawConn); err != nil { - return fmt.Errorf("transport: kcp configure socket write buffer: %w", err) - } - - flagCandidates := []int{ - linuxSOFTimestampingTXSched | - linuxSOFTimestampingTXSoftware | - linuxSOFTimestampingRXSoftware | - linuxSOFTimestampingSoftware | - linuxSOFTimestampingOptID | - linuxSOFTimestampingOptTSONLY, - linuxSOFTimestampingTXSched | - linuxSOFTimestampingTXSoftware | - linuxSOFTimestampingRXSoftware | - linuxSOFTimestampingSoftware | - linuxSOFTimestampingOptTSONLY, - } - - var lastErr error - for _, flags := range flagCandidates { - err := rawConn.Control(func(fd uintptr) { - lastErr = syscall.SetsockoptInt(int(fd), syscall.SOL_SOCKET, linuxSOTimestampingNew, flags) - }) - if err != nil { - return err - } - if lastErr == nil { - c.raw = rawConn - return nil - } - if !errors.Is(lastErr, syscall.EINVAL) { - return lastErr - } - } - - return lastErr -} - -func (c *platformKCPPacketConn) recvmsgRaw(buf []byte) (int, net.Addr, int64, error) { - var ( - n int - rxTimeNS int64 - from syscall.Sockaddr - opErr error - ) - - err := c.raw.Control(func(fd uintptr) { - oob := make([]byte, linuxTimestampControlBufferSize) - readN, oobN, _, sa, recvErr := syscall.Recvmsg(int(fd), buf, oob, 0) - if recvErr != nil { - opErr = recvErr - return - } - n = readN - from = sa - rxTimeNS = parseRXTimestampControlMessages(oob[:oobN]) - }) - if err != nil { - return 0, nil, 0, err - } - if opErr != nil { - return 0, nil, 0, opErr - } - - return n, sockaddrToUDPAddr(from), rxTimeNS, nil -} - -func (c *platformKCPPacketConn) sendmsgRaw(payload []byte, addr *net.UDPAddr) error { - var opErr error - sa := udpAddrToSockaddr(addr) - if sa == nil { - return fmt.Errorf("transport: invalid udp addr %v", addr) - } - - err := c.raw.Control(func(fd uintptr) { - opErr = syscall.Sendmsg(int(fd), payload, nil, sa, 0) - }) - if err != nil { - return err - } - return opErr -} - -func (c *platformKCPPacketConn) collectTXErrqueueLoop() { - for { - event, err := c.recvTXErrqueueOnce() - if err != nil { - if isWouldBlock(err) { - if c.isClosed() { - return - } - time.Sleep(linuxTXTimestampPollInterval) - continue - } - if c.isClosed() { - return - } - return - } - if event.EventName == "" || event.TSUnixNano <= 0 { - continue - } - - if event.EventName != latencylog.EventATXSched && event.EventName != latencylog.EventATXSoftware { - continue - } - - record, complete := c.recordPendingTXEvent(event) - if record == nil { - continue - } - - udpTxID := event.EEData - c.logKCPPacketDebugRecord(c.newKCPPacketDebugRecord( - event.EventName, - record.remoteAddr, - record.packetBytes, - event.TSUnixNano, - &udpTxID, - record.kcpConv, - record.segments, - )) - - if complete { - c.pendingMu.Lock() - delete(c.pendingTX, event.EEData) - c.pendingMu.Unlock() - } - } -} - -func (c *platformKCPPacketConn) recvTXErrqueueOnce() (txTimestampEvent, error) { - var ( - event txTimestampEvent - opErr error - ) - - err := c.raw.Control(func(fd uintptr) { - oob := make([]byte, linuxTimestampControlBufferSize) - _, oobN, _, _, recvErr := syscall.Recvmsg(int(fd), nil, oob, syscall.MSG_ERRQUEUE|syscall.MSG_DONTWAIT) - if recvErr != nil { - opErr = recvErr - return - } - event, _ = parseTXTimestampControlMessages(oob[:oobN]) - }) - if err != nil { - return txTimestampEvent{}, err - } - if opErr != nil { - return txTimestampEvent{}, opErr - } - return event, nil -} - -func (c *platformKCPPacketConn) reservePendingTX(remoteAddr net.Addr, packetBytes int, kcpConv *uint32, segments []KCPPacketDebugSegment) uint32 { - c.pendingMu.Lock() - defer c.pendingMu.Unlock() - - txID := c.nextTXID - c.nextTXID++ - c.pendingTX[txID] = kcpPendingPacketDebug{ - remoteAddr: remoteAddr, - packetBytes: packetBytes, - kcpConv: kcpConv, - segments: append([]KCPPacketDebugSegment(nil), segments...), - timestamps: make(map[string]int64, 2), - } - - return txID -} - -func (c *platformKCPPacketConn) rollbackPendingTX(txID uint32) { - c.pendingMu.Lock() - defer c.pendingMu.Unlock() - - delete(c.pendingTX, txID) - if c.nextTXID == txID+1 { - c.nextTXID = txID - } -} - -func (c *platformKCPPacketConn) recordPendingTXEvent(event txTimestampEvent) (*kcpPendingPacketDebug, bool) { - c.pendingMu.Lock() - defer c.pendingMu.Unlock() - - record, ok := c.pendingTX[event.EEData] - if !ok { - return nil, false - } - if existing, exists := record.timestamps[event.EventName]; !exists || event.TSUnixNano < existing { - record.timestamps[event.EventName] = event.TSUnixNano - } - c.pendingTX[event.EEData] = record - - complete := hasCompleteTXTimestampPair(record.timestamps) - copyRecord := record - return ©Record, complete -} - -func (c *platformKCPPacketConn) isClosed() bool { - select { - case <-c.closed: - return true - default: - } - return false -} diff --git a/cmd/internal/transport/kcp_packet_conn_linux_test.go b/cmd/internal/transport/kcp_packet_conn_linux_test.go deleted file mode 100644 index d66825c..0000000 --- a/cmd/internal/transport/kcp_packet_conn_linux_test.go +++ /dev/null @@ -1,62 +0,0 @@ -//go:build linux - -package transport - -import ( - "net" - "testing" -) - -func TestKCPPendingTXReservationRollbackRestoresSequence(t *testing.T) { - conn := &platformKCPPacketConn{ - kcpPacketConnBase: &kcpPacketConnBase{ - closed: make(chan struct{}), - }, - pendingTX: make(map[uint32]kcpPendingPacketDebug), - } - - conv := uint32(42) - txID := conn.reservePendingTX(&net.UDPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 9000}, 128, &conv, nil) - if txID != 0 { - t.Fatalf("reservePendingTX() txID = %d, want 0", txID) - } - if conn.nextTXID != 1 { - t.Fatalf("nextTXID after reserve = %d, want 1", conn.nextTXID) - } - if _, ok := conn.pendingTX[txID]; !ok { - t.Fatal("pendingTX missing reserved record") - } - - conn.rollbackPendingTX(txID) - - if conn.nextTXID != 0 { - t.Fatalf("nextTXID after rollback = %d, want 0", conn.nextTXID) - } - if _, ok := conn.pendingTX[txID]; ok { - t.Fatal("pendingTX still contains rolled back record") - } -} - -func TestKCPPendingTXReservationPreservesLaterSequenceOnOutOfOrderRollback(t *testing.T) { - conn := &platformKCPPacketConn{ - kcpPacketConnBase: &kcpPacketConnBase{ - closed: make(chan struct{}), - }, - pendingTX: make(map[uint32]kcpPendingPacketDebug), - } - - first := conn.reservePendingTX(&net.UDPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 9000}, 64, nil, nil) - second := conn.reservePendingTX(&net.UDPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 9001}, 64, nil, nil) - if first != 0 || second != 1 { - t.Fatalf("reserved tx IDs = %d,%d, want 0,1", first, second) - } - - conn.rollbackPendingTX(first) - - if conn.nextTXID != 2 { - t.Fatalf("nextTXID after out-of-order rollback = %d, want 2", conn.nextTXID) - } - if _, ok := conn.pendingTX[second]; !ok { - t.Fatal("pendingTX lost later reservation after rollback") - } -} diff --git a/cmd/internal/transport/kcp_packet_conn_other.go b/cmd/internal/transport/kcp_packet_conn_other.go deleted file mode 100644 index 8eea9ab..0000000 --- a/cmd/internal/transport/kcp_packet_conn_other.go +++ /dev/null @@ -1,29 +0,0 @@ -//go:build !linux - -package transport - -import "net" - -type platformKCPPacketConn struct { - *kcpPacketConnBase -} - -func newPlatformKCPPacketConn(conn *net.UDPConn, logger KCPPacketDebugLogger, nodeRole, nodeID string) (net.PacketConn, error) { - return &platformKCPPacketConn{ - kcpPacketConnBase: &kcpPacketConnBase{ - conn: conn, - logger: logger, - nodeRole: nodeRole, - nodeID: nodeID, - closed: make(chan struct{}), - }, - }, nil -} - -func (c *platformKCPPacketConn) ReadFrom(p []byte) (int, net.Addr, error) { - return c.conn.ReadFrom(p) -} - -func (c *platformKCPPacketConn) WriteTo(p []byte, addr net.Addr) (int, error) { - return c.conn.WriteTo(p, addr) -} diff --git a/cmd/internal/transport/kcp_packet_debug.go b/cmd/internal/transport/kcp_packet_debug.go deleted file mode 100644 index cfc289b..0000000 --- a/cmd/internal/transport/kcp_packet_debug.go +++ /dev/null @@ -1,87 +0,0 @@ -package transport - -import ( - "encoding/json" - "fmt" - "os" - "path/filepath" - "sync" -) - -// KCPPacketDebugRecord 是 KCP 底层 UDP packet kernel timestamp 的一条 JSONL 调试记录。 -type KCPPacketDebugRecord struct { - Event string `json:"event"` - NodeRole string `json:"node_role,omitempty"` - NodeID string `json:"node_id,omitempty"` - LocalAddr string `json:"local_addr,omitempty"` - RemoteAddr string `json:"remote_addr,omitempty"` - PacketBytes int `json:"packet_bytes"` - UDPTXID *uint32 `json:"udp_tx_id,omitempty"` - KCPConv *uint32 `json:"kcp_conv,omitempty"` - Segments []KCPPacketDebugSegment `json:"segments,omitempty"` - TSUnixNano int64 `json:"ts_unix_nano"` -} - -// KCPPacketDebugSegment 是一个 UDP datagram 中解析出的 KCP segment 头信息。 -type KCPPacketDebugSegment struct { - Cmd uint8 `json:"cmd"` - SN uint32 `json:"sn"` - UNA uint32 `json:"una"` - Frg uint8 `json:"frg"` - Wnd uint16 `json:"wnd"` - Len uint32 `json:"len"` -} - -// KCPPacketDebugLogger 接收 KCP packet 级调试记录。 -type KCPPacketDebugLogger interface { - LogKCPPacketDebugRecord(record KCPPacketDebugRecord) error -} - -// JSONLKCPPacketDebugLogger 以 JSONL 形式追加写 KCP packet 调试日志。 -type JSONLKCPPacketDebugLogger struct { - mu sync.Mutex - closeOnce sync.Once - closeErr error - file *os.File -} - -// NewJSONLKCPPacketDebugLogger 创建一个线程安全的 KCP packet JSONL 日志器。 -func NewJSONLKCPPacketDebugLogger(path string) (*JSONLKCPPacketDebugLogger, error) { - dir := filepath.Dir(path) - if err := os.MkdirAll(dir, 0o755); err != nil { - return nil, fmt.Errorf("transport: create kcp packet debug log dir %s: %w", dir, err) - } - - file, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644) - if err != nil { - return nil, fmt.Errorf("transport: open kcp packet debug log %s: %w", path, err) - } - - return &JSONLKCPPacketDebugLogger{file: file}, nil -} - -// LogKCPPacketDebugRecord 以单行 JSON 的形式追加一条 KCP packet 调试记录。 -func (l *JSONLKCPPacketDebugLogger) LogKCPPacketDebugRecord(record KCPPacketDebugRecord) error { - line, err := json.Marshal(record) - if err != nil { - return err - } - - l.mu.Lock() - defer l.mu.Unlock() - - if _, err := l.file.Write(append(line, '\n')); err != nil { - return err - } - - return nil -} - -// Close 关闭底层文件；重复调用是安全的。 -func (l *JSONLKCPPacketDebugLogger) Close() error { - l.closeOnce.Do(func() { - l.closeErr = l.file.Close() - }) - - return l.closeErr -} diff --git a/cmd/internal/transport/kcp_packet_metadata.go b/cmd/internal/transport/kcp_packet_metadata.go deleted file mode 100644 index 1469814..0000000 --- a/cmd/internal/transport/kcp_packet_metadata.go +++ /dev/null @@ -1,64 +0,0 @@ -package transport - -import "encoding/binary" - -const kcpPacketHeaderSize = 24 - -func parseKCPPacketMetadata(packet []byte) (*uint32, []KCPPacketDebugSegment) { - conv := parseKCPConversationID(packet) - segments, ok := parseKCPPacketSegments(packet) - if !ok { - return conv, nil - } - return conv, segments -} - -func parseKCPConversationID(packet []byte) *uint32 { - if len(packet) < 4 { - return nil - } - conv := binary.LittleEndian.Uint32(packet[:4]) - return &conv -} - -// ParseKCPConversationID 从原始 KCP UDP datagram 中提取 conv ID。 -func ParseKCPConversationID(packet []byte) (uint32, bool) { - conv := parseKCPConversationID(packet) - if conv == nil { - return 0, false - } - return *conv, true -} - -func parseKCPPacketSegments(packet []byte) ([]KCPPacketDebugSegment, bool) { - if len(packet) == 0 { - return nil, false - } - - data := packet - segments := make([]KCPPacketDebugSegment, 0, 1) - for len(data) > 0 { - if len(data) < kcpPacketHeaderSize { - return nil, false - } - - segmentLen := binary.LittleEndian.Uint32(data[20:]) - totalLen := kcpPacketHeaderSize + int(segmentLen) - if len(data) < totalLen { - return nil, false - } - - segments = append(segments, KCPPacketDebugSegment{ - Cmd: data[4], - Frg: data[5], - Wnd: binary.LittleEndian.Uint16(data[6:8]), - SN: binary.LittleEndian.Uint32(data[12:16]), - UNA: binary.LittleEndian.Uint32(data[16:20]), - Len: segmentLen, - }) - - data = data[totalLen:] - } - - return segments, true -} diff --git a/cmd/internal/transport/kcp_session_stats.go b/cmd/internal/transport/kcp_session_stats.go deleted file mode 100644 index ca4311d..0000000 --- a/cmd/internal/transport/kcp_session_stats.go +++ /dev/null @@ -1,444 +0,0 @@ -package transport - -import ( - "encoding/json" - "fmt" - "net" - "os" - "path/filepath" - "reflect" - "sync" - "time" - - kcp "github.com/xtaci/kcp-go/v5" -) - -const DefaultKCPSessionStatsInterval = 100 * time.Millisecond - -const ( - kcpSessionStatsRecordTypeSessionSample = "session_sample" - kcpSessionStatsRecordTypeProcessSNMPSample = "process_snmp_sample" - - kcpStatsSampleReasonPeriodic = "periodic" - kcpStatsSampleReasonSendHandoffBegin = "send_handoff_begin" - kcpStatsSampleReasonSendHandoffEnd = "send_handoff_end" - kcpStatsSampleReasonReceive = "receive" - kcpStatsSampleReasonClose = "close" -) - -// KCPSessionStatsRecord is a JSONL record for KCP session and process stats. -type KCPSessionStatsRecord struct { - RecordType string `json:"record_type"` - NodeRole string `json:"node_role,omitempty"` - NodeID string `json:"node_id,omitempty"` - LocalAddr string `json:"local_addr,omitempty"` - RemoteAddr string `json:"remote_addr,omitempty"` - Conv *uint32 `json:"conv,omitempty"` - TSUnixNano int64 `json:"ts_unix_nano"` - SampleReason string `json:"sample_reason"` - - RTOMillis *uint32 `json:"rto_ms,omitempty"` - SRTTMillis *int32 `json:"srtt_ms,omitempty"` - SRTTVarMillis *int32 `json:"srttvar_ms,omitempty"` - - BytesSent *uint64 `json:"bytes_sent,omitempty"` - BytesReceived *uint64 `json:"bytes_received,omitempty"` - InPkts *uint64 `json:"in_pkts,omitempty"` - OutPkts *uint64 `json:"out_pkts,omitempty"` - InSegs *uint64 `json:"in_segs,omitempty"` - OutSegs *uint64 `json:"out_segs,omitempty"` - RetransSegs *uint64 `json:"retrans_segs,omitempty"` - FastRetransSegs *uint64 `json:"fast_retrans_segs,omitempty"` - EarlyRetransSegs *uint64 `json:"early_retrans_segs,omitempty"` - LostSegs *uint64 `json:"lost_segs,omitempty"` - RepeatSegs *uint64 `json:"repeat_segs,omitempty"` - InErrs *uint64 `json:"in_errs,omitempty"` - KCPInErrs *uint64 `json:"kcp_in_errs,omitempty"` - - RingBufferSndQueue *uint64 `json:"ring_buffer_snd_queue,omitempty"` - RingBufferRcvQueue *uint64 `json:"ring_buffer_rcv_queue,omitempty"` - RingBufferSndBuffer *uint64 `json:"ring_buffer_snd_buffer,omitempty"` - CurrEstab *uint64 `json:"curr_estab,omitempty"` -} - -// KCPSessionStatsLogger receives KCP session stats records. -type KCPSessionStatsLogger interface { - LogKCPSessionStatsRecord(record KCPSessionStatsRecord) error -} - -// JSONLKCPSessionStatsLogger appends KCP session stats as JSONL. -type JSONLKCPSessionStatsLogger struct { - mu sync.Mutex - closeOnce sync.Once - closeErr error - file *os.File -} - -// NewJSONLKCPSessionStatsLogger creates a thread-safe JSONL stats logger. -func NewJSONLKCPSessionStatsLogger(path string) (*JSONLKCPSessionStatsLogger, error) { - dir := filepath.Dir(path) - if err := os.MkdirAll(dir, 0o755); err != nil { - return nil, fmt.Errorf("transport: create kcp session stats log dir %s: %w", dir, err) - } - - file, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644) - if err != nil { - return nil, fmt.Errorf("transport: open kcp session stats log %s: %w", path, err) - } - - return &JSONLKCPSessionStatsLogger{file: file}, nil -} - -// LogKCPSessionStatsRecord appends one JSONL record. -func (l *JSONLKCPSessionStatsLogger) LogKCPSessionStatsRecord(record KCPSessionStatsRecord) error { - line, err := json.Marshal(record) - if err != nil { - return err - } - - l.mu.Lock() - defer l.mu.Unlock() - - if _, err := l.file.Write(append(line, '\n')); err != nil { - return err - } - return nil -} - -// Close closes the underlying file. -func (l *JSONLKCPSessionStatsLogger) Close() error { - l.closeOnce.Do(func() { - l.closeErr = l.file.Close() - }) - return l.closeErr -} - -// ParseKCPSessionStatsInterval parses a duration string for stats sampling. -func ParseKCPSessionStatsInterval(raw string) (time.Duration, error) { - if raw == "" { - return DefaultKCPSessionStatsInterval, nil - } - - interval, err := time.ParseDuration(raw) - if err != nil { - return 0, fmt.Errorf("transport: parse kcp session stats interval %q: %w", raw, err) - } - if interval <= 0 { - return 0, fmt.Errorf("transport: kcp session stats interval must be greater than zero") - } - return interval, nil -} - -type kcpSessionStatsSource interface { - GetConv() uint32 - GetRTO() uint32 - GetSRTT() int32 - GetSRTTVar() int32 - LocalAddr() net.Addr - RemoteAddr() net.Addr -} - -type kcpSessionStatsSampler struct { - source kcpSessionStatsSource - logger KCPSessionStatsLogger - nodeRole string - nodeID string - interval time.Duration - processSampler *kcpProcessStatsSampler - - sampleMu sync.Mutex - stopOnce sync.Once - stopCh chan struct{} - stoppedCh chan struct{} -} - -func newKCPSessionStatsSampler(source kcpSessionStatsSource, logger KCPSessionStatsLogger, nodeRole, nodeID string, interval time.Duration) *kcpSessionStatsSampler { - if source == nil || logger == nil { - return nil - } - if interval <= 0 { - interval = DefaultKCPSessionStatsInterval - } - - sampler := &kcpSessionStatsSampler{ - source: source, - logger: logger, - nodeRole: nodeRole, - nodeID: nodeID, - interval: interval, - processSampler: acquireKCPProcessStatsSampler(logger, nodeRole, nodeID, interval), - stopCh: make(chan struct{}), - stoppedCh: make(chan struct{}), - } - - go sampler.run() - return sampler -} - -func (s *kcpSessionStatsSampler) run() { - ticker := time.NewTicker(s.interval) - defer ticker.Stop() - defer close(s.stoppedCh) - - for { - select { - case <-ticker.C: - s.logSessionSample(kcpStatsSampleReasonPeriodic) - case <-s.stopCh: - return - } - } -} - -func (s *kcpSessionStatsSampler) SampleEvent(reason string) { - if s == nil { - return - } - - s.logSessionSample(reason) - if s.processSampler == nil { - return - } - - if reason == kcpStatsSampleReasonClose { - s.processSampler.requestSampleAndWait(reason) - return - } - s.processSampler.requestSample(reason) -} - -func (s *kcpSessionStatsSampler) Close() { - if s == nil { - return - } - - s.stopOnce.Do(func() { - s.SampleEvent(kcpStatsSampleReasonClose) - close(s.stopCh) - <-s.stoppedCh - releaseKCPProcessStatsSampler(s.processSampler) - }) -} - -func (s *kcpSessionStatsSampler) logSessionSample(reason string) { - s.sampleMu.Lock() - defer s.sampleMu.Unlock() - - conv := s.source.GetConv() - rto := s.source.GetRTO() - srtt := s.source.GetSRTT() - srttVar := s.source.GetSRTTVar() - - record := KCPSessionStatsRecord{ - RecordType: kcpSessionStatsRecordTypeSessionSample, - NodeRole: s.nodeRole, - NodeID: s.nodeID, - LocalAddr: addrString(s.source.LocalAddr()), - RemoteAddr: addrString(s.source.RemoteAddr()), - Conv: uint32Ptr(conv), - TSUnixNano: time.Now().UTC().UnixNano(), - SampleReason: reason, - RTOMillis: uint32Ptr(rto), - SRTTMillis: int32Ptr(srtt), - SRTTVarMillis: int32Ptr(srttVar), - } - - _ = s.logger.LogKCPSessionStatsRecord(record) -} - -type kcpProcessSampleRequest struct { - reason string - done chan struct{} -} - -type kcpProcessStatsSampler struct { - key string - logger KCPSessionStatsLogger - nodeRole string - nodeID string - interval time.Duration - requestCh chan kcpProcessSampleRequest - stopCh chan struct{} - stoppedCh chan struct{} - - sampleMu sync.Mutex - previous *kcp.Snmp - refCount int -} - -var ( - kcpProcessSamplersMu sync.Mutex - kcpProcessSamplers = make(map[string]*kcpProcessStatsSampler) -) - -func acquireKCPProcessStatsSampler(logger KCPSessionStatsLogger, nodeRole, nodeID string, interval time.Duration) *kcpProcessStatsSampler { - if logger == nil { - return nil - } - if interval <= 0 { - interval = DefaultKCPSessionStatsInterval - } - - key := fmt.Sprintf("%s|%s|%s|%d", kcpStatsLoggerIdentity(logger), nodeRole, nodeID, interval) - - kcpProcessSamplersMu.Lock() - defer kcpProcessSamplersMu.Unlock() - - if sampler, ok := kcpProcessSamplers[key]; ok { - sampler.refCount++ - return sampler - } - - sampler := &kcpProcessStatsSampler{ - key: key, - logger: logger, - nodeRole: nodeRole, - nodeID: nodeID, - interval: interval, - requestCh: make(chan kcpProcessSampleRequest, 1), - stopCh: make(chan struct{}), - stoppedCh: make(chan struct{}), - previous: kcp.DefaultSnmp.Copy(), - refCount: 1, - } - kcpProcessSamplers[key] = sampler - go sampler.run() - return sampler -} - -func releaseKCPProcessStatsSampler(sampler *kcpProcessStatsSampler) { - if sampler == nil { - return - } - - kcpProcessSamplersMu.Lock() - sampler.refCount-- - if sampler.refCount > 0 { - kcpProcessSamplersMu.Unlock() - return - } - delete(kcpProcessSamplers, sampler.key) - close(sampler.stopCh) - kcpProcessSamplersMu.Unlock() - - <-sampler.stoppedCh -} - -func (s *kcpProcessStatsSampler) run() { - ticker := time.NewTicker(s.interval) - defer ticker.Stop() - defer close(s.stoppedCh) - - for { - select { - case <-ticker.C: - s.logProcessSample(kcpStatsSampleReasonPeriodic) - case req := <-s.requestCh: - s.logProcessSample(req.reason) - if req.done != nil { - close(req.done) - } - case <-s.stopCh: - return - } - } -} - -func (s *kcpProcessStatsSampler) requestSample(reason string) { - if s == nil { - return - } - - select { - case s.requestCh <- kcpProcessSampleRequest{reason: reason}: - default: - } -} - -func (s *kcpProcessStatsSampler) requestSampleAndWait(reason string) { - if s == nil { - return - } - - done := make(chan struct{}) - select { - case s.requestCh <- kcpProcessSampleRequest{reason: reason, done: done}: - <-done - default: - s.logProcessSample(reason) - } -} - -func (s *kcpProcessStatsSampler) logProcessSample(reason string) { - s.sampleMu.Lock() - defer s.sampleMu.Unlock() - - current := kcp.DefaultSnmp.Copy() - record := newKCPProcessSNMPSampleRecord(s.nodeRole, s.nodeID, reason, s.previous, current) - s.previous = current - - _ = s.logger.LogKCPSessionStatsRecord(record) -} - -func newKCPProcessSNMPSampleRecord(nodeRole, nodeID, reason string, previous, current *kcp.Snmp) KCPSessionStatsRecord { - return KCPSessionStatsRecord{ - RecordType: kcpSessionStatsRecordTypeProcessSNMPSample, - NodeRole: nodeRole, - NodeID: nodeID, - TSUnixNano: time.Now().UTC().UnixNano(), - SampleReason: reason, - BytesSent: uint64Ptr(diffUint64(previous.BytesSent, current.BytesSent)), - BytesReceived: uint64Ptr(diffUint64(previous.BytesReceived, current.BytesReceived)), - InPkts: uint64Ptr(diffUint64(previous.InPkts, current.InPkts)), - OutPkts: uint64Ptr(diffUint64(previous.OutPkts, current.OutPkts)), - InSegs: uint64Ptr(diffUint64(previous.InSegs, current.InSegs)), - OutSegs: uint64Ptr(diffUint64(previous.OutSegs, current.OutSegs)), - RetransSegs: uint64Ptr(diffUint64(previous.RetransSegs, current.RetransSegs)), - FastRetransSegs: uint64Ptr(diffUint64(previous.FastRetransSegs, current.FastRetransSegs)), - EarlyRetransSegs: uint64Ptr(diffUint64(previous.EarlyRetransSegs, current.EarlyRetransSegs)), - LostSegs: uint64Ptr(diffUint64(previous.LostSegs, current.LostSegs)), - RepeatSegs: uint64Ptr(diffUint64(previous.RepeatSegs, current.RepeatSegs)), - InErrs: uint64Ptr(diffUint64(previous.InErrs, current.InErrs)), - KCPInErrs: uint64Ptr(diffUint64(previous.KCPInErrors, current.KCPInErrors)), - RingBufferSndQueue: uint64Ptr(current.RingBufferSndQueue), - RingBufferRcvQueue: uint64Ptr(current.RingBufferRcvQueue), - RingBufferSndBuffer: uint64Ptr(current.RingBufferSndBuffer), - CurrEstab: uint64Ptr(current.CurrEstab), - } -} - -func kcpStatsLoggerIdentity(logger KCPSessionStatsLogger) string { - value := reflect.ValueOf(logger) - switch value.Kind() { - case reflect.Pointer, reflect.Map, reflect.Func, reflect.Slice, reflect.Chan, reflect.UnsafePointer: - return fmt.Sprintf("%T:%x", logger, value.Pointer()) - default: - return fmt.Sprintf("%T:%v", logger, logger) - } -} - -func diffUint64(previous, current uint64) uint64 { - if current < previous { - return 0 - } - return current - previous -} - -func addrString(addr net.Addr) string { - if addr == nil { - return "" - } - return addr.String() -} - -func uint32Ptr(value uint32) *uint32 { - return &value -} - -func uint64Ptr(value uint64) *uint64 { - return &value -} - -func int32Ptr(value int32) *int32 { - return &value -} diff --git a/cmd/internal/transport/kcp_session_stats_test.go b/cmd/internal/transport/kcp_session_stats_test.go deleted file mode 100644 index c542172..0000000 --- a/cmd/internal/transport/kcp_session_stats_test.go +++ /dev/null @@ -1,409 +0,0 @@ -package transport - -import ( - "encoding/binary" - "net" - "sync" - "testing" - "time" - - kcp "github.com/xtaci/kcp-go/v5" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -type recordingKCPSessionStatsLogger struct { - mu sync.Mutex - records []KCPSessionStatsRecord -} - -func (l *recordingKCPSessionStatsLogger) LogKCPSessionStatsRecord(record KCPSessionStatsRecord) error { - l.mu.Lock() - defer l.mu.Unlock() - - l.records = append(l.records, record) - return nil -} - -func (l *recordingKCPSessionStatsLogger) Records() []KCPSessionStatsRecord { - l.mu.Lock() - defer l.mu.Unlock() - - return append([]KCPSessionStatsRecord(nil), l.records...) -} - -type stubKCPSessionStatsSource struct { - conv uint32 - rto uint32 - srtt int32 - srttVar int32 - local net.Addr - remote net.Addr -} - -func (s stubKCPSessionStatsSource) GetConv() uint32 { return s.conv } -func (s stubKCPSessionStatsSource) GetRTO() uint32 { return s.rto } -func (s stubKCPSessionStatsSource) GetSRTT() int32 { return s.srtt } -func (s stubKCPSessionStatsSource) GetSRTTVar() int32 { return s.srttVar } -func (s stubKCPSessionStatsSource) LocalAddr() net.Addr { return s.local } -func (s stubKCPSessionStatsSource) RemoteAddr() net.Addr { return s.remote } - -func TestParseKCPPacketMetadataSingleSegment(t *testing.T) { - packet := buildTestKCPDatagram(42, []testKCPSegment{ - {cmd: 81, sn: 7, una: 3, frg: 1, wnd: 128, length: 5}, - }) - - conv, segments := parseKCPPacketMetadata(packet) - if conv == nil || *conv != 42 { - t.Fatalf("conv = %v, want 42", conv) - } - if len(segments) != 1 { - t.Fatalf("segment count = %d, want 1", len(segments)) - } - if segments[0].Cmd != 81 || segments[0].SN != 7 || segments[0].UNA != 3 || segments[0].Frg != 1 || segments[0].Wnd != 128 || segments[0].Len != 5 { - t.Fatalf("segment = %+v, want expected header values", segments[0]) - } -} - -func TestParseKCPPacketMetadataMultiSegment(t *testing.T) { - packet := buildTestKCPDatagram(99, []testKCPSegment{ - {cmd: 82, sn: 10, una: 5, frg: 2, wnd: 64, length: 3}, - {cmd: 83, sn: 11, una: 6, frg: 0, wnd: 96, length: 0}, - }) - - conv, segments := parseKCPPacketMetadata(packet) - if conv == nil || *conv != 99 { - t.Fatalf("conv = %v, want 99", conv) - } - if len(segments) != 2 { - t.Fatalf("segment count = %d, want 2", len(segments)) - } - if segments[0].SN != 10 || segments[1].SN != 11 { - t.Fatalf("segments = %+v, want in-order sequence numbers", segments) - } -} - -func TestParseKCPPacketMetadataReturnsNoSegmentsForTruncatedPacket(t *testing.T) { - packet := buildTestKCPDatagram(7, []testKCPSegment{ - {cmd: 81, sn: 1, una: 0, frg: 0, wnd: 32, length: 4}, - }) - packet = packet[:len(packet)-1] - - conv, segments := parseKCPPacketMetadata(packet) - if conv == nil || *conv != 7 { - t.Fatalf("conv = %v, want 7", conv) - } - if len(segments) != 0 { - t.Fatalf("segments = %+v, want empty on truncated packet", segments) - } -} - -func TestParseKCPSessionStatsInterval(t *testing.T) { - tests := []struct { - name string - raw string - want time.Duration - wantErr bool - }{ - {name: "default", raw: "", want: DefaultKCPSessionStatsInterval}, - {name: "valid", raw: "250ms", want: 250 * time.Millisecond}, - {name: "invalid format", raw: "soon", wantErr: true}, - {name: "invalid zero", raw: "0s", wantErr: true}, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got, err := ParseKCPSessionStatsInterval(tt.raw) - if tt.wantErr { - if err == nil { - t.Fatalf("ParseKCPSessionStatsInterval(%q) error = nil, want non-nil", tt.raw) - } - return - } - if err != nil { - t.Fatalf("ParseKCPSessionStatsInterval(%q) error = %v", tt.raw, err) - } - if got != tt.want { - t.Fatalf("ParseKCPSessionStatsInterval(%q) = %v, want %v", tt.raw, got, tt.want) - } - }) - } -} - -func TestKCPProcessSNMPSampleRecordUsesCounterDeltasAndGaugeSnapshots(t *testing.T) { - previous := &kcp.Snmp{ - BytesSent: 10, - BytesReceived: 20, - InPkts: 30, - OutPkts: 40, - InSegs: 50, - OutSegs: 60, - RetransSegs: 70, - FastRetransSegs: 80, - EarlyRetransSegs: 90, - LostSegs: 100, - RepeatSegs: 110, - InErrs: 120, - KCPInErrors: 130, - RingBufferSndQueue: 4, - RingBufferRcvQueue: 5, - RingBufferSndBuffer: 6, - CurrEstab: 1, - } - current := &kcp.Snmp{ - BytesSent: 15, - BytesReceived: 22, - InPkts: 31, - OutPkts: 44, - InSegs: 55, - OutSegs: 61, - RetransSegs: 79, - FastRetransSegs: 81, - EarlyRetransSegs: 95, - LostSegs: 101, - RepeatSegs: 115, - InErrs: 121, - KCPInErrors: 135, - RingBufferSndQueue: 9, - RingBufferRcvQueue: 8, - RingBufferSndBuffer: 7, - CurrEstab: 3, - } - - record := newKCPProcessSNMPSampleRecord(latencylog.NodeRoleServer, "hub", kcpStatsSampleReasonReceive, previous, current) - if record.RecordType != kcpSessionStatsRecordTypeProcessSNMPSample { - t.Fatalf("record type = %q, want %q", record.RecordType, kcpSessionStatsRecordTypeProcessSNMPSample) - } - if record.Conv != nil { - t.Fatalf("process sample conv = %v, want nil", record.Conv) - } - if record.BytesSent == nil || *record.BytesSent != 5 { - t.Fatalf("bytes_sent = %v, want 5", record.BytesSent) - } - if record.KCPInErrs == nil || *record.KCPInErrs != 5 { - t.Fatalf("kcp_in_errs = %v, want 5", record.KCPInErrs) - } - if record.RingBufferSndQueue == nil || *record.RingBufferSndQueue != 9 { - t.Fatalf("ring_buffer_snd_queue = %v, want 9", record.RingBufferSndQueue) - } - if record.CurrEstab == nil || *record.CurrEstab != 3 { - t.Fatalf("curr_estab = %v, want 3", record.CurrEstab) - } -} - -func TestKCPProcessStatsSamplerIsSharedPerLoggerRoleNodeAndInterval(t *testing.T) { - logger := &recordingKCPSessionStatsLogger{} - - first := acquireKCPProcessStatsSampler(logger, latencylog.NodeRoleServer, "hub", 10*time.Millisecond) - second := acquireKCPProcessStatsSampler(logger, latencylog.NodeRoleServer, "hub", 10*time.Millisecond) - if first != second { - t.Fatal("expected acquireKCPProcessStatsSampler to reuse sampler for same logger/role/node/interval") - } - - releaseKCPProcessStatsSampler(first) - releaseKCPProcessStatsSampler(second) -} - -func TestKCPSessionStatsSamplerRecordsEventAndPeriodicSamples(t *testing.T) { - kcp.DefaultSnmp.Reset() - - logger := &recordingKCPSessionStatsLogger{} - source := stubKCPSessionStatsSource{ - conv: 77, - rto: 25, - srtt: 10, - srttVar: 3, - local: &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 9001}, - remote: &net.UDPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 9002}, - } - - sampler := newKCPSessionStatsSampler(source, logger, latencylog.NodeRolePeer, "peer-a", 10*time.Millisecond) - waitForKCPSessionStatsRecords(t, logger, func(records []KCPSessionStatsRecord) bool { - return hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonPeriodic) - }, "periodic session sample") - - sampler.SampleEvent(kcpStatsSampleReasonSendHandoffBegin) - sampler.SampleEvent(kcpStatsSampleReasonSendHandoffEnd) - sampler.SampleEvent(kcpStatsSampleReasonReceive) - waitForKCPSessionStatsRecords(t, logger, func(records []KCPSessionStatsRecord) bool { - return hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonSendHandoffBegin) && - hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonSendHandoffEnd) && - hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonReceive) - }, "event-driven session samples") - - sampler.Close() - waitForKCPSessionStatsRecords(t, logger, func(records []KCPSessionStatsRecord) bool { - return hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonClose) && - hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeProcessSNMPSample, kcpStatsSampleReasonClose) - }, "close samples") - - recordCount := len(logger.Records()) - time.Sleep(30 * time.Millisecond) - if got := len(logger.Records()); got != recordCount { - t.Fatalf("record count after Close() = %d, want %d", got, recordCount) - } -} - -func TestKCPConnSessionStatsLoggingSparseTraffic(t *testing.T) { - senderLogger := &recordingKCPSessionStatsLogger{} - receiverLogger := &recordingKCPSessionStatsLogger{} - - sender, accepted, cleanup := newKCPConnPair( - t, - []KCPOption{ - WithKCPLogger(latencylog.NoopLogger{}, latencylog.NodeRolePeer, "peer-a"), - WithKCPSessionStatsLogger(senderLogger, 10*time.Millisecond), - }, - []KCPOption{ - WithKCPLogger(latencylog.NoopLogger{}, latencylog.NodeRolePeer, "peer-b"), - WithKCPSessionStatsLogger(receiverLogger, 10*time.Millisecond), - }, - nil, - nil, - ) - defer cleanup() - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello sparse"), - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - receiver := awaitAcceptedKCPConn(t, accepted) - if _, err := receiver.Receive(); err != nil { - t.Fatalf("receiver.Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("sender.Send() error = %v", err) - } - if err := sender.Close(); err != nil { - t.Fatalf("sender.Close() error = %v", err) - } - if err := receiver.Close(); err != nil { - t.Fatalf("receiver.Close() error = %v", err) - } - - waitForKCPSessionStatsRecords(t, senderLogger, func(records []KCPSessionStatsRecord) bool { - return hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonSendHandoffBegin) && - hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonSendHandoffEnd) && - hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonClose) && - hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeProcessSNMPSample, kcpStatsSampleReasonClose) - }, "sender sparse session stats") - waitForKCPSessionStatsRecords(t, receiverLogger, func(records []KCPSessionStatsRecord) bool { - return hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonReceive) && - hasKCPSessionStatsRecord(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonClose) - }, "receiver sparse session stats") -} - -func TestKCPConnSessionStatsLoggingContinuousTraffic(t *testing.T) { - logger := &recordingKCPSessionStatsLogger{} - - sender, accepted, cleanup := newKCPConnPair( - t, - []KCPOption{ - WithKCPLogger(latencylog.NoopLogger{}, latencylog.NodeRolePeer, "peer-a"), - WithKCPSessionStatsLogger(logger, 20*time.Millisecond), - }, - nil, - nil, - nil, - ) - defer cleanup() - - receiver := awaitAcceptedKCPConn(t, accepted) - receiveErr := make(chan error, 1) - go func() { - for i := 0; i < 12; i++ { - if _, err := receiver.Receive(); err != nil { - receiveErr <- err - return - } - } - receiveErr <- nil - }() - - for i := 0; i < 12; i++ { - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: uint64(i + 1), - From: "peer-a", - To: "peer-b", - Body: []byte("hello continuous"), - } - if err := sender.Send(msg); err != nil { - t.Fatalf("sender.Send(message %d) error = %v", i+1, err) - } - time.Sleep(15 * time.Millisecond) - } - - if err := <-receiveErr; err != nil { - t.Fatalf("receiver.Receive() error = %v", err) - } - - waitForKCPSessionStatsRecords(t, logger, func(records []KCPSessionStatsRecord) bool { - return countKCPSessionStatsRecords(records, kcpSessionStatsRecordTypeSessionSample, kcpStatsSampleReasonPeriodic) >= 2 && - countKCPSessionStatsRecords(records, kcpSessionStatsRecordTypeProcessSNMPSample, kcpStatsSampleReasonPeriodic) >= 2 - }, "continuous periodic session stats") -} - -type testKCPSegment struct { - cmd uint8 - sn uint32 - una uint32 - frg uint8 - wnd uint16 - length uint32 -} - -func buildTestKCPDatagram(conv uint32, segments []testKCPSegment) []byte { - packet := make([]byte, 0) - for _, segment := range segments { - entry := make([]byte, kcpPacketHeaderSize+int(segment.length)) - binary.LittleEndian.PutUint32(entry[0:4], conv) - entry[4] = segment.cmd - entry[5] = segment.frg - binary.LittleEndian.PutUint16(entry[6:8], segment.wnd) - binary.LittleEndian.PutUint32(entry[12:16], segment.sn) - binary.LittleEndian.PutUint32(entry[16:20], segment.una) - binary.LittleEndian.PutUint32(entry[20:24], segment.length) - packet = append(packet, entry...) - } - return packet -} - -func hasKCPSessionStatsRecord(records []KCPSessionStatsRecord, recordType, reason string) bool { - return countKCPSessionStatsRecords(records, recordType, reason) > 0 -} - -func countKCPSessionStatsRecords(records []KCPSessionStatsRecord, recordType, reason string) int { - count := 0 - for _, record := range records { - if record.RecordType == recordType && record.SampleReason == reason { - count++ - } - } - return count -} - -func waitForKCPSessionStatsRecords(t *testing.T, logger *recordingKCPSessionStatsLogger, condition func([]KCPSessionStatsRecord) bool, description string) { - t.Helper() - - deadline := time.Now().Add(2 * time.Second) - for time.Now().Before(deadline) { - records := logger.Records() - if condition(records) { - return - } - time.Sleep(10 * time.Millisecond) - } - - t.Fatalf("timed out waiting for %s", description) -} diff --git a/cmd/internal/transport/kcp_test.go b/cmd/internal/transport/kcp_test.go deleted file mode 100644 index da1e81a..0000000 --- a/cmd/internal/transport/kcp_test.go +++ /dev/null @@ -1,333 +0,0 @@ -package transport - -import ( - "reflect" - "strings" - "sync" - "testing" - "time" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -type recordingKCPPacketDebugLogger struct { - mu sync.Mutex - records []KCPPacketDebugRecord -} - -func (l *recordingKCPPacketDebugLogger) LogKCPPacketDebugRecord(record KCPPacketDebugRecord) error { - l.mu.Lock() - defer l.mu.Unlock() - - l.records = append(l.records, record) - return nil -} - -func (l *recordingKCPPacketDebugLogger) Records() []KCPPacketDebugRecord { - l.mu.Lock() - defer l.mu.Unlock() - - return append([]KCPPacketDebugRecord(nil), l.records...) -} - -type kcpAcceptResult struct { - conn *KCPConn - err error -} - -func TestKCPSendReceiveMessage(t *testing.T) { - tests := []struct { - name string - msg protocol.Message - }{ - { - name: "text", - msg: protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello kcp"), - }, - }, - { - name: "file", - msg: protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 2, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0x00, 0x10, 0xff}, - }, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - sender, accepted, cleanup := newKCPConnPair( - t, - nil, - []KCPOption{WithKCPLogger(latencylog.NoopLogger{}, latencylog.NodeRolePeer, "peer-b")}, - nil, - nil, - ) - defer cleanup() - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(tt.msg) - }() - - receiver := awaitAcceptedKCPConn(t, accepted) - got, err := receiver.Receive() - if err != nil { - t.Fatalf("receiver.Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("sender.Send() error = %v", err) - } - if !reflect.DeepEqual(got, tt.msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, tt.msg) - } - }) - } -} - -func TestKCPSendLogsHandoffEvents(t *testing.T) { - logger := &recordingLogger{} - sender, accepted, cleanup := newKCPConnPair( - t, - []KCPOption{WithKCPLogger(logger, latencylog.NodeRolePeer, "peer-a")}, - nil, - nil, - nil, - ) - defer cleanup() - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 7, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - receiver := awaitAcceptedKCPConn(t, accepted) - got, err := receiver.Receive() - if err != nil { - t.Fatalf("receiver.Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("sender.Send() error = %v", err) - } - if !reflect.DeepEqual(got, msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, msg) - } - - events := logger.Events() - if len(events) != 2 { - t.Fatalf("event count = %d, want 2", len(events)) - } - if events[0].Event != latencylog.EventSendHandoffBegin { - t.Fatalf("first event = %q, want %q", events[0].Event, latencylog.EventSendHandoffBegin) - } - if events[1].Event != latencylog.EventSendHandoffEnd { - t.Fatalf("second event = %q, want %q", events[1].Event, latencylog.EventSendHandoffEnd) - } -} - -func TestKCPReceiveLoopStopsOnClose(t *testing.T) { - sender, accepted, cleanup := newKCPConnPair(t, nil, nil, nil, nil) - defer cleanup() - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - receiver := awaitAcceptedKCPConn(t, accepted) - - var ( - mu sync.Mutex - got []protocol.Message - ) - loopErr := make(chan error, 1) - go func() { - loopErr <- receiver.ReceiveLoop(func(msg protocol.Message) error { - mu.Lock() - got = append(got, msg) - mu.Unlock() - return receiver.Close() - }) - }() - - if err := <-sendErr; err != nil { - t.Fatalf("sender.Send() error = %v", err) - } - - err := <-loopErr - if err == nil || (!strings.Contains(err.Error(), "closed") && !strings.Contains(err.Error(), "pipe")) { - t.Fatalf("ReceiveLoop() error = %v, want close-related error", err) - } - - mu.Lock() - defer mu.Unlock() - if len(got) != 1 || !reflect.DeepEqual(got[0], msg) { - t.Fatalf("received messages mismatch: got %+v want [%+v]", got, msg) - } -} - -func TestKCPCloseIsIdempotent(t *testing.T) { - sender, _, cleanup := newKCPConnPair(t, nil, nil, nil, nil) - defer cleanup() - - if err := sender.Close(); err != nil { - t.Fatalf("Close(first) error = %v", err) - } - if err := sender.Close(); err != nil { - t.Fatalf("Close(second) error = %v, want nil", err) - } -} - -func TestResolveUDPListenConfigSelectsSocketFamily(t *testing.T) { - tests := []struct { - name string - listenAddr string - wantNetwork string - wantAddr string - }{ - { - name: "ipv4 unspecified", - listenAddr: "0.0.0.0:10909", - wantNetwork: "udp4", - wantAddr: "0.0.0.0:10909", - }, - { - name: "ipv4 loopback", - listenAddr: "127.0.0.1:10909", - wantNetwork: "udp4", - wantAddr: "127.0.0.1:10909", - }, - { - name: "ipv6 loopback", - listenAddr: "[::1]:10909", - wantNetwork: "udp6", - wantAddr: "[::1]:10909", - }, - { - name: "host omitted", - listenAddr: ":10909", - wantNetwork: "udp", - wantAddr: ":10909", - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - gotNetwork, gotAddr, err := ResolveUDPListenConfig(tt.listenAddr) - if err != nil { - t.Fatalf("ResolveUDPListenConfig(%q) error = %v", tt.listenAddr, err) - } - if gotNetwork != tt.wantNetwork { - t.Fatalf("network = %q, want %q", gotNetwork, tt.wantNetwork) - } - if gotAddr.String() != tt.wantAddr { - t.Fatalf("addr = %q, want %q", gotAddr.String(), tt.wantAddr) - } - }) - } -} - -func newKCPConnPair(t *testing.T, senderOpts []KCPOption, receiverOpts []KCPOption, senderPacketLogger KCPPacketDebugLogger, receiverPacketLogger KCPPacketDebugLogger) (*KCPConn, <-chan kcpAcceptResult, func()) { - t.Helper() - - listener, packetConn, err := ListenKCPSessions("127.0.0.1:0", "", receiverPacketLogger, latencylog.NodeRolePeer, "peer-b") - if err != nil { - t.Fatalf("ListenKCPSessions() error = %v", err) - } - - accepted := make(chan kcpAcceptResult, 1) - go func() { - session, acceptErr := listener.AcceptKCP() - if acceptErr != nil { - accepted <- kcpAcceptResult{err: acceptErr} - return - } - - conn, connErr := NewKCPConn(session, receiverOpts...) - accepted <- kcpAcceptResult{conn: conn, err: connErr} - }() - - session, err := DialKCPSession(listener.Addr().String(), "", "", senderPacketLogger, latencylog.NodeRolePeer, "peer-a") - if err != nil { - _ = packetConn.Close() - _ = listener.Close() - t.Fatalf("DialKCPSession() error = %v", err) - } - - sender, err := NewKCPConn(session, senderOpts...) - if err != nil { - _ = session.Close() - _ = packetConn.Close() - _ = listener.Close() - t.Fatalf("NewKCPConn(sender) error = %v", err) - } - - cleanup := func() { - _ = sender.Close() - select { - case result := <-accepted: - if result.conn != nil { - _ = result.conn.Close() - } - default: - } - _ = listener.Close() - _ = packetConn.Close() - } - - return sender, accepted, cleanup -} - -func awaitAcceptedKCPConn(t *testing.T, accepted <-chan kcpAcceptResult) *KCPConn { - t.Helper() - - result := <-accepted - if result.err != nil { - t.Fatalf("AcceptKCP() error = %v", result.err) - } - if result.conn == nil { - t.Fatal("accepted KCP conn = nil") - } - return result.conn -} - -func waitForKCPPacketRecords(t *testing.T, logger *recordingKCPPacketDebugLogger, condition func([]KCPPacketDebugRecord) bool, description string) { - t.Helper() - - deadline := time.Now().Add(2 * time.Second) - for time.Now().Before(deadline) { - records := logger.Records() - if condition(records) { - return - } - time.Sleep(10 * time.Millisecond) - } - - t.Fatalf("timed out waiting for %s", description) -} diff --git a/cmd/internal/transport/tcp.go b/cmd/internal/transport/tcp.go deleted file mode 100644 index 6073ca6..0000000 --- a/cmd/internal/transport/tcp.go +++ /dev/null @@ -1,160 +0,0 @@ -package transport - -import ( - "errors" - "fmt" - "io" - "net" - "sync" - "syscall" - "time" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -// TCPConn 是对单条活跃 TCP 连接的轻量封装。 -// 它负责把协议层的单条消息读写，提升为可复用的收发接口。 -type TCPConn struct { - conn net.Conn - raw syscall.RawConn // 连接对应的底层 syscall 句柄，用于 Linux socket timestamping 收发。 - - logger latencylog.Logger - txTimestampDebugLogger TXTimestampDebugLogger - nodeRole string // 日志中记录的节点角色，例如 "server" 或 "peer" - nodeID string // 日志中记录的节点 ID，例如 peer 的 ID 或 server 的 "hub" - writeMu sync.Mutex // 保护 Send 方法的互斥锁，确保同一时刻只有一条完整协议消息被写入连接，防止多条消息字节交叉 - txWriteSeq uint32 // Linux TX timestamp OPT_ID_TCP 的本地镜像，按成功写出的字节推进。 - closeOnce sync.Once // 保护 Close 方法的 sync.Once，确保连接只被关闭一次 - closeErr error // 连接关闭时的错误，如果连接成功关闭则为 nil，重复调用 Close 时会返回同样的错误 -} - -// Option 用于为 TCPConn 注入可选行为，例如时延日志。 -type Option func(*TCPConn) - -// WithLogger 为连接发送路径注入业务消息日志上下文。 -func WithLogger(logger latencylog.Logger, nodeRole, nodeID string) Option { - return func(conn *TCPConn) { - conn.logger = logger - conn.nodeRole = nodeRole - conn.nodeID = nodeID - } -} - -// WithTXTimestampDebugLogger 为连接注入可选的 TX errqueue 调试日志器。 -func WithTXTimestampDebugLogger(logger TXTimestampDebugLogger) Option { - return func(conn *TCPConn) { - conn.txTimestampDebugLogger = logger - } -} - -// NewTCPConn 用已有的 net.Conn 创建 transport 连接封装。 -func NewTCPConn(conn net.Conn, opts ...Option) (*TCPConn, error) { - tcpConn := &TCPConn{ - conn: conn, - logger: latencylog.NoopLogger{}, - } - - for _, opt := range opts { - opt(tcpConn) - } - - if tcpConn.logger == nil { - tcpConn.logger = latencylog.NoopLogger{} - } - - if err := tcpConn.initLinuxTimestamping(); err != nil { - return nil, err - } - - return tcpConn, nil -} - -// Send 将一条协议消息完整写入底层连接。 -// 多个 goroutine 可以并发调用，内部会串行化写入。 -func (c *TCPConn) Send(msg protocol.Message) error { - c.writeMu.Lock() //“同一时刻只能有一条完整协议消息往连接里写，防止多条消息字节交叉 - defer c.writeMu.Unlock() - latencylog.LogMessageEvent(c.logger, c.nodeRole, c.nodeID, latencylog.EventSendHandoffBegin, msg) - - if err := c.sendMessageLinux(msg); err != nil { - return fmt.Errorf("transport: send message: %w", err) - } - //记录发送完成的时延日志事件，事件类型为 EventSendHandoffEnd，包含消息的基本信息（类型、ID、来源、目标）。 - latencylog.LogMessageEvent(c.logger, c.nodeRole, c.nodeID, latencylog.EventSendHandoffEnd, msg) - - return nil -} - -// Receive 从底层连接读取一条完整协议消息。 -// 同一条连接应只由单个 reader 持续调用该方法。 -func (c *TCPConn) Receive() (protocol.Message, error) { - msg, err := c.receiveMessageLinux() - if err != nil { - return protocol.Message{}, fmt.Errorf("transport: receive message: %w", err) - } - - return msg, nil -} - -// ReceiveLoop 持续读取消息并交给 handler 处理。 -// 读取错误、handler 错误或连接关闭都会结束循环，并关闭连接。 -func (c *TCPConn) ReceiveLoop(handler func(protocol.Message) error) error { - for { - msg, err := c.Receive() - if err != nil { - _ = c.Close() - return fmt.Errorf("transport: receive loop read: %w", err) - } - - if err := handler(msg); err != nil { - _ = c.Close() - return fmt.Errorf("transport: receive loop handler: %w", err) - } - } -} - -// CloseGracefully 在支持 half-close 的连接上先关闭写方向，给对端留出读取最终响应的机会， -// 然后在短暂等待后再彻底关闭连接。 -func (c *TCPConn) CloseGracefully(drainTimeout time.Duration) error { - if closeWriter, ok := c.conn.(interface{ CloseWrite() error }); ok { - if err := closeWriter.CloseWrite(); err != nil && !errors.Is(err, net.ErrClosed) { - return c.Close() - } - - if drainTimeout > 0 { - _ = c.conn.SetReadDeadline(time.Now().Add(drainTimeout)) - defer func() { - _ = c.conn.SetReadDeadline(time.Time{}) - }() - - var buf [256]byte - for { - _, err := c.conn.Read(buf[:]) - switch { - case err == nil: - continue - case errors.Is(err, io.EOF), errors.Is(err, net.ErrClosed): - return c.Close() - default: - var netErr net.Error - if errors.As(err, &netErr) && netErr.Timeout() { - return c.Close() - } - return c.Close() - } - } - } - } - - return c.Close() -} - -// Close 关闭底层连接，并保证重复调用是安全的。 -func (c *TCPConn) Close() error { - c.closeOnce.Do(func() { - c.closeErr = c.conn.Close() - }) - - return c.closeErr -} diff --git a/cmd/internal/transport/tcp_linux.go b/cmd/internal/transport/tcp_linux.go deleted file mode 100644 index 5641cc8..0000000 --- a/cmd/internal/transport/tcp_linux.go +++ /dev/null @@ -1,749 +0,0 @@ -//go:build linux - -package transport - -import ( - "encoding/binary" - "errors" - "fmt" - "io" - "syscall" - "time" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -const ( - linuxTimestampControlBufferSize = 2048 // 控制消息缓冲区。 - linuxSocketWriteBufferSize = 10 * 1024 * 1024 // 请求把 socket 发送缓冲区调到 10 MiB。 - linuxTXTimestampWaitTimeout = 5000 * time.Millisecond // 等待 TX 时间戳的上限。 - linuxTXTimestampPollInterval = time.Millisecond // 轮询 errqueue 的间隔。 - linuxDataPollInterval = time.Millisecond // 轮询普通收发的间隔。 - - linuxSOTimestampingNew = 0x41 - linuxSCMTimestampingNew = linuxSOTimestampingNew - linuxSOEEOriginTimestamping = 4 // timestamping errqueue 事件。 - linuxSCMTstampSnd = 0 // 对应 A_TX_SOFTWARE。 - linuxSCMTstampSched = 1 // 对应 A_TX_SCHED。 - - linuxSOFTimestampingTXSoftware = 1 << 1 // 打开 TX software timestamp。 - linuxSOFTimestampingRXSoftware = 1 << 3 // 打开 RX software timestamp。 - linuxSOFTimestampingSoftware = 1 << 4 // software timestamp 总开关。 - linuxSOFTimestampingOptID = 1 << 7 // 给时间戳关联 ID。 - linuxSOFTimestampingTXSched = 1 << 8 // 打开 TX sched timestamp。 - linuxSOFTimestampingOptTSONLY = 1 << 11 // 只回时间戳。 - linuxSOFTimestampingOptIDTCP = 1 << 16 // 让 TCP 也带 timestamp ID。 - - linuxTXTimestampPhasePreSendDrain = "pre_send_drain" - linuxTXTimestampPhasePostSendCollect = "post_send_collect" - linuxTXTimestampPhasePostSelectDrain = "post_select_drain" -) - -type txSendChunk struct { - SendCallIndex int - FrameOffsetStart int - FrameOffsetEnd int // 包含当前 sendmsg 成功写出的最后一个 frame 字节偏移。 - BytesWritten int - ExpectedTXID uint32 -} - -type txTimestampEvent struct { - EventName string - TSUnixNano int64 - EEInfo uint32 - EEData uint32 -} - -type observedTXTimestampEvent struct { - Phase string - ReadIndex int - Event txTimestampEvent -} - -type txTimestampSelection struct { - SelectedID uint32 - Timestamps map[string]int64 - HasEvent bool -} - -type socketExtendedErrInfo struct { - Info uint32 - Data uint32 -} - -// 拿到底层 fd，并打开 Linux timestamping。 -func (c *TCPConn) initLinuxTimestamping() error { - sysConn, ok := c.conn.(interface { - SyscallConn() (syscall.RawConn, error) - }) - if !ok { - return fmt.Errorf("transport: connection does not support SyscallConn") - } - - rawConn, err := sysConn.SyscallConn() - if err != nil || rawConn == nil { - if err != nil { - return fmt.Errorf("transport: get syscall conn: %w", err) - } - return fmt.Errorf("transport: missing syscall conn") - } - - if err := configureLinuxSocketWriteBuffer(rawConn); err != nil { - return fmt.Errorf("transport: configure socket write buffer: %w", err) - } - - //socket是否可以成功打开 timestamping 取决于内核版本和配置，尝试多个 flag 组合直到成功或遇到非 EINVAL 错误。 - if err := enableLinuxTimestamping(rawConn); err != nil { - return fmt.Errorf("transport: enable linux timestamping: %w", err) - } - //成功打开 timestamping 后，rawConn 就可以用来收 TX/RX 时间戳了。 - c.raw = rawConn - return nil -} - -// 设置 TCP缓冲区buffer size -func configureLinuxSocketWriteBuffer(rawConn syscall.RawConn) error { - var lastErr error - - err := rawConn.Control(func(fd uintptr) { - lastErr = syscall.SetsockoptInt(int(fd), syscall.SOL_SOCKET, syscall.SO_SNDBUF, linuxSocketWriteBufferSize) - }) - if err != nil { - return err - } - - return lastErr -} - -// 给 socket开权限打开TX software timestamping。 -func enableLinuxTimestamping(rawConn syscall.RawConn) error { - flagCandidates := []int{ //不同linux版本可能支持不同的 flag 组合，尝试多个组合直到成功。 - linuxSOFTimestampingTXSched | - linuxSOFTimestampingTXSoftware | - linuxSOFTimestampingRXSoftware | - linuxSOFTimestampingSoftware | - linuxSOFTimestampingOptID | //TCP 协议栈给每个时间戳生成一个序列号 - linuxSOFTimestampingOptIDTCP | - linuxSOFTimestampingOptTSONLY, - linuxSOFTimestampingTXSched | - linuxSOFTimestampingTXSoftware | - linuxSOFTimestampingRXSoftware | - linuxSOFTimestampingSoftware | - linuxSOFTimestampingOptID | - linuxSOFTimestampingOptTSONLY, - linuxSOFTimestampingTXSched | - linuxSOFTimestampingTXSoftware | - linuxSOFTimestampingRXSoftware | - linuxSOFTimestampingSoftware | - linuxSOFTimestampingOptTSONLY, - } - - var lastErr error - for _, flags := range flagCandidates { //尝试不同的 flag 组合，直到成功或遇到非 EINVAL 错误。 - // 内核根据 fd 找到对应的内存结构体（Socket 缓冲区） - err := rawConn.Control(func(fd uintptr) { //Control 方法保证在回调里 fd 是有效的，可以安全地调用 syscall.SetsockoptInt。 - lastErr = syscall.SetsockoptInt(int(fd), syscall.SOL_SOCKET, linuxSOTimestampingNew, flags) - }) - if err != nil { - return err - } - if lastErr == nil { - return nil - } - if !errors.Is(lastErr, syscall.EINVAL) { - return lastErr - } - } - - return lastErr -} - -// sendMessageLinux 编码消息、写完整帧，再记录 TX 时间戳。 -func (c *TCPConn) sendMessageLinux(msg protocol.Message) error { - payload, err := protocol.EncodeMessage(msg) - if err != nil { - return fmt.Errorf("protocol: encode message: %w", err) - } - - //编码后的消息 payload 前面加 4 字节长度，构成完整帧。 - frame := make([]byte, 4+len(payload)) - binary.BigEndian.PutUint32(frame[:4], uint32(len(payload))) - copy(frame[4:], payload) - - readIndex := 0 - c.drainPendingTXTimestampEvents(msg, nil, linuxTXTimestampPhasePreSendDrain, &readIndex) - - chunks, err := c.writeFrameLinux(frame, msg) - if err != nil { - return fmt.Errorf("protocol: write frame: %w", err) - } - //记录发送延时日志 - c.logTXTimestampEvents(msg, chunks, &readIndex) - return nil -} - -// writeFrameLinux 用 sendmsg 写完整帧。 -func (c *TCPConn) writeFrameLinux(frame []byte, msg protocol.Message) ([]txSendChunk, error) { - written := 0 - sendCallIndex := 0 - chunks := make([]txSendChunk, 0, 1) - - for written < len(frame) { - n, err := c.sendmsgDataOnce(frame[written:]) - switch { - case err == nil: - if n <= 0 { - return nil, io.ErrShortWrite - } - chunk := txSendChunk{ - SendCallIndex: sendCallIndex, - FrameOffsetStart: written, - FrameOffsetEnd: written + n - 1, - BytesWritten: n, - ExpectedTXID: c.txWriteSeq + uint32(n) - 1, - } - c.txWriteSeq += uint32(n) - chunks = append(chunks, chunk) - c.logTXSendChunkDebugRecord(msg, chunk) - sendCallIndex++ - written += n - case isWouldBlock(err): - time.Sleep(linuxDataPollInterval) - default: - return nil, err - } - } - - return chunks, nil -} - -// 把 A_TX_SCHED / A_TX_SOFTWARE 写入日志。(发送过程中） -func (c *TCPConn) logTXTimestampEvents(msg protocol.Message, chunks []txSendChunk, readIndex *int) { - timestamps := c.collectTXTimestampEvents(msg, chunks, readIndex) - - if ts, ok := timestamps[latencylog.EventATXSched]; ok { - latencylog.LogMessageEventAt(c.logger, c.nodeRole, c.nodeID, latencylog.EventATXSched, ts, msg) - } - if ts, ok := timestamps[latencylog.EventATXSoftware]; ok { - latencylog.LogMessageEventAt(c.logger, c.nodeRole, c.nodeID, latencylog.EventATXSoftware, ts, msg) - } -} - -// 在 errqueue 里等两类 TX 时间戳。 -func (c *TCPConn) collectTXTimestampEvents(msg protocol.Message, chunks []txSendChunk, readIndex *int) map[string]int64 { - deadline := time.Now().Add(linuxTXTimestampWaitTimeout) - observed := make([]observedTXTimestampEvent, 0, 4) - finalChunk, hasFinalChunk := finalTXSendChunk(chunks) - - // 轮询 errqueue，优先等待本次消息最后一个 sendmsg chunk 的 sched/software 两类时间戳。 - for time.Now().Before(deadline) { - event, err := c.recvTXTimestampOnce() - if err != nil { - if isWouldBlock(err) { - time.Sleep(linuxTXTimestampPollInterval) - continue - } - break - } - if event.EventName == "" || event.TSUnixNano <= 0 { - continue - } - observed = append(observed, observedTXTimestampEvent{ - Phase: linuxTXTimestampPhasePostSendCollect, - ReadIndex: *readIndex, - Event: event, - }) - *readIndex = *readIndex + 1 - - selection := selectTXTimestampEvents(observed, finalChunk.ExpectedTXID, hasFinalChunk) - if hasFinalChunk && selection.HasEvent && selection.SelectedID == finalChunk.ExpectedTXID && hasCompleteTXTimestampPair(selection.Timestamps) { - break - } - } - - selection := selectTXTimestampEvents(observed, finalChunk.ExpectedTXID, hasFinalChunk) - c.logObservedTXTimestampEvents(msg, chunks, observed, selection) - c.drainPendingTXTimestampEvents(msg, chunks, linuxTXTimestampPhasePostSelectDrain, readIndex) - return selection.Timestamps -} - -// recvTXTimestampOnce 从 errqueue 读一次时间戳事件。 -func (c *TCPConn) recvTXTimestampOnce() (txTimestampEvent, error) { - var ( - event txTimestampEvent - opErr error - ) - - err := c.raw.Control(func(fd uintptr) { - //设置足够大的 oob buffer 来接收控制消息，调用 recvmsg 从 errqueue 读一条消息。 - oob := make([]byte, linuxTimestampControlBufferSize) - //recvmsg 的 flags 里必须带 MSG_ERRQUEUE，才能从 errqueue 里读消息，非阻塞模式下如果没有消息可读会返回 EAGAIN。 - _, oobn, _, _, recvErr := syscall.Recvmsg(int(fd), nil, oob, syscall.MSG_ERRQUEUE|syscall.MSG_DONTWAIT) - if recvErr != nil { - opErr = recvErr - return - } - //解析控制消息，看看是不是我们关心的 TX 时间戳事件，如果是就拿到事件名和时间戳。 - event, _ = parseTXTimestampControlMessages(oob[:oobn]) - }) - if err != nil { - return txTimestampEvent{}, err - } - if opErr != nil { - return txTimestampEvent{}, opErr - } - - return event, nil -} - -// 把底层时间戳映射成日志事件名。 -func parseTXTimestampControlMessages(oob []byte) (txTimestampEvent, bool) { - if len(oob) == 0 { - return txTimestampEvent{}, false - } - //解析控制消息，看看是不是我们关心的 TX 时间戳事件，如果是就拿到事件名和时间戳。 - controlMessages, err := syscall.ParseSocketControlMessage(oob) - if err != nil { - return txTimestampEvent{}, false - } - - return parseTXTimestampSocketControlMessages(controlMessages) -} - -func parseTXTimestampSocketControlMessages(controlMessages []syscall.SocketControlMessage) (txTimestampEvent, bool) { - var ( - event txTimestampEvent - hasTS bool - hasKind bool - ) - - for _, controlMessage := range controlMessages { - switch { - case controlMessage.Header.Level == syscall.SOL_SOCKET && controlMessage.Header.Type == linuxSCMTimestampingNew: - if ts := parseSCMTimestampingData(controlMessage.Data); ts > 0 { - event.TSUnixNano = ts - hasTS = true - } - case isSocketExtendedErr(controlMessage): //判断时间戳是否进入了errqueue， - if info, ok := parseSocketExtendedErrInfo(controlMessage.Data); ok { - event.EEInfo = info.Info - event.EEData = info.Data - event.EventName = mapLinuxTXTimestampEventName(info.Info) - hasKind = true - } - } - } - - if !hasTS || !hasKind || event.EventName == "" { - return txTimestampEvent{}, false - } - - return event, true -} - -// 判断控制消息是否来自 socket extended err。 -// 内核产生的时间戳并不会混合在普通的数据流里，而是被包装成一种特殊的“错误消息”丢进 Error Queue。 -func isSocketExtendedErr(controlMessage syscall.SocketControlMessage) bool { - switch { - case controlMessage.Header.Level == syscall.SOL_IP && controlMessage.Header.Type == syscall.IP_RECVERR: - return true - case controlMessage.Header.Level == syscall.SOL_IPV6 && controlMessage.Header.Type == syscall.IPV6_RECVERR: - return true - default: - return false - } -} - -// 从 socket extended err 的数据里取 origin timestamping 信息。 -func parseSocketExtendedErrInfo(data []byte) (socketExtendedErrInfo, bool) { - if len(data) < 16 { - return socketExtendedErrInfo{}, false - } - if data[4] != linuxSOEEOriginTimestamping { - return socketExtendedErrInfo{}, false - } - - return socketExtendedErrInfo{ - Info: binary.NativeEndian.Uint32(data[8:12]), - Data: binary.NativeEndian.Uint32(data[12:16]), - }, true -} - -func mapLinuxTXTimestampEventName(tsKind uint32) string { - switch tsKind { - case linuxSCMTstampSched: - return latencylog.EventATXSched - case linuxSCMTstampSnd: - return latencylog.EventATXSoftware - default: - return fmt.Sprintf("TX_TIMESTAMP_KIND_%d", tsKind) - } -} - -func finalTXSendChunk(chunks []txSendChunk) (txSendChunk, bool) { - if len(chunks) == 0 { - return txSendChunk{}, false - } - - return chunks[len(chunks)-1], true -} - -func isBusinessTXTimestampEventName(eventName string) bool { - return eventName == latencylog.EventATXSched || eventName == latencylog.EventATXSoftware -} - -func hasCompleteTXTimestampPair(timestamps map[string]int64) bool { - if len(timestamps) == 0 { - return false - } - - return timestamps[latencylog.EventATXSched] > 0 && timestamps[latencylog.EventATXSoftware] > 0 -} - -func selectTXTimestampEvents(events []observedTXTimestampEvent, expectedFinalTXID uint32, hasExpectedFinalTXID bool) txTimestampSelection { - byID := make(map[uint32]map[string]int64) - selection := txTimestampSelection{ - Timestamps: make(map[string]int64, 2), - } - - var ( - highestID uint32 - haveHighest bool - ) - for _, observed := range events { - event := observed.Event - selection.HasEvent = true - if !haveHighest || event.EEData > highestID { - highestID = event.EEData - haveHighest = true - } - - if !isBusinessTXTimestampEventName(event.EventName) { - continue - } - if _, ok := byID[event.EEData]; !ok { - byID[event.EEData] = make(map[string]int64, 2) - } - if existing, exists := byID[event.EEData][event.EventName]; !exists || event.TSUnixNano < existing { - byID[event.EEData][event.EventName] = event.TSUnixNano - } - } - - if !selection.HasEvent { - return selection - } - - switch { - case hasExpectedFinalTXID && len(byID[expectedFinalTXID]) > 0: - selection.SelectedID = expectedFinalTXID - selection.Timestamps = copyTXTimestampMap(byID[expectedFinalTXID]) - case len(byID[highestID]) > 0: - selection.SelectedID = highestID - selection.Timestamps = copyTXTimestampMap(byID[highestID]) - default: - selection.SelectedID = highestID - } - - return selection -} - -func copyTXTimestampMap(src map[string]int64) map[string]int64 { - dst := make(map[string]int64, len(src)) - for key, value := range src { - dst[key] = value - } - return dst -} - -func (c *TCPConn) drainPendingTXTimestampEvents(msg protocol.Message, chunks []txSendChunk, phase string, readIndex *int) { - for { - event, err := c.recvTXTimestampOnce() - if err != nil { - if isWouldBlock(err) { - return - } - return - } - if event.EventName == "" || event.TSUnixNano <= 0 { - continue - } - c.logTXErrqueueDebugRecord(msg, chunks, observedTXTimestampEvent{ - Phase: phase, - ReadIndex: *readIndex, - Event: event, - }, false) - *readIndex = *readIndex + 1 - } -} - -func (c *TCPConn) logObservedTXTimestampEvents(msg protocol.Message, chunks []txSendChunk, observed []observedTXTimestampEvent, selection txTimestampSelection) { - if len(observed) == 0 { - return - } - - for _, entry := range observed { - selected := selection.HasEvent && - entry.Event.EEData == selection.SelectedID && - isBusinessTXTimestampEventName(entry.Event.EventName) && - selection.Timestamps[entry.Event.EventName] == entry.Event.TSUnixNano - c.logTXErrqueueDebugRecord(msg, chunks, entry, selected) - } -} - -func (c *TCPConn) logTXSendChunkDebugRecord(msg protocol.Message, chunk txSendChunk) { - if c.txTimestampDebugLogger == nil { - return - } - - sendCallIndex := chunk.SendCallIndex - frameOffsetStart := chunk.FrameOffsetStart - frameOffsetEnd := chunk.FrameOffsetEnd - bytesWritten := chunk.BytesWritten - expectedTXID := chunk.ExpectedTXID - - record := c.newTXTimestampDebugRecord(msg) - record.RecordType = txTimestampDebugRecordTypeSendChunk - record.SendCallIndex = &sendCallIndex - record.FrameOffsetStart = &frameOffsetStart - record.FrameOffsetEnd = &frameOffsetEnd - record.BytesWritten = &bytesWritten - record.ExpectedTXID = &expectedTXID - c.logTXTimestampDebugRecord(record) -} - -func (c *TCPConn) logTXErrqueueDebugRecord(msg protocol.Message, chunks []txSendChunk, observed observedTXTimestampEvent, selected bool) { - if c.txTimestampDebugLogger == nil { - return - } - - readIndex := observed.ReadIndex - tsUnixNano := observed.Event.TSUnixNano - eeInfo := observed.Event.EEInfo - eeData := observed.Event.EEData - selectedForLatency := selected - - record := c.newTXTimestampDebugRecord(msg) - record.RecordType = txTimestampDebugRecordTypeErrqueueEvent - record.Phase = observed.Phase - record.ReadIndex = &readIndex - record.EventName = observed.Event.EventName - record.TSUnixNano = &tsUnixNano - record.EEInfo = &eeInfo - record.EEData = &eeData - record.SelectedForLatency = &selectedForLatency - if matchedSendCallIndex, ok := matchTXTimestampEventToSendChunk(observed.Event.EEData, chunks); ok { - record.MatchedSendCallIndex = &matchedSendCallIndex - } - c.logTXTimestampDebugRecord(record) -} - -func matchTXTimestampEventToSendChunk(txID uint32, chunks []txSendChunk) (int, bool) { - for _, chunk := range chunks { - if chunk.ExpectedTXID == txID { - return chunk.SendCallIndex, true - } - } - - return 0, false -} - -func (c *TCPConn) newTXTimestampDebugRecord(msg protocol.Message) TXTimestampDebugRecord { - return TXTimestampDebugRecord{ - NodeRole: c.nodeRole, - NodeID: c.nodeID, - MessageType: msg.Type, - MessageID: msg.ID, - From: msg.From, - To: msg.To, - FileName: msg.FileName, - BodySize: len(msg.Body), - } -} - -func (c *TCPConn) logTXTimestampDebugRecord(record TXTimestampDebugRecord) { - if c.txTimestampDebugLogger == nil { - return - } - - _ = c.txTimestampDebugLogger.LogTXTimestampDebugRecord(record) -} - -// 读一条完整消息，并记录 B_RX_SOFTWARE。 -func (c *TCPConn) receiveMessageLinux() (protocol.Message, error) { - payload, rxTimestamp, err := c.readFrameLinux() - if err != nil { - return protocol.Message{}, fmt.Errorf("protocol: read frame: %w", err) - } - - msg, err := protocol.DecodeMessage(payload) - if err != nil { - return protocol.Message{}, fmt.Errorf("protocol: decode message: %w", err) - } - - if rxTimestamp > 0 { - latencylog.LogMessageEventAt(c.logger, c.nodeRole, c.nodeID, latencylog.EventBRXSoftware, rxTimestamp, msg) - } - - return msg, nil -} - -// readFrameLinux 先读 4 字节长度，再读整条 payload。 -func (c *TCPConn) readFrameLinux() ([]byte, int64, error) { - var frameHeader [4]byte - rxTimestamp, err := c.readFullLinux(frameHeader[:]) - if err != nil { - return nil, rxTimestamp, err - } - - size := binary.BigEndian.Uint32(frameHeader[:]) - switch { - case size == 0: - return nil, rxTimestamp, protocol.ErrInvalidFrameLength - case size > protocol.MaxFrameSize: - return nil, rxTimestamp, protocol.ErrFrameTooLarge - } - - payload := make([]byte, int(size)) - bodyTimestamp, err := c.readFullLinux(payload) - if rxTimestamp == 0 { - rxTimestamp = bodyTimestamp - } - if err != nil { - return nil, rxTimestamp, err - } - - return payload, rxTimestamp, nil -} - -// 读满 buf，并保留首个 RX_SOFTWARE（返回进入tcp协议栈的时间戳）。 -func (c *TCPConn) readFullLinux(buf []byte) (int64, error) { - if len(buf) == 0 { - return 0, nil - } - - var ( - offset int - firstRXTime int64 - ) - - for offset < len(buf) { - n, rxTimestamp, err := c.recvmsgLinux(buf[offset:]) - if firstRXTime == 0 && rxTimestamp > 0 { - firstRXTime = rxTimestamp - } - if err != nil { - if errors.Is(err, io.EOF) && offset > 0 { - return firstRXTime, io.ErrUnexpectedEOF - } - return firstRXTime, err - } - - offset += n - } - - return firstRXTime, nil -} - -// recvmsgLinux 用 recvmsg 同时读取数据和控制消息。 -func (c *TCPConn) recvmsgLinux(buf []byte) (int, int64, error) { - for { - n, rxTimeNS, err := c.recvmsgDataOnce(buf) - switch { - case err == nil: - if n == 0 { - return 0, 0, io.EOF - } - return n, rxTimeNS, nil - case isWouldBlock(err): - time.Sleep(linuxDataPollInterval) - default: - return 0, 0, err - } - } -} - -// 从控制消息里取 RX_SOFTWARE。 -func parseRXTimestampControlMessages(oob []byte) int64 { - if len(oob) == 0 { - return 0 - } - - controlMessages, err := syscall.ParseSocketControlMessage(oob) - if err != nil { - return 0 - } - - for _, controlMessage := range controlMessages { - if controlMessage.Header.Level != syscall.SOL_SOCKET || controlMessage.Header.Type != linuxSCMTimestampingNew { - continue - } - - if ts := parseSCMTimestampingData(controlMessage.Data); ts > 0 { - return ts - } - } - - return 0 -} - -// 取第一个非零 timespec。 -func parseSCMTimestampingData(data []byte) int64 { - const timespec64Size = 16 - - for offset := 0; offset+timespec64Size <= len(data); offset += timespec64Size { - sec := int64(binary.NativeEndian.Uint64(data[offset : offset+8])) - nsec := int64(binary.NativeEndian.Uint64(data[offset+8 : offset+16])) - if sec == 0 && nsec == 0 { - continue - } - return sec*int64(time.Second) + nsec - } - - return 0 -} - -// 判断错误是否是 EAGAIN 或 EWOULDBLOCK。 -func isWouldBlock(err error) bool { - return errors.Is(err, syscall.EAGAIN) || errors.Is(err, syscall.EWOULDBLOCK) -} - -func (c *TCPConn) sendmsgDataOnce(buf []byte) (int, error) { - var ( - n int - opErr error - ) - - err := c.raw.Control(func(fd uintptr) { - n, opErr = syscall.SendmsgN(int(fd), buf, nil, nil, 0) - }) - if err != nil { - return 0, err - } - - return n, opErr -} - -func (c *TCPConn) recvmsgDataOnce(buf []byte) (int, int64, error) { - var ( - n int - rxTimeNS int64 - opErr error - ) - - err := c.raw.Control(func(fd uintptr) { - oob := make([]byte, linuxTimestampControlBufferSize) - readN, oobN, _, _, recvErr := syscall.Recvmsg(int(fd), buf, oob, 0) - if recvErr != nil { - opErr = recvErr - return - } - n = readN - rxTimeNS = parseRXTimestampControlMessages(oob[:oobN]) - }) - if err != nil { - return 0, 0, err - } - - return n, rxTimeNS, opErr -} diff --git a/cmd/internal/transport/tcp_linux_test.go b/cmd/internal/transport/tcp_linux_test.go deleted file mode 100644 index 114f685..0000000 --- a/cmd/internal/transport/tcp_linux_test.go +++ /dev/null @@ -1,442 +0,0 @@ -//go:build linux - -package transport - -import ( - "encoding/binary" - "net" - "reflect" - "syscall" - "testing" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -func TestLinuxTimestampingRecordsKernelEvents(t *testing.T) { - tests := []struct { - name string - msg protocol.Message - }{ - { - name: "text", - msg: protocol.Message{ - Type: protocol.MessageTypeText, - ID: 41, - From: "peer-a", - To: "peer-b", - Body: []byte("hello over tcp"), - }, - }, - { - name: "file", - msg: protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 42, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0x00, 0x01, 0x02, 0xff}, - }, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - clientConn, serverConn := newTCPPair(t) - - senderLogger := &recordingLogger{} - receiverLogger := &recordingLogger{} - sender, err := NewTCPConn( - clientConn, - WithLogger(senderLogger, latencylog.NodeRolePeer, "peer-a"), - ) - if err != nil { - t.Fatalf("NewTCPConn(sender) error = %v", err) - } - receiver, err := NewTCPConn( - serverConn, - WithLogger(receiverLogger, latencylog.NodeRolePeer, "peer-b"), - ) - if err != nil { - t.Fatalf("NewTCPConn(receiver) error = %v", err) - } - t.Cleanup(func() { - _ = sender.Close() - _ = receiver.Close() - }) - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(tt.msg) - }() - - got, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v", err) - } - if !reflect.DeepEqual(got, tt.msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, tt.msg) - } - - assertHasEvent(t, senderLogger.Events(), latencylog.EventATXSched, tt.msg.ID) - assertHasEvent(t, senderLogger.Events(), latencylog.EventATXSoftware, tt.msg.ID) - assertHasEvent(t, receiverLogger.Events(), latencylog.EventBRXSoftware, tt.msg.ID) - }) - } -} - -func TestParseTXTimestampSocketControlMessagesExtractsEventKindAndID(t *testing.T) { - tests := []struct { - name string - kind uint32 - wantEvent string - }{ - { - name: "sched", - kind: linuxSCMTstampSched, - wantEvent: latencylog.EventATXSched, - }, - { - name: "software", - kind: linuxSCMTstampSnd, - wantEvent: latencylog.EventATXSoftware, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - controlMessages := []syscall.SocketControlMessage{ - makeTimestampSocketControlMessage(1_700_000_000_123_456_789), - makeSocketExtendedErrControlMessage(tt.kind, 42), - } - - event, ok := parseTXTimestampSocketControlMessages(controlMessages) - if !ok { - t.Fatal("parseTXTimestampSocketControlMessages() ok = false, want true") - } - if event.EventName != tt.wantEvent { - t.Fatalf("event name = %q, want %q", event.EventName, tt.wantEvent) - } - if event.TSUnixNano != 1_700_000_000_123_456_789 { - t.Fatalf("timestamp = %d, want %d", event.TSUnixNano, int64(1_700_000_000_123_456_789)) - } - if event.EEInfo != tt.kind { - t.Fatalf("ee_info = %d, want %d", event.EEInfo, tt.kind) - } - if event.EEData != 42 { - t.Fatalf("ee_data = %d, want 42", event.EEData) - } - }) - } -} - -func TestSelectTXTimestampEventsPrefersExactFinalChunkID(t *testing.T) { - events := []observedTXTimestampEvent{ - {Event: txTimestampEvent{EventName: latencylog.EventATXSched, TSUnixNano: 100, EEData: 7}}, - {Event: txTimestampEvent{EventName: latencylog.EventATXSoftware, TSUnixNano: 110, EEData: 7}}, - {Event: txTimestampEvent{EventName: latencylog.EventATXSched, TSUnixNano: 200, EEData: 9}}, - {Event: txTimestampEvent{EventName: latencylog.EventATXSoftware, TSUnixNano: 210, EEData: 9}}, - } - - selection := selectTXTimestampEvents(events, 9, true) - if !selection.HasEvent { - t.Fatal("selection.HasEvent = false, want true") - } - if selection.SelectedID != 9 { - t.Fatalf("SelectedID = %d, want 9", selection.SelectedID) - } - if got := selection.Timestamps[latencylog.EventATXSched]; got != 200 { - t.Fatalf("selected sched = %d, want 200", got) - } - if got := selection.Timestamps[latencylog.EventATXSoftware]; got != 210 { - t.Fatalf("selected software = %d, want 210", got) - } -} - -func TestSelectTXTimestampEventsFallsBackToHighestObservedID(t *testing.T) { - events := []observedTXTimestampEvent{ - {Event: txTimestampEvent{EventName: latencylog.EventATXSched, TSUnixNano: 100, EEData: 11}}, - {Event: txTimestampEvent{EventName: latencylog.EventATXSoftware, TSUnixNano: 120, EEData: 11}}, - {Event: txTimestampEvent{EventName: latencylog.EventATXSched, TSUnixNano: 200, EEData: 15}}, - {Event: txTimestampEvent{EventName: latencylog.EventATXSoftware, TSUnixNano: 220, EEData: 15}}, - } - - selection := selectTXTimestampEvents(events, 20, true) - if !selection.HasEvent { - t.Fatal("selection.HasEvent = false, want true") - } - if selection.SelectedID != 15 { - t.Fatalf("SelectedID = %d, want 15", selection.SelectedID) - } - if got := selection.Timestamps[latencylog.EventATXSched]; got != 200 { - t.Fatalf("selected sched = %d, want 200", got) - } - if got := selection.Timestamps[latencylog.EventATXSoftware]; got != 220 { - t.Fatalf("selected software = %d, want 220", got) - } -} - -func TestLinuxTimestampingDebugLoggerCapturesChunkAndErrqueueEvents(t *testing.T) { - clientConn, serverConn := newTCPPair(t) - setTCPWriteBuffer(t, clientConn, 10*1024*1024) - - debugLogger := &recordingTXTimestampDebugLogger{} - senderLogger := &recordingLogger{} - receiverLogger := &recordingLogger{} - sender, err := NewTCPConn( - clientConn, - WithLogger(senderLogger, latencylog.NodeRolePeer, "peer-a"), - WithTXTimestampDebugLogger(debugLogger), - ) - if err != nil { - t.Fatalf("NewTCPConn(sender) error = %v", err) - } - receiver, err := NewTCPConn( - serverConn, - WithLogger(receiverLogger, latencylog.NodeRolePeer, "peer-b"), - ) - if err != nil { - t.Fatalf("NewTCPConn(receiver) error = %v", err) - } - t.Cleanup(func() { - _ = sender.Close() - _ = receiver.Close() - }) - - body := make([]byte, 1<<20) - for i := range body { - body[i] = byte(i % 251) - } - msg := protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 99, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: body, - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - got, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v", err) - } - if !reflect.DeepEqual(got, msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, msg) - } - - assertHasEvent(t, senderLogger.Events(), latencylog.EventATXSched, msg.ID) - assertHasEvent(t, senderLogger.Events(), latencylog.EventATXSoftware, msg.ID) - assertHasEvent(t, receiverLogger.Events(), latencylog.EventBRXSoftware, msg.ID) - - sendChunkRecords := debugRecordsByType(debugLogger.Records(), txTimestampDebugRecordTypeSendChunk) - errqueueRecords := debugRecordsByType(debugLogger.Records(), txTimestampDebugRecordTypeErrqueueEvent) - if len(sendChunkRecords) == 0 { - t.Fatal("send_chunk debug records = 0, want at least 1") - } - if len(errqueueRecords) == 0 { - t.Fatal("errqueue_event debug records = 0, want at least 1") - } - - finalChunkRecord := sendChunkRecords[len(sendChunkRecords)-1] - if finalChunkRecord.ExpectedTXID == nil { - t.Fatal("final send_chunk expected_tx_id = nil, want non-nil") - } - finalExpectedTXID := *finalChunkRecord.ExpectedTXID - - selectedRecords := selectedErrqueueRecords(errqueueRecords) - if len(selectedRecords) == 0 { - t.Fatal("selected errqueue debug records = 0, want at least 1") - } - - highestObservedID := uint32(0) - haveHighestObservedID := false - haveExactFinalID := false - for _, record := range errqueueRecords { - if record.EEData == nil { - continue - } - if !haveHighestObservedID || *record.EEData > highestObservedID { - highestObservedID = *record.EEData - haveHighestObservedID = true - } - if *record.EEData == finalExpectedTXID && isBusinessTXTimestampRecord(record) { - haveExactFinalID = true - } - } - if !haveHighestObservedID { - t.Fatal("highestObservedID missing, want at least one ee_data") - } - - wantSelectedID := highestObservedID - if haveExactFinalID { - wantSelectedID = finalExpectedTXID - } - for _, record := range selectedRecords { - if record.EEData == nil { - t.Fatalf("selected record missing ee_data: %+v", record) - } - if *record.EEData != wantSelectedID { - t.Fatalf("selected ee_data = %d, want %d", *record.EEData, wantSelectedID) - } - } - - selectedByEventName := make(map[string]int64, len(selectedRecords)) - for _, record := range selectedRecords { - if record.TSUnixNano == nil { - t.Fatalf("selected record missing timestamp: %+v", record) - } - selectedByEventName[record.EventName] = *record.TSUnixNano - } - - senderEventsByName := make(map[string]int64) - for _, event := range senderLogger.Events() { - if event.MessageID != msg.ID { - continue - } - if !isBusinessTXTimestampEventName(event.Event) { - continue - } - senderEventsByName[event.Event] = event.TsUnixNano - } - - for eventName, selectedTS := range selectedByEventName { - if senderEventsByName[eventName] != selectedTS { - t.Fatalf("sender latency event %s = %d, want %d from selected debug record", eventName, senderEventsByName[eventName], selectedTS) - } - } -} - -func newTCPPair(t *testing.T) (net.Conn, net.Conn) { - t.Helper() - - listener, err := net.Listen("tcp", "127.0.0.1:0") - if err != nil { - t.Fatalf("net.Listen() error = %v", err) - } - - type acceptResult struct { - conn net.Conn - err error - } - - accepted := make(chan acceptResult, 1) - go func() { - conn, acceptErr := listener.Accept() - accepted <- acceptResult{conn: conn, err: acceptErr} - }() - - clientConn, err := net.Dial("tcp", listener.Addr().String()) - if err != nil { - _ = listener.Close() - t.Fatalf("net.Dial() error = %v", err) - } - - result := <-accepted - if err := listener.Close(); err != nil { - t.Fatalf("listener.Close() error = %v", err) - } - if result.err != nil { - _ = clientConn.Close() - t.Fatalf("listener.Accept() error = %v", result.err) - } - - return clientConn, result.conn -} - -func assertHasEvent(t *testing.T, events []latencylog.Event, wantEvent string, wantMessageID uint64) { - t.Helper() - - for _, event := range events { - if event.Event == wantEvent && event.MessageID == wantMessageID { - if event.TsUnixNano <= 0 { - t.Fatalf("event %s timestamp must be positive: %+v", wantEvent, event) - } - return - } - } - - t.Fatalf("missing event %s for message %d in %+v", wantEvent, wantMessageID, events) -} - -func makeTimestampSocketControlMessage(tsUnixNano int64) syscall.SocketControlMessage { - const timespec64Size = 16 - - data := make([]byte, timespec64Size*3) - sec := uint64(tsUnixNano / int64(1e9)) - nsec := uint64(tsUnixNano % int64(1e9)) - binary.NativeEndian.PutUint64(data[:8], sec) - binary.NativeEndian.PutUint64(data[8:16], nsec) - - return syscall.SocketControlMessage{ - Header: syscall.Cmsghdr{ - Level: syscall.SOL_SOCKET, - Type: linuxSCMTimestampingNew, - }, - Data: data, - } -} - -func makeSocketExtendedErrControlMessage(kind, id uint32) syscall.SocketControlMessage { - data := make([]byte, 16) - data[4] = linuxSOEEOriginTimestamping - binary.NativeEndian.PutUint32(data[8:12], kind) - binary.NativeEndian.PutUint32(data[12:16], id) - - return syscall.SocketControlMessage{ - Header: syscall.Cmsghdr{ - Level: syscall.SOL_IP, - Type: syscall.IP_RECVERR, - }, - Data: data, - } -} - -func setTCPWriteBuffer(t *testing.T, conn net.Conn, size int) { - t.Helper() - - tcpConn, ok := conn.(*net.TCPConn) - if !ok { - t.Fatalf("conn type %T does not support SetWriteBuffer", conn) - } - if err := tcpConn.SetWriteBuffer(size); err != nil { - t.Fatalf("SetWriteBuffer(%d) error = %v", size, err) - } -} - -func debugRecordsByType(records []TXTimestampDebugRecord, recordType string) []TXTimestampDebugRecord { - filtered := make([]TXTimestampDebugRecord, 0, len(records)) - for _, record := range records { - if record.RecordType != recordType { - continue - } - filtered = append(filtered, record) - } - return filtered -} - -func selectedErrqueueRecords(records []TXTimestampDebugRecord) []TXTimestampDebugRecord { - selected := make([]TXTimestampDebugRecord, 0, len(records)) - for _, record := range records { - if record.SelectedForLatency == nil || !*record.SelectedForLatency { - continue - } - selected = append(selected, record) - } - return selected -} - -func isBusinessTXTimestampRecord(record TXTimestampDebugRecord) bool { - return record.EventName == latencylog.EventATXSched || record.EventName == latencylog.EventATXSoftware -} diff --git a/cmd/internal/transport/tcp_test.go b/cmd/internal/transport/tcp_test.go deleted file mode 100644 index 878c759..0000000 --- a/cmd/internal/transport/tcp_test.go +++ /dev/null @@ -1,436 +0,0 @@ -package transport - -import ( - "errors" - "io" - "reflect" - "strings" - "sync" - "testing" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -type recordingLogger struct { - mu sync.Mutex - events []latencylog.Event -} - -func (l *recordingLogger) LogEvent(event latencylog.Event) error { - l.mu.Lock() - defer l.mu.Unlock() - - l.events = append(l.events, event) - return nil -} - -func (l *recordingLogger) Events() []latencylog.Event { - l.mu.Lock() - defer l.mu.Unlock() - - return append([]latencylog.Event(nil), l.events...) -} - -type failingLogger struct{} - -func (failingLogger) LogEvent(latencylog.Event) error { - return errors.New("log failed") -} - -type recordingTXTimestampDebugLogger struct { - mu sync.Mutex - records []TXTimestampDebugRecord -} - -func (l *recordingTXTimestampDebugLogger) LogTXTimestampDebugRecord(record TXTimestampDebugRecord) error { - l.mu.Lock() - defer l.mu.Unlock() - - l.records = append(l.records, record) - return nil -} - -func (l *recordingTXTimestampDebugLogger) Records() []TXTimestampDebugRecord { - l.mu.Lock() - defer l.mu.Unlock() - - return append([]TXTimestampDebugRecord(nil), l.records...) -} - -// TestSendReceiveMessage 验证 transport 可以在单条连接上正常收发 text 和 file 消息。 -func TestSendReceiveMessage(t *testing.T) { - tests := []struct { - name string - msg protocol.Message - }{ - { - name: "text", - msg: protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - }, - }, - { - name: "file", - msg: protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 2, - From: "peer-a", - To: "peer-b", - FileName: "data.bin", - Body: []byte{0x00, 0x10, 0xff}, - }, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - sender, receiver := newTransportConnPair(t, nil, nil) - //创建一个容量为1的缓冲通道sendErr，用于接收发送操作的错误结果。 - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(tt.msg) //发送消息，并将结果（错误或nil）发送到sendErr通道。 - }() - - got, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { //接受发送结果，如果发送过程中发生错误，则测试失败。 - t.Fatalf("Send() error = %v", err) - } - - if !reflect.DeepEqual(got, tt.msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, tt.msg) - } - }) - } -} - -func TestSendLogsHandoffEvents(t *testing.T) { - logger := &recordingLogger{} - sender, receiver := newTransportConnPair( - t, - []Option{WithLogger(logger, latencylog.NodeRolePeer, "peer-a")}, - nil, - ) - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 7, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - got, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v", err) - } - if !reflect.DeepEqual(got, msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, msg) - } - - events := logger.Events() - if len(events) != 4 { - t.Fatalf("event count = %d, want 4", len(events)) - } - if events[0].Event != latencylog.EventSendHandoffBegin { - t.Fatalf("first event = %q, want %q", events[0].Event, latencylog.EventSendHandoffBegin) - } - if events[1].Event != latencylog.EventATXSched { - t.Fatalf("second event = %q, want %q", events[1].Event, latencylog.EventATXSched) - } - if events[2].Event != latencylog.EventATXSoftware { - t.Fatalf("third event = %q, want %q", events[2].Event, latencylog.EventATXSoftware) - } - if events[3].Event != latencylog.EventSendHandoffEnd { - t.Fatalf("fourth event = %q, want %q", events[3].Event, latencylog.EventSendHandoffEnd) - } - for i, event := range events { - if event.MessageID != msg.ID { - t.Fatalf("event[%d] message ID = %d, want %d", i, event.MessageID, msg.ID) - } - } - if events[0].NodeRole != latencylog.NodeRolePeer || events[0].NodeID != "peer-a" { - t.Fatalf("node info = (%s,%s), want (%s,%s)", events[0].NodeRole, events[0].NodeID, latencylog.NodeRolePeer, "peer-a") - } - if events[0].TsUnixNano <= 0 || events[1].TsUnixNano <= 0 || events[2].TsUnixNano <= 0 || events[3].TsUnixNano <= 0 { - t.Fatalf("timestamps must be positive: %+v", events) - } -} - -func TestSendIgnoresLoggerFailure(t *testing.T) { - sender, receiver := newTransportConnPair( - t, - []Option{WithLogger(failingLogger{}, latencylog.NodeRolePeer, "peer-a")}, - nil, - ) - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 9, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - got, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v, want nil even if logger fails", err) - } - if !reflect.DeepEqual(got, msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, msg) - } -} - -// TestReceiveLoopDeliversMessages 验证 ReceiveLoop 会逐条交付连续到达的消息。 -func TestReceiveLoopDeliversMessages(t *testing.T) { - sender, receiver := newTransportConnPair(t, nil, nil) - - want := []protocol.Message{ - { - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - }, - { - Type: protocol.MessageTypeFile, - ID: 2, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0x01, 0x02, 0x03}, - }, - } - - var ( - mu sync.Mutex - got []protocol.Message - ) - loopErr := make(chan error, 1) - go func() { - loopErr <- receiver.ReceiveLoop(func(msg protocol.Message) error { - mu.Lock() - defer mu.Unlock() - got = append(got, msg) - return nil - }) - }() - - for _, msg := range want { - if err := sender.Send(msg); err != nil { - t.Fatalf("Send() error = %v", err) - } - } - if err := sender.Close(); err != nil { - t.Fatalf("sender.Close() error = %v", err) - } - - err := <-loopErr - if err == nil { - t.Fatal("ReceiveLoop() error = nil, want non-nil after peer close") - } - if !strings.Contains(err.Error(), "receive loop read") { - t.Fatalf("ReceiveLoop() error = %v, want read context", err) - } - - mu.Lock() - defer mu.Unlock() - if !reflect.DeepEqual(got, want) { - t.Fatalf("received messages mismatch: got %+v want %+v", got, want) - } -} - -// TestConcurrentSendKeepsMessagesIntact 验证并发发送时消息不会因为写入交叉而损坏。 -func TestConcurrentSendKeepsMessagesIntact(t *testing.T) { - sender, receiver := newTransportConnPair(t, nil, nil) - // 发送方将多条消息并发发送到接收方，接收方通过 ReceiveLoop 逐条读取并验证每条消息的完整性和正确性。 - want := []protocol.Message{ - {Type: protocol.MessageTypeText, ID: 1, From: "peer-a", To: "peer-b", Body: []byte("one")}, - {Type: protocol.MessageTypeText, ID: 2, From: "peer-a", To: "peer-b", Body: []byte("two")}, - {Type: protocol.MessageTypeText, ID: 3, From: "peer-a", To: "peer-b", Body: []byte("three")}, - {Type: protocol.MessageTypeText, ID: 4, From: "peer-a", To: "peer-b", Body: []byte("four")}, - } - - received := make(chan protocol.Message, len(want)) - readErr := make(chan error, 1) - go func() { //异步地运行一个 goroutine - for range want { - msg, err := receiver.Receive() - if err != nil { - readErr <- err - return - } - received <- msg - } - readErr <- nil - }() - - var wg sync.WaitGroup - for _, msg := range want { - msg := msg - wg.Add(1) - go func() { //异步处理 - defer wg.Done() - if err := sender.Send(msg); err != nil { - t.Errorf("Send() error = %v", err) - } - }() - } - wg.Wait() - - if err := <-readErr; err != nil { - t.Fatalf("Receive() error = %v", err) - } - - gotByID := make(map[uint64]protocol.Message, len(want)) - for range want { - msg := <-received - gotByID[msg.ID] = msg - } - - for _, msg := range want { - got, ok := gotByID[msg.ID] - if !ok { - t.Fatalf("missing message with ID %d", msg.ID) - } - if !reflect.DeepEqual(got, msg) { - t.Fatalf("message mismatch for ID %d: got %+v want %+v", msg.ID, got, msg) - } - } -} - -// TestReceiveLoopStopsOnHandlerError 验证 handler 返回错误时 ReceiveLoop 会退出并关闭连接。 -func TestReceiveLoopStopsOnHandlerError(t *testing.T) { - sender, receiver := newTransportConnPair(t, nil, nil) - - wantErr := errors.New("stop loop") - loopErr := make(chan error, 1) - go func() { - loopErr <- receiver.ReceiveLoop(func(msg protocol.Message) error { - return wantErr - }) - }() - - first := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - if err := sender.Send(first); err != nil { - t.Fatalf("Send(first) error = %v", err) - } - - err := <-loopErr - if !errors.Is(err, wantErr) { - t.Fatalf("ReceiveLoop() error = %v, want %v", err, wantErr) - } - if !strings.Contains(err.Error(), "receive loop handler") { - t.Fatalf("ReceiveLoop() error = %v, want handler context", err) - } -} - -// TestReceiveLoopStopsOnReadError 验证对端关闭时 ReceiveLoop 会以读取错误退出。 -func TestReceiveLoopStopsOnReadError(t *testing.T) { - sender, receiver := newTransportConnPair(t, nil, nil) - - loopErr := make(chan error, 1) - go func() { - loopErr <- receiver.ReceiveLoop(func(msg protocol.Message) error { - return nil - }) - }() - - if err := sender.Close(); err != nil { - t.Fatalf("sender.Close() error = %v", err) - } - - err := <-loopErr - if err == nil { - t.Fatal("ReceiveLoop() error = nil, want non-nil") - } - if !strings.Contains(err.Error(), "receive loop read") { - t.Fatalf("ReceiveLoop() error = %v, want read context", err) - } -} - -// TestCloseIsIdempotent 验证 Close 可以安全地被重复调用。 -func TestCloseIsIdempotent(t *testing.T) { - conn, peer := newTransportConnPair(t, nil, nil) - - if err := conn.Close(); err != nil { - t.Fatalf("Close(first) error = %v", err) - } - if err := conn.Close(); err != nil { - t.Fatalf("Close(second) error = %v, want nil", err) - } - if err := peer.Close(); err != nil && !strings.Contains(err.Error(), "closed") { - t.Fatalf("peer.Close() error = %v", err) - } -} - -// TestReceiveReturnsWrappedReadError 验证 Receive 在底层读取失败时会保留 transport 上下文。 -func TestReceiveReturnsWrappedReadError(t *testing.T) { - conn, peer := newTransportConnPair(t, nil, nil) - go func() { - _ = peer.Close() - }() - - _, err := conn.Receive() - if err == nil { - t.Fatal("Receive() error = nil, want non-nil") - } - if !strings.Contains(err.Error(), "transport: receive message") { - t.Fatalf("Receive() error = %v, want wrapped receive error", err) - } - if !errors.Is(err, io.EOF) && !strings.Contains(err.Error(), "closed") { - t.Fatalf("Receive() error = %v, want underlying read failure", err) - } -} - -func newTransportConnPair(t *testing.T, senderOpts []Option, receiverOpts []Option) (*TCPConn, *TCPConn) { - t.Helper() - - left, right := newTCPPair(t) - - sender, err := NewTCPConn(left, senderOpts...) - if err != nil { - t.Fatalf("NewTCPConn(sender) error = %v", err) - } - receiver, err := NewTCPConn(right, receiverOpts...) - if err != nil { - t.Fatalf("NewTCPConn(receiver) error = %v", err) - } - - t.Cleanup(func() { - _ = sender.Close() - _ = receiver.Close() - }) - - return sender, receiver -} diff --git a/cmd/internal/transport/tx_timestamp_debug.go b/cmd/internal/transport/tx_timestamp_debug.go deleted file mode 100644 index ad7712f..0000000 --- a/cmd/internal/transport/tx_timestamp_debug.go +++ /dev/null @@ -1,97 +0,0 @@ -package transport - -import ( - "encoding/json" - "fmt" - "os" - "path/filepath" - "sync" - - "omnisocketgo/cmd/internal/protocol" -) - -const ( - txTimestampDebugRecordTypeSendChunk = "send_chunk" - txTimestampDebugRecordTypeErrqueueEvent = "errqueue_event" -) - -// TXTimestampDebugRecord 是 TX errqueue 调试日志的一条 JSONL 记录。 -type TXTimestampDebugRecord struct { - RecordType string `json:"record_type"` - NodeRole string `json:"node_role,omitempty"` - NodeID string `json:"node_id,omitempty"` - MessageType protocol.MessageType `json:"message_type"` - MessageID uint64 `json:"message_id"` - From string `json:"from"` - To string `json:"to"` - FileName string `json:"file_name,omitempty"` - BodySize int `json:"body_size"` - - Phase string `json:"phase,omitempty"` - SendCallIndex *int `json:"send_call_index,omitempty"` - FrameOffsetStart *int `json:"frame_offset_start,omitempty"` - FrameOffsetEnd *int `json:"frame_offset_end,omitempty"` - BytesWritten *int `json:"bytes_written,omitempty"` - ExpectedTXID *uint32 `json:"expected_tx_id,omitempty"` - ReadIndex *int `json:"read_index,omitempty"` - EventName string `json:"event_name,omitempty"` - TSUnixNano *int64 `json:"ts_unix_nano,omitempty"` - EEInfo *uint32 `json:"ee_info,omitempty"` - EEData *uint32 `json:"ee_data,omitempty"` - MatchedSendCallIndex *int `json:"matched_send_call_index,omitempty"` - SelectedForLatency *bool `json:"selected_for_latency,omitempty"` -} - -// TXTimestampDebugLogger 接收 TX errqueue 调试记录。 -type TXTimestampDebugLogger interface { - LogTXTimestampDebugRecord(record TXTimestampDebugRecord) error -} - -// JSONLTXTimestampDebugLogger 以 JSONL 形式追加写 TX errqueue 调试日志。 -type JSONLTXTimestampDebugLogger struct { - mu sync.Mutex - closeOnce sync.Once - closeErr error - file *os.File -} - -// NewJSONLTXTimestampDebugLogger 创建一个线程安全的 TX errqueue JSONL 日志器。 -func NewJSONLTXTimestampDebugLogger(path string) (*JSONLTXTimestampDebugLogger, error) { - dir := filepath.Dir(path) - if err := os.MkdirAll(dir, 0o755); err != nil { - return nil, fmt.Errorf("transport: create tx timestamp debug log dir %s: %w", dir, err) - } - - file, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644) - if err != nil { - return nil, fmt.Errorf("transport: open tx timestamp debug log %s: %w", path, err) - } - - return &JSONLTXTimestampDebugLogger{file: file}, nil -} - -// LogTXTimestampDebugRecord 以单行 JSON 的形式追加一条调试记录。 -func (l *JSONLTXTimestampDebugLogger) LogTXTimestampDebugRecord(record TXTimestampDebugRecord) error { - line, err := json.Marshal(record) - if err != nil { - return err - } - - l.mu.Lock() - defer l.mu.Unlock() - - if _, err := l.file.Write(append(line, '\n')); err != nil { - return err - } - - return nil -} - -// Close 关闭底层文件；重复调用是安全的。 -func (l *JSONLTXTimestampDebugLogger) Close() error { - l.closeOnce.Do(func() { - l.closeErr = l.file.Close() - }) - - return l.closeErr -} diff --git a/cmd/internal/transport/udp.go b/cmd/internal/transport/udp.go deleted file mode 100644 index 87d1b31..0000000 --- a/cmd/internal/transport/udp.go +++ /dev/null @@ -1,151 +0,0 @@ -package transport - -import ( - "fmt" - "net" - "sync" - "syscall" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -// UDPConn wraps a UDP socket for protocol message send/receive. -type UDPConn struct { - conn *net.UDPConn - peerAddr *net.UDPAddr - raw syscall.RawConn - - linuxTimestampingEnabled bool - logger latencylog.Logger - txTimestampDebugLogger TXTimestampDebugLogger - txPacketSeq uint32 - pendingTX map[uint32]udpTXPendingRecord - nodeRole string - nodeID string - writeMu sync.Mutex - closeOnce sync.Once - closeErr error -} - -type udpTXPendingRecord struct { - msg protocol.Message - sendCallIndex int - bytesWritten int - expectedTXID uint32 - observedTimestamps map[string]int64 -} - -// UDPOption configures an optional behavior on UDPConn. -type UDPOption func(*UDPConn) - -// WithUDPLogger attaches latency logging context to a UDP connection. -func WithUDPLogger(logger latencylog.Logger, nodeRole, nodeID string) UDPOption { - return func(conn *UDPConn) { - conn.logger = logger - conn.nodeRole = nodeRole - conn.nodeID = nodeID - } -} - -// WithUDPTXTimestampDebugLogger attaches a TX errqueue debug logger. -func WithUDPTXTimestampDebugLogger(logger TXTimestampDebugLogger) UDPOption { - return func(conn *UDPConn) { - conn.txTimestampDebugLogger = logger - } -} - -// WithUDPLinuxTimestamping controls whether Linux UDP timestamping is enabled. -func WithUDPLinuxTimestamping(enabled bool) UDPOption { - return func(conn *UDPConn) { - conn.linuxTimestampingEnabled = enabled - } -} - -// NewUDPConn creates a UDP transport wrapper. -func NewUDPConn(conn *net.UDPConn, peerAddr *net.UDPAddr, opts ...UDPOption) (*UDPConn, error) { - udpConn := &UDPConn{ - conn: conn, - peerAddr: peerAddr, - linuxTimestampingEnabled: true, - logger: latencylog.NoopLogger{}, - pendingTX: make(map[uint32]udpTXPendingRecord), - } - - for _, opt := range opts { - opt(udpConn) - } - - if udpConn.logger == nil { - udpConn.logger = latencylog.NoopLogger{} - } - - if udpConn.linuxTimestampingEnabled { - if err := udpConn.initUDPLinuxTimestamping(); err != nil { - return nil, err - } - } - - return udpConn, nil -} - -// Send encodes and sends one protocol message over UDP. -func (c *UDPConn) Send(msg protocol.Message) error { - c.writeMu.Lock() - defer c.writeMu.Unlock() - - latencylog.LogMessageEvent(c.logger, c.nodeRole, c.nodeID, latencylog.EventSendHandoffBegin, msg) - if err := c.sendMessageLinux(msg); err != nil { - return fmt.Errorf("transport: udp send message: %w", err) - } - latencylog.LogMessageEvent(c.logger, c.nodeRole, c.nodeID, latencylog.EventSendHandoffEnd, msg) - - return nil -} - -// SendTo encodes and sends one protocol message to a specific UDP address. -func (c *UDPConn) SendTo(msg protocol.Message, addr *net.UDPAddr) error { - c.writeMu.Lock() - defer c.writeMu.Unlock() - - latencylog.LogMessageEvent(c.logger, c.nodeRole, c.nodeID, latencylog.EventSendHandoffBegin, msg) - if err := c.sendMessageToLinux(msg, addr); err != nil { - return fmt.Errorf("transport: udp send message to %s: %w", addr, err) - } - latencylog.LogMessageEvent(c.logger, c.nodeRole, c.nodeID, latencylog.EventSendHandoffEnd, msg) - - return nil -} - -// Receive reads one full protocol message from UDP. -func (c *UDPConn) Receive() (protocol.Message, *net.UDPAddr, error) { - msg, addr, err := c.receiveMessageLinux() - if err != nil { - return protocol.Message{}, nil, fmt.Errorf("transport: udp receive message: %w", err) - } - - return msg, addr, nil -} - -// ReceiveLoop continuously receives messages and passes them to handler. -func (c *UDPConn) ReceiveLoop(handler func(protocol.Message, *net.UDPAddr) error) error { - for { - msg, addr, err := c.Receive() - if err != nil { - return fmt.Errorf("transport: udp receive loop read: %w", err) - } - - if err := handler(msg, addr); err != nil { - return fmt.Errorf("transport: udp receive loop handler: %w", err) - } - } -} - -// Close closes the underlying UDP socket. -func (c *UDPConn) Close() error { - c.closeOnce.Do(func() { - c.closeErr = c.conn.Close() - }) - - return c.closeErr -} diff --git a/cmd/internal/transport/udp_device_linux.go b/cmd/internal/transport/udp_device_linux.go deleted file mode 100644 index b547ffe..0000000 --- a/cmd/internal/transport/udp_device_linux.go +++ /dev/null @@ -1,24 +0,0 @@ -//go:build linux - -package transport - -import ( - "fmt" - "syscall" -) - -// udpBindDeviceControl 返回一个 Control 函数，用于在 Linux 上将 UDP socket 绑定到指定网卡设备。 -func udpBindDeviceControl(device string) (func(string, string, syscall.RawConn) error, error) { - return func(_, _ string, rawConn syscall.RawConn) error { - var bindErr error - if err := rawConn.Control(func(fd uintptr) { - bindErr = syscall.BindToDevice(int(fd), device) - }); err != nil { - return err - } - if bindErr != nil { - return fmt.Errorf("transport: bind device %s: %w", device, bindErr) - } - return nil - }, nil -} diff --git a/cmd/internal/transport/udp_device_other.go b/cmd/internal/transport/udp_device_other.go deleted file mode 100644 index 3936003..0000000 --- a/cmd/internal/transport/udp_device_other.go +++ /dev/null @@ -1,12 +0,0 @@ -//go:build !linux - -package transport - -import ( - "fmt" - "syscall" -) - -func udpBindDeviceControl(device string) (func(string, string, syscall.RawConn) error, error) { - return nil, fmt.Errorf("transport: bind device %s is only supported on linux", device) -} diff --git a/cmd/internal/transport/udp_linux.go b/cmd/internal/transport/udp_linux.go deleted file mode 100644 index 0222a61..0000000 --- a/cmd/internal/transport/udp_linux.go +++ /dev/null @@ -1,556 +0,0 @@ -//go:build linux - -package transport - -import ( - "errors" - "fmt" - "net" - "syscall" - "time" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -// UDP 接收缓冲区需要容纳完整 datagram 和协议头。 -const udpReceiveBufferSize = protocol.MaxFrameSize + 1024 - -// initUDPLinuxTimestamping 拿到底层 fd，并打开 Linux timestamping。 -func (c *UDPConn) initUDPLinuxTimestamping() error { - rawConn, err := c.conn.SyscallConn() - if err != nil || rawConn == nil { - if err != nil { - return fmt.Errorf("transport: udp get syscall conn: %w", err) - } - return fmt.Errorf("transport: udp missing syscall conn") - } - - flagCandidates := []int{ - linuxSOFTimestampingTXSched | - linuxSOFTimestampingTXSoftware | - linuxSOFTimestampingRXSoftware | - linuxSOFTimestampingSoftware | - linuxSOFTimestampingOptID | - linuxSOFTimestampingOptTSONLY, - linuxSOFTimestampingTXSched | - linuxSOFTimestampingTXSoftware | - linuxSOFTimestampingRXSoftware | - linuxSOFTimestampingSoftware | - linuxSOFTimestampingOptTSONLY, - } - - var lastErr error - for _, flags := range flagCandidates { - err := rawConn.Control(func(fd uintptr) { - lastErr = syscall.SetsockoptInt(int(fd), syscall.SOL_SOCKET, linuxSOTimestampingNew, flags) - }) - if err != nil { - return err - } - if lastErr == nil { - c.raw = rawConn - return nil - } - if !errors.Is(lastErr, syscall.EINVAL) { - return lastErr - } - } - - return lastErr -} - -// sendMessageLinux 编码消息并通过 UDP 发送，采集 TX 时间戳。 -func (c *UDPConn) sendMessageLinux(msg protocol.Message) error { - payload, err := protocol.EncodeMessage(msg) - if err != nil { - return fmt.Errorf("protocol: encode message: %w", err) - } - - return c.sendUDPPayloadLinux(msg, payload, c.peerAddr) -} - -// sendMessageToLinux 编码消息并通过 UDP 发送到指定地址，采集 TX 时间戳。 -func (c *UDPConn) sendMessageToLinux(msg protocol.Message, addr *net.UDPAddr) error { - payload, err := protocol.EncodeMessage(msg) - if err != nil { - return fmt.Errorf("protocol: encode message: %w", err) - } - - return c.sendUDPPayloadLinux(msg, payload, addr) -} - -func (c *UDPConn) sendUDPPayloadLinux(msg protocol.Message, payload []byte, addr *net.UDPAddr) error { - readIndex := 0 - - // pre-send drain 可能读到上一条消息晚到的 errqueue 事件， - // 这里必须先清掉，并且按 ee_data 归还给原消息，不能污染当前消息。 - c.drainPendingUDPTXTimestampEvents(linuxTXTimestampPhasePreSendDrain, &readIndex) - - chunk := c.newUDPSendChunk(len(payload)) - var err error - if addr != nil { - err = c.udpSendTo(payload, addr) - } else { - err = c.udpSend(payload) - } - if err != nil { - return err - } - - c.commitUDPSend(msg, chunk) - c.collectAndLogUDPTXTimestampEvents(msg, chunk, &readIndex) - return nil -} - -func (c *UDPConn) newUDPSendChunk(payloadLen int) txSendChunk { - return txSendChunk{ - SendCallIndex: 0, - FrameOffsetStart: 0, - FrameOffsetEnd: payloadLen - 1, - BytesWritten: payloadLen, - ExpectedTXID: c.txPacketSeq, - } -} - -func (c *UDPConn) commitUDPSend(msg protocol.Message, chunk txSendChunk) { - // Linux 对 UDP datagram 的 ee_data 是按包递增的 ID。 - // 这里把 ID 和原始消息元数据绑定起来，后续 drain 到的晚到事件才能记回原消息。 - if c.txTimestampDebugLogger != nil { - c.pendingTX[chunk.ExpectedTXID] = udpTXPendingRecord{ - msg: msg, - sendCallIndex: chunk.SendCallIndex, - bytesWritten: chunk.BytesWritten, - expectedTXID: chunk.ExpectedTXID, - observedTimestamps: make(map[string]int64, 2), - } - c.logUDPTXSendChunkDebugRecord(msg, chunk) - } - - c.txPacketSeq++ -} - -// udpSend 通过已连接的 UDP socket 发送数据。 -func (c *UDPConn) udpSend(payload []byte) error { - if c.raw != nil { - return c.udpSendmsgRaw(payload, nil) - } - - _, err := c.conn.Write(payload) - return err -} - -// udpSendTo 通过 UDP socket 发送数据到指定地址。 -func (c *UDPConn) udpSendTo(payload []byte, addr *net.UDPAddr) error { - if c.raw != nil { - sa := udpAddrToSockaddr(addr) - if sa != nil { - return c.udpSendmsgRaw(payload, sa) - } - } - - _, err := c.conn.WriteToUDP(payload, addr) - return err -} - -// udpSendmsgRaw 通过 sendmsg syscall 发送 UDP 数据。 -func (c *UDPConn) udpSendmsgRaw(payload []byte, to syscall.Sockaddr) error { - var opErr error - - for { - err := c.raw.Control(func(fd uintptr) { - opErr = syscall.Sendmsg(int(fd), payload, nil, to, 0) - }) - if err != nil { - return err - } - if opErr == nil { - return nil - } - if isWouldBlock(opErr) { - time.Sleep(linuxDataPollInterval) - continue - } - return opErr - } -} - -// receiveMessageLinux 从 UDP 连接读取一条完整消息，并记录 RX 时间戳。 -func (c *UDPConn) receiveMessageLinux() (protocol.Message, *net.UDPAddr, error) { - payload, addr, rxTimestamp, err := c.udpRecvFrom() - if err != nil { - return protocol.Message{}, nil, fmt.Errorf("protocol: udp read: %w", err) - } - - msg, err := protocol.DecodeMessage(payload) - if err != nil { - return protocol.Message{}, nil, fmt.Errorf("protocol: decode message: %w", err) - } - - if rxTimestamp > 0 { - latencylog.LogMessageEventAt(c.logger, c.nodeRole, c.nodeID, latencylog.EventBRXSoftware, rxTimestamp, msg) - } - - return msg, addr, nil -} - -// udpRecvFrom 从 UDP socket 接收一个完整数据报，返回数据、来源地址和 RX 时间戳。 -func (c *UDPConn) udpRecvFrom() ([]byte, *net.UDPAddr, int64, error) { - if c.raw != nil { - return c.udpRecvmsgRaw() - } - - buf := make([]byte, udpReceiveBufferSize) - n, addr, err := c.conn.ReadFromUDP(buf) - if err != nil { - return nil, nil, 0, err - } - - return buf[:n], addr, 0, nil -} - -// udpRecvmsgRaw 通过 recvmsg syscall 接收 UDP 数据，同时采集 RX 时间戳。 -func (c *UDPConn) udpRecvmsgRaw() ([]byte, *net.UDPAddr, int64, error) { - for { - var ( - n int - rxTimeNS int64 - from syscall.Sockaddr - opErr error - ) - - buf := make([]byte, udpReceiveBufferSize) - err := c.raw.Control(func(fd uintptr) { - oob := make([]byte, linuxTimestampControlBufferSize) - readN, oobN, _, sa, recvErr := syscall.Recvmsg(int(fd), buf, oob, 0) - if recvErr != nil { - opErr = recvErr - return - } - n = readN - from = sa - rxTimeNS = parseRXTimestampControlMessages(oob[:oobN]) - }) - if err != nil { - return nil, nil, 0, err - } - if opErr != nil { - if isWouldBlock(opErr) { - time.Sleep(linuxDataPollInterval) - continue - } - return nil, nil, 0, opErr - } - - return buf[:n], sockaddrToUDPAddr(from), rxTimeNS, nil - } -} - -// collectAndLogUDPTXTimestampEvents 采集并记录 UDP 发送的 TX 时间戳事件。 -func (c *UDPConn) collectAndLogUDPTXTimestampEvents(msg protocol.Message, chunk txSendChunk, readIndex *int) { - timestamps := c.collectUDPTXTimestampEvents(msg, chunk, readIndex) - - if ts, ok := timestamps[latencylog.EventATXSched]; ok { - latencylog.LogMessageEventAt(c.logger, c.nodeRole, c.nodeID, latencylog.EventATXSched, ts, msg) - } - if ts, ok := timestamps[latencylog.EventATXSoftware]; ok { - latencylog.LogMessageEventAt(c.logger, c.nodeRole, c.nodeID, latencylog.EventATXSoftware, ts, msg) - } -} - -// collectUDPTXTimestampEvents 在 errqueue 中等待 TX 时间戳。 -func (c *UDPConn) collectUDPTXTimestampEvents(msg protocol.Message, chunk txSendChunk, readIndex *int) map[string]int64 { - if c.raw == nil { - return nil - } - - deadline := time.Now().Add(linuxTXTimestampWaitTimeout) - observed := make([]observedTXTimestampEvent, 0, 4) - - for time.Now().Before(deadline) { - event, err := c.recvUDPTXTimestampOnce() - if err != nil { - if isWouldBlock(err) { - time.Sleep(linuxTXTimestampPollInterval) - continue - } - break - } - if event.EventName == "" || event.TSUnixNano <= 0 { - continue - } - - observed = append(observed, observedTXTimestampEvent{ - Phase: linuxTXTimestampPhasePostSendCollect, - ReadIndex: *readIndex, - Event: event, - }) - c.recordUDPPendingEvent(event) - *readIndex = *readIndex + 1 - - selection := selectTXTimestampEvents(observed, chunk.ExpectedTXID, true) - if selection.HasEvent && selection.SelectedID == chunk.ExpectedTXID && hasCompleteTXTimestampPair(selection.Timestamps) { - break - } - } - - selection := selectTXTimestampEvents(observed, chunk.ExpectedTXID, true) - c.logObservedUDPTXTimestampEvents(msg, chunk, observed, selection) - c.releaseCompletedUDPPendingFromObserved(observed) - c.drainPendingUDPTXTimestampEvents(linuxTXTimestampPhasePostSelectDrain, readIndex) - c.releaseCompletedUDPPending(chunk.ExpectedTXID) - return selection.Timestamps -} - -// drainPendingUDPTXTimestampEvents 清空 errqueue 中残留的时间戳事件。 -func (c *UDPConn) drainPendingUDPTXTimestampEvents(phase string, readIndex *int) { - if c.raw == nil { - return - } - - for { - event, err := c.recvUDPTXTimestampOnce() - if err != nil { - if isWouldBlock(err) { - return - } - return - } - if event.EventName == "" || event.TSUnixNano <= 0 { - continue - } - - complete := c.recordUDPPendingEvent(event) - if msg, chunks, ok := c.lookupUDPPendingDebugContext(event.EEData); ok { - c.logUDPTXErrqueueDebugRecord(msg, chunks, observedTXTimestampEvent{ - Phase: phase, - ReadIndex: *readIndex, - Event: event, - }, false) - if complete { - delete(c.pendingTX, event.EEData) - } - } - *readIndex = *readIndex + 1 - } -} - -// recvUDPTXTimestampOnce 从 errqueue 读一次时间戳事件。 -func (c *UDPConn) recvUDPTXTimestampOnce() (txTimestampEvent, error) { - var ( - event txTimestampEvent - opErr error - ) - - err := c.raw.Control(func(fd uintptr) { - oob := make([]byte, linuxTimestampControlBufferSize) - _, oobn, _, _, recvErr := syscall.Recvmsg(int(fd), nil, oob, syscall.MSG_ERRQUEUE|syscall.MSG_DONTWAIT) - if recvErr != nil { - opErr = recvErr - return - } - event, _ = parseTXTimestampControlMessages(oob[:oobn]) - }) - if err != nil { - return txTimestampEvent{}, err - } - if opErr != nil { - return txTimestampEvent{}, opErr - } - - return event, nil -} - -func (c *UDPConn) recordUDPPendingEvent(event txTimestampEvent) bool { - if !isBusinessTXTimestampEventName(event.EventName) { - return false - } - - record, ok := c.pendingTX[event.EEData] - if !ok { - return false - } - if record.observedTimestamps == nil { - record.observedTimestamps = make(map[string]int64, 2) - } - if existing, exists := record.observedTimestamps[event.EventName]; !exists || event.TSUnixNano < existing { - record.observedTimestamps[event.EventName] = event.TSUnixNano - } - c.pendingTX[event.EEData] = record - - return hasCompleteTXTimestampPair(record.observedTimestamps) -} - -func (c *UDPConn) releaseCompletedUDPPending(txID uint32) { - record, ok := c.pendingTX[txID] - if !ok { - return - } - if hasCompleteTXTimestampPair(record.observedTimestamps) { - delete(c.pendingTX, txID) - } -} - -func (c *UDPConn) releaseCompletedUDPPendingFromObserved(observed []observedTXTimestampEvent) { - seen := make(map[uint32]struct{}, len(observed)) - for _, entry := range observed { - if _, ok := seen[entry.Event.EEData]; ok { - continue - } - c.releaseCompletedUDPPending(entry.Event.EEData) - seen[entry.Event.EEData] = struct{}{} - } -} - -func (c *UDPConn) lookupUDPPendingDebugContext(txID uint32) (protocol.Message, []txSendChunk, bool) { - record, ok := c.pendingTX[txID] - if !ok { - return protocol.Message{}, nil, false - } - - chunk := txSendChunk{ - SendCallIndex: record.sendCallIndex, - FrameOffsetStart: 0, - FrameOffsetEnd: record.bytesWritten - 1, - BytesWritten: record.bytesWritten, - ExpectedTXID: record.expectedTXID, - } - return record.msg, []txSendChunk{chunk}, true -} - -func (c *UDPConn) logObservedUDPTXTimestampEvents(msg protocol.Message, chunk txSendChunk, observed []observedTXTimestampEvent, selection txTimestampSelection) { - if len(observed) == 0 { - return - } - - currentChunks := []txSendChunk{chunk} - for _, entry := range observed { - recordMsg := msg - recordChunks := currentChunks - if pendingMsg, pendingChunks, ok := c.lookupUDPPendingDebugContext(entry.Event.EEData); ok { - recordMsg = pendingMsg - recordChunks = pendingChunks - } - - // 理想情况应命中本次 expectedTXID；如果等待窗口里只看到了更高的 ee_data， - // 就退回到本轮实际观察到的最新事件，至少保留调试和定位线索。 - selected := selection.HasEvent && - entry.Event.EEData == selection.SelectedID && - isBusinessTXTimestampEventName(entry.Event.EventName) && - selection.Timestamps[entry.Event.EventName] == entry.Event.TSUnixNano - c.logUDPTXErrqueueDebugRecord(recordMsg, recordChunks, entry, selected) - } -} - -func (c *UDPConn) logUDPTXSendChunkDebugRecord(msg protocol.Message, chunk txSendChunk) { - if c.txTimestampDebugLogger == nil { - return - } - - sendCallIndex := chunk.SendCallIndex - frameOffsetStart := chunk.FrameOffsetStart - frameOffsetEnd := chunk.FrameOffsetEnd - bytesWritten := chunk.BytesWritten - expectedTXID := chunk.ExpectedTXID - - record := c.newUDPTXTimestampDebugRecord(msg) - record.RecordType = txTimestampDebugRecordTypeSendChunk - record.SendCallIndex = &sendCallIndex - record.FrameOffsetStart = &frameOffsetStart - record.FrameOffsetEnd = &frameOffsetEnd - record.BytesWritten = &bytesWritten - record.ExpectedTXID = &expectedTXID - c.logUDPTXTimestampDebugRecord(record) -} - -func (c *UDPConn) logUDPTXErrqueueDebugRecord(msg protocol.Message, chunks []txSendChunk, observed observedTXTimestampEvent, selected bool) { - if c.txTimestampDebugLogger == nil { - return - } - - readIndex := observed.ReadIndex - tsUnixNano := observed.Event.TSUnixNano - eeInfo := observed.Event.EEInfo - eeData := observed.Event.EEData - selectedForLatency := selected - - record := c.newUDPTXTimestampDebugRecord(msg) - record.RecordType = txTimestampDebugRecordTypeErrqueueEvent - record.Phase = observed.Phase - record.ReadIndex = &readIndex - record.EventName = observed.Event.EventName - record.TSUnixNano = &tsUnixNano - record.EEInfo = &eeInfo - record.EEData = &eeData - record.SelectedForLatency = &selectedForLatency - if matchedSendCallIndex, ok := matchTXTimestampEventToSendChunk(observed.Event.EEData, chunks); ok { - record.MatchedSendCallIndex = &matchedSendCallIndex - } - c.logUDPTXTimestampDebugRecord(record) -} - -func (c *UDPConn) newUDPTXTimestampDebugRecord(msg protocol.Message) TXTimestampDebugRecord { - return TXTimestampDebugRecord{ - NodeRole: c.nodeRole, - NodeID: c.nodeID, - MessageType: msg.Type, - MessageID: msg.ID, - From: msg.From, - To: msg.To, - FileName: msg.FileName, - BodySize: len(msg.Body), - } -} - -func (c *UDPConn) logUDPTXTimestampDebugRecord(record TXTimestampDebugRecord) { - if c.txTimestampDebugLogger == nil { - return - } - - _ = c.txTimestampDebugLogger.LogTXTimestampDebugRecord(record) -} - -// udpAddrToSockaddr 将 net.UDPAddr 转换为 syscall.Sockaddr。 -func udpAddrToSockaddr(addr *net.UDPAddr) syscall.Sockaddr { - if ip4 := addr.IP.To4(); ip4 != nil { - sa := &syscall.SockaddrInet4{Port: addr.Port} - copy(sa.Addr[:], ip4) - return sa - } - if ip6 := addr.IP.To16(); ip6 != nil { - sa := &syscall.SockaddrInet6{Port: addr.Port} - copy(sa.Addr[:], ip6) - return sa - } - return nil -} - -// sockaddrToUDPAddr 将 syscall.Sockaddr 转换为 net.UDPAddr。 -func sockaddrToUDPAddr(sa syscall.Sockaddr) *net.UDPAddr { - switch addr := sa.(type) { - case *syscall.SockaddrInet4: - return &net.UDPAddr{ - IP: net.IP(addr.Addr[:]), - Port: addr.Port, - } - case *syscall.SockaddrInet6: - return &net.UDPAddr{ - IP: net.IP(addr.Addr[:]), - Port: addr.Port, - Zone: zoneToString(addr.ZoneId), - } - default: - return nil - } -} - -func zoneToString(zone uint32) string { - if zone == 0 { - return "" - } - iface, err := net.InterfaceByIndex(int(zone)) - if err != nil { - return "" - } - return iface.Name -} diff --git a/cmd/internal/transport/udp_linux_test.go b/cmd/internal/transport/udp_linux_test.go deleted file mode 100644 index 2472e92..0000000 --- a/cmd/internal/transport/udp_linux_test.go +++ /dev/null @@ -1,332 +0,0 @@ -//go:build linux - -package transport - -import ( - "net" - "reflect" - "testing" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -func TestUDPLinuxTimestampingRecordsKernelEvents(t *testing.T) { - tests := []struct { - name string - msg protocol.Message - }{ - { - name: "text", - msg: protocol.Message{ - Type: protocol.MessageTypeText, - ID: 41, - From: "peer-a", - To: "peer-b", - Body: []byte("hello over udp"), - }, - }, - { - name: "file", - msg: protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 42, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0x00, 0x01, 0x02, 0xff}, - }, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - senderLogger := &recordingLogger{} - receiverLogger := &recordingLogger{} - - serverAddr, err := net.ResolveUDPAddr("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ResolveUDPAddr() error = %v", err) - } - serverRaw, err := net.ListenUDP("udp", serverAddr) - if err != nil { - t.Fatalf("ListenUDP() error = %v", err) - } - receiver, err := NewUDPConn( - serverRaw, - nil, - WithUDPLogger(receiverLogger, latencylog.NodeRolePeer, "peer-b"), - ) - if err != nil { - _ = serverRaw.Close() - t.Fatalf("NewUDPConn(receiver) error = %v", err) - } - t.Cleanup(func() { _ = receiver.Close() }) - - peerRaw, err := net.DialUDP("udp", nil, serverRaw.LocalAddr().(*net.UDPAddr)) - if err != nil { - t.Fatalf("DialUDP() error = %v", err) - } - sender, err := NewUDPConn( - peerRaw, - nil, - WithUDPLogger(senderLogger, latencylog.NodeRolePeer, "peer-a"), - ) - if err != nil { - _ = peerRaw.Close() - t.Fatalf("NewUDPConn(sender) error = %v", err) - } - t.Cleanup(func() { _ = sender.Close() }) - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(tt.msg) - }() - - got, _, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v", err) - } - if !reflect.DeepEqual(got, tt.msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, tt.msg) - } - - assertHasEvent(t, senderLogger.Events(), latencylog.EventATXSched, tt.msg.ID) - assertHasEvent(t, senderLogger.Events(), latencylog.EventATXSoftware, tt.msg.ID) - assertHasEvent(t, receiverLogger.Events(), latencylog.EventBRXSoftware, tt.msg.ID) - }) - } -} - -func TestUDPLinuxTimestampingDebugLoggerCapturesDatagramAndErrqueueEvents(t *testing.T) { - debugLogger := &recordingTXTimestampDebugLogger{} - senderLogger := &recordingLogger{} - receiverLogger := &recordingLogger{} - - serverAddr, err := net.ResolveUDPAddr("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ResolveUDPAddr() error = %v", err) - } - serverRaw, err := net.ListenUDP("udp", serverAddr) - if err != nil { - t.Fatalf("ListenUDP() error = %v", err) - } - receiver, err := NewUDPConn( - serverRaw, - nil, - WithUDPLogger(receiverLogger, latencylog.NodeRolePeer, "peer-b"), - ) - if err != nil { - _ = serverRaw.Close() - t.Fatalf("NewUDPConn(receiver) error = %v", err) - } - t.Cleanup(func() { _ = receiver.Close() }) - - peerRaw, err := net.DialUDP("udp", nil, serverRaw.LocalAddr().(*net.UDPAddr)) - if err != nil { - t.Fatalf("DialUDP() error = %v", err) - } - sender, err := NewUDPConn( - peerRaw, - nil, - WithUDPLogger(senderLogger, latencylog.NodeRolePeer, "peer-a"), - WithUDPTXTimestampDebugLogger(debugLogger), - ) - if err != nil { - _ = peerRaw.Close() - t.Fatalf("NewUDPConn(sender) error = %v", err) - } - t.Cleanup(func() { _ = sender.Close() }) - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 99, - From: "peer-a", - To: "peer-b", - Body: []byte("hello udp debug"), - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - got, _, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v", err) - } - if !reflect.DeepEqual(got, msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, msg) - } - - assertHasEvent(t, senderLogger.Events(), latencylog.EventATXSched, msg.ID) - assertHasEvent(t, senderLogger.Events(), latencylog.EventATXSoftware, msg.ID) - assertHasEvent(t, receiverLogger.Events(), latencylog.EventBRXSoftware, msg.ID) - - sendChunkRecords := debugRecordsByType(debugLogger.Records(), txTimestampDebugRecordTypeSendChunk) - errqueueRecords := debugRecordsByType(debugLogger.Records(), txTimestampDebugRecordTypeErrqueueEvent) - if len(sendChunkRecords) == 0 { - t.Fatal("send_chunk debug records = 0, want at least 1") - } - if len(errqueueRecords) == 0 { - t.Fatal("errqueue_event debug records = 0, want at least 1") - } - - finalChunkRecord := sendChunkRecords[len(sendChunkRecords)-1] - if finalChunkRecord.ExpectedTXID == nil { - t.Fatal("final send_chunk expected_tx_id = nil, want non-nil") - } - finalExpectedTXID := *finalChunkRecord.ExpectedTXID - - selectedRecords := selectedErrqueueRecords(errqueueRecords) - if len(selectedRecords) == 0 { - t.Fatal("selected errqueue debug records = 0, want at least 1") - } - - highestObservedID := uint32(0) - haveHighestObservedID := false - haveExactFinalID := false - for _, record := range errqueueRecords { - if record.EEData == nil { - continue - } - if !haveHighestObservedID || *record.EEData > highestObservedID { - highestObservedID = *record.EEData - haveHighestObservedID = true - } - if *record.EEData == finalExpectedTXID && isBusinessTXTimestampRecord(record) { - haveExactFinalID = true - } - } - if !haveHighestObservedID { - t.Fatal("highestObservedID missing, want at least one ee_data") - } - - wantSelectedID := highestObservedID - if haveExactFinalID { - wantSelectedID = finalExpectedTXID - } - for _, record := range selectedRecords { - if record.EEData == nil { - t.Fatalf("selected record missing ee_data: %+v", record) - } - if *record.EEData != wantSelectedID { - t.Fatalf("selected ee_data = %d, want %d", *record.EEData, wantSelectedID) - } - } - - selectedByEventName := make(map[string]int64, len(selectedRecords)) - for _, record := range selectedRecords { - if record.TSUnixNano == nil { - t.Fatalf("selected record missing timestamp: %+v", record) - } - selectedByEventName[record.EventName] = *record.TSUnixNano - } - - senderEventsByName := make(map[string]int64) - for _, event := range senderLogger.Events() { - if event.MessageID != msg.ID { - continue - } - if !isBusinessTXTimestampEventName(event.Event) { - continue - } - senderEventsByName[event.Event] = event.TsUnixNano - } - - for eventName, selectedTS := range selectedByEventName { - if senderEventsByName[eventName] != selectedTS { - t.Fatalf("sender latency event %s = %d, want %d from selected debug record", eventName, senderEventsByName[eventName], selectedTS) - } - } -} - -func TestUDPLinuxTimestampingCanBeDisabled(t *testing.T) { - senderLogger := &recordingLogger{} - receiverLogger := &recordingLogger{} - - serverAddr, err := net.ResolveUDPAddr("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ResolveUDPAddr() error = %v", err) - } - serverRaw, err := net.ListenUDP("udp", serverAddr) - if err != nil { - t.Fatalf("ListenUDP() error = %v", err) - } - receiver, err := NewUDPConn( - serverRaw, - nil, - WithUDPLogger(receiverLogger, latencylog.NodeRolePeer, "peer-b"), - WithUDPLinuxTimestamping(false), - ) - if err != nil { - _ = serverRaw.Close() - t.Fatalf("NewUDPConn(receiver) error = %v", err) - } - t.Cleanup(func() { _ = receiver.Close() }) - - peerRaw, err := net.DialUDP("udp", nil, serverRaw.LocalAddr().(*net.UDPAddr)) - if err != nil { - t.Fatalf("DialUDP() error = %v", err) - } - sender, err := NewUDPConn( - peerRaw, - nil, - WithUDPLogger(senderLogger, latencylog.NodeRolePeer, "peer-a"), - WithUDPLinuxTimestamping(false), - ) - if err != nil { - _ = peerRaw.Close() - t.Fatalf("NewUDPConn(sender) error = %v", err) - } - t.Cleanup(func() { _ = sender.Close() }) - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 123, - From: "peer-a", - To: "peer-b", - Body: []byte("hello without udp timestamping"), - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - got, _, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v", err) - } - if !reflect.DeepEqual(got, msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, msg) - } - if sender.raw != nil { - t.Fatal("sender.raw != nil, want nil when linux timestamping is disabled") - } - if receiver.raw != nil { - t.Fatal("receiver.raw != nil, want nil when linux timestamping is disabled") - } - assertMissingEvent(t, senderLogger.Events(), latencylog.EventATXSched, msg.ID) - assertMissingEvent(t, senderLogger.Events(), latencylog.EventATXSoftware, msg.ID) - assertMissingEvent(t, receiverLogger.Events(), latencylog.EventBRXSoftware, msg.ID) -} - -func assertMissingEvent(t *testing.T, events []latencylog.Event, wantEvent string, wantMessageID uint64) { - t.Helper() - - for _, event := range events { - if event.Event == wantEvent && event.MessageID == wantMessageID { - t.Fatalf("unexpected event %s for message %d: %+v", wantEvent, wantMessageID, event) - } - } -} diff --git a/cmd/internal/transport/udp_test.go b/cmd/internal/transport/udp_test.go deleted file mode 100644 index 239e927..0000000 --- a/cmd/internal/transport/udp_test.go +++ /dev/null @@ -1,354 +0,0 @@ -package transport - -import ( - "net" - "reflect" - "strings" - "sync" - "testing" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/protocol" -) - -// TestUDPSendReceiveMessage 验证 UDP transport 可以正常收发 text 和 file 消息。 -func TestUDPSendReceiveMessage(t *testing.T) { - tests := []struct { - name string - msg protocol.Message - }{ - { - name: "text", - msg: protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello udp"), - }, - }, - { - name: "file", - msg: protocol.Message{ - Type: protocol.MessageTypeFile, - ID: 2, - From: "peer-a", - To: "peer-b", - FileName: "data.bin", - Body: []byte{0x00, 0x10, 0xff}, - }, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - sender, receiver := newUDPConnPair(t, nil, nil) - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(tt.msg) - }() - - got, _, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v", err) - } - - if !reflect.DeepEqual(got, tt.msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, tt.msg) - } - }) - } -} - -// TestUDPSendLogsHandoffEvents 验证 UDP Send 会记录 handoff 事件。 -func TestUDPSendLogsHandoffEvents(t *testing.T) { - logger := &recordingLogger{} - sender, receiver := newUDPConnPair( - t, - []UDPOption{WithUDPLogger(logger, latencylog.NodeRolePeer, "peer-a")}, - nil, - ) - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 7, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - } - - sendErr := make(chan error, 1) - go func() { - sendErr <- sender.Send(msg) - }() - - got, _, err := receiver.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v", err) - } - if !reflect.DeepEqual(got, msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, msg) - } - - events := logger.Events() - if len(events) < 2 { - t.Fatalf("event count = %d, want at least 2", len(events)) - } - if events[0].Event != latencylog.EventSendHandoffBegin { - t.Fatalf("first event = %q, want %q", events[0].Event, latencylog.EventSendHandoffBegin) - } - // 最后一个事件应该是 SendHandoffEnd - lastEvent := events[len(events)-1] - if lastEvent.Event != latencylog.EventSendHandoffEnd { - t.Fatalf("last event = %q, want %q", lastEvent.Event, latencylog.EventSendHandoffEnd) - } -} - -// TestUDPReceiveLoopDeliversMessages 验证 ReceiveLoop 会逐条交付连续到达的消息。 -func TestUDPReceiveLoopDeliversMessages(t *testing.T) { - sender, receiver := newUDPConnPair(t, nil, nil) - - want := []protocol.Message{ - { - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "peer-b", - Body: []byte("hello"), - }, - { - Type: protocol.MessageTypeFile, - ID: 2, - From: "peer-a", - To: "peer-b", - FileName: "payload.bin", - Body: []byte{0x01, 0x02, 0x03}, - }, - } - - var ( - mu sync.Mutex - got []protocol.Message - ) - loopErr := make(chan error, 1) - go func() { - loopErr <- receiver.ReceiveLoop(func(msg protocol.Message, _ *net.UDPAddr) error { - mu.Lock() - got = append(got, msg) - done := len(got) >= len(want) - mu.Unlock() - if done { - return receiver.Close() - } - return nil - }) - }() - - for _, msg := range want { - if err := sender.Send(msg); err != nil { - t.Fatalf("Send() error = %v", err) - } - } - - err := <-loopErr - if err == nil { - t.Fatal("ReceiveLoop() error = nil, want non-nil after receiver close") - } - if !strings.Contains(err.Error(), "closed") && !strings.Contains(err.Error(), "use of closed network connection") { - t.Fatalf("ReceiveLoop() error = %v, want close-related error", err) - } - - mu.Lock() - defer mu.Unlock() - if !reflect.DeepEqual(got, want) { - t.Fatalf("received messages mismatch: got %+v want %+v", got, want) - } -} - -// TestUDPCloseIsIdempotent 验证 Close 可以安全地被重复调用。 -func TestUDPCloseIsIdempotent(t *testing.T) { - conn, peer := newUDPConnPair(t, nil, nil) - - if err := conn.Close(); err != nil { - t.Fatalf("Close(first) error = %v", err) - } - if err := conn.Close(); err != nil { - t.Fatalf("Close(second) error = %v, want nil", err) - } - _ = peer.Close() -} - -// TestUDPSendToMessage 验证 SendTo 可以向指定地址发送消息。 -func TestUDPSendToMessage(t *testing.T) { - serverConn := newUDPListener(t) - peerConn := newUDPDialed(t, serverConn.conn.LocalAddr().(*net.UDPAddr)) - - msg := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 1, - From: "peer-a", - To: "server", - Body: []byte("hello sendto"), - } - - // peer 发送消息到 server - sendErr := make(chan error, 1) - go func() { - sendErr <- peerConn.Send(msg) - }() - - got, addr, err := serverConn.Receive() - if err != nil { - t.Fatalf("Receive() error = %v", err) - } - if err := <-sendErr; err != nil { - t.Fatalf("Send() error = %v", err) - } - if !reflect.DeepEqual(got, msg) { - t.Fatalf("message mismatch: got %+v want %+v", got, msg) - } - - // server 用 SendTo 回复到 peer 地址 - reply := protocol.Message{ - Type: protocol.MessageTypeText, - ID: 2, - From: "server", - To: "peer-a", - Body: []byte("reply"), - } - - sendErr2 := make(chan error, 1) - go func() { - sendErr2 <- serverConn.SendTo(reply, addr) - }() - - gotReply, _, err := peerConn.Receive() - if err != nil { - t.Fatalf("peer Receive() error = %v", err) - } - if err := <-sendErr2; err != nil { - t.Fatalf("SendTo() error = %v", err) - } - if !reflect.DeepEqual(gotReply, reply) { - t.Fatalf("reply mismatch: got %+v want %+v", gotReply, reply) - } -} - -// newUDPConnPair 创建一对互相连接的 UDP transport 连接，用于测试。 -func newUDPConnPair(t *testing.T, senderOpts []UDPOption, receiverOpts []UDPOption) (*UDPConn, *UDPConn) { - t.Helper() - - // 创建两个 UDP socket，通过 Dial 互相连接 - addr1, err := net.ResolveUDPAddr("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ResolveUDPAddr() error = %v", err) - } - addr2, err := net.ResolveUDPAddr("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ResolveUDPAddr() error = %v", err) - } - - conn1, err := net.ListenUDP("udp", addr1) - if err != nil { - t.Fatalf("ListenUDP(1) error = %v", err) - } - conn2, err := net.ListenUDP("udp", addr2) - if err != nil { - _ = conn1.Close() - t.Fatalf("ListenUDP(2) error = %v", err) - } - receiverLocalAddr := conn2.LocalAddr().(*net.UDPAddr) - - // 用 Dial 模式连接对端 - senderRaw, err := net.DialUDP("udp", nil, conn2.LocalAddr().(*net.UDPAddr)) - if err != nil { - _ = conn1.Close() - _ = conn2.Close() - t.Fatalf("DialUDP(sender) error = %v", err) - } - _ = conn1.Close() // 不再需要 conn1 - _ = conn2.Close() // 释放 receiver 计划使用的本地地址 - - receiverRaw, err := net.DialUDP("udp", receiverLocalAddr, senderRaw.LocalAddr().(*net.UDPAddr)) - if err != nil { - _ = senderRaw.Close() - t.Fatalf("DialUDP(receiver) error = %v", err) - } - - sender, err := NewUDPConn(senderRaw, nil, senderOpts...) - if err != nil { - _ = senderRaw.Close() - _ = receiverRaw.Close() - t.Fatalf("NewUDPConn(sender) error = %v", err) - } - - receiver, err := NewUDPConn(receiverRaw, nil, receiverOpts...) - if err != nil { - _ = sender.Close() - _ = receiverRaw.Close() - t.Fatalf("NewUDPConn(receiver) error = %v", err) - } - - t.Cleanup(func() { - _ = sender.Close() - _ = receiver.Close() - }) - - return sender, receiver -} - -// newUDPListener 创建一个监听模式的 UDP 连接，用于测试 server 场景。 -func newUDPListener(t *testing.T) *UDPConn { - t.Helper() - - addr, err := net.ResolveUDPAddr("udp", "127.0.0.1:0") - if err != nil { - t.Fatalf("ResolveUDPAddr() error = %v", err) - } - - raw, err := net.ListenUDP("udp", addr) - if err != nil { - t.Fatalf("ListenUDP() error = %v", err) - } - - conn, err := NewUDPConn(raw, nil) - if err != nil { - _ = raw.Close() - t.Fatalf("NewUDPConn() error = %v", err) - } - - t.Cleanup(func() { - _ = conn.Close() - }) - - return conn -} - -// newUDPDialed 创建一个已连接到指定地址的 UDP 连接，用于测试 peer 场景。 -func newUDPDialed(t *testing.T, serverAddr *net.UDPAddr) *UDPConn { - t.Helper() - - raw, err := net.DialUDP("udp", nil, serverAddr) - if err != nil { - t.Fatalf("DialUDP() error = %v", err) - } - - conn, err := NewUDPConn(raw, nil) - if err != nil { - _ = raw.Close() - t.Fatalf("NewUDPConn() error = %v", err) - } - - t.Cleanup(func() { - _ = conn.Close() - }) - - return conn -} diff --git a/c/cmd/kcppeer.c b/cmd/kcppeer.c similarity index 100% rename from c/cmd/kcppeer.c rename to cmd/kcppeer.c diff --git a/cmd/kcppeer/interactive.go b/cmd/kcppeer/interactive.go deleted file mode 100644 index 39852b8..0000000 --- a/cmd/kcppeer/interactive.go +++ /dev/null @@ -1,86 +0,0 @@ -package main - -import ( - "errors" - "fmt" - "io" - "strings" -) - -const ( - kcpInteractiveCommandHelp = "help" - kcpInteractiveCommandQuit = "quit" - kcpInteractiveCommandText = "text" - kcpInteractiveCommandFile = "file" -) - -var errKCPEmptyInteractiveCommand = errors.New("interactive command is empty") - -type kcpInteractiveCommand struct { - name string - to string - value string -} - -func parseKCPInteractiveCommand(line string) (kcpInteractiveCommand, error) { - commandName, rest, ok := cutKCPInteractiveField(strings.TrimSpace(line)) - if !ok { - return kcpInteractiveCommand{}, errKCPEmptyInteractiveCommand - } - - switch strings.ToLower(commandName) { - case "help", "h", "?": - return kcpInteractiveCommand{name: kcpInteractiveCommandHelp}, nil - case "quit", "exit": - return kcpInteractiveCommand{name: kcpInteractiveCommandQuit}, nil - case kcpInteractiveCommandText: - to, body, err := parseKCPInteractiveTargetValue(rest, kcpInteractiveCommandText) - if err != nil { - return kcpInteractiveCommand{}, err - } - return kcpInteractiveCommand{name: kcpInteractiveCommandText, to: to, value: body}, nil - case kcpInteractiveCommandFile: - to, path, err := parseKCPInteractiveTargetValue(rest, kcpInteractiveCommandFile) - if err != nil { - return kcpInteractiveCommand{}, err - } - return kcpInteractiveCommand{name: kcpInteractiveCommandFile, to: to, value: path}, nil - default: - return kcpInteractiveCommand{}, fmt.Errorf("unknown command %q; type help for usage", commandName) - } -} - -func parseKCPInteractiveTargetValue(rest, commandName string) (string, string, error) { - to, value, ok := cutKCPInteractiveField(strings.TrimSpace(rest)) - if !ok { - return "", "", fmt.Errorf("%s command requires a target peer and payload", commandName) - } - if strings.TrimSpace(value) == "" { - return "", "", fmt.Errorf("%s command requires a non-empty payload", commandName) - } - - return to, strings.TrimSpace(value), nil -} - -func cutKCPInteractiveField(input string) (string, string, bool) { - trimmed := strings.TrimSpace(input) - if trimmed == "" { - return "", "", false - } - - for i, r := range trimmed { - if r == ' ' || r == '\t' { - return trimmed[:i], strings.TrimSpace(trimmed[i+1:]), true - } - } - - return trimmed, "", true -} - -func printKCPInteractiveHelp(w io.Writer) { - _, _ = fmt.Fprintln(w, "interactive mode commands (KCP):") - _, _ = fmt.Fprintln(w, " help show this help") - _, _ = fmt.Fprintln(w, " text send one text message over the existing KCP session") - _, _ = fmt.Fprintln(w, " file send one file over the existing KCP session") - _, _ = fmt.Fprintln(w, " quit exit this peer process") -} diff --git a/cmd/kcppeer/main.go b/cmd/kcppeer/main.go deleted file mode 100644 index c3b4051..0000000 --- a/cmd/kcppeer/main.go +++ /dev/null @@ -1,213 +0,0 @@ -package main - -import ( - "bufio" - "flag" - "fmt" - "io" - "log" - "os" - - "omnisocketgo/cmd/internal/latencylog" - peerpkg "omnisocketgo/cmd/internal/peer" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -func main() { - peerID := flag.String("id", "peer-a", "peer identity") - serverAddr := flag.String("server", "127.0.0.1:9002", "logical KCP hub address; when -relay-via is set this may differ from the actual UDP dial target") - relayVia := flag.String("relay-via", "", "optional UDP relay address used as the actual KCP dial target") - targetPeer := flag.String("to", "", "optional target peer for one outgoing message") - text := flag.String("text", "", "optional text to send after connecting") - filePath := flag.String("file", "", "optional file path to send after connecting") - bindIP := flag.String("bind-ip", "", "optional local source IP used when dialing the server") - bindDevice := flag.String("bind-device", "", "optional Linux network device used when dialing the server") - inboxDir := flag.String("inbox-dir", "inbox", "directory used to persist received text and file messages") - logPath := flag.String("latency-log", "", "optional JSONL file path for latency timestamp logs") - kcpTimestampDebugLogPath := flag.String("kcp-ts-debug-log", "", "optional JSONL file path for KCP packet kernel timestamp debug records") - kcpSessionStatsLogPath := flag.String("kcp-session-stats-log", "", "optional JSONL file path for KCP session stats records") - kcpSessionStatsInterval := flag.String("kcp-session-stats-interval", transport.DefaultKCPSessionStatsInterval.String(), "sampling interval for KCP session stats, for example 100ms") - interactive := flag.Bool("interactive", true, "enable interactive REPL for repeated text/file sends on the same connection") - flag.Parse() - - statsInterval, err := transport.ParseKCPSessionStatsInterval(*kcpSessionStatsInterval) - if err != nil { - log.Fatalf("parse -kcp-session-stats-interval=%q: %v", *kcpSessionStatsInterval, err) - } - - clientOptions := make([]peerpkg.Option, 0, 6) - if *logPath != "" { - logger, err := latencylog.NewJSONLLogger(*logPath) - if err != nil { - log.Fatalf("create latency logger %s: %v", *logPath, err) - } - defer logger.Close() - clientOptions = append(clientOptions, peerpkg.WithLogger(logger)) - } - if *kcpTimestampDebugLogPath != "" { - logger, err := transport.NewJSONLKCPPacketDebugLogger(*kcpTimestampDebugLogPath) - if err != nil { - log.Fatalf("create kcp packet debug logger %s: %v", *kcpTimestampDebugLogPath, err) - } - defer logger.Close() - clientOptions = append(clientOptions, peerpkg.WithKCPPacketDebugLogger(logger)) - } - if *kcpSessionStatsLogPath != "" { - logger, err := transport.NewJSONLKCPSessionStatsLogger(*kcpSessionStatsLogPath) - if err != nil { - log.Fatalf("create kcp session stats logger %s: %v", *kcpSessionStatsLogPath, err) - } - defer logger.Close() - clientOptions = append(clientOptions, peerpkg.WithKCPSessionStatsLogger(logger, statsInterval)) - } - if *bindIP != "" { - clientOptions = append(clientOptions, peerpkg.WithBindIP(*bindIP)) - } - if *bindDevice != "" { - clientOptions = append(clientOptions, peerpkg.WithBindDevice(*bindDevice)) - } - if *relayVia != "" { - clientOptions = append(clientOptions, peerpkg.WithKCPDialAddress(*relayVia)) - } - - client, err := peerpkg.DialKCP(*serverAddr, *peerID, clientOptions...) - if err != nil { - log.Fatalf("dial kcp server %s: %v", *serverAddr, err) - } - defer client.Close() - - dialTarget := *serverAddr - if *relayVia != "" { - dialTarget = *relayVia - log.Printf("opened KCP session as %s; logical server=%s, actual dial target=%s via relay; register not yet confirmed", client.ID(), *serverAddr, dialTarget) - } else { - log.Printf("opened KCP session as %s; logical server=%s, actual dial target=%s; register not yet confirmed", client.ID(), *serverAddr, dialTarget) - } - - receiveErr := make(chan error, 1) - go func() { - receiveErr <- client.ReceiveLoop(func(msg protocol.Message) error { - switch msg.Type { - case protocol.MessageTypeText: - path, err := client.PersistMessage(msg, *inboxDir) - if err != nil { - return err - } - log.Printf("received text from %s to %s and persisted to %s", msg.From, msg.To, path) - case protocol.MessageTypeFile: - path, err := client.PersistMessage(msg, *inboxDir) - if err != nil { - return err - } - log.Printf("received file from %s to %s: %s (%d bytes) -> %s", msg.From, msg.To, msg.FileName, len(msg.Body), path) - case protocol.MessageTypeError: - log.Printf("received %s from %s to %s: %s", msg.Type, msg.From, msg.To, string(msg.Body)) - default: - log.Printf("received unexpected message type %s from %s", msg.Type, msg.From) - } - return nil - }) - }() - - if *text != "" && *filePath != "" { - log.Fatal("only one of -text or -file may be specified") - } - if (*text != "" || *filePath != "") && *targetPeer == "" { - log.Fatal("flag -to is required when sending text or file") - } - - if *targetPeer != "" && *text != "" { - if err := client.SendText(*targetPeer, *text); err != nil { - log.Fatalf("send text to %s: %v", *targetPeer, err) - } - log.Printf("sent text to %s", *targetPeer) - } - if *targetPeer != "" && *filePath != "" { - if err := client.SendFilePath(*targetPeer, *filePath); err != nil { - log.Fatalf("send file %s to %s: %v", *filePath, *targetPeer, err) - } - log.Printf("sent file %s to %s", *filePath, *targetPeer) - } - - if *interactive { - if err := runKCPInteractiveShell(client, os.Stdin, os.Stdout, receiveErr); err != nil { - log.Printf("interactive shell ended: %v", err) - } - return - } - - if err := <-receiveErr; err != nil { - log.Printf("receive loop ended: %v", err) - } -} - -func runKCPInteractiveShell(client *peerpkg.KCPClient, in io.Reader, out io.Writer, receiveErr <-chan error) error { - printKCPInteractiveHelp(out) - lines, inputErr := readKCPInteractiveLines(in, out, fmt.Sprintf("%s> ", client.ID())) - - for { - select { - case err := <-receiveErr: - return err - case line, ok := <-lines: - if !ok { - return <-inputErr - } - - command, err := parseKCPInteractiveCommand(line) - if err != nil { - if err == errKCPEmptyInteractiveCommand { - continue - } - log.Printf("interactive command error: %v", err) - continue - } - - switch command.name { - case kcpInteractiveCommandHelp: - printKCPInteractiveHelp(out) - case kcpInteractiveCommandQuit: - return nil - case kcpInteractiveCommandText: - if err := client.SendText(command.to, command.value); err != nil { - log.Printf("send text to %s: %v", command.to, err) - continue - } - log.Printf("sent text to %s", command.to) - case kcpInteractiveCommandFile: - if err := client.SendFilePath(command.to, command.value); err != nil { - log.Printf("send file %s to %s: %v", command.value, command.to, err) - continue - } - log.Printf("sent file %s to %s", command.value, command.to) - } - } - } -} - -func readKCPInteractiveLines(in io.Reader, out io.Writer, prompt string) (<-chan string, <-chan error) { - lines := make(chan string) - errs := make(chan error, 1) - - go func() { - defer close(lines) - - scanner := bufio.NewScanner(in) - scanner.Buffer(make([]byte, 0, 1024), 1024*1024) - - for { - if _, err := fmt.Fprint(out, prompt); err != nil { - errs <- err - return - } - if !scanner.Scan() { - errs <- scanner.Err() - return - } - lines <- scanner.Text() - } - }() - - return lines, errs -} diff --git a/c/cmd/kcpping.c b/cmd/kcpping.c similarity index 100% rename from c/cmd/kcpping.c rename to cmd/kcpping.c diff --git a/cmd/kcpping/main.go b/cmd/kcpping/main.go deleted file mode 100644 index 48edd94..0000000 --- a/cmd/kcpping/main.go +++ /dev/null @@ -1,392 +0,0 @@ -package main - -import ( - "encoding/json" - "errors" - "flag" - "fmt" - "io" - "math" - "os" - "sort" - "strings" - "time" -) - -const ( - defaultPeerID = "pinger" - defaultServer = "127.0.0.1:9002" - defaultCount = 100 - defaultInterval = 100 * time.Millisecond - defaultSize = 64 - defaultTimeout = 3 * time.Second - minExpiryPoll = 10 * time.Millisecond - maxExpiryPoll = 100 * time.Millisecond -) - -type config struct { - id string - server string - to string - echo bool - count int - interval time.Duration - size int - timeout time.Duration - bindIP string - bindDevice string - latencyLog string -} - -type pingPayload struct { - Seq uint64 `json:"seq"` - TSUnixNano int64 `json:"ts_ns"` - Pad string `json:"pad"` -} - -type pendingPing struct { - deadline time.Time -} - -type replyDisposition int - -const ( - replyMatched replyDisposition = iota - replyDuplicate - replyUnexpected -) - -type replyResult struct { - disposition replyDisposition - rtt time.Duration -} - -type pingTracker struct { - timeout time.Duration - sent int - duplicates int - pending map[uint64]pendingPing - seen map[uint64]struct{} - samples []time.Duration -} - -type rttSummary struct { - Sent int - Received int - Duplicates int - LossPct float64 - Min time.Duration - Avg time.Duration - Max time.Duration - P50 time.Duration - P95 time.Duration - P99 time.Duration - StdDev time.Duration - HasSamples bool -} - -func main() { - if err := runMain(os.Args[1:], os.Stdout, os.Stderr, time.Now); err != nil { - if errors.Is(err, flag.ErrHelp) { - return - } - _, _ = fmt.Fprintln(os.Stderr, err) - os.Exit(1) - } -} - -func runMain(args []string, stdout, stderr io.Writer, now func() time.Time) error { - cfg, err := parseConfig(args, stderr) - if err != nil { - return err - } - - return runPlatform(cfg, stdout, stderr, now) -} - -func parseConfig(args []string, stderr io.Writer) (config, error) { - cfg := config{} - - flags := flag.NewFlagSet("kcpping", flag.ContinueOnError) - flags.SetOutput(stderr) - flags.StringVar(&cfg.id, "id", defaultPeerID, "local peer identity") - flags.StringVar(&cfg.server, "server", defaultServer, "KCP server address") - flags.StringVar(&cfg.to, "to", "", "target peer identity in ping mode") - flags.BoolVar(&cfg.echo, "echo", false, "echo back every received text message") - flags.IntVar(&cfg.count, "count", defaultCount, "number of pings to send; 0 means run until interrupted") - flags.DurationVar(&cfg.interval, "interval", defaultInterval, "delay between ping sends") - flags.IntVar(&cfg.size, "size", defaultSize, "application payload size in bytes") - flags.DurationVar(&cfg.timeout, "timeout", defaultTimeout, "per-ping timeout") - flags.StringVar(&cfg.bindIP, "bind-ip", "", "optional local source IP used when dialing the server") - flags.StringVar(&cfg.bindDevice, "bind-device", "", "optional Linux network device used when dialing the server") - flags.StringVar(&cfg.latencyLog, "latency-log", "", "optional JSONL file path for latency timestamp logs") - - if err := flags.Parse(args); err != nil { - return config{}, err - } - if flags.NArg() > 0 { - return config{}, fmt.Errorf("unexpected positional arguments: %s", strings.Join(flags.Args(), " ")) - } - - cfg.id = strings.TrimSpace(cfg.id) - cfg.server = strings.TrimSpace(cfg.server) - cfg.to = strings.TrimSpace(cfg.to) - cfg.bindIP = strings.TrimSpace(cfg.bindIP) - cfg.bindDevice = strings.TrimSpace(cfg.bindDevice) - cfg.latencyLog = strings.TrimSpace(cfg.latencyLog) - - if err := cfg.validate(); err != nil { - return config{}, err - } - return cfg, nil -} - -func (c config) validate() error { - if c.id == "" { - return fmt.Errorf("flag -id is required") - } - if c.server == "" { - return fmt.Errorf("flag -server is required") - } - if !c.echo && c.to == "" { - return fmt.Errorf("flag -to is required unless -echo is set") - } - if c.count < 0 { - return fmt.Errorf("flag -count must be greater than or equal to zero") - } - if c.interval <= 0 { - return fmt.Errorf("flag -interval must be greater than zero") - } - if c.size <= 0 { - return fmt.Errorf("flag -size must be greater than zero") - } - if c.timeout <= 0 { - return fmt.Errorf("flag -timeout must be greater than zero") - } - return nil -} - -func buildPingPayload(seq uint64, tsUnixNano int64, size int) ([]byte, error) { - payload := pingPayload{ - Seq: seq, - TSUnixNano: tsUnixNano, - Pad: "", - } - - body, err := json.Marshal(payload) - if err != nil { - return nil, fmt.Errorf("encode ping payload: %w", err) - } - if len(body) > size { - return nil, fmt.Errorf("requested payload size %d is too small; minimum is %d", size, len(body)) - } - - payload.Pad = strings.Repeat("A", size-len(body)) - body, err = json.Marshal(payload) - if err != nil { - return nil, fmt.Errorf("encode padded ping payload: %w", err) - } - if len(body) != size { - return nil, fmt.Errorf("encode padded ping payload: got %d bytes, want %d", len(body), size) - } - return body, nil -} - -func parsePingPayload(body []byte) (pingPayload, error) { - var payload pingPayload - if err := json.Unmarshal(body, &payload); err != nil { - return pingPayload{}, fmt.Errorf("decode ping payload: %w", err) - } - if payload.Seq == 0 { - return pingPayload{}, fmt.Errorf("decode ping payload: seq must be greater than zero") - } - if payload.TSUnixNano <= 0 { - return pingPayload{}, fmt.Errorf("decode ping payload: ts_ns must be greater than zero") - } - return payload, nil -} - -func newPingTracker(timeout time.Duration) *pingTracker { - return &pingTracker{ - timeout: timeout, - pending: make(map[uint64]pendingPing), - seen: make(map[uint64]struct{}), - } -} - -func (t *pingTracker) markSent(seq uint64, sentAt time.Time) { - t.sent++ - t.pending[seq] = pendingPing{deadline: sentAt.Add(t.timeout)} - t.seen[seq] = struct{}{} -} - -func (t *pingTracker) observeReply(payload pingPayload, receivedAt time.Time) replyResult { - if _, ok := t.seen[payload.Seq]; !ok { - return replyResult{disposition: replyUnexpected} - } - - if _, ok := t.pending[payload.Seq]; !ok { - t.duplicates++ - return replyResult{disposition: replyDuplicate} - } - - delete(t.pending, payload.Seq) - rtt := receivedAt.Sub(time.Unix(0, payload.TSUnixNano)) - if rtt < 0 { - rtt = 0 - } - t.samples = append(t.samples, rtt) - return replyResult{ - disposition: replyMatched, - rtt: rtt, - } -} - -func (t *pingTracker) expire(now time.Time) []uint64 { - expired := make([]uint64, 0) - for seq, pending := range t.pending { - if !pending.deadline.After(now) { - expired = append(expired, seq) - delete(t.pending, seq) - } - } - sort.Slice(expired, func(i, j int) bool { - return expired[i] < expired[j] - }) - return expired -} - -func (t *pingTracker) pendingCount() int { - return len(t.pending) -} - -func (t *pingTracker) summary() rttSummary { - return calculateRTTSummary(t.samples, t.sent, t.duplicates) -} - -func calculateRTTSummary(samples []time.Duration, sent, duplicates int) rttSummary { - summary := rttSummary{ - Sent: sent, - Received: len(samples), - Duplicates: duplicates, - } - if sent > 0 { - summary.LossPct = float64(sent-len(samples)) * 100 / float64(sent) - } - if len(samples) == 0 { - return summary - } - - sorted := append([]time.Duration(nil), samples...) - sort.Slice(sorted, func(i, j int) bool { - return sorted[i] < sorted[j] - }) - - var sum float64 - for _, sample := range sorted { - sum += float64(sample) - } - avg := sum / float64(len(sorted)) - - var variance float64 - for _, sample := range sorted { - delta := float64(sample) - avg - variance += delta * delta - } - variance /= float64(len(sorted)) - - summary.Min = sorted[0] - summary.Avg = time.Duration(math.Round(avg)) - summary.Max = sorted[len(sorted)-1] - summary.P50 = percentileDuration(sorted, 0.50) - summary.P95 = percentileDuration(sorted, 0.95) - summary.P99 = percentileDuration(sorted, 0.99) - summary.StdDev = time.Duration(math.Round(math.Sqrt(variance))) - summary.HasSamples = true - return summary -} - -func percentileDuration(sorted []time.Duration, percentile float64) time.Duration { - if len(sorted) == 0 { - return 0 - } - if percentile <= 0 { - return sorted[0] - } - if percentile >= 1 { - return sorted[len(sorted)-1] - } - - index := int(math.Ceil(percentile*float64(len(sorted)))) - 1 - if index < 0 { - index = 0 - } - if index >= len(sorted) { - index = len(sorted) - 1 - } - return sorted[index] -} - -func formatRTT(duration time.Duration) string { - return fmt.Sprintf("%.2fms", float64(duration)/float64(time.Millisecond)) -} - -func writePingHeader(w io.Writer, cfg config) error { - _, err := fmt.Fprintf(w, "KCP PING %s via %s (payload=%d bytes, KCP)\n", cfg.to, cfg.server, cfg.size) - return err -} - -func writeMatchedReply(w io.Writer, seq uint64, rtt time.Duration) error { - _, err := fmt.Fprintf(w, "seq=%d rtt=%s\n", seq, formatRTT(rtt)) - return err -} - -func writeTimeout(w io.Writer, seq uint64) error { - _, err := fmt.Fprintf(w, "seq=%d timeout\n", seq) - return err -} - -func writeSummary(w io.Writer, target string, summary rttSummary) error { - if _, err := fmt.Fprintf(w, "--- %s kcp ping statistics ---\n", target); err != nil { - return err - } - if _, err := fmt.Fprintf( - w, - "%d packets transmitted, %d received, %d duplicates, %.2f%% packet loss\n", - summary.Sent, - summary.Received, - summary.Duplicates, - summary.LossPct, - ); err != nil { - return err - } - - if !summary.HasSamples { - _, err := fmt.Fprintln(w, "rtt min/avg/max/p50/p95/p99 = n/a/n/a/n/a/n/a/n/a/n/a, stddev=n/a") - return err - } - - _, err := fmt.Fprintf( - w, - "rtt min/avg/max/p50/p95/p99 = %s/%s/%s/%s/%s/%s, stddev=%s\n", - formatRTT(summary.Min), - formatRTT(summary.Avg), - formatRTT(summary.Max), - formatRTT(summary.P50), - formatRTT(summary.P95), - formatRTT(summary.P99), - formatRTT(summary.StdDev), - ) - return err -} - -func expiryPollInterval(timeout time.Duration) time.Duration { - interval := timeout / 4 - if interval < minExpiryPoll { - return minExpiryPoll - } - if interval > maxExpiryPoll { - return maxExpiryPoll - } - return interval -} diff --git a/cmd/kcpping/main_test.go b/cmd/kcpping/main_test.go deleted file mode 100644 index 48a2dd2..0000000 --- a/cmd/kcpping/main_test.go +++ /dev/null @@ -1,196 +0,0 @@ -package main - -import ( - "bytes" - "strings" - "testing" - "time" -) - -func TestParseConfigDefaults(t *testing.T) { - cfg, err := parseConfig([]string{"-to", "peer-b"}, ioDiscard{}) - if err != nil { - t.Fatalf("parseConfig() error = %v", err) - } - - if cfg.id != defaultPeerID { - t.Fatalf("id = %q, want %q", cfg.id, defaultPeerID) - } - if cfg.server != defaultServer { - t.Fatalf("server = %q, want %q", cfg.server, defaultServer) - } - if cfg.count != defaultCount { - t.Fatalf("count = %d, want %d", cfg.count, defaultCount) - } - if cfg.interval != defaultInterval { - t.Fatalf("interval = %s, want %s", cfg.interval, defaultInterval) - } - if cfg.size != defaultSize { - t.Fatalf("size = %d, want %d", cfg.size, defaultSize) - } - if cfg.timeout != defaultTimeout { - t.Fatalf("timeout = %s, want %s", cfg.timeout, defaultTimeout) - } -} - -func TestParseConfigRequiresTargetInPingMode(t *testing.T) { - _, err := parseConfig([]string{"-echo=false"}, ioDiscard{}) - if err == nil || !strings.Contains(err.Error(), "flag -to is required") { - t.Fatalf("parseConfig() error = %v, want missing -to error", err) - } -} - -func TestParseConfigAllowsEchoWithoutTarget(t *testing.T) { - cfg, err := parseConfig([]string{"-echo"}, ioDiscard{}) - if err != nil { - t.Fatalf("parseConfig() error = %v", err) - } - if !cfg.echo { - t.Fatal("echo = false, want true") - } -} - -func TestBuildPingPayloadUsesExactSize(t *testing.T) { - body, err := buildPingPayload(7, 123456789, 96) - if err != nil { - t.Fatalf("buildPingPayload() error = %v", err) - } - if len(body) != 96 { - t.Fatalf("len(body) = %d, want 96", len(body)) - } - - payload, err := parsePingPayload(body) - if err != nil { - t.Fatalf("parsePingPayload() error = %v", err) - } - if payload.Seq != 7 { - t.Fatalf("seq = %d, want 7", payload.Seq) - } - if payload.TSUnixNano != 123456789 { - t.Fatalf("ts_ns = %d, want 123456789", payload.TSUnixNano) - } -} - -func TestBuildPingPayloadRejectsTooSmallSize(t *testing.T) { - _, err := buildPingPayload(1, 123456789, 8) - if err == nil || !strings.Contains(err.Error(), "too small") { - t.Fatalf("buildPingPayload() error = %v, want size too small error", err) - } -} - -func TestParsePingPayloadRejectsInvalidJSON(t *testing.T) { - _, err := parsePingPayload([]byte("not-json")) - if err == nil || !strings.Contains(err.Error(), "decode ping payload") { - t.Fatalf("parsePingPayload() error = %v, want decode error", err) - } -} - -func TestPingTrackerHandlesMatchedDuplicateAndTimeout(t *testing.T) { - tracker := newPingTracker(50 * time.Millisecond) - sentAt := time.Unix(0, 100) - tracker.markSent(1, sentAt) - - match := tracker.observeReply(pingPayload{Seq: 1, TSUnixNano: sentAt.UnixNano()}, sentAt.Add(12*time.Millisecond)) - if match.disposition != replyMatched { - t.Fatalf("first disposition = %v, want matched", match.disposition) - } - if match.rtt != 12*time.Millisecond { - t.Fatalf("first rtt = %s, want 12ms", match.rtt) - } - - duplicate := tracker.observeReply(pingPayload{Seq: 1, TSUnixNano: sentAt.UnixNano()}, sentAt.Add(20*time.Millisecond)) - if duplicate.disposition != replyDuplicate { - t.Fatalf("second disposition = %v, want duplicate", duplicate.disposition) - } - - tracker.markSent(2, sentAt) - expired := tracker.expire(sentAt.Add(60 * time.Millisecond)) - if len(expired) != 1 || expired[0] != 2 { - t.Fatalf("expired = %v, want [2]", expired) - } - - late := tracker.observeReply(pingPayload{Seq: 2, TSUnixNano: sentAt.UnixNano()}, sentAt.Add(70*time.Millisecond)) - if late.disposition != replyDuplicate { - t.Fatalf("late disposition = %v, want duplicate", late.disposition) - } - - unexpected := tracker.observeReply(pingPayload{Seq: 99, TSUnixNano: sentAt.UnixNano()}, sentAt.Add(80*time.Millisecond)) - if unexpected.disposition != replyUnexpected { - t.Fatalf("unexpected disposition = %v, want unexpected", unexpected.disposition) - } -} - -func TestCalculateRTTSummary(t *testing.T) { - summary := calculateRTTSummary( - []time.Duration{ - 10 * time.Millisecond, - 20 * time.Millisecond, - 30 * time.Millisecond, - 40 * time.Millisecond, - 50 * time.Millisecond, - }, - 6, - 2, - ) - - if summary.Sent != 6 { - t.Fatalf("Sent = %d, want 6", summary.Sent) - } - if summary.Received != 5 { - t.Fatalf("Received = %d, want 5", summary.Received) - } - if summary.Duplicates != 2 { - t.Fatalf("Duplicates = %d, want 2", summary.Duplicates) - } - if summary.LossPct != (float64(1) * 100 / 6) { - t.Fatalf("LossPct = %f, want %f", summary.LossPct, float64(1)*100/6) - } - if summary.Min != 10*time.Millisecond { - t.Fatalf("Min = %s, want 10ms", summary.Min) - } - if summary.Avg != 30*time.Millisecond { - t.Fatalf("Avg = %s, want 30ms", summary.Avg) - } - if summary.Max != 50*time.Millisecond { - t.Fatalf("Max = %s, want 50ms", summary.Max) - } - if summary.P50 != 30*time.Millisecond { - t.Fatalf("P50 = %s, want 30ms", summary.P50) - } - if summary.P95 != 50*time.Millisecond { - t.Fatalf("P95 = %s, want 50ms", summary.P95) - } - if summary.P99 != 50*time.Millisecond { - t.Fatalf("P99 = %s, want 50ms", summary.P99) - } - if summary.StdDev == 0 { - t.Fatal("StdDev = 0, want non-zero") - } -} - -func TestWriteSummaryUsesNAWithoutSamples(t *testing.T) { - var buf bytes.Buffer - err := writeSummary(&buf, "host", rttSummary{ - Sent: 3, - Received: 0, - Duplicates: 1, - LossPct: 100, - }) - if err != nil { - t.Fatalf("writeSummary() error = %v", err) - } - - out := buf.String() - if !strings.Contains(out, "3 packets transmitted, 0 received, 1 duplicates, 100.00% packet loss") { - t.Fatalf("summary output missing counters: %q", out) - } - if !strings.Contains(out, "n/a/n/a/n/a/n/a/n/a/n/a") { - t.Fatalf("summary output missing n/a metrics: %q", out) - } -} - -type ioDiscard struct{} - -func (ioDiscard) Write(p []byte) (int, error) { - return len(p), nil -} diff --git a/cmd/kcpping/platform_linux.go b/cmd/kcpping/platform_linux.go deleted file mode 100644 index 1eebe87..0000000 --- a/cmd/kcpping/platform_linux.go +++ /dev/null @@ -1,246 +0,0 @@ -//go:build linux - -package main - -import ( - "context" - "errors" - "fmt" - "io" - "os" - "os/signal" - "strings" - "time" - - "omnisocketgo/cmd/internal/latencylog" - peerpkg "omnisocketgo/cmd/internal/peer" - "omnisocketgo/cmd/internal/protocol" -) - -func runPlatform(cfg config, stdout, stderr io.Writer, now func() time.Time) error { - ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt) - defer stop() - - client, closeLogger, err := dialKCPClient(cfg) - if err != nil { - return err - } - defer closeLogger() - defer client.Close() - - if cfg.echo { - return runEchoMode(ctx, client, stderr) - } - return runPingMode(ctx, client, cfg, stdout, stderr, now) -} - -func dialKCPClient(cfg config) (*peerpkg.KCPClient, func(), error) { - options := make([]peerpkg.Option, 0, 3) - closeLogger := func() {} - - if cfg.latencyLog != "" { - logger, err := latencylog.NewJSONLLogger(cfg.latencyLog) - if err != nil { - return nil, nil, fmt.Errorf("create latency logger %s: %w", cfg.latencyLog, err) - } - options = append(options, peerpkg.WithLogger(logger)) - closeLogger = func() { - _ = logger.Close() - } - } - if cfg.bindIP != "" { - options = append(options, peerpkg.WithBindIP(cfg.bindIP)) - } - if cfg.bindDevice != "" { - options = append(options, peerpkg.WithBindDevice(cfg.bindDevice)) - } - - client, err := peerpkg.DialKCP(cfg.server, cfg.id, options...) - if err != nil { - closeLogger() - return nil, nil, fmt.Errorf("dial kcp server %s: %w", cfg.server, err) - } - return client, closeLogger, nil -} - -func runPingMode(ctx context.Context, client *peerpkg.KCPClient, cfg config, stdout, stderr io.Writer, now func() time.Time) error { - if err := writePingHeader(stdout, cfg); err != nil { - return err - } - - receiveCh := make(chan protocol.Message, 32) - receiveErrCh := make(chan error, 1) - go func() { - for { - msg, err := client.Receive() - if err != nil { - receiveErrCh <- err - return - } - receiveCh <- msg - } - }() - - tracker := newPingTracker(cfg.timeout) - expiryTicker := time.NewTicker(expiryPollInterval(cfg.timeout)) - defer expiryTicker.Stop() - - var sendTicker *time.Ticker - if cfg.count == 0 || cfg.count > 1 { - sendTicker = time.NewTicker(cfg.interval) - defer sendTicker.Stop() - } - - nextSeq := uint64(1) - if err := sendPing(client, tracker, cfg, nextSeq, now); err != nil { - return err - } - nextSeq++ - - stopSending := cfg.count == 1 - receiveErrSeen := false - - for { - if stopSending && tracker.pendingCount() == 0 { - break - } - - var sendTick <-chan time.Time - if !stopSending && sendTicker != nil { - sendTick = sendTicker.C - } - - select { - case <-ctx.Done(): - stopSending = true - case <-expiryTicker.C: - for _, seq := range tracker.expire(now()) { - if err := writeTimeout(stdout, seq); err != nil { - return err - } - } - case msg := <-receiveCh: - if err := handlePingMessage(tracker, msg, stdout, stderr, now); err != nil { - return err - } - case err := <-receiveErrCh: - receiveErrSeen = true - if ctx.Err() != nil && isExpectedCloseError(err) { - break - } - if stopSending && tracker.pendingCount() == 0 && isExpectedCloseError(err) { - break - } - return fmt.Errorf("receive reply: %w", err) - case <-sendTick: - if cfg.count > 0 && tracker.sent >= cfg.count { - stopSending = true - continue - } - if err := sendPing(client, tracker, cfg, nextSeq, now); err != nil { - return err - } - nextSeq++ - if cfg.count > 0 && tracker.sent >= cfg.count { - stopSending = true - } - } - - if receiveErrSeen && stopSending && tracker.pendingCount() == 0 { - break - } - } - - for _, seq := range tracker.expire(now()) { - if err := writeTimeout(stdout, seq); err != nil { - return err - } - } - return writeSummary(stdout, cfg.to, tracker.summary()) -} - -func sendPing(client *peerpkg.KCPClient, tracker *pingTracker, cfg config, seq uint64, now func() time.Time) error { - sentAt := now() - payload, err := buildPingPayload(seq, sentAt.UnixNano(), cfg.size) - if err != nil { - return err - } - if err := client.SendText(cfg.to, string(payload)); err != nil { - return fmt.Errorf("send ping seq=%d: %w", seq, err) - } - tracker.markSent(seq, sentAt) - return nil -} - -func handlePingMessage(tracker *pingTracker, msg protocol.Message, stdout, stderr io.Writer, now func() time.Time) error { - switch msg.Type { - case protocol.MessageTypeText: - payload, err := parsePingPayload(msg.Body) - if err != nil { - _, writeErr := fmt.Fprintf(stderr, "ignore non-ping text message from %s: %v\n", msg.From, err) - if writeErr != nil { - return writeErr - } - return nil - } - - result := tracker.observeReply(payload, now()) - switch result.disposition { - case replyMatched: - return writeMatchedReply(stdout, payload.Seq, result.rtt) - case replyDuplicate: - _, err := fmt.Fprintf(stderr, "seq=%d duplicate or late reply ignored\n", payload.Seq) - return err - case replyUnexpected: - _, err := fmt.Fprintf(stderr, "seq=%d unexpected reply ignored\n", payload.Seq) - return err - default: - return nil - } - case protocol.MessageTypeError: - _, err := fmt.Fprintf(stderr, "server error: %s\n", strings.TrimSpace(string(msg.Body))) - return err - default: - _, err := fmt.Fprintf(stderr, "unexpected message type %s from %s ignored\n", msg.Type, msg.From) - return err - } -} - -func runEchoMode(ctx context.Context, client *peerpkg.KCPClient, stderr io.Writer) error { - receiveErrCh := make(chan error, 1) - go func() { - receiveErrCh <- client.ReceiveLoop(func(msg protocol.Message) error { - switch msg.Type { - case protocol.MessageTypeText: - return client.SendText(msg.From, string(msg.Body)) - case protocol.MessageTypeError: - _, err := fmt.Fprintf(stderr, "server error: %s\n", strings.TrimSpace(string(msg.Body))) - return err - default: - _, err := fmt.Fprintf(stderr, "unexpected message type %s from %s ignored\n", msg.Type, msg.From) - return err - } - }) - }() - - select { - case <-ctx.Done(): - return nil - case err := <-receiveErrCh: - if err == nil || (ctx.Err() != nil && isExpectedCloseError(err)) { - return nil - } - return fmt.Errorf("echo receive loop: %w", err) - } -} - -func isExpectedCloseError(err error) bool { - if err == nil { - return true - } - message := err.Error() - return errors.Is(err, context.Canceled) || - strings.Contains(message, "closed") || - strings.Contains(message, "broken pipe") || - strings.Contains(message, "io: read/write on closed pipe") -} diff --git a/cmd/kcpping/platform_other.go b/cmd/kcpping/platform_other.go deleted file mode 100644 index 8483cd9..0000000 --- a/cmd/kcpping/platform_other.go +++ /dev/null @@ -1,14 +0,0 @@ -//go:build !linux - -package main - -import ( - "fmt" - "io" - "runtime" - "time" -) - -func runPlatform(cfg config, stdout, stderr io.Writer, now func() time.Time) error { - return fmt.Errorf("kcpping is only supported on linux; current GOOS=%s", runtime.GOOS) -} diff --git a/c/cmd/kcpserver.c b/cmd/kcpserver.c similarity index 100% rename from c/cmd/kcpserver.c rename to cmd/kcpserver.c diff --git a/cmd/kcpserver/main.go b/cmd/kcpserver/main.go deleted file mode 100644 index e45621b..0000000 --- a/cmd/kcpserver/main.go +++ /dev/null @@ -1,186 +0,0 @@ -package main - -import ( - "flag" - "log" - "net" - "strings" - "time" - - kcp "github.com/xtaci/kcp-go/v5" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/server" - "omnisocketgo/cmd/internal/transport" -) - -const ( - kcpServerModeHub = "hub" - kcpServerModeRelay = "relay" -) - -func main() { - mode := flag.String("mode", kcpServerModeHub, "kcpserver mode: hub or relay") - listenAddr := flag.String("listen", ":9002", "listen address; KCP listener in hub mode, UDP relay listener in relay mode") - bindDevice := flag.String("bind-device", "", "optional Linux network device used when listening") - logPath := flag.String("latency-log", "", "optional JSONL file path for latency timestamp logs") - kcpTimestampDebugLogPath := flag.String("kcp-ts-debug-log", "", "optional JSONL file path for KCP packet kernel timestamp debug records") - kcpSessionStatsLogPath := flag.String("kcp-session-stats-log", "", "optional JSONL file path for KCP session stats records") - kcpSessionStatsInterval := flag.String("kcp-session-stats-interval", transport.DefaultKCPSessionStatsInterval.String(), "sampling interval for KCP session stats, for example 100ms") - relayListenAddr := flag.String("relay-listen", "", "deprecated alias for -listen in relay mode") - relayRemoteAddr := flag.String("relay-remote", "", "fixed remote UDP address used in relay mode") - relayPeerAddr := flag.String("relay-peer", "", "deprecated alias for -relay-remote") - flag.Parse() - - var relayRemoteFlagSet bool - var relayPeerFlagSet bool - var relayListenFlagSet bool - flag.Visit(func(f *flag.Flag) { - switch f.Name { - case "relay-listen": - relayListenFlagSet = true - case "relay-remote": - relayRemoteFlagSet = true - case "relay-peer": - relayPeerFlagSet = true - } - }) - - switch { - case relayRemoteFlagSet && relayPeerFlagSet && *relayRemoteAddr != *relayPeerAddr: - log.Fatal("flags -relay-remote and -relay-peer must match when both are set") - case *relayRemoteAddr == "" && *relayPeerAddr != "": - *relayRemoteAddr = *relayPeerAddr - } - if relayPeerFlagSet { - log.Printf("warning: flag -relay-peer is deprecated; use -relay-remote instead") - } - if relayListenFlagSet { - if *relayListenAddr == "" { - log.Fatal("flag -relay-listen must not be empty when set") - } - if *mode != kcpServerModeRelay { - log.Fatal("flag -relay-listen may only be used in relay mode") - } - if *listenAddr != ":9002" && *listenAddr != *relayListenAddr { - log.Fatal("flags -listen and -relay-listen must match when both are set in relay mode") - } - *listenAddr = *relayListenAddr - log.Printf("warning: flag -relay-listen is deprecated; use -listen with -mode=relay instead") - } - - statsInterval, err := transport.ParseKCPSessionStatsInterval(*kcpSessionStatsInterval) - if err != nil { - log.Fatalf("parse -kcp-session-stats-interval=%q: %v", *kcpSessionStatsInterval, err) - } - - switch *mode { - case kcpServerModeHub: - if *relayRemoteAddr != "" { - log.Fatal("flag -relay-remote may only be used in relay mode") - } - runHubServer(*listenAddr, *bindDevice, *logPath, *kcpTimestampDebugLogPath, *kcpSessionStatsLogPath, statsInterval) - case kcpServerModeRelay: - if *bindDevice != "" { - log.Fatal("flag -bind-device is not supported in relay mode") - } - if *relayRemoteAddr == "" { - log.Fatal("flag -relay-remote is required in relay mode") - } - runUDPRelayServer(*listenAddr, *relayRemoteAddr) - default: - log.Fatalf("unsupported -mode=%q; want %q or %q", *mode, kcpServerModeHub, kcpServerModeRelay) - } -} - -func runHubServer(listenAddr, bindDevice, logPath, packetDebugLogPath, sessionStatsLogPath string, statsInterval time.Duration) { - listenNetwork, _, err := transport.ResolveUDPListenConfig(listenAddr) - if err != nil { - log.Fatalf("resolve kcp listen address %s: %v", listenAddr, err) - } - - hubOptions := make([]server.KCPOption, 0, 2) - if logPath != "" { - logger, err := latencylog.NewJSONLLogger(logPath) - if err != nil { - log.Fatalf("create latency logger %s: %v", logPath, err) - } - defer logger.Close() - hubOptions = append(hubOptions, server.WithKCPLogger(logger)) - } - - var packetLogger transport.KCPPacketDebugLogger - if packetDebugLogPath != "" { - logger, err := transport.NewJSONLKCPPacketDebugLogger(packetDebugLogPath) - if err != nil { - log.Fatalf("create kcp packet debug logger %s: %v", packetDebugLogPath, err) - } - defer logger.Close() - packetLogger = logger - } - if sessionStatsLogPath != "" { - logger, err := transport.NewJSONLKCPSessionStatsLogger(sessionStatsLogPath) - if err != nil { - log.Fatalf("create kcp session stats logger %s: %v", sessionStatsLogPath, err) - } - defer logger.Close() - hubOptions = append(hubOptions, server.WithKCPSessionStatsLogger(logger, statsInterval)) - } - - listener, packetConn, err := transport.ListenKCPSessions(listenAddr, bindDevice, packetLogger, latencylog.NodeRoleServer, "hub") - if err != nil { - log.Fatalf("listen kcp on %s: %v", listenAddr, err) - } - defer packetConn.Close() - defer listener.Close() - - hub := server.NewKCPHub(hubOptions...) - - log.Printf("kcp hub listening on %s %s", listenNetwork, packetConn.LocalAddr()) - - for { - session, err := listener.AcceptKCP() - if err != nil { - if strings.Contains(err.Error(), "closed") { - return - } - log.Printf("accept kcp session: %v", err) - continue - } - - go func(sess *kcp.UDPSession) { - if serveErr := hub.ServeSession(sess); serveErr != nil { - log.Printf("kcp session closed: %v", serveErr) - } - }(session) - } -} - -func runUDPRelayServer(listenAddr, remoteAddr string) { - listenNetwork, udpListenAddr, err := transport.ResolveUDPListenConfig(listenAddr) - if err != nil { - log.Fatalf("resolve udp relay listen address %s: %v", listenAddr, err) - } - - conn, err := net.ListenPacket(listenNetwork, udpListenAddr.String()) - if err != nil { - log.Fatalf("listen %s relay on %s: %v", listenNetwork, udpListenAddr, err) - } - defer conn.Close() - - remote, err := net.ResolveUDPAddr("udp", remoteAddr) - if err != nil { - log.Fatalf("resolve relay remote %s: %v", remoteAddr, err) - } - - relay, err := server.NewUDPRelay(conn, remote) - if err != nil { - _ = conn.Close() - log.Fatalf("create udp relay: %v", err) - } - - log.Printf("udp relay listening on %s %s and forwarding to %s", listenNetwork, conn.LocalAddr(), remote) - if err := relay.Serve(); err != nil { - log.Fatalf("udp relay stopped: %v", err) - } -} diff --git a/cmd/peer/interactive.go b/cmd/peer/interactive.go deleted file mode 100644 index 8137bc0..0000000 --- a/cmd/peer/interactive.go +++ /dev/null @@ -1,89 +0,0 @@ -package main - -import ( - "errors" - "fmt" - "io" - "strings" -) - -const ( - interactiveCommandHelp = "help" - interactiveCommandQuit = "quit" - interactiveCommandText = "text" - interactiveCommandFile = "file" -) - -// 交互式命令行界面，允许用户在连接建立后反复发送文本或文件消息。 -var errEmptyInteractiveCommand = errors.New("interactive command is empty") - -type interactiveCommand struct { - name string - to string - value string -} - -// 解析用户输入的交互式命令，支持发送文本或文件消息，以及查看帮助和退出。 -func parseInteractiveCommand(line string) (interactiveCommand, error) { - commandName, rest, ok := cutInteractiveField(strings.TrimSpace(line)) - if !ok { - return interactiveCommand{}, errEmptyInteractiveCommand - } - - switch strings.ToLower(commandName) { - case "help", "h", "?": - return interactiveCommand{name: interactiveCommandHelp}, nil - case "quit", "exit": - return interactiveCommand{name: interactiveCommandQuit}, nil - case interactiveCommandText: - to, body, err := parseInteractiveTargetValue(rest, interactiveCommandText) - if err != nil { - return interactiveCommand{}, err - } - return interactiveCommand{name: interactiveCommandText, to: to, value: body}, nil - case interactiveCommandFile: - to, path, err := parseInteractiveTargetValue(rest, interactiveCommandFile) - if err != nil { - return interactiveCommand{}, err - } - return interactiveCommand{name: interactiveCommandFile, to: to, value: path}, nil - default: - return interactiveCommand{}, fmt.Errorf("unknown command %q; type help for usage", commandName) - } -} - -func parseInteractiveTargetValue(rest, commandName string) (string, string, error) { - to, value, ok := cutInteractiveField(strings.TrimSpace(rest)) - if !ok { - return "", "", fmt.Errorf("%s command requires a target peer and payload", commandName) - } - if strings.TrimSpace(value) == "" { - return "", "", fmt.Errorf("%s command requires a non-empty payload", commandName) - } - - return to, strings.TrimSpace(value), nil -} - -func cutInteractiveField(input string) (string, string, bool) { - trimmed := strings.TrimSpace(input) - if trimmed == "" { - return "", "", false - } - - for i, r := range trimmed { - if r == ' ' || r == '\t' { - return trimmed[:i], strings.TrimSpace(trimmed[i+1:]), true - } - } - - return trimmed, "", true -} - -// 打印交互式命令帮助信息，列出可用的命令和用法说明。 -func printInteractiveHelp(w io.Writer) { - _, _ = fmt.Fprintln(w, "interactive mode commands:") - _, _ = fmt.Fprintln(w, " help show this help") - _, _ = fmt.Fprintln(w, " text send one text message over the existing connection") - _, _ = fmt.Fprintln(w, " file send one file over the existing connection") - _, _ = fmt.Fprintln(w, " quit exit this peer process") -} diff --git a/cmd/peer/interactive_test.go b/cmd/peer/interactive_test.go deleted file mode 100644 index 74abf87..0000000 --- a/cmd/peer/interactive_test.go +++ /dev/null @@ -1,87 +0,0 @@ -package main - -import "testing" - -func TestParseInteractiveCommand(t *testing.T) { - tests := []struct { - name string - line string - want interactiveCommand - wantErr string - }{ - { - name: "text command preserves spaces in body", - line: "text peer-b hello over the same connection", - want: interactiveCommand{ - name: interactiveCommandText, - to: "peer-b", - value: "hello over the same connection", - }, - }, - { - name: "file command preserves spaces in path", - line: "file peer-b /tmp/demo payload.bin", - want: interactiveCommand{ - name: interactiveCommandFile, - to: "peer-b", - value: "/tmp/demo payload.bin", - }, - }, - { - name: "help alias", - line: "?", - want: interactiveCommand{ - name: interactiveCommandHelp, - }, - }, - { - name: "quit alias", - line: "exit", - want: interactiveCommand{ - name: interactiveCommandQuit, - }, - }, - { - name: "empty command", - line: " ", - wantErr: errEmptyInteractiveCommand.Error(), - }, - { - name: "text requires payload", - line: "text peer-b", - wantErr: "text command requires a non-empty payload", - }, - { - name: "file requires target and payload", - line: "file", - wantErr: "file command requires a target peer and payload", - }, - { - name: "unknown command", - line: "ping peer-b", - wantErr: `unknown command "ping"; type help for usage`, - }, - } - - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - got, err := parseInteractiveCommand(tt.line) - if tt.wantErr != "" { - if err == nil { - t.Fatalf("parseInteractiveCommand(%q) error = nil, want %q", tt.line, tt.wantErr) - } - if err.Error() != tt.wantErr { - t.Fatalf("parseInteractiveCommand(%q) error = %q, want %q", tt.line, err.Error(), tt.wantErr) - } - return - } - - if err != nil { - t.Fatalf("parseInteractiveCommand(%q) error = %v", tt.line, err) - } - if got != tt.want { - t.Fatalf("parseInteractiveCommand(%q) = %+v, want %+v", tt.line, got, tt.want) - } - }) - } -} diff --git a/cmd/peer/main.go b/cmd/peer/main.go deleted file mode 100644 index 845cab8..0000000 --- a/cmd/peer/main.go +++ /dev/null @@ -1,192 +0,0 @@ -package main - -import ( - "bufio" - "flag" - "fmt" - "io" - "log" - "os" - - "omnisocketgo/cmd/internal/latencylog" - peerpkg "omnisocketgo/cmd/internal/peer" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -func main() { - peerID := flag.String("id", "peer-a", "peer identity") // peer 的标识 - serverAddr := flag.String("server", "127.0.0.1:9000", "server address") // server 的地址 - targetPeer := flag.String("to", "", "optional target peer for one outgoing message") // 可选的目标 peer 标识 - text := flag.String("text", "", "optional text to send after connecting") // 可选的文本消息内容 - filePath := flag.String("file", "", "optional file path to send after connecting") - bindIP := flag.String("bind-ip", "", "optional local source IP used when dialing the server") - bindDevice := flag.String("bind-device", "", "optional Linux network device used when dialing the server") - inboxDir := flag.String("inbox-dir", "inbox", "directory used to persist received text and file messages") - logPath := flag.String("latency-log", "", "optional JSONL file path for latency timestamp logs") - txTimestampDebugLogPath := flag.String("tx-ts-debug-log", "", "optional JSONL file path for TX errqueue debug records") - interactive := flag.Bool("interactive", true, "enable interactive REPL for repeated text/file sends on the same connection") - flag.Parse() - - clientOptions := make([]peerpkg.Option, 0, 4) - if *logPath != "" { - logger, err := latencylog.NewJSONLLogger(*logPath) - if err != nil { - log.Fatalf("create latency logger %s: %v", *logPath, err) - } - defer logger.Close() - clientOptions = append(clientOptions, peerpkg.WithLogger(logger)) - } - if *txTimestampDebugLogPath != "" { - logger, err := transport.NewJSONLTXTimestampDebugLogger(*txTimestampDebugLogPath) - if err != nil { - log.Fatalf("create tx timestamp debug logger %s: %v", *txTimestampDebugLogPath, err) - } - defer logger.Close() - clientOptions = append(clientOptions, peerpkg.WithTXTimestampDebugLogger(logger)) - } - if *bindIP != "" { - clientOptions = append(clientOptions, peerpkg.WithBindIP(*bindIP)) - } - if *bindDevice != "" { - clientOptions = append(clientOptions, peerpkg.WithBindDevice(*bindDevice)) - } - - client, err := peerpkg.Dial(*serverAddr, *peerID, clientOptions...) - if err != nil { - log.Fatalf("dial server %s: %v", *serverAddr, err) - } - defer client.Close() - - log.Printf("connected to %s as %s", *serverAddr, client.ID()) - - receiveErr := make(chan error, 1) - go func() { - receiveErr <- client.ReceiveLoop(func(msg protocol.Message) error { - switch msg.Type { - case protocol.MessageTypeText: - path, err := client.PersistMessage(msg, *inboxDir) - if err != nil { - return err - } - log.Printf("received text from %s to %s and persisted to %s", msg.From, msg.To, path) - case protocol.MessageTypeFile: - path, err := client.PersistMessage(msg, *inboxDir) - if err != nil { - return err - } - log.Printf("received file from %s to %s: %s (%d bytes) -> %s", msg.From, msg.To, msg.FileName, len(msg.Body), path) - case protocol.MessageTypeError: - log.Printf("received %s from %s to %s: %s", msg.Type, msg.From, msg.To, string(msg.Body)) - default: - log.Printf("received unexpected message type %s from %s", msg.Type, msg.From) - } - return nil - }) - }() - - if *text != "" && *filePath != "" { - log.Fatal("only one of -text or -file may be specified") - } - - if (*text != "" || *filePath != "") && *targetPeer == "" { - log.Fatal("flag -to is required when sending text or file") - } - - //如果指定了目标 peer 和文本消息内容，则向目标 peer 发送一条文本消息，如果发送失败，打印错误日志并退出。 - if *targetPeer != "" && *text != "" { - if err := client.SendText(*targetPeer, *text); err != nil { - log.Fatalf("send text to %s: %v", *targetPeer, err) - } - log.Printf("sent text to %s", *targetPeer) - } - - if *targetPeer != "" && *filePath != "" { - if err := client.SendFilePath(*targetPeer, *filePath); err != nil { - log.Fatalf("send file %s to %s: %v", *filePath, *targetPeer, err) - } - log.Printf("sent file %s to %s", *filePath, *targetPeer) - } - //交互式模式：如果启用了交互式模式，则启动一个 REPL 循环，允许用户在命令行输入多条发送文本或文件的命令，直到用户输入退出命令或接收循环发生错误。如果没有启用交互式模式，则等待接收循环结束，如果接收循环发生错误，则打印错误日志。 - if *interactive { - if err := runInteractiveShell(client, os.Stdin, os.Stdout, receiveErr); err != nil { - log.Printf("interactive shell ended: %v", err) - } - return - } - - if err := <-receiveErr; err != nil { - log.Printf("receive loop ended: %v", err) - } -} - -// 在终端面板中打印报错和交互消息 -func runInteractiveShell(client *peerpkg.Client, in io.Reader, out io.Writer, receiveErr <-chan error) error { - printInteractiveHelp(out) - lines, inputErr := readInteractiveLines(in, out, fmt.Sprintf("%s> ", client.ID())) - - for { - select { - case err := <-receiveErr: - return err - case line, ok := <-lines: - if !ok { - return <-inputErr - } - - command, err := parseInteractiveCommand(line) - if err != nil { - if err == errEmptyInteractiveCommand { - continue - } - log.Printf("interactive command error: %v", err) - continue - } - - switch command.name { - case interactiveCommandHelp: - printInteractiveHelp(out) - case interactiveCommandQuit: - return nil - case interactiveCommandText: - if err := client.SendText(command.to, command.value); err != nil { - log.Printf("send text to %s: %v", command.to, err) - continue - } - log.Printf("sent text to %s", command.to) - case interactiveCommandFile: - if err := client.SendFilePath(command.to, command.value); err != nil { - log.Printf("send file %s to %s: %v", command.value, command.to, err) - continue - } - log.Printf("sent file %s to %s", command.value, command.to) - } - } - } -} - -func readInteractiveLines(in io.Reader, out io.Writer, prompt string) (<-chan string, <-chan error) { - lines := make(chan string) - errs := make(chan error, 1) - - go func() { - defer close(lines) - - scanner := bufio.NewScanner(in) - scanner.Buffer(make([]byte, 0, 1024), 1024*1024) - - for { - if _, err := fmt.Fprint(out, prompt); err != nil { - errs <- err - return - } - if !scanner.Scan() { - errs <- scanner.Err() - return - } - lines <- scanner.Text() - } - }() - - return lines, errs -} diff --git a/cmd/server/main.go b/cmd/server/main.go deleted file mode 100644 index 390e145..0000000 --- a/cmd/server/main.go +++ /dev/null @@ -1,49 +0,0 @@ -package main - -import ( - "flag" - "log" - "net" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/server" -) - -func main() { - listenAddr := flag.String("listen", ":9000", "server listen address") //监听地址 - logPath := flag.String("latency-log", "", "optional JSONL file path for latency timestamp logs") - flag.Parse() //查看命令行参数 - - hubOptions := make([]server.Option, 0, 1) - if *logPath != "" { - logger, err := latencylog.NewJSONLLogger(*logPath) - if err != nil { - log.Fatalf("create latency logger %s: %v", *logPath, err) - } - defer logger.Close() - hubOptions = append(hubOptions, server.WithLogger(logger)) - } - - listener, err := net.Listen("tcp", *listenAddr) //开启tcp监听器，监听来自客户端的连接请求 - if err != nil { - log.Fatalf("listen on %s: %v", *listenAddr, err) - } - defer listener.Close() //确保在 main 函数退出时关闭监听器 - - hub := server.NewHub(hubOptions...) //创建一个新的 Hub 实例，负责管理客户端连接和消息转发 - log.Printf("server listening on %s", listener.Addr()) - - for { - conn, err := listener.Accept() - if err != nil { - log.Printf("accept connection: %v", err) - continue - } - - go func(rawConn net.Conn) { - if err := hub.ServeConn(rawConn); err != nil { - log.Printf("connection closed: %v", err) - } - }(conn) - } -} diff --git a/c/cmd/udppeer.c b/cmd/udppeer.c similarity index 100% rename from c/cmd/udppeer.c rename to cmd/udppeer.c diff --git a/cmd/udppeer/interactive.go b/cmd/udppeer/interactive.go deleted file mode 100644 index fb4f08d..0000000 --- a/cmd/udppeer/interactive.go +++ /dev/null @@ -1,70 +0,0 @@ -package main - -import ( - "errors" - "fmt" - "strings" -) - -var errUDPEmptyCommand = errors.New("interactive command is empty") - -type udpInteractiveCommand struct { - name string - to string - value string -} - -func parseUDPInteractiveCommand(line string) (udpInteractiveCommand, error) { - commandName, rest, ok := cutUDPField(strings.TrimSpace(line)) - if !ok { - return udpInteractiveCommand{}, errUDPEmptyCommand - } - - switch strings.ToLower(commandName) { - case "help", "h", "?": - return udpInteractiveCommand{name: "help"}, nil - case "quit", "exit": - return udpInteractiveCommand{name: "quit"}, nil - case "text": - to, body, err := parseUDPTargetValue(rest, "text") - if err != nil { - return udpInteractiveCommand{}, err - } - return udpInteractiveCommand{name: "text", to: to, value: body}, nil - case "file": - to, path, err := parseUDPTargetValue(rest, "file") - if err != nil { - return udpInteractiveCommand{}, err - } - return udpInteractiveCommand{name: "file", to: to, value: path}, nil - default: - return udpInteractiveCommand{}, fmt.Errorf("unknown command %q; type help for usage", commandName) - } -} - -func parseUDPTargetValue(rest, commandName string) (string, string, error) { - to, value, ok := cutUDPField(strings.TrimSpace(rest)) - if !ok { - return "", "", fmt.Errorf("%s command requires a target peer and payload", commandName) - } - if strings.TrimSpace(value) == "" { - return "", "", fmt.Errorf("%s command requires a non-empty payload", commandName) - } - - return to, strings.TrimSpace(value), nil -} - -func cutUDPField(input string) (string, string, bool) { - trimmed := strings.TrimSpace(input) - if trimmed == "" { - return "", "", false - } - - for i, r := range trimmed { - if r == ' ' || r == '\t' { - return trimmed[:i], strings.TrimSpace(trimmed[i+1:]), true - } - } - - return trimmed, "", true -} diff --git a/cmd/udppeer/main.go b/cmd/udppeer/main.go deleted file mode 100644 index 18f9345..0000000 --- a/cmd/udppeer/main.go +++ /dev/null @@ -1,194 +0,0 @@ -package main - -import ( - "bufio" - "flag" - "fmt" - "io" - "log" - "os" - - "omnisocketgo/cmd/internal/latencylog" - peerpkg "omnisocketgo/cmd/internal/peer" - "omnisocketgo/cmd/internal/protocol" - "omnisocketgo/cmd/internal/transport" -) - -func main() { - peerID := flag.String("id", "peer-a", "peer identity") - serverAddr := flag.String("server", "127.0.0.1:9001", "UDP server address") - targetPeer := flag.String("to", "", "optional target peer for one outgoing message") - text := flag.String("text", "", "optional text to send after connecting") - filePath := flag.String("file", "", "optional file path to send after connecting") - bindIP := flag.String("bind-ip", "", "optional local source IP used when dialing the server") - inboxDir := flag.String("inbox-dir", "inbox", "directory used to persist received text and file messages") - logPath := flag.String("latency-log", "", "optional JSONL file path for latency timestamp logs") - txTimestampDebugLogPath := flag.String("tx-ts-debug-log", "", "optional JSONL file path for TX errqueue debug records") - interactive := flag.Bool("interactive", true, "enable interactive REPL for repeated text/file sends on the same connection") - flag.Parse() - - clientOptions := make([]peerpkg.Option, 0, 4) - if *logPath != "" { - logger, err := latencylog.NewJSONLLogger(*logPath) - if err != nil { - log.Fatalf("create latency logger %s: %v", *logPath, err) - } - defer logger.Close() - clientOptions = append(clientOptions, peerpkg.WithLogger(logger)) - } - if *txTimestampDebugLogPath != "" { - logger, err := transport.NewJSONLTXTimestampDebugLogger(*txTimestampDebugLogPath) - if err != nil { - log.Fatalf("create tx timestamp debug logger %s: %v", *txTimestampDebugLogPath, err) - } - defer logger.Close() - clientOptions = append(clientOptions, peerpkg.WithTXTimestampDebugLogger(logger)) - } - if *bindIP != "" { - clientOptions = append(clientOptions, peerpkg.WithBindIP(*bindIP)) - } - - client, err := peerpkg.DialUDP(*serverAddr, *peerID, clientOptions...) - if err != nil { - log.Fatalf("dial udp server %s: %v", *serverAddr, err) - } - defer client.Close() - - log.Printf("connected to %s as %s (UDP)", *serverAddr, client.ID()) - - receiveErr := make(chan error, 1) - go func() { - receiveErr <- client.ReceiveLoop(func(msg protocol.Message) error { - switch msg.Type { - case protocol.MessageTypeText: - path, err := client.PersistMessage(msg, *inboxDir) - if err != nil { - return err - } - log.Printf("received text from %s to %s and persisted to %s", msg.From, msg.To, path) - case protocol.MessageTypeFile: - path, err := client.PersistMessage(msg, *inboxDir) - if err != nil { - return err - } - log.Printf("received file from %s to %s: %s (%d bytes) -> %s", msg.From, msg.To, msg.FileName, len(msg.Body), path) - case protocol.MessageTypeError: - log.Printf("received %s from %s to %s: %s", msg.Type, msg.From, msg.To, string(msg.Body)) - default: - log.Printf("received unexpected message type %s from %s", msg.Type, msg.From) - } - return nil - }) - }() - - if *text != "" && *filePath != "" { - log.Fatal("only one of -text or -file may be specified") - } - - if (*text != "" || *filePath != "") && *targetPeer == "" { - log.Fatal("flag -to is required when sending text or file") - } - - if *targetPeer != "" && *text != "" { - if err := client.SendText(*targetPeer, *text); err != nil { - log.Fatalf("send text to %s: %v", *targetPeer, err) - } - log.Printf("sent text to %s", *targetPeer) - } - - if *targetPeer != "" && *filePath != "" { - if err := client.SendFilePath(*targetPeer, *filePath); err != nil { - log.Fatalf("send file %s to %s: %v", *filePath, *targetPeer, err) - } - log.Printf("sent file %s to %s", *filePath, *targetPeer) - } - - if *interactive { - if err := runUDPInteractiveShell(client, os.Stdin, os.Stdout, receiveErr); err != nil { - log.Printf("interactive shell ended: %v", err) - } - return - } - - if err := <-receiveErr; err != nil { - log.Printf("receive loop ended: %v", err) - } -} - -func runUDPInteractiveShell(client *peerpkg.UDPClient, in io.Reader, out io.Writer, receiveErr <-chan error) error { - printUDPInteractiveHelp(out) - lines, inputErr := readUDPInteractiveLines(in, out, fmt.Sprintf("%s> ", client.ID())) - - for { - select { - case err := <-receiveErr: - return err - case line, ok := <-lines: - if !ok { - return <-inputErr - } - - command, err := parseUDPInteractiveCommand(line) - if err != nil { - if err == errUDPEmptyCommand { - continue - } - log.Printf("interactive command error: %v", err) - continue - } - - switch command.name { - case "help": - printUDPInteractiveHelp(out) - case "quit": - return nil - case "text": - if err := client.SendText(command.to, command.value); err != nil { - log.Printf("send text to %s: %v", command.to, err) - continue - } - log.Printf("sent text to %s", command.to) - case "file": - if err := client.SendFilePath(command.to, command.value); err != nil { - log.Printf("send file %s to %s: %v", command.value, command.to, err) - continue - } - log.Printf("sent file %s to %s", command.value, command.to) - } - } - } -} - -func readUDPInteractiveLines(in io.Reader, out io.Writer, prompt string) (<-chan string, <-chan error) { - lines := make(chan string) - errs := make(chan error, 1) - - go func() { - defer close(lines) - - scanner := bufio.NewScanner(in) - scanner.Buffer(make([]byte, 0, 1024), 1024*1024) - - for { - if _, err := fmt.Fprint(out, prompt); err != nil { - errs <- err - return - } - if !scanner.Scan() { - errs <- scanner.Err() - return - } - lines <- scanner.Text() - } - }() - - return lines, errs -} - -func printUDPInteractiveHelp(w io.Writer) { - _, _ = fmt.Fprintln(w, "interactive mode commands (UDP):") - _, _ = fmt.Fprintln(w, " help show this help") - _, _ = fmt.Fprintln(w, " text send one text message over UDP") - _, _ = fmt.Fprintln(w, " file send one file over UDP") - _, _ = fmt.Fprintln(w, " quit exit this peer process") -} diff --git a/c/cmd/udpping.c b/cmd/udpping.c similarity index 100% rename from c/cmd/udpping.c rename to cmd/udpping.c diff --git a/cmd/udpping/main.go b/cmd/udpping/main.go deleted file mode 100644 index 2b416fc..0000000 --- a/cmd/udpping/main.go +++ /dev/null @@ -1,389 +0,0 @@ -package main - -import ( - "encoding/json" - "errors" - "flag" - "fmt" - "io" - "math" - "os" - "sort" - "strings" - "time" -) - -const ( - defaultPeerID = "pinger" - defaultServer = "127.0.0.1:9001" - defaultCount = 100 - defaultInterval = 100 * time.Millisecond - defaultSize = 64 - defaultTimeout = 3 * time.Second - minExpiryPoll = 10 * time.Millisecond - maxExpiryPoll = 100 * time.Millisecond -) - -type config struct { - id string - server string - to string - echo bool - count int - interval time.Duration - size int - timeout time.Duration - bindIP string - latencyLog string -} - -type pingPayload struct { - Seq uint64 `json:"seq"` - TSUnixNano int64 `json:"ts_ns"` - Pad string `json:"pad"` -} - -type pendingPing struct { - deadline time.Time -} - -type replyDisposition int - -const ( - replyMatched replyDisposition = iota - replyDuplicate - replyUnexpected -) - -type replyResult struct { - disposition replyDisposition - rtt time.Duration -} - -type pingTracker struct { - timeout time.Duration - sent int - duplicates int - pending map[uint64]pendingPing - seen map[uint64]struct{} - samples []time.Duration -} - -type rttSummary struct { - Sent int - Received int - Duplicates int - LossPct float64 - Min time.Duration - Avg time.Duration - Max time.Duration - P50 time.Duration - P95 time.Duration - P99 time.Duration - StdDev time.Duration - HasSamples bool -} - -func main() { - if err := runMain(os.Args[1:], os.Stdout, os.Stderr, time.Now); err != nil { - if errors.Is(err, flag.ErrHelp) { - return - } - _, _ = fmt.Fprintln(os.Stderr, err) - os.Exit(1) - } -} - -func runMain(args []string, stdout, stderr io.Writer, now func() time.Time) error { - cfg, err := parseConfig(args, stderr) - if err != nil { - return err - } - - return runPlatform(cfg, stdout, stderr, now) -} - -func parseConfig(args []string, stderr io.Writer) (config, error) { - cfg := config{} - - flags := flag.NewFlagSet("udpping", flag.ContinueOnError) - flags.SetOutput(stderr) - flags.StringVar(&cfg.id, "id", defaultPeerID, "local peer identity") - flags.StringVar(&cfg.server, "server", defaultServer, "UDP server address") - flags.StringVar(&cfg.to, "to", "", "target peer identity in ping mode") - flags.BoolVar(&cfg.echo, "echo", false, "echo back every received text message") - flags.IntVar(&cfg.count, "count", defaultCount, "number of pings to send; 0 means run until interrupted") - flags.DurationVar(&cfg.interval, "interval", defaultInterval, "delay between ping sends") - flags.IntVar(&cfg.size, "size", defaultSize, "application payload size in bytes") - flags.DurationVar(&cfg.timeout, "timeout", defaultTimeout, "per-ping timeout") - flags.StringVar(&cfg.bindIP, "bind-ip", "", "optional local source IP used when dialing the server") - flags.StringVar(&cfg.latencyLog, "latency-log", "", "optional JSONL file path for latency timestamp logs") - - if err := flags.Parse(args); err != nil { - return config{}, err - } - if flags.NArg() > 0 { - return config{}, fmt.Errorf("unexpected positional arguments: %s", strings.Join(flags.Args(), " ")) - } - - cfg.id = strings.TrimSpace(cfg.id) - cfg.server = strings.TrimSpace(cfg.server) - cfg.to = strings.TrimSpace(cfg.to) - cfg.bindIP = strings.TrimSpace(cfg.bindIP) - cfg.latencyLog = strings.TrimSpace(cfg.latencyLog) - - if err := cfg.validate(); err != nil { - return config{}, err - } - return cfg, nil -} - -func (c config) validate() error { - if c.id == "" { - return fmt.Errorf("flag -id is required") - } - if c.server == "" { - return fmt.Errorf("flag -server is required") - } - if !c.echo && c.to == "" { - return fmt.Errorf("flag -to is required unless -echo is set") - } - if c.count < 0 { - return fmt.Errorf("flag -count must be greater than or equal to zero") - } - if c.interval <= 0 { - return fmt.Errorf("flag -interval must be greater than zero") - } - if c.size <= 0 { - return fmt.Errorf("flag -size must be greater than zero") - } - if c.timeout <= 0 { - return fmt.Errorf("flag -timeout must be greater than zero") - } - return nil -} - -func buildPingPayload(seq uint64, tsUnixNano int64, size int) ([]byte, error) { - payload := pingPayload{ - Seq: seq, - TSUnixNano: tsUnixNano, - Pad: "", - } - - body, err := json.Marshal(payload) - if err != nil { - return nil, fmt.Errorf("encode ping payload: %w", err) - } - if len(body) > size { - return nil, fmt.Errorf("requested payload size %d is too small; minimum is %d", size, len(body)) - } - - payload.Pad = strings.Repeat("A", size-len(body)) - body, err = json.Marshal(payload) - if err != nil { - return nil, fmt.Errorf("encode padded ping payload: %w", err) - } - if len(body) != size { - return nil, fmt.Errorf("encode padded ping payload: got %d bytes, want %d", len(body), size) - } - return body, nil -} - -func parsePingPayload(body []byte) (pingPayload, error) { - var payload pingPayload - if err := json.Unmarshal(body, &payload); err != nil { - return pingPayload{}, fmt.Errorf("decode ping payload: %w", err) - } - if payload.Seq == 0 { - return pingPayload{}, fmt.Errorf("decode ping payload: seq must be greater than zero") - } - if payload.TSUnixNano <= 0 { - return pingPayload{}, fmt.Errorf("decode ping payload: ts_ns must be greater than zero") - } - return payload, nil -} - -func newPingTracker(timeout time.Duration) *pingTracker { - return &pingTracker{ - timeout: timeout, - pending: make(map[uint64]pendingPing), - seen: make(map[uint64]struct{}), - } -} - -func (t *pingTracker) markSent(seq uint64, sentAt time.Time) { - t.sent++ - t.pending[seq] = pendingPing{deadline: sentAt.Add(t.timeout)} - t.seen[seq] = struct{}{} -} - -func (t *pingTracker) observeReply(payload pingPayload, receivedAt time.Time) replyResult { - if _, ok := t.seen[payload.Seq]; !ok { - return replyResult{disposition: replyUnexpected} - } - - if _, ok := t.pending[payload.Seq]; !ok { - t.duplicates++ - return replyResult{disposition: replyDuplicate} - } - - delete(t.pending, payload.Seq) - rtt := receivedAt.Sub(time.Unix(0, payload.TSUnixNano)) - if rtt < 0 { - rtt = 0 - } - t.samples = append(t.samples, rtt) - return replyResult{ - disposition: replyMatched, - rtt: rtt, - } -} - -func (t *pingTracker) expire(now time.Time) []uint64 { - expired := make([]uint64, 0) - for seq, pending := range t.pending { - if !pending.deadline.After(now) { - expired = append(expired, seq) - delete(t.pending, seq) - } - } - sort.Slice(expired, func(i, j int) bool { - return expired[i] < expired[j] - }) - return expired -} - -func (t *pingTracker) pendingCount() int { - return len(t.pending) -} - -func (t *pingTracker) summary() rttSummary { - return calculateRTTSummary(t.samples, t.sent, t.duplicates) -} - -func calculateRTTSummary(samples []time.Duration, sent, duplicates int) rttSummary { - summary := rttSummary{ - Sent: sent, - Received: len(samples), - Duplicates: duplicates, - } - if sent > 0 { - summary.LossPct = float64(sent-len(samples)) * 100 / float64(sent) - } - if len(samples) == 0 { - return summary - } - - sorted := append([]time.Duration(nil), samples...) - sort.Slice(sorted, func(i, j int) bool { - return sorted[i] < sorted[j] - }) - - var sum float64 - for _, sample := range sorted { - sum += float64(sample) - } - avg := sum / float64(len(sorted)) - - var variance float64 - for _, sample := range sorted { - delta := float64(sample) - avg - variance += delta * delta - } - variance /= float64(len(sorted)) - - summary.Min = sorted[0] - summary.Avg = time.Duration(math.Round(avg)) - summary.Max = sorted[len(sorted)-1] - summary.P50 = percentileDuration(sorted, 0.50) - summary.P95 = percentileDuration(sorted, 0.95) - summary.P99 = percentileDuration(sorted, 0.99) - summary.StdDev = time.Duration(math.Round(math.Sqrt(variance))) - summary.HasSamples = true - return summary -} - -func percentileDuration(sorted []time.Duration, percentile float64) time.Duration { - if len(sorted) == 0 { - return 0 - } - if percentile <= 0 { - return sorted[0] - } - if percentile >= 1 { - return sorted[len(sorted)-1] - } - - index := int(math.Ceil(percentile*float64(len(sorted)))) - 1 - if index < 0 { - index = 0 - } - if index >= len(sorted) { - index = len(sorted) - 1 - } - return sorted[index] -} - -func formatRTT(duration time.Duration) string { - return fmt.Sprintf("%.2fms", float64(duration)/float64(time.Millisecond)) -} - -func writePingHeader(w io.Writer, cfg config) error { - _, err := fmt.Fprintf(w, "UDP PING %s via %s (payload=%d bytes, UDP)\n", cfg.to, cfg.server, cfg.size) - return err -} - -func writeMatchedReply(w io.Writer, seq uint64, rtt time.Duration) error { - _, err := fmt.Fprintf(w, "seq=%d rtt=%s\n", seq, formatRTT(rtt)) - return err -} - -func writeTimeout(w io.Writer, seq uint64) error { - _, err := fmt.Fprintf(w, "seq=%d timeout\n", seq) - return err -} - -func writeSummary(w io.Writer, target string, summary rttSummary) error { - if _, err := fmt.Fprintf(w, "--- %s udp ping statistics ---\n", target); err != nil { - return err - } - if _, err := fmt.Fprintf( - w, - "%d packets transmitted, %d received, %d duplicates, %.2f%% packet loss\n", - summary.Sent, - summary.Received, - summary.Duplicates, - summary.LossPct, - ); err != nil { - return err - } - - if !summary.HasSamples { - _, err := fmt.Fprintln(w, "rtt min/avg/max/p50/p95/p99 = n/a/n/a/n/a/n/a/n/a/n/a, stddev=n/a") - return err - } - - _, err := fmt.Fprintf( - w, - "rtt min/avg/max/p50/p95/p99 = %s/%s/%s/%s/%s/%s, stddev=%s\n", - formatRTT(summary.Min), - formatRTT(summary.Avg), - formatRTT(summary.Max), - formatRTT(summary.P50), - formatRTT(summary.P95), - formatRTT(summary.P99), - formatRTT(summary.StdDev), - ) - return err -} - -func expiryPollInterval(timeout time.Duration) time.Duration { - interval := timeout / 4 - if interval < minExpiryPoll { - return minExpiryPoll - } - if interval > maxExpiryPoll { - return maxExpiryPoll - } - return interval -} diff --git a/cmd/udpping/main_test.go b/cmd/udpping/main_test.go deleted file mode 100644 index adb1915..0000000 --- a/cmd/udpping/main_test.go +++ /dev/null @@ -1,203 +0,0 @@ -package main - -import ( - "bytes" - "strings" - "testing" - "time" -) - -func TestParseConfigDefaults(t *testing.T) { - cfg, err := parseConfig([]string{"-to", "peer-b"}, ioDiscard{}) - if err != nil { - t.Fatalf("parseConfig() error = %v", err) - } - - if cfg.id != defaultPeerID { - t.Fatalf("id = %q, want %q", cfg.id, defaultPeerID) - } - if cfg.server != defaultServer { - t.Fatalf("server = %q, want %q", cfg.server, defaultServer) - } - if cfg.count != defaultCount { - t.Fatalf("count = %d, want %d", cfg.count, defaultCount) - } - if cfg.interval != defaultInterval { - t.Fatalf("interval = %s, want %s", cfg.interval, defaultInterval) - } - if cfg.size != defaultSize { - t.Fatalf("size = %d, want %d", cfg.size, defaultSize) - } - if cfg.timeout != defaultTimeout { - t.Fatalf("timeout = %s, want %s", cfg.timeout, defaultTimeout) - } -} - -func TestParseConfigRequiresTargetInPingMode(t *testing.T) { - _, err := parseConfig([]string{"-echo=false"}, ioDiscard{}) - if err == nil || !strings.Contains(err.Error(), "flag -to is required") { - t.Fatalf("parseConfig() error = %v, want missing -to error", err) - } -} - -func TestParseConfigAllowsEchoWithoutTarget(t *testing.T) { - cfg, err := parseConfig([]string{"-echo"}, ioDiscard{}) - if err != nil { - t.Fatalf("parseConfig() error = %v", err) - } - if !cfg.echo { - t.Fatal("echo = false, want true") - } -} - -func TestParseConfigRejectsBindDeviceFlag(t *testing.T) { - _, err := parseConfig([]string{"-to", "peer-b", "-bind-device", "wwan0"}, ioDiscard{}) - if err == nil || !strings.Contains(err.Error(), "flag provided but not defined") { - t.Fatalf("parseConfig() error = %v, want unknown flag error", err) - } -} - -func TestBuildPingPayloadUsesExactSize(t *testing.T) { - body, err := buildPingPayload(7, 123456789, 96) - if err != nil { - t.Fatalf("buildPingPayload() error = %v", err) - } - if len(body) != 96 { - t.Fatalf("len(body) = %d, want 96", len(body)) - } - - payload, err := parsePingPayload(body) - if err != nil { - t.Fatalf("parsePingPayload() error = %v", err) - } - if payload.Seq != 7 { - t.Fatalf("seq = %d, want 7", payload.Seq) - } - if payload.TSUnixNano != 123456789 { - t.Fatalf("ts_ns = %d, want 123456789", payload.TSUnixNano) - } -} - -func TestBuildPingPayloadRejectsTooSmallSize(t *testing.T) { - _, err := buildPingPayload(1, 123456789, 8) - if err == nil || !strings.Contains(err.Error(), "too small") { - t.Fatalf("buildPingPayload() error = %v, want size too small error", err) - } -} - -func TestParsePingPayloadRejectsInvalidJSON(t *testing.T) { - _, err := parsePingPayload([]byte("not-json")) - if err == nil || !strings.Contains(err.Error(), "decode ping payload") { - t.Fatalf("parsePingPayload() error = %v, want decode error", err) - } -} - -func TestPingTrackerHandlesMatchedDuplicateAndTimeout(t *testing.T) { - tracker := newPingTracker(50 * time.Millisecond) - sentAt := time.Unix(0, 100) - tracker.markSent(1, sentAt) - - match := tracker.observeReply(pingPayload{Seq: 1, TSUnixNano: sentAt.UnixNano()}, sentAt.Add(12*time.Millisecond)) - if match.disposition != replyMatched { - t.Fatalf("first disposition = %v, want matched", match.disposition) - } - if match.rtt != 12*time.Millisecond { - t.Fatalf("first rtt = %s, want 12ms", match.rtt) - } - - duplicate := tracker.observeReply(pingPayload{Seq: 1, TSUnixNano: sentAt.UnixNano()}, sentAt.Add(20*time.Millisecond)) - if duplicate.disposition != replyDuplicate { - t.Fatalf("second disposition = %v, want duplicate", duplicate.disposition) - } - - tracker.markSent(2, sentAt) - expired := tracker.expire(sentAt.Add(60 * time.Millisecond)) - if len(expired) != 1 || expired[0] != 2 { - t.Fatalf("expired = %v, want [2]", expired) - } - - late := tracker.observeReply(pingPayload{Seq: 2, TSUnixNano: sentAt.UnixNano()}, sentAt.Add(70*time.Millisecond)) - if late.disposition != replyDuplicate { - t.Fatalf("late disposition = %v, want duplicate", late.disposition) - } - - unexpected := tracker.observeReply(pingPayload{Seq: 99, TSUnixNano: sentAt.UnixNano()}, sentAt.Add(80*time.Millisecond)) - if unexpected.disposition != replyUnexpected { - t.Fatalf("unexpected disposition = %v, want unexpected", unexpected.disposition) - } -} - -func TestCalculateRTTSummary(t *testing.T) { - summary := calculateRTTSummary( - []time.Duration{ - 10 * time.Millisecond, - 20 * time.Millisecond, - 30 * time.Millisecond, - 40 * time.Millisecond, - 50 * time.Millisecond, - }, - 6, - 2, - ) - - if summary.Sent != 6 { - t.Fatalf("Sent = %d, want 6", summary.Sent) - } - if summary.Received != 5 { - t.Fatalf("Received = %d, want 5", summary.Received) - } - if summary.Duplicates != 2 { - t.Fatalf("Duplicates = %d, want 2", summary.Duplicates) - } - if summary.LossPct != (float64(1) * 100 / 6) { - t.Fatalf("LossPct = %f, want %f", summary.LossPct, float64(1)*100/6) - } - if summary.Min != 10*time.Millisecond { - t.Fatalf("Min = %s, want 10ms", summary.Min) - } - if summary.Avg != 30*time.Millisecond { - t.Fatalf("Avg = %s, want 30ms", summary.Avg) - } - if summary.Max != 50*time.Millisecond { - t.Fatalf("Max = %s, want 50ms", summary.Max) - } - if summary.P50 != 30*time.Millisecond { - t.Fatalf("P50 = %s, want 30ms", summary.P50) - } - if summary.P95 != 50*time.Millisecond { - t.Fatalf("P95 = %s, want 50ms", summary.P95) - } - if summary.P99 != 50*time.Millisecond { - t.Fatalf("P99 = %s, want 50ms", summary.P99) - } - if summary.StdDev == 0 { - t.Fatal("StdDev = 0, want non-zero") - } -} - -func TestWriteSummaryUsesNAWithoutSamples(t *testing.T) { - var buf bytes.Buffer - err := writeSummary(&buf, "host", rttSummary{ - Sent: 3, - Received: 0, - Duplicates: 1, - LossPct: 100, - }) - if err != nil { - t.Fatalf("writeSummary() error = %v", err) - } - - out := buf.String() - if !strings.Contains(out, "3 packets transmitted, 0 received, 1 duplicates, 100.00% packet loss") { - t.Fatalf("summary output missing counters: %q", out) - } - if !strings.Contains(out, "n/a/n/a/n/a/n/a/n/a/n/a") { - t.Fatalf("summary output missing n/a metrics: %q", out) - } -} - -type ioDiscard struct{} - -func (ioDiscard) Write(p []byte) (int, error) { - return len(p), nil -} diff --git a/cmd/udpping/platform_linux.go b/cmd/udpping/platform_linux.go deleted file mode 100644 index 150dc2e..0000000 --- a/cmd/udpping/platform_linux.go +++ /dev/null @@ -1,244 +0,0 @@ -//go:build linux - -package main - -import ( - "context" - "errors" - "fmt" - "io" - "os" - "os/signal" - "strings" - "time" - - "omnisocketgo/cmd/internal/latencylog" - peerpkg "omnisocketgo/cmd/internal/peer" - "omnisocketgo/cmd/internal/protocol" -) - -func runPlatform(cfg config, stdout, stderr io.Writer, now func() time.Time) error { - ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt) - defer stop() - - client, closeLogger, err := dialUDPClient(cfg) - if err != nil { - return err - } - defer closeLogger() - defer client.Close() - - if cfg.echo { - return runEchoMode(ctx, client, stderr) - } - return runPingMode(ctx, client, cfg, stdout, stderr, now) -} - -func dialUDPClient(cfg config) (*peerpkg.UDPClient, func(), error) { - options := make([]peerpkg.Option, 0, 3) - closeLogger := func() {} - options = append(options, peerpkg.WithUDPLinuxTimestamping(false)) - - if cfg.latencyLog != "" { - logger, err := latencylog.NewJSONLLogger(cfg.latencyLog) - if err != nil { - return nil, nil, fmt.Errorf("create latency logger %s: %w", cfg.latencyLog, err) - } - options = append(options, peerpkg.WithLogger(logger)) - closeLogger = func() { - _ = logger.Close() - } - } - if cfg.bindIP != "" { - options = append(options, peerpkg.WithBindIP(cfg.bindIP)) - } - - client, err := peerpkg.DialUDP(cfg.server, cfg.id, options...) - if err != nil { - closeLogger() - return nil, nil, fmt.Errorf("dial udp server %s: %w", cfg.server, err) - } - return client, closeLogger, nil -} - -func runPingMode(ctx context.Context, client *peerpkg.UDPClient, cfg config, stdout, stderr io.Writer, now func() time.Time) error { - if err := writePingHeader(stdout, cfg); err != nil { - return err - } - - receiveCh := make(chan protocol.Message, 32) - receiveErrCh := make(chan error, 1) - go func() { - for { - msg, err := client.Receive() - if err != nil { - receiveErrCh <- err - return - } - receiveCh <- msg - } - }() - - tracker := newPingTracker(cfg.timeout) - expiryTicker := time.NewTicker(expiryPollInterval(cfg.timeout)) - defer expiryTicker.Stop() - - var sendTicker *time.Ticker - if cfg.count == 0 || cfg.count > 1 { - sendTicker = time.NewTicker(cfg.interval) - defer sendTicker.Stop() - } - - nextSeq := uint64(1) - if err := sendPing(client, tracker, cfg, nextSeq, now); err != nil { - return err - } - nextSeq++ - - stopSending := cfg.count == 1 - receiveErrSeen := false - - for { - if stopSending && tracker.pendingCount() == 0 { - break - } - - var sendTick <-chan time.Time - if !stopSending && sendTicker != nil { - sendTick = sendTicker.C - } - - select { - case <-ctx.Done(): - stopSending = true - case <-expiryTicker.C: - for _, seq := range tracker.expire(now()) { - if err := writeTimeout(stdout, seq); err != nil { - return err - } - } - case msg := <-receiveCh: - if err := handlePingMessage(tracker, msg, stdout, stderr, now); err != nil { - return err - } - case err := <-receiveErrCh: - receiveErrSeen = true - if ctx.Err() != nil && isExpectedCloseError(err) { - break - } - if stopSending && tracker.pendingCount() == 0 && isExpectedCloseError(err) { - break - } - return fmt.Errorf("receive reply: %w", err) - case <-sendTick: - if cfg.count > 0 && tracker.sent >= cfg.count { - stopSending = true - continue - } - if err := sendPing(client, tracker, cfg, nextSeq, now); err != nil { - return err - } - nextSeq++ - if cfg.count > 0 && tracker.sent >= cfg.count { - stopSending = true - } - } - - if receiveErrSeen && stopSending && tracker.pendingCount() == 0 { - break - } - } - - for _, seq := range tracker.expire(now()) { - if err := writeTimeout(stdout, seq); err != nil { - return err - } - } - return writeSummary(stdout, cfg.to, tracker.summary()) -} - -func sendPing(client *peerpkg.UDPClient, tracker *pingTracker, cfg config, seq uint64, now func() time.Time) error { - sentAt := now() - payload, err := buildPingPayload(seq, sentAt.UnixNano(), cfg.size) - if err != nil { - return err - } - if err := client.SendText(cfg.to, string(payload)); err != nil { - return fmt.Errorf("send ping seq=%d: %w", seq, err) - } - tracker.markSent(seq, sentAt) - return nil -} - -func handlePingMessage(tracker *pingTracker, msg protocol.Message, stdout, stderr io.Writer, now func() time.Time) error { - switch msg.Type { - case protocol.MessageTypeText: - payload, err := parsePingPayload(msg.Body) - if err != nil { - _, writeErr := fmt.Fprintf(stderr, "ignore non-ping text message from %s: %v\n", msg.From, err) - if writeErr != nil { - return writeErr - } - return nil - } - - result := tracker.observeReply(payload, now()) - switch result.disposition { - case replyMatched: - return writeMatchedReply(stdout, payload.Seq, result.rtt) - case replyDuplicate: - _, err := fmt.Fprintf(stderr, "seq=%d duplicate or late reply ignored\n", payload.Seq) - return err - case replyUnexpected: - _, err := fmt.Fprintf(stderr, "seq=%d unexpected reply ignored\n", payload.Seq) - return err - default: - return nil - } - case protocol.MessageTypeError: - _, err := fmt.Fprintf(stderr, "server error: %s\n", strings.TrimSpace(string(msg.Body))) - return err - default: - _, err := fmt.Fprintf(stderr, "unexpected message type %s from %s ignored\n", msg.Type, msg.From) - return err - } -} - -func runEchoMode(ctx context.Context, client *peerpkg.UDPClient, stderr io.Writer) error { - receiveErrCh := make(chan error, 1) - go func() { - receiveErrCh <- client.ReceiveLoop(func(msg protocol.Message) error { - switch msg.Type { - case protocol.MessageTypeText: - return client.SendText(msg.From, string(msg.Body)) - case protocol.MessageTypeError: - _, err := fmt.Fprintf(stderr, "server error: %s\n", strings.TrimSpace(string(msg.Body))) - return err - default: - _, err := fmt.Fprintf(stderr, "unexpected message type %s from %s ignored\n", msg.Type, msg.From) - return err - } - }) - }() - - select { - case <-ctx.Done(): - return nil - case err := <-receiveErrCh: - if err == nil || (ctx.Err() != nil && isExpectedCloseError(err)) { - return nil - } - return fmt.Errorf("echo receive loop: %w", err) - } -} - -func isExpectedCloseError(err error) bool { - if err == nil { - return true - } - message := err.Error() - return errors.Is(err, context.Canceled) || - strings.Contains(message, "closed") || - strings.Contains(message, "broken pipe") || - strings.Contains(message, "io: read/write on closed pipe") -} diff --git a/cmd/udpping/platform_other.go b/cmd/udpping/platform_other.go deleted file mode 100644 index d13b792..0000000 --- a/cmd/udpping/platform_other.go +++ /dev/null @@ -1,14 +0,0 @@ -//go:build !linux - -package main - -import ( - "fmt" - "io" - "runtime" - "time" -) - -func runPlatform(cfg config, stdout, stderr io.Writer, now func() time.Time) error { - return fmt.Errorf("udpping is only supported on linux; current GOOS=%s", runtime.GOOS) -} diff --git a/c/cmd/udprelay.c b/cmd/udprelay.c similarity index 100% rename from c/cmd/udprelay.c rename to cmd/udprelay.c diff --git a/cmd/udprelay/main.go b/cmd/udprelay/main.go deleted file mode 100644 index 6108f26..0000000 --- a/cmd/udprelay/main.go +++ /dev/null @@ -1,43 +0,0 @@ -package main - -import ( - "flag" - "log" - "net" - - "omnisocketgo/cmd/internal/server" - "omnisocketgo/cmd/internal/transport" -) - -func main() { - listenAddr := flag.String("listen", ":9003", "UDP relay listen address (downstream, for KCP peer to connect)") - upstreamAddr := flag.String("upstream", "127.0.0.1:9002", "upstream KCP server address (server D)") - flag.Parse() - - upstreamUDPAddr, err := net.ResolveUDPAddr("udp", *upstreamAddr) - if err != nil { - log.Fatalf("resolve upstream address %s: %v", *upstreamAddr, err) - } - - listenNetwork, udpListenAddr, err := transport.ResolveUDPListenConfig(*listenAddr) - if err != nil { - log.Fatalf("resolve udp relay listen address %s: %v", *listenAddr, err) - } - - conn, err := net.ListenPacket(listenNetwork, udpListenAddr.String()) - if err != nil { - log.Fatalf("listen %s on %s: %v", listenNetwork, udpListenAddr, err) - } - - relay, err := server.NewUDPRelay(conn, upstreamUDPAddr) - if err != nil { - _ = conn.Close() - log.Fatalf("create udp relay: %v", err) - } - - log.Printf("udp relay listening on %s %s, upstream %s", listenNetwork, conn.LocalAddr(), *upstreamAddr) - - if err := relay.Serve(); err != nil { - log.Fatalf("udp relay serve: %v", err) - } -} diff --git a/c/cmd/udpserver.c b/cmd/udpserver.c similarity index 100% rename from c/cmd/udpserver.c rename to cmd/udpserver.c diff --git a/cmd/udpserver/main.go b/cmd/udpserver/main.go deleted file mode 100644 index ecb65c3..0000000 --- a/cmd/udpserver/main.go +++ /dev/null @@ -1,60 +0,0 @@ -package main - -import ( - "flag" - "log" - "net" - - "omnisocketgo/cmd/internal/latencylog" - "omnisocketgo/cmd/internal/server" - "omnisocketgo/cmd/internal/transport" -) - -func main() { - listenAddr := flag.String("listen", ":9001", "UDP server listen address") - logPath := flag.String("latency-log", "", "optional JSONL file path for latency timestamp logs") - txTimestampDebugLogPath := flag.String("tx-ts-debug-log", "", "optional JSONL file path for TX errqueue debug records; enables Linux UDP timestamping") - flag.Parse() - - hubOptions := make([]server.UDPOption, 0, 2) - hubOptions = append(hubOptions, server.WithUDPLinuxTimestamping(false)) - if *logPath != "" { - logger, err := latencylog.NewJSONLLogger(*logPath) - if err != nil { - log.Fatalf("create latency logger %s: %v", *logPath, err) - } - defer logger.Close() - hubOptions = append(hubOptions, server.WithUDPLogger(logger)) - } - if *txTimestampDebugLogPath != "" { - logger, err := transport.NewJSONLTXTimestampDebugLogger(*txTimestampDebugLogPath) - if err != nil { - log.Fatalf("create tx timestamp debug logger %s: %v", *txTimestampDebugLogPath, err) - } - defer logger.Close() - hubOptions = append(hubOptions, server.WithUDPLinuxTimestamping(true)) - hubOptions = append(hubOptions, server.WithUDPTXTimestampDebugLogger(logger)) - } - - udpAddr, err := net.ResolveUDPAddr("udp", *listenAddr) - if err != nil { - log.Fatalf("resolve udp address %s: %v", *listenAddr, err) - } - - conn, err := net.ListenUDP("udp", udpAddr) - if err != nil { - log.Fatalf("listen udp on %s: %v", *listenAddr, err) - } - defer conn.Close() - - hub, err := server.NewUDPHub(conn, hubOptions...) - if err != nil { - log.Fatalf("create udp hub: %v", err) - } - - log.Printf("udp server listening on %s", conn.LocalAddr()) - - if err := hub.Serve(); err != nil { - log.Fatalf("udp hub serve: %v", err) - } -} diff --git a/doc/project-guide.md b/doc/project-guide.md deleted file mode 100644 index ccceca5..0000000 --- a/doc/project-guide.md +++ /dev/null @@ -1,571 +0,0 @@ -# OmniSocketGo 项目导读 - -本文面向第一次接手 `OmniSocketGo` 的开发者，目标是帮助你在阅读源码之前先建立整体认知。它不替代根目录的 `README.md`；`README.md` 仍然适合作为快速构建和运行说明，这份文档更偏向项目结构、模块职责、消息流转和源码阅读导航。 - -## 1. 项目定位 - -`OmniSocketGo` 是一个 Linux-only 的 Go 1.22 项目，核心模型是： - -- 一个中心 `server` -- 多个连接到 `server` 的 `peer` -- 一个独立的时延日志汇总工具 `latencysummary` - -从职责上看： - -- `server` 负责接收 TCP 连接、校验 `peer` 注册、维护在线连接表，并在不同 `peer` 之间转发业务消息。 -- `peer` 负责连接 `server`、完成注册、发送文本或文件、接收转发消息，并将接收到的内容持久化到本地目录。 -- `latencysummary` 负责读取原始时延 JSONL 日志，按消息维度聚合为摘要结果，并额外生成一个 HTML 图表，方便观察端到端时延。 - -这个项目不是点对点直连通信，而是典型的“中心转发”模式。`peer-a` 想发给 `peer-b` 时，消息路径是： - -`peer-a -> server -> peer-b` - -## 2. 顶层结构 - -当前仓库的核心内容可以按下面理解： - -```text -OmniSocketGo/ -├─ README.md -├─ go.mod -├─ doc/ -│ └─ project-guide.md -├─ cmd/ -│ ├─ server/ -│ │ └─ main.go -│ ├─ peer/ -│ │ ├─ main.go -│ │ ├─ interactive.go -│ │ └─ interactive_test.go -│ ├─ latencysummary/ -│ │ └─ main.go -│ └─ internal/ -│ ├─ protocol/ -│ ├─ transport/ -│ ├─ server/ -│ ├─ peer/ -│ └─ latencylog/ -└─ latencysummary -``` - -其中： - -- `cmd/server`、`cmd/peer`、`cmd/latencysummary` 是 3 个可执行程序入口。 -- `cmd/internal/*` 是这些入口依赖的内部包，真正的核心逻辑都在这里。 -- 根目录中还存在一个名为 `latencysummary` 的文件；本文聚焦源码结构，因此不把它作为主要源码入口展开。 - -## 3. 分层视角看项目 - -如果从“分层”而不是“目录”来看，项目可以分成 5 层： - -1. 命令入口层 -2. 业务层 -3. 协议层 -4. 传输层 -5. 观测与分析层 - -对应关系如下： - -| 层次 | 目录/包 | 主要职责 | -| --- | --- | --- | -| 命令入口层 | `cmd/server` `cmd/peer` `cmd/latencysummary` | 解析参数，组装对象，启动主循环 | -| 业务层 | `cmd/internal/server` `cmd/internal/peer` | 注册、转发、发送、接收、持久化 | -| 协议层 | `cmd/internal/protocol` | 定义消息类型、编码解码、消息合法性校验 | -| 传输层 | `cmd/internal/transport` | 基于 TCP 发送/接收完整消息，处理并发写和 Linux 时间戳 | -| 观测与分析层 | `cmd/internal/latencylog` | 记录时延事件、汇总 JSONL、生成图表 | - -推荐把它理解为： - -- `protocol` 决定“消息长什么样” -- `transport` 决定“消息怎么在 TCP 上被完整收发” -- `server` 和 `peer` 决定“业务如何使用这些消息” -- `latencylog` 决定“如何观察一次消息从发送到落盘经历了什么” - -## 4. 三个可执行程序分别做什么 - -### 4.1 `server` - -入口在 `cmd/server/main.go`。 - -它做的事情很直接： - -1. 解析命令行参数，比如 `-listen` 和 `-latency-log` -2. 创建 `Hub` -3. 监听 TCP 地址 -4. 每接受一个连接，就交给 `Hub.ServeConn` 处理 - -`server` 自身不理解“聊天”“文件同步”之类的上层业务语义，它只知道： - -- 首条消息必须是 `register` -- 已注册 `peer` 只能发送 `text` 或 `file` -- 如果目标 `peer` 不存在，要回一个 `error` -- 如果目标连接失效，要清理连接并回一个 `error` - -可以把 `server` 看成一个中心路由器。 - -### 4.2 `peer` - -入口在 `cmd/peer/main.go`。 - -它负责： - -1. 用 `Dial` 连接 `server` -2. 连接建立后立刻发送 `register` -3. 根据参数决定是否发送一条初始消息 -4. 启动接收循环 -5. 把收到的文本或文件落盘到 `inbox-dir` -6. 如果启用了交互模式，在同一条长连接上反复发送多条消息 - -它支持的典型模式有两种： - -- 一次性模式：启动后发送一条 `text` 或 `file` -- 交互模式：启动后进入简单 REPL，持续复用同一条连接发消息 - -### 4.3 `latencysummary` - -入口在 `cmd/latencysummary/main.go`。 - -它不参与在线通信，只处理离线日志： - -1. 读取一个或多个原始时延 JSONL 文件 -2. 按消息聚合事件 -3. 计算多类时延指标 -4. 输出汇总 JSONL -5. 额外输出一个 HTML 图表 - -如果说 `server` 和 `peer` 是数据面，那么 `latencysummary` 就是观测面的离线分析工具。 - -## 5. 核心内部模块导读 - -### 5.1 `cmd/internal/protocol` - -这是协议层，决定消息的结构和线上的编码方式。 - -#### 主要类型 - -`Message` 是整个项目最核心的结构体，字段包括： - -- `Type` -- `ID` -- `From` -- `To` -- `FileName` -- `Body` - -#### 支持的消息类型 - -- `text`：正文按 UTF-8 文本解释 -- `file`：正文是原始文件字节，必须有 `FileName` -- `register`：`peer` 向 `server` 注册身份 -- `error`：`server` 返回错误信息 - -#### 它负责的事情 - -- 校验不同消息类型的字段约束 -- 把结构化消息编码为字节流 -- 从字节流还原为结构化消息 -- 处理帧边界，避免 TCP 粘包/拆包问题 - -#### 关键约束 - -- 所有消息都必须有 `From` 和 `To` -- `text` 不能带 `FileName` -- `text` 的 `Body` 必须是合法 UTF-8 -- `file` 必须有 `FileName` -- `register` 的目标必须是 `server` -- `register` 不能带正文 -- `error` 必须由 `server` 发出 -- 单帧最大大小为 `8 * 1024 * 1024` - -### 5.2 `cmd/internal/transport` - -这是传输层，负责把协议消息稳定地跑在一条 TCP 连接上。 - -核心类型是 `TCPConn`，它对 `net.Conn` 做了封装，提供： - -- `Send` -- `Receive` -- `ReceiveLoop` -- `Close` -- `CloseGracefully` - -#### 它解决了哪些问题 - -- 保证发送的是一整条消息，而不是半条 -- 通过写锁避免多个 goroutine 并发写时字节流互相交错 -- 在接收侧持续读取，直到拿到完整帧 -- 在 Linux 上启用 socket timestamping，记录发送和接收链路中的内核时间戳 - -#### Linux 相关实现 - -`tcp_linux.go` 是这个项目比较有特色的一块。它会尝试打开 Linux 的 timestamping 能力，并记录部分关键事件，例如： - -- `A_TX_SCHED` -- `A_TX_SOFTWARE` -- `B_RX_SOFTWARE` - -这也是为什么项目明确写了 `Linux only`。不是只有部署目标是 Linux，而是代码本身依赖 Linux 的 socket timestamping 能力。 - -### 5.3 `cmd/internal/server` - -这里只有一个核心概念：`Hub`。 - -可以把 `Hub` 理解为“在线连接中心”，它维护： - -- `peer ID -> TCPConn` 的映射 - -#### `Hub` 的主要职责 - -- 处理新连接的注册流程 -- 拒绝未注册连接直接发业务消息 -- 拒绝重复 `peer ID` -- 按目标 `peer ID` 查找连接并转发消息 -- 连接关闭或转发失败时清理注册表 - -#### `Hub` 的行为边界 - -它只负责转发和协议约束，不负责： - -- 业务持久化 -- 文件存储管理 -- 聊天记录管理 -- 权限控制 - -也就是说，目前它是一个很轻量的转发中心，而不是功能复杂的消息中间件。 - -### 5.4 `cmd/internal/peer` - -这个包是 `peer` 端的业务层。 - -#### `Client` - -`Client` 表示一个已经连接并注册到 `server` 的节点，主要方法包括： - -- `Dial` -- `SendText` -- `SendFile` -- `SendFilePath` -- `Receive` -- `ReceiveLoop` -- `PersistMessage` -- `Close` - -#### 它负责的事情 - -- 建立到 `server` 的 TCP 连接 -- 发送 `register` 完成身份注册 -- 为业务消息分配自增 `MessageID` -- 发送文本消息和文件消息 -- 接收来自 `server` 的转发消息或错误消息 -- 把收到的业务消息落盘 - -#### 持久化策略 - -接收侧落盘逻辑在 `persist.go`： - -- 文本消息会被追加到 `messages.log` -- 文件消息会被写成单独文件 -- 文件名格式是：`--` - -这样做的好处是： - -- 文本消息便于顺序追踪 -- 文件消息天然避免覆盖 -- 文件名里直接带了来源和消息 ID，方便回溯 - -#### 网络绑定能力 - -`peer` 还支持： - -- `-bind-ip`：指定本地源 IP -- `-bind-device`：指定 Linux 网络设备，例如 `eth0`、`wwan0` - -这对多网卡环境或特殊链路测试比较有用。 - -### 5.5 `cmd/internal/latencylog` - -这个包负责“记录”和“分析”两件事。 - -#### 记录侧 - -`logger.go` 定义了： - -- `Event` -- `Logger` -- `JSONLLogger` -- 一组 `LogMessageEvent` / `LogMessageEventAt` 辅助函数 - -当前业务上最重要的事件有： - -- `A_APP_PREP_BEGIN` -- `A_TX_SCHED` -- `A_TX_SOFTWARE` -- `B_RX_SOFTWARE` -- `B_APP_RECV` -- `B_PERSIST_BEGIN` -- `B_PERSIST_END` - -其中： - -- `A_*` 表示发送侧 -- `B_*` 表示接收侧 -- `TX/RX` 更偏内核或传输链路 -- `APP/PERSIST` 更偏应用层 - -#### 分析侧 - -`summary.go` 会把原始事件按消息聚合，并计算： - -- `AProcessingLatencyNS` -- `AQueueLatencyNS` -- `ABTransportPropagationNS` -- `BKernelReceivePathLatencyNS` -- `BProcessingLatencyNS` -- `EndToEndLatencyNS` -- `ApproxRTTNS` - -`summary_chart.go` 则把这些摘要结果渲染成 HTML 页面，方便快速观察不同消息的时延分布。 - -## 6. 核心消息流 - -这一节按一次完整消息生命周期来梳理。 - -### 6.1 连接与注册 - -1. `peer` 启动后调用 `Dial(serverAddr, peerID, ...)` -2. 底层建立 TCP 连接 -3. `peer` 立刻发送一条 `register` 消息 -4. `server` 的 `Hub.ServeConn` 先读取首条消息 -5. 如果首条消息不是 `register`，连接会被拒绝 -6. 如果 `peer ID` 重复，连接会被拒绝并收到 `error` -7. 注册成功后，`Hub` 把该连接加入在线表 - -注册阶段决定了后续所有转发的寻址基础。 - -### 6.2 发送文本或文件 - -发送侧调用： - -- `SendText(to, body)` 或 -- `SendFile(to, fileName, body)` 或 -- `SendFilePath(to, path)` - -发送前会生成新的 `MessageID`，然后： - -1. 记录发送前的应用层事件 `A_APP_PREP_BEGIN` -2. 交给 `transport.TCPConn.Send` -3. `transport` 调用协议层编码 -4. 编码后的消息被写入 TCP 连接 -5. Linux 侧尽量采集 `A_TX_SCHED` 和 `A_TX_SOFTWARE` - -### 6.3 `server` 转发 - -`server` 收到消息后： - -1. 确认消息类型只能是 `text` 或 `file` -2. 强制把 `msg.From` 改成当前已注册的 `peer ID` -3. 通过 `msg.To` 查找目标连接 -4. 找不到目标时返回 `error` -5. 找到目标就直接转发 - -这里有一个重要细节：`server` 不信任客户端自己填写的 `From`。即使发送端伪造了 `From`，`Hub` 也会用实际注册身份覆盖它。 - -### 6.4 接收与落盘 - -接收侧 `peer` 的接收循环拿到消息后： - -1. `transport` 在 Linux 下尝试记录 `B_RX_SOFTWARE` -2. `Client.Receive` / `ReceiveLoop` 记录 `B_APP_RECV` -3. 根据消息类型调用 `PersistMessage` -4. 持久化开始时记录 `B_PERSIST_BEGIN` -5. 写盘完成后记录 `B_PERSIST_END` - -因此，一条业务消息从“发送端开始准备”到“接收端落盘完成”形成了相对完整的一条时延链路。 - -### 6.5 时延日志汇总 - -后处理阶段由 `latencysummary` 完成： - -1. 用一个或多个 `-input` 指定原始 JSONL 日志 -2. 加载所有事件 -3. 按消息聚合 -4. 计算摘要时延 -5. 输出一个汇总 JSONL -6. 按输出文件名自动生成一个同名 HTML 图表 - -## 7. 协议说明 - -### 7.1 消息结构 - -业务层统一使用 `protocol.Message`： - -```text -Type / ID / From / To / FileName / Body -``` - -其中： - -- `FileName` 仅对 `file` 消息有意义 -- `Body` 不进入 header JSON，而是作为二进制正文附加在后面 - -### 7.2 TCP 上传输的帧格式 - -从 TCP 视角，完整格式可以理解为： - -```text -[4-byte frameLength][4-byte headerLen][header JSON][body bytes] -``` - -更细一点说： - -- `WriteFrame` 负责最外层的 `frameLength` -- `EncodeMessage` 负责 payload 内部的 `headerLen + header JSON + body` - -这样做的目的很明确：TCP 是字节流，不天然保留消息边界，所以要自己在协议层补齐边界信息。 - -### 7.3 错误语义 - -当前协议里的错误消息由 `server` 发送，类型是 `error`。常见场景包括： - -- 首条消息不是 `register` -- 重复注册相同 `peer ID` -- 已注册 `peer` 再次发送 `register` -- 目标 `peer` 不存在 -- 发送了不支持的消息类型 - -从设计上看，`error` 仍然走同一条消息通道，而不是额外开一个控制通道。 - -## 8. 时延日志机制 - -### 8.1 为什么项目里有这套日志 - -这个仓库不只是做“能发消息”，还明显在关注消息经过网络栈时的细粒度时延。尤其是： - -- 应用层开始准备消息的时间 -- 消息进入发送调度队列的时间 -- 消息进入软件发送路径的时间 -- 接收侧内核把数据交给协议栈的时间 -- 接收侧应用真正读到消息的时间 -- 接收侧写盘完成的时间 - -这些点能帮助区分： - -- 应用侧处理慢 -- 发送侧排队慢 -- 网络传输慢 -- 接收侧内核路径慢 -- 接收侧持久化慢 - -### 8.2 当前谁在打点 - -当前实际打点来源主要有两类： - -- `peer` 应用层：发送、接收、持久化 -- `transport` 传输层：Linux kernel timestamping - -`server` 代码里保留了 `WithLogger` 和 `-latency-log` 相关入口，但当前实现仍然把 `server` 视为黑盒转发器，不主动为转发过程写入业务级端到端事件。这一点从现有测试也能看出来：服务端转发路径默认不产出这类事件。 - -### 8.3 汇总结果怎么看 - -`latencysummary` 输出的摘要结果按“单条消息”聚合。阅读时可以重点看： - -- `EndToEndLatencyNS`：从发送侧准备开始到接收侧写盘完成 -- `AQueueLatencyNS`：发送端从进入调度到真正进入软件发送路径 -- `ABTransportPropagationNS`：从发送侧真正发出到接收侧应用读到 -- `BProcessingLatencyNS`：接收端应用读到后到写盘完成 - -如果某些事件缺失，摘要里会带 `MissingTimestamps`，告诉你少了哪些关键时间点。 - -## 9. 运行与调试补充 - -### 9.1 构建入口 - -按 `README.md` 当前给出的方式，主要构建命令是： - -```bash -go build -o bin/server ./cmd/server -go build -o bin/peer ./cmd/peer -go build -o bin/latencysummary ./cmd/latencysummary -``` - -也可以按不同 Linux 架构交叉编译。 - -### 9.2 `server` 常用参数 - -- `-listen`：监听地址，默认 `:9000` -- `-latency-log`：原始时延 JSONL 输出路径 - -### 9.3 `peer` 常用参数 - -- `-id`：当前节点 ID -- `-server`：服务端地址 -- `-to`：一次性发送时的目标 `peer` -- `-text`：一次性发送文本 -- `-file`：一次性发送文件 -- `-inbox-dir`：接收内容的落盘目录 -- `-bind-ip`：本地源 IP -- `-bind-device`：本地网络设备 -- `-latency-log`：原始时延 JSONL 输出路径 -- `-interactive`：是否启用交互式 REPL，默认开启 - -### 9.4 交互命令 - -交互模式支持： - -```text -help -text -file -quit -``` - -这让你可以在同一条长连接上连续发送多次，而不用每发一条消息就重启一次进程。 - -### 9.5 Linux-only 限制 - -这个项目应当被当作 Linux 项目来理解。 - -需要注意两层含义： - -- 部署目标是 Linux -- 代码实现也依赖 Linux 特性 - -在当前 Windows 环境下执行 `go test ./...`，会因为 `cmd/internal/transport` 中的 Linux 专属实现而构建失败。这属于平台限制，不代表仓库当前代码损坏。换句话说，这个失败更接近“当前平台不支持这套实现”，而不是“代码逻辑错误”。 - -## 10. 推荐阅读顺序 - -如果你刚接手项目，建议按下面顺序读： - -1. `README.md` -2. `cmd/server/main.go` -3. `cmd/peer/main.go` -4. `cmd/internal/protocol/message.go` -5. `cmd/internal/protocol/codec.go` -6. `cmd/internal/transport/tcp.go` -7. `cmd/internal/transport/tcp_linux.go` -8. `cmd/internal/server/hub.go` -9. `cmd/internal/peer/client.go` -10. `cmd/internal/peer/persist.go` -11. `cmd/internal/latencylog/logger.go` -12. `cmd/internal/latencylog/summary.go` -13. `cmd/internal/latencylog/summary_chart.go` - -这样阅读的好处是： - -- 先知道程序怎么启动 -- 再知道消息长什么样 -- 再知道消息怎么传 -- 再知道服务端和客户端各自做什么 -- 最后再看时延观测和分析 - -## 11. 你接手后最值得先记住的几件事 - -- 这是一个“中心转发”的系统，不是 `peer` 直连。 -- `register` 是连接建立后的第一条消息，缺了它后续都不成立。 -- `server` 会用已注册身份覆盖消息里的 `From`，不会信任客户端自报身份。 -- 文本消息和文件消息共享同一套协议，只是约束不同。 -- `transport` 不只是收发 TCP，还承担 Linux 时间戳采集。 -- 接收侧持久化是 `peer` 的职责，不是 `server` 的职责。 -- `latencysummary` 是离线分析工具，不在在线转发链路里。 - -如果后续你准备改协议、改传输层，或者新增消息类型，建议先把 `protocol -> transport -> peer/server -> latencylog` 这一整条链路一起过一遍，再开始动代码。 diff --git a/go/README.md b/go/README.md new file mode 100644 index 0000000..770f75f --- /dev/null +++ b/go/README.md @@ -0,0 +1,94 @@ +# OmniSocketGo + +Linux only. Go 1.22. + +如果目标机器只运行 `server`，只需要编译并拷贝 `server` 二进制。 +如果目标机器只运行 `peer`，只需要编译并拷贝 `peer` 二进制。 + +`go build ./cmd/server` 和 `go build ./cmd/peer` 会把各自依赖到的功能一起编译进最终二进制，不需要再单独编译 `cmd/internal/...` 包。 + +- `server` 二进制会包含它依赖到的转发、协议、传输等代码 +- `peer` 二进制会包含它依赖到的注册、交互发送、接收落盘、协议、传输等代码 +- 只有没有被这个可执行程序引用的其他命令，才不在该二进制里，比如 `cmd/latencysummary` + +## Build + +按目标架构分别编译。 + mkdir -p bin + go build -o bin/server ./cmd/server + go build -o bin/peer ./cmd/peer + go build -o bin/latencysummary ./cmd/latencysummary + +### Linux amd64 + +```bash +CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o bin/server-linux-amd64 ./cmd/server +``` + +### Linux arm64 + +```bash +CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o bin/peer-linux-arm64 ./cmd/peer +``` + +### Linux armv7 + +```bash +CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -o bin/server-linux-armv7 ./cmd/server +CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -o bin/peer-linux-armv7 ./cmd/peer +``` + + + +## Run On Different Machines + +`server D` 所在机器监听 `0.0.0.0:10909`。 + +```bash +go run cmd/kcpserver/ -listen 0.0.0.0:10909 +-kcp-ts-debug-log logs/d-kcp-ts.jsonl -kcp-session-stats-log logs/d-kcp-stats.jsonl +``` + +`relay server C` 所在机器 + +```bash +go run ./cmd/kcpserver/ -mode=relay -listen 0.0.0.0:10909 -relay-remote 172.21.32.15:10909 + +2>&1 | tee logs/c.stdout.log +``` + +### peer-a (A) + +```bash +go run ./cmd/kcppeer/ -id peer-a -server 172.21.32.15:10909 -relay-via 106.55.173.235:10909 -inbox-dir inbox/a + +-latency-log logs/a-latency.jsonl -kcp-ts-debug-log logs/a-kcp-ts.jsonl -kcp-session-stats-log logs/a-kcp-stats.jsonl + +go run ./cmd/kcpping/ -id peer-a -server 106.55.173.235:10909 -echo +``` + +### peer-b (B) + +```bash +go run ./cmd/kcppeer/ -id peer-b -server 81.70.156.140:10909 -inbox-dir inbox/b + +-latency-log logs/b-latency.jsonl -kcp-ts-debug-log logs/b-kcp-ts.jsonl -kcp-session-stats-log logs/b-kcp-stats.jsonl + +go run ./cmd/kcpping -id peer-b -server 81.70.156.140:10909 -to peer-a -count 20 -interval 100ms +``` + +## Interactive Commands + +`peer` 启动后可以在终端里持续使用同一条长连接发送多次消息。 + +```text +help +text peer-b hello +text peer-a hi +file peer-a /tmp/test125.bin +file peer-a /tmp/test5.bin +quit +``` +### 自动化拉取更新汇总数据 +cd /home/limingjie/LMJ_Work/OmniSocketGo +./scripts/refresh-latency-summary.sh \ No newline at end of file diff --git a/change_to_c.md b/go/change_to_c.md similarity index 100% rename from change_to_c.md rename to go/change_to_c.md diff --git a/cmd/internal/latencylog/logger.go b/go/cmd/internal/latencylog/logger.go similarity index 100% rename from cmd/internal/latencylog/logger.go rename to go/cmd/internal/latencylog/logger.go diff --git a/cmd/internal/latencylog/logger_test.go b/go/cmd/internal/latencylog/logger_test.go similarity index 100% rename from cmd/internal/latencylog/logger_test.go rename to go/cmd/internal/latencylog/logger_test.go diff --git a/cmd/internal/latencylog/summary.go b/go/cmd/internal/latencylog/summary.go similarity index 100% rename from cmd/internal/latencylog/summary.go rename to go/cmd/internal/latencylog/summary.go diff --git a/cmd/internal/latencylog/summary_chart.go b/go/cmd/internal/latencylog/summary_chart.go similarity index 100% rename from cmd/internal/latencylog/summary_chart.go rename to go/cmd/internal/latencylog/summary_chart.go diff --git a/cmd/internal/latencylog/summary_chart_test.go b/go/cmd/internal/latencylog/summary_chart_test.go similarity index 100% rename from cmd/internal/latencylog/summary_chart_test.go rename to go/cmd/internal/latencylog/summary_chart_test.go diff --git a/cmd/internal/latencylog/summary_test.go b/go/cmd/internal/latencylog/summary_test.go similarity index 100% rename from cmd/internal/latencylog/summary_test.go rename to go/cmd/internal/latencylog/summary_test.go diff --git a/cmd/internal/protocol/codec.go b/go/cmd/internal/protocol/codec.go similarity index 100% rename from cmd/internal/protocol/codec.go rename to go/cmd/internal/protocol/codec.go diff --git a/cmd/internal/protocol/codec_test.go b/go/cmd/internal/protocol/codec_test.go similarity index 100% rename from cmd/internal/protocol/codec_test.go rename to go/cmd/internal/protocol/codec_test.go diff --git a/cmd/internal/protocol/message.go b/go/cmd/internal/protocol/message.go similarity index 100% rename from cmd/internal/protocol/message.go rename to go/cmd/internal/protocol/message.go diff --git a/cmd/latencysummary/main.go b/go/cmd/latencysummary/main.go similarity index 100% rename from cmd/latencysummary/main.go rename to go/cmd/latencysummary/main.go diff --git a/go.mod b/go/go.mod similarity index 100% rename from go.mod rename to go/go.mod diff --git a/go.sum b/go/go.sum similarity index 100% rename from go.sum rename to go/go.sum diff --git a/c/include/cli_parse.h b/include/cli_parse.h similarity index 100% rename from c/include/cli_parse.h rename to include/cli_parse.h diff --git a/c/include/interactive.h b/include/interactive.h similarity index 100% rename from c/include/interactive.h rename to include/interactive.h diff --git a/c/include/kcp_packet_debug.h b/include/kcp_packet_debug.h similarity index 100% rename from c/include/kcp_packet_debug.h rename to include/kcp_packet_debug.h diff --git a/c/include/kcp_session_stats.h b/include/kcp_session_stats.h similarity index 100% rename from c/include/kcp_session_stats.h rename to include/kcp_session_stats.h diff --git a/c/include/latencylog.h b/include/latencylog.h similarity index 100% rename from c/include/latencylog.h rename to include/latencylog.h diff --git a/c/include/linux_timestamping.h b/include/linux_timestamping.h similarity index 100% rename from c/include/linux_timestamping.h rename to include/linux_timestamping.h diff --git a/c/include/omni_common.h b/include/omni_common.h similarity index 100% rename from c/include/omni_common.h rename to include/omni_common.h diff --git a/c/include/peer_kcp_client.h b/include/peer_kcp_client.h similarity index 100% rename from c/include/peer_kcp_client.h rename to include/peer_kcp_client.h diff --git a/c/include/peer_udp_client.h b/include/peer_udp_client.h similarity index 100% rename from c/include/peer_udp_client.h rename to include/peer_udp_client.h diff --git a/c/include/protocol.h b/include/protocol.h similarity index 100% rename from c/include/protocol.h rename to include/protocol.h diff --git a/c/include/server_kcp_hub.h b/include/server_kcp_hub.h similarity index 100% rename from c/include/server_kcp_hub.h rename to include/server_kcp_hub.h diff --git a/c/include/server_udp_hub.h b/include/server_udp_hub.h similarity index 100% rename from c/include/server_udp_hub.h rename to include/server_udp_hub.h diff --git a/c/include/server_udp_relay.h b/include/server_udp_relay.h similarity index 100% rename from c/include/server_udp_relay.h rename to include/server_udp_relay.h diff --git a/c/include/transport_kcp.h b/include/transport_kcp.h similarity index 100% rename from c/include/transport_kcp.h rename to include/transport_kcp.h diff --git a/c/include/transport_udp.h b/include/transport_udp.h similarity index 100% rename from c/include/transport_udp.h rename to include/transport_udp.h diff --git a/c/include/tx_timestamp_debug.h b/include/tx_timestamp_debug.h similarity index 100% rename from c/include/tx_timestamp_debug.h rename to include/tx_timestamp_debug.h diff --git a/c/src/interactive.c b/src/interactive.c similarity index 100% rename from c/src/interactive.c rename to src/interactive.c diff --git a/c/src/kcp_packet_debug.c b/src/kcp_packet_debug.c similarity index 100% rename from c/src/kcp_packet_debug.c rename to src/kcp_packet_debug.c diff --git a/c/src/kcp_session_stats.c b/src/kcp_session_stats.c similarity index 100% rename from c/src/kcp_session_stats.c rename to src/kcp_session_stats.c diff --git a/c/src/latencylog.c b/src/latencylog.c similarity index 100% rename from c/src/latencylog.c rename to src/latencylog.c diff --git a/c/src/linux_timestamping.c b/src/linux_timestamping.c similarity index 100% rename from c/src/linux_timestamping.c rename to src/linux_timestamping.c diff --git a/c/src/omni_common.c b/src/omni_common.c similarity index 100% rename from c/src/omni_common.c rename to src/omni_common.c diff --git a/c/src/peer_kcp_client.c b/src/peer_kcp_client.c similarity index 100% rename from c/src/peer_kcp_client.c rename to src/peer_kcp_client.c diff --git a/c/src/peer_udp_client.c b/src/peer_udp_client.c similarity index 100% rename from c/src/peer_udp_client.c rename to src/peer_udp_client.c diff --git a/c/src/protocol.c b/src/protocol.c similarity index 100% rename from c/src/protocol.c rename to src/protocol.c diff --git a/c/src/server_kcp_hub.c b/src/server_kcp_hub.c similarity index 100% rename from c/src/server_kcp_hub.c rename to src/server_kcp_hub.c diff --git a/c/src/server_udp_hub.c b/src/server_udp_hub.c similarity index 100% rename from c/src/server_udp_hub.c rename to src/server_udp_hub.c diff --git a/c/src/server_udp_relay.c b/src/server_udp_relay.c similarity index 100% rename from c/src/server_udp_relay.c rename to src/server_udp_relay.c diff --git a/c/src/transport_kcp.c b/src/transport_kcp.c similarity index 100% rename from c/src/transport_kcp.c rename to src/transport_kcp.c diff --git a/c/src/transport_udp.c b/src/transport_udp.c similarity index 100% rename from c/src/transport_udp.c rename to src/transport_udp.c diff --git a/c/src/tx_timestamp_debug.c b/src/tx_timestamp_debug.c similarity index 100% rename from c/src/tx_timestamp_debug.c rename to src/tx_timestamp_debug.c diff --git a/c/third_party/cjson/cJSON.c b/third_party/cjson/cJSON.c similarity index 100% rename from c/third_party/cjson/cJSON.c rename to third_party/cjson/cJSON.c diff --git a/c/third_party/cjson/cJSON.h b/third_party/cjson/cJSON.h similarity index 100% rename from c/third_party/cjson/cJSON.h rename to third_party/cjson/cJSON.h diff --git a/c/third_party/kcp/ikcp.c b/third_party/kcp/ikcp.c similarity index 100% rename from c/third_party/kcp/ikcp.c rename to third_party/kcp/ikcp.c diff --git a/c/third_party/kcp/ikcp.h b/third_party/kcp/ikcp.h similarity index 100% rename from c/third_party/kcp/ikcp.h rename to third_party/kcp/ikcp.h