AlexStocks commented on code in PR #3502: URL: https://github.com/apache/dubbo-go/pull/3502#discussion_r3662224633
########## tools/benchmark/README.md: ########## @@ -0,0 +1,300 @@ +# Dubbo-Go Benchmark Suite + +English | [中文](README_CN.md) + +Performance benchmark suite for comparing **Dubbo-Go / Dubbo-Java / gRPC** frameworks. + +## Environment Requirements + +- **Go**: 1.23+ +- **Java**: 8+ +- **Maven**: 3.6+ +- **protoc**: 3.0+ + +## Default Port Configuration + +| Framework | Default Port | +|-----------|-------------| +| Dubbo-Go | 20000 | +| Dubbo-Java | 20001 | +| gRPC | 50051 | + +## Directory Structure + +``` +tools/benchmark +├── client/ # Benchmark client +│ ├── main.go # Entry point +│ ├── clients/ # Client implementations +│ │ ├── dubbo_client.go # Dubbo-Go client +│ │ └── grpc_client.go # gRPC client +│ ├── engine/ # Benchmark engine +│ │ ├── engine.go # Engine logic +│ │ ├── statistics.go # Statistics calculation +│ │ └── metrics.go # Metrics collection +│ ├── monitor/ # System monitor +│ │ └── system_monitor.go # CPU/Memory monitor +│ └── payload/ # Payload generator +│ └── payload.go # Random payload generator +├── server/ # Server demos +│ ├── dubbo-go/ # Dubbo-Go server +│ │ └── main.go +│ ├── dubbo-java/ # Dubbo-Java server +│ │ └── pom.xml +│ └── grpc/ # gRPC server +│ └── main.go +├── proto/ # Protocol definitions and generated code +│ ├── benchmark.proto # Protobuf definition +│ ├── benchmark.pb.go # Generated Go code +│ ├── benchmark_grpc.pb.go # Generated gRPC code +│ └── benchmark.triple.go # Generated Triple code +├── scripts/ # Automation scripts +│ ├── gen_code.sh # Protobuf code generation +│ ├── run_all.sh # Run all benchmarks +│ └── run_single.sh # Run single benchmark +├── config.yaml # Benchmark configuration +├── go.mod/go.sum # Go dependencies Review Comment: [P2] go.mod/go.sum 实际未包含在本 PR 本 PR 的 30 个文件中不存在 tools/benchmark/go.mod/go.sum，但 README 目录结构在此行列出 go.mod/go.sum。仓库 tools/ 下其余子工具（dubbogo-cli、imports-formatter、protoc-gen-go-triple、protoc-gen-triple-openapi）均为独立 Go 模块。缺 go.mod 使 benchmark 并入根模块参与 go build ./...（RISC-V CI 虽通过），但与同级工具约定不一致且 README 与事实不符。建议补 module dubbo.apache.org/dubbo-go/v3/tools/benchmark 并 replace dubbo.apache.org/dubbo-go/v3 => ../../..，或从 README 移除该行。 ########## tools/benchmark/scripts/run_all.sh: ########## @@ -0,0 +1,168 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +set -e + +BASE_DIR=$(cd "$(dirname "$0")/.." && pwd) +LOG_DIR="$BASE_DIR/logs" +REPORT_DIR="$BASE_DIR/report" +DATA_DIR="$BASE_DIR/data" +PID_FILE="/tmp/benchmark_server.pid" +SEPARATOR="========================================" + +mkdir -p "$LOG_DIR" +mkdir -p "$REPORT_DIR" +mkdir -p "$DATA_DIR" + +echo "$SEPARATOR" +echo " Dubbo-Go Benchmark - Full Test Suite" +echo "$SEPARATOR" + +echo "[INFO] Checking environment dependencies..." + +if ! command -v go > /dev/null 2>&1; then + echo "[ERROR] Go not installed, please install Go 1.25+" + exit 1 +fi + +echo "[INFO] Environment check passed" + +cleanup() { + echo "[INFO] Cleaning up resources..." + if [ -f "$PID_FILE" ]; then + pid=$(cat "$PID_FILE") + if kill -0 "$pid" 2>/dev/null; then + kill "$pid" 2>/dev/null || true + sleep 2 + if kill -0 "$pid" 2>/dev/null; then + kill -9 "$pid" 2>/dev/null || true + fi + fi + rm -f "$PID_FILE" + fi +} + +trap cleanup EXIT + +echo "" +echo "[INFO] Compiling Dubbo-Go server..." +cd "$BASE_DIR/server/dubbo-go" +go build -o benchmark-dubbo-go main.go + +echo "[INFO] Compiling gRPC server..." +cd "$BASE_DIR/server/grpc" +go build -o benchmark-grpc main.go + +echo "" +echo "[INFO] Compiling benchmark client..." +cd "$BASE_DIR/client" +go build -o benchmark-client main.go + +echo "" +echo "[INFO] Starting full benchmark suite..." + +FRAMEWORKS=("dubbo-go" "grpc") Review Comment: [P2] run_all.sh 不覆盖 dubbo-java FRAMEWORKS 仅含 dubbo-go/grpc，无 Maven/java -jar 分支；但 PR 标题与 README 宣称 Dubbo-Go/Dubbo-Java/gRPC 三者对比且结果表含 dubbo-java 列。dubbo-java 用 dubbo+Hessian2 POJO（与 dubbo-go Triple+protobuf 协议不互通）且无 streamCall，故 full suite 无法复现 dubbo-java 结果。建议接入 dubbo-java 自动运行，或在 README 明示其需单独运行、结果表来源。 ########## tools/benchmark/client/main.go: ########## @@ -0,0 +1,245 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package main + +import ( + "context" + "encoding/json" + "flag" + "fmt" + "os" + "path/filepath" + "time" +) + +import ( + "github.com/dubbogo/gost/log/logger" +) + +import ( + "dubbo.apache.org/dubbo-go/v3/tools/benchmark/client/clients" + "dubbo.apache.org/dubbo-go/v3/tools/benchmark/client/engine" + "dubbo.apache.org/dubbo-go/v3/tools/benchmark/client/monitor" + "dubbo.apache.org/dubbo-go/v3/tools/benchmark/client/payload" +) + +const ( + FrameworkDubboGo = "dubbo-go" + FrameworkGRPC = "grpc" + Separator = "========================================" +) + +var ( + framework = flag.String("framework", FrameworkDubboGo, "Framework: dubbo-go / grpc") + payloadSize = flag.Int("payload", 1024, "Payload size (bytes)") + serialization = flag.String("serialization", "protobuf", "Serialization protocol: hessian2 / protobuf / msgpack") + compression = flag.String("compression", "none", "Compression strategy: none / default / fastest") + concurrency = flag.Int("concurrency", 100, "Concurrency level") + callMode = flag.String("mode", "unary", "Call mode: unary / streaming") + testDuration = flag.String("duration", "60s", "Test duration") + warmupDuration = flag.String("warmup", "10s", "Warmup duration") + serverAddr = flag.String("addr", "", "Server address") + serverPID = flag.Int("pid", 0, "Server process PID (for system monitoring)") +) + +type Caller interface { + Call(ctx context.Context) error + Close() error + String() string +} + +type BenchmarkResult struct { + Framework string `json:"framework"` + PayloadSize int `json:"payload_size"` + Serialization string `json:"serialization"` + Compression string `json:"compression"` + Concurrency int `json:"concurrency"` + CallMode string `json:"call_mode"` + Timestamp string `json:"timestamp"` + QPS float64 `json:"qps"` + SuccessRate float64 `json:"success_rate"` + TotalRequests int64 `json:"total_requests"` + SuccessRequests int64 `json:"success_requests"` + FailureRequests int64 `json:"failure_requests"` + LatencyP50 float64 `json:"latency_p50_ms"` + LatencyP90 float64 `json:"latency_p90_ms"` + LatencyP95 float64 `json:"latency_p95_ms"` + LatencyP99 float64 `json:"latency_p99_ms"` + LatencyMin float64 `json:"latency_min_ms"` + LatencyMax float64 `json:"latency_max_ms"` + LatencyAvg float64 `json:"latency_avg_ms"` + CPUAvg float64 `json:"cpu_avg_percent"` + MemoryPeak float64 `json:"memory_peak_mb"` +} + +func main() { + flag.Parse() + + logger.Info(Separator) + logger.Info(" Dubbo-Go Benchmark Client") + logger.Info(Separator) + logger.Infof("Framework: %s", *framework) + logger.Infof("Payload Size: %d bytes", *payloadSize) + logger.Infof("Serialization: %s", *serialization) + logger.Infof("Compression: %s", *compression) + logger.Infof("Concurrency: %d", *concurrency) + logger.Infof("Call Mode: %s", *callMode) + logger.Infof("Warmup Duration: %s", *warmupDuration) + logger.Infof("Test Duration: %s", *testDuration) + if *serverAddr != "" { + logger.Infof("Server Address: %s", *serverAddr) + } + if *serverPID != 0 { + logger.Infof("Server PID: %d", *serverPID) + } + logger.Info(Separator) + + testDur, err := time.ParseDuration(*testDuration) + if err != nil { + logger.Fatalf("Invalid test duration: %v", err) + } + + warmupDur, err := time.ParseDuration(*warmupDuration) + if err != nil { + logger.Fatalf("Invalid warmup duration: %v", err) + } + + pg := payload.NewPayloadGenerator() + data := pg.Generate(*payloadSize) + logger.Infof("[INFO] Payload data generated, size: %d bytes", len(data)) + + caller, err := createCaller(data) + if err != nil { + logger.Fatalf("Failed to create client: %v", err) + } + defer caller.Close() + + var sysMonitor *monitor.SystemMonitor + if *serverPID != 0 { + sysMonitor = monitor.NewSystemMonitor(*serverPID, 1*time.Second) + sysMonitor.Start() + defer sysMonitor.Stop() + logger.Infof("[INFO] System monitor started, monitoring PID: %d", *serverPID) + } + + benchEngine := engine.NewEngine(*concurrency, warmupDur, testDur, 30*time.Second) + + logger.Info("[INFO] Starting benchmark...") + stats := benchEngine.Run(func(ctx context.Context) (time.Duration, error) { + start := time.Now() + err := caller.Call(ctx) + return time.Since(start), err + }) + + logger.Info(stats.String()) + + cpuAvg, memoryPeakBytes := 0.0, uint64(0) + if sysMonitor != nil { + cpuAvg, memoryPeakBytes = sysMonitor.GetSummary() + logger.Info(sysMonitor.String()) + } + + saveResults(stats, cpuAvg, float64(memoryPeakBytes)/1024/1024) +} + +func createCaller(data []byte) (Caller, error) { + addr := *serverAddr + if addr == "" { + switch *framework { + case FrameworkDubboGo: + addr = "127.0.0.1:20000" + case FrameworkGRPC: + addr = "127.0.0.1:50051" + default: + addr = "127.0.0.1:20000" + } + } + + switch *framework { + case FrameworkDubboGo: + return clients.NewDubboGoClient(addr, *serialization, *compression, *callMode, data) + case FrameworkGRPC: + return clients.NewGrpcClient(addr, *callMode, data) + default: + return nil, fmt.Errorf("unsupported framework: %s", *framework) + } +} + +func saveResults(stats *engine.Statistics, cpuAvg, memoryPeak float64) { + result := &BenchmarkResult{ + Framework: *framework, + PayloadSize: *payloadSize, + Serialization: *serialization, + Compression: *compression, + Concurrency: *concurrency, + CallMode: *callMode, + Timestamp: time.Now().Format("2006-01-02 15:04:05"), + QPS: stats.QPS, + SuccessRate: stats.SuccessRate, + TotalRequests: stats.Total, + SuccessRequests: stats.Success, + FailureRequests: stats.Failure, + LatencyP50: float64(stats.P50) / float64(time.Millisecond), + LatencyP90: float64(stats.P90) / float64(time.Millisecond), + LatencyP95: float64(stats.P95) / float64(time.Millisecond), + LatencyP99: float64(stats.P99) / float64(time.Millisecond), + LatencyMin: float64(stats.Min) / float64(time.Millisecond), + LatencyMax: float64(stats.Max) / float64(time.Millisecond), + LatencyAvg: float64(stats.Avg) / float64(time.Millisecond), + CPUAvg: cpuAvg, + MemoryPeak: memoryPeak, + } + + execPath, err := os.Executable() + if err != nil { + logger.Warnf("[WARN] Failed to get executable path: %v", err) + return + } + baseDir := filepath.Dir(filepath.Dir(execPath)) Review Comment: [P2] go run 下结果写入临时目录 baseDir 取自 os.Executable() 的祖辈目录；README 文档的 go run client/main.go 方式下可执行文件位于临时构建目录，结果 data/ 被写入临时目录而非 benchmark 仓（仅 go build -o 编译二进制方式才正确）。建议改用工作目录或显式 --output 参数解析输出路径。 ########## tools/benchmark/client/main.go: ########## @@ -0,0 +1,245 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package main + +import ( + "context" + "encoding/json" + "flag" + "fmt" + "os" + "path/filepath" + "time" +) + +import ( + "github.com/dubbogo/gost/log/logger" +) + +import ( + "dubbo.apache.org/dubbo-go/v3/tools/benchmark/client/clients" + "dubbo.apache.org/dubbo-go/v3/tools/benchmark/client/engine" + "dubbo.apache.org/dubbo-go/v3/tools/benchmark/client/monitor" + "dubbo.apache.org/dubbo-go/v3/tools/benchmark/client/payload" +) + +const ( + FrameworkDubboGo = "dubbo-go" + FrameworkGRPC = "grpc" + Separator = "========================================" +) + +var ( + framework = flag.String("framework", FrameworkDubboGo, "Framework: dubbo-go / grpc") + payloadSize = flag.Int("payload", 1024, "Payload size (bytes)") + serialization = flag.String("serialization", "protobuf", "Serialization protocol: hessian2 / protobuf / msgpack") + compression = flag.String("compression", "none", "Compression strategy: none / default / fastest") + concurrency = flag.Int("concurrency", 100, "Concurrency level") + callMode = flag.String("mode", "unary", "Call mode: unary / streaming") + testDuration = flag.String("duration", "60s", "Test duration") + warmupDuration = flag.String("warmup", "10s", "Warmup duration") + serverAddr = flag.String("addr", "", "Server address") + serverPID = flag.Int("pid", 0, "Server process PID (for system monitoring)") +) + +type Caller interface { + Call(ctx context.Context) error + Close() error + String() string +} + +type BenchmarkResult struct { + Framework string `json:"framework"` + PayloadSize int `json:"payload_size"` + Serialization string `json:"serialization"` + Compression string `json:"compression"` + Concurrency int `json:"concurrency"` + CallMode string `json:"call_mode"` + Timestamp string `json:"timestamp"` + QPS float64 `json:"qps"` + SuccessRate float64 `json:"success_rate"` + TotalRequests int64 `json:"total_requests"` + SuccessRequests int64 `json:"success_requests"` + FailureRequests int64 `json:"failure_requests"` + LatencyP50 float64 `json:"latency_p50_ms"` + LatencyP90 float64 `json:"latency_p90_ms"` + LatencyP95 float64 `json:"latency_p95_ms"` + LatencyP99 float64 `json:"latency_p99_ms"` + LatencyMin float64 `json:"latency_min_ms"` + LatencyMax float64 `json:"latency_max_ms"` + LatencyAvg float64 `json:"latency_avg_ms"` + CPUAvg float64 `json:"cpu_avg_percent"` + MemoryPeak float64 `json:"memory_peak_mb"` +} + +func main() { + flag.Parse() + + logger.Info(Separator) + logger.Info(" Dubbo-Go Benchmark Client") + logger.Info(Separator) + logger.Infof("Framework: %s", *framework) + logger.Infof("Payload Size: %d bytes", *payloadSize) + logger.Infof("Serialization: %s", *serialization) + logger.Infof("Compression: %s", *compression) + logger.Infof("Concurrency: %d", *concurrency) + logger.Infof("Call Mode: %s", *callMode) + logger.Infof("Warmup Duration: %s", *warmupDuration) + logger.Infof("Test Duration: %s", *testDuration) + if *serverAddr != "" { + logger.Infof("Server Address: %s", *serverAddr) + } + if *serverPID != 0 { + logger.Infof("Server PID: %d", *serverPID) + } + logger.Info(Separator) + + testDur, err := time.ParseDuration(*testDuration) + if err != nil { + logger.Fatalf("Invalid test duration: %v", err) + } + + warmupDur, err := time.ParseDuration(*warmupDuration) + if err != nil { + logger.Fatalf("Invalid warmup duration: %v", err) + } + + pg := payload.NewPayloadGenerator() + data := pg.Generate(*payloadSize) + logger.Infof("[INFO] Payload data generated, size: %d bytes", len(data)) + + caller, err := createCaller(data) + if err != nil { + logger.Fatalf("Failed to create client: %v", err) + } + defer caller.Close() Review Comment: [P2] 客户端缺少信号处理 此处 defer caller.Close()（含 graceful_shutdown.Shutdown）在收到 SIGINT 时不会执行（client/main.go 无 signal.Notify），与 PR 描述 All ports properly released 不一致；server/dubbo-go 与 server/grpc 均已处理 SIGINT。建议客户端也捕获 SIGINT/SIGTERM 并走 graceful shutdown。 ########## tools/benchmark/client/monitor/system_monitor.go: ########## @@ -0,0 +1,181 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package monitor + +import ( + "fmt" + "os/exec" + "strconv" + "strings" + "sync" + "time" +) + +type SystemMetrics struct { + CPUUsage float64 + MemoryUsage uint64 + Timestamp time.Time +} + +type SystemMonitor struct { + pid int + interval time.Duration + metrics []SystemMetrics + mu sync.Mutex + stopChan chan struct{} + wg sync.WaitGroup +} + +func NewSystemMonitor(pid int, interval time.Duration) *SystemMonitor { + return &SystemMonitor{ + pid: pid, + interval: interval, + metrics: make([]SystemMetrics, 0), + stopChan: make(chan struct{}), + } +} + +func (sm *SystemMonitor) Start() { + sm.wg.Add(1) + go sm.monitor() +} + +func (sm *SystemMonitor) Stop() { + close(sm.stopChan) + sm.wg.Wait() +} + +func (sm *SystemMonitor) monitor() { + defer sm.wg.Done() + + ticker := time.NewTicker(sm.interval) + defer ticker.Stop() + + for { + select { + case <-sm.stopChan: + return + case <-ticker.C: + metrics, err := sm.collectMetrics() + if err != nil { + continue + } + sm.mu.Lock() + sm.metrics = append(sm.metrics, metrics) + sm.mu.Unlock() + } + } +} + +func (sm *SystemMonitor) collectMetrics() (SystemMetrics, error) { + metrics := SystemMetrics{ + Timestamp: time.Now(), + } + + cpu, err := sm.getCPUUsage() + if err != nil { + return metrics, err + } + metrics.CPUUsage = cpu + + mem, err := sm.getMemoryUsage() + if err != nil { + return metrics, err + } + metrics.MemoryUsage = mem + + return metrics, nil +} + +func (sm *SystemMonitor) getCPUUsage() (float64, error) { + psPath, err := exec.LookPath("ps") Review Comment: [P2] CPU/内存监控测量方法与平台限制 ps -p PID -o %cpu= 返回进程生命周期平均 CPU，非测试窗口瞬时值，GetSummary 的 Avg CPU 实为累计平均的再平均；且 ps 在 Windows 不存在，--pid 监控在 Windows 静默返回 0（README 未标注）。建议用跨平台库（如 gopsutil）取瞬时 CPU，或在 README 标注系统监控仅支持 Unix。 ########## tools/benchmark/config.yaml: ########## @@ -0,0 +1,55 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +service: + name: BenchmarkService + port: + dubbo-go: 20000 + dubbo-java: 20001 + grpc: 50051 + +payload_sizes: + - 128 + - 1024 + - 16384 + - 1048576 + +serializations: Review Comment: [P2] config.yaml 未被二进制加载 main.go 仅用 flag，从不读取 config.yaml；run_all.sh 又硬编码子集（SERIALIZATIONS=(protobuf) 等），与本文件列出的 hessian2/protobuf/msgpack、none/default/fastest、unary/streaming 不一致。建议让二进制读取 config.yaml，或从仓库移除该文件以免误导。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
