sreekanth-db opened a new pull request, #3664: URL: https://github.com/apache/arrow-adbc/pull/3664
This PR implements phase 2 ([telemetry-activity-based-design.md](https://github.com/apache/arrow-adbc/pull/3636/files)) of the Activity-based telemetry system for the Databricks ADBC C# driver. It builds upon [Phase 1](https://github.com/apache/arrow-adbc/pull/3653) by adding the foundational infrastructure for collecting and exporting telemetry metrics. ## Changes ### New Files Added #### 1. **TelemetryConfiguration.cs** Configuration settings for Databricks telemetry collection and export. **Key properties:** - `Enabled`: Enable/disable telemetry - `BatchSize`: Number of metrics to batch before sending - `FlushIntervalMs`: Interval for periodic flush - `MaxRetries`, `RetryDelayMs`: Retry configuration - `CircuitBreakerEnabled`, `CircuitBreakerThreshold`, `CircuitBreakerTimeout`: Circuit breaker settings #### 2. **TelemetryMetric.cs** Data model representing a single telemetry event derived from Activity data. **Properties:** - `EventType`: Type of telemetry event (ConnectionOpen, StatementExecution, Error) - `Timestamp`: When the event occurred - `Tags`: Flexible dictionary for event-specific metadata #### 3. **ITelemetryExporter.cs** Interface for exporting telemetry metrics to external services. **Contract:** - Single async method: `ExportAsync(IReadOnlyList<TelemetryMetric> metrics, CancellationToken cancellationToken)` - **Critical requirement**: Implementations must never throw exceptions #### 4. **DatabricksTelemetryExporter.cs** HTTP-based exporter implementation that sends metrics to Databricks telemetry service. **Features:** - Sends metrics to `/telemetry-ext` endpoint - Uses `UriBuilder` for proper URL construction - Non-blocking error handling (never throws exceptions) ## Design Decisions - **Simplified first version**: Focuses on core functionality without retry logic, circuit breaker implementation or unauthenticated telemetry support (can be enhanced later) ## What's Left in Phase 2 Two components remain for separate review (because of their complex implementation, will raise a separate PR for better review process): - **MetricsAggregator**: Batching logic, event type aggregation - **DatabricksActivityListener**: Activity event subscription, conversion to `TelemetryMetric` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
