rkhachatryan commented on code in PR #23845: URL: https://github.com/apache/flink/pull/23845#discussion_r1417402466
########## docs/content/docs/ops/traces.md: ########## @@ -0,0 +1,126 @@ +--- +title: "Metrics" +weight: 6 +type: docs +aliases: + - /ops/traces.html + - /apis/traces.html + - /monitoring/traces.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Traces + +Flink exposes a tracing system that allows gathering and exposing traces to external systems. + +## Reporting traces + +You can access the tracing system from any user function that extends [RichFunction]({{< ref "docs/dev/datastream/user_defined_functions" >}}#rich-functions) by calling `getRuntimeContext().getMetricGroup()`. +This method returns a `MetricGroup` object via which you can report a new single span trace. + +### Reporting single Span + + +A `Span` represents something that happened in Flink at certain point of time, that will be reported to a `TraceReporter`. +To report a `Span` you can use the `MetricGroup#addSpan(SpanBuilder)` method. + +Currently we don't support traces with multiple spans. Each `Span` is self-contained and represents things like a checkpoint or recovery. +{{< tabs "9612d275-bdda-4322-a01f-ae6da805e917" >}} +{{< tab "Java" >}} +```java +public class MyClass { + void doSomething() { + // (...) + metricGroup.addSpan( + Span.builder(MyClass.class, "SomeAction") + .setStartTsMillis(startTs) // Optional + .setEndTsMillis(endTs) // Optional + .setAttribute("foo", "bar"); + } +} +``` +{{< /tab >}} +{{< tab "Python" >}} +```python +Currently reporting Spans from Python is not supported. +``` +{{< /tab >}} +{{< /tabs >}} + +## Reporter + +For information on how to set up Flink's trace reporters please take a look at the [trace reporters documentation]({{< ref "docs/deployment/trace_reporters" >}}). + +## System traces + +Flink reports traces listed below. + +The tables below generally feature 5 columns: Review Comment: typo: feature**s** ? ########## docs/content/docs/ops/traces.md: ########## @@ -0,0 +1,126 @@ +--- +title: "Metrics" +weight: 6 +type: docs +aliases: + - /ops/traces.html + - /apis/traces.html + - /monitoring/traces.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Traces + +Flink exposes a tracing system that allows gathering and exposing traces to external systems. + +## Reporting traces + +You can access the tracing system from any user function that extends [RichFunction]({{< ref "docs/dev/datastream/user_defined_functions" >}}#rich-functions) by calling `getRuntimeContext().getMetricGroup()`. +This method returns a `MetricGroup` object via which you can report a new single span trace. + +### Reporting single Span + + +A `Span` represents something that happened in Flink at certain point of time, that will be reported to a `TraceReporter`. +To report a `Span` you can use the `MetricGroup#addSpan(SpanBuilder)` method. + +Currently we don't support traces with multiple spans. Each `Span` is self-contained and represents things like a checkpoint or recovery. +{{< tabs "9612d275-bdda-4322-a01f-ae6da805e917" >}} +{{< tab "Java" >}} +```java +public class MyClass { + void doSomething() { + // (...) + metricGroup.addSpan( + Span.builder(MyClass.class, "SomeAction") + .setStartTsMillis(startTs) // Optional + .setEndTsMillis(endTs) // Optional + .setAttribute("foo", "bar"); + } +} +``` +{{< /tab >}} +{{< tab "Python" >}} +```python +Currently reporting Spans from Python is not supported. +``` +{{< /tab >}} +{{< /tabs >}} + +## Reporter + +For information on how to set up Flink's trace reporters please take a look at the [trace reporters documentation]({{< ref "docs/deployment/trace_reporters" >}}). + +## System traces + +Flink reports traces listed below. + +The tables below generally feature 5 columns: + +* The "Scope" column describes what is that trace reported scope. + +* The "Name" column describes the name of the reported trace. + +* The "Attributes" column lists the names of all attributes that are reported with the given trace. + +* The "Description" column provides information as to what a given attribute is reporting. + +### Checkpointing and recovery + Review Comment: Should we clarify that the trace is reported only once the checkpoint is completed/failed? ########## docs/layouts/shortcodes/generated/trace_configuration.html: ########## @@ -0,0 +1,36 @@ +<table class="configuration table table-bordered"> Review Comment: I couldn't find this section. Shouldn't it be added explicitly to `config.md`? ########## docs/content/docs/ops/traces.md: ########## @@ -0,0 +1,126 @@ +--- +title: "Metrics" +weight: 6 +type: docs +aliases: + - /ops/traces.html + - /apis/traces.html + - /monitoring/traces.html +--- +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Traces + +Flink exposes a tracing system that allows gathering and exposing traces to external systems. + +## Reporting traces + +You can access the tracing system from any user function that extends [RichFunction]({{< ref "docs/dev/datastream/user_defined_functions" >}}#rich-functions) by calling `getRuntimeContext().getMetricGroup()`. +This method returns a `MetricGroup` object via which you can report a new single span trace. + +### Reporting single Span + + +A `Span` represents something that happened in Flink at certain point of time, that will be reported to a `TraceReporter`. +To report a `Span` you can use the `MetricGroup#addSpan(SpanBuilder)` method. + +Currently we don't support traces with multiple spans. Each `Span` is self-contained and represents things like a checkpoint or recovery. +{{< tabs "9612d275-bdda-4322-a01f-ae6da805e917" >}} +{{< tab "Java" >}} +```java +public class MyClass { + void doSomething() { + // (...) + metricGroup.addSpan( + Span.builder(MyClass.class, "SomeAction") + .setStartTsMillis(startTs) // Optional + .setEndTsMillis(endTs) // Optional + .setAttribute("foo", "bar"); + } +} +``` +{{< /tab >}} +{{< tab "Python" >}} +```python +Currently reporting Spans from Python is not supported. +``` +{{< /tab >}} +{{< /tabs >}} + +## Reporter + +For information on how to set up Flink's trace reporters please take a look at the [trace reporters documentation]({{< ref "docs/deployment/trace_reporters" >}}). + +## System traces + +Flink reports traces listed below. + +The tables below generally feature 5 columns: + +* The "Scope" column describes what is that trace reported scope. + +* The "Name" column describes the name of the reported trace. + +* The "Attributes" column lists the names of all attributes that are reported with the given trace. + +* The "Description" column provides information as to what a given attribute is reporting. + +### Checkpointing and recovery + +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 18%">Scope</th> + <th class="text-left" style="width: 22%">Name</th> + <th class="text-left" style="width: 20%">Attributes</th> + <th class="text-left" style="width: 32%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <th rowspan="6">org.apache.flink.runtime.checkpoint.CheckpointStatsTracker</th> + <th rowspan="6"><strong>Checkpoint</strong></th> + <td>startTs</td> + <td>Timestamp when the checkpoint has started.</td> + </tr> + <tr> + <td>endTs</td> + <td>Timestamp when the checkpoint has finished.</td> + </tr> + <tr> + <td>checkpointId</td> + <td>Id of the checkpoint.</td> + </tr> + <tr> + <td>checkpointSize</td> Review Comment: Isn't it checkpoint**ed**Size? ########## docs/content/docs/ops/traces.md: ########## @@ -0,0 +1,126 @@ +--- +title: "Metrics" Review Comment: Metrics -> Traces (I see two identical pages currently in navigation) ########## docs/layouts/shortcodes/generated/akka_configuration.html: ########## @@ -98,6 +80,24 @@ <td>Duration</td> <td>Timeout used for the lookup of the JobManager. The timeout value has to contain a time-unit specifier (ms/s/min/h/d).</td> </tr> + <tr> + <td><h5>pekko.remote-fork-join-executor.parallelism-factor</h5></td> + <td style="word-wrap: break-word;">2.0</td> + <td>Double</td> + <td>The parallelism factor is used to determine thread pool size using the following formula: ceil(available processors * factor). Resulting size is then bounded by the parallelism-min and parallelism-max values.</td> + </tr> + <tr> + <td><h5>pekko.remote-fork-join-executor.parallelism-max</h5></td> + <td style="word-wrap: break-word;">16</td> + <td>Integer</td> + <td>Max number of threads to cap factor-based parallelism number to.</td> + </tr> + <tr> + <td><h5>pekko.remote-fork-join-executor.parallelism-min</h5></td> + <td style="word-wrap: break-word;">8</td> + <td>Integer</td> + <td>Min number of threads to cap factor-based parallelism number to.</td> + </tr> Review Comment: Unnecessary change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
