This is an automated email from the ASF dual-hosted git repository. wusheng pushed a commit to branch fix/docs-oal-mal-lal-improvements in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit a382f89c0fade201afeea69b0d43547e856874dc Author: Wu Sheng <[email protected]> AuthorDate: Mon Jan 19 10:27:00 2026 +0800 Polish OAL, MAL, and LAL documentation - OAL: Add cpm function docs, improve filter operators formatting, clarify downsampling/TimeBucket behavior - MAL: Fix typos (valueNotEqual, tagNotMatch, combinates, fourse), fix wrong region in example, add missing Layer param to endpoint - LAL: Fix endpoint description, fix extractor typo, clarify rpm rate limit comments, remove grok TODO - CLAUDE.md: Add tips for markdown rendering and relative paths --- CLAUDE.md | 2 ++ docs/en/concepts-and-designs/lal.md | 17 ++++++----------- docs/en/concepts-and-designs/mal.md | 12 +++++++----- docs/en/concepts-and-designs/oal.md | 33 +++++++++++++++++++++++---------- 4 files changed, 38 insertions(+), 26 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 7f48ff5219..19704b8c23 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -321,3 +321,5 @@ Follow the PR template in `.github/PULL_REQUEST_TEMPLATE`. Key requirements: 6. **OAL generates code**: Don't manually edit generated metrics classes 7. **Use Lombok**: Prefer annotations over boilerplate code 8. **Test both unit and integration**: Different test patterns for different scopes +9. **Documentation is rendered via markdown**: When reviewing docs, consider how they will be rendered by a markdown engine +10. **Relative paths in docs are valid**: Relative file paths (e.g., `../../../oap-server/...`) in documentation work both in the repo and on the documentation website, supported by website build tooling diff --git a/docs/en/concepts-and-designs/lal.md b/docs/en/concepts-and-designs/lal.md index 4fc330be24..f843871cf8 100644 --- a/docs/en/concepts-and-designs/lal.md +++ b/docs/en/concepts-and-designs/lal.md @@ -140,11 +140,6 @@ filter { } ``` -- `grok` (TODO) - -We're aware of certain performance issues in the grok Java library, and so we're currently conducting investigations and benchmarking. Contributions are -welcome. - ### Extractor Extractors aim to extract metadata from the logs. The metadata can be a service name, a service instance name, an @@ -162,7 +157,7 @@ persisted (if not dropped) and is used to associate with traces / metrics. - `endpoint` -`endpoint` extracts the service instance name from the `parsed` result, and set it into the `LogData`, which will be +`endpoint` extracts the endpoint name from the `parsed` result, and set it into the `LogData`, which will be persisted (if not dropped) and is used to associate with traces / metrics. - `traceId` @@ -341,7 +336,7 @@ persisted (if not dropped) and is used to associate with TopNDatabaseStatement. `id` extracts the id from the `parsed` result, and set it into the `DatabaseSlowStatement`, which will be persisted (if not dropped) and is used to associate with TopNDatabaseStatement. -A Example of LAL to distinguish slow logs: +An example of LAL to distinguish slow logs: ```groovy filter { @@ -460,11 +455,11 @@ filter { sampler { if (parsed.service == "ImportantApp") { rateLimit("ImportantAppSampler") { - rpm 1800 // samples 1800 pieces of logs every minute for service "ImportantApp" + rpm 1800 // samples at most 1800 logs per minute for service "ImportantApp" } } else { rateLimit("OtherSampler") { - rpm 180 // samples 180 pieces of logs every minute for other services than "ImportantApp" + rpm 180 // samples at most 180 logs per minute for other services than "ImportantApp" } } } @@ -526,8 +521,8 @@ filter { // filter A: this is for persistence } } filter { // filter B: - // ... extractors to generate many metrics - extractors { + // ... extractor to generate many metrics + extractor { metrics { // ... metrics } diff --git a/docs/en/concepts-and-designs/mal.md b/docs/en/concepts-and-designs/mal.md index aa391aeac6..32f4b23199 100644 --- a/docs/en/concepts-and-designs/mal.md +++ b/docs/en/concepts-and-designs/mal.md @@ -39,19 +39,20 @@ MAL supports four type operations to filter samples in a sample family by tag: - tagEqual: Filter tags exactly equal to the string provided. - tagNotEqual: Filter tags not equal to the string provided. - tagMatch: Filter tags that regex-match the string provided. - - tagNotMatch: Filter labels that do not regex-match the string provided. + - tagNotMatch: Filter tags that do not regex-match the string provided. For example, this filters all instance_trace_count samples for us-west and asia-north region and az-1 az: ``` instance_trace_count.tagMatch("region", "us-west|asia-north").tagEqual("az", "az-1") ``` + ### Value filter MAL supports six type operations to filter samples in a sample family by value: - valueEqual: Filter values exactly equal to the value provided. -- valueNotEqual: Filter values equal to the value provided. +- valueNotEqual: Filter values not equal to the value provided. - valueGreater: Filter values greater than the value provided. - valueGreaterEqual: Filter values greater than or equal to the value provided. - valueLess: Filter values less than the value provided. @@ -62,6 +63,7 @@ For example, this filters all instance_trace_count samples for values >= 33: ``` instance_trace_count.valueGreaterEqual(33) ``` + ### Tag manipulator MAL allows tag manipulators to change (i.e. add/delete/update) tags and their values. @@ -142,7 +144,7 @@ Example expression: instance_trace_analysis_error_count / instance_trace_count ``` -This returns a resulting sample family containing the error rate of trace analysis. Samples with region us-west and az az-3 +This returns a resulting sample family containing the error rate of trace analysis. Samples with region us-east and az az-3 have no match and will not show up in the result: ``` @@ -264,7 +266,7 @@ They extract level relevant labels from metric labels, then informs the meter-sy - `service([svc_label1, svc_label2...], Layer)` extracts service level labels from the array argument, extracts layer from `Layer` argument. - `instance([svc_label1, svc_label2...], [ins_label1, ins_label2...], Layer, Closure<Map<String, String>> propertiesExtractor)` extracts service level labels from the first array argument, extracts instance level labels from the second array argument, extracts layer from `Layer` argument, `propertiesExtractor` is an optional closure that extracts instance properties from `tags`, e.g. `{ tags -> ['pod': tags.pod, 'namespace': tags.namespace] }`. - - `endpoint([svc_label1, svc_label2...], [ep_label1, ep_label2...])` extracts service level labels from the first array argument, + - `endpoint([svc_label1, svc_label2...], [ep_label1, ep_label2...], Layer)` extracts service level labels from the first array argument, extracts endpoint level labels from the second array argument, extracts layer from `Layer` argument. - `process([svc_label1, svc_label2...], [ins_label1, ins_label2...], [ps_label1, ps_label2...], layer_lable)` extracts service level labels from the first array argument, extracts instance level labels from the second array argument, extracts process level labels from the third array argument, extracts layer label from fourse argument. @@ -310,7 +312,7 @@ metricsRules: ### <metric_rules> ```yaml -# The name of rule, which combinates with a prefix 'meter_' as the index/table name in storage. +# The name of rule, which combines with a prefix 'meter_' as the index/table name in storage. name: <string> # MAL expression. exp: <string> diff --git a/docs/en/concepts-and-designs/oal.md b/docs/en/concepts-and-designs/oal.md index 13ea3f8d4c..038f14dc3f 100644 --- a/docs/en/concepts-and-designs/oal.md +++ b/docs/en/concepts-and-designs/oal.md @@ -23,7 +23,7 @@ However, the OAL script is a compiled language, and the OAL Runtime generates ja the changes of those scripts in the runtime. If your OAP servers are running in a cluster mode, these script defined metrics should be aligned. -You can open set `SW_OAL_ENGINE_DEBUG=Y` at system env to see which classes are generated. +You can set `SW_OAL_ENGINE_DEBUG=Y` at system env to see which classes are generated. ## Grammar Scripts should be named `*.oal` @@ -47,10 +47,16 @@ See [Scope Definitions](scope-definitions.md), where you can find all existing S ## Filter -Use filter to build conditions for the value of fields by using field name and expression. +Use filter to build conditions for the value of fields by using field name and expression. -The filter expressions run as a chain, generally connected with `logic AND`. -The OPs support `==`, `!=`, `>`, `<`, `>=`, `<=`, `in [...]` ,`like %...`, `like ...%` , `like %...%` , `contain` and `not contain`, with type detection based on field type. In the event of incompatibility, compile or code generation errors may be triggered. +The filter expressions run as a chain, generally connected with `logic AND`. +Type detection is based on field type. In the event of incompatibility, compile or code generation errors may be triggered. + +Supported operators: +- Comparison: `==`, `!=`, `>`, `<`, `>=`, `<=` +- Collection: `in [...]` (e.g., `name in ("Endpoint1", "Endpoint2")`) +- String matching: `like %...`, `like ...%`, `like %...%` (e.g., `name like "serv%"`) +- Tag matching: `contain`, `not contain` (e.g., `tags contain "http.method:GET"`) ## Aggregation Function The default functions are provided by the SkyWalking OAP core, and it is possible to implement additional functions. @@ -77,7 +83,12 @@ Parameter (2) is the `denominator` condition. - `count`. The sum of calls per scope entity. > service_calls_sum = from(Service.*).count(); -In this case, the number of calls of each service. +In this case, the number of calls of each service. + +- `cpm`. Calls Per Minute. The total number of calls divided by the duration in minutes. +> service_cpm = from(Service.*).cpm(); + +In this case, the CPM of each service. - `histogram`. See [Heatmap in WIKI](https://en.wikipedia.org/wiki/Heat_map). > service_heatmap = from(Service.latency).histogram(100, 20); @@ -116,10 +127,12 @@ In this case, the avg of the duration of each browser resource file, max support The metrics name for storage implementor, alarm and query modules. The type inference is supported by core. ## Group -All metrics data will be grouped by Scope.ID and min-level TimeBucket. +All metrics data will be grouped by Scope.ID and min-level TimeBucket. - In the `Endpoint` scope, the Scope.ID is same as the Endpoint ID (i.e. the unique ID based on service and its endpoint). +The `Minute` level aggregation is enforced by default. Additional time bucket aggregations (`Hour`, `Day`) are also enabled by default and can be configured via `core/downsampling` in `application.yml`. + ## Cast Fields of source are static type. In some cases, the type required by the filter expression and aggregation function doesn't match the type in the source, such as tag value in the source is String type, most aggregation calculation requires numeric. @@ -154,11 +167,11 @@ By default, none of them are disabled. ## Examples ``` -// Calculate p99 of both Endpoint1 and Endpoint2 -endpoint_p99 = from(Endpoint.latency).filter(name in ("Endpoint1", "Endpoint2")).summary(0.99) +// Calculate percentile(s) of both Endpoint1 and Endpoint2 +endpoint_p99 = from(Endpoint.latency).filter(name in ("Endpoint1", "Endpoint2")).percentile2(10) -// Calculate p99 of Endpoint name started with `serv` -serv_Endpoint_p99 = from(Endpoint.latency).filter(name like "serv%").summary(0.99) +// Calculate percentile(s) of Endpoint name started with `serv` +serv_Endpoint_p99 = from(Endpoint.latency).filter(name like "serv%").percentile2(10) // Calculate the avg response time of each Endpoint endpoint_resp_time = from(Endpoint.latency).avg()
