[jira] [Updated] (IGNITE-17092) Implement topology command
[ https://issues.apache.org/jira/browse/IGNITE-17092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr updated IGNITE-17092: --- Description: The topology commandd should call a special endpoint for getting the cluster topology. There are two kinds of topology: physical and logical. was:Topology should call a special endpoint for getting the cluster topology. Also there shold be a difference between logical and physical topology. > Implement topology command > -- > > Key: IGNITE-17092 > URL: https://issues.apache.org/jira/browse/IGNITE-17092 > Project: Ignite > Issue Type: Task >Reporter: Aleksandr >Priority: Major > Labels: ignite-3, ignite-3-cli-tool > > The topology commandd should call a special endpoint for getting the cluster > topology. > There are two kinds of topology: physical and logical. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16907) Add ability to use Raft log as storage WAL within the scope of local recovery
[
https://issues.apache.org/jira/browse/IGNITE-16907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-16907:
---
Epic Link: IGNITE-16923
> Add ability to use Raft log as storage WAL within the scope of local recovery
> -
>
> Key: IGNITE-16907
> URL: https://issues.apache.org/jira/browse/IGNITE-16907
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> h4. Problem
> From the birds eye view raft-to-storage flow looks similar to
> #
> {code:java}
> RaftGroupService#run(writeCommand());{code}
> # Inner raft replication logic, when replicated on majority adjust
> raft.commitedIndex.
> # Propagate command to RaftGroupListener (raft state machine).
> {code:java}
> RaftGroupListener#onWrite(closure(writeCommand()));{code}
> # Within state machine insert data from writeCommand to underneath storage:
> {code:java}
> var insertRes = storage.insert(cmd.getRow(), cmd.getTimestamp());{code}
> # ack that data was applied successfully
> {code:java}
> clo.result(insertRes);{code}
> # move raft.appliedIndex to corresponding value, meaning that the data for
> this index is applied to the state machine.
> The most interesting part, especially for given ticket, relates to step 4.
> In real world storage doesn't flush every mutator on disk, instead it buffers
> some amount of such mutators and flush them all-together as a part of some
> checkpointing process. Thus, if node fails before mutatorsBuffer.flush() it
> might lost some data because raft will apply data starting from appliedIndex
> + 1 on recovery.
> h4. Possible solutions:
> There are several possibilities to solve this issue:
> # In-storage WAL. Bad solution, because there's already raft log that can be
> used as a WAL. Such duplication is redundant.
> # Local recovery starting from appliedIndex - mutatorsBuffer.size. Bad
> solution. Won't work for not-idempotent operations. Exposes inner storage
> details such as mutatorBuffer.size.
> # proposedIndex propagation + checkpointIndex synchonization. Seems fine.
> More details below:
> * First off all, in order to coordinate raft replicator and storage,
> proposedIndex should be propagated to raftGroupListener and storage.
> * On every checkpoint, storage will persist corresponding proposed index as
> checkpointIndex.
> ** In case of storage inner checkpoints, storage won't notify raft
> replicator about new checkpointIndex. This kind of notification is an
> optimization that does not affect the correctness of the protocol.
> ** In case of outer checkpoint intention, e.g. raft snapshotting for the
> purposes of raft log truncation, corresponding checkpointIndex will be
> propagated to raft replicator within a callback "onShapshotDone".
> * During local recovery raft will apply raft log entries from the very
> begging. If checkpointIndex occurred to be bigger than proposedIndex on an
> another raft log entity it fails the proposed closure with
> IndexMismatchException(checkpointIndex) that leads to proposedIndex shift and
> optional async raft log truncation.
> Let's consider following example:
> ] checkpointBuffer = 3. [P] - perisisted entities, [!P] - not perisisted/in
> memory one.
> # raft.put(k1,v1)
> ## -> raftlog[cmd(k1,v1, index:1)]
> ## -> storage[(k1,v1), index:1]
> ## -> appliedIndex:1
> # raft.put(k2,v2)
> ## -> raftlog[cmd(k1,v1, index:1), \\{*}cmd(k2,v2, index:2)\\{*}]
> ## -> storage[(k1,v1), \\{*}(k2,v2)\\{*}, ** index:\\{*}2\\{*}]
> ## -> appliedIndex:{*}2{*}
> # raft.put(k3,v3)
> ## -> raftlog[cmd(k1,v1, index:1), cmd(k2,v2, index:2), \\{*}cmd(k3,v3,
> index:3)\\{*}]
> ## -> storage[(k1,v1), (k2,v2), \\{*}(k3,v3)\\{*}, index:\\{*}3\\{*}]
> ## -> appliedIndex:{*}3{*}
> ## *inner storage checkpoint*
> ### raftlog[cmd(k1,v1, index:1), cmd(k2,v2, index:2), cmd(k3,v3, index:3)]
> ### storage[(k1,v1, proposedIndex:1), (k2,v2, proposedIndex:2), (k3,v3,
> proposedIndex:3)]
> ### {*}checkpointedData[(k1,v1),* *(k2,v2),* \\{*}(k3,v3),
> checkpointIndex:3\\{*}{*}\\{*}{*}]{*}{*}{{*}}
> # raft.put(k4,v4)
> ## -> raftlog[cmd(k1,v1, index:1), cmd(k2,v2, index:2), cmd(k3,v3,
> index:3), \\{*}cmd(k4,v4, index:4)\\{*}]
> ## -> storage[(k1,v1), (k2,v2), (k3,v3), *(k4,v4)* index:\\{*}4\\{*}]
> ## -> checkpointedData[(k1,v1), (k2,v2), (k3,v3), checkpointIndex:3]
> ## -> appliedIndex:{*}4{*}
> # Node failure
> # Node restart
> ## StorageRecovery: storage.apply(checkpointedData)
> ## raft-to-storage data application starting from index: 1 // raft doesn't
> know checkpointedIndex at this point.
> ### ->
[jira] [Updated] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17
[
https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amelchev Nikita updated IGNITE-16650:
-
Fix Version/s: 2.14
> Exclude ignite-log4j, log4j 1.2.17
> --
>
> Key: IGNITE-16650
> URL: https://issues.apache.org/jira/browse/IGNITE-16650
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergei Ryzhov
>Assignee: Mikhail Petrov
>Priority: Major
> Labels: ise
> Fix For: 2.14
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> log4j 1.2.17 is not supported and contains critical vulnerabilities
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> I suggest excluding the ignite-log4j module from ignite
> Direct vulnerabilities:
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
> As a result of the mentioned migration, the following changes will be applied:
> 1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after
> the refactoring we will get two configuration ignite-log4j.xml and
> ignite-log4j2.xml both in log4j2 format because ignite-log4j2.xml is in use
> now and but provide log formatitng different from ignite-log4j.xml.
> 2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because
> it is used with compatibility tests.
> 3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j
> format. The current version of core/src/test/config/log4j2-test.xml is
> moved to the log4j2/src/test/config folder.
> 4. osgi-paxlogging will be removed because it's only meant to provide some
> log4j dependencies. We have no need in them now.
> 5. Exception logging format will change slightly:
> Before:
> {code:java}
> class org.apache.ignite.IgniteException: Platform error:System.Exception:
> EXCEPTION_TEST_Warn
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
> at
> org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
> {code}
> After:
> {code:java}
> org.apache.ignite.IgniteException: Platform error:System.Exception:
> EXCEPTION_TEST_Warn
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
> at
> org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
> {code}
> As you can see, only the first word "class" is omitted.
> 6. All other files containing log4j configuration will be refactored to suite
> log4j2 and will be renamed if previously their name allowed log4j to
> automatically find them in the class path (e.g. log4j.xml -> log4j2.xml and
> so on)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17022) Implement interceptors for Ignite Services
[
https://issues.apache.org/jira/browse/IGNITE-17022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-17022:
--
Ignite Flags: Release Notes Required (was: Docs Required,Release Notes
Required)
> Implement interceptors for Ignite Services
> --
>
> Key: IGNITE-17022
> URL: https://issues.apache.org/jira/browse/IGNITE-17022
> Project: Ignite
> Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Pavel Pereslegin
>Priority: Major
> Labels: iep-79, important, ise
> Fix For: 2.14
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currenlty, Ignite Services has a lack of support execution a custom
> middleware process before and after a service execution. This leads to a lot
> of boilerplate code.
> The IEP-79 describes a draft API of such a feature implementation:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191334119
> However, we still need to consider additional crucial points of such API:
> - sync or async listeners/interceptors execution (some of processes may
> require to set and clear threadlocals);
> - an exception in the inteceptrod should or should not interrupt a service
> execution;
> - {{context}} may need to be accessible inside interceptors and services;
> Also we need to add description about what kind of the {{guarantees}} are
> provided by Ignite server node during a service execution. It may be
> necessary to consider the service-per-thread execution logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (IGNITE-17022) Implement interceptors for Ignite Services
[
https://issues.apache.org/jira/browse/IGNITE-17022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin resolved IGNITE-17022.
---
Release Note: Added the ability to intercept calls to service methods.
Resolution: Done
> Implement interceptors for Ignite Services
> --
>
> Key: IGNITE-17022
> URL: https://issues.apache.org/jira/browse/IGNITE-17022
> Project: Ignite
> Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Pavel Pereslegin
>Priority: Major
> Labels: iep-79, important, ise
> Fix For: 2.14
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currenlty, Ignite Services has a lack of support execution a custom
> middleware process before and after a service execution. This leads to a lot
> of boilerplate code.
> The IEP-79 describes a draft API of such a feature implementation:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=191334119
> However, we still need to consider additional crucial points of such API:
> - sync or async listeners/interceptors execution (some of processes may
> require to set and clear threadlocals);
> - an exception in the inteceptrod should or should not interrupt a service
> execution;
> - {{context}} may need to be accessible inside interceptors and services;
> Also we need to add description about what kind of the {{guarantees}} are
> provided by Ignite server node during a service execution. It may be
> necessary to consider the service-per-thread execution logic.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17366) Document the service call context and interceptors.
[ https://issues.apache.org/jira/browse/IGNITE-17366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17569087#comment-17569087 ] Pavel Pereslegin commented on IGNITE-17366: --- [~NSAmelchev], [~Zabotkin], thanks for the review! > Document the service call context and interceptors. > --- > > Key: IGNITE-17366 > URL: https://issues.apache.org/jira/browse/IGNITE-17366 > Project: Ignite > Issue Type: Sub-task > Components: documentation >Reporter: Pavel Pereslegin >Assignee: Pavel Pereslegin >Priority: Major > Labels: iep-79 > Fix For: 2.14 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Add documentation and examples how to use service interceptors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17398) Opencensus metrics do not work well with Prometheus since it do not have tags
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Kadaner updated IGNITE-17398:
Summary: Opencensus metrics do not work well with Prometheus since it do
not have tags (was: Opencensus metrics do not work well with Prometheus)
> Opencensus metrics do not work well with Prometheus since it do not have tags
> -
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
> Attachments: image-2022-07-20-17-51-07-217.png
>
>
> The metrics created by the [new metrics
> system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
> are very inconvenient to use with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
> For example with tags it is very easy to get a graph in Grafana with a cache
> hit percentage that includes every existing cache. It automatically extracts
> the cache name from the "cache" tag. Unfortunately, it is not possible if
> there are no tags.
> !image-2022-07-20-17-51-07-217.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17398) Opencensus metrics do not work well with Prometheus
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Kadaner updated IGNITE-17398:
Description:
The metrics created by the [new metrics
system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
are very inconvenient to use with Prometheus since it does not use tags.
For example, Spring metric generated for the same cache looks like the
following in Prometheus and is very convenient to use:
{noformat}
cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
1387.0{noformat}
The native Ignite metric looks like the following:
{noformat}
cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
The Spring reported statistics can be easily filtered by cache name and other
attributes, while the build-in Ignite metrics do not provide an easy way to
access cache names. The only option seems to be parsing the
"cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by Grafana.
For example with tags it is very easy to get a graph in Grafana with a cache
hit percentage that includes every existing cache. It automatically extracts
the cache name from the "cache" tag. Unfortunately, it is not possible if there
are no tags.
!image-2022-07-20-17-51-07-217.png!
was:
The metrics created by the [new metrics
system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
are very inconvenient to use with Prometheus since it does not use tags.
For example, Spring metric generated for the same cache looks like the
following in Prometheus and is very convenient to use:
{noformat}
cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
1387.0{noformat}
The native Ignite metric looks like the following:
{noformat}
cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
The Spring reported statistics can be easily filtered by cache name and other
attributes, while the build-in Ignite metrics do not provide an easy way to
access cache names. The only option seems to be parsing the
"cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by Grafana.
> Opencensus metrics do not work well with Prometheus
> ---
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
> Attachments: image-2022-07-20-17-51-07-217.png
>
>
> The metrics created by the [new metrics
> system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
> are very inconvenient to use with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
> For example with tags it is very easy to get a graph in Grafana with a cache
> hit percentage that includes every existing cache. It automatically extracts
> the cache name from the "cache" tag. Unfortunately, it is not possible if
> there are no tags.
> !image-2022-07-20-17-51-07-217.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17398) Opencensus metrics do not work well with Prometheus
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Kadaner updated IGNITE-17398:
Attachment: image-2022-07-20-17-51-07-217.png
> Opencensus metrics do not work well with Prometheus
> ---
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
> Attachments: image-2022-07-20-17-51-07-217.png
>
>
> The metrics created by the [new metrics
> system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
> are very inconvenient to use with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17398) Opencensus metrics do not work well with Prometheus
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Kadaner updated IGNITE-17398:
Description:
The metrics created by the [new metrics
system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
are very inconvenient to use with Prometheus since it does not use tags.
For example, Spring metric generated for the same cache looks like the
following in Prometheus and is very convenient to use:
{noformat}
cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
1387.0{noformat}
The native Ignite metric looks like the following:
{noformat}
cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
The Spring reported statistics can be easily filtered by cache name and other
attributes, while the build-in Ignite metrics do not provide an easy way to
access cache names. The only option seems to be parsing the
"cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by Grafana.
was:
The metrics created by the new metrics system are very inconvenient to use with
Prometheus since it does not use tags.
For example, Spring metric generated for the same cache looks like the
following in Prometheus and is very convenient to use:
{noformat}
cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
1387.0{noformat}
The native Ignite metric looks like the following:
{noformat}
cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
The Spring reported statistics can be easily filtered by cache name and other
attributes, while the build-in Ignite metrics do not provide an easy way to
access cache names. The only option seems to be parsing the
"cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by Grafana.
> Opencensus metrics do not work well with Prometheus
> ---
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
>
> The metrics created by the [new metrics
> system|https://ignite.apache.org/docs/latest/monitoring-metrics/new-metrics]
> are very inconvenient to use with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17398) Opencensus metrics do not work well with Prometheus
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Kadaner updated IGNITE-17398:
Labels: metrics (was: )
> Opencensus metrics do not work well with Prometheus
> ---
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
> Labels: metrics
>
> The metrics created by the new metrics system are very inconvenient to use
> with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17398) Opencensus metrics do not work well with Prometheus
[
https://issues.apache.org/jira/browse/IGNITE-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Kadaner updated IGNITE-17398:
Affects Version/s: 2.9.1
> Opencensus metrics do not work well with Prometheus
> ---
>
> Key: IGNITE-17398
> URL: https://issues.apache.org/jira/browse/IGNITE-17398
> Project: Ignite
> Issue Type: Improvement
>Affects Versions: 2.9.1
>Reporter: Sergey Kadaner
>Priority: Major
>
> The metrics created by the new metrics system are very inconvenient to use
> with Prometheus since it does not use tags.
> For example, Spring metric generated for the same cache looks like the
> following in Prometheus and is very convenient to use:
>
> {noformat}
> cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
> 1387.0{noformat}
>
> The native Ignite metric looks like the following:
>
> {noformat}
> cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
> The Spring reported statistics can be easily filtered by cache name and other
> attributes, while the build-in Ignite metrics do not provide an easy way to
> access cache names. The only option seems to be parsing the
> "cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by
> Grafana.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17398) Opencensus metrics do not work well with Prometheus
Sergey Kadaner created IGNITE-17398:
---
Summary: Opencensus metrics do not work well with Prometheus
Key: IGNITE-17398
URL: https://issues.apache.org/jira/browse/IGNITE-17398
Project: Ignite
Issue Type: Improvement
Reporter: Sergey Kadaner
The metrics created by the new metrics system are very inconvenient to use with
Prometheus since it does not use tags.
For example, Spring metric generated for the same cache looks like the
following in Prometheus and is very convenient to use:
{noformat}
cache_gets_total{cache="MY_CACHE_NAME",name="MY_CACHE_NAME",result="hit",}
1387.0{noformat}
The native Ignite metric looks like the following:
{noformat}
cache_MY_CACHE_NAME_CacheHits 1387.0{noformat}
The Spring reported statistics can be easily filtered by cache name and other
attributes, while the build-in Ignite metrics do not provide an easy way to
access cache names. The only option seems to be parsing the
"cache_MY_CACHE_NAME_CacheHits" strings which AFAIK is not supported by Grafana.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-16650) Exclude ignite-log4j, log4j 1.2.17
[
https://issues.apache.org/jira/browse/IGNITE-16650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17569058#comment-17569058
]
Ignite TC Bot commented on IGNITE-16650:
{panel:title=Branch: [pull/10152/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10152/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6688631buildTypeId=IgniteTests24Java8_RunAll]
> Exclude ignite-log4j, log4j 1.2.17
> --
>
> Key: IGNITE-16650
> URL: https://issues.apache.org/jira/browse/IGNITE-16650
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergei Ryzhov
>Assignee: Mikhail Petrov
>Priority: Major
> Labels: ise
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> log4j 1.2.17 is not supported and contains critical vulnerabilities
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> I suggest excluding the ignite-log4j module from ignite
> Direct vulnerabilities:
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23305
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23302
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-4104
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-17571
> As a result of the mentioned migration, the following changes will be applied:
> 1. ignite-log4j.xml will be migrated to log4j2 format. Unfortunately after
> the refactoring we will get two configuration ignite-log4j.xml and
> ignite-log4j2.xml both in log4j2 format because ignite-log4j2.xml is in use
> now and but provide log formatitng different from ignite-log4j.xml.
> 2. core/src/test/config/log4j-test.xml will not be migrated to log4j2 because
> it is used with compatibility tests.
> 3. core/src/test/config/log4j2-test.xml is refactored to suite current log4j
> format. The current version of core/src/test/config/log4j2-test.xml is
> moved to the log4j2/src/test/config folder.
> 4. osgi-paxlogging will be removed because it's only meant to provide some
> log4j dependencies. We have no need in them now.
> 5. Exception logging format will change slightly:
> Before:
> {code:java}
> class org.apache.ignite.IgniteException: Platform error:System.Exception:
> EXCEPTION_TEST_Warn
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
> at
> org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
> {code}
> After:
> {code:java}
> org.apache.ignite.IgniteException: Platform error:System.Exception:
> EXCEPTION_TEST_Warn
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:449)
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:511)
> at
> org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:575)
> at
> org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
> {code}
> As you can see, only the first word "class" is omitted.
> 6. All other files containing log4j configuration will be refactored to suite
> log4j2 and will be renamed if previously their name allowed log4j to
> automatically find them in the class path (e.g. log4j.xml -> log4j2.xml and
> so on)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-15837) Read Repair should support binary keys
[
https://issues.apache.org/jira/browse/IGNITE-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17569023#comment-17569023
]
Ignite TC Bot commented on IGNITE-15837:
{panel:title=Branch: [pull/10165/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10165/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6688909buildTypeId=IgniteTests24Java8_RunAll]
> Read Repair should support binary keys
> --
>
> Key: IGNITE-15837
> URL: https://issues.apache.org/jira/browse/IGNITE-15837
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Labels: iep-31
> Fix For: 2.14
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17366) Document the service call context and interceptors.
[
https://issues.apache.org/jira/browse/IGNITE-17366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17568998#comment-17568998
]
Ignite TC Bot commented on IGNITE-17366:
{panel:title=Branch: [pull/10162/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10162/head] Base: [master] : New Tests
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Examples{color} [[tests
2|https://ci2.ignite.apache.org/viewLog.html?buildId=6526128]]
* {color:#013220}IgniteExamplesSelfTestSuite:
IgniteServiceExamplesSelfTest.testServicesExample - PASSED{color}
* {color:#013220}IgniteExamplesSelfTestSuite:
IgniteServiceExamplesSelfTest.testServicesMiddlewareExample - PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=6526192buildTypeId=IgniteTests24Java8_RunAll]
> Document the service call context and interceptors.
> ---
>
> Key: IGNITE-17366
> URL: https://issues.apache.org/jira/browse/IGNITE-17366
> Project: Ignite
> Issue Type: Sub-task
> Components: documentation
>Reporter: Pavel Pereslegin
>Assignee: Pavel Pereslegin
>Priority: Major
> Labels: iep-79
> Fix For: 2.14
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Add documentation and examples how to use service interceptors.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-15837) Read Repair should support binary keys
[ https://issues.apache.org/jira/browse/IGNITE-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-15837: -- Fix Version/s: 2.14 > Read Repair should support binary keys > -- > > Key: IGNITE-15837 > URL: https://issues.apache.org/jira/browse/IGNITE-15837 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Labels: iep-31 > Fix For: 2.14 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17397) Investigate the ability to read a empty page in FilePageStore
Kirill Tkalenko created IGNITE-17397: Summary: Investigate the ability to read a empty page in FilePageStore Key: IGNITE-17397 URL: https://issues.apache.org/jira/browse/IGNITE-17397 Project: Ignite Issue Type: Task Reporter: Kirill Tkalenko Fix For: 3.0.0-alpha6 I found that in *FilePageStore#read(long, java.nio.ByteBuffer, boolean)* there may be a situation that we allocated the page without writing it to disk, tried to read it and instead of an error, we simply write an empty page to the buffer. It is necessary to understand better the reason for this behavior and either comment on this moment or redo it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17396) NPE during SQL query preparation when distributedJoins enabled
Ilya Korol created IGNITE-17396:
---
Summary: NPE during SQL query preparation when distributedJoins
enabled
Key: IGNITE-17396
URL: https://issues.apache.org/jira/browse/IGNITE-17396
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 2.13
Reporter: Ilya Korol
Strange NPE when distributedJoins enabled:
{code:java}
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteException;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.query.SqlFieldsQuery;
import org.apache.ignite.configuration.IgniteConfiguration;
import java.util.List;
public class DistributedJoinsNPEReproducer {
public static void main(String[] args) throws IgniteException {
try (Ignite ignite = Ignition.start(new IgniteConfiguration())) {
ignite.getOrCreateCache("tmp").query(new SqlFieldsQuery("CREATE
TABLE IF NOT EXISTS FOO (" +
" HOUR DECIMAL," +
" MINUTE DECIMAL," +
" DATE VARCHAR(8)," +
" PART_DATE VARCHAR(8)," +
" PART_KEY VARCHAR(2)," +
" FILLER INTEGER," +
" CONSTRAINT Foo_PK PRIMARY KEY (" +
"HOUR," +
"MINUTE," +
"DATE," +
"PART_DATE," +
"PART_KEY" +
" )" +
") WITH \"" +
" CACHE_NAME=foo," +
" KEY_TYPE=FooKey," +
" VALUE_TYPE=FooValue" +
"\"")).getAll();
ignite.getOrCreateCache("tmp").query(new SqlFieldsQuery("INSERT
INTO FOO (HOUR, MINUTE, DATE, PART_KEY, PART_DATE, FILLER) VALUES (1, 1,
'20220501', '01', '20220501', 1)")).getAll();
List> result = ignite.getOrCreateCache("tmp").query(new
SqlFieldsQuery(QUERY).setDistributedJoins(true)).getAll();
System.out.println(result.get(0));
}
}
static final String QUERY = "WITH param AS (" +
" SELECT CAST('20220501' AS varchar) AS pd" +
")" +
"SELECT B.pd " +
"FROM FOO, param B " +
"WHERE part_date=B.pd " +
" AND date=B.pd " +
" AND hour >=1 " +
"LIMIT 10";
}
{code}
Error:
{code:java}
Exception in thread "main" javax.cache.CacheException: Failed to parse query.
Внутренняя ошибка: "java.lang.NullPointerException"
General error: "java.lang.NullPointerException" [5-197]
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:859)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:792)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:432)
at
org.solveme.playground.ignite.examples.DistributedJoinsNPEReproducer.main(DistributedJoinsNPEReproducer.java:37)
Caused by: class
org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to parse
query. Внутренняя ошибка: "java.lang.NullPointerException"
General error: "java.lang.NullPointerException" [5-197]
at
org.apache.ignite.internal.processors.query.h2.H2Connection.prepareStatementNoCache(H2Connection.java:194)
at
org.apache.ignite.internal.processors.query.h2.H2Connection.prepareStatement(H2Connection.java:141)
at
org.apache.ignite.internal.processors.query.h2.H2PooledConnection.prepareStatement(H2PooledConnection.java:86)
at
org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.prepare(GridSqlQuerySplitter.java:1803)
at
org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.split0(GridSqlQuerySplitter.java:303)
at
org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.split(GridSqlQuerySplitter.java:222)
at
org.apache.ignite.internal.processors.query.h2.QueryParser.parseH2(QueryParser.java:551)
at
org.apache.ignite.internal.processors.query.h2.QueryParser.parse0(QueryParser.java:228)
at
org.apache.ignite.internal.processors.query.h2.QueryParser.parse(QueryParser.java:141)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1142)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$2.applyx(GridQueryProcessor.java:3016)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor$2.applyx(GridQueryProcessor.java:2989)
at
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:3661)
at
[jira] [Updated] (IGNITE-17393) Make JRaftServiceFactory properly configurable
[ https://issues.apache.org/jira/browse/IGNITE-17393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Bessonov updated IGNITE-17393: --- Reviewer: Semyon Danilov > Make JRaftServiceFactory properly configurable > -- > > Key: IGNITE-17393 > URL: https://issues.apache.org/jira/browse/IGNITE-17393 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Ivan Bessonov >Priority: Major > Labels: ignite-3 > Time Spent: 10m > Remaining Estimate: 0h > > Currently, ported JRaft code is polluted with unnecessary changes. > On top of that, the way that we configure raft service is far from optimal. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16337) Update lucene version up to 8.11.2 or above
[ https://issues.apache.org/jira/browse/IGNITE-16337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-16337: -- Fix Version/s: 2.14 > Update lucene version up to 8.11.2 or above > --- > > Key: IGNITE-16337 > URL: https://issues.apache.org/jira/browse/IGNITE-16337 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.11 >Reporter: Ilya Korol >Assignee: Ilya Korol >Priority: Major > Fix For: 2.14 > > Time Spent: 1h > Remaining Estimate: 0h > > Our Lucene version is quite outdated right now (7.4), while there are 2 major > releases available: 8.x.x with latest 8.11.1 and 9.0.0. So it would be good > to update this library because there were dozens of bugfixes, improvements > and optimizations since 7.4.x. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17312) Thin 3.0: Unified exception handling
[ https://issues.apache.org/jira/browse/IGNITE-17312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17568900#comment-17568900 ] Igor Sapego commented on IGNITE-17312: -- [~ptupitsyn] please take a look on my comments in PR > Thin 3.0: Unified exception handling > > > Key: IGNITE-17312 > URL: https://issues.apache.org/jira/browse/IGNITE-17312 > Project: Ignite > Issue Type: Improvement > Components: thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 10m > Remaining Estimate: 0h > > Java thin client uses the same public API interfaces as the server side. > Those user-facing APIs should behave in the same way on the server and client > sides, including exceptions that may be thrown. > # Remove *IgniteClientException*. > # Keep *IgniteClientConnectionException* and other client-specific types. > Inherit them from *IgniteException*. > # When sending exception details from server to client, include the following: > ## Code (group + error, see IgniteException.code) > ## TraceId > ## Message > ## Fully-qualified class name > ## OPTIONALLY: stack trace (Sensitive info. Send only when enabled in > server-side ClientConnectorConfig. Disabled by default.) -- This message was sent by Atlassian Jira (v8.20.10#820010)
