[jira] [Resolved] (IGNITE-17453) Calcite Engine: ORDER BY Optimization
[
https://issues.apache.org/jira/browse/IGNITE-17453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
YuJue Li resolved IGNITE-17453.
---
Resolution: Fixed
> Calcite Engine: ORDER BY Optimization
> -
>
> Key: IGNITE-17453
> URL: https://issues.apache.org/jira/browse/IGNITE-17453
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: YuJue Li
>Priority: Critical
> Fix For: 2.14
>
>
> 1.start a node;
> 2.create table:
> CREATE TABLE PI_COM_DAY
> (COM_ID VARCHAR(30) NOT NULL ,
> ITEM_ID VARCHAR(30) NOT NULL ,
> DATE1 VARCHAR(8) NOT NULL ,
> KIND VARCHAR(1),
> QTY_IOD DECIMAL(18, 6) ,
> AMT_IOD DECIMAL(18, 6) ,
> PRIMARYKEY (COM_ID,ITEM_ID,DATE1))
> WITH"template=partitioned,CACHE_NAME=PI_COM_DAY";
> CREATE INDEX IDX_PI_COM_DAY_ITEM_DATE ON PI_COM_DAY(ITEM_ID);
>
> 3.insert some data, for example, 1m;
> 4.execute sql:
> select * from pi_com_day order by item_id desc limit 10;
> normal.
> select /*+ QUERY_ENGINE('calcite') */ * from pi_com_day order by item_id desc
> limit 10;
> then OOM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17447) Support transform for IndexQuery
[ https://issues.apache.org/jira/browse/IGNITE-17447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Timonin reassigned IGNITE-17447: --- Assignee: Maksim Timonin > Support transform for IndexQuery > > > Key: IGNITE-17447 > URL: https://issues.apache.org/jira/browse/IGNITE-17447 > Project: Ignite > Issue Type: New Feature >Reporter: Maksim Timonin >Assignee: Maksim Timonin >Priority: Major > > Currently IndexQuery doesn't support transformation with > `cache#query(Query, IgniteClosure)`. > > But as IndexQuery re-use some infrastructure of ScanQuery, it's possible to > implement this for IndexQuery too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17455) IndexQuery should support setPartition
Maksim Timonin created IGNITE-17455: --- Summary: IndexQuery should support setPartition Key: IGNITE-17455 URL: https://issues.apache.org/jira/browse/IGNITE-17455 Project: Ignite Issue Type: New Feature Reporter: Maksim Timonin Assignee: Maksim Timonin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17082) Validate hocon output in REST endpoints
[
https://issues.apache.org/jira/browse/IGNITE-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573794#comment-17573794
]
Ivan Bessonov commented on IGNITE-17082:
Looks good to me!
> Validate hocon output in REST endpoints
> ---
>
> Key: IGNITE-17082
> URL: https://issues.apache.org/jira/browse/IGNITE-17082
> Project: Ignite
> Issue Type: Task
>Reporter: Aleksandr
>Assignee: Aleksandr
>Priority: Major
> Labels: ignite-3, rest
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Now all endpoints return plain/text. That looks strange in case the endpoint
> returns a hocon string. It's not just _any_ plain text. Here
> [https://www.w3schools.io/file/hocon-introduction/] they suggest
> {{{}application/hocon{}}}; it does not seem to be registered with IANA, but
> it looks logical.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-14890) Unify a log string with placeholders for various log-library implementation
[
https://issues.apache.org/jira/browse/IGNITE-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky updated IGNITE-14890:
Ignite Flags: (was: Docs Required,Release Notes Required)
> Unify a log string with placeholders for various log-library implementation
> ---
>
> Key: IGNITE-14890
> URL: https://issues.apache.org/jira/browse/IGNITE-14890
> Project: Ignite
> Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha3
>
>
> After the fix, we would use Ignite own specific logger pattern format.
> This format is based by widely know SLF4J:
> 1. We would pass several parameters after a pattern.
> 2. The last one parameter might be exception.
> 3. `{}` is used as a formatting anchor.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17453) Calcite Engine: ORDER BY Optimization
[
https://issues.apache.org/jira/browse/IGNITE-17453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573788#comment-17573788
]
Aleksey Plekhanov commented on IGNITE-17453:
[~liyuj], can you please check this issue on the master branch? I hope it was
fixed by IGNITE-16013.
> Calcite Engine: ORDER BY Optimization
> -
>
> Key: IGNITE-17453
> URL: https://issues.apache.org/jira/browse/IGNITE-17453
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: YuJue Li
>Priority: Critical
> Fix For: 2.14
>
>
> 1.start a node;
> 2.create table:
> CREATE TABLE PI_COM_DAY
> (COM_ID VARCHAR(30) NOT NULL ,
> ITEM_ID VARCHAR(30) NOT NULL ,
> DATE1 VARCHAR(8) NOT NULL ,
> KIND VARCHAR(1),
> QTY_IOD DECIMAL(18, 6) ,
> AMT_IOD DECIMAL(18, 6) ,
> PRIMARYKEY (COM_ID,ITEM_ID,DATE1))
> WITH"template=partitioned,CACHE_NAME=PI_COM_DAY";
> CREATE INDEX IDX_PI_COM_DAY_ITEM_DATE ON PI_COM_DAY(ITEM_ID);
>
> 3.insert some data, for example, 1m;
> 4.execute sql:
> select * from pi_com_day order by item_id desc limit 10;
> normal.
> select /*+ QUERY_ENGINE('calcite') */ * from pi_com_day order by item_id desc
> limit 10;
> then OOM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17454) Logger need to highlight somehow about wrong formatting template.
Evgeny Stanilovsky created IGNITE-17454:
---
Summary: Logger need to highlight somehow about wrong formatting
template.
Key: IGNITE-17454
URL: https://issues.apache.org/jira/browse/IGNITE-17454
Project: Ignite
Issue Type: Improvement
Affects Versions: 3.0.0-alpha5
Reporter: Evgeny Stanilovsky
Assignee: Vladislav Pyatkov
Seems it`s easy to implement without any performance impact about wrong
formatting (possible human factor) in logging.
Check:
*LOG.info("test [{]", 4);* -> outputs : *INFO: test [{]*
*LOG.info("test [{} {]", 4, 5);* -> outputs : *INFO: test [4 {]*
Warn or possibly exception is useless to raise in such a case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-14937) Index schema & Index management integration
[ https://issues.apache.org/jira/browse/IGNITE-14937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov reassigned IGNITE-14937: - Assignee: Konstantin Orlov > Index schema & Index management integration > --- > > Key: IGNITE-14937 > URL: https://issues.apache.org/jira/browse/IGNITE-14937 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Taras Ledkov >Assignee: Konstantin Orlov >Priority: Major > Labels: ignite-3 > > Public index schema (required indexes) and current indexes state on the > cluster are different. > We have to track it, store it and provide actual indexes schema state for any > components: select query, DDL query etc.. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17453) Calcite Engine: ORDER BY Optimization
YuJue Li created IGNITE-17453:
-
Summary: Calcite Engine: ORDER BY Optimization
Key: IGNITE-17453
URL: https://issues.apache.org/jira/browse/IGNITE-17453
Project: Ignite
Issue Type: Bug
Components: sql
Affects Versions: 2.13
Reporter: YuJue Li
Fix For: 2.14
1.start a node;
2.create table:
CREATE TABLE PI_COM_DAY
(COM_ID VARCHAR(30) NOT NULL ,
ITEM_ID VARCHAR(30) NOT NULL ,
DATE1 VARCHAR(8) NOT NULL ,
KIND VARCHAR(1),
QTY_IOD DECIMAL(18, 6) ,
AMT_IOD DECIMAL(18, 6) ,
PRIMARYKEY (COM_ID,ITEM_ID,DATE1))
WITH"template=partitioned,CACHE_NAME=PI_COM_DAY";
CREATE INDEX IDX_PI_COM_DAY_ITEM_DATE ON PI_COM_DAY(ITEM_ID);
3.insert some data, for example, 1m;
4.execute sql:
select * from pi_com_day order by item_id desc limit 10;
normal.
select /*+ QUERY_ENGINE('calcite') */ * from pi_com_day order by item_id desc
limit 10;
then OOM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17452) [Extensions] Implement Kafka to thin client CDC streamer
Amelchev Nikita created IGNITE-17452: Summary: [Extensions] Implement Kafka to thin client CDC streamer Key: IGNITE-17452 URL: https://issues.apache.org/jira/browse/IGNITE-17452 Project: Ignite Issue Type: Task Reporter: Amelchev Nikita Implement Kafka to thin client CDC streamer -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16743) [Extensions] Implement thin client CDC streamer
[ https://issues.apache.org/jira/browse/IGNITE-16743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amelchev Nikita updated IGNITE-16743: - Labels: IEP-59 ise (was: ise) > [Extensions] Implement thin client CDC streamer > --- > > Key: IGNITE-16743 > URL: https://issues.apache.org/jira/browse/IGNITE-16743 > Project: Ignite > Issue Type: New Feature > Components: extensions >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Labels: IEP-59, ise > > Implement consumer that streams all data changes to thin client. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-14487) Table view bulk methods refactoring
[ https://issues.apache.org/jira/browse/IGNITE-14487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573723#comment-17573723 ] Evgeny Stanilovsky commented on IGNITE-14487: - fill one comment, overall looks good and seems ticket need to be renamed, no streams improvements found for now. > Table view bulk methods refactoring > --- > > Key: IGNITE-14487 > URL: https://issues.apache.org/jira/browse/IGNITE-14487 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Original Estimate: 120h > Time Spent: 10m > Remaining Estimate: 119h 50m > > org.apache.ignite.internal.table.RecordBinaryViewImpl#wrap use streams that > have performance impact. Let's rewrite it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17451) [Extensions] Implement thin client CDC streamer metadata replication
[ https://issues.apache.org/jira/browse/IGNITE-17451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-17451: - Labels: IEP-59 ise (was: ise) > [Extensions] Implement thin client CDC streamer metadata replication > > > Key: IGNITE-17451 > URL: https://issues.apache.org/jira/browse/IGNITE-17451 > Project: Ignite > Issue Type: Task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Labels: IEP-59, ise > > Implement thin client CDC streamer metadata replication -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17451) [Extensions] Implement thin client CDC streamer metadata replication
Amelchev Nikita created IGNITE-17451: Summary: [Extensions] Implement thin client CDC streamer metadata replication Key: IGNITE-17451 URL: https://issues.apache.org/jira/browse/IGNITE-17451 Project: Ignite Issue Type: Task Reporter: Amelchev Nikita Assignee: Amelchev Nikita Implement thin client CDC streamer metadata replication -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17451) [Extensions] Implement thin client CDC streamer metadata replication
[ https://issues.apache.org/jira/browse/IGNITE-17451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amelchev Nikita updated IGNITE-17451: - Labels: ise (was: ) > [Extensions] Implement thin client CDC streamer metadata replication > > > Key: IGNITE-17451 > URL: https://issues.apache.org/jira/browse/IGNITE-17451 > Project: Ignite > Issue Type: Task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Labels: ise > > Implement thin client CDC streamer metadata replication -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17450) Documentation: thin client CDC streamer
[ https://issues.apache.org/jira/browse/IGNITE-17450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-17450: - Component/s: IEP-59 > Documentation: thin client CDC streamer > --- > > Key: IGNITE-17450 > URL: https://issues.apache.org/jira/browse/IGNITE-17450 > Project: Ignite > Issue Type: Task > Components: IEP-59 >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Minor > > Add thin client CDC streamer documentation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17450) Documentation: thin client CDC streamer
[ https://issues.apache.org/jira/browse/IGNITE-17450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-17450: - Labels: IEP-59 (was: ) > Documentation: thin client CDC streamer > --- > > Key: IGNITE-17450 > URL: https://issues.apache.org/jira/browse/IGNITE-17450 > Project: Ignite > Issue Type: Task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Minor > Labels: IEP-59 > > Add thin client CDC streamer documentation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17450) Documentation: thin client CDC streamer
[ https://issues.apache.org/jira/browse/IGNITE-17450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-17450: - Component/s: (was: IEP-59) > Documentation: thin client CDC streamer > --- > > Key: IGNITE-17450 > URL: https://issues.apache.org/jira/browse/IGNITE-17450 > Project: Ignite > Issue Type: Task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Minor > > Add thin client CDC streamer documentation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-14986) Re-work error handling in meta storage component in accordance with error groups
[ https://issues.apache.org/jira/browse/IGNITE-14986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573692#comment-17573692 ] Ivan Bessonov commented on IGNITE-14986: Looks good to me, thank you! > Re-work error handling in meta storage component in accordance with error > groups > > > Key: IGNITE-14986 > URL: https://issues.apache.org/jira/browse/IGNITE-14986 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Assignee: Vyacheslav Koptilin >Priority: Major > Labels: iep-84, ignite-3 > Time Spent: 1h 50m > Remaining Estimate: 0h > > Need to introduce a new error group related to Meta Storage Service and add > all needed error codes. > Also, the implementation should using _IgniteInternalException_ and > _IgniteInternalCheckedException_ with specific error codes. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17450) Documentation: thin client CDC streamer
Amelchev Nikita created IGNITE-17450: Summary: Documentation: thin client CDC streamer Key: IGNITE-17450 URL: https://issues.apache.org/jira/browse/IGNITE-17450 Project: Ignite Issue Type: Task Reporter: Amelchev Nikita Assignee: Amelchev Nikita Add thin client CDC streamer documentation -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17166) Simplify the configuration annotation processor
[ https://issues.apache.org/jira/browse/IGNITE-17166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich updated IGNITE-17166: --- Labels: iep-55 ignite-3 technical-debt (was: iep-55 ignite-3 tech-debt) > Simplify the configuration annotation processor > --- > > Key: IGNITE-17166 > URL: https://issues.apache.org/jira/browse/IGNITE-17166 > Project: Ignite > Issue Type: Task >Reporter: Kirill Tkalenko >Priority: Major > Labels: iep-55, ignite-3, technical-debt > Fix For: 3.0.0-alpha6 > > > At the moment, the > *org.apache.ignite.internal.configuration.processor.Processor* looks > complicated due to the addition of internal, polymorphic and abstract > configuration, the code has become harder to read and edit. > It is proposed to think about how and to divide this class into methods or > subclasses for each type of configuration. > It would also be nice to write validation for class fields, for example that > a field (if not static) can only have one of the annotations *Value*, > *ConfigValue* and *NamedConfigValue*, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17167) Simplify the configuration asm generator
[ https://issues.apache.org/jira/browse/IGNITE-17167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich updated IGNITE-17167: --- Labels: iep-55 ignite-3 technical-debt (was: iep-55 ignite-3 tech-debt) > Simplify the configuration asm generator > > > Key: IGNITE-17167 > URL: https://issues.apache.org/jira/browse/IGNITE-17167 > Project: Ignite > Issue Type: Task >Reporter: Kirill Tkalenko >Priority: Major > Labels: iep-55, ignite-3, technical-debt > Fix For: 3.0.0-alpha6 > > > At the moment, the > *org.apache.ignite.internal.configuration.asm.ConfigurationAsmGenerator* > looks complicated due to the addition of internal, polymorphic and abstract > configuration, the code has become harder to read and edit. > It is proposed to think about how and to divide this class into methods or > subclasses for each type of configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15628) Get rid of waitForTopology method from ITNodeTest
[
https://issues.apache.org/jira/browse/IGNITE-15628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yury Gerzhedovich updated IGNITE-15628:
---
Labels: ignite-3 technical-debt (was: ignite-3 tech-debt)
> Get rid of waitForTopology method from ITNodeTest
> -
>
> Key: IGNITE-15628
> URL: https://issues.apache.org/jira/browse/IGNITE-15628
> Project: Ignite
> Issue Type: Improvement
>Reporter: Mirza Aliev
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3, technical-debt
>
> Get rid of {{waitForTopology}} method from
> {{org.apache.ignite.raft.jraft.core.ITNodeTest}}
> Ticket was created after decompostion of
> https://issues.apache.org/jira/browse/IGNITE-14832
> See todo on {{org.apache.ignite.raft.jraft.core.ITNodeTest#waitForTopology}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17449) CDC: investigate eviction policy of entries
[
https://issues.apache.org/jira/browse/IGNITE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov updated IGNITE-17449:
-
Labels: IEP-59 ise (was: ise)
> CDC: investigate eviction policy of entries
> ---
>
> Key: IGNITE-17449
> URL: https://issues.apache.org/jira/browse/IGNITE-17449
> Project: Ignite
> Issue Type: Task
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
> Labels: IEP-59, ise
>
> The eviction policy can be set not only by configuration but via
> {{IgniteCache#withExpiryPolicy}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17449) CDC: investigate eviction policy of entries
[
https://issues.apache.org/jira/browse/IGNITE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amelchev Nikita reassigned IGNITE-17449:
Assignee: Amelchev Nikita
> CDC: investigate eviction policy of entries
> ---
>
> Key: IGNITE-17449
> URL: https://issues.apache.org/jira/browse/IGNITE-17449
> Project: Ignite
> Issue Type: Task
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Major
> Labels: ise
>
> The eviction policy can be set not only by configuration but via
> {{IgniteCache#withExpiryPolicy}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17449) CDC: investigate eviction policy of entries
[
https://issues.apache.org/jira/browse/IGNITE-17449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amelchev Nikita updated IGNITE-17449:
-
Labels: ise (was: )
> CDC: investigate eviction policy of entries
> ---
>
> Key: IGNITE-17449
> URL: https://issues.apache.org/jira/browse/IGNITE-17449
> Project: Ignite
> Issue Type: Task
>Reporter: Amelchev Nikita
>Priority: Major
> Labels: ise
>
> The eviction policy can be set not only by configuration but via
> {{IgniteCache#withExpiryPolicy}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17355) CLI management of metrics
[ https://issues.apache.org/jira/browse/IGNITE-17355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Chudov updated IGNITE-17355: -- Description: It should be possible to enable or disable metrics sources by metrics source name and to get list of all metrics sources names via Ignite Shell. It should be possible per node basis as well as cluster wide Possible syntax: ignite metrics enable ignite metrics disable ignite metrics list CLI must return error in case of specified name of metrics source doesn't exists. CLI should not return any error in cases of state of metrics source wasn't changed (e.g. metrics source is allready enabled) For receiving metric values via CLI, should be implemented separate exporter, it should be done in separate ticket. was: It should be possible to enable or disable metrics sources by metrics source name and to get list of all metrics sources names via Ignite Shell. It should be possible per node basis as well as cluster wide Possible syntax: ignite metrics enable ignite metrics disable ignite metrics list CLI must return error in case of specified name of metrics source doesn't exists. CLI should not return any error in cases of state of metrics source wasn't changed (e.g. metrics source is allready enabled) > CLI management of metrics > - > > Key: IGNITE-17355 > URL: https://issues.apache.org/jira/browse/IGNITE-17355 > Project: Ignite > Issue Type: New Feature >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > It should be possible to enable or disable metrics sources by metrics source > name and to get list of all metrics sources names via Ignite Shell. > It should be possible per node basis as well as cluster wide > Possible syntax: > ignite metrics enable > ignite metrics disable > ignite metrics list > CLI must return error in case of specified name of metrics source doesn't > exists. > CLI should not return any error in cases of state of metrics source wasn't > changed (e.g. metrics source is allready enabled) > For receiving metric values via CLI, should be implemented separate exporter, > it should be done in separate ticket. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17449) CDC: investigate eviction policy of entries
Amelchev Nikita created IGNITE-17449:
Summary: CDC: investigate eviction policy of entries
Key: IGNITE-17449
URL: https://issues.apache.org/jira/browse/IGNITE-17449
Project: Ignite
Issue Type: Task
Reporter: Amelchev Nikita
The eviction policy can be set not only by configuration but via
{{IgniteCache#withExpiryPolicy}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17444) Consider possibility of introducing pluggable metric exporters
[ https://issues.apache.org/jira/browse/IGNITE-17444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Chudov updated IGNITE-17444: -- Description: Consider possibility of introducing pluggable metric exporters to provide ability for users to create their own exporters thay need. Pluggable metric exporters should implement the common interface, and implementations should be loaded to the application using Java ServiceLoader. The problems of configuration exporters and turning on and off certain implementations should be worked out within IGNITE-17358 . was:Consider possibility of introducing pluggable metric exporters to provide ability for users to create their own exporters thay need. > Consider possibility of introducing pluggable metric exporters > -- > > Key: IGNITE-17444 > URL: https://issues.apache.org/jira/browse/IGNITE-17444 > Project: Ignite > Issue Type: New Feature >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > Consider possibility of introducing pluggable metric exporters to provide > ability for users to create their own exporters thay need. > Pluggable metric exporters should implement the common interface, and > implementations should be loaded to the application using Java ServiceLoader. > The problems of configuration exporters and turning on and off certain > implementations should be worked out within IGNITE-17358 . -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17445) RocksDbKeyValueStorage recreates DB on start, so data can't be found until Raft log is replayed
[
https://issues.apache.org/jira/browse/IGNITE-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573666#comment-17573666
]
Ivan Bessonov edited comment on IGNITE-17445 at 8/1/22 9:56 AM:
This bug is pretty stable in IGNITE-17081, flaky rate of
ItIgniteNodeRestartTest#nodeWithDataTest goes through the roof on the warmed-up
JVM.
The reason is that, presumably, node startup (partitions recovery) became
faster in that branch.
was (Author: ibessonov):
This bug is pretty stable in IGNITE-17081, flaky rate of
ItIgniteNodeRestartTest#nodeWithDataTest goes through the roof on the warmed-up
JVM.
The reason is that node startup (partitions recovery) became faster in that
branch.
> RocksDbKeyValueStorage recreates DB on start, so data can't be found until
> Raft log is replayed
> ---
>
> Key: IGNITE-17445
> URL: https://issues.apache.org/jira/browse/IGNITE-17445
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Chudov
>Priority: Major
>
> RocksDbKeyValueStorage recreates DB on start. This means that entries that
> were put to this storage earlier, can or cant be found until raft log is
> replayed, i.e. the behavior is undefined. For example, this can cause
> assertion on node recovery:
> {code:java}
> java.lang.AssertionError: Configuration revision must be greater than local
> node applied revision [msRev=0, appliedRev=1
> {code}
> which means that applied revision in vault is 1 but only 0 is found in meta
> storage, as the storage of meta storage is being recreated.
> For now, the only thing that saves us from this assertion to be thrown every
> time, is that operations related to node recovery, applied from distributed
> configuration (see IgniteImpl#notifyConfigurationListeners ), take some time
> and raft log is small enough to replay faster than the performing of recovery
> operations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17445) RocksDbKeyValueStorage recreates DB on start, so data can't be found until Raft log is replayed
[
https://issues.apache.org/jira/browse/IGNITE-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573666#comment-17573666
]
Ivan Bessonov commented on IGNITE-17445:
This bug is pretty stable in IGNITE-17081, flaky rate of
ItIgniteNodeRestartTest#nodeWithDataTest goes through the roof on the warmed-up
JVM.
The reason is that node startup (partitions recovery) became faster in that
branch.
> RocksDbKeyValueStorage recreates DB on start, so data can't be found until
> Raft log is replayed
> ---
>
> Key: IGNITE-17445
> URL: https://issues.apache.org/jira/browse/IGNITE-17445
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Chudov
>Priority: Major
>
> RocksDbKeyValueStorage recreates DB on start. This means that entries that
> were put to this storage earlier, can or cant be found until raft log is
> replayed, i.e. the behavior is undefined. For example, this can cause
> assertion on node recovery:
> {code:java}
> java.lang.AssertionError: Configuration revision must be greater than local
> node applied revision [msRev=0, appliedRev=1
> {code}
> which means that applied revision in vault is 1 but only 0 is found in meta
> storage, as the storage of meta storage is being recreated.
> For now, the only thing that saves us from this assertion to be thrown every
> time, is that operations related to node recovery, applied from distributed
> configuration (see IgniteImpl#notifyConfigurationListeners ), take some time
> and raft log is small enough to replay faster than the performing of recovery
> operations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17376) Sql. Investigate of support default value for TIMESTAMP WITH LOCAL TIME ZONE type
[
https://issues.apache.org/jira/browse/IGNITE-17376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yury Gerzhedovich updated IGNITE-17376:
---
Summary: Sql. Investigate of support default value for TIMESTAMP WITH LOCAL
TIME ZONE type (was: Sql. Support default value for TIMESTAMP WITH LOCAL TIME
ZONE type )
> Sql. Investigate of support default value for TIMESTAMP WITH LOCAL TIME ZONE
> type
> --
>
> Key: IGNITE-17376
> URL: https://issues.apache.org/jira/browse/IGNITE-17376
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Konstantin Orlov
>Priority: Major
> Labels: ignite-3, tech-debt
>
> Currently default value is not supported for columns with type {{TIMESTAMP
> WITH LOCAL TIME ZONE}}, because behaviour is not clear:
> * if default value literal MUST contain time zone, then sql parser should be
> extended to provide ability to specify timestamp with time zone literal
> * if default value literal MIGHT NOT contain time zone, then some research is
> needed. Consider the following case:
> {code:java}
> CREATE TABLE t (id INT PRIMARY KEY, val TIMESTAMP WITH LOCAL TIME ZONE
> DEFAULT TIMESTAMP '2021-01-01 01:01:01')
> {code}
> Which timezone should be chosen for converting this literal to UTC: timezone
> of the server or timezone of the client who create this table? Should the
> inserted values be the same in case the user inserts values explicitly or
> implicitly (though DEFAULT) like that:
> {code:java}
> INSERT INTO t (id) VALUES (0);
>VS
> INSERT INTO t (id, val) VALUES (0, DEFAULT);
>VS
> INSERT INTO t (id, val) VALUES (0, TIMESTAMP '2021-01-01 01:01:01');
> {code}
> Let's provide some research on this topic and chose the proper way to address
> the issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-15807) In case of port is absent in IgniteClient first of all try connect on default port
[ https://issues.apache.org/jira/browse/IGNITE-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573648#comment-17573648 ] Maksim Timonin edited comment on IGNITE-15807 at 8/1/22 9:10 AM: - > Did we have any kind of "balancing" before? I think the fix preserves >existing logic. Yes, it chose channel by Random, now it's always 0. See `ReliableChannel#initChannelHolders`: now here `dfltChannelIdx = 0` , before it was random. > user might expect us to connect to the specified addresses in the specified > order I believe that user will configure only stable nodes. Otherwise, it can configure it with ClientAddressFinder that was implemented for dynamic cases. was (Author: timonin.maksim): > Did we have any kind of "balancing" before? I think the fix preserves >existing logic. Yes, it chose channel by Random, not it's always 0. See `ReliableChannel#initChannelHolders`: now here `dfltChannelIdx = 0` , before it was random. > user might expect us to connect to the specified addresses in the specified > order I believe that user will configure only stable nodes. In such case, otherwise it can configure it with ClientAddressFinder that was implemented for dynamic cases. > In case of port is absent in IgniteClient first of all try connect on default > port > -- > > Key: IGNITE-15807 > URL: https://issues.apache.org/jira/browse/IGNITE-15807 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.11 >Reporter: Ilya Kazakov >Assignee: Ilya Kazakov >Priority: Minor > Fix For: 2.12 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Now if in ClientConfiguration address specified without port, client will try > to connect to default port range in random order. But often server listen on > default port. This random order leads to unexpected connection time. > Will be better if firstly client will try to connect to default port, before > randomly scan all port range. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-15807) In case of port is absent in IgniteClient first of all try connect on default port
[ https://issues.apache.org/jira/browse/IGNITE-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573648#comment-17573648 ] Maksim Timonin commented on IGNITE-15807: - > Did we have any kind of "balancing" before? I think the fix preserves >existing logic. Yes, it chose channel by Random, not it's always 0. See `ReliableChannel#initChannelHolders`: now here `dfltChannelIdx = 0` , before it was random. > user might expect us to connect to the specified addresses in the specified > order I believe that user will configure only stable nodes. In such case, otherwise it can configure it with ClientAddressFinder that was implemented for dynamic cases. > In case of port is absent in IgniteClient first of all try connect on default > port > -- > > Key: IGNITE-15807 > URL: https://issues.apache.org/jira/browse/IGNITE-15807 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.11 >Reporter: Ilya Kazakov >Assignee: Ilya Kazakov >Priority: Minor > Fix For: 2.12 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Now if in ClientConfiguration address specified without port, client will try > to connect to default port range in random order. But often server listen on > default port. This random order leads to unexpected connection time. > Will be better if firstly client will try to connect to default port, before > randomly scan all port range. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17359) Sql. Implement session auto expiration
[ https://issues.apache.org/jira/browse/IGNITE-17359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich updated IGNITE-17359: --- Description: Currently server-side sessions are created but never deleted. Session objects clogs the memory and lead to OOME. Need to introduce background task to check each session from time to time and clean up those that have expired or use Caffeine. start point - org.apache.ignite.internal.sql.engine.session.SessionManager#activeSessions was: Currently server-side sessions are created but never deleted. Session objects clogs the memory and lead to OOME. Need to introduce background task to check each session from time to time and clean up those that have expired. > Sql. Implement session auto expiration > --- > > Key: IGNITE-17359 > URL: https://issues.apache.org/jira/browse/IGNITE-17359 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Konstantin Orlov >Priority: Major > Labels: ignite-3 > > Currently server-side sessions are created but never deleted. Session objects > clogs the memory and lead to OOME. > Need to introduce background task to check each session from time to time and > clean up those that have expired or use Caffeine. > start point - > org.apache.ignite.internal.sql.engine.session.SessionManager#activeSessions -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17448) Implementation of metric source for cluster state
[ https://issues.apache.org/jira/browse/IGNITE-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Chudov updated IGNITE-17448: -- Issue Type: New Feature (was: Bug) > Implementation of metric source for cluster state > - > > Key: IGNITE-17448 > URL: https://issues.apache.org/jira/browse/IGNITE-17448 > Project: Ignite > Issue Type: New Feature >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > At least, could be implemented metric showing *topology version*. > What is needed to achieve this: > Define metrics source class and metrics holder class for ignite compute. > Define all metric variables in the holder class, if needed. > Implement initialization logic which is responsible for creation of actual > instances of metrics and building an immutable instance of metrics set (will > be executed on metrics enabling). Implement additional clean up logic if > needed (will be executed on metrics disabled). > Define methods that implement desirable manipulations with metrics. Every > such method must check the state of the metrics source (enabled\disabled) by > testing the holder reference to null (some kind of guard) before modifying a > metric value. Read holder reference only once and use it in scope of the > method in order to avoid data races. > Metrics source should be registered in metrics registry on component start > and should be unregistered on component stop. Initial metrics source state > (enabled\disabled) should be set according to configuration for each > particular component. > Add logic related with metrics values modification (add invocations of > metrics source methods). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-15807) In case of port is absent in IgniteClient first of all try connect on default port
[ https://issues.apache.org/jira/browse/IGNITE-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573639#comment-17573639 ] Pavel Tupitsyn commented on IGNITE-15807: - [~timonin.maksim] > this fix breaks balancing Did we have any kind of "balancing" before? I think the fix preserves existing logic. Regarding "primary" addresses: for example, there might be a dynamic cluster where a few nodes always exist, but other nodes are added/removed based on the load, so the user might expect us to connect to the specified addresses in the specified order, not randomly. If the user wants round-robin or random order, they can reorder the addresses in config easily. > In case of port is absent in IgniteClient first of all try connect on default > port > -- > > Key: IGNITE-15807 > URL: https://issues.apache.org/jira/browse/IGNITE-15807 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.11 >Reporter: Ilya Kazakov >Assignee: Ilya Kazakov >Priority: Minor > Fix For: 2.12 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Now if in ClientConfiguration address specified without port, client will try > to connect to default port range in random order. But often server listen on > default port. This random order leads to unexpected connection time. > Will be better if firstly client will try to connect to default port, before > randomly scan all port range. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-16040) Calcite. Unable to plan query with several correlated sub-queries in select list
[
https://issues.apache.org/jira/browse/IGNITE-16040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573631#comment-17573631
]
Yury Gerzhedovich commented on IGNITE-16040:
Also we need to unmute tests related to the bug
> Calcite. Unable to plan query with several correlated sub-queries in select
> list
>
>
> Key: IGNITE-16040
> URL: https://issues.apache.org/jira/browse/IGNITE-16040
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Konstantin Orlov
>Assignee: Konstantin Orlov
>Priority: Major
> Labels: calcite, calcite2-required, calcite3-required
>
> Currently the query like below can't be planned by calcite-based sql engine:
> {code:java}
> SELECT a+b*2,
>(a+b+c+d+e)/5,
>(SELECT count(*) FROM t1 AS x WHERE x.c>t1.c AND x.d(SELECT count(*) FROM t1 AS x WHERE x.babs(b-c),
>a-b
> FROM t1
> WHERE EXISTS(SELECT 1 FROM t1 AS x WHERE x.bAND c>d
> ORDER BY 6,5,4,1,3,2
> {code}
> Need to figure it out how to fix this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-15807) In case of port is absent in IgniteClient first of all try connect on default port
[ https://issues.apache.org/jira/browse/IGNITE-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573628#comment-17573628 ] Maksim Timonin edited comment on IGNITE-15807 at 8/1/22 8:35 AM: - [~ptupitsyn] [~kazakov] Hi guys! Could you please explain what is a "primary address" in the PR discussion[1] ? It looks like this fix breaks balancing of thin clients connection to cluster (by addresses). Is any explanation why this is needed? I agree that default port should be preferable, but why the first address? In case somebody needs predefined order of addresses, one can use ClientAddressFinder that return strict order of addresses. [1] [https://github.com/apache/ignite/pull/9522/files/274f488875a9ed8097b3aab2bd30765e8b98373a#r738304340] was (Author: timonin.maksim): [~ptupitsyn] [~kazakov] Hi guys! Could you please explain what is a "primary address" in the PR discussion[1] ? It looks like this fix breaks balancing of thin clients connection to cluster (by addresses). Is any explanation why this is needed? I agree that default port should be preferable, but why the first address? [1] https://github.com/apache/ignite/pull/9522/files/274f488875a9ed8097b3aab2bd30765e8b98373a#r738304340 > In case of port is absent in IgniteClient first of all try connect on default > port > -- > > Key: IGNITE-15807 > URL: https://issues.apache.org/jira/browse/IGNITE-15807 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.11 >Reporter: Ilya Kazakov >Assignee: Ilya Kazakov >Priority: Minor > Fix For: 2.12 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Now if in ClientConfiguration address specified without port, client will try > to connect to default port range in random order. But often server listen on > default port. This random order leads to unexpected connection time. > Will be better if firstly client will try to connect to default port, before > randomly scan all port range. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-15807) In case of port is absent in IgniteClient first of all try connect on default port
[ https://issues.apache.org/jira/browse/IGNITE-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573628#comment-17573628 ] Maksim Timonin commented on IGNITE-15807: - [~ptupitsyn] [~kazakov] Hi guys! Could you please explain what is a "primary address" in the PR discussion[1] ? It looks like this fix breaks balancing of thin clients connection to cluster (by addresses). Is any explanation why this is needed? I agree that default port should be preferable, but why the first address? [1] https://github.com/apache/ignite/pull/9522/files/274f488875a9ed8097b3aab2bd30765e8b98373a#r738304340 > In case of port is absent in IgniteClient first of all try connect on default > port > -- > > Key: IGNITE-15807 > URL: https://issues.apache.org/jira/browse/IGNITE-15807 > Project: Ignite > Issue Type: Improvement > Components: thin client >Affects Versions: 2.11 >Reporter: Ilya Kazakov >Assignee: Ilya Kazakov >Priority: Minor > Fix For: 2.12 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Now if in ClientConfiguration address specified without port, client will try > to connect to default port range in random order. But often server listen on > default port. This random order leads to unexpected connection time. > Will be better if firstly client will try to connect to default port, before > randomly scan all port range. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17434) Extend implicit sql transactions functionality for RO|RW transactions support.
[ https://issues.apache.org/jira/browse/IGNITE-17434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evgeny Stanilovsky updated IGNITE-17434: Epic Link: IGNITE-15081 > Extend implicit sql transactions functionality for RO|RW transactions support. > -- > > Key: IGNITE-17434 > URL: https://issues.apache.org/jira/browse/IGNITE-17434 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 3.0.0-alpha5 >Reporter: Evgeny Stanilovsky >Priority: Major > Labels: ignite-3 > > After [1] was implemented, there are some improvements seems still required: > # If no explicit tx is specified it`s possible to obtain situation when keys > from one bulk enlisted under different tx [2] > # Implicit tx (not tx at all but tx meta - timestamp, first enlisted > partition and so on ..) need to be replicated into all calcite execution > fragments. > Thus in [3] seems we need to start implicit tx if no explicit defined and > correctly process it in [4] > [1] https://issues.apache.org/jira/browse/IGNITE-17328 > [2] ModifyNode#flushTuples > [3] ExecutionServiceImpl#executeQuery > [4] DistributedQueryManager#execute -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17448) Implementation of metric source for cluster state
[ https://issues.apache.org/jira/browse/IGNITE-17448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Chudov updated IGNITE-17448: -- Description: At least, could be implemented metric showing *topology version*. What is needed to achieve this: Define metrics source class and metrics holder class for ignite compute. Define all metric variables in the holder class, if needed. Implement initialization logic which is responsible for creation of actual instances of metrics and building an immutable instance of metrics set (will be executed on metrics enabling). Implement additional clean up logic if needed (will be executed on metrics disabled). Define methods that implement desirable manipulations with metrics. Every such method must check the state of the metrics source (enabled\disabled) by testing the holder reference to null (some kind of guard) before modifying a metric value. Read holder reference only once and use it in scope of the method in order to avoid data races. Metrics source should be registered in metrics registry on component start and should be unregistered on component stop. Initial metrics source state (enabled\disabled) should be set according to configuration for each particular component. Add logic related with metrics values modification (add invocations of metrics source methods). was: At least, could be implemented metric showing *topology version*. What is needed to achieve this: Define metrics source class and metrics holder class for ignite compute. Define all metric variables in the holder class, if needed. Implement initialization logic which is responsible for creation of actual instances of metrics and building an immutable instance of metrics set (will be executed on metrics enabling). Implement additional clean up logic if needed (will be executed on metrics disabled). Define methods that implement desirable manipulations with metrics. Every such method must check the state of the metrics source (enabled\disabled) by testing the holder reference to null (some kind of guard) before modifying a metric value. Read holder reference only once and use it in scope of the method in order to avoid data races. Metrics source should be registered in metrics registry on component start and should be unregistered on component stop. Initial metrics source state (enabled\disabled) should be set according to configuration for each particular component. Add logic related with metrics values modification (add invocations of metrics source methods). > Implementation of metric source for cluster state > - > > Key: IGNITE-17448 > URL: https://issues.apache.org/jira/browse/IGNITE-17448 > Project: Ignite > Issue Type: Bug >Reporter: Denis Chudov >Priority: Major > Labels: ignite-3 > > At least, could be implemented metric showing *topology version*. > What is needed to achieve this: > Define metrics source class and metrics holder class for ignite compute. > Define all metric variables in the holder class, if needed. > Implement initialization logic which is responsible for creation of actual > instances of metrics and building an immutable instance of metrics set (will > be executed on metrics enabling). Implement additional clean up logic if > needed (will be executed on metrics disabled). > Define methods that implement desirable manipulations with metrics. Every > such method must check the state of the metrics source (enabled\disabled) by > testing the holder reference to null (some kind of guard) before modifying a > metric value. Read holder reference only once and use it in scope of the > method in order to avoid data races. > Metrics source should be registered in metrics registry on component start > and should be unregistered on component stop. Initial metrics source state > (enabled\disabled) should be set according to configuration for each > particular component. > Add logic related with metrics values modification (add invocations of > metrics source methods). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17448) Implementation of metric source for cluster state
Denis Chudov created IGNITE-17448: - Summary: Implementation of metric source for cluster state Key: IGNITE-17448 URL: https://issues.apache.org/jira/browse/IGNITE-17448 Project: Ignite Issue Type: Bug Reporter: Denis Chudov At least, could be implemented metric showing *topology version*. What is needed to achieve this: Define metrics source class and metrics holder class for ignite compute. Define all metric variables in the holder class, if needed. Implement initialization logic which is responsible for creation of actual instances of metrics and building an immutable instance of metrics set (will be executed on metrics enabling). Implement additional clean up logic if needed (will be executed on metrics disabled). Define methods that implement desirable manipulations with metrics. Every such method must check the state of the metrics source (enabled\disabled) by testing the holder reference to null (some kind of guard) before modifying a metric value. Read holder reference only once and use it in scope of the method in order to avoid data races. Metrics source should be registered in metrics registry on component start and should be unregistered on component stop. Initial metrics source state (enabled\disabled) should be set according to configuration for each particular component. Add logic related with metrics values modification (add invocations of metrics source methods). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17447) Support transform for IndexQuery
Maksim Timonin created IGNITE-17447: --- Summary: Support transform for IndexQuery Key: IGNITE-17447 URL: https://issues.apache.org/jira/browse/IGNITE-17447 Project: Ignite Issue Type: New Feature Reporter: Maksim Timonin Currently IndexQuery doesn't support transformation with `cache#query(Query, IgniteClosure)`. But as IndexQuery re-use some infrastructure of ScanQuery, it's possible to implement this for IndexQuery too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16657) [Native Persistence 3.0] Implement partitions chunks merger
[ https://issues.apache.org/jira/browse/IGNITE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-16657: - Description: At the moment, after each checkpoint, delta files are created for each modified partition (which have dirty pages). We need to implement merging the delta file into the main partition file to reduce read amplification. Suggested solution: * Merge to make multi-thread, the number of threads to set in the configuration; * Each thread processes one delta file at a time; * Merge to start as soon as there are delta files. The question remains, what to do when a checkpoint and merge of delta files are working at the same time, who should be given priority and how to deal with it? It is proposed to think over and do in IGNITE-17446. was: At the moment, after each checkpoint, delta files are created for each modified partition (which have dirty pages). We need to implement merging the delta file into the main partition file to reduce read amplification. Suggested solution: * Merge to make multi-thread, the number of threads to set in the configuration; * Each thread processes one delta file at a time; * Merge to start as soon as there are delta files. The question remains, what to do when a checkpoint and merge of delta files are working at the same time, who should be given priority and how to deal with it? It is proposed to think over and do in 123. > [Native Persistence 3.0] Implement partitions chunks merger > --- > > Key: IGNITE-16657 > URL: https://issues.apache.org/jira/browse/IGNITE-16657 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > > At the moment, after each checkpoint, delta files are created for each > modified partition (which have dirty pages). > We need to implement merging the delta file into the main partition file to > reduce read amplification. > Suggested solution: > * Merge to make multi-thread, the number of threads to set in the > configuration; > * Each thread processes one delta file at a time; > * Merge to start as soon as there are delta files. > The question remains, what to do when a checkpoint and merge of delta files > are working at the same time, who should be given priority and how to deal > with it? It is proposed to think over and do in IGNITE-17446. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17446) Determine priority between checkpoint and merge delta files
Kirill Tkalenko created IGNITE-17446: Summary: Determine priority between checkpoint and merge delta files Key: IGNITE-17446 URL: https://issues.apache.org/jira/browse/IGNITE-17446 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko Fix For: 3.0.0-alpha6 When creating delta files in parallel and merging them with the main partition file, the write amplification may increase, and they may begin to interfere with each other. In RocksDb, these processes run in parallel, but checkpoint threads have higher thread priority. I think it can be done by analogy, but without thread priorities, but with slowing down the threads of the delta file merge. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16657) [Native Persistence 3.0] Implement partitions chunks merger
[ https://issues.apache.org/jira/browse/IGNITE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-16657: - Description: At the moment, after each checkpoint, delta files are created for each modified partition (which have dirty pages). We need to implement merging the delta file into the main partition file to reduce read amplification. Suggested solution: * Merge to make multi-thread, the number of threads to set in the configuration; * Each thread processes one delta file at a time; * Merge to start as soon as there are delta files. The question remains, what to do when a checkpoint and merge of delta files are working at the same time, who should be given priority and how to deal with it? It is proposed to think over and do in 123. was: At the moment, after each checkpoint, delta files are created for each modified partition (which have dirty pages). We need to implement merging the delta file into the main partition file to reduce read amplification. Suggested solution: * Merge to make multi-thread, the number of threads to set in the configuration; * Each thread processes one delta file at a time; * Merge to start as soon as there are delta files. > [Native Persistence 3.0] Implement partitions chunks merger > --- > > Key: IGNITE-16657 > URL: https://issues.apache.org/jira/browse/IGNITE-16657 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > > At the moment, after each checkpoint, delta files are created for each > modified partition (which have dirty pages). > We need to implement merging the delta file into the main partition file to > reduce read amplification. > Suggested solution: > * Merge to make multi-thread, the number of threads to set in the > configuration; > * Each thread processes one delta file at a time; > * Merge to start as soon as there are delta files. > The question remains, what to do when a checkpoint and merge of delta files > are working at the same time, who should be given priority and how to deal > with it? It is proposed to think over and do in 123. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16657) [Native Persistence 3.0] Implement partitions chunks merger
[ https://issues.apache.org/jira/browse/IGNITE-16657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-16657: - Description: At the moment, after each checkpoint, delta files are created for each modified partition (which have dirty pages). We need to implement merging the delta file into the main partition file to reduce read amplification. Suggested solution: * Merge to make multi-thread, the number of threads to set in the configuration; * Each thread processes one delta file at a time; * Merge to start as soon as there are delta files. was:Please refer to https://issues.apache.org/jira/browse/IGNITE-15818 for the description. This issue should include the merge process itself. > [Native Persistence 3.0] Implement partitions chunks merger > --- > > Key: IGNITE-16657 > URL: https://issues.apache.org/jira/browse/IGNITE-16657 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > > At the moment, after each checkpoint, delta files are created for each > modified partition (which have dirty pages). > We need to implement merging the delta file into the main partition file to > reduce read amplification. > Suggested solution: > * Merge to make multi-thread, the number of threads to set in the > configuration; > * Each thread processes one delta file at a time; > * Merge to start as soon as there are delta files. -- This message was sent by Atlassian Jira (v8.20.10#820010)
