[jira] [Commented] (IGNITE-9272) PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace.
[ https://issues.apache.org/jira/browse/IGNITE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632809#comment-16632809 ] Nikolay Izhikov commented on IGNITE-9272: - [~zstan] Please, reolve my comments in PR: https://github.com/apache/ignite/pull/4619 > PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace. > -- > > Key: IGNITE-9272 > URL: https://issues.apache.org/jira/browse/IGNITE-9272 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.8 > > Attachments: BenchmarkCRC.java > > > I see that Ignite has its own crc32 realization called: PureJavaCrc32 and > from desc it seems to be : _The current version is ~10x to 1.8x as fast as > Sun's native java.util.zip.CRC32 in Java 1.6_ But my jmh tests show opposite > results. > + If it really so, looks like backward compatibility would be easy, all that > need is just to take lower part of long form zip.crc32 realization. > jmh results: > Benchmark Mode CntScoreError Units > BenchmarkCRC.Crc32 avgt5 1521060.716 ± 44083.424 ns/op > BenchmarkCRC.pureJavaCrc32 avgt5 4657756.671 ± 177243.254 ns/op > JMH version: 1.21 > VM version: JDK 1.8.0_131, Java HotSpot(TM) 64-Bit Server VM, 25.131-b11 > VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java > op system : ubuntu 16.10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-9272) PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace.
[ https://issues.apache.org/jira/browse/IGNITE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632809#comment-16632809 ] Nikolay Izhikov edited comment on IGNITE-9272 at 9/29/18 5:51 AM: -- [~zstan] Please, resolve my comments in PR: https://github.com/apache/ignite/pull/4619 was (Author: nizhikov): [~zstan] Please, reolve my comments in PR: https://github.com/apache/ignite/pull/4619 > PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace. > -- > > Key: IGNITE-9272 > URL: https://issues.apache.org/jira/browse/IGNITE-9272 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.8 > > Attachments: BenchmarkCRC.java > > > I see that Ignite has its own crc32 realization called: PureJavaCrc32 and > from desc it seems to be : _The current version is ~10x to 1.8x as fast as > Sun's native java.util.zip.CRC32 in Java 1.6_ But my jmh tests show opposite > results. > + If it really so, looks like backward compatibility would be easy, all that > need is just to take lower part of long form zip.crc32 realization. > jmh results: > Benchmark Mode CntScoreError Units > BenchmarkCRC.Crc32 avgt5 1521060.716 ± 44083.424 ns/op > BenchmarkCRC.pureJavaCrc32 avgt5 4657756.671 ± 177243.254 ns/op > JMH version: 1.21 > VM version: JDK 1.8.0_131, Java HotSpot(TM) 64-Bit Server VM, 25.131-b11 > VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java > op system : ubuntu 16.10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-9740) [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up to IGNITE-9711)
[
https://issues.apache.org/jira/browse/IGNITE-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632597#comment-16632597
]
Oleg Ignatenko edited comment on IGNITE-9740 at 9/28/18 10:20 PM:
--
[~chief] as promised when we spoke today I checked ml unit tests. It looks like
the only test class that involves {{IgniteThread}} is the one reported in this
ticket (EvaluatorTest).
was (Author: oignatenko):
[~chief] as promised when we spoke today I checked ml unit tests. It looks like
the only test class that involves {{IgniteThread}} is the one reported in this
ticket.
> [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up
> to IGNITE-9711)
> ---
>
> Key: IGNITE-9740
> URL: https://issues.apache.org/jira/browse/IGNITE-9740
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Priority: Minor
> Fix For: 2.8
>
>
> [EvaluatorTest|https://github.com/apache/ignite/blob/master/modules/ml/src/test/java/org/apache/ignite/ml/selection/scoring/evaluator/EvaluatorTest.java]
> involves {{IgniteThread}} which is in fact not needed there and should be
> removed.
> {{IgniteThread}} usage is a remainder / copy-paste from older tests and
> examples that were using API requiring it. This API has been removed and
> there is no need for wrapping like that anymore. For the reference on how to
> perform suggested cleanup check changes made to ml examples per IGNITE-9711.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9740) [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up to IGNITE-9711)
[
https://issues.apache.org/jira/browse/IGNITE-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632597#comment-16632597
]
Oleg Ignatenko commented on IGNITE-9740:
[~chief] as promised when we spoke today I checked ml unit tests. It looks like
the only test class that involves {{IgniteThread}} is the one reported in this
ticket.
> [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up
> to IGNITE-9711)
> ---
>
> Key: IGNITE-9740
> URL: https://issues.apache.org/jira/browse/IGNITE-9740
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Priority: Minor
> Fix For: 2.8
>
>
> [EvaluatorTest|https://github.com/apache/ignite/blob/master/modules/ml/src/test/java/org/apache/ignite/ml/selection/scoring/evaluator/EvaluatorTest.java]
> involves {{IgniteThread}} which is in fact not needed there and should be
> removed.
> {{IgniteThread}} usage is a remainder / copy-paste from older tests and
> examples that were using API requiring it. This API has been removed and
> there is no need for wrapping like that anymore. For the reference on how to
> perform suggested cleanup check changes made to ml examples per IGNITE-9711.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9740) [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up to IGNITE-9711)
[
https://issues.apache.org/jira/browse/IGNITE-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko updated IGNITE-9740:
---
Fix Version/s: 2.8
> [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up
> to IGNITE-9711)
> ---
>
> Key: IGNITE-9740
> URL: https://issues.apache.org/jira/browse/IGNITE-9740
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Priority: Minor
> Fix For: 2.8
>
>
> [EvaluatorTest|https://github.com/apache/ignite/blob/master/modules/ml/src/test/java/org/apache/ignite/ml/selection/scoring/evaluator/EvaluatorTest.java]
> involves {{IgniteThread}} which is in fact not needed there and should be
> removed.
> {{IgniteThread}} usage is a remainder / copy-paste from older tests and
> examples that were using API requiring it. This API has been removed and
> there is no need for wrapping like that anymore. For the reference on how to
> perform suggested cleanup check changes made to ml examples per IGNITE-9711.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9740) [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up to IGNITE-9711)
[
https://issues.apache.org/jira/browse/IGNITE-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Ignatenko updated IGNITE-9740:
---
Ignite Flags: (was: Docs Required)
> [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up
> to IGNITE-9711)
> ---
>
> Key: IGNITE-9740
> URL: https://issues.apache.org/jira/browse/IGNITE-9740
> Project: Ignite
> Issue Type: Bug
> Components: ml
>Affects Versions: 2.6
>Reporter: Oleg Ignatenko
>Priority: Minor
> Fix For: 2.8
>
>
> [EvaluatorTest|https://github.com/apache/ignite/blob/master/modules/ml/src/test/java/org/apache/ignite/ml/selection/scoring/evaluator/EvaluatorTest.java]
> involves {{IgniteThread}} which is in fact not needed there and should be
> removed.
> {{IgniteThread}} usage is a remainder / copy-paste from older tests and
> examples that were using API requiring it. This API has been removed and
> there is no need for wrapping like that anymore. For the reference on how to
> perform suggested cleanup check changes made to ml examples per IGNITE-9711.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-9740) [ML] Remove IgniteThread wrapper from ml unit test EvaluatorTest (follow up to IGNITE-9711)
Oleg Ignatenko created IGNITE-9740:
--
Summary: [ML] Remove IgniteThread wrapper from ml unit test
EvaluatorTest (follow up to IGNITE-9711)
Key: IGNITE-9740
URL: https://issues.apache.org/jira/browse/IGNITE-9740
Project: Ignite
Issue Type: Bug
Components: ml
Affects Versions: 2.6
Reporter: Oleg Ignatenko
[EvaluatorTest|https://github.com/apache/ignite/blob/master/modules/ml/src/test/java/org/apache/ignite/ml/selection/scoring/evaluator/EvaluatorTest.java]
involves {{IgniteThread}} which is in fact not needed there and should be
removed.
{{IgniteThread}} usage is a remainder / copy-paste from older tests and
examples that were using API requiring it. This API has been removed and there
is no need for wrapping like that anymore. For the reference on how to perform
suggested cleanup check changes made to ml examples per IGNITE-9711.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-7651) Assertion error on cache start while reading meta pages
[
https://issues.apache.org/jira/browse/IGNITE-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eduard Shangareev updated IGNITE-7651:
--
Fix Version/s: (was: 2.8)
> Assertion error on cache start while reading meta pages
> ---
>
> Key: IGNITE-7651
> URL: https://issues.apache.org/jira/browse/IGNITE-7651
> Project: Ignite
> Issue Type: Bug
>Reporter: Eduard Shangareev
>Priority: Critical
>
> We recently have added assertion. Now it has found issue with start cache
> after restart.
> {code:java}
> java.lang.AssertionError: calculatedOffset=16384, allocated=8192,
> headerSize=4096
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:341)
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:322)
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:306)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:658)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:578)
> at
> org.apache.ignite.internal.processors.cache.persistence.DataStructure.acquirePage(DataStructure.java:130)
> at
> org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.init(PagesList.java:167)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.reuse.ReuseListImpl.(ReuseListImpl.java:56)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.initDataStructures(GridCacheOffheapManager.java:103)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.start(IgniteCacheOffheapManagerImpl.java:131)
> at
> org.apache.ignite.internal.processors.cache.CacheGroupContext.start(CacheGroupContext.java:893)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCacheGroup(GridCacheProcessor.java:1979)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1869)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:1759)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:769)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:651)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2337)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IGNITE-7651) Assertion error on cache start while reading meta pages
[
https://issues.apache.org/jira/browse/IGNITE-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eduard Shangareev resolved IGNITE-7651.
---
Resolution: Duplicate
I assume that it should be the same issue with a dirty reads of the metastore.
> Assertion error on cache start while reading meta pages
> ---
>
> Key: IGNITE-7651
> URL: https://issues.apache.org/jira/browse/IGNITE-7651
> Project: Ignite
> Issue Type: Bug
>Reporter: Eduard Shangareev
>Priority: Critical
> Fix For: 2.8
>
>
> We recently have added assertion. Now it has found issue with start cache
> after restart.
> {code:java}
> java.lang.AssertionError: calculatedOffset=16384, allocated=8192,
> headerSize=4096
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:341)
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:322)
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.read(FilePageStoreManager.java:306)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:658)
> at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.acquirePage(PageMemoryImpl.java:578)
> at
> org.apache.ignite.internal.processors.cache.persistence.DataStructure.acquirePage(DataStructure.java:130)
> at
> org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.init(PagesList.java:167)
> at
> org.apache.ignite.internal.processors.cache.persistence.tree.reuse.ReuseListImpl.(ReuseListImpl.java:56)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.initDataStructures(GridCacheOffheapManager.java:103)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.start(IgniteCacheOffheapManagerImpl.java:131)
> at
> org.apache.ignite.internal.processors.cache.CacheGroupContext.start(CacheGroupContext.java:893)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCacheGroup(GridCacheProcessor.java:1979)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1869)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:1759)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:769)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:651)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2337)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-7054) S3 IP finder: support client side encryption
[
https://issues.apache.org/jira/browse/IGNITE-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Valentin Kulichenko updated IGNITE-7054:
Fix Version/s: (was: 2.7)
2.8
> S3 IP finder: support client side encryption
>
>
> Key: IGNITE-7054
> URL: https://issues.apache.org/jira/browse/IGNITE-7054
> Project: Ignite
> Issue Type: Improvement
> Components: s3
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Uday Kale
>Priority: Major
> Fix For: 2.8
>
>
> In case client side encryption [1] is used, it may be required to use
> {{AmazonS3EncryptionClient}} instead of regular {{AmazonS3Client}}. We need
> to add this option to the S3 IP finder, along with any applicable
> configuration parameters.
> [1]
> http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9732) Add joins to Spark Dataframe examples
[
https://issues.apache.org/jira/browse/IGNITE-9732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632529#comment-16632529
]
Valentin Kulichenko commented on IGNITE-9732:
-
[~vozerov], moved to 2.8
> Add joins to Spark Dataframe examples
> -
>
> Key: IGNITE-9732
> URL: https://issues.apache.org/jira/browse/IGNITE-9732
> Project: Ignite
> Issue Type: Improvement
> Components: examples, spark
>Affects Versions: 2.6
>Reporter: Valentin Kulichenko
>Priority: Major
> Fix For: 2.8
>
>
> {{IgniteDataFrameExample}} creates two tables - {{city}} and {{person}}, but
> only {{person}} is actually used. Need to add join examples.
> Would also be great to demonstrate the fact that optimization is working and
> joins are executed in Ignite, not Spark (using {{explain()}}, maybe?).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9732) Add joins to Spark Dataframe examples
[
https://issues.apache.org/jira/browse/IGNITE-9732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Valentin Kulichenko updated IGNITE-9732:
Fix Version/s: 2.8
> Add joins to Spark Dataframe examples
> -
>
> Key: IGNITE-9732
> URL: https://issues.apache.org/jira/browse/IGNITE-9732
> Project: Ignite
> Issue Type: Improvement
> Components: examples, spark
>Affects Versions: 2.6
>Reporter: Valentin Kulichenko
>Priority: Major
> Fix For: 2.8
>
>
> {{IgniteDataFrameExample}} creates two tables - {{city}} and {{person}}, but
> only {{person}} is actually used. Need to add join examples.
> Would also be great to demonstrate the fact that optimization is working and
> joins are executed in Ignite, not Spark (using {{explain()}}, maybe?).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9732) Add joins to Spark Dataframe examples
[
https://issues.apache.org/jira/browse/IGNITE-9732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Valentin Kulichenko updated IGNITE-9732:
Fix Version/s: (was: 2.7)
> Add joins to Spark Dataframe examples
> -
>
> Key: IGNITE-9732
> URL: https://issues.apache.org/jira/browse/IGNITE-9732
> Project: Ignite
> Issue Type: Improvement
> Components: examples, spark
>Affects Versions: 2.6
>Reporter: Valentin Kulichenko
>Priority: Major
> Fix For: 2.8
>
>
> {{IgniteDataFrameExample}} creates two tables - {{city}} and {{person}}, but
> only {{person}} is actually used. Need to add join examples.
> Would also be great to demonstrate the fact that optimization is working and
> joins are executed in Ignite, not Spark (using {{explain()}}, maybe?).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9710) Ignite watchdog service handles longrunning cache creation
[
https://issues.apache.org/jira/browse/IGNITE-9710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632230#comment-16632230
]
Andrey Kuznetsov commented on IGNITE-9710:
--
Wrapping entire {{GridDhtPartitionsExchangeFuture.init()}} into blocking
section is too rough solution. By doing this we in fact make
partition-exchanger thread not critical.
To fix the issue thoroughly, {{init()}} body should be examined, and long
running operations should be interleaved with {{GridWorker.updateHeartbeat()}}
calls or wrapped into critical sections, depending on the situation.
> Ignite watchdog service handles longrunning cache creation
> --
>
> Key: IGNITE-9710
> URL: https://issues.apache.org/jira/browse/IGNITE-9710
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.6
>Reporter: Vyacheslav Daradur
>Assignee: Andrey Kuznetsov
>Priority: Major
> Fix For: 2.7
>
> Attachments: LongRunningCacheCreationTest.java
>
>
> Ignite watchdog service introduced by IGNITE-6587 handles long running cache
> creation.
> Action in {{GridDhtPartitionsExchangeFuture#init}} may take significant time
> and possibly should be covered by blocking section of warchdog service.
> Reproducer was attached: [^LongRunningCacheCreationTest.java].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-9710) Ignite watchdog service handles longrunning cache creation
[
https://issues.apache.org/jira/browse/IGNITE-9710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Kuznetsov reassigned IGNITE-9710:
Assignee: Andrey Kuznetsov
> Ignite watchdog service handles longrunning cache creation
> --
>
> Key: IGNITE-9710
> URL: https://issues.apache.org/jira/browse/IGNITE-9710
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.6
>Reporter: Vyacheslav Daradur
>Assignee: Andrey Kuznetsov
>Priority: Major
> Fix For: 2.7
>
> Attachments: LongRunningCacheCreationTest.java
>
>
> Ignite watchdog service introduced by IGNITE-6587 handles long running cache
> creation.
> Action in {{GridDhtPartitionsExchangeFuture#init}} may take significant time
> and possibly should be covered by blocking section of warchdog service.
> Reproducer was attached: [^LongRunningCacheCreationTest.java].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9737) Ignite WatchDog service should be configurable
[
https://issues.apache.org/jira/browse/IGNITE-9737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632210#comment-16632210
]
Andrey Kuznetsov commented on IGNITE-9737:
--
Critical threads monitoring should be configured in a more flexible way than it
was originally suggested in issue description.
# Separate timeout setting should be introduced for {{SYSTEM_WORKER_BLOCKED}}
detection in {{IgniteConfiguration}}, falling back to
{{IgniteConfiguration.failureDetectionTimeout}} if omitted. This can be
overridden by a system property or changed in runtime by newly created
management bean. Timeout value of 0 should denote infinite timeout, thus it has
an effect of detection disabling.
# Separate timeout should be introduced for checkpoint read lock acquisition in
the same manner.
# When timed out, checkpoint read lock acquisition should not throw an
exception unless failure handler invalidated the node. This will guarantee
neutral behavior of {{NoOpFailureHandler}}.
> Ignite WatchDog service should be configurable
> --
>
> Key: IGNITE-9737
> URL: https://issues.apache.org/jira/browse/IGNITE-9737
> Project: Ignite
> Issue Type: Bug
>Reporter: Nikolay Izhikov
>Assignee: Andrey Kuznetsov
>Priority: Blocker
> Fix For: 2.7
>
>
> At the moment, there is no way to disable Ignite WatchDog service from config
> or JVM option.
> In any corner case or bug in that feature Ignite can become fully unusable
> due to unpredictable shutdown.
> We should provide a way to enable/disable this feature from config or from
> JVM option.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9739) Critical exception in transaction processing in case we have nodes out of baseline and non-persisted cache
[
https://issues.apache.org/jira/browse/IGNITE-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Kosarev updated IGNITE-9739:
---
Description:
Activation finished
{code:java}
2018-09-20 20:47:05.169 [INFO
][sys-#307%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
Successfully performed final activation steps
[nodeId=382437eb-fd8a-4f92-acd5-d9ea562c8557, client=false,
topVer=AffinityTopologyVersion [topVer=160, minorTopVer=1]]
{code}
but we have nodes not in base line
{code:java}
2018-09-20 20:45:36.116 [INFO
][sys-#305%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
Local node is not included in Baseline Topology and will not be used for
persistent data storage. Use control.(sh|bat) script or IgniteCluster interface
to include the node to Baseline Topology.
{code}
And we have cache (869481129) in the data region with persistanceEnabled=false
{code:java}
2018-09-20 20:49:01.825 [INFO
][exchange-worker-#154%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheProcessor]
Started cache [name=DPL_PUBLISHED_CACHES_REGISTRY$, *id=869481129*, group=SY
STEM_CACHEGROUP_PUBLISHED_REGISTRY, memoryPolicyName=not-persisted,
mode=PARTITIONED, atomicity=TRANSACTIONAL, backups=3]
{code}
Transaction on this cache(869481129)
{code:java}
869481129{code}
leads to critical error causing nodes by faulure handler:
{code:java}
2018-09-20 20:50:24.275
[ERROR][sys-stripe-41-#42%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
Failed processing message [senderId=62e986f0-62b5-4ec8-8cc7-27b74d345235,
msg=GridDhtTxPrepareRequest [nearNodeId=814af7c4-2de5-4511-b1ea-065b91eaa774,
futId=520e308f561-255fdea5-a996-4102-a120-afa380c54570, miniId=1,
topVer=AffinityTopologyVersion [topVer=160, minorTopVer=2],
invalidateNearEntries={}, nearWrites=null, owned=null,
nearXidVer=GridCacheVersion [topVer=148944365, order=1537511036821,
nodeOrder=132], subjId=814af7c4-2de5-4511-b1ea-065b91eaa774, taskNameHash=0,
preloadKeys=null, skipCompletedVers=false,
super=GridDistributedTxPrepareRequest [threadId=58, concurrency=PESSIMISTIC,
isolation=READ_COMMITTED, writeVer=GridCacheVersion [topVer=148944365,
order=1537511036824, nodeOrder=7], timeout=299970, reads=null, writes=ArrayList
[
IgniteTxEntry [key=KeyCacheObjectImpl [part=27254,
val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
*cacheId=869481129*,
txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=27254,
val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
*cacheId=869481129*], val=[op=CREATE,
val=com.sbt.dpl.gridgain.PublishedRegistry$PublishedCacheTuple
[idHash=811765531, hash=1522508040,
cacheName=com.sbt.gbk.entities.DocType_DPL_union-module,indexes=ArrayList
{com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
[idHash=1583970836, hash=363194492, isSoftReference=false,
unselectiveBuckets=4096, fieldNames=ArrayList
\{isDeleted},moduleName=union-module
, cachedUnselectives=1, selectors=ArrayList {isDeleted},
exceptUnselectives=false, primitiveCollection=false, isVersioned=false,
isComposite=false, isSystemTypeBelongs=false,
name=com.sbt.gbk.entities.DocType_DPL_isDeleted, isIndexedCollection=false,
isGlobal=false, maxSelective=1000],
com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
[idHash=2060926101, hash=1983794578, isSoftReference=false,
unselectiveBuckets=4096, fieldNames=ArrayList ,moduleName=union-module,
cachedUnselectives=1, selectors=ArrayList, exceptUnselectives=false,
primitiveCollection=false, isVersioned=false, isComposite=false,
isSystemTypeBelongs=false, name=com.sbt.gbk.entities.DocType_DPL_code,
isIndexedCollection=false, isGlobal=true, maxSelective=1000]
, com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
[idHash=1821682714, hash=-1245813786, isSoftReference=false,
unselectiveBuckets=4096, fieldNames=ArrayList {globalId},
moduleName=union-module, cachedUnselectives=1, selectors=ArrayList
{globalId}, exceptUnselectives=false, primitiveCollection=false,
isVersioned=false, isComposite=false, isSystemTypeBelongs=false,
name=com.sbt.gbk.entities.DocType_DPL_globalId, isIndexedCollection=false,
isGlobal=false, maxSelective=1000]
}, partitionDependencyClassName=null, moduleName=union-module,
cacheModuleName=union-module]
], prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null],
entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null,
explicitVer=null, dhtVer=null, filters=CacheEntryPredicate[] [],
filtersPassed=false, filtersSet=false, entry=GridCacheMapEntry
[key=KeyCacheObjectImpl [part=27254,
val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true], val=null,
startVer=1537511036806, ver=GridCacheVersion [topVer=148944365,
order=1537511036806, nodeOrder=86], hash=-1897857526,
extras=GridCacheMvccEntryExtras [mvcc=GridCacheMvcc [locs=null, rmts=Link
edList
[jira] [Updated] (IGNITE-9739) Critical exception in transaction processing in case we have nodes out of baseline and non-persisted cache
[
https://issues.apache.org/jira/browse/IGNITE-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Kosarev updated IGNITE-9739:
---
Description:
Activation finished
{code:java}
2018-09-20 20:47:05.169 [INFO
][sys-#307%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
Successfully performed final activation steps
[nodeId=382437eb-fd8a-4f92-acd5-d9ea562c8557, client=false,
topVer=AffinityTopologyVersion [topVer=160, minorTopVer=1]]
{code}
but we have nodes not in base line
{code:java}
2018-09-20 20:45:36.116 [INFO
][sys-#305%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
Local node is not included in Baseline Topology and will not be used for
persistent data storage. Use control.(sh|bat) script or IgniteCluster interface
to include the node to Baseline Topology.
{code}
And we have cache (869481129) in the data region with persistanceEnabled=false
{code:java}
2018-09-20 20:49:01.825 [INFO
][exchange-worker-#154%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheProcessor]
Started cache [name=DPL_PUBLISHED_CACHES_REGISTRY$, *id=869481129*, group=SY
STEM_CACHEGROUP_PUBLISHED_REGISTRY, memoryPolicyName=not-persisted,
mode=PARTITIONED, atomicity=TRANSACTIONAL, backups=3]
{code}
Transaction on this cache(869481129)
{code:java}
869481129{code}
leads to critical error causing nodes by faulure handler:
{code:java}
2018-09-20 20:50:24.275
[ERROR][sys-stripe-41-#42%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
Failed processing message [senderId=62e986f0-62b5-4ec8-8cc7-27b74d345235,
msg=GridDhtTxPrepareRequest [nearNodeId=814af7c4-2de5-4511-b1ea-065b91eaa774,
futId=520e308f561-255fdea5-a996-4102-a120-afa380c54570, miniId=1,
topVer=AffinityTopologyVersion [topVer=160, minorTopVer=2],
invalidateNearEntries={}, nearWrites=null, owned=null,
nearXidVer=GridCacheVersion [topVer=148944365, order=1537511036821,
nodeOrder=132], subjId=814af7c4-2de5-4511-b1ea-065b91eaa774, taskNameHash=0,
preloadKeys=null, skipCompletedVers=false,
super=GridDistributedTxPrepareRequest [threadId=58, concurrency=PESSIMISTIC,
isolation=READ_COMMITTED, writeVer=GridCacheVersion [topVer=148944365,
order=1537511036824, nodeOrder=7], timeout=299970, reads=null, writes=ArrayList
[
IgniteTxEntry [key=KeyCacheObjectImpl [part=27254,
val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
*cacheId=869481129*,
txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=27254,
val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
*cacheId=869481129*], val=[op=CREATE,
val=com.sbt.dpl.gridgain.PublishedRegistry$PublishedCacheTuple
[idHash=811765531, hash=1522508040,
cacheName=com.sbt.gbk.entities.DocType_DPL_union-module,indexes=ArrayList
{com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
[idHash=1583970836, hash=363194492, isSoftReference=false,
unselectiveBuckets=4096, fieldNames=ArrayList
\{isDeleted},moduleName=union-module
, cachedUnselectives=1, selectors=ArrayList {isDeleted},
exceptUnselectives=false, primitiveCollection=false, isVersioned=false,
isComposite=false, isSystemTypeBelongs=false,
name=com.sbt.gbk.entities.DocType_DPL_isDeleted, isIndexedCollection=false,
isGlobal=false, maxSelective=1000],
com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
[idHash=2060926101, hash=1983794578, isSoftReference=false,
unselectiveBuckets=4096, fieldNames=ArrayList ,moduleName=union-module,
cachedUnselectives=1, selectors=ArrayList, exceptUnselectives=false,
primitiveCollection=false, isVersioned=false, isComposite=false,
isSystemTypeBelongs=false, name=com.sbt.gbk.entities.DocType_DPL_code,
isIndexedCollection=false, isGlobal=true, maxSelective=1000]
, com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
[idHash=1821682714, hash=-1245813786, isSoftReference=false,
unselectiveBuckets=4096, fieldNames=ArrayList {globalId},
moduleName=union-module, cachedUnselectives=1, selectors=ArrayList
{globalId}, exceptUnselectives=false, primitiveCollection=false,
isVersioned=false, isComposite=false, isSystemTypeBelongs=false,
name=com.sbt.gbk.entities.DocType_DPL_globalId, isIndexedCollection=false,
isGlobal=false, maxSelective=1000]
}, partitionDependencyClassName=null, moduleName=union-module,
cacheModuleName=union-module]
], prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null],
entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null,
explicitVer=null, dhtVer=null, filters=CacheEntryPredicate[] [],
filtersPassed=false, filtersSet=false, entry=GridCacheMapEntry
[key=KeyCacheObjectImpl [part=27254,
val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true], val=null,
startVer=1537511036806, ver=GridCacheVersion [topVer=148944365,
order=1537511036806, nodeOrder=86], hash=-1897857526,
extras=GridCacheMvccEntryExtras [mvcc=GridCacheMvcc [locs=null, rmts=Link
edList
[jira] [Issue Comment Deleted] (IGNITE-7616) GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect values due to invalid interface registration.
[
https://issues.apache.org/jira/browse/IGNITE-7616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy Pavlov updated IGNITE-7616:
---
Comment: was deleted
(was: {panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}MVCC Cache{color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=1928601]]
* IgniteCacheMvccTestSuite:
CacheMvccTransactionsTest.testAccountsTxScan_WithRemoves_ClientServer_Backups1
- 0,0% fails in last 100 master runs.
{panel}
[TeamCity Run
All|http://ci.ignite.apache.org/viewLog.html?buildId=1928604buildTypeId=IgniteTests24Java8_RunAll])
> GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect
> values due to invalid interface registration.
>
>
> Key: IGNITE-7616
> URL: https://issues.apache.org/jira/browse/IGNITE-7616
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.4
>Reporter: Max Shonichev
>Assignee: David Harvey
>Priority: Major
> Labels: jmx
> Fix For: 2.7
>
> Attachments: master_1b3742f4d7_mxbeans_threads2.patch
>
>
> Two of newly added management beans as a result of implementing feature
> request https://issues.apache.org/jira/browse/IGNITE-7217 have bugs:
> # GridDataStreamExecutor is registered as conforming to ThreadPoolMXBean
> interface, though actually it is an incompatible StripedExecutor.
> # GridCallbackExecutor is registered as conforming to ThreadPoolMXBean
> interface, though actually it is an incompatible
> IgniteStripedThreadPoolExecutor.
> # ThreadPoolMXBeanAdapter checks whether adapted instance is
> ThreadPoolExecutor, and as interfaces are incompatible, most of the JMX
> attributes of GridCallbackExecutor and GridDataStreamExecutor are returned as
> -1 or null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-9658) Add ability to disable memory deallocation on deactivation.
[ https://issues.apache.org/jira/browse/IGNITE-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632127#comment-16632127 ] Alexei Scherbakov edited comment on IGNITE-9658 at 9/28/18 5:11 PM: [~ivandasch], Your implementation has memory leak issues. I've made fixes in separate branch ignite-9658-2, PR [1], TC [2] If you don't mind I'll finish the ticket. [1] https://github.com/apache/ignite/pull/4874 [2] https://ci.ignite.apache.org/viewQueued.html?itemId=1963633=queuedBuildOverviewTab was (Author: ascherbakov): [~ivandasch], Your implementation has memory leak issues. I've made fixes in separate branch ignite-9658-2, PR [1], TC [2] If you don't mind I'll finish the ticket. [1] https://github.com/apache/ignite/pull/4874 [2] https://ci.ignite.apache.org/viewLog.html?buildId=1963526=buildResultsDiv=IgniteTests24Java8_RunAll > Add ability to disable memory deallocation on deactivation. > --- > > Key: IGNITE-9658 > URL: https://issues.apache.org/jira/browse/IGNITE-9658 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Daschinskiy >Assignee: Alexei Scherbakov >Priority: Major > Fix For: 2.8 > > > Currently, in some systems (i.e. RHEL 7.4), we can see, that during massive > UNSAFE.freeMemory process freezes. This behaviour can lead to SEGMENTATION of > node, especcially when ZookeeperDiscoverySPI is used. There should be an > abillity to disable memory deallocation during deactivation of cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-7616) GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect values due to invalid interface registration.
[
https://issues.apache.org/jira/browse/IGNITE-7616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632143#comment-16632143
]
Ignite TC Bot commented on IGNITE-7616:
---
{panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}MVCC Cache{color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=1928601]]
* IgniteCacheMvccTestSuite:
CacheMvccTransactionsTest.testAccountsTxScan_WithRemoves_ClientServer_Backups1
- 0,0% fails in last 100 master runs.
{panel}
[TeamCity Run
All|http://ci.ignite.apache.org/viewLog.html?buildId=1928604buildTypeId=IgniteTests24Java8_RunAll]
> GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect
> values due to invalid interface registration.
>
>
> Key: IGNITE-7616
> URL: https://issues.apache.org/jira/browse/IGNITE-7616
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.4
>Reporter: Max Shonichev
>Assignee: David Harvey
>Priority: Major
> Labels: jmx
> Fix For: 2.7
>
> Attachments: master_1b3742f4d7_mxbeans_threads2.patch
>
>
> Two of newly added management beans as a result of implementing feature
> request https://issues.apache.org/jira/browse/IGNITE-7217 have bugs:
> # GridDataStreamExecutor is registered as conforming to ThreadPoolMXBean
> interface, though actually it is an incompatible StripedExecutor.
> # GridCallbackExecutor is registered as conforming to ThreadPoolMXBean
> interface, though actually it is an incompatible
> IgniteStripedThreadPoolExecutor.
> # ThreadPoolMXBeanAdapter checks whether adapted instance is
> ThreadPoolExecutor, and as interfaces are incompatible, most of the JMX
> attributes of GridCallbackExecutor and GridDataStreamExecutor are returned as
> -1 or null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-7616) GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect values due to invalid interface registration.
[
https://issues.apache.org/jira/browse/IGNITE-7616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632141#comment-16632141
]
Ignite TC Bot commented on IGNITE-7616:
---
{panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}MVCC Cache{color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=1928601]]
* IgniteCacheMvccTestSuite:
CacheMvccTransactionsTest.testAccountsTxScan_WithRemoves_ClientServer_Backups1
- 0,0% fails in last 100 master runs.
{panel}
[TeamCity Run
All|http://ci.ignite.apache.org/viewLog.html?buildId=1928604buildTypeId=IgniteTests24Java8_RunAll]
> GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect
> values due to invalid interface registration.
>
>
> Key: IGNITE-7616
> URL: https://issues.apache.org/jira/browse/IGNITE-7616
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.4
>Reporter: Max Shonichev
>Assignee: David Harvey
>Priority: Major
> Labels: jmx
> Fix For: 2.7
>
> Attachments: master_1b3742f4d7_mxbeans_threads2.patch
>
>
> Two of newly added management beans as a result of implementing feature
> request https://issues.apache.org/jira/browse/IGNITE-7217 have bugs:
> # GridDataStreamExecutor is registered as conforming to ThreadPoolMXBean
> interface, though actually it is an incompatible StripedExecutor.
> # GridCallbackExecutor is registered as conforming to ThreadPoolMXBean
> interface, though actually it is an incompatible
> IgniteStripedThreadPoolExecutor.
> # ThreadPoolMXBeanAdapter checks whether adapted instance is
> ThreadPoolExecutor, and as interfaces are incompatible, most of the JMX
> attributes of GridCallbackExecutor and GridDataStreamExecutor are returned as
> -1 or null.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-9658) Add ability to disable memory deallocation on deactivation.
[ https://issues.apache.org/jira/browse/IGNITE-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632127#comment-16632127 ] Alexei Scherbakov edited comment on IGNITE-9658 at 9/28/18 5:01 PM: [~ivandasch], Your implementation has memory leak issues. I've made fixes in separate branch ignite-9658-2, PR [1], TC [2] If you don't mind I'll finish the ticket. [1] https://github.com/apache/ignite/pull/4874 [2] https://ci.ignite.apache.org/viewLog.html?buildId=1963526=buildResultsDiv=IgniteTests24Java8_RunAll was (Author: ascherbakov): [~ivandasch], Your implementation has memory leak issues. I've made fixes in separate branch ignite-9658-2, PR [1], TC [2] If you don't mind I'll finish the ticket. [1] https://github.com/apache/ignite/pull/4874 [2] https://ci.ignite.apache.org/viewQueued.html?itemId=1963202=queuedBuildOverviewTab > Add ability to disable memory deallocation on deactivation. > --- > > Key: IGNITE-9658 > URL: https://issues.apache.org/jira/browse/IGNITE-9658 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Daschinskiy >Assignee: Alexei Scherbakov >Priority: Major > Fix For: 2.8 > > > Currently, in some systems (i.e. RHEL 7.4), we can see, that during massive > UNSAFE.freeMemory process freezes. This behaviour can lead to SEGMENTATION of > node, especcially when ZookeeperDiscoverySPI is used. There should be an > abillity to disable memory deallocation during deactivation of cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9658) Add ability to disable memory deallocation on deactivation.
[ https://issues.apache.org/jira/browse/IGNITE-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632127#comment-16632127 ] Alexei Scherbakov commented on IGNITE-9658: --- [~ivandasch], Your implementation has memory leak issues. I've made fixes in separate branch ignite-9658-2, PR [1], TC [2] If you don't mind I'll finish the ticket. [1] https://github.com/apache/ignite/pull/4874 [2] https://ci.ignite.apache.org/viewQueued.html?itemId=1963202=queuedBuildOverviewTab > Add ability to disable memory deallocation on deactivation. > --- > > Key: IGNITE-9658 > URL: https://issues.apache.org/jira/browse/IGNITE-9658 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Daschinskiy >Assignee: Alexei Scherbakov >Priority: Major > Fix For: 2.8 > > > Currently, in some systems (i.e. RHEL 7.4), we can see, that during massive > UNSAFE.freeMemory process freezes. This behaviour can lead to SEGMENTATION of > node, especcially when ZookeeperDiscoverySPI is used. There should be an > abillity to disable memory deallocation during deactivation of cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9451) Consistent Cache.size for cache API methods over MVCC tables
[ https://issues.apache.org/jira/browse/IGNITE-9451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632101#comment-16632101 ] ASF GitHub Bot commented on IGNITE-9451: Github user pavlukhin closed the pull request at: https://github.com/apache/ignite/pull/4818 > Consistent Cache.size for cache API methods over MVCC tables > > > Key: IGNITE-9451 > URL: https://issues.apache.org/jira/browse/IGNITE-9451 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Ivan Pavlukhin >Assignee: Ivan Pavlukhin >Priority: Major > > In scope of supporting Cache API for MVCC caches we should ensure that > Cache.size is updated consistently and only on transaction commit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-5935) MVCC TX: Tx recovery protocol
[ https://issues.apache.org/jira/browse/IGNITE-5935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632098#comment-16632098 ] ASF GitHub Bot commented on IGNITE-5935: GitHub user pavlukhin opened a pull request: https://github.com/apache/ignite/pull/4873 IGNITE-5935: MVCC transaction recovery You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-5935 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4873.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4873 commit 77dc283027793bee8831fe75c8947a825f81a539 Author: ipavlukhin Date: 2018-09-28T14:50:59Z accumulate backups participating in transaction in GridDistributedTxMapping, retrieve them from received enlist response commit 781dab6e0a2068acaaf859f4848f35c52327cc4f Author: ipavlukhin Date: 2018-09-28T15:14:11Z GridDistributedTxMapping normalize null backups to empty list commit 090d17e40e56f45258a5a3f27c4da6e065343ed9 Author: ipavlukhin Date: 2018-09-28T15:32:01Z pass backups to transaction mappings upon receiving response in GridNearTxEnlistFuture commit 0d2446b27da80944d203cfd3c8a218a6904daa1c Author: ipavlukhin Date: 2018-09-28T15:36:28Z putIfAbsent -> put commit 8d3eb13b40ba1bcd3fa70fa6af4c7af5ed521454 Author: ipavlukhin Date: 2018-09-28T15:48:16Z remove outdated todo commit 36a22a8b0ffc487dd47ab430057d185afdb81da9 Author: ipavlukhin Date: 2018-09-28T16:05:52Z make backups in GridDistributedTxMapping thread-safe commit 3ff50a1d2553d249f30858b478fc89986f344f8e Author: ipavlukhin Date: 2018-09-28T16:07:41Z add draft of a test > MVCC TX: Tx recovery protocol > - > > Key: IGNITE-5935 > URL: https://issues.apache.org/jira/browse/IGNITE-5935 > Project: Ignite > Issue Type: Task > Components: cache, mvcc >Reporter: Semen Boikov >Assignee: Ivan Pavlukhin >Priority: Major > Fix For: 2.7 > > > Transaction recovery procedure is initiated when near node failed before > transaction was finished. > In MVCC transactions _partition update counter_ modification is started on > prepare phase. If a transaction was prepared at least on one node we need to > finish _partition update counter_ modification consistently on all > participating nodes. > Also recovered transaction should be removed from active transactions list on > mvcc coordinator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-6343) Index is not used properly if changing sort order.
[
https://issues.apache.org/jira/browse/IGNITE-6343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632078#comment-16632078
]
ASF GitHub Bot commented on IGNITE-6343:
Github user rkondakov closed the pull request at:
https://github.com/apache/ignite/pull/3212
> Index is not used properly if changing sort order.
> --
>
> Key: IGNITE-6343
> URL: https://issues.apache.org/jira/browse/IGNITE-6343
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.0
>Reporter: Alexei Scherbakov
>Priority: Major
> Labels: performance
>
> Unit test reproducer:
> {noformat}
> /*
> * Licensed to the Apache Software Foundation (ASF) under one or more
> * contributor license agreements. See the NOTICE file distributed with
> * this work for additional information regarding copyright ownership.
> * The ASF licenses this file to You under the Apache License, Version 2.0
> * (the "License"); you may not use this file except in compliance with
> * the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> package org.apache.ignite.internal.processors.cache;
> import java.util.Calendar;
> import java.util.Collections;
> import java.util.Date;
> import java.util.LinkedHashMap;
> import java.util.List;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.cache.CacheMode;
> import org.apache.ignite.cache.QueryEntity;
> import org.apache.ignite.cache.QueryIndex;
> import org.apache.ignite.cache.QueryIndexType;
> import org.apache.ignite.cache.query.SqlFieldsQuery;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.configuration.MemoryConfiguration;
> import org.apache.ignite.configuration.MemoryPolicyConfiguration;
> import org.apache.ignite.internal.util.typedef.internal.U;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.TcpDiscoveryIpFinder;
> import org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import org.apache.ignite.testframework.junits.common.GridCommonAbstractTest;
> import static org.apache.ignite.cache.CacheMode.PARTITIONED;
> import static org.apache.ignite.cache.CacheWriteSynchronizationMode.FULL_SYNC;
> import static java.util.Calendar.*;
> /**
> * Tests for cache query results serialization.
> */
> public class GridCacheQueryIndexUsageSelfTest extends GridCommonAbstractTest {
> /** */
> private static final int GRID_CNT = 1;
> /** */
> private static final String CACHE_NAME = "A";
> /** */
> private static final CacheMode CACHE_MODE = PARTITIONED;
> /** */
> private static TcpDiscoveryIpFinder ipFinder = new
> TcpDiscoveryVmIpFinder(true);
> /** {@inheritDoc} */
> @Override protected void beforeTest() throws Exception {
> startGridsMultiThreaded(GRID_CNT);
> }
> /** {@inheritDoc} */
> @Override protected void afterTest() throws Exception {
> stopAllGrids();
> }
> /** {@inheritDoc} */
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> MemoryPolicyConfiguration mpcfg = new MemoryPolicyConfiguration();
> //mpcfg.setMaxSize(2 * 1024 * 1024 * 1024L);
> mpcfg.setName("def");
> MemoryConfiguration mcfg = new MemoryConfiguration();
> mcfg.setDefaultMemoryPolicyName("def");
> mcfg.setMemoryPolicies(mpcfg);
> cfg.setMemoryConfiguration(mcfg);
> TcpDiscoverySpi disco = new TcpDiscoverySpi();
> disco.setIpFinder(ipFinder);
> cfg.setDiscoverySpi(disco);
> CacheConfiguration cacheCfg = defaultCacheConfiguration();
> cacheCfg.setName(CACHE_NAME);
> cacheCfg.setCacheMode(CACHE_MODE);
> cacheCfg.setWriteSynchronizationMode(FULL_SYNC);
> QueryEntity qe = new QueryEntity();
> qe.setKeyType(Long.class.getName());
> qe.setValueType(IndexedValue.class.getName());
> LinkedHashMap fields = U.newLinkedHashMap(3);
> fields.put("id", Long.class.getName());
> fields.put("startDate", Date.class.getName());
> qe.setFields(fields);
> QueryIndex idx = new QueryIndex();
>
[jira] [Commented] (IGNITE-1436) C++: Port to MAC OS.
[ https://issues.apache.org/jira/browse/IGNITE-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632076#comment-16632076 ] ASF GitHub Bot commented on IGNITE-1436: GitHub user sdarlington opened a pull request: https://github.com/apache/ignite/pull/4872 IGNITE-1436: Build C++ API on the Mac Creating PR to 'advertise' that it's being worked on. Tasks still to complete: * Test on Linux / Windows to make sure I've not broken anything * Final step in build: need to `install_name_tool -add_rpath $(JAVA_HOME)/jre/lib/server ignite` but I've not found a way to automate that * Tests, ODBC driver and thin-client don't currently build You can merge this pull request into a Git repository by running: $ git pull https://github.com/sdarlington/ignite cpp-mac Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4872.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4872 commit 46a923720f64c8e920ff53d807f9c42d2b9270ef Author: Stephen Darlington Date: 2018-09-25T12:52:30Z Get past missing CLOCK_MONOTONIC on Macs error commit 2707b263f27fd708b5525df0eb3b4398203f7549 Author: Stephen Darlington Date: 2018-09-25T14:08:47Z Set right headers to allow Mac compile of C++ client commit 0ee352f012ef7a802b52a1e9bfba7fb96ba3a335 Author: Stephen Darlington Date: 2018-09-25T14:34:53Z Mac dynamic libraries are not '.so' files commit 249588f2525c7c5b7588f01f02db876264d6df0c Author: Stephen Darlington Date: 2018-09-28T16:03:06Z Update DEVNOTES > C++: Port to MAC OS. > > > Key: IGNITE-1436 > URL: https://issues.apache.org/jira/browse/IGNITE-1436 > Project: Ignite > Issue Type: Task > Components: platforms >Affects Versions: 1.1.4 >Reporter: Vladimir Ozerov >Priority: Major > Labels: cpp > > It will require minimal porting of "common" and "utils" stuff. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9638) .NET: JVM keeps track of CLR Threads, even when they are finished
[
https://issues.apache.org/jira/browse/IGNITE-9638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632062#comment-16632062
]
Pavel Tupitsyn commented on IGNITE-9638:
This is caused by {{AttachCurrentThread}} JNI call without matching
{{DetachCurrentThread}}.
Detaching JVM thread on every call will severely degrade the performance.
We have a number of options:
* Provide a user-facing API to manually detach the thread (really ugly from
user perspective)
* Use platform-specific ways to detect thread exit (DLL_THREAD_DETACH on
Windows, pthread_key_create on Linux) - this will likely require compiling
unmanaged DLLs for every platform (ugly from our perspective, less
cross-platform)
* Find a way to detach a thread that has exited: this way we can rely on GC,
ThreadLocals and finalizers to detect a thread exit
> .NET: JVM keeps track of CLR Threads, even when they are finished
> --
>
> Key: IGNITE-9638
> URL: https://issues.apache.org/jira/browse/IGNITE-9638
> Project: Ignite
> Issue Type: Bug
> Components: platforms
>Affects Versions: 2.6
>Reporter: Ilya Kasnacheev
>Assignee: Pavel Tupitsyn
>Priority: Major
> Labels: .NET
> Fix For: 2.8
>
> Attachments: IgniteRepro.zip
>
>
> When you create a Thread in C#, JVM creates corresponding thread
> "Thread-" which is visible in jstack. When C# joins this thread, it is
> not removed from JVM and is kept around. This means that jstack may show
> thousands of threads which are not there. Reproducer is attached. It is
> presumed that memory will be exhausted eventually.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9612) Improve checkpoint mark phase speed.
[
https://issues.apache.org/jira/browse/IGNITE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632056#comment-16632056
]
ASF GitHub Bot commented on IGNITE-9612:
Github user ascherbakoff closed the pull request at:
https://github.com/apache/ignite/pull/4864
> Improve checkpoint mark phase speed.
>
>
> Key: IGNITE-9612
> URL: https://issues.apache.org/jira/browse/IGNITE-9612
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>Priority: Major
> Fix For: 2.7
>
>
> I'm observing regular slow checkpoints due to long mark duration, which is
> not related to dirty pages number:
> {noformat}
> 2018-09-01 14:55:20.408 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint started [checkpointId=01e0c7bf-842f-4ed6-8589-b4904063434f,
> startPtr=FileWALPointer [idx=19814, fileOff=948996096, len=5233457],
> checkpointLockWait=0ms, checkpointLockHoldTime=951ms,
> walCpRecordFsyncDuration=39ms, pages=78477, reason='timeout']
> 2018-09-01 14:55:21.307 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint finished [cpId=01e0c7bf-842f-4ed6-8589-b4904063434f, pages=78477,
> markPos=FileWALPointer [idx=19814, fileOff=948996096, len=5233457],
> walSegmentsCleared=0, walSegmentsCovered=[], *markDuration=1002m*s,
> pagesWrite=478ms, fsync=421ms, total=1901ms]
> {noformat}
> {noformat}
> 2018-09-01 14:58:20.355 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint started [checkpointId=09d1f4bc-d3f3-4a16-b291-89d7fa745ea5,
> startPtr=FileWALPointer [idx=19814, fileOff=124208, len=5233457],
> checkpointLockWait=0ms, checkpointLockHoldTime=926ms,
> walCpRecordFsyncDuration=14ms, pages=10837, reason='timeout']
> 2018-09-01 14:58:20.480 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint finished [cpId=09d1f4bc-d3f3-4a16-b291-89d7fa745ea5, pages=10837,
> markPos=FileWALPointer [idx=19814, fileOff=124208, len=5233457],
> walSegmentsCleared=0, walSegmentsCovered=[], *markDuration=943ms*,
> pagesWrite=64ms, fsync=61ms, total=1068ms]
> {noformat}
> Debugging has revealed what this is due to large amount of work required to
> save metadata for metapages and free/reuse lists. Because this is done under
> checkpoint write lock, all other activities are blocked, resulting in
> increased tx and atomic ops latency.
> Simple solution: parallelize metadata processing during mark phase.
> Best way to solve the problem is described in IGNITE-9520.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9612) Improve checkpoint mark phase speed.
[
https://issues.apache.org/jira/browse/IGNITE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632055#comment-16632055
]
Alexei Scherbakov commented on IGNITE-9612:
---
Another issue is related to snapshots.
Fixed in commit 2f3b567f61c2a9a1080ee7fda57f3a18231b1dae
> Improve checkpoint mark phase speed.
>
>
> Key: IGNITE-9612
> URL: https://issues.apache.org/jira/browse/IGNITE-9612
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>Priority: Major
> Fix For: 2.7
>
>
> I'm observing regular slow checkpoints due to long mark duration, which is
> not related to dirty pages number:
> {noformat}
> 2018-09-01 14:55:20.408 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint started [checkpointId=01e0c7bf-842f-4ed6-8589-b4904063434f,
> startPtr=FileWALPointer [idx=19814, fileOff=948996096, len=5233457],
> checkpointLockWait=0ms, checkpointLockHoldTime=951ms,
> walCpRecordFsyncDuration=39ms, pages=78477, reason='timeout']
> 2018-09-01 14:55:21.307 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint finished [cpId=01e0c7bf-842f-4ed6-8589-b4904063434f, pages=78477,
> markPos=FileWALPointer [idx=19814, fileOff=948996096, len=5233457],
> walSegmentsCleared=0, walSegmentsCovered=[], *markDuration=1002m*s,
> pagesWrite=478ms, fsync=421ms, total=1901ms]
> {noformat}
> {noformat}
> 2018-09-01 14:58:20.355 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint started [checkpointId=09d1f4bc-d3f3-4a16-b291-89d7fa745ea5,
> startPtr=FileWALPointer [idx=19814, fileOff=124208, len=5233457],
> checkpointLockWait=0ms, checkpointLockHoldTime=926ms,
> walCpRecordFsyncDuration=14ms, pages=10837, reason='timeout']
> 2018-09-01 14:58:20.480 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint finished [cpId=09d1f4bc-d3f3-4a16-b291-89d7fa745ea5, pages=10837,
> markPos=FileWALPointer [idx=19814, fileOff=124208, len=5233457],
> walSegmentsCleared=0, walSegmentsCovered=[], *markDuration=943ms*,
> pagesWrite=64ms, fsync=61ms, total=1068ms]
> {noformat}
> Debugging has revealed what this is due to large amount of work required to
> save metadata for metapages and free/reuse lists. Because this is done under
> checkpoint write lock, all other activities are blocked, resulting in
> increased tx and atomic ops latency.
> Simple solution: parallelize metadata processing during mark phase.
> Best way to solve the problem is described in IGNITE-9520.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9612) Improve checkpoint mark phase speed.
[
https://issues.apache.org/jira/browse/IGNITE-9612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632054#comment-16632054
]
Alexey Goncharuk commented on IGNITE-9612:
--
Another fix with checkpoint listener logic is merged to master and ignite-2.7
branch:
2f3b567f61c2a9a1080ee7fda57f3a18231b1dae
> Improve checkpoint mark phase speed.
>
>
> Key: IGNITE-9612
> URL: https://issues.apache.org/jira/browse/IGNITE-9612
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexei Scherbakov
>Assignee: Alexei Scherbakov
>Priority: Major
> Fix For: 2.7
>
>
> I'm observing regular slow checkpoints due to long mark duration, which is
> not related to dirty pages number:
> {noformat}
> 2018-09-01 14:55:20.408 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint started [checkpointId=01e0c7bf-842f-4ed6-8589-b4904063434f,
> startPtr=FileWALPointer [idx=19814, fileOff=948996096, len=5233457],
> checkpointLockWait=0ms, checkpointLockHoldTime=951ms,
> walCpRecordFsyncDuration=39ms, pages=78477, reason='timeout']
> 2018-09-01 14:55:21.307 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint finished [cpId=01e0c7bf-842f-4ed6-8589-b4904063434f, pages=78477,
> markPos=FileWALPointer [idx=19814, fileOff=948996096, len=5233457],
> walSegmentsCleared=0, walSegmentsCovered=[], *markDuration=1002m*s,
> pagesWrite=478ms, fsync=421ms, total=1901ms]
> {noformat}
> {noformat}
> 2018-09-01 14:58:20.355 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint started [checkpointId=09d1f4bc-d3f3-4a16-b291-89d7fa745ea5,
> startPtr=FileWALPointer [idx=19814, fileOff=124208, len=5233457],
> checkpointLockWait=0ms, checkpointLockHoldTime=926ms,
> walCpRecordFsyncDuration=14ms, pages=10837, reason='timeout']
> 2018-09-01 14:58:20.480 [INFO
> ][db-checkpoint-thread-#241%DPL_GRID%DplGridNodeName%][o.a.i.i.p.c.p.GridCacheDatabaseSharedManager]
> Checkpoint finished [cpId=09d1f4bc-d3f3-4a16-b291-89d7fa745ea5, pages=10837,
> markPos=FileWALPointer [idx=19814, fileOff=124208, len=5233457],
> walSegmentsCleared=0, walSegmentsCovered=[], *markDuration=943ms*,
> pagesWrite=64ms, fsync=61ms, total=1068ms]
> {noformat}
> Debugging has revealed what this is due to large amount of work required to
> save metadata for metapages and free/reuse lists. Because this is done under
> checkpoint write lock, all other activities are blocked, resulting in
> increased tx and atomic ops latency.
> Simple solution: parallelize metadata processing during mark phase.
> Best way to solve the problem is described in IGNITE-9520.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-9739) Critical exception in transaction processing in case we have nodes out of baseline and non-persisted cache
Sergey Kosarev created IGNITE-9739:
--
Summary: Critical exception in transaction processing in case we
have nodes out of baseline and non-persisted cache
Key: IGNITE-9739
URL: https://issues.apache.org/jira/browse/IGNITE-9739
Project: Ignite
Issue Type: Bug
Reporter: Sergey Kosarev
Activation finished
{code}
2018-09-20 20:47:05.169 [INFO
][sys-#307%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
Successfully performed final activation steps
[nodeId=382437eb-fd8a-4f92-acd5-d9ea562c8557, client=false,
topVer=AffinityTopologyVersion [topVer=160, minorTopVer=1]]
{code}
but we have nodes not in base line
{code}
2018-09-20 20:45:36.116 [INFO
][sys-#305%DPL_GRID%DplGridNodeName%][o.g.g.i.p.c.d.GridSnapshotAwareClusterStateProcessorImpl]
Local node is not included in Baseline Topology and will not be used for
persistent data storage. Use control.(sh|bat) script or IgniteCluster interface
to include the node to Baseline Topology.
{code}
And we have cache in the data region with persistanceEnabled=false
{code}
2018-09-20 20:49:01.825 [INFO
][exchange-worker-#154%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheProcessor]
Started cache [name=DPL_PUBLISHED_CACHES_REGISTRY$, *id=869481129*, group=SY
STEM_CACHEGROUP_PUBLISHED_REGISTRY, memoryPolicyName=not-persisted,
mode=PARTITIONED, atomicity=TRANSACTIONAL, backups=3]
{code}
Transaction on this cache leads to critical error causing nodes by faulure
handler:
{code}
2018-09-20 20:50:24.275
[ERROR][sys-stripe-41-#42%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
Failed processing message [senderId=62e986f0-62b5-4ec8-8cc7-27b74d345235,
msg=GridDhtTxPrepareRequest [nearNodeId=814af7c4-2de5-4511-b1ea-065b91eaa774,
futId=520e308f561-255fdea5-a996-4102-a120-afa380c54570, miniId=1,
topVer=AffinityTopologyVersion [topVer=160, minorTopVer=2],
invalidateNearEntries={}, nearWrites=null, owned=null,
nearXidVer=GridCacheVersion [topVer=148944365, order=1537511036821,
nodeOrder=132], subjId=814af7c4-2de5-4511-b1ea-065b91eaa774, taskNameHash=0,
preloadKeys=null, skipCompletedVers=false,
super=GridDistributedTxPrepareRequest [threadId=58, concurrency=PESSIMISTIC,
isolation=READ_COMMITTED, writeVer=GridCacheVersion [topVer=148944365,
order=1537511036824, nodeOrder=7], timeout=299970, reads=null, writes=ArrayList
[
IgniteTxEntry [key=KeyCacheObjectImpl [part=27254,
val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
*cacheId=869481129*,
txKey=IgniteTxKey [key=KeyCacheObjectImpl [part=27254,
val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true],
*cacheId=869481129*], val=[op=CREATE,
val=com.sbt.dpl.gridgain.PublishedRegistry$PublishedCacheTuple
[idHash=811765531, hash=1522508040,
cacheName=com.sbt.gbk.entities.DocType_DPL_union-module,indexes=ArrayList
{com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
[idHash=1583970836, hash=363194492, isSoftReference=false,
unselectiveBuckets=4096, fieldNames=ArrayList {isDeleted},
moduleName=union-module
, cachedUnselectives=1, selectors=ArrayList {isDeleted},
exceptUnselectives=false, primitiveCollection=false, isVersioned=false,
isComposite=false, isSystemTypeBelongs=false,
name=com.sbt.gbk.entities.DocType_DPL_isDeleted, isIndexedCollection=false,
isGlobal=false, maxSelective=1000],
com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
[idHash=2060926101, hash=1983794578, isSoftReference=false,
unselectiveBuckets=4096, fieldNames=ArrayList {code},moduleName=union-module,
cachedUnselectives=1, selectors=ArrayList {code}, exceptUnselectives=false,
primitiveCollection=false, isVersioned=false, isComposite=false,
isSystemTypeBelongs=false, name=com.sbt.gbk.entities.DocType_DPL_code,
isIndexedCollection=false, isGlobal=true, maxSelective=1000]
, com.sbt.dpl.gridgain.newModel.base.indexes.PublishedIndexType
[idHash=1821682714, hash=-1245813786, isSoftReference=false,
unselectiveBuckets=4096, fieldNames=ArrayList {globalId},
moduleName=union-module, cachedUnselectives=1, selectors=ArrayList
{globalId}, exceptUnselectives=false, primitiveCollection=false,
isVersioned=false, isComposite=false, isSystemTypeBelongs=false,
name=com.sbt.gbk.entities.DocType_DPL_globalId, isIndexedCollection=false,
isGlobal=false, maxSelective=1000]
}, partitionDependencyClassName=null, moduleName=union-module,
cacheModuleName=union-module]
], prevVal=[op=NOOP, val=null], oldVal=[op=NOOP, val=null],
entryProcessorsCol=null, ttl=-1, conflictExpireTime=-1, conflictVer=null,
explicitVer=null, dhtVer=null, filters=CacheEntryPredicate[] [],
filtersPassed=false, filtersSet=false, entry=GridCacheMapEntry
[key=KeyCacheObjectImpl [part=27254,
val=com.sbt.api.entities.out.IPublishedDocType, hasValBytes=true], val=null,
startVer=1537511036806, ver=GridCacheVersion [topVer=148944365,
[jira] [Commented] (IGNITE-9727) JDK9/10/11: ignite run script must be works with JDK9/10/11
[
https://issues.apache.org/jira/browse/IGNITE-9727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632045#comment-16632045
]
ASF GitHub Bot commented on IGNITE-9727:
GitHub user tledkov-gridgain opened a pull request:
https://github.com/apache/ignite/pull/4871
IGNITE-9727: fix ignite.sh & ignite.bat start scripts for JDK 9/10/11
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-9727
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/4871.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4871
commit 79d170c61bb1614d92b47a42dba8d539a4cc85a2
Author: tledkov-gridgain
Date: 2018-09-28T12:31:16Z
IGNITE-9727: fix ignite.sh & ignite.bat start scripts for JDK 9/10/11
> JDK9/10/11: ignite run script must be works with JDK9/10/11
> ---
>
> Key: IGNITE-9727
> URL: https://issues.apache.org/jira/browse/IGNITE-9727
> Project: Ignite
> Issue Type: Task
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Fix For: 2.7
>
>
> {{ignite.sh/bat}} and other scripts must be works properly with JDK 9/10/11
> versions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9723) Critial section should be wrapped correctly for GridCacheDatabaseSharedManager
[
https://issues.apache.org/jira/browse/IGNITE-9723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-9723:
Fix Version/s: 2.7
> Critial section should be wrapped correctly for GridCacheDatabaseSharedManager
> --
>
> Key: IGNITE-9723
> URL: https://issues.apache.org/jira/browse/IGNITE-9723
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.6
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.7
>
>
> `blockingSectionBegin` should be used here
> {code:java|title=GridCachePartitionExchangeManager:2687}
> blockingSectionEnd();
> try {
> resVer = exchFut.get(exchTimeout, TimeUnit.MILLISECONDS);
> } finally {
> blockingSectionEnd();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9723) Critial section should be wrapped correctly for GridCacheDatabaseSharedManager
[
https://issues.apache.org/jira/browse/IGNITE-9723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632024#comment-16632024
]
Ignite TC Bot commented on IGNITE-9723:
---
{panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}PDS 2{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=1961438]]
{color:#d04437}Cache 7 (With Persistence){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=1961466]]
* IgniteCacheTestSuite7:
TransactionIntegrityWithPrimaryIndexCorruptionTest.testPrimaryIndexCorruptionDuringCommitOnPrimaryNode2
- 0,0% fails in last 100 master runs.
{panel}
[TeamCity Run
All|http://ci.ignite.apache.org/viewLog.html?buildId=1961481buildTypeId=IgniteTests24Java8_RunAll]
> Critial section should be wrapped correctly for GridCacheDatabaseSharedManager
> --
>
> Key: IGNITE-9723
> URL: https://issues.apache.org/jira/browse/IGNITE-9723
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.6
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
>
> `blockingSectionBegin` should be used here
> {code:java|title=GridCachePartitionExchangeManager:2687}
> blockingSectionEnd();
> try {
> resVer = exchFut.get(exchTimeout, TimeUnit.MILLISECONDS);
> } finally {
> blockingSectionEnd();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9731) NPE is possible during WAL flushing
[
https://issues.apache.org/jira/browse/IGNITE-9731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632012#comment-16632012
]
ASF GitHub Bot commented on IGNITE-9731:
Github user asfgit closed the pull request at:
https://github.com/apache/ignite/pull/4863
> NPE is possible during WAL flushing
> ---
>
> Key: IGNITE-9731
> URL: https://issues.apache.org/jira/browse/IGNITE-9731
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Kuznetsov
>Assignee: Anton Kalashnikov
>Priority: Critical
> Fix For: 2.7
>
> Attachments: WalRolloverRecordLoggingTest.java
>
>
> {{FileWriteAheadLogManager.flush()}} seems to be not thread-safe anymore in
> master branch. The test attached produces the following NPE:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileHandle.getSegmentId(FileWriteAheadLogManager.java:2371)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.needFsync(FileWriteAheadLogManager.java:2642)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.fsync(FileWriteAheadLogManager.java:2668)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.access$1900(FileWriteAheadLogManager.java:2445)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.flush(FileWriteAheadLogManager.java:866)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:3633)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:3126)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:3025)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> This could be possibly brought by commit [1].
> [1]
> https://github.com/apache/ignite/commit/2f72fe758d4256c4eb4610e5922ad3d174b43dc5
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9731) NPE is possible during WAL flushing
[
https://issues.apache.org/jira/browse/IGNITE-9731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632008#comment-16632008
]
Alexey Goncharuk commented on IGNITE-9731:
--
Re-checked, licenses were broken in master. Good to merge.
> NPE is possible during WAL flushing
> ---
>
> Key: IGNITE-9731
> URL: https://issues.apache.org/jira/browse/IGNITE-9731
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Kuznetsov
>Assignee: Anton Kalashnikov
>Priority: Critical
> Fix For: 2.7
>
> Attachments: WalRolloverRecordLoggingTest.java
>
>
> {{FileWriteAheadLogManager.flush()}} seems to be not thread-safe anymore in
> master branch. The test attached produces the following NPE:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileHandle.getSegmentId(FileWriteAheadLogManager.java:2371)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.needFsync(FileWriteAheadLogManager.java:2642)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.fsync(FileWriteAheadLogManager.java:2668)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.access$1900(FileWriteAheadLogManager.java:2445)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.flush(FileWriteAheadLogManager.java:866)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:3633)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:3126)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:3025)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> This could be possibly brought by commit [1].
> [1]
> https://github.com/apache/ignite/commit/2f72fe758d4256c4eb4610e5922ad3d174b43dc5
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9731) NPE is possible during WAL flushing
[
https://issues.apache.org/jira/browse/IGNITE-9731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632007#comment-16632007
]
Ignite TC Bot commented on IGNITE-9731:
---
{panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}_Licenses Headers_{color} [[tests 0 Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=1961144]]
{panel}
[TeamCity Run
All|http://ci.ignite.apache.org/viewLog.html?buildId=1960190buildTypeId=IgniteTests24Java8_RunAll]
> NPE is possible during WAL flushing
> ---
>
> Key: IGNITE-9731
> URL: https://issues.apache.org/jira/browse/IGNITE-9731
> Project: Ignite
> Issue Type: Bug
>Reporter: Andrey Kuznetsov
>Assignee: Anton Kalashnikov
>Priority: Critical
> Fix For: 2.7
>
> Attachments: WalRolloverRecordLoggingTest.java
>
>
> {{FileWriteAheadLogManager.flush()}} seems to be not thread-safe anymore in
> master branch. The test attached produces the following NPE:
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileHandle.getSegmentId(FileWriteAheadLogManager.java:2371)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.needFsync(FileWriteAheadLogManager.java:2642)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.fsync(FileWriteAheadLogManager.java:2668)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileWriteHandle.access$1900(FileWriteAheadLogManager.java:2445)
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager.flush(FileWriteAheadLogManager.java:866)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:3633)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:3126)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:3025)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> This could be possibly brought by commit [1].
> [1]
> https://github.com/apache/ignite/commit/2f72fe758d4256c4eb4610e5922ad3d174b43dc5
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
[
https://issues.apache.org/jira/browse/IGNITE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632001#comment-16632001
]
ASF GitHub Bot commented on IGNITE-7251:
Github user vveider closed the pull request at:
https://github.com/apache/ignite/pull/4804
> Remove term "fabric" from Ignite deliverables
> -
>
> Key: IGNITE-7251
> URL: https://issues.apache.org/jira/browse/IGNITE-7251
> Project: Ignite
> Issue Type: Task
>Reporter: Denis Magda
>Assignee: Anton Vinogradov
>Priority: Blocker
> Labels: important
> Fix For: 2.7
>
>
> Apache Ignite binary releases still include “fabric” word in their names:
> https://ignite.apache.org/download.cgi#binaries
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> Remove “fabric” from the name and have the binary releases named as -
> {{apache-ignite-\{version}-bin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
[
https://issues.apache.org/jira/browse/IGNITE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631908#comment-16631908
]
Peter Ivanov edited comment on IGNITE-7251 at 9/28/18 3:13 PM:
---
TeamCity check is complete,
[PR#4502|https://github.com/apache/ignite/pull/4502] recommended for merge.
was (Author: vveider):
TeamCity check is complete,
[PR#4804|https://github.com/apache/ignite/pull/4804] recommended for merge.
> Remove term "fabric" from Ignite deliverables
> -
>
> Key: IGNITE-7251
> URL: https://issues.apache.org/jira/browse/IGNITE-7251
> Project: Ignite
> Issue Type: Task
>Reporter: Denis Magda
>Assignee: Anton Vinogradov
>Priority: Blocker
> Labels: important
> Fix For: 2.7
>
>
> Apache Ignite binary releases still include “fabric” word in their names:
> https://ignite.apache.org/download.cgi#binaries
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> Remove “fabric” from the name and have the binary releases named as -
> {{apache-ignite-\{version}-bin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9693) Scale up wal compression workers to increase performance
[ https://issues.apache.org/jira/browse/IGNITE-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631996#comment-16631996 ] ASF GitHub Bot commented on IGNITE-9693: Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4831 > Scale up wal compression workers to increase performance > > > Key: IGNITE-9693 > URL: https://issues.apache.org/jira/browse/IGNITE-9693 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Kosarev >Assignee: Ivan Daschinskiy >Priority: Major > Fix For: 2.8 > > Attachments: successfull_local_test.PNG > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9693) Scale up wal compression workers to increase performance
[ https://issues.apache.org/jira/browse/IGNITE-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-9693: --- Component/s: persistence > Scale up wal compression workers to increase performance > > > Key: IGNITE-9693 > URL: https://issues.apache.org/jira/browse/IGNITE-9693 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Kosarev >Assignee: Ivan Daschinskiy >Priority: Major > Fix For: 2.8 > > Attachments: successfull_local_test.PNG > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9621) MVCC: sqlline warning that transactions are not supported
[
https://issues.apache.org/jira/browse/IGNITE-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov updated IGNITE-9621:
Ignite Flags: (was: Docs Required)
> MVCC: sqlline warning that transactions are not supported
> -
>
> Key: IGNITE-9621
> URL: https://issues.apache.org/jira/browse/IGNITE-9621
> Project: Ignite
> Issue Type: Bug
> Components: jdbc, mvcc
>Reporter: Stepan Pilschikov
>Assignee: Vladimir Ozerov
>Priority: Major
> Fix For: 2.7
>
>
> MVCC enabled
> --autoCommit=false
> In sqlline first initial lines throwing warning message:
> {code}
> [1;34missuing: !connect jdbc:ignite:thin://127.0.0.1:10800 '' ''
> org.apache.ignite.IgniteJdbcThinDriver[m
> [1;34mConnecting to jdbc:ignite:thin://127.0.0.1:10800[m
> [1;34mConnected to: Apache Ignite (version 2.7.1#19700101-sha1:)[m
> [1;34mDriver: Apache Ignite Thin JDBC Driver (version
> 2.7.1#19700101-sha1:)[m
> [1;34mAutocommit status: false[m
> Sep 17, 2018 5:44:32 PM
> org.apache.ignite.internal.jdbc.thin.JdbcThinConnection
> setTransactionIsolation
> WARNING: Transactions are not supported.
> Sep 17, 2018 5:44:32 PM
> org.apache.ignite.internal.jdbc.thin.JdbcThinConnection
> getTransactionIsolation
> WARNING: Transactions are not supported.
> [1;34mTransaction isolation: TRANSACTION_REPEATABLE_READ[m
> sqlline version 1.3.0
> 0: jdbc:ignite:thin://127.0.0.1:10800>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IGNITE-9621) MVCC: sqlline warning that transactions are not supported
[
https://issues.apache.org/jira/browse/IGNITE-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov resolved IGNITE-9621.
-
Resolution: Fixed
Assignee: Vladimir Ozerov
> MVCC: sqlline warning that transactions are not supported
> -
>
> Key: IGNITE-9621
> URL: https://issues.apache.org/jira/browse/IGNITE-9621
> Project: Ignite
> Issue Type: Bug
> Components: jdbc, mvcc
>Reporter: Stepan Pilschikov
>Assignee: Vladimir Ozerov
>Priority: Major
> Fix For: 2.7
>
>
> MVCC enabled
> --autoCommit=false
> In sqlline first initial lines throwing warning message:
> {code}
> [1;34missuing: !connect jdbc:ignite:thin://127.0.0.1:10800 '' ''
> org.apache.ignite.IgniteJdbcThinDriver[m
> [1;34mConnecting to jdbc:ignite:thin://127.0.0.1:10800[m
> [1;34mConnected to: Apache Ignite (version 2.7.1#19700101-sha1:)[m
> [1;34mDriver: Apache Ignite Thin JDBC Driver (version
> 2.7.1#19700101-sha1:)[m
> [1;34mAutocommit status: false[m
> Sep 17, 2018 5:44:32 PM
> org.apache.ignite.internal.jdbc.thin.JdbcThinConnection
> setTransactionIsolation
> WARNING: Transactions are not supported.
> Sep 17, 2018 5:44:32 PM
> org.apache.ignite.internal.jdbc.thin.JdbcThinConnection
> getTransactionIsolation
> WARNING: Transactions are not supported.
> [1;34mTransaction isolation: TRANSACTION_REPEATABLE_READ[m
> sqlline version 1.3.0
> 0: jdbc:ignite:thin://127.0.0.1:10800>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-9693) Scale up wal compression workers to increase performance
[ https://issues.apache.org/jira/browse/IGNITE-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631992#comment-16631992 ] Ivan Rakov edited comment on IGNITE-9693 at 9/28/18 3:11 PM: - [~ivandasch], thanks for your contribution! Merged to master. was (Author: ivan.glukos): [~ivandasch], thanks fro your contribution! Merged to master. > Scale up wal compression workers to increase performance > > > Key: IGNITE-9693 > URL: https://issues.apache.org/jira/browse/IGNITE-9693 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Kosarev >Assignee: Ivan Daschinskiy >Priority: Major > Fix For: 2.8 > > Attachments: successfull_local_test.PNG > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9693) Scale up wal compression workers to increase performance
[ https://issues.apache.org/jira/browse/IGNITE-9693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Rakov updated IGNITE-9693: --- Ignite Flags: (was: Docs Required) > Scale up wal compression workers to increase performance > > > Key: IGNITE-9693 > URL: https://issues.apache.org/jira/browse/IGNITE-9693 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Kosarev >Assignee: Ivan Daschinskiy >Priority: Major > Fix For: 2.8 > > Attachments: successfull_local_test.PNG > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov updated IGNITE-9171:
Issue Type: Task (was: Improvement)
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Task
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-6677) Collect QueryDetailMetrics for cache-less SQL queries
[ https://issues.apache.org/jira/browse/IGNITE-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-6677: Issue Type: Bug (was: Improvement) > Collect QueryDetailMetrics for cache-less SQL queries > - > > Key: IGNITE-6677 > URL: https://issues.apache.org/jira/browse/IGNITE-6677 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.2 >Reporter: Alexey Kuznetsov >Assignee: Sergey Grimstad >Priority: Blocker > Fix For: 2.7 > > > We have logic that collect query history per caches. > We need: > # Add history size param on IgniteConfiguration > # Implement logic that will collect history for SQL queries executed directly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9540) MVCC TX: make cache invoke\invokeAll operations support Mvcc tx mode.
[
https://issues.apache.org/jira/browse/IGNITE-9540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov updated IGNITE-9540:
Issue Type: Task (was: New Feature)
> MVCC TX: make cache invoke\invokeAll operations support Mvcc tx mode.
> -
>
> Key: IGNITE-9540
> URL: https://issues.apache.org/jira/browse/IGNITE-9540
> Project: Ignite
> Issue Type: Task
> Components: cache, mvcc
>Reporter: Andrew Mashenkov
>Assignee: Andrew Mashenkov
>Priority: Major
> Fix For: 2.7
>
>
> {color:#FF}{color}Make invoke/invokeAll operations consistent with SQL
> queries in MVCC mode.
> This requires changes in protocol.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9390) MVCC: add MVCC support to .NET configuration.
[ https://issues.apache.org/jira/browse/IGNITE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-9390: Issue Type: Task (was: Improvement) > MVCC: add MVCC support to .NET configuration. > - > > Key: IGNITE-9390 > URL: https://issues.apache.org/jira/browse/IGNITE-9390 > Project: Ignite > Issue Type: Task > Components: mvcc, platforms >Reporter: Andrew Mashenkov >Assignee: Vladimir Ozerov >Priority: Blocker > Fix For: 2.7 > > > We should make MVCC configurable from .NET client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov updated IGNITE-9171:
Priority: Blocker (was: Major)
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Blocker
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-6044) SQL insert waits for transaction commit, but it must be executed right away
[ https://issues.apache.org/jira/browse/IGNITE-6044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-6044: Fix Version/s: (was: 2.7) 2.8 > SQL insert waits for transaction commit, but it must be executed right away > --- > > Key: IGNITE-6044 > URL: https://issues.apache.org/jira/browse/IGNITE-6044 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.1 >Reporter: Mikhail Cherkasov >Assignee: Yury Gerzhedovich >Priority: Critical > Labels: sql-stability, usability > Fix For: 2.8 > > > Doc says: > ""Presently, DML supports the atomic mode only meaning that if there is a DML > query that is executed as a part of an Ignite transaction then it will not be > enlisted in the transaction's writing queue and will be executed right away."" > https://apacheignite.readme.io/docs/dml#section-transactional-support > However the data will be added to cache only after transaction commit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-6044) SQL insert waits for transaction commit, but it must be executed right away
[ https://issues.apache.org/jira/browse/IGNITE-6044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631973#comment-16631973 ] Vladimir Ozerov commented on IGNITE-6044: - Let's move to AI 2.8. We have a slight chance to finalize it by Monday (including tests, another review, etc), but not sure. > SQL insert waits for transaction commit, but it must be executed right away > --- > > Key: IGNITE-6044 > URL: https://issues.apache.org/jira/browse/IGNITE-6044 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.1 >Reporter: Mikhail Cherkasov >Assignee: Yury Gerzhedovich >Priority: Critical > Labels: sql-stability, usability > Fix For: 2.8 > > > Doc says: > ""Presently, DML supports the atomic mode only meaning that if there is a DML > query that is executed as a part of an Ignite transaction then it will not be > enlisted in the transaction's writing queue and will be executed right away."" > https://apacheignite.readme.io/docs/dml#section-transactional-support > However the data will be added to cache only after transaction commit. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9738) Client node can suddenly fail on start
[
https://issues.apache.org/jira/browse/IGNITE-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladislav Pyatkov updated IGNITE-9738:
--
Description:
If client joining to large topology it can to spend some time on waiting
{{TcpDiscoveryNodeAddFinishedMessage}}, but in that time it can not to send
{{TcpDiscoveryClientMetricsUpdateMessage.}} By that reason server can to reset
client from topology.
{noformat}
Client node considered as unreachable and will be dropped from cluster, because
no metrics update messages received in interval:
TcpDiscoverySpi.clientFailureDetectionTimeout() ms. It may be caused by network
problems or long GC pause on client node, try to increase this parameter.
[nodeId=a3493895-7c13-403c-bf0d-94eab011,
clientFailureDetectionTimeout=3];
{noformat}
We should to sent {{TcpDiscoveryClientMetricsUpdateMessage as soon as
possible}} without, waiting finish of join procedure.
was:
If client joining to large topology it can to spend some time on waiting
{{TcpDiscoveryNodeAddFinishedMessage}}, but in that time it can not to send
{{TcpDiscoveryClientMetricsUpdateMessage.}} By that reason server can to reset
client from topology.
We should to sent {{TcpDiscoveryClientMetricsUpdateMessage as soon as
possible}} without, waiting finish of join procedure.
> Client node can suddenly fail on start
> --
>
> Key: IGNITE-9738
> URL: https://issues.apache.org/jira/browse/IGNITE-9738
> Project: Ignite
> Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Priority: Major
>
> If client joining to large topology it can to spend some time on waiting
> {{TcpDiscoveryNodeAddFinishedMessage}}, but in that time it can not to send
> {{TcpDiscoveryClientMetricsUpdateMessage.}} By that reason server can to
> reset client from topology.
> {noformat}
> Client node considered as unreachable and will be dropped from cluster,
> because no metrics update messages received in interval:
> TcpDiscoverySpi.clientFailureDetectionTimeout() ms. It may be caused by
> network problems or long GC pause on client node, try to increase this
> parameter. [nodeId=a3493895-7c13-403c-bf0d-94eab011,
> clientFailureDetectionTimeout=3];
> {noformat}
> We should to sent {{TcpDiscoveryClientMetricsUpdateMessage as soon as
> possible}} without, waiting finish of join procedure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-9583) PHP thin: php directory should be included in binary release
[
https://issues.apache.org/jira/browse/IGNITE-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov reassigned IGNITE-9583:
---
Assignee: Peter Ivanov
> PHP thin: php directory should be included in binary release
>
>
> Key: IGNITE-9583
> URL: https://issues.apache.org/jira/browse/IGNITE-9583
> Project: Ignite
> Issue Type: Improvement
> Components: build, thin client
>Reporter: Igor Sapego
>Assignee: Peter Ivanov
>Priority: Major
> Labels: php
> Fix For: 2.7
>
>
> Currently, php directory is not generated by the {{maven install}} command.
> Need to add appropriate copy steps to {{assembly/release-fabric-base.xml}}
> file.
> The following components should be copied:
> {noformat}
> ignite/modules/platforms/php/composer.json
> ignite/modules/platforms/php/src
> ignite/modules/platforms/php/examples
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-9559) Node.js thin: nodejs directory should be included binary release
[
https://issues.apache.org/jira/browse/IGNITE-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Ozerov reassigned IGNITE-9559:
---
Assignee: Peter Ivanov
> Node.js thin: nodejs directory should be included binary release
>
>
> Key: IGNITE-9559
> URL: https://issues.apache.org/jira/browse/IGNITE-9559
> Project: Ignite
> Issue Type: Improvement
> Components: build, thin client
>Reporter: Igor Sapego
>Assignee: Peter Ivanov
>Priority: Major
> Labels: nodejs
> Fix For: 2.7
>
>
> Currently, nodejs directory is not generated by the {{maven install}}
> command. Need to add appropriate copy steps to
> {{assembly/release-fabric-base.xml}} file.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IGNITE-9272) PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace.
[ https://issues.apache.org/jira/browse/IGNITE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-9272: Fix Version/s: (was: 2.7) 2.8 > PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace. > -- > > Key: IGNITE-9272 > URL: https://issues.apache.org/jira/browse/IGNITE-9272 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.8 > > Attachments: BenchmarkCRC.java > > > I see that Ignite has its own crc32 realization called: PureJavaCrc32 and > from desc it seems to be : _The current version is ~10x to 1.8x as fast as > Sun's native java.util.zip.CRC32 in Java 1.6_ But my jmh tests show opposite > results. > + If it really so, looks like backward compatibility would be easy, all that > need is just to take lower part of long form zip.crc32 realization. > jmh results: > Benchmark Mode CntScoreError Units > BenchmarkCRC.Crc32 avgt5 1521060.716 ± 44083.424 ns/op > BenchmarkCRC.pureJavaCrc32 avgt5 4657756.671 ± 177243.254 ns/op > JMH version: 1.21 > VM version: JDK 1.8.0_131, Java HotSpot(TM) 64-Bit Server VM, 25.131-b11 > VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java > op system : ubuntu 16.10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9687) JDK9: JTA tests failed
[
https://issues.apache.org/jira/browse/IGNITE-9687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631968#comment-16631968
]
ASF GitHub Bot commented on IGNITE-9687:
Github user asfgit closed the pull request at:
https://github.com/apache/ignite/pull/4858
> JDK9: JTA tests failed
> --
>
> Key: IGNITE-9687
> URL: https://issues.apache.org/jira/browse/IGNITE-9687
> Project: Ignite
> Issue Type: Task
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: jdk9
> Fix For: 2.7
>
>
> JTA tests fail on JDK9 with error:
> {{java.lang.NoClassDefFoundError: javax/rmi/PortableRemoteObject}}
> the option {{--add-modules java.se.ee}}
> changes the error to:
> {{java.lang.NoClassDefFoundError: javax/transaction/UserTransaction}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-9738) Client node can suddenly fail on start
Vladislav Pyatkov created IGNITE-9738:
-
Summary: Client node can suddenly fail on start
Key: IGNITE-9738
URL: https://issues.apache.org/jira/browse/IGNITE-9738
Project: Ignite
Issue Type: Bug
Reporter: Vladislav Pyatkov
If client joining to large topology it can to spend some time on waiting
{{TcpDiscoveryNodeAddFinishedMessage}}, but in that time it can not to send
{{TcpDiscoveryClientMetricsUpdateMessage.}} By that reason server can to reset
client from topology.
We should to sent {{TcpDiscoveryClientMetricsUpdateMessage as soon as
possible}} without, waiting finish of join procedure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9272) PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace.
[ https://issues.apache.org/jira/browse/IGNITE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631967#comment-16631967 ] Vladimir Ozerov commented on IGNITE-9272: - Agreed to move the ticket to the next release. > PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace. > -- > > Key: IGNITE-9272 > URL: https://issues.apache.org/jira/browse/IGNITE-9272 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.8 > > Attachments: BenchmarkCRC.java > > > I see that Ignite has its own crc32 realization called: PureJavaCrc32 and > from desc it seems to be : _The current version is ~10x to 1.8x as fast as > Sun's native java.util.zip.CRC32 in Java 1.6_ But my jmh tests show opposite > results. > + If it really so, looks like backward compatibility would be easy, all that > need is just to take lower part of long form zip.crc32 realization. > jmh results: > Benchmark Mode CntScoreError Units > BenchmarkCRC.Crc32 avgt5 1521060.716 ± 44083.424 ns/op > BenchmarkCRC.pureJavaCrc32 avgt5 4657756.671 ± 177243.254 ns/op > JMH version: 1.21 > VM version: JDK 1.8.0_131, Java HotSpot(TM) 64-Bit Server VM, 25.131-b11 > VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java > op system : ubuntu 16.10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-8873) Optimize cache scans with enabled persistence.
[ https://issues.apache.org/jira/browse/IGNITE-8873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-8873: Fix Version/s: (was: 2.7) 2.8 > Optimize cache scans with enabled persistence. > -- > > Key: IGNITE-8873 > URL: https://issues.apache.org/jira/browse/IGNITE-8873 > Project: Ignite > Issue Type: Improvement >Reporter: Alexei Scherbakov >Assignee: Alexei Scherbakov >Priority: Major > Fix For: 2.8 > > > Currently cache scans with enabled persistence involve link resolution, which > can lead to radom disk access resulting in bad performace on SAS disks. > One possibility is to preload cache data pages to remove slow random disk > access. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9390) MVCC: add MVCC support to .NET configuration.
[ https://issues.apache.org/jira/browse/IGNITE-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-9390: Priority: Blocker (was: Major) > MVCC: add MVCC support to .NET configuration. > - > > Key: IGNITE-9390 > URL: https://issues.apache.org/jira/browse/IGNITE-9390 > Project: Ignite > Issue Type: Improvement > Components: mvcc, platforms >Reporter: Andrew Mashenkov >Assignee: Vladimir Ozerov >Priority: Blocker > Fix For: 2.7 > > > We should make MVCC configurable from .NET client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9171) Use lazy mode with results pre-fetch
[
https://issues.apache.org/jira/browse/IGNITE-9171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631952#comment-16631952
]
Vladimir Ozerov commented on IGNITE-9171:
-
[~tledkov-gridgain], I reviewed the patch. It looks good in general. But I
found several suspicious places, where concurrency may be broken. Could you
please take care of these places (mentioned in PR)?
> Use lazy mode with results pre-fetch
>
>
> Key: IGNITE-9171
> URL: https://issues.apache.org/jira/browse/IGNITE-9171
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.6
>Reporter: Taras Ledkov
>Assignee: Taras Ledkov
>Priority: Major
> Labels: sql-stability
> Fix For: 2.7
>
>
> Current implementation of the {{lazy}} mode always starts separate thread for
> {{MapQueryLazyWorker}}. It causes excessive overhead for requests that
> produces small results set.
> We have to begin execute query at the {{QUERY_POOL}} thread pool and fetch
> first page of the results. In case results set is bigger than one page
> {{MapQueryLazyWorker}} is started and link with {{MapNodeResults}} to handle
> next pages lazy.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-9702) Make IGNITE_BINARY_SORT_OBJECT_FIELDS enabled by default
[ https://issues.apache.org/jira/browse/IGNITE-9702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Cherkasov reassigned IGNITE-9702: - Assignee: (was: Mikhail Cherkasov) > Make IGNITE_BINARY_SORT_OBJECT_FIELDS enabled by default > > > Key: IGNITE-9702 > URL: https://issues.apache.org/jira/browse/IGNITE-9702 > Project: Ignite > Issue Type: Bug >Reporter: Mikhail Cherkasov >Priority: Blocker > Fix For: 3.0 > > > Right now: > BinObjectImpl(a=1,b=2) != BinObjectImpl(b=2,a=1), so hash code of binary > object depends of fields order and for ignite it's too different objects. > It's unclear and counter-intuitive for a user. > However, this can not be changed till 3.0, because it breaks compatibility > with existent storage and requites migration with downtime and an util that > will migrate binary objects to a new internal fields order. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-9702) Make IGNITE_BINARY_SORT_OBJECT_FIELDS enabled by default
[ https://issues.apache.org/jira/browse/IGNITE-9702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Cherkasov updated IGNITE-9702: -- Description: Right now: BinObjectImpl(a=1,b=2) != BinObjectImpl(b=2,a=1), so hash code of binary object depends of fields order and for ignite it's too different objects. It's unclear and counter-intuitive for a user. However, this can not be changed till 3.0, because it breaks compatibility with existent storage and requites migration with downtime and an util that will migrate binary objects to a new internal fields order. was: Right now: BinObjectImpl(a=1,b=2) != BinObjectImpl(a=2,b=1), so hash code of binary object depends of fields order and for ignite it's too different objects. It's unclear and counter-intuitive for a user. However, this can not be changed till 3.0, because it breaks compatibility with existent storage and requites migration with downtime and an util that will migrate binary objects to a new internal fields order. > Make IGNITE_BINARY_SORT_OBJECT_FIELDS enabled by default > > > Key: IGNITE-9702 > URL: https://issues.apache.org/jira/browse/IGNITE-9702 > Project: Ignite > Issue Type: Bug >Reporter: Mikhail Cherkasov >Assignee: Mikhail Cherkasov >Priority: Blocker > Fix For: 3.0 > > > Right now: > BinObjectImpl(a=1,b=2) != BinObjectImpl(b=2,a=1), so hash code of binary > object depends of fields order and for ignite it's too different objects. > It's unclear and counter-intuitive for a user. > However, this can not be changed till 3.0, because it breaks compatibility > with existent storage and requites migration with downtime and an util that > will migrate binary objects to a new internal fields order. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-9702) Make IGNITE_BINARY_SORT_OBJECT_FIELDS enabled by default
[ https://issues.apache.org/jira/browse/IGNITE-9702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Cherkasov reassigned IGNITE-9702: - Assignee: Mikhail Cherkasov > Make IGNITE_BINARY_SORT_OBJECT_FIELDS enabled by default > > > Key: IGNITE-9702 > URL: https://issues.apache.org/jira/browse/IGNITE-9702 > Project: Ignite > Issue Type: Bug >Reporter: Mikhail Cherkasov >Assignee: Mikhail Cherkasov >Priority: Blocker > Fix For: 3.0 > > > Right now: > BinObjectImpl(a=1,b=2) != BinObjectImpl(a=2,b=1), so hash code of binary > object depends of fields order and for ignite it's too different objects. > It's unclear and counter-intuitive for a user. > However, this can not be changed till 3.0, because it breaks compatibility > with existent storage and requites migration with downtime and an util that > will migrate binary objects to a new internal fields order. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
[
https://issues.apache.org/jira/browse/IGNITE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631908#comment-16631908
]
Peter Ivanov edited comment on IGNITE-7251 at 9/28/18 2:16 PM:
---
TeamCity check is complete,
[PR#4804|https://github.com/apache/ignite/pull/4804] recommended for merge.
was (Author: vveider):
TeamCity check is complete, patch recommended for merge.
> Remove term "fabric" from Ignite deliverables
> -
>
> Key: IGNITE-7251
> URL: https://issues.apache.org/jira/browse/IGNITE-7251
> Project: Ignite
> Issue Type: Task
>Reporter: Denis Magda
>Assignee: Anton Vinogradov
>Priority: Blocker
> Labels: important
> Fix For: 2.7
>
>
> Apache Ignite binary releases still include “fabric” word in their names:
> https://ignite.apache.org/download.cgi#binaries
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> Remove “fabric” from the name and have the binary releases named as -
> {{apache-ignite-\{version}-bin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
[
https://issues.apache.org/jira/browse/IGNITE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631866#comment-16631866
]
Peter Ivanov edited comment on IGNITE-7251 at 9/28/18 2:16 PM:
---
Test plan:
* (/) [AI tests
2.4+|https://ci.ignite.apache.org/viewLog.html?buildId=1924462]: no problems
found
* (/) [AI release|https://ci.ignite.apache.org/viewLog.html?buildId=1948138]:
release task was modified for accepting new assembly descriptors
* (/) Release script {{vote_3_step_1[packages]build.sh}}: fixed package
assembly (already merged to _ignite-release_)
* (!) AI nightly release: will be fixed right after merge
was (Author: vveider):
Test plan:
* (/) [AI tests
2.4+|https://ci.ignite.apache.org/viewLog.html?buildId=1924462]: no problems
found
* (/) [AI release|https://ci.ignite.apache.org/viewLog.html?buildId=1948138]:
release task was modified for accepting new assembly descriptors
* Release script {{vote_3_step_1[packages]build.sh}}: fixed package assembly
(already merged to _ignite-release_)
* (!) AI nightly release: will be fixed right after merge
> Remove term "fabric" from Ignite deliverables
> -
>
> Key: IGNITE-7251
> URL: https://issues.apache.org/jira/browse/IGNITE-7251
> Project: Ignite
> Issue Type: Task
>Reporter: Denis Magda
>Assignee: Anton Vinogradov
>Priority: Blocker
> Labels: important
> Fix For: 2.7
>
>
> Apache Ignite binary releases still include “fabric” word in their names:
> https://ignite.apache.org/download.cgi#binaries
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> Remove “fabric” from the name and have the binary releases named as -
> {{apache-ignite-\{version}-bin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9458) SQL benchmarks: absence of "--mvcc" benchmark flag should not overwrite config values
[ https://issues.apache.org/jira/browse/IGNITE-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631909#comment-16631909 ] Pavel Kuznetsov commented on IGNITE-9458: - --mvcc removed, since we turn on mvcc by setting cache mode to TRANSACTIONAL_SNAPSHOT > SQL benchmarks: absence of "--mvcc" benchmark flag should not overwrite > config values > - > > Key: IGNITE-9458 > URL: https://issues.apache.org/jira/browse/IGNITE-9458 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Minor > > Currently, IgniteNode benchmark application ignores mvccEnable property of > IgniteConfiguration. > Absence of --mvcc flag is treateted as "turn mvcc off forcibly", existence as > "turn mvcc on forcibly". > We need to change sematics of absence of this flag as "Use value from config". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
[
https://issues.apache.org/jira/browse/IGNITE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631866#comment-16631866
]
Peter Ivanov edited comment on IGNITE-7251 at 9/28/18 2:15 PM:
---
Test plan:
* (/) [AI tests
2.4+|https://ci.ignite.apache.org/viewLog.html?buildId=1924462]: no problems
found
* (/) [AI release|https://ci.ignite.apache.org/viewLog.html?buildId=1948138]:
release task was modified for accepting new assembly descriptors
* Release script {{vote_3_step_1[packages]build.sh}}: fixed package assembly
(already merged to _ignite-release_)
* (!) AI nightly release: will be fixed right after merge
was (Author: vveider):
Test plan:
* (/) [AI tests 2.4+|https://ci.ignite.apache.org/viewLog.html?buildId=1924462]
* (/) [AI release|https://ci.ignite.apache.org/viewLog.html?buildId=1948138]
Release scripts
** (/) vote_3_step_1[packages]build.sh
** (/) vote_3_step_2[pgp]sign_artifacts.sh
* (!) AI nightly release (will be fixed right after merge)
> Remove term "fabric" from Ignite deliverables
> -
>
> Key: IGNITE-7251
> URL: https://issues.apache.org/jira/browse/IGNITE-7251
> Project: Ignite
> Issue Type: Task
>Reporter: Denis Magda
>Assignee: Anton Vinogradov
>Priority: Blocker
> Labels: important
> Fix For: 2.7
>
>
> Apache Ignite binary releases still include “fabric” word in their names:
> https://ignite.apache.org/download.cgi#binaries
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> Remove “fabric” from the name and have the binary releases named as -
> {{apache-ignite-\{version}-bin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IGNITE-9458) SQL benchmarks: absence of "--mvcc" benchmark flag should not overwrite config values
[ https://issues.apache.org/jira/browse/IGNITE-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Kuznetsov resolved IGNITE-9458. - Resolution: Won't Fix > SQL benchmarks: absence of "--mvcc" benchmark flag should not overwrite > config values > - > > Key: IGNITE-9458 > URL: https://issues.apache.org/jira/browse/IGNITE-9458 > Project: Ignite > Issue Type: Bug >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Minor > > Currently, IgniteNode benchmark application ignores mvccEnable property of > IgniteConfiguration. > Absence of --mvcc flag is treateted as "turn mvcc off forcibly", existence as > "turn mvcc on forcibly". > We need to change sematics of absence of this flag as "Use value from config". -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
[
https://issues.apache.org/jira/browse/IGNITE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631908#comment-16631908
]
Peter Ivanov commented on IGNITE-7251:
--
TeamCity check is complete, patch recommended for merge.
> Remove term "fabric" from Ignite deliverables
> -
>
> Key: IGNITE-7251
> URL: https://issues.apache.org/jira/browse/IGNITE-7251
> Project: Ignite
> Issue Type: Task
>Reporter: Denis Magda
>Assignee: Anton Vinogradov
>Priority: Blocker
> Labels: important
> Fix For: 2.7
>
>
> Apache Ignite binary releases still include “fabric” word in their names:
> https://ignite.apache.org/download.cgi#binaries
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> Remove “fabric” from the name and have the binary releases named as -
> {{apache-ignite-\{version}-bin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-5733) Activate/deactivate cluster through http-rest api
[ https://issues.apache.org/jira/browse/IGNITE-5733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631877#comment-16631877 ] Roman Shtykh commented on IGNITE-5733: -- [~pgarg] Does IGNITE-9253 address the issues? if yes, let's document. > Activate/deactivate cluster through http-rest api > - > > Key: IGNITE-5733 > URL: https://issues.apache.org/jira/browse/IGNITE-5733 > Project: Ignite > Issue Type: Improvement >Affects Versions: 2.0 >Reporter: Alexander Belyak >Assignee: Alexander Belyak >Priority: Critical > Labels: usability > Fix For: 2.8 > > > Need to add command to get/set cluster active flag into http rest api. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8006) Starting multiple caches slows down exchange process on joining node
[
https://issues.apache.org/jira/browse/IGNITE-8006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631871#comment-16631871
]
Pavel Kovalenko commented on IGNITE-8006:
-
[~akalashnikov] I've reviewed your changes and left several comments in
Upsource review. Most of them are minor.
> Starting multiple caches slows down exchange process on joining node
>
>
> Key: IGNITE-8006
> URL: https://issues.apache.org/jira/browse/IGNITE-8006
> Project: Ignite
> Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Anton Kalashnikov
>Priority: Major
> Fix For: 2.8
>
>
> In some cases when we starts multiple caches (over 2K caches), we can get a
> stop on exchange when new node joining to the cluster.
> Coordinator-node wait to receive a single message from all other nodes, but
> last node (which want to joining to the cluster) stopped on starting caches:
>
> {noformat}
> Stack trace
> at java.lang.Thread.dumpStack(Thread.java:1329)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1159)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1900)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:1764)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:740)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:622)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2329)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> It blocks cluster exchange process until all caches started on the last node.
> We should start caches in parallel threads or exclude the action from
> exchange init process.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
[
https://issues.apache.org/jira/browse/IGNITE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631865#comment-16631865
]
Peter Ivanov commented on IGNITE-7251:
--
Test plan:
* (/) [AI tests 2.4+|https://ci.ignite.apache.org/viewLog.html?buildId=1924462]
* (/) [AI release|https://ci.ignite.apache.org/viewLog.html?buildId=1948138]
Release scripts
** (/) vote_3_step_1[packages]build.sh
** (/) vote_3_step_2[pgp]sign_artifacts.sh
* (!) AI nightly release (will be fixed right after merge)
> Remove term "fabric" from Ignite deliverables
> -
>
> Key: IGNITE-7251
> URL: https://issues.apache.org/jira/browse/IGNITE-7251
> Project: Ignite
> Issue Type: Task
>Reporter: Denis Magda
>Assignee: Anton Vinogradov
>Priority: Blocker
> Labels: important
> Fix For: 2.7
>
>
> Apache Ignite binary releases still include “fabric” word in their names:
> https://ignite.apache.org/download.cgi#binaries
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> Remove “fabric” from the name and have the binary releases named as -
> {{apache-ignite-\{version}-bin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
[
https://issues.apache.org/jira/browse/IGNITE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Ivanov updated IGNITE-7251:
-
Comment: was deleted
(was: Test plan:
* (/) [AI tests 2.4+|https://ci.ignite.apache.org/viewLog.html?buildId=1924462]
* (/) [AI release|https://ci.ignite.apache.org/viewLog.html?buildId=1948138]
Release scripts
** (/) vote_3_step_1[packages]build.sh
** (/) vote_3_step_2[pgp]sign_artifacts.sh
* (!) AI nightly release (will be fixed right after merge))
> Remove term "fabric" from Ignite deliverables
> -
>
> Key: IGNITE-7251
> URL: https://issues.apache.org/jira/browse/IGNITE-7251
> Project: Ignite
> Issue Type: Task
>Reporter: Denis Magda
>Assignee: Anton Vinogradov
>Priority: Blocker
> Labels: important
> Fix For: 2.7
>
>
> Apache Ignite binary releases still include “fabric” word in their names:
> https://ignite.apache.org/download.cgi#binaries
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> Remove “fabric” from the name and have the binary releases named as -
> {{apache-ignite-\{version}-bin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-7251) Remove term "fabric" from Ignite deliverables
[
https://issues.apache.org/jira/browse/IGNITE-7251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631866#comment-16631866
]
Peter Ivanov commented on IGNITE-7251:
--
Test plan:
* (/) [AI tests 2.4+|https://ci.ignite.apache.org/viewLog.html?buildId=1924462]
* (/) [AI release|https://ci.ignite.apache.org/viewLog.html?buildId=1948138]
Release scripts
** (/) vote_3_step_1[packages]build.sh
** (/) vote_3_step_2[pgp]sign_artifacts.sh
* (!) AI nightly release (will be fixed right after merge)
> Remove term "fabric" from Ignite deliverables
> -
>
> Key: IGNITE-7251
> URL: https://issues.apache.org/jira/browse/IGNITE-7251
> Project: Ignite
> Issue Type: Task
>Reporter: Denis Magda
>Assignee: Anton Vinogradov
>Priority: Blocker
> Labels: important
> Fix For: 2.7
>
>
> Apache Ignite binary releases still include “fabric” word in their names:
> https://ignite.apache.org/download.cgi#binaries
> For instance, this is a full name of the previous release -
> apache-ignite-fabric-2.3.0-bin.
> It’s a little oversight on our side because the project has not been
> positioned as a fabric for a while.
> Remove “fabric” from the name and have the binary releases named as -
> {{apache-ignite-\{version}-bin}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-7616) GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect values due to invalid interface registration.
[ https://issues.apache.org/jira/browse/IGNITE-7616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631861#comment-16631861 ] Alexey Goncharuk commented on IGNITE-7616: -- [~syssoftsol], thanks for the submission, I will take a look at the nearest time. > GridDataStreamExecutor and GridCallbackExecutor JMX beans return incorrect > values due to invalid interface registration. > > > Key: IGNITE-7616 > URL: https://issues.apache.org/jira/browse/IGNITE-7616 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.4 >Reporter: Max Shonichev >Assignee: David Harvey >Priority: Major > Labels: jmx > Fix For: 2.7 > > Attachments: master_1b3742f4d7_mxbeans_threads2.patch > > > Two of newly added management beans as a result of implementing feature > request https://issues.apache.org/jira/browse/IGNITE-7217 have bugs: > # GridDataStreamExecutor is registered as conforming to ThreadPoolMXBean > interface, though actually it is an incompatible StripedExecutor. > # GridCallbackExecutor is registered as conforming to ThreadPoolMXBean > interface, though actually it is an incompatible > IgniteStripedThreadPoolExecutor. > # ThreadPoolMXBeanAdapter checks whether adapted instance is > ThreadPoolExecutor, and as interfaces are incompatible, most of the JMX > attributes of GridCallbackExecutor and GridDataStreamExecutor are returned as > -1 or null. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9282) [ML] Add Naive Bayes classifier
[ https://issues.apache.org/jira/browse/IGNITE-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631854#comment-16631854 ] ASF GitHub Bot commented on IGNITE-9282: GitHub user dehasi opened a pull request: https://github.com/apache/ignite/pull/4869 IGNITE-9282 Add Gaussian Naive Bayes Classifier You can merge this pull request into a Git repository by running: $ git pull https://github.com/dehasi/ignite feature/ignite-9282-naive-bayes Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4869.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4869 commit a397a4e89bd57fd1062b9d2a8f4be35be74bf9e7 Author: dehasi Date: 2018-09-08T17:21:17Z IGNITE-9282: Create bayes packages commit 814d45516334b10264f49c6ff31b05883f7c7535 Author: dehasi Date: 2018-09-08T18:55:33Z IGNITE-9282: Add gauss density function commit 7600f9814806cfa377dc8c350d2540f5ffee34f7 Author: dehasi Date: 2018-09-08T20:16:07Z IGNITE-9282: Implement model commit 8b7a9e8ba7f423ffbec0153ad9f6dcafe1921c85 Author: dehasi Date: 2018-09-08T20:25:41Z IGNITE-9282: Add java doc commit bf943bd29395fed9a266c13a5c58960f9b41d5cd Author: dehasi Date: 2018-09-08T20:26:04Z IGNITE-9282: Add dot commit a26865860f7fb5601178f02b93e930686a6669a1 Author: dehasi Date: 2018-09-09T15:46:16Z IGNITE-9282: Update java doc commit e266041820d4dddec6e3158a4d31731906ade8ec Author: dehasi Date: 2018-09-09T15:54:26Z IGNITE-9282: Add a trainer skeleton commit c91bd79d3eaa84afa7557374cddae5053887266f Author: dehasi Date: 2018-09-17T17:31:05Z IGNITE-9282: Add javadoc commit e9498bb5eac4fcb36244270a7d38239ebc31520e Author: dehasi Date: 2018-09-21T12:22:18Z IGNITE-9282: Add computation skeleton commit bc98cc65e44ba3f98386007ff74d4fc9d6c6c36d Author: dehasi Date: 2018-09-21T12:46:19Z IGNITE-9282: count means for each label commit 977b2393734ceabbe858d5aee8e9861039bb2ce5 Author: dehasi Date: 2018-09-23T20:50:55Z IGNITE-9282: count means with map reduce commit 117939be9a7f537d057d3454846b4389aba6e6d9 Author: dehasi Date: 2018-09-23T21:47:16Z IGNITE-9282: Add trainer unit test commit 2c8d29a51cde34e6356e922e8eccf378ca979221 Author: dehasi Date: 2018-09-23T22:03:40Z IGNITE-9282: Add test checker commit 3b05686321c77f0c36aaefe9757dbf43d04e0d8c Author: dehasi Date: 2018-09-24T17:02:31Z IGNITE-9282: compute variances commit d2b42f94f9e2c4ff107457c5c0e39f3cfad738ee Author: dehasi Date: 2018-09-24T17:22:26Z IGNITE-9282: fix variance calc commit 664b175f4d817ea17b5ede96bd8956978dbab3a6 Author: dehasi Date: 2018-09-24T18:54:01Z IGNITE-9282: Add java docs to model commit 4730a24d4594f6a77a1cfd9682b5bd911f9d6210 Author: dehasi Date: 2018-09-24T19:12:48Z IGNITE-9282: Add java docs to trainer commit 6982b4d5cfac77d118e7d922a292f0c2814383bf Author: dehasi Date: 2018-09-24T19:12:59Z IGNITE-9282: Add java docs to trainer commit 8337b890ca2916566dda7f58ff31222b5d0b920c Author: dehasi Date: 2018-09-24T19:23:30Z IGNITE-9282: Add a test to check variance commit c6e5eb2d11b3fff3ceac25769df5471c879bd5b9 Author: dehasi Date: 2018-09-24T19:43:52Z Merge branch 'master' into feature/ignite-9282-naive-bayes commit fdcf94ac8286c0212fd537982adf4369e704dfaa Author: dehasi Date: 2018-09-24T20:30:11Z IGNITE-9282: Add Naive Bayes Example commit aa76b8dcc00f17d49627e0717318fa1ce0d1cac1 Author: dehasi Date: 2018-09-26T16:47:56Z IGNITE-9282: Use double instead of object commit 5ae89d1415b775f287cba371e37627129fd19fa6 Author: dehasi Date: 2018-09-26T16:55:01Z IGNITE-9282: Use TrainerTest commit 17b39e8a64b3d2c89411ffc0d3742af7cf816d84 Author: dehasi Date: 2018-09-26T17:56:08Z IGNITE-9282: Use array instead of vector commit 23fec409973d2635e9556fa8de3b21bb80067f65 Author: dehasi Date: 2018-09-26T18:01:07Z IGNITE-9282: Add abitily to set class probabilities commit a5021023bbe07ec5740e6b91085a631cb3f9d3ca Author: dehasi Date: 2018-09-26T18:19:31Z IGNITE-9282: Add abitily to set class probabilities commit a5faf74b28ae3a7ea0a456a67d4fae7e7f16d66b Author: dehasi Date: 2018-09-26T18:25:20Z IGNITE-9282: Fix model test commit 23b6ebc51416e6fafb92580046f4837c750a0cd2 Author: dehasi Date: 2018-09-26T18:32:36Z IGNITE-9282: Update trainer setters commit 92dabe2ec6f4bf6599e23db2f17c1a9fab5c4020 Author: dehasi Date: 2018-09-26T18:42:05Z IGNITE-9282: Add wikipedia dataset test commit 71a10b041faac953ea8a874561210f1299441a76 Author: dehasi Date: 2018-09-26T19:08:23Z IGNITE-9282: Add scikit learn dataset test > [ML] Add Naive Bayes classifier > --- > > Key:
[jira] [Commented] (IGNITE-8867) [ML] Bagging on learning sample
[ https://issues.apache.org/jira/browse/IGNITE-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631814#comment-16631814 ] Yury Babak commented on IGNITE-8867: Moved to 2.8 > [ML] Bagging on learning sample > --- > > Key: IGNITE-8867 > URL: https://issues.apache.org/jira/browse/IGNITE-8867 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Artem Malykh >Priority: Major > Fix For: 2.8 > > > Need to implement bootstrapping algorithm in Bagging-classifier -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9717) [ML] Add setters methods to Logistic Regression and fix examples/tests
[ https://issues.apache.org/jira/browse/IGNITE-9717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631815#comment-16631815 ] ASF GitHub Bot commented on IGNITE-9717: Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4865 > [ML] Add setters methods to Logistic Regression and fix examples/tests > -- > > Key: IGNITE-9717 > URL: https://issues.apache.org/jira/browse/IGNITE-9717 > Project: Ignite > Issue Type: Bug > Components: ml >Affects Versions: 2.7 >Reporter: Aleksey Zinoviev >Assignee: Aleksey Zinoviev >Priority: Major > Fix For: 2.7 > > > Logistic Regression and Multilayered Perceptron can not be used in Pipeline > due to unexisting setter methods .withFieldName. > Also examples and tests shoould be fixed -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IGNITE-8662) Umbrella: ML data preprocessing for 2.7 release
[ https://issues.apache.org/jira/browse/IGNITE-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Babak resolved IGNITE-8662. Resolution: Done All related features planned for 2.7 were completed. > Umbrella: ML data preprocessing for 2.7 release > --- > > Key: IGNITE-8662 > URL: https://issues.apache.org/jira/browse/IGNITE-8662 > Project: Ignite > Issue Type: New Feature > Components: ml >Reporter: Yury Babak >Assignee: Yury Babak >Priority: Major > Fix For: 2.7 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-8867) [ML] Bagging on learning sample
[ https://issues.apache.org/jira/browse/IGNITE-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Babak updated IGNITE-8867: --- Fix Version/s: (was: 2.7) 2.8 > [ML] Bagging on learning sample > --- > > Key: IGNITE-8867 > URL: https://issues.apache.org/jira/browse/IGNITE-8867 > Project: Ignite > Issue Type: Improvement > Components: ml >Reporter: Yury Babak >Assignee: Artem Malykh >Priority: Major > Fix For: 2.8 > > > Need to implement bootstrapping algorithm in Bagging-classifier -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9713) [ML] Fix JavaDocs in ML Prerpocessing package
[ https://issues.apache.org/jira/browse/IGNITE-9713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631809#comment-16631809 ] ASF GitHub Bot commented on IGNITE-9713: Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4867 > [ML] Fix JavaDocs in ML Prerpocessing package > - > > Key: IGNITE-9713 > URL: https://issues.apache.org/jira/browse/IGNITE-9713 > Project: Ignite > Issue Type: Bug > Components: ml >Affects Versions: 2.7 >Reporter: Aleksey Zinoviev >Assignee: Aleksey Zinoviev >Priority: Major > Fix For: 2.7 > > > JavaDocs are incorrect in StringEncoder Preprocessor, Encoder Trainer, > Binarization. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-7764) MVCC TX: make cache basic operations support Mvcc tx mode.
[ https://issues.apache.org/jira/browse/IGNITE-7764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631804#comment-16631804 ] ASF GitHub Bot commented on IGNITE-7764: Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4725 > MVCC TX: make cache basic operations support Mvcc tx mode. > -- > > Key: IGNITE-7764 > URL: https://issues.apache.org/jira/browse/IGNITE-7764 > Project: Ignite > Issue Type: New Feature > Components: mvcc, sql >Reporter: Roman Kondakov >Assignee: Andrew Mashenkov >Priority: Major > Fix For: 2.7 > > > We need to implement an MVCC-compatible locking protocol for Key-Value API. > At the moment during transactions with KV operations if entry we are going to > change is unlocked we do not check if it has been changed by the previous > transaction. See IGNITE-6935. > > Lets make get/getAll, put/PutAll/getAndPut, remove/removeAll/getAndRemove > operations consistent with SQL queries in MVCC mode at first > as it blocks many other tickets. Other operations can be implemented within > separate tickets in parallel once this ticket will be resolved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9272) PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace.
[ https://issues.apache.org/jira/browse/IGNITE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631798#comment-16631798 ] Vladimir Ozerov commented on IGNITE-9272: - On the other hand, I see that you did multiple runs, and performance improvement is consistent between runs. I would suggest to do the following: 1) If TC is OK, ticket is reviewed and storage compatibility is not broken by 30 Sept - then merge to AI 2.7 2) Otherwise - move to AI 2.8. What do you think? > PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace. > -- > > Key: IGNITE-9272 > URL: https://issues.apache.org/jira/browse/IGNITE-9272 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.7 > > Attachments: BenchmarkCRC.java > > > I see that Ignite has its own crc32 realization called: PureJavaCrc32 and > from desc it seems to be : _The current version is ~10x to 1.8x as fast as > Sun's native java.util.zip.CRC32 in Java 1.6_ But my jmh tests show opposite > results. > + If it really so, looks like backward compatibility would be easy, all that > need is just to take lower part of long form zip.crc32 realization. > jmh results: > Benchmark Mode CntScoreError Units > BenchmarkCRC.Crc32 avgt5 1521060.716 ± 44083.424 ns/op > BenchmarkCRC.pureJavaCrc32 avgt5 4657756.671 ± 177243.254 ns/op > JMH version: 1.21 > VM version: JDK 1.8.0_131, Java HotSpot(TM) 64-Bit Server VM, 25.131-b11 > VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java > op system : ubuntu 16.10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9272) PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace.
[ https://issues.apache.org/jira/browse/IGNITE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631795#comment-16631795 ] Vladimir Ozerov commented on IGNITE-9272: - 4-5% is very close to typical measurement error. > PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace. > -- > > Key: IGNITE-9272 > URL: https://issues.apache.org/jira/browse/IGNITE-9272 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.6 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Fix For: 2.7 > > Attachments: BenchmarkCRC.java > > > I see that Ignite has its own crc32 realization called: PureJavaCrc32 and > from desc it seems to be : _The current version is ~10x to 1.8x as fast as > Sun's native java.util.zip.CRC32 in Java 1.6_ But my jmh tests show opposite > results. > + If it really so, looks like backward compatibility would be easy, all that > need is just to take lower part of long form zip.crc32 realization. > jmh results: > Benchmark Mode CntScoreError Units > BenchmarkCRC.Crc32 avgt5 1521060.716 ± 44083.424 ns/op > BenchmarkCRC.pureJavaCrc32 avgt5 4657756.671 ± 177243.254 ns/op > JMH version: 1.21 > VM version: JDK 1.8.0_131, Java HotSpot(TM) 64-Bit Server VM, 25.131-b11 > VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java > op system : ubuntu 16.10 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-9712) SQL: Select query benchmarks suite
[ https://issues.apache.org/jira/browse/IGNITE-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631785#comment-16631785 ] Pavel Kuznetsov edited comment on IGNITE-9712 at 9/28/18 12:47 PM: --- Results on aws: 2 servers 1(2) clients, 4 threads ||Configuration ||Avg ops/sec|| |sql-select-pk-jdbc-thin-r1-1-backup|9,791.36| |sql-select-pk-jdbc-thick-r1-1-backup |18,185.57| |sql-select-pk-native-sql-r1-1-backup |18,578.95| |Sql-select-val-jdbc-thin-r1-1-backup |9,371.92| |Sql-select-val-jdbc-thick-r1-1-backup |16,147.76| |Sql-select-val-native-sql-r1-1-backup |16,167.79| |sql-select-pk-jdbc-thin-r1000-1-backup |3,107.89| |sql-select-pk-jdbc-thick-r1000-1-backup|3,897.32| |sql-select-pk-native-sql-r1000-1-backup|4,132.67| |Sql-select-val-jdbc-thin-r1000-1-backup|2,837.67| |Sql-select-val-jdbc-thick-r1000-1-backup |3,392.78| |Sql-select-val-native-sql-r1000-1-backup |3,689.99| was (Author: pkouznet): Results on aws: 2 servers 1(2) clients ||Configuration ||Avg ops/sec|| |sql-select-pk-jdbc-thin-r1-1-backup|9,791.36| |sql-select-pk-jdbc-thick-r1-1-backup |18,185.57| |sql-select-pk-native-sql-r1-1-backup |18,578.95| |Sql-select-val-jdbc-thin-r1-1-backup |9,371.92| |Sql-select-val-jdbc-thick-r1-1-backup |16,147.76| |Sql-select-val-native-sql-r1-1-backup |16,167.79| |sql-select-pk-jdbc-thin-r1000-1-backup |3,107.89| |sql-select-pk-jdbc-thick-r1000-1-backup|3,897.32| |sql-select-pk-native-sql-r1000-1-backup|4,132.67| |Sql-select-val-jdbc-thin-r1000-1-backup|2,837.67| |Sql-select-val-jdbc-thick-r1000-1-backup |3,392.78| |Sql-select-val-native-sql-r1000-1-backup |3,689.99| > SQL: Select query benchmarks suite > -- > > Key: IGNITE-9712 > URL: https://issues.apache.org/jira/browse/IGNITE-9712 > Project: Ignite > Issue Type: Task >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We need to create benchmark suite for select operations. > We already have benchmarks for selects, so we need to update this code to > support > 1) select the single row with by PK; indexed field > 2) select the range of rows filtered with PK; indexed field > Selects by non-indexed fields are out of scope. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9712) SQL: Select query benchmarks suite
[ https://issues.apache.org/jira/browse/IGNITE-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631785#comment-16631785 ] Pavel Kuznetsov commented on IGNITE-9712: - Results on aws: 2 servers 1(2) clients ||Configuration ||Avg ops/sec|| |sql-select-pk-jdbc-thin-r1-1-backup|9,791.36| |sql-select-pk-jdbc-thick-r1-1-backup |18,185.57| |sql-select-pk-native-sql-r1-1-backup |18,578.95| |Sql-select-val-jdbc-thin-r1-1-backup |9,371.92| |Sql-select-val-jdbc-thick-r1-1-backup |16,147.76| |Sql-select-val-native-sql-r1-1-backup |16,167.79| |sql-select-pk-jdbc-thin-r1000-1-backup |3,107.89| |sql-select-pk-jdbc-thick-r1000-1-backup|3,897.32| |sql-select-pk-native-sql-r1000-1-backup|4,132.67| |Sql-select-val-jdbc-thin-r1000-1-backup|2,837.67| |Sql-select-val-jdbc-thick-r1000-1-backup |3,392.78| |Sql-select-val-native-sql-r1000-1-backup |3,689.99| > SQL: Select query benchmarks suite > -- > > Key: IGNITE-9712 > URL: https://issues.apache.org/jira/browse/IGNITE-9712 > Project: Ignite > Issue Type: Task >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We need to create benchmark suite for select operations. > We already have benchmarks for selects, so we need to update this code to > support > 1) select the single row with by PK; indexed field > 2) select the range of rows filtered with PK; indexed field > Selects by non-indexed fields are out of scope. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9736) Public interface DiscoverySpiListener.onDiscovery returns private API class
[ https://issues.apache.org/jira/browse/IGNITE-9736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631783#comment-16631783 ] ASF GitHub Bot commented on IGNITE-9736: GitHub user Jokser opened a pull request: https://github.com/apache/ignite/pull/4868 IGNITE-9736 Fixed public API for Discovery SPI listener You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9736 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/4868.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4868 commit dc96fe07f4b73984e03e12dcc7f4201dc7bc7e05 Author: Pavel Kovalenko Date: 2018-09-28T12:37:41Z IGNITE-9736 Changed return type of Discovery SPI listener for public API. commit b67a61c7a2b8200e026d53aaebc0e274a38c7993 Author: Pavel Kovalenko Date: 2018-09-28T12:41:23Z IGNITE-9736 Fixed usages of Discovery SPI listener. > Public interface DiscoverySpiListener.onDiscovery returns private API class > --- > > Key: IGNITE-9736 > URL: https://issues.apache.org/jira/browse/IGNITE-9736 > Project: Ignite > Issue Type: Bug > Components: general >Reporter: Andrew Mashenkov >Assignee: Pavel Kovalenko >Priority: Critical > Fix For: 2.7 > > > Public interface DiscoverySpiListener.onDiscovery returns > IgniteInternalFuture of 'internal' package. > We should wrap internal future into IgniteFuture of public API. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (IGNITE-9330) Multiple CacheMetricsManageTest tests are failing
[
https://issues.apache.org/jira/browse/IGNITE-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16625835#comment-16625835
]
Oleg Ignatenko edited comment on IGNITE-9330 at 9/28/18 12:34 PM:
--
(i) this also fixed some other flaky tests:
{{CacheMetricsManageTest.testClusterApiClearStatistics}} and
{{CacheMetricsManageTest.testCacheApiClearStatistics}} (for the latter,
Teamcity history for last few weeks shows that test case failures stopped
appearing after this change was merged to master)
was (Author: oignatenko):
(i) this also fixed another flaky test:
{{CacheMetricsManageTest.testClusterApiClearStatistics}}
> Multiple CacheMetricsManageTest tests are failing
> -
>
> Key: IGNITE-9330
> URL: https://issues.apache.org/jira/browse/IGNITE-9330
> Project: Ignite
> Issue Type: Bug
>Reporter: Ilya Kasnacheev
>Assignee: Amelchev Nikita
>Priority: Major
> Labels: MakeTeamcityGreenAgain
> Fix For: 2.7
>
>
> testCacheManagerStatisticsEnable and testClusterApiClearStatistics fail every
> so often, such as in
> https://ci.ignite.apache.org/viewLog.html?buildId=1676464
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-9126) Update Apache Kafka dependency
[ https://issues.apache.org/jira/browse/IGNITE-9126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Pudov reassigned IGNITE-9126: --- Assignee: Maxim Pudov > Update Apache Kafka dependency > -- > > Key: IGNITE-9126 > URL: https://issues.apache.org/jira/browse/IGNITE-9126 > Project: Ignite > Issue Type: Task >Reporter: Dmitriy Pavlov >Assignee: Maxim Pudov >Priority: Major > Fix For: 2.7 > > > It is suggested to update kafka in accordance with scala update, e.g. to > https://mvnrepository.com/artifact/org.apache.kafka/kafka_2.11/1.0.2 > or to Kafka 1.1.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9155) Exception during cluster state change terminates ExchangeWorker
[
https://issues.apache.org/jira/browse/IGNITE-9155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631768#comment-16631768
]
Ignite TC Bot commented on IGNITE-9155:
---
{panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}MVCC Queries{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=1931984]]
* IgniteCacheMvccSqlTestSuite:
CacheMvccPartitionedSqlCoordinatorFailoverTest.testUpdate_N_Objects_ClientServer_Backups0_Sql_Persistence
- 0,0% fails in last 100 master runs.
{color:#d04437}MVCC Cache{color} [[tests
5|https://ci.ignite.apache.org/viewLog.html?buildId=1931982]]
* IgniteCacheMvccTestSuite:
CacheMvccProcessorLazyStartTest.testMvccRestartedWithDynamicCache - 3,3% fails
in last 100 master runs.
* IgniteCacheMvccTestSuite: CacheMvccSizeWithConcurrentTransactionTest.testSize
- 2,9% fails in last 100 master runs.
* IgniteCacheMvccTestSuite:
CacheMvccPartitionedCoordinatorFailoverTest.testReadInProgressCoordinatorFailsSimple_FromServerPutGet
- 0,0% fails in last 100 master runs.
{color:#d04437}Binary Objects (Simple Mapper Basic){color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=1931882]]
* IgniteBinarySimpleNameMapperBasicTestSuite:
IgniteCacheP2pUnmarshallingErrorTest.testResponseMessageOnUnmarshallingFailed -
1,0% fails in last 100 master runs.
{color:#d04437}Cache (Restarts) 1{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=1931959]]
* IgniteCacheRestartTestSuite:
GridCachePartitionedNodeRestartTest.testRestartWithTxEightNodesTwoBackups -
0,0% fails in last 100 master runs.
{panel}
[TeamCity Run
All|http://ci.ignite.apache.org/viewLog.html?buildId=1931985buildTypeId=IgniteTests24Java8_RunAll]
> Exception during cluster state change terminates ExchangeWorker
> ---
>
> Key: IGNITE-9155
> URL: https://issues.apache.org/jira/browse/IGNITE-9155
> Project: Ignite
> Issue Type: Bug
>Reporter: Ilya Lantukh
>Assignee: Ilya Lantukh
>Priority: Major
>
> After IGNITE-8311 we throw an exception in ExchangeFuture instead swallowing
> it.
> ClusterStateChangeProcessor has it's own exception handling mechanism, which
> doesn't require ExchangeWorker termination (and leaving node in broken state).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9723) Critial section should be wrapped correctly for GridCacheDatabaseSharedManager
[
https://issues.apache.org/jira/browse/IGNITE-9723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631749#comment-16631749
]
Ignite TC Bot commented on IGNITE-9723:
---
{panel:title=Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}JDBC Driver{color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=1959421]]
{color:#d04437}MVCC Queries{color} [[tests
2|https://ci.ignite.apache.org/viewLog.html?buildId=1959494]]
* IgniteCacheMvccSqlTestSuite:
CacheMvccPartitionedSelectForUpdateQueryTest.testSelectForUpdateAfterAbortedTx
- 3,2% fails in last 100 master runs.
{color:#d04437}MVCC Cache{color} [[tests
4|https://ci.ignite.apache.org/viewLog.html?buildId=1959492]]
* IgniteCacheMvccTestSuite: CacheMvccSizeWithConcurrentTransactionTest.testSize
- 2,9% fails in last 100 master runs.
* IgniteCacheMvccTestSuite:
CacheMvccPartitionedCoordinatorFailoverTest.testGetReadInProgressCoordinatorFails_ReadDelay
- 0,0% fails in last 100 master runs.
{color:#d04437}Cache 2{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=1959473]]
* IgniteCacheTestSuite2:
GridNearCacheStoreUpdateTest.testTransactionAtomicUpdateNear - 0,0% fails in
last 100 master runs.
{color:#d04437}Cache 7 (With Persistence){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=1959480]]
* IgniteCacheTestSuite7:
TransactionIntegrityWithPrimaryIndexCorruptionTest.testPrimaryIndexCorruptionDuringCommitOnPrimaryNode2
- 0,0% fails in last 100 master runs.
{color:#d04437}SPI (URI Deploy){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=1959430]]
* IgniteUriDeploymentTestSuite:
GridUriDeploymentFileProcessorSelfTest.testTaskCorrect - 0,0% fails in last 100
master runs.
{panel}
[TeamCity Run
All|http://ci.ignite.apache.org/viewLog.html?buildId=1959495buildTypeId=IgniteTests24Java8_RunAll]
> Critial section should be wrapped correctly for GridCacheDatabaseSharedManager
> --
>
> Key: IGNITE-9723
> URL: https://issues.apache.org/jira/browse/IGNITE-9723
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.6
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
>
> `blockingSectionBegin` should be used here
> {code:java|title=GridCachePartitionExchangeManager:2687}
> blockingSectionEnd();
> try {
> resVer = exchFut.get(exchTimeout, TimeUnit.MILLISECONDS);
> } finally {
> blockingSectionEnd();
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9272) PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace.
[
https://issues.apache.org/jira/browse/IGNITE-9272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631745#comment-16631745
]
Stanilovsky Evgeny commented on IGNITE-9272:
yep, 25% i mentioned above, it`s our inner benchmarks, yardstick shows:
atomic-get-and-put 6000s duration.
Throughput (more - better).
{code:java}
ignite-9272:
65530.42
65523.72
65536.26
65487.34
65811.76
65202.05
65372.89
65627.32
master:
62740.62
62772.30
62928.18
62989.85
62926.43
62668.95
63026.49
62885.89
{code}
= 4-5% profit, thoughts?
> PureJavaCrc32 vs j.u.zip.CRC32 benchmark and probably replace.
> --
>
> Key: IGNITE-9272
> URL: https://issues.apache.org/jira/browse/IGNITE-9272
> Project: Ignite
> Issue Type: Improvement
> Components: general
>Affects Versions: 2.6
>Reporter: Stanilovsky Evgeny
>Assignee: Stanilovsky Evgeny
>Priority: Major
> Fix For: 2.7
>
> Attachments: BenchmarkCRC.java
>
>
> I see that Ignite has its own crc32 realization called: PureJavaCrc32 and
> from desc it seems to be : _The current version is ~10x to 1.8x as fast as
> Sun's native java.util.zip.CRC32 in Java 1.6_ But my jmh tests show opposite
> results.
> + If it really so, looks like backward compatibility would be easy, all that
> need is just to take lower part of long form zip.crc32 realization.
> jmh results:
> Benchmark Mode CntScoreError Units
> BenchmarkCRC.Crc32 avgt5 1521060.716 ± 44083.424 ns/op
> BenchmarkCRC.pureJavaCrc32 avgt5 4657756.671 ± 177243.254 ns/op
> JMH version: 1.21
> VM version: JDK 1.8.0_131, Java HotSpot(TM) 64-Bit Server VM, 25.131-b11
> VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java
> op system : ubuntu 16.10
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9658) Add ability to disable memory deallocation on deactivation.
[ https://issues.apache.org/jira/browse/IGNITE-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631735#comment-16631735 ] Ivan Daschinskiy commented on IGNITE-9658: -- [~ascherbakov] Please, could you take a look at some suspicious failures in TC? > Add ability to disable memory deallocation on deactivation. > --- > > Key: IGNITE-9658 > URL: https://issues.apache.org/jira/browse/IGNITE-9658 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Daschinskiy >Assignee: Alexei Scherbakov >Priority: Major > Fix For: 2.8 > > > Currently, in some systems (i.e. RHEL 7.4), we can see, that during massive > UNSAFE.freeMemory process freezes. This behaviour can lead to SEGMENTATION of > node, especcially when ZookeeperDiscoverySPI is used. There should be an > abillity to disable memory deallocation during deactivation of cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-9712) SQL: Select query benchmarks suite
[ https://issues.apache.org/jira/browse/IGNITE-9712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631736#comment-16631736 ] Pavel Kuznetsov commented on IGNITE-9712: - I've updated existing data model to support indexes. Have updated SELECT benchmark to support indexed value field in WHERE clause (in addition to existing queries which use PK field). [~tledkov-gridgain] Could you please take a look at the patch? > SQL: Select query benchmarks suite > -- > > Key: IGNITE-9712 > URL: https://issues.apache.org/jira/browse/IGNITE-9712 > Project: Ignite > Issue Type: Task >Reporter: Pavel Kuznetsov >Assignee: Pavel Kuznetsov >Priority: Major > > We need to create benchmark suite for select operations. > We already have benchmarks for selects, so we need to update this code to > support > 1) select the single row with by PK; indexed field > 2) select the range of rows filtered with PK; indexed field > Selects by non-indexed fields are out of scope. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-9658) Add ability to disable memory deallocation on deactivation.
[ https://issues.apache.org/jira/browse/IGNITE-9658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Daschinskiy reassigned IGNITE-9658: Assignee: Alexei Scherbakov (was: Ivan Daschinskiy) > Add ability to disable memory deallocation on deactivation. > --- > > Key: IGNITE-9658 > URL: https://issues.apache.org/jira/browse/IGNITE-9658 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Daschinskiy >Assignee: Alexei Scherbakov >Priority: Major > Fix For: 2.8 > > > Currently, in some systems (i.e. RHEL 7.4), we can see, that during massive > UNSAFE.freeMemory process freezes. This behaviour can lead to SEGMENTATION of > node, especcially when ZookeeperDiscoverySPI is used. There should be an > abillity to disable memory deallocation during deactivation of cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8006) Starting multiple caches slows down exchange process on joining node
[
https://issues.apache.org/jira/browse/IGNITE-8006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16631732#comment-16631732
]
Anton Kalashnikov commented on IGNITE-8006:
---
[~Jokser], could you please get a look this changes.
> Starting multiple caches slows down exchange process on joining node
>
>
> Key: IGNITE-8006
> URL: https://issues.apache.org/jira/browse/IGNITE-8006
> Project: Ignite
> Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Anton Kalashnikov
>Priority: Major
> Fix For: 2.8
>
>
> In some cases when we starts multiple caches (over 2K caches), we can get a
> stop on exchange when new node joining to the cluster.
> Coordinator-node wait to receive a single message from all other nodes, but
> last node (which want to joining to the cluster) stopped on starting caches:
>
> {noformat}
> Stack trace
> at java.lang.Thread.dumpStack(Thread.java:1329)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1159)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1900)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:1764)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:740)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:622)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2329)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> It blocks cluster exchange process until all caches started on the last node.
> We should start caches in parallel threads or exclude the action from
> exchange init process.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
