[jira] [Created] (IGNITE-12413) .NET: XMLDoc does not work when using Ignite NuGet from .NET Core
Pavel Tupitsyn created IGNITE-12413: --- Summary: .NET: XMLDoc does not work when using Ignite NuGet from .NET Core Key: IGNITE-12413 URL: https://issues.apache.org/jira/browse/IGNITE-12413 Project: Ignite Issue Type: Bug Components: platforms Affects Versions: 2.8 Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 2.8 * Create new .NET Core project (2.x or 3.x): dotnet new console * Install nightly build of 2.8.0: dotnet add package Apache.Ignite -v 2.8.0-alpha20191118 * Open the project in any IDE (VS, VSCode, Rider) and start using Ignite APIs There is no documentation in IDE tooltips. NuGet package is malformed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[
https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986677#comment-16986677
]
Stanilovsky Evgeny edited comment on IGNITE-12374 at 12/3/19 7:27 AM:
--
simple tuning give me perf boost:
===
NumPartial: 50
NumCompleteRec: 2000
TolRec: 10
Duration: 53 seconds
TPS: 1887
===
what i do:
1. change your config, will attach it (my-config-zstan.xml)
2. simple GC tuning: ignite.sh changing :
{code:java}
if [ $version -eq 8 ] ; then
JVM_OPTS="\
-Xmx4g -Xmx4g -XX:+UseG1GC \
${JVM_OPTS}"
{code}
i hope its enough for u ?
further steps : remove bigdecimal, use index only with long or some simple
types, tune your network (my servers are all bare with real 10G network), ones
more read perf tuning documentation.
is it ok ?
was (Author: zstan):
simple tuning give me perf boost:
===
NumPartial: 50
NumCompleteRec: 2000
TolRec: 10
Duration: 53 seconds
TPS: 1887
===
what i do:
1. change your config, will attach it (my-config-zstan.xml)
2. simple GC tuning: ignite.sh changing :
{code:java}
if [ $version -eq 8 ] ; then
JVM_OPTS="\
-Xmx4g -Xmx4g -XX:+UseG1GC \
${JVM_OPTS}"
{code}
i hope its enough for u ?
> Too low performance ~200TPS for single ODBC client
> --
>
> Key: IGNITE-12374
> URL: https://issues.apache.org/jira/browse/IGNITE-12374
> Project: Ignite
> Issue Type: Improvement
> Components: clients, odbc
>Affects Versions: 2.7.5
> Environment: Ignite server run on top of Kubernetes, with 2 server
> nodes, persistence enabled. Both CPU and RAM at server/client server is
> sufficient according to system reports.
>Reporter: swy
>Priority: Major
> Attachments: ignite-9bb53f34.0.log,
> ignite-logs-config-source-20191128.zip,
> ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample,
> odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png,
> profiling03.png, profling02.png, screenshot-1.png,
> snapshot-1574845275597.nps, threaddump-1573207804944.tdump,
> threaddump-1574845414243.tdump, values.yaml
>
>
> Hi, in our test ignite performance with ODBC connection is too bad to proceed
> with product integration. It is about ~200 TPS, each transaction with
> select+update operation.
> Please refer to attach sample program. It is just a simple test case.
> Based on the profiling most of the time consumed by sql execution. Please
> advice if the application did not do the right thing.
> Thank you.
> local ignite server, but the result same to remote container Ignite
> deployment too.
> cat /etc/apache-ignite/my-config.xml
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd;>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration"/>
>
>
>
>
>
> class="org.apache.ignite.binary.BinaryBasicIdMapper">
>
>
>
>
>
>
>
> g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include
> -I./ignite/common/os/linux/include -I./ignite/common/os/win/include
> -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample
> -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[
https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986677#comment-16986677
]
Stanilovsky Evgeny edited comment on IGNITE-12374 at 12/3/19 7:25 AM:
--
simple tuning give me perf boost:
===
NumPartial: 50
NumCompleteRec: 2000
TolRec: 10
Duration: 53 seconds
TPS: 1887
===
what i do:
1. change your config, will attach it (my-config-zstan.xml)
2. simple GC tuning: ignite.sh changing :
{code:java}
if [ $version -eq 8 ] ; then
JVM_OPTS="\
-Xmx4g -Xmx4g -XX:+UseG1GC \
${JVM_OPTS}"
{code}
i hope its enough for u ?
was (Author: zstan):
simple tuning give me perf boost:
===
NumPartial: 50
NumCompleteRec: 2000
TolRec: 10
Duration: 53 seconds
TPS: 1887
===
what i do:
1. change your config, will attach it
2. simple GC tuning: ignite.sh changing :
{code:java}
if [ $version -eq 8 ] ; then
JVM_OPTS="\
-Xmx4g -Xmx4g -XX:+UseG1GC \
${JVM_OPTS}"
{code}
i hope its enough for u ?
> Too low performance ~200TPS for single ODBC client
> --
>
> Key: IGNITE-12374
> URL: https://issues.apache.org/jira/browse/IGNITE-12374
> Project: Ignite
> Issue Type: Improvement
> Components: clients, odbc
>Affects Versions: 2.7.5
> Environment: Ignite server run on top of Kubernetes, with 2 server
> nodes, persistence enabled. Both CPU and RAM at server/client server is
> sufficient according to system reports.
>Reporter: swy
>Priority: Major
> Attachments: ignite-9bb53f34.0.log,
> ignite-logs-config-source-20191128.zip,
> ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample,
> odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png,
> profiling03.png, profling02.png, screenshot-1.png,
> snapshot-1574845275597.nps, threaddump-1573207804944.tdump,
> threaddump-1574845414243.tdump, values.yaml
>
>
> Hi, in our test ignite performance with ODBC connection is too bad to proceed
> with product integration. It is about ~200 TPS, each transaction with
> select+update operation.
> Please refer to attach sample program. It is just a simple test case.
> Based on the profiling most of the time consumed by sql execution. Please
> advice if the application did not do the right thing.
> Thank you.
> local ignite server, but the result same to remote container Ignite
> deployment too.
> cat /etc/apache-ignite/my-config.xml
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd;>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration"/>
>
>
>
>
>
> class="org.apache.ignite.binary.BinaryBasicIdMapper">
>
>
>
>
>
>
>
> g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include
> -I./ignite/common/os/linux/include -I./ignite/common/os/win/include
> -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample
> -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanilovsky Evgeny updated IGNITE-12374: Attachment: my-config-zstan.xml > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, my-config-zstan.xml, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[
https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986677#comment-16986677
]
Stanilovsky Evgeny commented on IGNITE-12374:
-
simple tuning give me perf boost:
===
NumPartial: 50
NumCompleteRec: 2000
TolRec: 10
Duration: 53 seconds
TPS: 1887
===
what i do:
1. change your config, will attach it
2. simple GC tuning: ignite.sh changing :
{code:java}
if [ $version -eq 8 ] ; then
JVM_OPTS="\
-Xmx4g -Xmx4g -XX:+UseG1GC \
${JVM_OPTS}"
{code}
i hope its enough for u ?
> Too low performance ~200TPS for single ODBC client
> --
>
> Key: IGNITE-12374
> URL: https://issues.apache.org/jira/browse/IGNITE-12374
> Project: Ignite
> Issue Type: Improvement
> Components: clients, odbc
>Affects Versions: 2.7.5
> Environment: Ignite server run on top of Kubernetes, with 2 server
> nodes, persistence enabled. Both CPU and RAM at server/client server is
> sufficient according to system reports.
>Reporter: swy
>Priority: Major
> Attachments: ignite-9bb53f34.0.log,
> ignite-logs-config-source-20191128.zip,
> ignite-logs-config-source-20191129.zip, odbcsample,
> odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png,
> profiling03.png, profling02.png, screenshot-1.png,
> snapshot-1574845275597.nps, threaddump-1573207804944.tdump,
> threaddump-1574845414243.tdump, values.yaml
>
>
> Hi, in our test ignite performance with ODBC connection is too bad to proceed
> with product integration. It is about ~200 TPS, each transaction with
> select+update operation.
> Please refer to attach sample program. It is just a simple test case.
> Based on the profiling most of the time consumed by sql execution. Please
> advice if the application did not do the right thing.
> Thank you.
> local ignite server, but the result same to remote container Ignite
> deployment too.
> cat /etc/apache-ignite/my-config.xml
>
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="http://www.springframework.org/schema/beans
> http://www.springframework.org/schema/beans/spring-beans.xsd;>
> class="org.apache.ignite.configuration.IgniteConfiguration">
>
> class="org.apache.ignite.configuration.ClientConnectorConfiguration"/>
>
>
>
>
>
> class="org.apache.ignite.binary.BinaryBasicIdMapper">
>
>
>
>
>
>
>
> g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include
> -I./ignite/common/os/linux/include -I./ignite/common/os/win/include
> -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample
> -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986673#comment-16986673 ] Stanilovsky Evgeny commented on IGNITE-12374: - initial run with your "my-config": [10:08:41] Ignite node started OK (id=fb9bdbf8) [10:08:41] Topology snapshot [ver=7, locNode=fb9bdbf8, servers=3, clients=0, state=ACTIVE, CPUs=48, offheap=57.0GB, heap=63.0GB] [10:08:41] ^-- Baseline [id=0, size=3, online=3, offline=0] [estanilovskiy@lab34 ~]$ ./run.sh 50 2000 [10:08:51] New version is available at gridgain.com: 8.7.7 top === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 74 seconds TPS: 1351 === i will try to tune it now. 2x Xeon X5570 96Gb 512GB SSD 2048GB HDD 10GB/s uname -a Linux lab34.me.local 3.10.0-862.2.3.el7.x86_64 #1 SMP Wed May 9 18:05:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986591#comment-16986591 ] swy commented on IGNITE-12374: -- 'long' should be enough. By using "PRIMARY_SYNC" performance is slightly better, but still poor.. === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 148 seconds TPS: 676 === > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11942) IGFS and Hadoop Accelerator Discontinuation
[ https://issues.apache.org/jira/browse/IGNITE-11942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986346#comment-16986346 ] Denis A. Magda commented on IGNITE-11942: - [~zaleslaw], [~mmuzaf], ok folks, let's move to the next release. > IGFS and Hadoop Accelerator Discontinuation > --- > > Key: IGNITE-11942 > URL: https://issues.apache.org/jira/browse/IGNITE-11942 > Project: Ignite > Issue Type: Task >Reporter: Denis A. Magda >Priority: Blocker > Fix For: 2.9 > > > The community has voted for the following decision: > * IGFS and In-Memory Hadoop Accelerator components are to be discontinued and > no longer supported by the community > * The existing source code of IGFS and In-Memory Hadoop Accelerator is to be > removed from Ignite master. Before that, a special branch like > "ignite-igfs-and-hadoop-accelerator" to be forked off the master in order to > preserve the sources in Git history for those who might need it. > The voting thread: > http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html > Once the changes are made for Ignite 2.8, please contact Denis Magda to > update a public documentation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11410) Sandbox for user-defined code
[
https://issues.apache.org/jira/browse/IGNITE-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986284#comment-16986284
]
Ignite TC Bot commented on IGNITE-11410:
{panel:title=Branch: [pull/6707/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4801905buildTypeId=IgniteTests24Java8_RunAll]
> Sandbox for user-defined code
> -
>
> Key: IGNITE-11410
> URL: https://issues.apache.org/jira/browse/IGNITE-11410
> Project: Ignite
> Issue Type: New Feature
>Reporter: Anton Vinogradov
>Assignee: Denis Garus
>Priority: Critical
> Labels: iep-38
> Fix For: 2.8
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> We should provide a restricted environment (sandbox) in which to run
> user-defined code securely. To get it done, we would use the java sandbox
> model.
> The java sandbox model allows restricting access from user-defined code to
> the system resources or security-sensitive feature of java, for example,
> reflection.
> The user-defined code contains:
> - StreamReceiver for DataStreamer:
> - EntryProcessor;
> - ComputeJob;
> - filter and transformer for ScanQuery.
> The user-defined code will get permissions from GridSecuerityProcessor
> (security plugin).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12188) Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly if there is more then one cache in cache group
[
https://issues.apache.org/jira/browse/IGNITE-12188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986211#comment-16986211
]
Amelchev Nikita edited comment on IGNITE-12188 at 12/2/19 5:26 PM:
---
[~alex_pl], HI, thanks for taking a look.
Yes, that didn't work before. Moreover, this metric will not be reseted when
{{parallelism lvl=1}} or {{thread idx=0}}, see
{{SchemaIndexCacheVisitorImpl#visit}} (There is no handler for exceptions from
{{processPartitions}} in init thread).
Should we fix it in this issue?
was (Author: nsamelchev):
[~alex_pl], HI, thanks for taking a look.
Yes, that didn't work before. Moreover, this metric will not be reseted when
{{parallelism lvl=1}} or {{thread idx=0}}, see
{{SchemaIndexCacheVisitorImpl#visit}} (There is no handler for exceptions from
{{processPartitions}} in init thread).
Should we fix this in this issue?
> Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly
> if there is more then one cache in cache group
>
>
> Key: IGNITE-12188
> URL: https://issues.apache.org/jira/browse/IGNITE-12188
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Amelchev Nikita
>Priority: Major
> Labels: IEP-35, await
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Initial partitions counter is set to a total number of partitions for cache
> group but decremented each time partition processed for each cache.
> Reproducer:
> {code:java}
> @Test
> public void testIndexRebuildMetrics() throws Exception {
> IgniteEx ignite = startGrid(new IgniteConfiguration()
> .setConsistentId("ignite")
> .setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new
> DataRegionConfiguration().setPersistenceEnabled(true)))
> .setCacheConfiguration(
> new CacheConfiguration Integer>("c1").setGroupName("grp").setIndexedTypes(Integer.class,
> Integer.class),
> new CacheConfiguration Integer>("c2").setGroupName("grp").setIndexedTypes(Integer.class,
> Integer.class)
> ));
> ignite.cluster().active(true);
> for (int i = 0; i < 10_000; i++)
> ignite.cache("c1").put(i, i);
> ignite.cluster().active(false);
> File dir = U.resolveWorkDirectory(U.defaultWorkDirectory(),
> DFLT_STORE_DIR, false);
> U.delete(new File(dir, "ignite/cacheGroup-grp/index.bin"));
> ignite.cluster().active(true);
> doSleep(1_000L);
>
> assertTrue(ignite.context().cache().cache("c1").context().group().metrics().getIndexBuildCountPartitionsLeft()
> >= 0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12188) Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly if there is more then one cache in cache group
[
https://issues.apache.org/jira/browse/IGNITE-12188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986211#comment-16986211
]
Amelchev Nikita commented on IGNITE-12188:
--
[~alex_pl], HI, thanks for taking a look.
Yes, that didn't work before. Moreover, this metric will not be reseted when
{{parallelism lvl=1}} or {{thread idx=0}}, see
{{SchemaIndexCacheVisitorImpl#visit}} (There is no handler for exceptions from
{{processPartitions}} in init thread).
Should we fix this in this issue?
> Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly
> if there is more then one cache in cache group
>
>
> Key: IGNITE-12188
> URL: https://issues.apache.org/jira/browse/IGNITE-12188
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Amelchev Nikita
>Priority: Major
> Labels: IEP-35, await
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Initial partitions counter is set to a total number of partitions for cache
> group but decremented each time partition processed for each cache.
> Reproducer:
> {code:java}
> @Test
> public void testIndexRebuildMetrics() throws Exception {
> IgniteEx ignite = startGrid(new IgniteConfiguration()
> .setConsistentId("ignite")
> .setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new
> DataRegionConfiguration().setPersistenceEnabled(true)))
> .setCacheConfiguration(
> new CacheConfiguration Integer>("c1").setGroupName("grp").setIndexedTypes(Integer.class,
> Integer.class),
> new CacheConfiguration Integer>("c2").setGroupName("grp").setIndexedTypes(Integer.class,
> Integer.class)
> ));
> ignite.cluster().active(true);
> for (int i = 0; i < 10_000; i++)
> ignite.cache("c1").put(i, i);
> ignite.cluster().active(false);
> File dir = U.resolveWorkDirectory(U.defaultWorkDirectory(),
> DFLT_STORE_DIR, false);
> U.delete(new File(dir, "ignite/cacheGroup-grp/index.bin"));
> ignite.cluster().active(true);
> doSleep(1_000L);
>
> assertTrue(ignite.context().cache().cache("c1").context().group().metrics().getIndexBuildCountPartitionsLeft()
> >= 0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12412) Incomplete check for ABA problem in PageMemoryImpl#PagePool
[
https://issues.apache.org/jira/browse/IGNITE-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986209#comment-16986209
]
Alexey Goncharuk commented on IGNITE-12412:
---
The test alongside with a small refactoring required to create the test are in
the attached PR.
> Incomplete check for ABA problem in PageMemoryImpl#PagePool
> ---
>
> Key: IGNITE-12412
> URL: https://issues.apache.org/jira/browse/IGNITE-12412
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In current implementation, {{PagePool#releasePage}} clears the counter part
> of the returned page ID, which effectively disables the ABA check intended in
> the class. This issue can be rarely reproduced on zOS during checkpoints
> (when pages are being taken and returned to the checkpoint pages pool).
> I managed to write a unit-test to reproduce this issue on x86.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12412) Incomplete check for ABA problem in PageMemoryImpl#PagePool
Alexey Goncharuk created IGNITE-12412:
-
Summary: Incomplete check for ABA problem in
PageMemoryImpl#PagePool
Key: IGNITE-12412
URL: https://issues.apache.org/jira/browse/IGNITE-12412
Project: Ignite
Issue Type: Improvement
Reporter: Alexey Goncharuk
Assignee: Alexey Goncharuk
In current implementation, {{PagePool#releasePage}} clears the counter part of
the returned page ID, which effectively disables the ABA check intended in the
class. This issue can be rarely reproduced on zOS during checkpoints (when
pages are being taken and returned to the checkpoint pages pool).
I managed to write a unit-test to reproduce this issue on x86.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12412) Incomplete check for ABA problem in PageMemoryImpl#PagePool
[
https://issues.apache.org/jira/browse/IGNITE-12412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12412:
--
Ignite Flags: Release Notes Required (was: Docs Required,Release Notes
Required)
> Incomplete check for ABA problem in PageMemoryImpl#PagePool
> ---
>
> Key: IGNITE-12412
> URL: https://issues.apache.org/jira/browse/IGNITE-12412
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Alexey Goncharuk
>Priority: Critical
>
> In current implementation, {{PagePool#releasePage}} clears the counter part
> of the returned page ID, which effectively disables the ABA check intended in
> the class. This issue can be rarely reproduced on zOS during checkpoints
> (when pages are being taken and returned to the checkpoint pages pool).
> I managed to write a unit-test to reproduce this issue on x86.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12405) .NET: WithReadRepair does not work
[ https://issues.apache.org/jira/browse/IGNITE-12405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986141#comment-16986141 ] Pavel Tupitsyn commented on IGNITE-12405: - Merged to master: a5bc7283b6829dfa7f23b6a94c28bd95f7f6af70 > .NET: WithReadRepair does not work > -- > > Key: IGNITE-12405 > URL: https://issues.apache.org/jira/browse/IGNITE-12405 > Project: Ignite > Issue Type: Bug > Components: platforms >Affects Versions: 2.8 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Blocker > Labels: .NET > Fix For: 2.8 > > Time Spent: 20m > Remaining Estimate: 0h > > WithReadRepair in .NET was partially implemented as part of IGNITE-10663, > however: > * It does not work due to missing Java part (no handling of > CacheOp.WithReadRepair in PlatformCache) > * There are no tests > This new API is not in any release yet, so we should either fix it or remove > from the public API. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12049) Allow custom authenticators to use SSL certificates
[ https://issues.apache.org/jira/browse/IGNITE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986102#comment-16986102 ] Alexei Scherbakov commented on IGNITE-12049: [~SomeFire] Sounds good. Attributes for jdbc/odbc can be passed as base64 encoded strings to driver, the factory is also fine. > Allow custom authenticators to use SSL certificates > --- > > Key: IGNITE-12049 > URL: https://issues.apache.org/jira/browse/IGNITE-12049 > Project: Ignite > Issue Type: Improvement >Reporter: Ryabov Dmitrii >Assignee: Ryabov Dmitrii >Priority: Minor > Time Spent: 1.5h > Remaining Estimate: 0h > > Add SSL certificates to AuthenticationContext, so, authenticators can make > additional checks based on SSL certificates. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11992) Improvements for new security approach
[ https://issues.apache.org/jira/browse/IGNITE-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986098#comment-16986098 ] Maxim Muzafarov commented on IGNITE-11992: -- Folks, [~mstepachev], [~garus.d.g] What we should do next? Who will review the issue? > Improvements for new security approach > -- > > Key: IGNITE-11992 > URL: https://issues.apache.org/jira/browse/IGNITE-11992 > Project: Ignite > Issue Type: Improvement > Components: security >Affects Versions: 2.8 >Reporter: Stepachev Maksim >Assignee: Stepachev Maksim >Priority: Major > Fix For: 2.8 > > Time Spent: 3.5h > Remaining Estimate: 0h > > 1. The visor tasks lost permission. > The method VisorQueryUtils#scheduleQueryStart makes a new thread and loses > context. > 2. The GridRestProcessor does tasks outside "withContext" section. As result > context loses. > In additional: > Change a java docs for TaskEvent, CacheEvent, CacheQueryExecutedEvent and > CacheQueryReadEvent. > "Gets security subject ID initiated this task event, if available. > This property is not available for GridEventType#EVT_TASK_SESSION_ATTR_SET > task event. > Subject ID will be set either to node ID or client ID initiated task > execution." > by: > "Gets security subject ID initiated this task event if IgniteSecurity is > enabled, otherwise returns null." > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-8617) Node Discovery Using AWS Application ELB
[ https://issues.apache.org/jira/browse/IGNITE-8617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-8617: Priority: Critical (was: Major) > Node Discovery Using AWS Application ELB > > > Key: IGNITE-8617 > URL: https://issues.apache.org/jira/browse/IGNITE-8617 > Project: Ignite > Issue Type: New Feature > Components: aws >Reporter: Uday Kale >Assignee: Uday Kale >Priority: Critical > Fix For: 2.8 > > > Support for Node discovery using AWS Application ELB. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12203) Rebalance is loading partitions already loading after cancellation without WAL
[ https://issues.apache.org/jira/browse/IGNITE-12203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-12203: - Fix Version/s: (was: 2.8) 2.9 > Rebalance is loading partitions already loading after cancellation without WAL > -- > > Key: IGNITE-12203 > URL: https://issues.apache.org/jira/browse/IGNITE-12203 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Assignee: Vladislav Pyatkov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > I have seen from log partition miss warnings and after that rebelance was > canceled and was forcibly restarted but already over another suppliers. In > case when added nodes without a storage in persistent cluster it can lead to > several times fully rebalance. It seem to bee because we have not updated > partitions state until rebalance finished. > Should to prevent partition eviction until rebalance on all nodes completed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-9720) Initialize partition free lists lazily
[
https://issues.apache.org/jira/browse/IGNITE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986096#comment-16986096
]
Maxim Muzafarov commented on IGNITE-9720:
-
Folks,
I've cancelled the patch. PR is outdated.
Please, feel free to set the PA status again when the issue will be ready for
review.
> Initialize partition free lists lazily
> --
>
> Key: IGNITE-9720
> URL: https://issues.apache.org/jira/browse/IGNITE-9720
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Goncharuk
>Assignee: Semen Boikov
>Priority: Major
> Labels: performance
> Fix For: 2.8
>
>
> When persistence is enabled, partition free lists metadata may take quite a
> lot of pages.
> This results in a very long start time because
> {{GridCacheOffheapManager.GridCacheDataStore#init0}} will read all metadata
> for free list in each partition on exchange start (this is done in the
> {{CacheFreeListImpl}} constructor)
> We should only read required information on exchange and defer actual free
> list initialization to the first access.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12246) [Spark] Add support of changes in External Catalog
[ https://issues.apache.org/jira/browse/IGNITE-12246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12246: - Labels: (was: await) > [Spark] Add support of changes in External Catalog > -- > > Key: IGNITE-12246 > URL: https://issues.apache.org/jira/browse/IGNITE-12246 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.8 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Major > Fix For: 2.8 > > > It leads to problems with schemas > > For example, next tests are muted now in Spark 2.4 > "Additional features for IgniteSparkSession" > and > "Should allow Spark SQL to create a table" > "Should disallow creation of tables in non-PUBLIC schemas" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12244) [Spark] Fix test with Multiple Joins
[ https://issues.apache.org/jira/browse/IGNITE-12244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12244: - Affects Version/s: (was: 2.8) 2.9 > [Spark] Fix test with Multiple Joins > > > Key: IGNITE-12244 > URL: https://issues.apache.org/jira/browse/IGNITE-12244 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Minor > Labels: await > Fix For: 2.8 > > > Fix test "JOIN 3 TABLE" after investigation in strange join plan generation > in spark 2.4. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12245) [Spark] Support of Null Handling in JOIN condition
[
https://issues.apache.org/jira/browse/IGNITE-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Zinoviev updated IGNITE-12245:
-
Fix Version/s: (was: 2.8)
2.9
> [Spark] Support of Null Handling in JOIN condition
> --
>
> Key: IGNITE-12245
> URL: https://issues.apache.org/jira/browse/IGNITE-12245
> Project: Ignite
> Issue Type: Sub-task
> Components: spark
>Affects Versions: 2.9
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Major
> Fix For: 2.9
>
>
> Also, in Spark was fixed bug with incorrect null handling on columns in
> codition
> https://issues.apache.org/jira/browse/SPARK-21479
> It leads to IgniteOptimizationJoinSpec fixes (the same thing was in the
> previous migration from 2.2 to 2.3)
>
> Also, in Spark was fixed bug with incorrect null handling on columns in
> codition
> https://issues.apache.org/jira/browse/SPARK-21479
> It leads to IgniteOptimizationJoinSpec fixes (the same thing was in the
> previous migration from 2.2 to 2.3)
>
>
> I made experiment with Spark code for version 2.3 it generates the next plan
> == Parsed Logical Plan ==
> 'Project 'jt1.id AS id1#28, 'jt1.val1, 'jt2.id AS id2#29, 'jt2.val2
> +- 'Join LeftOuter, ('jt1.val1 = 'jt2.val2)
> :- 'UnresolvedRelation `jt1`
> +- 'UnresolvedRelation `jt2`
> == Analyzed Logical Plan ==
> id1: string, val1: string, id2: string, val2: string
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- SubqueryAlias jt1
> : +- Relationid#10,val1#11 csv
> +- SubqueryAlias jt2
> +- Relationid#24,val2#25 csv
> == Optimized Logical Plan ==
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- Relationid#10,val1#11 csv
> +- Relationid#24,val2#25 csv
>
> The 2.4 generates
> == Parsed Logical Plan ==
> 'Project 'jt1.id AS id1#28, 'jt1.val1, 'jt2.id AS id2#29, 'jt2.val2
> +- 'Join LeftOuter, ('jt1.val1 = 'jt2.val2)
> :- 'UnresolvedRelation `jt1`
> +- 'UnresolvedRelation `jt2`
> == Analyzed Logical Plan ==
> id1: string, val1: string, id2: string, val2: string
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- SubqueryAlias `jt1`
> : +- Relationid#10,val1#11 csv
> +- SubqueryAlias `jt2`
> +- Relationid#24,val2#25 csv
> == Optimized Logical Plan ==
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- Relationid#10,val1#11 csv
> +- Filter isnotnull(val2#25)
> +- Relationid#24,val2#25 csv
>
> The +- Filter isnotnull(val2#25) is added in optimized logical plan
> But in reality it doesn't work properly (and doesn't filter in Spark), but
> wow! It works for Ignite (because we honestly work with Spark plan)
>
> If you enable next option
> .config("ignite.disableSparkSQLOptimization", "true") - the behaviour will be
> the same in Ignite and Spark and will not add the filter
>
> The best approach for Spark 2.4 - disableSparkOptimization before fixing on
> Spark side (I could create a bug for this)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12244) [Spark] Fix test with Multiple Joins
[ https://issues.apache.org/jira/browse/IGNITE-12244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12244: - Fix Version/s: (was: 2.8) 2.9 > [Spark] Fix test with Multiple Joins > > > Key: IGNITE-12244 > URL: https://issues.apache.org/jira/browse/IGNITE-12244 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Minor > Fix For: 2.9 > > > Fix test "JOIN 3 TABLE" after investigation in strange join plan generation > in spark 2.4. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12246) [Spark] Add support of changes in External Catalog
[ https://issues.apache.org/jira/browse/IGNITE-12246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12246: - Fix Version/s: (was: 2.8) 2.9 > [Spark] Add support of changes in External Catalog > -- > > Key: IGNITE-12246 > URL: https://issues.apache.org/jira/browse/IGNITE-12246 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.8 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Major > Fix For: 2.9 > > > It leads to problems with schemas > > For example, next tests are muted now in Spark 2.4 > "Additional features for IgniteSparkSession" > and > "Should allow Spark SQL to create a table" > "Should disallow creation of tables in non-PUBLIC schemas" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12245) [Spark] Support of Null Handling in JOIN condition
[
https://issues.apache.org/jira/browse/IGNITE-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Zinoviev updated IGNITE-12245:
-
Affects Version/s: (was: 2.8)
2.9
> [Spark] Support of Null Handling in JOIN condition
> --
>
> Key: IGNITE-12245
> URL: https://issues.apache.org/jira/browse/IGNITE-12245
> Project: Ignite
> Issue Type: Sub-task
> Components: spark
>Affects Versions: 2.9
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Major
> Labels: await
> Fix For: 2.8
>
>
> Also, in Spark was fixed bug with incorrect null handling on columns in
> codition
> https://issues.apache.org/jira/browse/SPARK-21479
> It leads to IgniteOptimizationJoinSpec fixes (the same thing was in the
> previous migration from 2.2 to 2.3)
>
> Also, in Spark was fixed bug with incorrect null handling on columns in
> codition
> https://issues.apache.org/jira/browse/SPARK-21479
> It leads to IgniteOptimizationJoinSpec fixes (the same thing was in the
> previous migration from 2.2 to 2.3)
>
>
> I made experiment with Spark code for version 2.3 it generates the next plan
> == Parsed Logical Plan ==
> 'Project 'jt1.id AS id1#28, 'jt1.val1, 'jt2.id AS id2#29, 'jt2.val2
> +- 'Join LeftOuter, ('jt1.val1 = 'jt2.val2)
> :- 'UnresolvedRelation `jt1`
> +- 'UnresolvedRelation `jt2`
> == Analyzed Logical Plan ==
> id1: string, val1: string, id2: string, val2: string
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- SubqueryAlias jt1
> : +- Relationid#10,val1#11 csv
> +- SubqueryAlias jt2
> +- Relationid#24,val2#25 csv
> == Optimized Logical Plan ==
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- Relationid#10,val1#11 csv
> +- Relationid#24,val2#25 csv
>
> The 2.4 generates
> == Parsed Logical Plan ==
> 'Project 'jt1.id AS id1#28, 'jt1.val1, 'jt2.id AS id2#29, 'jt2.val2
> +- 'Join LeftOuter, ('jt1.val1 = 'jt2.val2)
> :- 'UnresolvedRelation `jt1`
> +- 'UnresolvedRelation `jt2`
> == Analyzed Logical Plan ==
> id1: string, val1: string, id2: string, val2: string
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- SubqueryAlias `jt1`
> : +- Relationid#10,val1#11 csv
> +- SubqueryAlias `jt2`
> +- Relationid#24,val2#25 csv
> == Optimized Logical Plan ==
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- Relationid#10,val1#11 csv
> +- Filter isnotnull(val2#25)
> +- Relationid#24,val2#25 csv
>
> The +- Filter isnotnull(val2#25) is added in optimized logical plan
> But in reality it doesn't work properly (and doesn't filter in Spark), but
> wow! It works for Ignite (because we honestly work with Spark plan)
>
> If you enable next option
> .config("ignite.disableSparkSQLOptimization", "true") - the behaviour will be
> the same in Ignite and Spark and will not add the filter
>
> The best approach for Spark 2.4 - disableSparkOptimization before fixing on
> Spark side (I could create a bug for this)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12245) [Spark] Support of Null Handling in JOIN condition
[
https://issues.apache.org/jira/browse/IGNITE-12245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Zinoviev updated IGNITE-12245:
-
Labels: (was: await)
> [Spark] Support of Null Handling in JOIN condition
> --
>
> Key: IGNITE-12245
> URL: https://issues.apache.org/jira/browse/IGNITE-12245
> Project: Ignite
> Issue Type: Sub-task
> Components: spark
>Affects Versions: 2.9
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Major
> Fix For: 2.8
>
>
> Also, in Spark was fixed bug with incorrect null handling on columns in
> codition
> https://issues.apache.org/jira/browse/SPARK-21479
> It leads to IgniteOptimizationJoinSpec fixes (the same thing was in the
> previous migration from 2.2 to 2.3)
>
> Also, in Spark was fixed bug with incorrect null handling on columns in
> codition
> https://issues.apache.org/jira/browse/SPARK-21479
> It leads to IgniteOptimizationJoinSpec fixes (the same thing was in the
> previous migration from 2.2 to 2.3)
>
>
> I made experiment with Spark code for version 2.3 it generates the next plan
> == Parsed Logical Plan ==
> 'Project 'jt1.id AS id1#28, 'jt1.val1, 'jt2.id AS id2#29, 'jt2.val2
> +- 'Join LeftOuter, ('jt1.val1 = 'jt2.val2)
> :- 'UnresolvedRelation `jt1`
> +- 'UnresolvedRelation `jt2`
> == Analyzed Logical Plan ==
> id1: string, val1: string, id2: string, val2: string
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- SubqueryAlias jt1
> : +- Relationid#10,val1#11 csv
> +- SubqueryAlias jt2
> +- Relationid#24,val2#25 csv
> == Optimized Logical Plan ==
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- Relationid#10,val1#11 csv
> +- Relationid#24,val2#25 csv
>
> The 2.4 generates
> == Parsed Logical Plan ==
> 'Project 'jt1.id AS id1#28, 'jt1.val1, 'jt2.id AS id2#29, 'jt2.val2
> +- 'Join LeftOuter, ('jt1.val1 = 'jt2.val2)
> :- 'UnresolvedRelation `jt1`
> +- 'UnresolvedRelation `jt2`
> == Analyzed Logical Plan ==
> id1: string, val1: string, id2: string, val2: string
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- SubqueryAlias `jt1`
> : +- Relationid#10,val1#11 csv
> +- SubqueryAlias `jt2`
> +- Relationid#24,val2#25 csv
> == Optimized Logical Plan ==
> Project id#10 AS id1#28, val1#11, id#24 AS id2#29, val2#25
> +- Join LeftOuter, (val1#11 = val2#25)
> :- Relationid#10,val1#11 csv
> +- Filter isnotnull(val2#25)
> +- Relationid#24,val2#25 csv
>
> The +- Filter isnotnull(val2#25) is added in optimized logical plan
> But in reality it doesn't work properly (and doesn't filter in Spark), but
> wow! It works for Ignite (because we honestly work with Spark plan)
>
> If you enable next option
> .config("ignite.disableSparkSQLOptimization", "true") - the behaviour will be
> the same in Ignite and Spark and will not add the filter
>
> The best approach for Spark 2.4 - disableSparkOptimization before fixing on
> Spark side (I could create a bug for this)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12405) .NET: WithReadRepair does not work
[ https://issues.apache.org/jira/browse/IGNITE-12405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986095#comment-16986095 ] Igor Sapego commented on IGNITE-12405: -- [~ptupitsyn] looks good to me. > .NET: WithReadRepair does not work > -- > > Key: IGNITE-12405 > URL: https://issues.apache.org/jira/browse/IGNITE-12405 > Project: Ignite > Issue Type: Bug > Components: platforms >Affects Versions: 2.8 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Blocker > Labels: .NET > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > WithReadRepair in .NET was partially implemented as part of IGNITE-10663, > however: > * It does not work due to missing Java part (no handling of > CacheOp.WithReadRepair in PlatformCache) > * There are no tests > This new API is not in any release yet, so we should either fix it or remove > from the public API. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12244) [Spark] Fix test with Multiple Joins
[ https://issues.apache.org/jira/browse/IGNITE-12244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12244: - Labels: (was: await) > [Spark] Fix test with Multiple Joins > > > Key: IGNITE-12244 > URL: https://issues.apache.org/jira/browse/IGNITE-12244 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Minor > Fix For: 2.8 > > > Fix test "JOIN 3 TABLE" after investigation in strange join plan generation > in spark 2.4. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12246) [Spark] Add support of changes in External Catalog
[ https://issues.apache.org/jira/browse/IGNITE-12246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12246: - Affects Version/s: (was: 2.8) 2.9 > [Spark] Add support of changes in External Catalog > -- > > Key: IGNITE-12246 > URL: https://issues.apache.org/jira/browse/IGNITE-12246 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Major > Fix For: 2.9 > > > It leads to problems with schemas > > For example, next tests are muted now in Spark 2.4 > "Additional features for IgniteSparkSession" > and > "Should allow Spark SQL to create a table" > "Should disallow creation of tables in non-PUBLIC schemas" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12243) [Spark] Add support of HAVING without GROUP BY
[ https://issues.apache.org/jira/browse/IGNITE-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986093#comment-16986093 ] Alexey Zinoviev commented on IGNITE-12243: -- [~mmuzaf] NO, it's for the next releases, I've moved to 2.9 > [Spark] Add support of HAVING without GROUP BY > -- > > Key: IGNITE-12243 > URL: https://issues.apache.org/jira/browse/IGNITE-12243 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 2.9 > > > Also the semantic of Having support was changed in Spark > [https://github.com/apache/spark/pull/22696/files] > https://issues.apache.org/jira/browse/SPARK-25708 > Now Having could be a legal operation without GroupBY > > Rewrite the test "SELECT id FROM city HAVING id > 1" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12243) [Spark] Add support of HAVING without GROUP BY
[ https://issues.apache.org/jira/browse/IGNITE-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12243: - Affects Version/s: (was: 2.8) 2.9 > [Spark] Add support of HAVING without GROUP BY > -- > > Key: IGNITE-12243 > URL: https://issues.apache.org/jira/browse/IGNITE-12243 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 2.8 > > > Also the semantic of Having support was changed in Spark > [https://github.com/apache/spark/pull/22696/files] > https://issues.apache.org/jira/browse/SPARK-25708 > Now Having could be a legal operation without GroupBY > > Rewrite the test "SELECT id FROM city HAVING id > 1" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12243) [Spark] Add support of HAVING without GROUP BY
[ https://issues.apache.org/jira/browse/IGNITE-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Zinoviev updated IGNITE-12243: - Fix Version/s: (was: 2.8) 2.9 > [Spark] Add support of HAVING without GROUP BY > -- > > Key: IGNITE-12243 > URL: https://issues.apache.org/jira/browse/IGNITE-12243 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 2.9 > > > Also the semantic of Having support was changed in Spark > [https://github.com/apache/spark/pull/22696/files] > https://issues.apache.org/jira/browse/SPARK-25708 > Now Having could be a legal operation without GroupBY > > Rewrite the test "SELECT id FROM city HAVING id > 1" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12049) Allow custom authenticators to use SSL certificates
[ https://issues.apache.org/jira/browse/IGNITE-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986090#comment-16986090 ] Ryabov Dmitrii commented on IGNITE-12049: - [~ascherbakov], I thought about this task and agree that passing certificates to node attributes is enough. For a common client, attributes can be configured in `ClientConfiguration`. For JDBC/ODBC, attributes can't be passed directly to the driver, so, I propose to pass a factory class name and create attributes inside the factory. Is it ok for you? > Allow custom authenticators to use SSL certificates > --- > > Key: IGNITE-12049 > URL: https://issues.apache.org/jira/browse/IGNITE-12049 > Project: Ignite > Issue Type: Improvement >Reporter: Ryabov Dmitrii >Assignee: Ryabov Dmitrii >Priority: Minor > Time Spent: 1.5h > Remaining Estimate: 0h > > Add SSL certificates to AuthenticationContext, so, authenticators can make > additional checks based on SSL certificates. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986085#comment-16986085 ] Stanilovsky Evgeny commented on IGNITE-12374: - ok, check it ! can you avoid of bigint usage ? may be long would be enough ? i would check multiple nodes usage. Check [1] may be WRITE_SYNCHRONIZATION_MODE=PRIMARY_SYNC grantee would be enough for you ? Otherwise looks like all your performance drop due to network hops... [1] https://apacheignite-sql.readme.io/docs/create-table > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986056#comment-16986056 ] swy commented on IGNITE-12374: -- [~zstan] in my lab, 1 node with persistence=true: === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 62 seconds TPS: 1613 === 3 nodes with persistence=true: === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 173 seconds TPS: 578 === Could you try with 3 nodes cluster as well? And any specific reason why the performance drop so much between 1 node cluster and 3 nodes cluster? besides the reason of data transfer overhead between hosts. The application now has been simplify to all varchar columns so can run with 578 TPS. In the first version which bigint column involved it can run only with ~200 TPS. > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-7285) Add default query timeout
[
https://issues.apache.org/jira/browse/IGNITE-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986040#comment-16986040
]
Ivan Pavlukhin commented on IGNITE-7285:
[~samaitra], could you please fill in release notes field, create a follow-up
documentation ticket and resolve this ticket?
> Add default query timeout
> -
>
> Key: IGNITE-7285
> URL: https://issues.apache.org/jira/browse/IGNITE-7285
> Project: Ignite
> Issue Type: Improvement
> Components: cache, sql
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Saikat Maitra
>Priority: Major
> Labels: sql-stability
> Fix For: 2.8
>
> Time Spent: 10h 40m
> Remaining Estimate: 0h
>
> Currently it's possible to provide timeout only on query level. It would be
> very useful to have default timeout value provided on cache startup. Let's
> add {{CacheConfiguration#defaultQueryTimeout}} configuration property.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12409) Destroying a cache during cache load may lead to a hang
[
https://issues.apache.org/jira/browse/IGNITE-12409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986015#comment-16986015
]
Ignite TC Bot commented on IGNITE-12409:
{panel:title=Branch: [pull/7092/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4803331buildTypeId=IgniteTests24Java8_RunAll]
> Destroying a cache during cache load may lead to a hang
> ---
>
> Key: IGNITE-12409
> URL: https://issues.apache.org/jira/browse/IGNITE-12409
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7.6
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Critical
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Observed this hang in JCache Suite (only relevant threads are shown):
> {noformat}
> "main" #1 prio=5 os_prio=0 tid=0x7f801c00f800 nid=0x6edc waiting on
> condition [0x7f80255c2000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
> at
> org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
> at
> org.apache.ignite.internal.util.future.IgniteFutureImpl.get(IgniteFutureImpl.java:133)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.destroy(GatewayProtectedCacheProxy.java:1318)
> at
> org.apache.ignite.cache.CacheManager.destroyCache(CacheManager.java:282)
> at
> org.jsr107.tck.testutil.CacheTestSupport.teardown(CacheTestSupport.java:56)
> at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray2(ReflectionUtils.java:186)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:161)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:84)
> at
> org.apache.maven.plugin.surefire.InPluginVMSurefireStarter.runSuitesInProcess(InPluginVMSurefireStarter.java:87)
> at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1144)
> at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1002)
> at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:848)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
> at
>
[jira] [Updated] (IGNITE-11410) Sandbox for user-defined code
[ https://issues.apache.org/jira/browse/IGNITE-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11410: - Issue Type: New Feature (was: Task) > Sandbox for user-defined code > - > > Key: IGNITE-11410 > URL: https://issues.apache.org/jira/browse/IGNITE-11410 > Project: Ignite > Issue Type: New Feature >Reporter: Anton Vinogradov >Assignee: Denis Garus >Priority: Critical > Labels: iep-38 > Fix For: 2.8 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > We should provide a restricted environment (sandbox) in which to run > user-defined code securely. To get it done, we would use the java sandbox > model. > The java sandbox model allows restricting access from user-defined code to > the system resources or security-sensitive feature of java, for example, > reflection. > The user-defined code contains: > - StreamReceiver for DataStreamer: > - EntryProcessor; > - ComputeJob; > - filter and transformer for ScanQuery. > The user-defined code will get permissions from GridSecuerityProcessor > (security plugin). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12410) Too long muted tests should be ignored with annotation
[
https://issues.apache.org/jira/browse/IGNITE-12410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986004#comment-16986004
]
Ignite TC Bot commented on IGNITE-12410:
{panel:title=Branch: [pull/7094/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4805973buildTypeId=IgniteTests24Java8_RunAll]
> Too long muted tests should be ignored with annotation
> --
>
> Key: IGNITE-12410
> URL: https://issues.apache.org/jira/browse/IGNITE-12410
> Project: Ignite
> Issue Type: Task
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Trivial
> Time Spent: 10m
> Remaining Estimate: 0h
>
> There are some tests that muted and having long execution time:
> Test: CacheGroupsMetricsRebalanceTest.testRebalanceEstimateFinishTime
> Duration: 2m:37s,377ms (x2, 2 suite)
> Muted by: https://issues.apache.org/jira/browse/IGNITE-9391
> Fail rate: 97%
> Test: IgniteOsgiServiceTest.testServiceExposedAndCallbacksInvoked
> Duration: 3m:04s,307ms
> Muted by: https://issues.apache.org/jira/browse/IGNITE-8300
> Fail rate: 93%
> Test:
> IgniteKarafFeaturesInstallationTest.testAllBundlesActiveAndFeaturesInstalled
> Duration: 3m:01s,369ms
> Muted by: https://issues.apache.org/jira/browse/IGNITE-8254
> Fail rate: 100%
> Test:
> JoinInActiveNodeToActiveClusterWithPersistence.testJoinClientStaticCacheConfigurationInCluster
> Duration: 9s,194ms
> Muted by: https://issues.apache.org/jira/browse/IGNITE-5518
> Fail rate: 100%
> Total time: 11+ minutes
> They should be ignored with the ignored annotation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12088) Cache or template name should be validated before attempt to start
[
https://issues.apache.org/jira/browse/IGNITE-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-12088:
-
Fix Version/s: (was: 2.8)
2.9
> Cache or template name should be validated before attempt to start
> --
>
> Key: IGNITE-12088
> URL: https://issues.apache.org/jira/browse/IGNITE-12088
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Pavel Kovalenko
>Assignee: kcheng.mvp
>Priority: Critical
> Labels: usability
> Fix For: 2.9
>
>
> If set too long cache name it can be a cause of impossibility to create work
> directory for that cache:
> {noformat}
> [2019-08-20
> 19:35:42,139][ERROR][exchange-worker-#172%node1%][IgniteTestResources]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
> SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=class o.a.i.IgniteCheckedException: Failed to
> initialize cache working directory (failed to create, make sure the work
> folder has correct permissions):
> /home/gridgain/projects/incubator-ignite/work/db/node1/cache-CacheConfiguration
> [name=ccfg3staticTemplate*, grpName=null, memPlcName=null,
> storeConcurrentLoadAllThreshold=5, rebalancePoolSize=1,
> rebalanceTimeout=1, evictPlc=null, evictPlcFactory=null,
> onheapCache=false, sqlOnheapCache=false, sqlOnheapCacheMaxSize=0,
> evictFilter=null, eagerTtl=true, dfltLockTimeout=0, nearCfg=null,
> writeSync=null, storeFactory=null, storeKeepBinary=false, loadPrevVal=false,
> aff=null, cacheMode=PARTITIONED, atomicityMode=null, backups=6,
> invalidate=false, tmLookupClsName=null, rebalanceMode=ASYNC,
> rebalanceOrder=0, rebalanceBatchSize=524288, rebalanceBatchesPrefetchCnt=2,
> maxConcurrentAsyncOps=500, sqlIdxMaxInlineSize=-1, writeBehindEnabled=false,
> writeBehindFlushSize=10240, writeBehindFlushFreq=5000,
> writeBehindFlushThreadCnt=1, writeBehindBatchSize=512,
> writeBehindCoalescing=true, maxQryIterCnt=1024, affMapper=null,
> rebalanceDelay=0, rebalanceThrottle=0, interceptor=null,
> longQryWarnTimeout=3000, qryDetailMetricsSz=0, readFromBackup=true,
> nodeFilter=null, sqlSchema=null, sqlEscapeAll=false, cpOnRead=true,
> topValidator=null, partLossPlc=IGNORE, qryParallelism=1, evtsDisabled=false,
> encryptionEnabled=false, diskPageCompression=null,
> diskPageCompressionLevel=null]0]]
> class org.apache.ignite.IgniteCheckedException: Failed to initialize cache
> working directory (failed to create, make sure the work folder has correct
> permissions):
> /home/gridgain/projects/incubator-ignite/work/db/node1/cache-CacheConfiguration
> [name=ccfg3staticTemplate*, grpName=null, memPlcName=null,
> storeConcurrentLoadAllThreshold=5, rebalancePoolSize=1,
> rebalanceTimeout=1, evictPlc=null, evictPlcFactory=null,
> onheapCache=false, sqlOnheapCache=false, sqlOnheapCacheMaxSize=0,
> evictFilter=null, eagerTtl=true, dfltLockTimeout=0, nearCfg=null,
> writeSync=null, storeFactory=null, storeKeepBinary=false, loadPrevVal=false,
> aff=null, cacheMode=PARTITIONED, atomicityMode=null, backups=6,
> invalidate=false, tmLookupClsName=null, rebalanceMode=ASYNC,
> rebalanceOrder=0, rebalanceBatchSize=524288, rebalanceBatchesPrefetchCnt=2,
> maxConcurrentAsyncOps=500, sqlIdxMaxInlineSize=-1, writeBehindEnabled=false,
> writeBehindFlushSize=10240, writeBehindFlushFreq=5000,
> writeBehindFlushThreadCnt=1, writeBehindBatchSize=512,
> writeBehindCoalescing=true, maxQryIterCnt=1024, affMapper=null,
> rebalanceDelay=0, rebalanceThrottle=0, interceptor=null,
> longQryWarnTimeout=3000, qryDetailMetricsSz=0, readFromBackup=true,
> nodeFilter=null, sqlSchema=null, sqlEscapeAll=false, cpOnRead=true,
> topValidator=null, partLossPlc=IGNORE, qryParallelism=1, evtsDisabled=false,
> encryptionEnabled=false, diskPageCompression=null,
> diskPageCompressionLevel=null]0
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.checkAndInitCacheWorkDir(FilePageStoreManager.java:769)
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.checkAndInitCacheWorkDir(FilePageStoreManager.java:748)
> at
> org.apache.ignite.internal.processors.cache.CachesRegistry.persistCacheConfigurations(CachesRegistry.java:289)
> at
> org.apache.ignite.internal.processors.cache.CachesRegistry.registerAllCachesAndGroups(CachesRegistry.java:264)
> at
> org.apache.ignite.internal.processors.cache.CachesRegistry.update(CachesRegistry.java:202)
> at
>
[jira] [Commented] (IGNITE-10417) notifyDiscoveryListener() call can be lost.
[ https://issues.apache.org/jira/browse/IGNITE-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986001#comment-16986001 ] Maxim Muzafarov commented on IGNITE-10417: -- [~voropava] I see that comments in PR are still not resolved. Will you fix them? > notifyDiscoveryListener() call can be lost. > --- > > Key: IGNITE-10417 > URL: https://issues.apache.org/jira/browse/IGNITE-10417 > Project: Ignite > Issue Type: Bug >Affects Versions: 2.7 >Reporter: Pavel Voronkin >Assignee: Pavel Voronkin >Priority: Critical > Fix For: 2.8 > > > 1) ServerImpl:5455 notifyDiscoveryListener can not be called in case of > spiState != CONNECTED, for example DISCONNECTING which is valid state > nevetheless inside notifyDiscoveryListener we check spiState == CONNECTED || > spiState == DISCONNECTING, also we need to improve logging in > notifyDiscoveryListener(). > 2) Improve logging on duplicated custom discovery event. > 3) Add assert if msg.creatorNodeId() doesn't exist in topology this is bad > behaviour taht should lead to node fail. > > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-6324) Transactional cache data partially available after crash.
[ https://issues.apache.org/jira/browse/IGNITE-6324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-6324: Fix Version/s: (was: 2.8) 2.9 > Transactional cache data partially available after crash. > - > > Key: IGNITE-6324 > URL: https://issues.apache.org/jira/browse/IGNITE-6324 > Project: Ignite > Issue Type: Bug > Components: persistence >Affects Versions: 1.9, 2.1 >Reporter: Stanilovsky Evgeny >Priority: Critical > Fix For: 2.9 > > Attachments: InterruptCommitedThreadTest.java > > > If InterruptedException raise in client code during pds store operations we > can obtain inconsistent cache after restart. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12188) Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly if there is more then one cache in cache group
[
https://issues.apache.org/jira/browse/IGNITE-12188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986000#comment-16986000
]
Aleksey Plekhanov commented on IGNITE-12188:
[~NSAmelchev], There is still the problem with
{{cctx.group().metrics().setIndexBuildCountPartitionsLeft(0);}}
usage. If any error occurred during index rebuild, the metric value becomes
negative.
> Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly
> if there is more then one cache in cache group
>
>
> Key: IGNITE-12188
> URL: https://issues.apache.org/jira/browse/IGNITE-12188
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.8
>Reporter: Aleksey Plekhanov
>Assignee: Amelchev Nikita
>Priority: Major
> Labels: IEP-35, await
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Initial partitions counter is set to a total number of partitions for cache
> group but decremented each time partition processed for each cache.
> Reproducer:
> {code:java}
> @Test
> public void testIndexRebuildMetrics() throws Exception {
> IgniteEx ignite = startGrid(new IgniteConfiguration()
> .setConsistentId("ignite")
> .setDataStorageConfiguration(new DataStorageConfiguration()
> .setDefaultDataRegionConfiguration(new
> DataRegionConfiguration().setPersistenceEnabled(true)))
> .setCacheConfiguration(
> new CacheConfiguration Integer>("c1").setGroupName("grp").setIndexedTypes(Integer.class,
> Integer.class),
> new CacheConfiguration Integer>("c2").setGroupName("grp").setIndexedTypes(Integer.class,
> Integer.class)
> ));
> ignite.cluster().active(true);
> for (int i = 0; i < 10_000; i++)
> ignite.cache("c1").put(i, i);
> ignite.cluster().active(false);
> File dir = U.resolveWorkDirectory(U.defaultWorkDirectory(),
> DFLT_STORE_DIR, false);
> U.delete(new File(dir, "ignite/cacheGroup-grp/index.bin"));
> ignite.cluster().active(true);
> doSleep(1_000L);
>
> assertTrue(ignite.context().cache().cache("c1").context().group().metrics().getIndexBuildCountPartitionsLeft()
> >= 0);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-7414) Cluster restart may lead to cluster activation error
[
https://issues.apache.org/jira/browse/IGNITE-7414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985999#comment-16985999
]
Maxim Muzafarov commented on IGNITE-7414:
-
[~agoncharuk]
Hello, should we fix this issue before the 2.8 release occurs?
> Cluster restart may lead to cluster activation error
>
>
> Key: IGNITE-7414
> URL: https://issues.apache.org/jira/browse/IGNITE-7414
> Project: Ignite
> Issue Type: Bug
> Components: general
>Affects Versions: 2.3
>Reporter: Vyacheslav Koptilin
>Assignee: Alexey Goncharuk
>Priority: Critical
> Fix For: 2.8
>
> Attachments: Reproducer.java
>
>
> Attempt to execute the following reproducer twice result in an error (please
> see attached)
> {code}
> public static void main(String[] args) throws IgniteException {
> Ignite ignite = Ignition.start(createIgniteConfiguration());
> ignite.active(true);
> CacheConfiguration cacheCfg = new CacheConfiguration("test-cache");
> cacheCfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
> cacheCfg.setCacheMode(CacheMode.PARTITIONED);
> cacheCfg.setDataRegionName("inmemory");
> cacheCfg.setIndexedTypes(Integer.class, String.class);
> IgniteCache cache = ignite.getOrCreateCache(cacheCfg);
> cache.put(42, "value-42");
> ignite.close();
> }
> private static IgniteConfiguration createIgniteConfiguration() {
> IgniteConfiguration ignCfg = IgniteConfiguration();
> ...
> DataStorageConfiguration dc = new DataStorageConfiguration();
> // persistence enabled region
> DataRegionConfiguration persistenceRegion = new
> DataRegionConfiguration();
> persistenceRegion.setName("persistence");
> persistenceRegion.setPersistenceEnabled(true);
> // persistence disabled region
> DataRegionConfiguration inmemoryRegion = new
> DataRegionConfiguration();
> inmemoryRegion.setName("inmemory");
> inmemoryRegion.setInitialSize(100L * 1024 * 1024);
> inmemoryRegion.setMaxSize(500L * 1024 * 1024);
> inmemoryRegion.setPersistenceEnabled(false);
> dc.setDataRegionConfigurations(persistenceRegion, inmemoryRegion);
> ignCfg.setDataStorageConfiguration(dc);
> return ignCfg;
> }
> {code}
> The second execution failed with the exception as follows:
> {code}
> [2018-01-15
> 17:12:52,431][ERROR][exchange-worker-#42%test-grid%][GridDhtPartitionsExchangeFuture]
> Failed to activate node components
> [nodeId=4ba59aa1-cbc5-4d67-8ca5-b6dd1628c5dc, client=false,
> topVer=AffinityTopologyVersion [topVer=1, minorTopVer=1]]
> class org.apache.ignite.IgniteCheckedException: Failed to find cache group
> descriptor [grpId=623628935]
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.getPageMemoryForCacheGroup(GridCacheDatabaseSharedManager.java:1602)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1544)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:570)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest(GridDhtPartitionsExchangeFuture.java:820)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:583)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2279)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:748)
> Exception in thread "main" class org.apache.ignite.IgniteException: Failed to
> activate cluster
> at
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:980)
> at
> org.apache.ignite.internal.IgniteKernal.active(IgniteKernal.java:3318)
> at org.apache.ignite.examples.Reproducer.main(Reproducer.java:33)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to activate
> cluster
> Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to
> find cache group descriptor [grpId=623628935]
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.getPageMemoryForCacheGroup(GridCacheDatabaseSharedManager.java:1602)
> at
>
[jira] [Updated] (IGNITE-9379) Ignite node hangs after OOM in a thread from thin client thread pool
[ https://issues.apache.org/jira/browse/IGNITE-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-9379: Fix Version/s: (was: 2.8) 2.9 > Ignite node hangs after OOM in a thread from thin client thread pool > > > Key: IGNITE-9379 > URL: https://issues.apache.org/jira/browse/IGNITE-9379 > Project: Ignite > Issue Type: Bug > Components: sql >Affects Versions: 2.6 >Reporter: Taras Ledkov >Priority: Critical > Fix For: 2.9 > > > OOM exception handler isn't set up for thin client thread pool. > The issue is described in details at the [dev > list|http://apache-ignite-evelopers.2346864.n4.nabble.com/Binary-Client-Protocol-client-hangs-in-case-of-OOM-on-server-td34224.html]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-9090) When client node make cache.QueryCursorImpl.getAll they have OOM and continue working
[
https://issues.apache.org/jira/browse/IGNITE-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-9090:
Fix Version/s: (was: 2.8)
2.9
> When client node make cache.QueryCursorImpl.getAll they have OOM and continue
> working
> -
>
> Key: IGNITE-9090
> URL: https://issues.apache.org/jira/browse/IGNITE-9090
> Project: Ignite
> Issue Type: Bug
> Components: clients
>Affects Versions: 2.4
> Environment: 2 server node, 1 client, 1 cache with 15 kk size
>Reporter: ARomantsov
>Priority: Critical
> Fix For: 2.9
>
>
> {code:java}
> [12:21:22,390][SEVERE][query-#69][GridCacheIoManager] Failed to process
> message [senderId=30cab4ec-1da7-4e9f-a262-bdfa4d466865, messageType=class
> o.a.i.i.processors.cache.query.GridCacheQueryResponse]
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.lang.Long.valueOf(Long.java:840)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:250)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:421)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponseEntry.readExternal(GridCacheQueryResponseEntry.java:90)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readExternalizable(OptimizedObjectInputStream.java:555)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedClassDescriptor.read(OptimizedClassDescriptor.java:917)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObject0(OptimizedObjectInputStream.java:346)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedObjectInputStream.readObjectOverride(OptimizedObjectInputStream.java:198)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:421)
> at
> org.apache.ignite.internal.marshaller.optimized.OptimizedMarshaller.unmarshal0(OptimizedMarshaller.java:227)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
> at
> org.apache.ignite.internal.binary.BinaryUtils.doReadOptimized(BinaryUtils.java:1777)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1964)
> at
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
> at
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:310)
> at
> org.apache.ignite.internal.binary.BinaryMarshaller.unmarshal0(BinaryMarshaller.java:99)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.unmarshalCollection0(GridCacheQueryResponse.java:189)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryResponse.finishUnmarshal(GridCacheQueryResponse.java:162)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1530)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:576)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:101)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1613)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>
[jira] [Commented] (IGNITE-12243) [Spark] Add support of HAVING without GROUP BY
[ https://issues.apache.org/jira/browse/IGNITE-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985996#comment-16985996 ] Maxim Muzafarov commented on IGNITE-12243: -- [~zaleslaw] Hello, should we still wait for this issue to be included in the 2.8 release? > [Spark] Add support of HAVING without GROUP BY > -- > > Key: IGNITE-12243 > URL: https://issues.apache.org/jira/browse/IGNITE-12243 > Project: Ignite > Issue Type: Sub-task > Components: spark >Affects Versions: 2.8 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 2.8 > > > Also the semantic of Having support was changed in Spark > [https://github.com/apache/spark/pull/22696/files] > https://issues.apache.org/jira/browse/SPARK-25708 > Now Having could be a legal operation without GroupBY > > Rewrite the test "SELECT id FROM city HAVING id > 1" -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-8719: Fix Version/s: (was: 2.8) 2.9 > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Priority: Critical > Fix For: 2.9 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-8688) Pending tree is initialized outside of checkpoint lock
[
https://issues.apache.org/jira/browse/IGNITE-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-8688:
Fix Version/s: (was: 2.8)
2.9
> Pending tree is initialized outside of checkpoint lock
> --
>
> Key: IGNITE-8688
> URL: https://issues.apache.org/jira/browse/IGNITE-8688
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.5
>Reporter: Pavel Kovalenko
>Priority: Critical
> Fix For: 2.9
>
>
> This may lead to possible page corruption.
> {noformat}
> handled accordingly to configured handler [hnd=class
> o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext
> [type=SYSTEM_WORKER_TERMINATION, err=java.lang.AssertionError]]
> [00:11:56]W: [org.gridgain:gridgain-compatibility]
> java.lang.AssertionError
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.allocatePage(PageMemoryImpl.java:463)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.allocateForTree(IgniteCacheOffheapManagerImpl.java:818)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.initPendingTree(IgniteCacheOffheapManagerImpl.java:164)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.onCacheStarted(IgniteCacheOffheapManagerImpl.java:151)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.CacheGroupContext.onCacheStarted(CacheGroupContext.java:283)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1965)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:791)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest(GridDhtPartitionsExchangeFuture.java:946)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:651)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2458)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2338)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> [00:11:56]W: [org.gridgain:gridgain-compatibility] at
> java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-9489) CorruptedTreeException on index create.
[
https://issues.apache.org/jira/browse/IGNITE-9489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985992#comment-16985992
]
Maxim Muzafarov commented on IGNITE-9489:
-
[~gvvinblade]
Hello, do you have any additional info about this issue? Does this issue
already fixed?
> CorruptedTreeException on index create.
> ---
>
> Key: IGNITE-9489
> URL: https://issues.apache.org/jira/browse/IGNITE-9489
> Project: Ignite
> Issue Type: Bug
> Components: cache, sql
>Affects Versions: 2.4, 2.5, 2.6
>Reporter: Igor Seliverstov
>Priority: Blocker
> Fix For: 2.8
>
> Attachments: Test.java
>
>
> Currently on dynamic index drop with enabled persistence H2TreeIndex
> instances aren't destroyed. That means that their root pages aren't removed
> from meta tree (see
> {{org.apache.ignite.internal.processors.cache.persistence.IndexStorageImpl#getOrAllocateForTree}})
> and reused on subsequent dynamic index create that leads
> CorruptedTreeException on initial index rebuild because there are some items
> with broken links on the root page.
> Reproducer attached.
> Error log:
> {noformat}
> Error during parallel index create/rebuild.
> org.h2.message.DbException: Внутренняя ошибка: "class
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> Runtime failure on row: Row@7745722d[ key:
> org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$KeyClass
> [idHash=2038596277, hash=-1388553726, id=1], val:
> org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$ValueClass
> [idHash=2109544797, hash=-898815788, field1=val1], ver: GridCacheVersion
> [topVer=147733489, order=1536253488473, nodeOrder=2] ][ 1, val1, null ]"
> General error: "class
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> Runtime failure on row: Row@7745722d[ key:
> org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$KeyClass
> [idHash=2038596277, hash=-1388553726, id=1], val:
> org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$ValueClass
> [idHash=2109544797, hash=-898815788, field1=val1], ver: GridCacheVersion
> [topVer=147733489, order=1536253488473, nodeOrder=2] ][ 1, val1, null ]"
> [5-195]
> at org.h2.message.DbException.get(DbException.java:168)
> at org.h2.message.DbException.convert(DbException.java:295)
> at
> org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:251)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$3.apply(IgniteH2Indexing.java:890)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.updateIndex(GridCacheMapEntry.java:4320)
> at
> org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.processKey(SchemaIndexCacheVisitorImpl.java:244)
> at
> org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.processPartition(SchemaIndexCacheVisitorImpl.java:207)
> at
> org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.processPartitions(SchemaIndexCacheVisitorImpl.java:166)
> at
> org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.access$100(SchemaIndexCacheVisitorImpl.java:50)
> at
> org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl$AsyncWorker.body(SchemaIndexCacheVisitorImpl.java:317)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.h2.jdbc.JdbcSQLException: Внутренняя ошибка: "class
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> Runtime failure on row: Row@7745722d[ key:
> org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$KeyClass
> [idHash=2038596277, hash=-1388553726, id=1], val:
> org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$ValueClass
> [idHash=2109544797, hash=-898815788, field1=val1], ver: GridCacheVersion
> [topVer=147733489, order=1536253488473, nodeOrder=2] ][ 1, val1, null ]"
> General error: "class
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> Runtime failure on row: Row@7745722d[ key:
> org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$KeyClass
> [idHash=2038596277, hash=-1388553726, id=1], val:
> org.apache.ignite.internal.processors.cache.index.AbstractSchemaSelfTest$ValueClass
> [idHash=2109544797, hash=-898815788, field1=val1], ver: GridCacheVersion
> [topVer=147733489, order=1536253488473, nodeOrder=2] ][ 1, val1, null ]"
> [5-195]
> at
[jira] [Updated] (IGNITE-8695) RPM|DEB: Remove requirement to run Ignite stand-along under root
[ https://issues.apache.org/jira/browse/IGNITE-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-8695: Fix Version/s: (was: 2.8) 2.9 > RPM|DEB: Remove requirement to run Ignite stand-along under root > > > Key: IGNITE-8695 > URL: https://issues.apache.org/jira/browse/IGNITE-8695 > Project: Ignite > Issue Type: Task >Reporter: Denis A. Magda >Priority: Critical > Fix For: 2.9 > > > If Ignite is started as a stand-alone process we require to do that under the > root user: > https://apacheignite.readme.io/docs/getting-started#section-run-ignite-as-a-stand-alone-process > Noone would do that in prod. We need to remove this requirement. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11461) Automatic modules support for Apache Ignite: find and resolve packages conflicts
[ https://issues.apache.org/jira/browse/IGNITE-11461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11461: - Fix Version/s: (was: 2.8) 2.9 > Automatic modules support for Apache Ignite: find and resolve packages > conflicts > > > Key: IGNITE-11461 > URL: https://issues.apache.org/jira/browse/IGNITE-11461 > Project: Ignite > Issue Type: Improvement >Reporter: Dmitry Pavlov >Priority: Critical > Fix For: 2.9 > > Time Spent: 1h > Remaining Estimate: 0h > > Example of failure in a modular environment: > Error:java: the unnamed module reads package > org.apache.ignite.internal.processors.cache.persistence.file from both > ignite.core and ignite.direct.io > This type of failure is named package inference, but it is strictly > prohibited > http://openjdk.java.net/projects/jigsaw/spec/reqs/#non-interference > Ignite compatibility with Jigsaw is tested in a separate project. See details > in > https://github.com/apache/ignite/tree/ignite-11461-java11/modules/dev-utils/ignite-modules-test#ignite-modular-environment-test-project > > Following table contains currenly investigated Ignite modules if this > applicability as automatic modules: > ||Module||Run In Modular Environment||Changeable using private API only || > Notes || > |ignite-code|(/)|(/)| | > |ignite-indexing|(x) [IGNITE-11464] | (?) Refacrtoing to use > ignite-indexing-text may be a breaking change | Lucene artifacts exclusion is > required by user manually. | > |ignite-compress | (x) | (/) not releaseed | > org.apache.ignite.internal.processors.compress package conflict | > |ignite-direct-io|(x) blocked by indexind | (/) | > org.apache.ignite.internal.processors.cache.persistence.file package conflict > | > |ignite-spring|(x) [IGNITE-11467] blocked by indexing | (x) > org.apache.ignite.IgniteSpringBean affected | | > |ignite-ml |(x) blocked by indexing | | | > |ignite-log4j|(/)|(/) | But may not compile with other logging dependencies - > EOL https://blogs.apache.org/logging/entry/moving_on_to_log4j_2 | > |ignite-log4j2|(/)|(/)| | > |ignite-slf4j | (/)|(/)| | > |ignite-rest-http | (x) IGNITE-11469 & Mirgate to log4j2x [IGNITE-11486] | > (/) | Usage with slf4j may break compilation because conflict of packages | > |ignite-hibernate_5.3 and others | (x) [IGNITE-11485] | (?) | avoid of API > breaking is possibleif hibernate core classes not used by third party code | > |ignite-zookeeper| (x) IGNITE-11486 | (/) | | > |ignite-spring-data_2-0 | (x) blocked by spring | org.apache.commons.logging > from both commons.logging and spring.jcl conflict | > https://jira.spring.io/browse/SPR-16605 | > |ignite-ml | (/) master (x) 2.7 | | | > |ignite-cassandra-store | (x) [IGNITE-11467] blocked by spring | (/) | > Only spring needs to be fixed | -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11147) Re-balance cancellation occur by non-affected event
[ https://issues.apache.org/jira/browse/IGNITE-11147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11147: - Fix Version/s: (was: 2.8) 2.9 > Re-balance cancellation occur by non-affected event > --- > > Key: IGNITE-11147 > URL: https://issues.apache.org/jira/browse/IGNITE-11147 > Project: Ignite > Issue Type: Test > Components: cache >Affects Versions: 2.7 >Reporter: Sergey Antonov >Assignee: Vladislav Pyatkov >Priority: Critical > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > Re-balance cancels by non-affected events, for examples: > 1) joining non affinity node > 2) stating snapshot > 3) starting/stopping other cache > Try to skip as more as possible events instead of cancellation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-9867) Add ability to block out of range IP on discovery request
[
https://issues.apache.org/jira/browse/IGNITE-9867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-9867:
Fix Version/s: (was: 2.8)
2.9
> Add ability to block out of range IP on discovery request
> -
>
> Key: IGNITE-9867
> URL: https://issues.apache.org/jira/browse/IGNITE-9867
> Project: Ignite
> Issue Type: Improvement
> Components: general
>Affects Versions: 2.7
>Reporter: ARomantsov
>Priority: Critical
> Fix For: 2.9
>
>
> Now we can set list of cluster collector node, but cannot deny another ips to
> connect to our cluster
> {code:java}
>
>
>
>
>
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>
>
>
>
>
>
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-12330) Assertion error in CachedDeploymentInfo
[
https://issues.apache.org/jira/browse/IGNITE-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-12330:
-
Fix Version/s: (was: 2.8)
2.9
> Assertion error in CachedDeploymentInfo
> ---
>
> Key: IGNITE-12330
> URL: https://issues.apache.org/jira/browse/IGNITE-12330
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7.6
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Blocker
> Fix For: 2.9
>
>
> The following exception occured on some production environment.
> It leads to the node fail.
> {noformat}
> 2019-09-17
> 18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
> Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
> msg=GridCacheQueryRequest [id=4922,
> cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
> type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
> rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
> incMeta=false, all=false, keepBinary=true,
> subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
> topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
> super=GridCacheIdMessage [cacheId=-724666788]]]2019-09-17
> 18:29:29.890[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][o.a.i.i.p.cache.GridCacheIoManager]
> Failed to process message [senderId=4c071d12-325a-4bb1-a68d-cc910f636562,
> msg=GridCacheQueryRequest [id=4922,
> cacheName=com.sbt.limits.data.entity.LimitTemplateV1Entity_DPL_union-module,
> type=SCAN, fields=false, clause=null, clsName=null, keyValFilter=null,
> rdc=null, trans=null, pageSize=1024, incBackups=false, cancel=false,
> incMeta=false, all=false, keepBinary=true,
> subjId=4c071d12-325a-4bb1-a68d-cc910f636562, taskHash=0, part=-1,
> topVer=AffinityTopologyVersion [topVer=191, minorTopVer=0],
> super=GridCacheIdMessage [cacheId=-724666788]]]
> java.lang.AssertionError: null
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:918)
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:889)
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.p2pContext(GridCacheDeploymentManager.java:422)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1547)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:582)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:386)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:312)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:102)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:301)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1091)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2019-09-17
> 18:29:29.912[ERROR][query-#1577440%DPL_GRID%DplGridNodeName%][org.apache.ignite.Ignite]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=java.lang.AssertionError]]
> java.lang.AssertionError: null
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:918)
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager$CachedDeploymentInfo.(GridCacheDeploymentManager.java:889)
> at
> org.apache.ignite.internal.processors.cache.GridCacheDeploymentManager.p2pContext(GridCacheDeploymentManager.java:422)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.unmarshall(GridCacheIoManager.java:1547)
> at
>
[jira] [Updated] (IGNITE-11942) IGFS and Hadoop Accelerator Discontinuation
[ https://issues.apache.org/jira/browse/IGNITE-11942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11942: - Fix Version/s: (was: 2.8) 2.9 > IGFS and Hadoop Accelerator Discontinuation > --- > > Key: IGNITE-11942 > URL: https://issues.apache.org/jira/browse/IGNITE-11942 > Project: Ignite > Issue Type: Task >Reporter: Denis A. Magda >Priority: Blocker > Fix For: 2.9 > > > The community has voted for the following decision: > * IGFS and In-Memory Hadoop Accelerator components are to be discontinued and > no longer supported by the community > * The existing source code of IGFS and In-Memory Hadoop Accelerator is to be > removed from Ignite master. Before that, a special branch like > "ignite-igfs-and-hadoop-accelerator" to be forked off the master in order to > preserve the sources in Git history for those who might need it. > The voting thread: > http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42405.html > Once the changes are made for Ignite 2.8, please contact Denis Magda to > update a public documentation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11905) [IEP-35] Monitoring Phase 2
[ https://issues.apache.org/jira/browse/IGNITE-11905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11905: - Priority: Blocker (was: Major) > [IEP-35] Monitoring Phase 2 > -- > > Key: IGNITE-11905 > URL: https://issues.apache.org/jira/browse/IGNITE-11905 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Blocker > Labels: IEP-35, await, important > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Phase 2 should introduce: > Ability to collect lists of some internal object Ignite manage. > Examples of such objects: > * Caches > * Queries (including continuous queries) > * Services > * Compute tasks > * Distributed Data Structures > * etc... > 1. Fields for each list should be discussed in separate tickets > 2. Metric Exporters (optionally) can support list export. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-10696) Java 9/10/11 Support
[ https://issues.apache.org/jira/browse/IGNITE-10696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-10696: - Fix Version/s: (was: 2.8) 2.9 > Java 9/10/11 Support > > > Key: IGNITE-10696 > URL: https://issues.apache.org/jira/browse/IGNITE-10696 > Project: Ignite > Issue Type: Improvement >Reporter: Peter Ivanov >Assignee: Peter Ivanov >Priority: Major > Labels: important, jdk11 > Fix For: 2.9 > > > Enhance Apache Ignite with ability to build, test and run under JDK 9-11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-11410) Sandbox for user-defined code
[ https://issues.apache.org/jira/browse/IGNITE-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985971#comment-16985971 ] Aleksey Plekhanov commented on IGNITE-11410: [~garus.d.g], I've looked at your patch, it looks good to me. > Sandbox for user-defined code > - > > Key: IGNITE-11410 > URL: https://issues.apache.org/jira/browse/IGNITE-11410 > Project: Ignite > Issue Type: Task >Reporter: Anton Vinogradov >Assignee: Denis Garus >Priority: Critical > Labels: iep-38 > Fix For: 2.8 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > We should provide a restricted environment (sandbox) in which to run > user-defined code securely. To get it done, we would use the java sandbox > model. > The java sandbox model allows restricting access from user-defined code to > the system resources or security-sensitive feature of java, for example, > reflection. > The user-defined code contains: > - StreamReceiver for DataStreamer: > - EntryProcessor; > - ComputeJob; > - filter and transformer for ScanQuery. > The user-defined code will get permissions from GridSecuerityProcessor > (security plugin). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12411) [ML] Finish ML API and fix typos in method names
Alexey Zinoviev created IGNITE-12411: Summary: [ML] Finish ML API and fix typos in method names Key: IGNITE-12411 URL: https://issues.apache.org/jira/browse/IGNITE-12411 Project: Ignite Issue Type: Improvement Components: ml Affects Versions: 2.8 Reporter: Alexey Zinoviev Assignee: Alexey Zinoviev Fix For: 2.8 There are a few typos in method naming that should be fixed before 2.8 release -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-11290) History of server node IDs should be passed to new nodes with NodeAddedMessage
[ https://issues.apache.org/jira/browse/IGNITE-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11290: - Fix Version/s: (was: 2.8) 2.9 > History of server node IDs should be passed to new nodes with NodeAddedMessage > -- > > Key: IGNITE-11290 > URL: https://issues.apache.org/jira/browse/IGNITE-11290 > Project: Ignite > Issue Type: Bug >Reporter: Sergey Chugunov >Assignee: Sergey Chugunov >Priority: Major > Fix For: 2.9 > > Time Spent: 10m > Remaining Estimate: 0h > > As part of IGNITE-5569 (bounded) history of IDs of all server nodes existed > in the cluster is introduced to prevent join requests with duplicate IDs if > network glitch happens during node's join process. > Initial implementation maintains the history locally on each node and isn't > transferred to successfully joined nodes. > It is needed to pass it (in NodeAdded messages) to new nodes to cover > edge-case scenarios of coordinator failover. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanilovsky Evgeny updated IGNITE-12374: Attachment: odbcsample > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985937#comment-16985937 ] Stanilovsky Evgeny commented on IGNITE-12374: - i attach mine odbcsample may be would helpful. > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, odbcsample, > odbcsample.allchar.rebind.cc, odbcsample.cc, profiling01.png, > profiling03.png, profling02.png, screenshot-1.png, > snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12393) Striped thread pool queue system view
[
https://issues.apache.org/jira/browse/IGNITE-12393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985891#comment-16985891
]
Nikolay Izhikov commented on IGNITE-12393:
--
[~v.pyatkov]
> I think, view of queue through system view will not be enough to understanding
You are right.
This view is only one view of Ignite's internal processes.
We should take other monitoring features in the count to see the whole picture.
> What is duration of handle of particular message?
It seems it should be the metric that shows the time when the last {{Runnable}}
was peeked up from stripe.
> How long the message hold in the queue?
Yes, you are right.
I thought to add this number to the view.
But this will require changes in {{StripedPool}} implementation.
We also can have performance issues with the extra long field for each Runnable
in the pool.
I think we should add this in the other ticket.
What do you think?
> What is the message mean? (in human readable format)
I don't this we should implement it in {{SystemView}}.
Translation from Ignite specific to human-readable if a feature of a
third-party tool.
> How to determine not typical duration of message?
This covers by the Failure Handler feature for Ignite internals, isn't it?
> Striped thread pool queue system view
> -
>
> Key: IGNITE-12393
> URL: https://issues.apache.org/jira/browse/IGNITE-12393
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When in the production environment exist some cluster performance issues
> usually it leads to the large striped executor queue size.
> The number of tasks in the queue can observe by
> {StripedExecutorMXBean#getTotalQueueSize} metric. In the case queue size
> becomes large it's useful to have the ability to know what tasks are waiting
> for execution in the thread pool.
> Especially, for dealing with failover scenarios.
> We should create a system views to expose information about striped executor
> services queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12393) Striped thread pool queue system view
[
https://issues.apache.org/jira/browse/IGNITE-12393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985882#comment-16985882
]
Nikolay Izhikov edited comment on IGNITE-12393 at 12/2/19 8:00 AM:
---
[~zstan] Thanks for the feedback.
> tx lock, aquire, commit phases log, who takes long lock,
I think this should be covered with a tracing feature.
> long queries
We already have {{SYS.SQL_QUERIES}}, {{SYS.SCAN_QUERIES}},
{{SYS.SQL_QUERIES_HISTORY}}.
these views provide data similar to what you expected.
Can you, please, take a look at this view?
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/SqlQueryView.java
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/SqlQueryHistoryView.java
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/ScanQueryView.java
was (Author: nizhikov):
[~zstan] Thanks for the feedback.
> tx lock, aquire, commit phases log, who takes long lock,
I think this should be covered with a tracing feature.
> long queries
We already have {SYS.SQL_QUERIES}, {SYS.SCAN_QUERIES},
{SYS.SQL_QUERIES_HISTORY}.
these views provide data similar to what you expected.
Can you, please, take a look at this view?
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/SqlQueryView.java
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/SqlQueryHistoryView.java
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/ScanQueryView.java
> Striped thread pool queue system view
> -
>
> Key: IGNITE-12393
> URL: https://issues.apache.org/jira/browse/IGNITE-12393
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When in the production environment exist some cluster performance issues
> usually it leads to the large striped executor queue size.
> The number of tasks in the queue can observe by
> {StripedExecutorMXBean#getTotalQueueSize} metric. In the case queue size
> becomes large it's useful to have the ability to know what tasks are waiting
> for execution in the thread pool.
> Especially, for dealing with failover scenarios.
> We should create a system views to expose information about striped executor
> services queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12374) Too low performance ~200TPS for single ODBC client
[ https://issues.apache.org/jira/browse/IGNITE-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985883#comment-16985883 ] Stanilovsky Evgeny commented on IGNITE-12374: - additionally: forgot to apply your config, with config : ./run.sh 50 2000 === NumPartial: 50 NumCompleteRec: 2000 TolRec: 10 Duration: 44 seconds TPS: 2273 === main difference with you is : one node cluster, can you confirm that your 1 node installation shows the same ? > Too low performance ~200TPS for single ODBC client > -- > > Key: IGNITE-12374 > URL: https://issues.apache.org/jira/browse/IGNITE-12374 > Project: Ignite > Issue Type: Improvement > Components: clients, odbc >Affects Versions: 2.7.5 > Environment: Ignite server run on top of Kubernetes, with 2 server > nodes, persistence enabled. Both CPU and RAM at server/client server is > sufficient according to system reports. >Reporter: swy >Priority: Major > Attachments: ignite-9bb53f34.0.log, > ignite-logs-config-source-20191128.zip, > ignite-logs-config-source-20191129.zip, odbcsample.allchar.rebind.cc, > odbcsample.cc, profiling01.png, profiling03.png, profling02.png, > screenshot-1.png, snapshot-1574845275597.nps, threaddump-1573207804944.tdump, > threaddump-1574845414243.tdump, values.yaml > > > Hi, in our test ignite performance with ODBC connection is too bad to proceed > with product integration. It is about ~200 TPS, each transaction with > select+update operation. > Please refer to attach sample program. It is just a simple test case. > Based on the profiling most of the time consumed by sql execution. Please > advice if the application did not do the right thing. > Thank you. > local ignite server, but the result same to remote container Ignite > deployment too. > cat /etc/apache-ignite/my-config.xml > > http://www.springframework.org/schema/beans; >xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; >xsi:schemaLocation="http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd;> > class="org.apache.ignite.configuration.IgniteConfiguration"> > > class="org.apache.ignite.configuration.ClientConnectorConfiguration"/> > > > > > > class="org.apache.ignite.binary.BinaryBasicIdMapper"> > > > > > > > > g++ -I/usr/include -I./ignite/binary/include -I./ignite/common/include > -I./ignite/common/os/linux/include -I./ignite/common/os/win/include > -I./ignite/core/include -I./ignite/jni/include odbcsample.cc -o odbcsample > -lodbc -L./ignite-libs/libignite.so -L./ignite-libs/libignite-odbc.so -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12393) Striped thread pool queue system view
[
https://issues.apache.org/jira/browse/IGNITE-12393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985882#comment-16985882
]
Nikolay Izhikov commented on IGNITE-12393:
--
[~zstan] Thanks for the feedback.
> tx lock, aquire, commit phases log, who takes long lock,
I think this should be covered with a tracing feature.
> long queries
We already have {SYS.SQL_QUERIES}, {SYS.SCAN_QUERIES},
{SYS.SQL_QUERIES_HISTORY}.
these views provide data similar to what you expected.
Can you, please, take a look at this view?
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/SqlQueryView.java
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/SqlQueryHistoryView.java
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/systemview/view/ScanQueryView.java
> Striped thread pool queue system view
> -
>
> Key: IGNITE-12393
> URL: https://issues.apache.org/jira/browse/IGNITE-12393
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: IEP-35
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When in the production environment exist some cluster performance issues
> usually it leads to the large striped executor queue size.
> The number of tasks in the queue can observe by
> {StripedExecutorMXBean#getTotalQueueSize} metric. In the case queue size
> becomes large it's useful to have the ability to know what tasks are waiting
> for execution in the thread pool.
> Especially, for dealing with failover scenarios.
> We should create a system views to expose information about striped executor
> services queue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
