[jira] [Created] (IGNITE-13239) Document APIs to view and change Cluster ID and Tag

2020-07-09 Thread Sergey Chugunov (Jira)
Sergey Chugunov created IGNITE-13239:


 Summary: Document APIs to view and change Cluster ID and Tag
 Key: IGNITE-13239
 URL: https://issues.apache.org/jira/browse/IGNITE-13239
 Project: Ignite
  Issue Type: Task
Reporter: Sergey Chugunov


In IGNITE-13185 new APIs and changes were introduced to view Cluster ID and Tag 
and change Tag.

These APIs and use cases need to be documented.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13238) Instant not supported whithin the scope of jdbc thin.

2020-07-09 Thread Alexander Lapin (Jira)
Alexander Lapin created IGNITE-13238:


 Summary: Instant not supported whithin the scope of jdbc thin.
 Key: IGNITE-13238
 URL: https://issues.apache.org/jira/browse/IGNITE-13238
 Project: Ignite
  Issue Type: Bug
  Components: jdbc
Affects Versions: 2.9
Reporter: Alexander Lapin


Reproducer: 
org.apache.ignite.jdbc.thin.JdbcThinCacheToJdbcDataTypesCoverageTest#testInstantDataType

 
{code:java}
[2020-07-10 
09:12:36,183][ERROR][client-connector-#196%thin.JdbcThinCacheToJdbcDataTypesCoverageTest0%][JdbcRequestHandler]
 Failed to execute SQL query [reqId=11, req=JdbcQueryExecuteRequest 
[schemaName=cachebb4b3282_3ad0_4ee3_9959_806c33521406, pageSize=1024, 
maxRows=0, sqlQry=SELECT * FROM tablebb4b3282_3ad0_4ee3_9959_806c33521406 WHERE 
_key = ?, args=Object[] [-292275055-05-16T16:47:04.192Z], 
stmtType=SELECT_STATEMENT_TYPE, autoCommit=true, partResReq=false, 
super=JdbcRequest [type=2, reqId=11]]]
javax.cache.CacheException: Failed to calculate derived partitions for query.
at 
org.apache.ignite.internal.sql.optimizer.affinity.PartitionResult.calculatePartitions(PartitionResult.java:132)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSelectDistributed(IgniteH2Indexing.java:1682)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSelect0(IgniteH2Indexing.java:1412)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.executeSelect(IgniteH2Indexing.java:1291)
at 
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1128)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2779)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2775)
at 
org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:3338)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$2(GridQueryProcessor.java:2795)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2833)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2769)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2727)
at 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.executeQuery(JdbcRequestHandler.java:647)
at 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.doHandle(JdbcRequestHandler.java:320)
at 
org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.handle(JdbcRequestHandler.java:257)
at 
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:200)
at 
org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:54)
at 
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
at 
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
at 
org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at 
org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to wrap 
value[type=24, value=-292275055-05-16T16:47:04.192Z]
at 
org.apache.ignite.internal.processors.query.h2.H2Utils.wrap(H2Utils.java:668)
at 
org.apache.ignite.internal.processors.query.h2.H2Utils.convert(H2Utils.java:520)
at 
org.apache.ignite.internal.processors.query.h2.affinity.H2PartitionResolver.partition(H2PartitionResolver.java:43)
at 
org.apache.ignite.internal.sql.optimizer.affinity.PartitionParameterNode.applySingle(PartitionParameterNode.java:72)
at 
org.apache.ignite.internal.sql.optimizer.affinity.PartitionSingleNode.apply(PartitionSingleNode.java:47)
at 
org.apache.ignite.internal.sql.optimizer.affinity.PartitionResult.calculatePartitions(PartitionResult.java:114)
... 25 more
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13237) In some cases IgniteClient#query does not fire CacheQueryExecutedEvent

2020-07-09 Thread Ryabov Dmitrii (Jira)
Ryabov Dmitrii created IGNITE-13237:
---

 Summary: In some cases IgniteClient#query does not fire 
CacheQueryExecutedEvent
 Key: IGNITE-13237
 URL: https://issues.apache.org/jira/browse/IGNITE-13237
 Project: Ignite
  Issue Type: Task
Reporter: Ryabov Dmitrii
Assignee: Ryabov Dmitrii
 Fix For: 2.10


In some cases {{IgniteClient#query}} does not fire {{CacheQueryExecutedEvent}}.
 Reproducer:
{code:java|title=IgniteCacheAbstractQuerySelfTest.java}
@Test
public void testClientSqlQueryEvents() throws Exception {
CountDownLatch execLatch = new CountDownLatch(9);

IgnitePredicate lsnr = evt -> {
assert evt instanceof CacheQueryExecutedEvent;

System.out.println(">>> EVENT: " + evt);

CacheQueryExecutedEvent qe = (CacheQueryExecutedEvent)evt;

assertEquals("SQL_PUBLIC_TEST_TABLE", qe.cacheName());
assertNotNull(qe.clause());
assertNull(qe.scanQueryFilter());
assertNull(qe.continuousQueryFilter());

execLatch.countDown();

return true;
};

ignite().events().localListen(lsnr, EVT_CACHE_QUERY_EXECUTED);

ClientConfiguration cc = new 
ClientConfiguration().setAddresses(Config.SERVER);

try (IgniteClient client = Ignition.startClient(cc)) {
client.query(new SqlFieldsQuery("create table TEST_TABLE(key int 
primary key, val int)"))
.getAll();

client.query(new SqlFieldsQuery("insert into TEST_TABLE values (?, 
?)").setArgs(1, 1))
.getAll();

client.query(new SqlFieldsQuery("update TEST_TABLE set val = ?2 where 
key = ?1").setArgs(1, 2))
.getAll();

client.query(new SqlFieldsQuery("select * from TEST_TABLE"))
.getAll();

client.query(new SqlFieldsQuery("create index idx_1 on 
TEST_TABLE(key)"))
.getAll();

client.query(new SqlFieldsQuery("drop index idx_1"))
.getAll();

client.query(new SqlFieldsQuery("alter table TEST_TABLE add column val2 
int"))
.getAll();

client.query(new SqlFieldsQuery("alter table TEST_TABLE drop val2"))
.getAll();

client.query(new SqlFieldsQuery("drop table TEST_TABLE"))
.getAll();

assertTrue(execLatch.await(3_000, MILLISECONDS));
}
finally {
ignite().events().stopLocalListen(lsnr, EVT_CACHE_QUERY_EXECUTED);
}
}
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [DISCUSSION] Ignite integration testing framework.

2020-07-09 Thread Anton Vinogradov
> Have you had a chance to deploy ducktests in bare metal?
Working on servers obtaining.

On Thu, Jul 9, 2020 at 10:11 AM Max Shonichev  wrote:

> Anton,
>
> well, strange thing, but clean up and rerun helped.
>
>
> Ubuntu 18.04
>
>
> 
> SESSION REPORT (ALL TESTS)
> ducktape version: 0.7.7
> session_id:   2020-07-06--003
> run time: 4 minutes 44.835 seconds
> tests run:5
> passed:   5
> failed:   0
> ignored:  0
>
> 
> test_id:
>
> ignitetest.tests.benchmarks.add_node_rebalance_test.AddNodeRebalanceTest.test_add_node.version=2.8.1
> status: PASS
> run time:   41.927 seconds
> {"Rebalanced in (sec)": 1.02205491065979}
>
> 
> test_id:
>
> ignitetest.tests.benchmarks.add_node_rebalance_test.AddNodeRebalanceTest.test_add_node.version=dev
> status: PASS
> run time:   51.985 seconds
> {"Rebalanced in (sec)": 0.0760810375213623}
>
> 
> test_id:
>
> ignitetest.tests.benchmarks.pme_free_switch_test.PmeFreeSwitchTest.test.version=2.7.6
> status: PASS
> run time:   1 minute 4.283 seconds
> {"Streamed txs": "1900", "Measure duration (ms)": "34818", "Worst
> latency (ms)": "31035"}
>
> 
> test_id:
>
> ignitetest.tests.benchmarks.pme_free_switch_test.PmeFreeSwitchTest.test.version=dev
> status: PASS
> run time:   1 minute 13.089 seconds
> {"Streamed txs": "73134", "Measure duration (ms)": "35843", "Worst
> latency (ms)": "139"}
>
> 
> test_id:
>
> ignitetest.tests.spark_integration_test.SparkIntegrationTest.test_spark_client
> status: PASS
> run time:   53.332 seconds
>
> 
>
>
> MacBook
>
> 
> SESSION REPORT (ALL TESTS)
> ducktape version: 0.7.7
> session_id:   2020-07-06--001
> run time: 6 minutes 58.612 seconds
> tests run:5
> passed:   5
> failed:   0
> ignored:  0
>
> 
> test_id:
>
> ignitetest.tests.benchmarks.add_node_rebalance_test.AddNodeRebalanceTest.test_add_node.version=2.8.1
> status: PASS
> run time:   48.724 seconds
> {"Rebalanced in (sec)": 3.2574470043182373}
>
> 
> test_id:
>
> ignitetest.tests.benchmarks.add_node_rebalance_test.AddNodeRebalanceTest.test_add_node.version=dev
> status: PASS
> run time:   1 minute 23.210 seconds
> {"Rebalanced in (sec)": 2.165921211242676}
>
> 
> test_id:
>
> ignitetest.tests.benchmarks.pme_free_switch_test.PmeFreeSwitchTest.test.version=2.7.6
> status: PASS
> run time:   1 minute 12.659 seconds
> {"Streamed txs": "642", "Measure duration (ms)": "33177", "Worst latency
> (ms)": "31063"}
>
> 
> test_id:
>
> ignitetest.tests.benchmarks.pme_free_switch_test.PmeFreeSwitchTest.test.version=dev
> status: PASS
> run time:   1 minute 57.257 seconds
> {"Streamed txs": "32924", "Measure duration (ms)": "48252", "Worst
> latency (ms)": "1010"}
>
> 
> test_id:
>
> ignitetest.tests.spark_integration_test.SparkIntegrationTest.test_spark_client
> status: PASS
> run time:   1 minute 36.317 seconds
>
> =
>
> while relative numbers proportion remains the same for different Ignite
> versions, absolute number for mac/linux differ more then twice.
>
> I'm finalizing code with 'local Tiden' appliance for your tests.  PR
> would be ready soon.
>
> Have you had a chance to deploy ducktests in bare metal?
>
>
>
> On 06.07.2020 14:27, Anton Vinogradov wrote:
> > Max,
> >
> > Thanks for the check!
> >
> >> Is it OK for those tests to fail?
> > No.
> > I see really strange things at logs.
> > Looks like you have concurrent ducktests run started not expected
> services,
> > and this broke the tests.
> > Could you please clean up the docker (use clean-up script [1]).
> > Compile sources (use script [2]) and rerun the tests.
> >
> > [1]
> >
> https://github.com/anton-vinogradov/ignite/blob/dc98ee9df90b25eb5d928090b0e78b48cae2392e/modules/ducktests/tests/docker/clean_up.sh
> > [2]
> >
> https://github.com/ant

[jira] [Created] (IGNITE-13236) Rebalance doesn't happen after restart if node failed at the moment of rebalance

2020-07-09 Thread Alexander Lapin (Jira)
Alexander Lapin created IGNITE-13236:


 Summary: Rebalance doesn't happen after restart if node failed at 
the moment of rebalance
 Key: IGNITE-13236
 URL: https://issues.apache.org/jira/browse/IGNITE-13236
 Project: Ignite
  Issue Type: Bug
 Environment: !Screen Shot 2019-08-05 at 18.20.46.png!
Reporter: Alexander Lapin
 Attachments: Screen Shot 2019-08-05 at 18.20.46.png, 
image-2020-07-09-14-55-51-574.png

Steps to reproduce:
1. Start 2 nodes in baseline
2. Stream a lot of data.
3. Stop the second node, clean the persistence and start it again(with the same 
consistent id
4. Kill the first node at the moment of rebalancing (make sure that not all 
data was rebalanced)
5. Partition loss will be detected.
6. Return first node to the cluster.
7. Result:
!image-2020-07-09-14-55-51-574.png!


Data is not rebalanced, All data(primary and backup) is stored on the first 
node only, while the second node has the same small subset of the data. It's 
reproducible with any Partition loss policy. Resetting lost partitions, 
activation/deactivation doesn't help.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-09 Thread Pavel Tupitsyn
We are breaking backwards compatibility,
so this can be only done for Ignite 3.0, am I right?

On Thu, Jul 9, 2020 at 1:46 PM Anton Kalashnikov  wrote:

> Hi everyone,
>
> The task of removal IGFS and Hadoop accelerator is ready to review.(
> https://issues.apache.org/jira/browse/IGNITE-11942)
> I've already asked some guys to take a look at it but if somebody familiar
> with this part of code, feel free to take a look at the changes
> too(especially scripts changes).
>
> I also think it is good to decide which release it should be planned on.
> This task planned for 2.9 right now but I should notice that first of all
> there are a lot of changes and secondly there are some changes in public
> API(removed some methods from configuration). So maybe it makes sense to
> move this ticket to the next release. What do you think?
>
> --
> Best regards,
> Anton Kalashnikov
>
>
> 10.02.2020, 15:45, "Alexey Zinoviev" :
> > Thank you so you much! Will wait:)
> >
> > пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk <
> alexey.goncha...@gmail.com>:
> >
> >>  Got it, then no need to rush, let's wait for the TF-IGFS decoupling.
> >>
> >>  пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev  >:
> >>
> >>  > Tensorflow integration uses IGFS, if you have any idea how to store
> files
> >>  > in memory by another way, please suggest something.
> >>  > I hope to decouple Ignite-TF integration to the separate repository
> >>  before
> >>  > release 2.9 with its own file system over Ignite Caches
> >>  >
> >>  > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin :
> >>  >
> >>  > > Is not it blocked by
> >>  > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in
> JIRA?
> >>  > >
> >>  > > @Alex Zinoviev could you please shed some light on this?
> >>  > >
> >>  > > Best regards,
> >>  > > Ivan Pavlukhin
> >>  > >
> >>  > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov  >:
> >>  > >
> >>  > > >
> >>  > > > I found the correct ticket for such activity -
> >>  > > https://issues.apache.org/jira/browse/IGNITE-11942
> >>  > > >
> >>  > > > --
> >>  > > > Best regards,
> >>  > > > Anton Kalashnikov
> >>  > > >
> >>  > > >
> >>  > > > 10.02.2020, 12:16, "Anton Kalashnikov" :
> >>  > > > > Hello.
> >>  > > > >
> >>  > > > > I created a ticket for this activity -
> >>  > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are
> >>  still
> >>  > > in consensus I'll do it at the nearest time(I've already had the
> >>  prepared
> >>  > > code).
> >>  > > > >
> >>  > > > > --
> >>  > > > > Best regards,
> >>  > > > > Anton Kalashnikov
> >>  > > > >
> >>  > > > > 10.02.2020, 12:07, "Alexey Goncharuk" <
> alexey.goncha...@gmail.com
> >>  >:
> >>  > > > >> Folks,
> >>  > > > >>
> >>  > > > >> I think there is a consensus here, but we did not remove IGFS
> >>  > > neither in
> >>  > > > >> 2.7 nor in 2.8, did we? Should we schedule a corresponding
> ticket
> >>  > > for 2.9?
> >>  > >
> >>  >
>


Re: [DISCUSSION] Complete Discontinuation of IGFS and Hadoop Accelerator

2020-07-09 Thread Anton Kalashnikov
Hi everyone,

The task of removal IGFS and Hadoop accelerator is ready to 
review.(https://issues.apache.org/jira/browse/IGNITE-11942)
I've already asked some guys to take a look at it but if somebody familiar with 
this part of code, feel free to take a look at the changes too(especially 
scripts changes).

I also think it is good to decide which release it should be planned on. This 
task planned for 2.9 right now but I should notice that first of all there are 
a lot of changes and secondly there are some changes in public API(removed some 
methods from configuration). So maybe it makes sense to move this ticket to the 
next release. What do you think?

-- 
Best regards,
Anton Kalashnikov


10.02.2020, 15:45, "Alexey Zinoviev" :
> Thank you so you much! Will wait:)
>
> пн, 10 февр. 2020 г. в 15:13, Alexey Goncharuk :
>
>>  Got it, then no need to rush, let's wait for the TF-IGFS decoupling.
>>
>>  пн, 10 февр. 2020 г. в 13:15, Alexey Zinoviev :
>>
>>  > Tensorflow integration uses IGFS, if you have any idea how to store files
>>  > in memory by another way, please suggest something.
>>  > I hope to decouple Ignite-TF integration to the separate repository
>>  before
>>  > release 2.9 with its own file system over Ignite Caches
>>  >
>>  > пн, 10 февр. 2020 г. в 12:49, Ivan Pavlukhin :
>>  >
>>  > > Is not it blocked by
>>  > > https://issues.apache.org/jira/browse/IGNITE-10292 as stated in JIRA?
>>  > >
>>  > > @Alex Zinoviev could you please shed some light on this?
>>  > >
>>  > > Best regards,
>>  > > Ivan Pavlukhin
>>  > >
>>  > > пн, 10 февр. 2020 г. в 12:46, Anton Kalashnikov :
>>  > >
>>  > > >
>>  > > > I found the correct ticket for such activity -
>>  > > https://issues.apache.org/jira/browse/IGNITE-11942
>>  > > >
>>  > > > --
>>  > > > Best regards,
>>  > > > Anton Kalashnikov
>>  > > >
>>  > > >
>>  > > > 10.02.2020, 12:16, "Anton Kalashnikov" :
>>  > > > > Hello.
>>  > > > >
>>  > > > > I created a ticket for this activity -
>>  > > https://issues.apache.org/jira/browse/IGNITE-12647. And if we are
>>  still
>>  > > in consensus I'll do it at the nearest time(I've already had the
>>  prepared
>>  > > code).
>>  > > > >
>>  > > > > --
>>  > > > > Best regards,
>>  > > > > Anton Kalashnikov
>>  > > > >
>>  > > > > 10.02.2020, 12:07, "Alexey Goncharuk" >  >:
>>  > > > >> Folks,
>>  > > > >>
>>  > > > >> I think there is a consensus here, but we did not remove IGFS
>>  > > neither in
>>  > > > >> 2.7 nor in 2.8, did we? Should we schedule a corresponding ticket
>>  > > for 2.9?
>>  > >
>>  >


[jira] [Created] (IGNITE-13235) Deadlock in IgniteServiceProcessor

2020-07-09 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-13235:
--

 Summary: Deadlock in IgniteServiceProcessor
 Key: IGNITE-13235
 URL: https://issues.apache.org/jira/browse/IGNITE-13235
 Project: Ignite
  Issue Type: Bug
Reporter: Ivan Bessonov
Assignee: Ivan Bessonov
 Fix For: 2.9


{code:java}
"main" #1 prio=5 os_prio=0 tid=0x7ff9ac00f000 nid=0x86d in Object.wait() 
[0x7ff9b418b000]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at 
org.apache.ignite.internal.util.worker.GridWorker.join(GridWorker.java:242)
- locked <0x000776ee2028> (a java.lang.Object)
at 
org.apache.ignite.internal.util.IgniteUtils.join(IgniteUtils.java:5009)
at 
org.apache.ignite.internal.processors.service.ServiceDeploymentManager.stopProcessing(ServiceDeploymentManager.java:145)
at 
org.apache.ignite.internal.processors.service.IgniteServiceProcessor.stopProcessor(IgniteServiceProcessor.java:261)
at 
org.apache.ignite.internal.processors.service.IgniteServiceProcessor.onKernalStop(IgniteServiceProcessor.java:248)
at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2466)
at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2414)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2577)
- locked <0x000776424138> (a 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2540)
at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:333)
at org.apache.ignite.Ignition.stop(Ignition.java:221)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.stopGrid(GridAbstractTest.java:1225)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.stopAllGrids(GridAbstractTest.java:1268)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.stopAllGrids(GridAbstractTest.java:1246)
at 
org.apache.ignite.events.ClusterActivationStartedEventTest.afterTest(ClusterActivationStartedEventTest.java:41)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.cleanUpTestEnviroment(GridAbstractTest.java:701)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2165)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.access$600(GridAbstractTest.java:172)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$2.evaluate(GridAbstractTest.java:207)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at 
org.apache.ignite.testframework.junits.SystemPropertiesRule.lambda$methodStatement$1(SystemPropertiesRule.java:109)
at 
org.apache.ignite.testframework.junits.SystemPropertiesRule$$Lambda$6/167185492.evaluate(Unknown
 Source)
at 
org.apache.ignite.testframework.junits.DelegatingJUnitStatement.evaluate(DelegatingJUnitStatement.java:48)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at 
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.evaluateInsideFixture(GridAbstractTest.java:2669)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.access$500(GridAbstractTest.java:172)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$BeforeFirstAndAfterLastTestRule$1.evaluate(GridAbstractTest.java:2649)
at 
org.apache.ignite.testframework.junits.SystemPropertiesRule.lambda$classStatement$0(SystemPropertiesRule.java:93)
at 
org.apache.ignite.testframework.junits.SystemPropertiesRule$$Lambda$2/1879492184.evaluate(Unknown
 Source)
at 
org.apache.ignite.testframework.junits.DelegatingJUnitStatement.evaluate(DelegatingJUnitStatement.java:48)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at 
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at 
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRu

[jira] [Created] (IGNITE-13234) SqlFieldsQuery as ContinuousQuery.InitialQuery

2020-07-09 Thread Stephen Darlington (Jira)
Stephen Darlington created IGNITE-13234:
---

 Summary: SqlFieldsQuery as ContinuousQuery.InitialQuery
 Key: IGNITE-13234
 URL: https://issues.apache.org/jira/browse/IGNITE-13234
 Project: Ignite
  Issue Type: New Feature
  Components: platforms
Affects Versions: 2.8.1, 2.8
Reporter: Stephen Darlington


SqlQuery has been deprecated in favour of SqlFieldsQuery, but ContinuousQuery 
in Ignite does not allow SqlFieldsQuery as InitialQuery.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13233) Invalid milliseconds value of java.util.Date converted to java.sql.Timestamp via JDBC Thin.

2020-07-09 Thread Alexander Lapin (Jira)
Alexander Lapin created IGNITE-13233:


 Summary: Invalid milliseconds value of java.util.Date converted to 
java.sql.Timestamp via JDBC Thin.
 Key: IGNITE-13233
 URL: https://issues.apache.org/jira/browse/IGNITE-13233
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Lapin


In some cases, for example in case of new java.util.Date(Long.MAX_VALUE), 
milliseconds part of java.sql.Timestamp retrieved via JDBC has different value 
than previously putted via cache API java.util.Date.
Date.getTime() 9223372036854775*807*
Timestamp.getTime() 9223372036854775*191*

Reproducer 
org.apache.ignite.jdbc.thin.JdbcThinCacheToJdbcDataTypesCoverageTest#testDateDataType



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13232) JDBC thin: it's not possible to use array of objects as value in where _key=

2020-07-09 Thread Alexander Lapin (Jira)
Alexander Lapin created IGNITE-13232:


 Summary: JDBC thin: it's not possible to use array of objects as 
value in where _key=
 Key: IGNITE-13232
 URL: https://issues.apache.org/jira/browse/IGNITE-13232
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Lapin


Reproducers:

org.apache.ignite.jdbc.thin.JdbcThinCacheToJdbcDataTypesCoverageTest#testObjectArrayDataType



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13231) Unable to use character in where _key= clause via JDBC Thin, if previously putted within cache API.

2020-07-09 Thread Alexander Lapin (Jira)
Alexander Lapin created IGNITE-13231:


 Summary: Unable to use character in where _key= clause 
via JDBC Thin, if previously putted within cache API.
 Key: IGNITE-13231
 URL: https://issues.apache.org/jira/browse/IGNITE-13231
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Lapin


Unable to use character in where _key= clause via JDBC Thin, if 
previously putted within cache API.

Following exception will be got:

{color:#172b4d}{{java.sql.{color:#6554c0}SQLException{color}: 
{color:#6554c0}Failed{color} {color:#0052cc}to{color} parse query. 
{color:#6554c0}Hexadecimal{color} string {color:#0052cc}with{color} odd number 
of characters: {color:#36b37e}"a"{color}; SQL statement:SELECT * FROM 
tableefa88adf_7a52_4cac_9a2b_253c4880a369 {color:#6554c0}WHERE{color} _key = 
{color:#36b37e}'a'{color} 
[{color:#0052cc}90003{color}-{color:#0052cc}199{color}]}}{color}

Reproducer: 
org.apache.ignite.jdbc.thin.JdbcThinCacheToJdbcDataTypesCoverageTest#testCharacterDataType



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (IGNITE-13230) Ignite duplicate key and NullPointerException

2020-07-09 Thread Abhay (Jira)
Abhay created IGNITE-13230:
--

 Summary: Ignite duplicate key and NullPointerException
 Key: IGNITE-13230
 URL: https://issues.apache.org/jira/browse/IGNITE-13230
 Project: Ignite
  Issue Type: Bug
  Components: clients
Affects Versions: 2.8.1
Reporter: Abhay
 Attachments: logFile.txt

[^logFile.txt]

Following steps lead to crash 
 # Start ignite node with persistence enabled , and use ODBC client like isql 
or pyignite
 # Fire create table command e.g
 ## CREATE TABLE ct_countries(id bigint PRIMARY KEY NOT NULL,code VARCHAR(50) 
DEFAULT '',name VARCHAR(100) DEFAULT '',timezonecheck VARCHAR DEFAULT 
'N',dstcheck VARCHAR DEFAULT 'N',phonecodelength VARCHAR(20) DEFAULT '',status 
varchar(10) DEFAULT 'INACTIVE')WITH 
"template=partitioned,backups=0,affinity_key=id";
 # Create index without giving index name and create two such index
 ## CREATE INDEX ON ct_countries(code);
 ## CREATE INDEX ON ct_countries(name);

Restart ignite and it will crash with the follolwing logs 

java.lang.IllegalStateException: Duplicate key
 at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:233)
 at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:184)
 at 
org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)

 

java.lang.IllegalStateException: Duplicate key
 at org.apache.ignite.cache.QueryEntity.checkIndexes(QueryEntity.java:233)
 at org.apache.ignite.cache.QueryEntity.makePatch(QueryEntity.java:184)
 at 
org.apache.ignite.internal.processors.query.QuerySchema.makePatch(QuerySchema.java:114)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [DISCUSSION] Ignite integration testing framework.

2020-07-09 Thread Max Shonichev

Anton,

well, strange thing, but clean up and rerun helped.


Ubuntu 18.04


SESSION REPORT (ALL TESTS)
ducktape version: 0.7.7
session_id:   2020-07-06--003
run time: 4 minutes 44.835 seconds
tests run:5
passed:   5
failed:   0
ignored:  0

test_id: 
ignitetest.tests.benchmarks.add_node_rebalance_test.AddNodeRebalanceTest.test_add_node.version=2.8.1

status: PASS
run time:   41.927 seconds
{"Rebalanced in (sec)": 1.02205491065979}

test_id: 
ignitetest.tests.benchmarks.add_node_rebalance_test.AddNodeRebalanceTest.test_add_node.version=dev

status: PASS
run time:   51.985 seconds
{"Rebalanced in (sec)": 0.0760810375213623}

test_id: 
ignitetest.tests.benchmarks.pme_free_switch_test.PmeFreeSwitchTest.test.version=2.7.6

status: PASS
run time:   1 minute 4.283 seconds
{"Streamed txs": "1900", "Measure duration (ms)": "34818", "Worst 
latency (ms)": "31035"}


test_id: 
ignitetest.tests.benchmarks.pme_free_switch_test.PmeFreeSwitchTest.test.version=dev

status: PASS
run time:   1 minute 13.089 seconds
{"Streamed txs": "73134", "Measure duration (ms)": "35843", "Worst 
latency (ms)": "139"}


test_id: 
ignitetest.tests.spark_integration_test.SparkIntegrationTest.test_spark_client

status: PASS
run time:   53.332 seconds



MacBook

SESSION REPORT (ALL TESTS)
ducktape version: 0.7.7
session_id:   2020-07-06--001
run time: 6 minutes 58.612 seconds
tests run:5
passed:   5
failed:   0
ignored:  0

test_id: 
ignitetest.tests.benchmarks.add_node_rebalance_test.AddNodeRebalanceTest.test_add_node.version=2.8.1

status: PASS
run time:   48.724 seconds
{"Rebalanced in (sec)": 3.2574470043182373}

test_id: 
ignitetest.tests.benchmarks.add_node_rebalance_test.AddNodeRebalanceTest.test_add_node.version=dev

status: PASS
run time:   1 minute 23.210 seconds
{"Rebalanced in (sec)": 2.165921211242676}

test_id: 
ignitetest.tests.benchmarks.pme_free_switch_test.PmeFreeSwitchTest.test.version=2.7.6

status: PASS
run time:   1 minute 12.659 seconds
{"Streamed txs": "642", "Measure duration (ms)": "33177", "Worst latency 
(ms)": "31063"}


test_id: 
ignitetest.tests.benchmarks.pme_free_switch_test.PmeFreeSwitchTest.test.version=dev

status: PASS
run time:   1 minute 57.257 seconds
{"Streamed txs": "32924", "Measure duration (ms)": "48252", "Worst 
latency (ms)": "1010"}


test_id: 
ignitetest.tests.spark_integration_test.SparkIntegrationTest.test_spark_client

status: PASS
run time:   1 minute 36.317 seconds

=

while relative numbers proportion remains the same for different Ignite 
versions, absolute number for mac/linux differ more then twice.


I'm finalizing code with 'local Tiden' appliance for your tests.  PR 
would be ready soon.


Have you had a chance to deploy ducktests in bare metal?



On 06.07.2020 14:27, Anton Vinogradov wrote:

Max,

Thanks for the check!


Is it OK for those tests to fail?

No.
I see really strange things at logs.
Looks like you have concurrent ducktests run started not expected services,
and this broke the tests.
Could you please clean up the docker (use clean-up script [1]).
Compile sources (use script [2]) and rerun the tests.

[1]
https://github.com/anton-vinogradov/ignite/blob/dc98ee9df90b25eb5d928090b0e78b48cae2392e/modules/ducktests/tests/docker/clean_up.sh
[2]
https://github.com/anton-vinogradov/ignite/blob/3c39983005bd9eaf8cb458950d942fb592fff85c/scripts/build.sh

On Mon, Jul 6, 2020 at 12:03 PM Nikolay Izhikov  wrote:


Hello, Maxim.

Thanks for writing down the minutes.

There is no such thing as «Nikolay team» on the dev-list.
I propose to focus on product requirements and what we want to gain from
the framework instead of taking into account the needs of some team.

Can you, please, write