Hadoop and hive not working after setting ignite igfs.

[pawanpawar]

Hi team I am getting error after configuring ignite with hadoop and hive.
when I try to run any hadoop cmd it show me this error.

pawanpawar@cloud:~$ hadoop fs -ls -R /
-ls: Fatal internal error
java.lang.RuntimeException: java.lang.NoSuchMethodException:
org.apache.ignite.hadoop.fs.v2.IgniteHadoopFileSystem.()
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131)
at 
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2595)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at 
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:354)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:325)
at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:224)
at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:207)
at 
org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:190)
at org.apache.hadoop.fs.shell.Command.run(Command.java:154)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:287)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
Caused by: java.lang.NoSuchMethodException:
org.apache.ignite.hadoop.fs.v2.IgniteHadoopFileSystem.()
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.getDeclaredConstructor(Class.java:2178)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:125)
... 17 more


and when I try to start hiveserver2 it gives me same error.

I have setup the HADOOP_CLASSPATH and other configuration as shown in
tutorial https://apacheignite-fs.readme.io/v1.5.0



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/Hadoop-and-hive-not-working-after-setting-ignite-igfs-tp8178.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

[jira] [Created] (IGNITE-2921) ScanQueries over local partitions are not optimal

[Denis Magda (JIRA)]

Denis Magda created IGNITE-2921:
---

 Summary: ScanQueries over local partitions are not optimal
 Key: IGNITE-2921
 URL: https://issues.apache.org/jira/browse/IGNITE-2921
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.5.0.final
Reporter: Denis Magda
Priority: Blocker
 Fix For: 1.6


Presently scan queries over local partitions are not executed optimally. 

If to run a scan query over a specific partition (by setting 
{{query.setPartition(...)}} parameter and or {{query.setLocal(true)}}) and 
start iterating over entries we will see that the Thread, that iterates over 
the data, waits for some event to happen.

In fact the Thread waits while a system pool's thread prepares an iterator with 
entries for it and only after that iterates over the returned result set. The 
flow looks this way:
- {{GridCacheLocalQueryFuture}} is created;
- when {{QueryCursor.iterator().next}} is called from the app thread (the 
Thread above), {{GridCacheLocalQueryFuture.execute()}} methods puts closure 
that will prepare content for the iterator in the system pool.
-  a system Thread execute {{GridCacheQueryManager.runQuery()}} reading all the 
entries from partition and passing them back to the Thread at line 1553 by 
calling {{onPageReady(...)}} method.

The other bottleneck is that a system thread gets all the entries and passes 
them to the Thread which will lead to more garbaged Java heap especially if 
cache is {{OFFHEAP_TIRED}}.

Run attached test ({{ScanQueryStuff}}) and you will see with Visual VM that 
most of the time the test spends executing the code from system threads.

Finally, what have to be done:
- if ScanQuery is supposed to be executed locally (setPartition() refers to 
local partition or setLocal is set to true) then the calling application thread 
has to iterate over the data avoiding usage of the system pool;
- internal code mustn't read all entries from a partition initially. The 
iterator has to get one entry next after another. This will be a memory 
backpressure mechanism especially for {{OFFHEAP_TIRED}}.

My assumption is that the fixed version has to work in a similar way to 
iteration over local entries - {{cache.localEntries(CachePeekMode.PRIMARY);}}. 
Run attached {{LocalIteratorStuff}} to see with Visual VM that the application 
thread is fully utilized and system threads are idle. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request: IGNITE-2920: IgniteError now derives from std...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/589

IGNITE-2920: IgniteError now derives from std::runtime_error.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/isapego/ignite ignite-2920

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/589.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #589


commit 2b1db571cea0cad26498939e927453a9092adad1
Author: isapego 
Date:   2016-03-30T17:46:04Z

IGNITE-2920: IgniteError now derives from std::runtime_error.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-2920) CPP: IgniteError should be derived from std::runtime_error not from std::exception

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-2920:
---

 Summary: CPP: IgniteError should be derived from 
std::runtime_error not from std::exception
 Key: IGNITE-2920
 URL: https://issues.apache.org/jira/browse/IGNITE-2920
 Project: Ignite
  Issue Type: Task
  Components: platforms
Affects Versions: 1.5.0.final
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 1.6


It seems like std::exception is not supposed to be used as a base class by user 
classes. It is suggested to use std::runtime_error instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-2919) Visor CMD: Add ability to attach custom scripts to alerts

[Denis Magda (JIRA)]

Denis Magda created IGNITE-2919:
---

 Summary: Visor CMD: Add ability to attach custom scripts to alerts
 Key: IGNITE-2919
 URL: https://issues.apache.org/jira/browse/IGNITE-2919
 Project: Ignite
  Issue Type: Bug
  Components: visor
Affects Versions: 1.5.0.final
Reporter: Denis Magda
Assignee: Alexey Kuznetsov
 Fix For: 1.6


Visor cmd support "alert" command that allows to listen on particular triggers 
(heap usage, nodes count, ...).

If a trigger is fired Visor will print info into the log file. 

However it makes sense specify a custom script that will be triggered when a 
trigger is fired.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Apache Flink <=> Apache Ignite integration

[Raul Kripalani]

On Wed, Mar 30, 2016 at 2:20 PM, Roman  wrote:

> Raul,
>
> Small comment from me.
>
> >* As a Flink sink => inject data directly into a cache via a DataStreamer.
> After reviews, IGNITE-813 is exactly this functionality.
>
>
That's cool, Roman! The idea would be to host these (richer) modules as
Flink connectors, like they do with others:

https://github.com/apache/flink/tree/master/flink-streaming-connectors
https://github.com/apache/flink/tree/master/flink-batch-connectors

[GitHub] ignite pull request: IGNITE-2694 .NET: AnyCPU build

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/546


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (IGNITE-2918) CPP: Add C++ test suits to TeamCity.

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-2918:
---

 Summary: CPP: Add C++ test suits to TeamCity.
 Key: IGNITE-2918
 URL: https://issues.apache.org/jira/browse/IGNITE-2918
 Project: Ignite
  Issue Type: Task
  Components: platforms
Affects Versions: 1.5.0.final
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 1.6


We need to add to TC at least following test suits:
1. Win 64 suit
2. Win 32 suit
3 Linux suit




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request: GG-10988 : GridCacheConcurrentMap redesign.

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/587

GG-10988 : GridCacheConcurrentMap redesign.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ilantukh/ignite ignite-gg-10988

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/587.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #587


commit 81650236aa12e8a45f1b8a815596f37904e77b33
Author: Ilya Lantukh 
Date:   2016-03-14T07:44:09Z

gg-10988 : GridCacheConcurrentMap usage redesign.

commit 734686dea51390b26cf649fad7c55ce897bf1e88
Author: Ilya Lantukh 
Date:   2016-03-14T13:22:05Z

Merge branch 'master' of https://github.com/gridgain/apache-ignite into 
gg-10988

commit 2c4bd18a8177d01d709ba677531358ee0f910e79
Author: Ilya Lantukh 
Date:   2016-03-14T14:59:18Z

gg-10988 : Continued GridCacheConcurrentMap usage redesign.

commit b73ebfeec3d61562836b91fbde38fd7004fb223b
Author: Ilya Lantukh 
Date:   2016-03-15T15:45:34Z

gg-10988 : WIP

commit 6af51f4f8d1fe7bbd15270d888a4457fd2d44892
Author: Ilya Lantukh 
Date:   2016-03-16T13:31:10Z

gg-10988 : Deferred delete fix.

commit 9769e1dbcfd4a19d86b2665a47b90f0cbdb290dd
Author: Ilya Lantukh 
Date:   2016-03-16T16:40:16Z

gg-10988 : WIP

commit aed503574c64a37168121c2ec803b80ca84d125b
Author: Ilya Lantukh 
Date:   2016-03-17T12:16:26Z

gg-10988 : Minors.

commit d6d6b35d91252f570f26868b2c144f124b3b5a00
Author: Ilya Lantukh 
Date:   2016-03-17T13:15:04Z

gg-10988 : Fixed memory usage and type casting problems.

commit e6486508c1ec728279c07eefa0b87fed1fa3e65c
Author: Ilya Lantukh 
Date:   2016-03-17T13:16:16Z

Merge branch 'master' of https://github.com/gridgain/apache-ignite into 
gg-10988

commit c38b37962c89a1f35f245bd4fd31a297b65751bd
Author: Ilya Lantukh 
Date:   2016-03-17T14:15:39Z

gg-10988 : Restored previously outcommented functionality.

commit 66e1b5edb7d96caa69739dba048c1d4d04b2b0a4
Author: Ilya Lantukh 
Date:   2016-03-17T14:30:17Z

gg-10988 : Fixed local cache.

commit 3fcc98e856bf1a711fe82a847a0194235c2adf7d
Author: Ilya Lantukh 
Date:   2016-03-18T09:58:36Z

gg-10988 : Fixed java 7 compatibility.

commit 7196c080917dcc8d7e2b121d939ed3ffa1f315c8
Author: Ilya Lantukh 
Date:   2016-03-18T12:17:34Z

gg-10988 : Fixed NPE on GridCacheAdapter initialization.

commit b18a96e2b7462baedb74bb849a39dc954af82c58
Author: Ilya Lantukh 
Date:   2016-03-18T12:19:18Z

Merge remote-tracking branch 'origin/gg-10988' into gg-10988

commit 53c6efdc6cd34c9cecb4dde0c775c6647e39d508
Author: Ilya Lantukh 
Date:   2016-03-18T12:37:10Z

gg-10988 : Fixed old class casts.

commit 01833e3911f33f4627201572cc1b00495445150a
Author: Ilya Lantukh 
Date:   2016-03-18T13:32:09Z

gg-10988 : Javadocs, renaming.

commit 64e5a935d26a50f119efa7452275216a2d5eca9f
Author: Ilya Lantukh 
Date:   2016-03-18T14:47:48Z

gg-10988 : Fixed near cache entry removal.

commit c022bb7eb0772092212d66db1defc68acbf23015
Author: Ilya Lantukh 
Date:   2016-03-21T09:14:17Z

gg-10988 : Fixed possible deadlock on entry removal.

commit a0d728803f8820b3f2ecc1e83e059db5310ef21f
Author: Ilya Lantukh 
Date:   2016-03-21T12:14:19Z

gg-10988 : Fixed KeyCacheObject conversion.

commit e67e284a8964175516fc268b4f80bf4c32776d99
Author: Ilya Lantukh 
Date:   2016-03-21T12:15:05Z

Merge remote-tracking branch 'origin/gg-10988' into gg-10988

commit 412d9a4c8307eeaa5f3fcadd64837ece008ab88d
Author: Ilya Lantukh 
Date:   2016-03-21T13:38:30Z

gg-10988 : Deferred delete - WIP.

commit dfc8f4c0724620b7281fc865e5758c5041a31304
Author: Ilya Lantukh 
Date:   2016-03-21T14:07:19Z

gg-10988 : Deferred delete and public map size.

commit eb11e459e7bd0f3b7e93501f4a413d5b8e0c22dc
Author: Ilya Lantukh 
Date:   2016-03-21T14:12:32Z

gg-10988 : Synchronization.

commit d3cec70e2d994673eec283e1bd7feb089e774632
Author: Ilya Lantukh 
Date:   2016-03-21T14:27:45Z

gg-10988 : Restored deferred delete logic for near cache.

commit 69d9dee4161ec61d6235f6fc48b3478dabb88b1b
Author: Ilya Lantukh 
Date:   2016-03-22T12:03:37Z

gg-10988 : Fixed #putEntryIfObsoleteOrAbsent logic

commit 79ae8a4606ee40748df410487857f99e0ec3ec3d
Author: Ilya Lantukh 

[jira] [Created] (IGNITE-2916) ODBC: Merge host and port into a single property.

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-2916:
---

 Summary: ODBC: Merge host and port into a single property.
 Key: IGNITE-2916
 URL: https://issues.apache.org/jira/browse/IGNITE-2916
 Project: Ignite
  Issue Type: Task
  Components: odbc
Affects Versions: 1.5.0.final
Reporter: Vladimir Ozerov
Assignee: Igor Sapego
Priority: Critical
 Fix For: 1.6


See {{OdbcConfiguration}}. Currently we specify host and port separately. 
Instead, we should do the following:
1) Merge them in a single String property "address".
2) It might be set as either "host:port" or "host:port1..port2". In the last 
case we should iterate over the list of available ports and bind to the first 
not occupied.
3) Add validation code which should be triggered on ODBC processor start.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-2915) ODBC: Remove fine-grained properties from OdbcConfiguration.

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-2915:
---

 Summary: ODBC: Remove fine-grained properties from 
OdbcConfiguration.
 Key: IGNITE-2915
 URL: https://issues.apache.org/jira/browse/IGNITE-2915
 Project: Ignite
  Issue Type: Task
  Components: odbc
Affects Versions: 1.5.0.final
Reporter: Vladimir Ozerov
Assignee: Igor Sapego
Priority: Critical
 Fix For: 1.6


We need to hide all fine-grained properties from configuration for now. All of 
them are used for performance tuning. Probably it make sense to move them to a 
separate bean or so. Anyway, we can do that in future. For now let's just 
remove them and set sensible defaults to NIO server.

- idleTimeout
- receiveBufferSize
- selectorCount
- sendBufferSize
- sendQueueLimit
- directBuffer
- noDelay



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] ignite pull request: IGNITE-2914: Removed MakeDate and MakeTimesta...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/586

IGNITE-2914: Removed MakeDate and MakeTimestamp.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/isapego/ignite ignite-2914

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/586.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #586


commit 1b9a1b6cda95cdd9b83884859b7f010de4b0304f
Author: isapego 
Date:   2016-03-15T14:39:24Z

IGNITE-2823: Splitted common in two libs.

commit 6368c25720270817f12fee6f751178096d9a1010
Author: isapego 
Date:   2016-03-15T14:53:43Z

IGNITE-2823: Moved utils lib files to "utils" include dir.

commit 5d01177e4e9b89c7b7d7d4dc3002ce69b2a61438
Author: isapego 
Date:   2016-03-15T15:04:16Z

IGNITE-2823: Moved utils to its own namespace.

commit 73f9ab6af21a2b8aaac06b836bf7ca59da16182e
Author: Igor Sapego 
Date:   2016-03-15T15:37:16Z

IGNITE-2823: Autotools build of utils implemented.

commit c41bad6f60d02c297c95d6698d98773a0c07b100
Author: Igor Sapego 
Date:   2016-03-15T16:23:08Z

IGNITE-2823: Fixes for the Autotools build system.

commit 7ab9732f8d169f3e7bde8f12fc15aa76721d4c00
Author: isapego 
Date:   2016-03-15T16:56:19Z

IGNITE-2823: minor refactoring.

commit b26cdc41a96f51780991e67e171c5000e931a33e
Author: isapego 
Date:   2016-03-17T15:45:53Z

Merge branch 'ignite-1786' into ignite-2823

commit d41afdd528abdfe05c8a3782a0b4cd6d907ae83d
Author: isapego 
Date:   2016-03-23T12:25:52Z

Merge branch 'ignite-1786' into ignite-2823

commit 7e2357d27f592564a5e8a312307348e66d98cbac
Author: isapego 
Date:   2016-03-23T12:43:04Z

IGNITE-2823: Merge-related fixes.

commit c146fc4d5c136706044a1b19a2a5b16552d9550c
Author: isapego 
Date:   2016-03-23T13:54:51Z

IGNITE-2823: Renamed: common => jni, utils => common.

commit c68d7907367a9c27218cbe7590cb8baddc44a26d
Author: Igor Sapego 
Date:   2016-03-23T14:30:08Z

IGNITE-2823: Fixes for Linux.

commit 21f20335d7ba1cb9e48908053a342f188d33d7fb
Author: isapego 
Date:   2016-03-23T14:48:09Z

IGNITE-2823: Renamed namspaces: common => jni, utils => common.

commit 7450a5bfc1aba6e91ac22ca66622e27928d6f117
Author: isapego 
Date:   2016-03-23T15:05:17Z

IGNITE-2823: Fix for Debug version.

commit a1da841d566b2df1221d6e1d2a836704ca9f1ef3
Author: isapego 
Date:   2016-03-23T15:38:19Z

IGNITE-2823: Fixes for .NET

commit 9d291715997984d72db112dd7cdea888b5df2209
Author: Igor Sapego 
Date:   2016-03-23T17:56:27Z

Save

commit 24e16cd2d95e9c310deaf8d95360a3f96527067e
Author: Igor Sapego 
Date:   2016-03-28T16:11:33Z

IGNITE-2823: Autotools build system reworked.

commit 34ef5166f6b63c6e14f3c4d4e6adbbec8fe317d9
Author: Igor Sapego 
Date:   2016-03-28T16:49:19Z

IGNITE-2823: Added headers to DIST.

commit a377091a16bb5125a2a376ae24091bdeffae01a1
Author: Igor Sapego 
Date:   2016-03-28T17:06:41Z

IGNITE-2823: Removed examples from the root cfg.

commit 0fd7b014d037bcf210253de6a83b5e70789cc47d
Author: Igor Sapego 
Date:   2016-03-29T14:01:41Z

IGNITE-2823: Minor fixes. column_test.cpp added to odbc-tests.

commit e46afd89217e99dc6ed1dc7f5695c1883559aec6
Author: Igor Sapego 
Date:   2016-03-29T14:23:36Z

IGNITE-2823: Added core/ignite.pc to AC_CONFIG_FILES.

commit ac15b2c1fd7b06c96e97bde2302440c2c1331d8f
Author: Igor Sapego 
Date:   2016-03-29T14:33:31Z

IGNITE-2823: Added release version of configure.ac and Makefile.am.

commit 052bc51d722cbe8d88a20e7287b57ad09535ce32
Author: Igor Sapego 
Date:   2016-03-29T14:57:06Z

IGNITE-2823: Examples build reworked

commit c77723d80f35928b8f38b54bff35a3f89b0713cd
Author: Igor Sapego 
Date:   2016-03-29T15:28:54Z

IGNITE-2823: Fixed assembly.

commit 05f762a828b041a75d61296251f8b9eefba17564
Author: Igor Sapego 
Date:   2016-03-29T15:33:36Z

Merge branch 'ignite-1786' into ignite-2823

commit 5ea069a4bf76016c42d76d86d05b35ce6907
Author: Igor Sapego 
Date:   2016-03-29T15:39:04Z

IGNITE-2823: Enable ODBC and Tests by default for non-release builds.

commit 3f1e5087f4241aac2bd1edfb3927d7f6960bb4e6
Author: Igor Sapego 
Date:   2016-03-29T16:36:45Z

IGNITE-2823: Fixes for 'make [un]install' command.

commit 

Apache Flink <=> Apache Ignite integration

[Raul Kripalani]

Hello from the Apache Ignite community!

Last year there was an interesting thread [1] about such integration.
Unfortunately there's been little follow-through, so let's try and fix that
in 2016 ;-)

I'm sure a lot has changed in the Flink community, with the recent
graduation and 1.0 release, so I'd like to make a new (updated) list of
synergies and areas of integration I can think of:

+++ *Ignite as a bidirectional Connector* +++

The first and obvious integration point is Ignite as a source and a sink of
Flink. An Ignite contributor has already sent a pull request [2] to serve
as a sink into Ignite Queues, but I feel this integration can be deeper and
more functional. Moreover, it should be hosted in the Flink source tree as
a Connector (like the Kafka, or ES connectors). Particularly, we could
offer these features:

* As a Flink sink => inject data directly into a cache via a DataStreamer.
* As a Flink source => run a continuous query against one or multiple
caches [4].

+++ *Ignite as a state backend* +++

Either natively [5] or via the IGFS (Ignite Filesystem) interface which can
run as a Hadoop Filesystem [6].

This would allow Flink to store intermediate states in Ignite. I believe
this is what you called "distributed backup for Streaming Operator State"
in the initial exchange, is it?

+++ *Ignite as a DataSet API connector* +++

Ability to use Ignite as a source for batch pipelines, by executing Ignite
SQL queries [7] against a cache and feeding the results into a Flink
pipeline. Basically a batch counterpart to the streaming continuous query
idea above.

+++ *Ignite as an execution backend* +++

You already mentioned this in [1] and I think it makes for a perfect
synergy between both projects, through Ignite's Compute API.

Still agree with this? Any changes since last year I should take into
account?

+++ *Ignite as a parameter server* +++

This was in the initial proposal [1], but it's not clear to me. I have
found references to the idea of a Parameter Server in Flink, but only as
proposed ideas. Was this feature finally implemented, or is it in the
future roadmap?



This is just a newer, updated proposal from my side, but I'm sure that both
communities can, and will want to, chime in!

Cheers,

[1]
https://mail-archives.apache.org/mod_mbox/flink-dev/201504.mbox/%3CCANC1h_u__KgsdOo2SZ4M=8jf3zomozs3xbekq0erjj9p4wf...@mail.gmail.com%3E
[2] https://issues.apache.org/jira/browse/IGNITE-813
[3] https://ignite.apache.org/features/streaming.html
[4] http://apacheignite.gridgain.org/v1.5/docs/continuous-queries
[5] https://apacheignite-fs.readme.io/docs/igfs
[6] https://apacheignite-fs.readme.io/docs/file-system
[7] https://apacheignite.readme.io/docs/sql-queries

*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
Blog: raul.io | twitter: @raulvk 

[jira] [Created] (IGNITE-2913) SQL query with EXISTS() expression fails on partitioned cache.

[Sergi Vladykin (JIRA)]

Sergi Vladykin created IGNITE-2913:
--

 Summary: SQL query with EXISTS() expression fails on partitioned 
cache.
 Key: IGNITE-2913
 URL: https://issues.apache.org/jira/browse/IGNITE-2913
 Project: Ignite
  Issue Type: Bug
Reporter: Sergi Vladykin


select name from Person p where exists(select 1 from Organization o where 
p.orgId = o.id)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: API for asynchronous execution.

[Yakov Zhdanov]

I think this approach works unless user does not initiate number of
concurrent cache operations greater than MSG_QUEUE_SIZE.  Where msg queue
size default is 1024, but still configurable.

Thanks!
--
Yakov Zhdanov, Director R
*GridGain Systems*
www.gridgain.com

2016-03-30 11:44 GMT+03:00 Vladimir Ozerov :

> Does it mean that if cache update rate is greater than filter execution
> rate, then at some point we will stop reading messages from socket? If yes,
> then it seems we still cannot execute cache operations:
> 1) Filter starts cache operation for a key. Current node is backup for this
> key.
> 2) Cache message is sent to primary node
> 3) Primary sends message back to current node.
> 4) Message is never read because of backpressure. Cache operation and
> filter never complete.
>
> Am I missing something?
>
> On Wed, Mar 30, 2016 at 11:23 AM, Yakov Zhdanov 
> wrote:
>
> > Vladimir,
> >
> > Communication should stop reading from connection is there are too many
> > unprocessed messages. Sender will be blocked on putting message to queue.
> >
> > --Yakov
> >
> > 2016-03-30 11:11 GMT+03:00 Vladimir Ozerov :
> >
> > > Guys,
> > >
> > > Can you explain how backpressure control is implemented? What if event
> > > arrival speed is greater than filter processing speed?
> > >
> > > On Wed, Mar 30, 2016 at 10:37 AM, Semyon Boikov 
> > > wrote:
> > >
> > > > Andrey,
> > > >
> > > > I agree that current situation with threading in Ignite is very
> > > > inconvenient when user callbacks execute some non-trivial code. But
> > > > changing this to async dispatch is huge refactoring, even changing
> this
> > > > just for continuous queries callback is not so easy task.
> > > >
> > > > We can start with https://issues.apache.org/jira/browse/IGNITE-2004,
> > and
> > > > if
> > > > more users complains arise we can think about changing others parts
> of
> > > > system.
> > > >
> > > > For now we need decisions for these points:
> > > > - how to specify that callback should be run asynchronously (Nikolay
> > > > suggested marker interface IgniteAsyncCallback, or
> > @IgniteAsyncCallback)
> > > > - where these callbacks are executed, AFAIK Nikolay added special
> pool
> > > > which is configured in IgniteConfiguration (something like
> > > > IgniteConfiguration.asyncCallbackThreadPoolSize)
> > > >
> > > > Regards
> > > >
> > > >
> > > > On Tue, Mar 29, 2016 at 10:45 PM, Andrey Kornev <
> > > andrewkor...@hotmail.com>
> > > > wrote:
> > > >
> > > > > Vladimir, Igniters
> > > > >
> > > > > Here are my 2 cents.
> > > > >
> > > > > The current situation with threading when it comes to executing
> user
> > > > > callbacks -- the CQ filters (either local or remote), the CQ
> > listeners,
> > > > the
> > > > > event listeners, the messaging listeners, the entry processors
> (did I
> > > > miss
> > > > > anything?) -- is pretty sad. The callbacks may get executed on a
> > system
> > > > > pool's thread, public pool's, utility pool's, discovery worker
> > thread,
> > > > > application thread, to name a few. It causes a lot of grief and
> > > > suffering,
> > > > > hard-to-fix races, dead locks and other bugs.
> > > > >
> > > > > I guess it's always possible to come up with a more or less
> > reasonable
> > > > > explanation to such predicament (which usually boils down to "It is
> > so
> > > > > because this is how it's implemented"), but I, as a user, could not
> > > care
> > > > > less. I want consistency. I want all my callbacks (including Entry
> > > > > Processors!) to be executed on the public pool's threads, to be
> > > precise.
> > > > > This is not the first time I complain about this, and I really
> think
> > > it's
> > > > > time to fix this mess.
> > > > >
> > > > > For a good example of how to implement ordered async dispatch of
> > > > callbacks
> > > > > on large scale, one only needs to look at Akka (or Reactor
> > > > > https://github.com/reactor/reactor).  Coherence also managed to
> get
> > it
> > > > > right (in my opinion, that is).
> > > > >
> > > > > Regards
> > > > > Andrey
> > > > >
> > > > >
> > > >
> > >
> >
>

[jira] [Created] (IGNITE-2912) Implement support focus out to composite field

[Dmitriyff (JIRA)]

Dmitriyff created IGNITE-2912:
-

 Summary: Implement support focus out to composite field
 Key: IGNITE-2912
 URL: https://issues.apache.org/jira/browse/IGNITE-2912
 Project: Ignite
  Issue Type: Sub-task
Reporter: Dmitriyff
Assignee: Dmitriyff






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (IGNITE-2911) ClusterGroup.forDaemons does not work

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-2911:
--

 Summary: ClusterGroup.forDaemons does not work
 Key: IGNITE-2911
 URL: https://issues.apache.org/jira/browse/IGNITE-2911
 Project: Ignite
  Issue Type: Bug
  Components: compute
Affects Versions: 1.1.4
Reporter: Pavel Tupitsyn
Priority: Critical
 Fix For: 1.6


* Start a node in daemon mode
* ClusterGroup.forDaemons returns an empty projection

The only existing test for this method only checks for empty result.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: API for asynchronous execution.

[Yakov Zhdanov]

Vladimir,

Communication should stop reading from connection is there are too many
unprocessed messages. Sender will be blocked on putting message to queue.

--Yakov

2016-03-30 11:11 GMT+03:00 Vladimir Ozerov :

> Guys,
>
> Can you explain how backpressure control is implemented? What if event
> arrival speed is greater than filter processing speed?
>
> On Wed, Mar 30, 2016 at 10:37 AM, Semyon Boikov 
> wrote:
>
> > Andrey,
> >
> > I agree that current situation with threading in Ignite is very
> > inconvenient when user callbacks execute some non-trivial code. But
> > changing this to async dispatch is huge refactoring, even changing this
> > just for continuous queries callback is not so easy task.
> >
> > We can start with https://issues.apache.org/jira/browse/IGNITE-2004, and
> > if
> > more users complains arise we can think about changing others parts of
> > system.
> >
> > For now we need decisions for these points:
> > - how to specify that callback should be run asynchronously (Nikolay
> > suggested marker interface IgniteAsyncCallback, or @IgniteAsyncCallback)
> > - where these callbacks are executed, AFAIK Nikolay added special pool
> > which is configured in IgniteConfiguration (something like
> > IgniteConfiguration.asyncCallbackThreadPoolSize)
> >
> > Regards
> >
> >
> > On Tue, Mar 29, 2016 at 10:45 PM, Andrey Kornev <
> andrewkor...@hotmail.com>
> > wrote:
> >
> > > Vladimir, Igniters
> > >
> > > Here are my 2 cents.
> > >
> > > The current situation with threading when it comes to executing user
> > > callbacks -- the CQ filters (either local or remote), the CQ listeners,
> > the
> > > event listeners, the messaging listeners, the entry processors (did I
> > miss
> > > anything?) -- is pretty sad. The callbacks may get executed on a system
> > > pool's thread, public pool's, utility pool's, discovery worker thread,
> > > application thread, to name a few. It causes a lot of grief and
> > suffering,
> > > hard-to-fix races, dead locks and other bugs.
> > >
> > > I guess it's always possible to come up with a more or less reasonable
> > > explanation to such predicament (which usually boils down to "It is so
> > > because this is how it's implemented"), but I, as a user, could not
> care
> > > less. I want consistency. I want all my callbacks (including Entry
> > > Processors!) to be executed on the public pool's threads, to be
> precise.
> > > This is not the first time I complain about this, and I really think
> it's
> > > time to fix this mess.
> > >
> > > For a good example of how to implement ordered async dispatch of
> > callbacks
> > > on large scale, one only needs to look at Akka (or Reactor
> > > https://github.com/reactor/reactor).  Coherence also managed to get it
> > > right (in my opinion, that is).
> > >
> > > Regards
> > > Andrey
> > >
> > >
> >
>

Re: API for asynchronous execution.

[Vladimir Ozerov]

Guys,

Can you explain how backpressure control is implemented? What if event
arrival speed is greater than filter processing speed?

On Wed, Mar 30, 2016 at 10:37 AM, Semyon Boikov 
wrote:

> Andrey,
>
> I agree that current situation with threading in Ignite is very
> inconvenient when user callbacks execute some non-trivial code. But
> changing this to async dispatch is huge refactoring, even changing this
> just for continuous queries callback is not so easy task.
>
> We can start with https://issues.apache.org/jira/browse/IGNITE-2004, and
> if
> more users complains arise we can think about changing others parts of
> system.
>
> For now we need decisions for these points:
> - how to specify that callback should be run asynchronously (Nikolay
> suggested marker interface IgniteAsyncCallback, or @IgniteAsyncCallback)
> - where these callbacks are executed, AFAIK Nikolay added special pool
> which is configured in IgniteConfiguration (something like
> IgniteConfiguration.asyncCallbackThreadPoolSize)
>
> Regards
>
>
> On Tue, Mar 29, 2016 at 10:45 PM, Andrey Kornev 
> wrote:
>
> > Vladimir, Igniters
> >
> > Here are my 2 cents.
> >
> > The current situation with threading when it comes to executing user
> > callbacks -- the CQ filters (either local or remote), the CQ listeners,
> the
> > event listeners, the messaging listeners, the entry processors (did I
> miss
> > anything?) -- is pretty sad. The callbacks may get executed on a system
> > pool's thread, public pool's, utility pool's, discovery worker thread,
> > application thread, to name a few. It causes a lot of grief and
> suffering,
> > hard-to-fix races, dead locks and other bugs.
> >
> > I guess it's always possible to come up with a more or less reasonable
> > explanation to such predicament (which usually boils down to "It is so
> > because this is how it's implemented"), but I, as a user, could not care
> > less. I want consistency. I want all my callbacks (including Entry
> > Processors!) to be executed on the public pool's threads, to be precise.
> > This is not the first time I complain about this, and I really think it's
> > time to fix this mess.
> >
> > For a good example of how to implement ordered async dispatch of
> callbacks
> > on large scale, one only needs to look at Akka (or Reactor
> > https://github.com/reactor/reactor).  Coherence also managed to get it
> > right (in my opinion, that is).
> >
> > Regards
> > Andrey
> >
> >
>

[GitHub] ignite pull request: Ignite-2730: Ignite Events Source Streaming t...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/560


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: API for asynchronous execution.

[Semyon Boikov]

Andrey,

I agree that current situation with threading in Ignite is very
inconvenient when user callbacks execute some non-trivial code. But
changing this to async dispatch is huge refactoring, even changing this
just for continuous queries callback is not so easy task.

We can start with https://issues.apache.org/jira/browse/IGNITE-2004, and if
more users complains arise we can think about changing others parts of
system.

For now we need decisions for these points:
- how to specify that callback should be run asynchronously (Nikolay
suggested marker interface IgniteAsyncCallback, or @IgniteAsyncCallback)
- where these callbacks are executed, AFAIK Nikolay added special pool
which is configured in IgniteConfiguration (something like
IgniteConfiguration.asyncCallbackThreadPoolSize)

Regards


On Tue, Mar 29, 2016 at 10:45 PM, Andrey Kornev 
wrote:

> Vladimir, Igniters
>
> Here are my 2 cents.
>
> The current situation with threading when it comes to executing user
> callbacks -- the CQ filters (either local or remote), the CQ listeners, the
> event listeners, the messaging listeners, the entry processors (did I miss
> anything?) -- is pretty sad. The callbacks may get executed on a system
> pool's thread, public pool's, utility pool's, discovery worker thread,
> application thread, to name a few. It causes a lot of grief and suffering,
> hard-to-fix races, dead locks and other bugs.
>
> I guess it's always possible to come up with a more or less reasonable
> explanation to such predicament (which usually boils down to "It is so
> because this is how it's implemented"), but I, as a user, could not care
> less. I want consistency. I want all my callbacks (including Entry
> Processors!) to be executed on the public pool's threads, to be precise.
> This is not the first time I complain about this, and I really think it's
> time to fix this mess.
>
> For a good example of how to implement ordered async dispatch of callbacks
> on large scale, one only needs to look at Akka (or Reactor
> https://github.com/reactor/reactor).  Coherence also managed to get it
> right (in my opinion, that is).
>
> Regards
> Andrey
>
>

Re: JCache dependency

[Romain Manni-Bucau]

Just checked and our spec jar passes sigtest. Not sure for this week
but think we can run a vote next one if nobody objects - don't
hesitate to ping if nothing happens ;).

Romain Manni-Bucau
@rmannibucau |  Blog | Github | LinkedIn | Tomitriber


2016-03-30 9:20 GMT+02:00 Dmitriy Setrakyan :
> TCK does contain the sigtest:
> https://github.com/jsr107/jsr107tck/tree/master/sigtest
>
> Looking forward to getting the 1.0 version :)
>
> D.
>
> On Tue, Mar 29, 2016 at 11:12 PM, Romain Manni-Bucau 
> wrote:
>>
>> Le 30 mars 2016 01:45, "Dmitriy Setrakyan"  a écrit
>> :
>> >
>> > I just mention to mention that Apache Ignite passes JCache TCK with
>> flying colors :)
>> >
>>
>> True! Totally forgot tck were open! Didn't check sigtest, is it there too?
>> If so nothing blocking a 1.0.
>>
>> > We have it integrated into our build routine and verify it using our CI
>> tests. In addition, it was verified by one of the JCache spec leads, Greg
>> Luck, who confirmed that Ignite complies with the spec.
>> >
>> > Given the above, can Geronimo provide us with JCache 1.0 spec JAR?
>> >
>> > D.
>> >
>> > On Tue, Mar 29, 2016 at 1:41 PM, Romain Manni-Bucau
>> > 
>> wrote:
>> >>
>> >> ok, let me try to make it clearer (and don't hesitate to shout if still
>> not ;)):
>> >>
>> >> TCK are not only @Test but also some bianary validations (aka sigtest
>> >> or signature tests) the spec jars need to pass. It basically checks
>> >> you respect the spec signature for the supported java version of the
>> >> spec. Not having TCK and not being related to a public spec (like BVal
>> >> or JBatch) makes this sigtest validation missing @asf side so until we
>> >> get this or somebody checks generated bytecode of spec jars (and not
>> >> sources) then we'll not use final versions to not show a spec
>> >> compliance we maybe don't have.
>> >>
>> >> Romain Manni-Bucau
>> >> @rmannibucau |  Blog | Github | LinkedIn | Tomitriber
>> >>
>> >>
>> >> 2016-03-29 21:33 GMT+02:00 John D. Ament :
>> >> >
>> >> >
>> >> > On Tue, Mar 29, 2016 at 3:04 PM Dmitriy Setrakyan <
>> dsetrak...@apache.org>
>> >> > wrote:
>> >> >>
>> >> >> We will switch the Ignite JAR to the 1.0-alpha-1 version from
>> Geronimo,
>> >> >> but I am still very confused.
>> >> >>
>> >> >> I do not understand why we need to check any TCK compliance when
>> creating
>> >> >> a JAR for the JSR107 spec. The TCK compliance should be checked
>> against an
>> >> >> implementation, not a spec.
>> >> >>
>> >> >
>> >> > I'm confused by this statement as well.  TCK is only applied to impl
>> so not
>> >> > sure why you might think that.
>> >> >
>> >> > What Romain was trying to convey was that the alpha-1 release
>> indicates that
>> >> > no implementation has checked it as TCK compliant.  One of the JSR
>> >> > requirements though is to produce a valid API JAR.  If someone can do
>> that,
>> >> > then this can likely be promoted to a 1.0 release.
>> >> >
>> >> >>
>> >> >> Is there any place in Apache documentation explaining this process?
>> >> >>
>> >> >> D.
>> >> >>
>> >> >> On Mon, Mar 28, 2016 at 1:57 AM, Romain Manni-Bucau
>> >> >>  wrote:
>> >> >>>
>> >> >>> Le 28 mars 2016 10:15, "Dmitriy Setrakyan" 
>> >> >>> a
>> >> >>> écrit :
>> >> >>> >
>> >> >>> > John,
>> >> >>> >
>> >> >>> > I am still a bit confused. I was talking about the version of the
>> >> >>> > JCache
>> >> >>> spec API, essentially only interfaces. The spec does not have any
>> >> >>> implementation, nor implies that every project importing or
>> depending on
>> >> >>> the spec must be compliant with the spec.
>> >> >>> >
>> >> >>> > In my view implementation and TCK compliance are a different
>> matter,
>> >> >>> > and
>> >> >>> it should be up to the project community itself to declare the
>> compliance
>> >> >>> with a certain spec and pass the TCK.
>> >> >>> >
>> >> >>> > Am I wrong?
>> >> >>> >
>> >> >>>
>> >> >>> Yes, while not passing sigtest practise is to not release 1.0.
>> >> >>>
>> >> >>> > D.
>> >> >>> >
>> >> >>> >
>> >> >>> > On Sun, Mar 27, 2016 at 9:01 AM, John D. Ament <
>> johndam...@apache.org>
>> >> >>> wrote:
>> >> >>> >>
>> >> >>> >> Dmitriy,
>> >> >>> >>
>> >> >>> >> I think what Romain is referring to is other TCKs.  Generally,
>> >> >>> >> geronimo
>> >> >>> JAR versions don't reflect the version of spec that they implement.
>> >> >>> There
>> >> >>> may be alpha releases that match EDRs, or alphas that are based on
>> the
>> >> >>> final version but with minor tweaks.
>> >> >>> >>
>> >> >>> >> For reference, Apache ActiveMQ Artemis relies on alpha2 of the
>> JMS 2
>> >> >>> spec.
>> https://github.com/apache/activemq-artemis/blob/master/pom.xml#L131
>> >> >>> >> It's feature complete, and Artemis passes the TCK, its just
>> >> >>> >> alpha2
>> >> >>> because we haven't done a thorough enough job of making sure the
>> >> >>> API
>> is

Re: JCache dependency

[Dmitriy Setrakyan]

TCK does contain the sigtest:
https://github.com/jsr107/jsr107tck/tree/master/sigtest

Looking forward to getting the 1.0 version :)

D.

On Tue, Mar 29, 2016 at 11:12 PM, Romain Manni-Bucau 
wrote:

> Le 30 mars 2016 01:45, "Dmitriy Setrakyan"  a
> écrit :
> >
> > I just mention to mention that Apache Ignite passes JCache TCK with
> flying colors :)
> >
>
> True! Totally forgot tck were open! Didn't check sigtest, is it there too?
> If so nothing blocking a 1.0.
>
> > We have it integrated into our build routine and verify it using our CI
> tests. In addition, it was verified by one of the JCache spec leads, Greg
> Luck, who confirmed that Ignite complies with the spec.
> >
> > Given the above, can Geronimo provide us with JCache 1.0 spec JAR?
> >
> > D.
> >
> > On Tue, Mar 29, 2016 at 1:41 PM, Romain Manni-Bucau <
> rmannibu...@gmail.com>
> wrote:
> >>
> >> ok, let me try to make it clearer (and don't hesitate to shout if still
> not ;)):
> >>
> >> TCK are not only @Test but also some bianary validations (aka sigtest
> >> or signature tests) the spec jars need to pass. It basically checks
> >> you respect the spec signature for the supported java version of the
> >> spec. Not having TCK and not being related to a public spec (like BVal
> >> or JBatch) makes this sigtest validation missing @asf side so until we
> >> get this or somebody checks generated bytecode of spec jars (and not
> >> sources) then we'll not use final versions to not show a spec
> >> compliance we maybe don't have.
> >>
> >> Romain Manni-Bucau
> >> @rmannibucau |  Blog | Github | LinkedIn | Tomitriber
> >>
> >>
> >> 2016-03-29 21:33 GMT+02:00 John D. Ament :
> >> >
> >> >
> >> > On Tue, Mar 29, 2016 at 3:04 PM Dmitriy Setrakyan <
> dsetrak...@apache.org>
> >> > wrote:
> >> >>
> >> >> We will switch the Ignite JAR to the 1.0-alpha-1 version from
> Geronimo,
> >> >> but I am still very confused.
> >> >>
> >> >> I do not understand why we need to check any TCK compliance when
> creating
> >> >> a JAR for the JSR107 spec. The TCK compliance should be checked
> against an
> >> >> implementation, not a spec.
> >> >>
> >> >
> >> > I'm confused by this statement as well.  TCK is only applied to impl
> so not
> >> > sure why you might think that.
> >> >
> >> > What Romain was trying to convey was that the alpha-1 release
> indicates that
> >> > no implementation has checked it as TCK compliant.  One of the JSR
> >> > requirements though is to produce a valid API JAR.  If someone can do
> that,
> >> > then this can likely be promoted to a 1.0 release.
> >> >
> >> >>
> >> >> Is there any place in Apache documentation explaining this process?
> >> >>
> >> >> D.
> >> >>
> >> >> On Mon, Mar 28, 2016 at 1:57 AM, Romain Manni-Bucau
> >> >>  wrote:
> >> >>>
> >> >>> Le 28 mars 2016 10:15, "Dmitriy Setrakyan" 
> a
> >> >>> écrit :
> >> >>> >
> >> >>> > John,
> >> >>> >
> >> >>> > I am still a bit confused. I was talking about the version of the
> >> >>> > JCache
> >> >>> spec API, essentially only interfaces. The spec does not have any
> >> >>> implementation, nor implies that every project importing or
> depending on
> >> >>> the spec must be compliant with the spec.
> >> >>> >
> >> >>> > In my view implementation and TCK compliance are a different
> matter,
> >> >>> > and
> >> >>> it should be up to the project community itself to declare the
> compliance
> >> >>> with a certain spec and pass the TCK.
> >> >>> >
> >> >>> > Am I wrong?
> >> >>> >
> >> >>>
> >> >>> Yes, while not passing sigtest practise is to not release 1.0.
> >> >>>
> >> >>> > D.
> >> >>> >
> >> >>> >
> >> >>> > On Sun, Mar 27, 2016 at 9:01 AM, John D. Ament <
> johndam...@apache.org>
> >> >>> wrote:
> >> >>> >>
> >> >>> >> Dmitriy,
> >> >>> >>
> >> >>> >> I think what Romain is referring to is other TCKs.  Generally,
> >> >>> >> geronimo
> >> >>> JAR versions don't reflect the version of spec that they implement.
> >> >>> There
> >> >>> may be alpha releases that match EDRs, or alphas that are based on
> the
> >> >>> final version but with minor tweaks.
> >> >>> >>
> >> >>> >> For reference, Apache ActiveMQ Artemis relies on alpha2 of the
> JMS 2
> >> >>> spec.
> https://github.com/apache/activemq-artemis/blob/master/pom.xml#L131
> >> >>> >> It's feature complete, and Artemis passes the TCK, its just
> alpha2
> >> >>> because we haven't done a thorough enough job of making sure the API
> is
> >> >>> sane.
> >> >>> >>
> >> >>> >> John
> >> >>> >>
> >> >>> >>
> >> >>> >> On Sun, Mar 27, 2016 at 11:54 AM Dmitriy Setrakyan
> >> >>> >> 
> >> >>> wrote:
> >> >>> >>>
> >> >>> >>> Romain, I am not sure what you mean by not having access to TCK.
> Are
> >> >>> you talking about validating compatibility with JCAche using the TCK
> [1]?
> >> >>> In this case, Apache Ignite does pass the TCK. Moreover, the TCK
> seems to
> >> >>> be licensed under Apache 

[jira] [Created] (IGNITE-2910) VisorNodeDataCollectorTask should collect info about offheap and swap keys per partitions

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-2910:


 Summary: VisorNodeDataCollectorTask should collect info about 
offheap and swap keys per partitions
 Key: IGNITE-2910
 URL: https://issues.apache.org/jira/browse/IGNITE-2910
 Project: Ignite
  Issue Type: Task
  Components: visor
Affects Versions: 1.5.0.final
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 1.6


For now only keys in heap is collected, but offheap and swap also should be 
handled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Multimap implementation attempt

[Valentin Kulichenko]

Hi Konstantin,

Do you have any progress on this ticket? When do you plan to finish it?

-Val

On Wed, Mar 2, 2016 at 2:54 AM, Yakov Zhdanov  wrote:

> Hi Konstantin!
>
> I have added a comment to the ticket.
>
> --Yakov
>
> 2016-03-01 16:13 GMT+03:00 Konstantin Margorin :
>
> > Hello Igniters.
> >
> > I'm trying now to implement Multimap datastructure. I wrote a comment
> about
> > implementation details here:
> >
> > https://issues.apache.org/jira/browse/IGNITE-640
> >
> > Now I got Multimap prototype with put/get methods:
> >
> >
> >
> https://github.com/apache/ignite/compare/master...ruskim:ignite-640?expand=1
> >
> > This basic test for put/get successfully passed:
> >
> >
> >
> https://github.com/ruskim/ignite/blob/2af6274f9234dda1550696014ffd8e327d75da57/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/datastructures/GridCacheMultimapSelfTest.java
> >
> > A lot of thing I did by analogy with IgniteQueue and IgniteSet. I
> realize,
> > that most probably I made a lot of errors.
> >
> > Before moving further I want to be sure that basic things, like put/get,
> > implemented properly.
> >
> > Probably someone could look at my code and point me at most serious
> errors.
> >
> > Thank you!
> >
> > Konstantin
> >
>

Re: JCache dependency

[Romain Manni-Bucau]

Le 30 mars 2016 01:45, "Dmitriy Setrakyan"  a écrit :
>
> I just mention to mention that Apache Ignite passes JCache TCK with
flying colors :)
>

True! Totally forgot tck were open! Didn't check sigtest, is it there too?
If so nothing blocking a 1.0.

> We have it integrated into our build routine and verify it using our CI
tests. In addition, it was verified by one of the JCache spec leads, Greg
Luck, who confirmed that Ignite complies with the spec.
>
> Given the above, can Geronimo provide us with JCache 1.0 spec JAR?
>
> D.
>
> On Tue, Mar 29, 2016 at 1:41 PM, Romain Manni-Bucau 
wrote:
>>
>> ok, let me try to make it clearer (and don't hesitate to shout if still
not ;)):
>>
>> TCK are not only @Test but also some bianary validations (aka sigtest
>> or signature tests) the spec jars need to pass. It basically checks
>> you respect the spec signature for the supported java version of the
>> spec. Not having TCK and not being related to a public spec (like BVal
>> or JBatch) makes this sigtest validation missing @asf side so until we
>> get this or somebody checks generated bytecode of spec jars (and not
>> sources) then we'll not use final versions to not show a spec
>> compliance we maybe don't have.
>>
>> Romain Manni-Bucau
>> @rmannibucau |  Blog | Github | LinkedIn | Tomitriber
>>
>>
>> 2016-03-29 21:33 GMT+02:00 John D. Ament :
>> >
>> >
>> > On Tue, Mar 29, 2016 at 3:04 PM Dmitriy Setrakyan <
dsetrak...@apache.org>
>> > wrote:
>> >>
>> >> We will switch the Ignite JAR to the 1.0-alpha-1 version from
Geronimo,
>> >> but I am still very confused.
>> >>
>> >> I do not understand why we need to check any TCK compliance when
creating
>> >> a JAR for the JSR107 spec. The TCK compliance should be checked
against an
>> >> implementation, not a spec.
>> >>
>> >
>> > I'm confused by this statement as well.  TCK is only applied to impl
so not
>> > sure why you might think that.
>> >
>> > What Romain was trying to convey was that the alpha-1 release
indicates that
>> > no implementation has checked it as TCK compliant.  One of the JSR
>> > requirements though is to produce a valid API JAR.  If someone can do
that,
>> > then this can likely be promoted to a 1.0 release.
>> >
>> >>
>> >> Is there any place in Apache documentation explaining this process?
>> >>
>> >> D.
>> >>
>> >> On Mon, Mar 28, 2016 at 1:57 AM, Romain Manni-Bucau
>> >>  wrote:
>> >>>
>> >>> Le 28 mars 2016 10:15, "Dmitriy Setrakyan"  a
>> >>> écrit :
>> >>> >
>> >>> > John,
>> >>> >
>> >>> > I am still a bit confused. I was talking about the version of the
>> >>> > JCache
>> >>> spec API, essentially only interfaces. The spec does not have any
>> >>> implementation, nor implies that every project importing or
depending on
>> >>> the spec must be compliant with the spec.
>> >>> >
>> >>> > In my view implementation and TCK compliance are a different
matter,
>> >>> > and
>> >>> it should be up to the project community itself to declare the
compliance
>> >>> with a certain spec and pass the TCK.
>> >>> >
>> >>> > Am I wrong?
>> >>> >
>> >>>
>> >>> Yes, while not passing sigtest practise is to not release 1.0.
>> >>>
>> >>> > D.
>> >>> >
>> >>> >
>> >>> > On Sun, Mar 27, 2016 at 9:01 AM, John D. Ament <
johndam...@apache.org>
>> >>> wrote:
>> >>> >>
>> >>> >> Dmitriy,
>> >>> >>
>> >>> >> I think what Romain is referring to is other TCKs.  Generally,
>> >>> >> geronimo
>> >>> JAR versions don't reflect the version of spec that they implement.
>> >>> There
>> >>> may be alpha releases that match EDRs, or alphas that are based on
the
>> >>> final version but with minor tweaks.
>> >>> >>
>> >>> >> For reference, Apache ActiveMQ Artemis relies on alpha2 of the
JMS 2
>> >>> spec.
https://github.com/apache/activemq-artemis/blob/master/pom.xml#L131
>> >>> >> It's feature complete, and Artemis passes the TCK, its just alpha2
>> >>> because we haven't done a thorough enough job of making sure the API
is
>> >>> sane.
>> >>> >>
>> >>> >> John
>> >>> >>
>> >>> >>
>> >>> >> On Sun, Mar 27, 2016 at 11:54 AM Dmitriy Setrakyan
>> >>> >> 
>> >>> wrote:
>> >>> >>>
>> >>> >>> Romain, I am not sure what you mean by not having access to TCK.
Are
>> >>> you talking about validating compatibility with JCAche using the TCK
[1]?
>> >>> In this case, Apache Ignite does pass the TCK. Moreover, the TCK
seems to
>> >>> be licensed under Apache 2.0 [2]. Can you please explain?
>> >>> >>>
>> >>> >>> [1] https://github.com/jsr107/jsr107tck
>> >>> >>> [2] https://github.com/jsr107/jsr107tck/blob/master/LICENSE.txt
>> >>> >>>
>> >>> >>> On Sun, Mar 27, 2016 at 2:35 AM, Romain Manni-Bucau <
>> >>> rmannibu...@gmail.com> wrote:
>> >>> 
>> >>>  Alpha cause asf doesnt have oracle tck so we cant validate
binary
>> >>> compat
>> >>>  but it targets jcache 1.0. More a legal thing than anything
else. If
>> >>> you
>> >>>  have access to