[GitHub] ignite pull request #1866: Ignite 4799 2.0

[abeliak]

GitHub user abeliak opened a pull request:

https://github.com/apache/ignite/pull/1866

Ignite 4799 2.0



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-4799-2.0

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1866.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1866


commit 358a7b7cefca699a5fb94cf642aada5ed982feeb
Author: Alexander Belyak 
Date:   2017-04-25T04:11:57Z

ignite-4799 - removed TCP Discovery heartbeat frequency property, fixed 
compilation and logic, fixed xml configs
IGNITE-4799: altered IgniteConfiguration.metricsUpdateFrequency 
documentation.
.NET: Remove TcpDiscoverySpi.HeartbeatFrequency
ignite-4799 review
Add test for Server to Server failure detection timeout

commit 57bafd4fcadc0b2f0caad80f351521034d2d5cbe
Author: Alexander Belyak 
Date:   2017-04-25T04:12:38Z

IGNITE-4799: altered IgniteConfiguration.metricsUpdateFrequency 
documentation.

commit e975d5f83464e2a7a356c8f17d64445307f44684
Author: Alexander Belyak 
Date:   2017-04-25T04:15:36Z

.NET: Remove TcpDiscoverySpi.HeartbeatFrequency

commit 37efd98c8d40f2fff5f8cf87ea3053aa6e07334f
Author: Alexander Belyak 
Date:   2017-04-25T04:24:40Z

Remove maxMissed*Heartbeats, rename TcpDiscoveryClientHeartbeatMessage to 
TDCMetrics message, add clientFailureDetectionTimeout

commit 3ea761e1cfa414fa7486cfb36a9e170e36166ac6
Author: Alexander Belyak 
Date:   2017-04-25T04:25:15Z

Remove -1 and 0 special values from 
IgniteConfiguration.metricsUpdateFrequency

commit 723cba960c44648a32503fe019ea1549100a7b67
Author: Alexander Belyak 
Date:   2017-04-25T04:25:48Z

Use clientFailureDetectionTimeout in ServerImpl, fix comment

commit c6eac07c90151e627d290c2c3abc975b4a6bc14b
Author: Alexander Belyak 
Date:   2017-04-25T04:27:22Z

ignite-4799 review

commit 81f1a37b94d91da6e7a51dc4aacd2c87c45f1f06
Author: Alexander Belyak 
Date:   2017-04-25T04:27:41Z

Rename MetricsMessage to MetricsUpdateMessage, muFreq to metricsUpdateFreq, 
fix metricsUpdateInterval

commit ecacef60e07e34c448d28c626156870b7768deb0
Author: Alexander Belyak 
Date:   2017-04-25T04:37:03Z

Merge commit 'f6d5974fb95f7fc0bc40d0dc4df2457dda4bc697' into ignite-4799

commit e64a29faf80ddbdb26b1618422445d846ccb9df4
Author: Alexander Belyak 
Date:   2017-04-25T04:38:03Z

Use clientFailureDetectionTimeout in ServerImpl, add test

commit 998b80bc99834de14497ed09d4ca8991e75cf8b7
Author: Alexander Belyak 
Date:   2017-04-25T04:38:46Z

Add test for Server to Server failure detection timeout

commit 55c60b5c88fbb6422848e467a89e9668dda40f4e
Author: Alexander Belyak 
Date:   2017-04-25T04:39:16Z

ignite-4799 review

commit 0955bf7d913701f48051201ecd9ec2b596b77c58
Author: Alexander Belyak 
Date:   2017-04-25T04:39:48Z

Simplify tests for failure detect timeouts

commit 6453888b38e4a256b371657565f0f2da7b40316b
Author: Alexander Belyak 
Date:   2017-04-25T04:41:04Z

ignite-4799 review

commit 57bcc56bc40775b13e33a004873be21dc04be20b
Author: Alexander Belyak 
Date:   2017-04-25T04:41:51Z

ignite-4799 log messages

commit a842e6241ba13dcf37ec382e6d3f51b8fb17fae5
Author: Alexander Belyak 
Date:   2017-04-25T04:42:21Z

Formatting and always use clientFailureDetectionTimeout

commit 7aca66f296b6c8f9a33dde56e6abad7c1175cec3
Author: Alexander Belyak 
Date:   2017-04-25T04:42:59Z

Formatting

commit d3d8a788d0d4b7b5241cf8a458509195f2375b93
Author: Alexander Belyak 
Date:   2017-04-25T04:43:28Z

Fix tests

commit 8e50b310cddfd622e309eaf5767783d4db4b3709
Author: Alexander Belyak 
Date:   2017-04-25T04:44:08Z

Tear down detection timeout in test (from Long.MAX_VALUE to 
Integer.MAX_VALUE)

commit f6e3aca5ed5f440dc99f47df25fd5b426a7e8ad8
Author: Alexander Belyak 
Date:   2017-04-25T04:44:53Z

Tear down failure detection timeout

commit e66969cdde2f4dae1b538d1941a051802280fc87
Author: Alexander Belyak 
Date:   2017-04-25T04:46:15Z

ignite-4799 minor

commit dea4be64a4b8547cf6e6b9b4594d869dda269014
Author: Alexander Belyak 
Date:   2017-04-25T04:46:43Z

Fix tests

commit aa0468c04d94112a9816a31e058cf0bd2034
Author: Alexander Belyak 

[GitHub] ignite pull request #1862: Ignite 4799

[abeliak]

Github user abeliak closed the pull request at:

https://github.com/apache/ignite/pull/1862


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Null cache names

[Roman Shtykh]

Igor, +1 from me.We can add a field to ConnectorConfiguration (not sure if it's 
a proper place, but it's shared by REST, memcached and Redis). A user will have 
to create a cache, configure as needed and specify the name in 
ConnectorConfiguration.
Roman



On Monday, April 24, 2017 10:34 PM, Seliverstov Igor  
wrote:
 

 Dear Igniters,

Seems we have almost the same issue with Memcached protocol.

There is an ability to define a cache name via operation extras message
part (
https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped#packet-structure)
but it looks a bit complicated from my point of view...

Different client implementations might provide such functionality or not (I
mean an additional info in an operation extras), so, potential user might
have some difficultes using Ignite as a Memcached server because of this
ignite-specific message part becomes mandatory.

An alternative (an the best way, I think) is introducing a configuration
property to define which cache to use in case it hasn't be defined in a
message.

I'll appreciate any thoughts on that.

Regards,
Igor

2017-04-24 12:43 GMT+03:00 Roman Shtykh :

> Vladimir,
> Probably we may set the cache name via https://redis.io/commands/
> client-setname, save it and use until the user specifies another name.
> #That will be the name for the default cache (mainly for STRING data). For
> other data types, like hashes (https://redis.io/topics/data-types), I am
> thinking about putting data into caches specified by key.
> Or use https://redis.io/commands/config-set CONFIG SET DFLT_CACHE
> cache_name,and save cache name somewhere in Ignite cluster (what is the
> proper place to store such info?).
> For that, we have to implement one of the above-mentioned commands.
> What do you think?
>
>
>
>    On Monday, April 24, 2017 4:34 PM, Vladimir Ozerov <
> voze...@gridgain.com> wrote:
>
>
>  Roman,
> Is it possible to define a kind of property in Redis connection string (or
> property map) for this purpose? IMO ideally we should "externalize" cache
> name somehow, so that it can be changed with no changes to user's code.
>
> Alex,
> Not sure if this is a good idea as you will end up with PARTITIONED cache
> without backups with no ability to change that.
>
> On Mon, Apr 24, 2017 at 9:35 AM, Alexey Kuznetsov 
> wrote:
>
> > Roman,
> >
> > Just as idea, how about in case of user does not configured "REDIS_CACHE"
> >  then create it via ignite.getOrCreateCache(new
> > CacheConfiguration("REDIS_CACHE"))
> > and prin warning to log "REDIS_CACHE not configured, using default
> > partitioned cache".
> >
> > What do you think?
> >
> > On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh  >
> > wrote:
> >
> > > Denis, Igor,
> > > What can be done now
> > > 1. create a default cache name for Redis data, e.g. "REDIS_CACHE" that
> > has
> > > to be configured explicitly in xml file (as it is done with other
> caches)
> > > by a user if he/she needs Redis protocol.
> > > 2. Force users to specify cache names as prefix to their keys, so that
> we
> > > can parse and switch between caches.
> > > The 1st one is a very quick fix I can do today. This can be extended in
> > > future to have a separate cache for each data type.
> > > Roman
> > >
> > >
> > >    On Monday, April 24, 2017 12:15 AM, Denis Magda <
> dma...@gridgain.com
> > >
> > > wrote:
> > >
> > >
> > >  Roman, would you suggest a quick solution for the redis integration or
> > > even
> > > implement it in the nearest couple of days? We need to change the
> defaul
> > > cache name in 2.0. Otherwise, it can be done only in an year or so in
> > 3.0.
> > >
> > > Denis
> > >
> > > On Sunday, April 23, 2017, Seliverstov Igor 
> > wrote:
> > >
> > > > Hi Roman,
> > > >
> > > > The ticket number is IGNITE-3488.
> > > >
> > > > It's planned not to have null-named or default caches at all. All the
> > > > caches must be defined explicitly or via templates and have names.
> The
> > > > current implementation uses a cache with null name, so, we need some
> > > > configuration property to define which cache to use or mapping in
> case
> > of
> > > > using Redis databases.
> > > >
> > > > Regards,
> > > > Igor
> > > >
> > > > 22 апр. 2017 г. 8:47 пользователь "Roman Shtykh"
> > >  > > > >
> > > > написал:
> > > >
> > > > > Hi Igor,
> > > > > The current implementation supports only STRING data type of Redis.
> > In
> > > > > addition, AFAIK, scaling Redis per dataset is normally done via
> > running
> > > > > separate instances of Redis for each dataset. Therefore my choice
> was
> > > > > storing to the default cache. It looks fine from Redis'
> perspective,
> > > but
> > > > > probably not from Ignite's.For other data types like HASH or SORTED
> > > SET,
> > > > I
> > > > > planned to specify cache name by key and treat value inside as
> > Ignite's
> > > > > 

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[npordash]

Hi Denis,

>> if you want to deserialize an entry then most likely you’re doing this on
>> the app side that already has the class in the class path

In this particular case the app is a deployed service and the hosting node
doesn't have the class files on its classpath. I implemented a way to deploy
and run services on the grid even if the class files are not known by ignite
(which is current service grid limitation). It works fine except for this
deserialization issue.

-Nick



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/Re-BinaryObjectImpl-deserializeValue-with-specific-ClassLoader-tp17126p17173.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

[Denis Magda]

Hi guys,


Cache entries don’t store an identification of the class loader a key or value 
was created with. This is why binary marshaller picks the system class loader 
at deserialization time by default and you get class not found exception.

>> In general, I think ignite should try to resolve a class based on the 
>> caller's context first instead of only relying on
>> ignite's classloader or what it was configured with.

That’s probably worth looking into but for the cache entries we tried to 
simplify the things - if you want to deserialize an entry then most likely 
you’re doing this on the app side that already has the class in the class path. 
For the server nodes, that usually doesn’t have the classes it’s suggest to use 
BinaryObject and BinaryObjectBuilder wrappers on top of serialized data:
https://apacheignite.readme.io/docs/binary-marshaller#binaryobject-cache-api

Vovan, what do you think on this?

—
Denis

> On Apr 24, 2017, at 12:55 AM, Dmitriy Karachentsev 
>  wrote:
> 
> Crosspost to dev list.
> 
> Igniters,
> 
> This proposal looks quite reasonable. Do we have any restrictions that
> could prevent implementing such feature?
> I think it's nice to have in Ignite.
> 
> Thanks!
> -Dmitry.
> 
> On Sun, Apr 23, 2017 at 1:37 AM, npordash  wrote:
> 
>> Thanks!
>> 
>> That would definitely help address the hack I've implemented where I have
>> to
>> reference classes in Ignite's internal package.
>> 
>> However, I still have to work with the caches in binary which is less than
>> ideal. It's pretty common in use-cases like this to first try to use
>> Thread.currentThread().getContextClassLoader() and if that's not set then
>> fallback to something else. In general, I think ignite should try to
>> resolve
>> a class based on the caller's context first instead of only relying on
>> ignite's classloader or what it was configured with.
>> 
>> In the spirit of the webinar that Denis just had, I think this kind of
>> behavior will become mandatory for the service grid to get to the point
>> where it can do deployments of services without requiring class files to be
>> on ignite's classpath. I've heard that is something that's still
>> tentatively
>> planned and the use-case I outlined is an attempt to get around the current
>> service grid limitations. :)
>> 
>> WDYT?
>> 
>> -Nick
>> 
>> 
>> 
>> --
>> View this message in context: http://apache-ignite-users.
>> 70518.x6.nabble.com/BinaryObjectImpl-deserializeValue-with-
>> specific-ClassLoader-tp12055p12171.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>> 

[jira] [Created] (IGNITE-5071) Web Console: Support custom table names on model import

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-5071:


 Summary: Web Console: Support custom table names on model import
 Key: IGNITE-5071
 URL: https://issues.apache.org/jira/browse/IGNITE-5071
 Project: Ignite
  Issue Type: Test
  Components: wizards
Affects Versions: 1.9
Reporter: Alexey Kuznetsov
Assignee: Vasiliy Sisko
 Fix For: 2.0


We need to support table aliases aka custom table name on models import from 
RDBMS.

See QueryEntity.TableName property.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5070) Update Affinity Functions Documentation

[Denis Magda (JIRA)]

Denis Magda created IGNITE-5070:
---

 Summary: Update Affinity Functions Documentation
 Key: IGNITE-5070
 URL: https://issues.apache.org/jira/browse/IGNITE-5070
 Project: Ignite
  Issue Type: Sub-task
Reporter: Denis Magda
Assignee: Denis Magda
Priority: Critical


Review and update the existing documentation on the affinity functions:
https://apacheignite.readme.io/docs/affinity-collocation#affinity-function

At least the fair affinity function has to be removed from there and mentioned 
in the migration guide.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Optimizations on titles and h1

[Denis Magda]

Mauricio, thanks!

I’ve merged all your changes except the ones from CSS files. Why do we need to 
change the background? Please elaborate and attach a screenshot that shows what 
the changes will affect. BTW, these sort of changes have to be reviewed by 
Prachi Garg.


--- css/all.css (revision 1792160)
+++ css/all.css (working copy)
@@ -8469,7 +8469,7 @@
   margin: auto;
   width: 80px;
   height: 80px;
-  background: #d7d7d7;
+  background: #CF4F54;
 }
 .features-icon:hover {
   background: #c51017;

Index: scss/ignite.scss
===
--- scss/ignite.scss(revision 1792160)
+++ scss/ignite.scss(working copy)
@@ -949,8 +949,9 @@
 margin: auto;
 width: 80px;
 height: 80px;
-background: #d7d7d7;
+background: #CF4F54;
 
+
 &:hover {
 background: #c51017;
 }

> On Apr 20, 2017, at 8:55 PM, Mauricio Stekl  wrote:
> 
> Hi Igniters, 
> I am attaching a patch with keyword optimizations on several page titles and 
> H1 across the website.
> 
> I would appreciate if any of the committers could merge this to the website.  
> Please let me know if you have any question.
> 
> 
> Thanks in advance. 
> 
> Best, 
> Mauricio Stekl
> 
> 
> 
> 

Re: MemoryMetrics interface inconsistencies

[Alexey Goncharuk]

Agree, I overlooked this during the review. However, the changes to fix
this is pretty simple - we just move all mutator methods to MBean, like
Vladimir suggested and make MBean return the live data, while the public
API will return a serializable snapshot.

Agreed?

2017-04-24 19:33 GMT+03:00 Vladimir Ozerov :

> It seems that all mutator methods should be simply moved to MBean interface
> and change MBytes to bytes. In this case metrics interface will be
> consistent.
>
> On Mon, Apr 24, 2017 at 7:29 PM, Vladimir Ozerov 
> wrote:
>
> > Sergey,
> >
> > We need to keep API consistent. If we usually return bytes, then these
> > metrics should return bytes as well. What is more important, when I look
> at
> > API I understand that this thing is not metrics at all. Metrics in Ignte
> > terms is a set of read-only numeric properties. But this interface
> contains
> > strange things like "name", "swapPilePath". What even more strange, I do
> > not see how to get these metrics from public API.
> >
> > All in all, looks like this entity is not "metrics", but "MBean" in usual
> > Ignite terms.
> >
> > Vladimir.
> >
> > On Mon, Apr 24, 2017 at 7:20 PM, Pavel Tupitsyn 
> > wrote:
> >
> >> Sergey, I disagree.
> >>
> >> 1) As a user, I would expect MemoryMetrics instance to be
> >> read-only and serializable, so I can send it somewhere, store,
> >> put into a collection and draw a graph in UI, etc.
> >>
> >> Other metrics APIs allow this, MemoryMetrics does not.
> >>
> >> 2) Methods like enableMetrics and rateTimeInterval placed in
> MemoryMetrics
> >> break single responsibility principle and are confusing.
> >> Why do I need to Get metrics to Enable them?
> >>
> >> These methods should be placed somewhere else.
> >>
> >> I would suggest some thing like this:
> >> - introduce Memory class with getMetrics, enableMetrics,
> >> setRateTimeInterval, etc
> >> - add Ignite.getMemory method
> >>
> >>
> >> On Mon, Apr 24, 2017 at 6:53 PM, Sergey Chugunov <
> >> sergey.chugu...@gmail.com>
> >> wrote:
> >>
> >> > I guess the main reason to have IgniteCache returning snapshot metrics
> >> was
> >> > to collect metrics about distributed cache across the cluster.
> >> > At the same time MemoryMetrics were initially designed to be local on
> >> each
> >> > node, there were no requirements about collecting cluster-wide
> >> > MemoryMetrics.
> >> >
> >> > Collecting MemoryMetrics is considered as an investigation action when
> >> > something goes wrong, that's why it was decided to add enable/disable
> >> > actions to the interface.
> >> > So for me it sounds reasonable.
> >> >
> >> > The only debatable thing is having MemoryMetrics returning size in
> >> > megabytes, however I think about these metrics as designed only for
> >> user,
> >> > so I think it makes sense to return this metric in human-readable
> form.
> >> >
> >> > On Mon, Apr 24, 2017 at 6:41 PM, Vladimir Ozerov <
> voze...@gridgain.com>
> >> > wrote:
> >> >
> >> > > Agree, looks inconsistent to me.
> >> > >
> >> > > Alex G., could you chime in?
> >> > >
> >> > > On Mon, Apr 24, 2017 at 6:30 PM, Pavel Tupitsyn <
> ptupit...@apache.org
> >> >
> >> > > wrote:
> >> > >
> >> > > > Igniters,
> >> > > >
> >> > > > I have noticed quite a lot of inconsistencies with the rest of our
> >> APIs
> >> > > in
> >> > > > our new MemoryMetrics API:
> >> > > >
> >> > > > 1) MemoryMetrics is not a snapshot. It is a "live" instance which
> >> > returns
> >> > > > different values each time you access some property.
> >> > > (IgniteCache.metrics()
> >> > > > returns a snapshot).
> >> > > >
> >> > > > 2) MemoryMetrics has methods that modify state, like enableMetrics
> >> > > > and rateTimeInterval
> >> > > > (IgniteCache.metrics() returns a read-only serializable snapshot)
> >> > > >
> >> > > > 3) getSize method returns size in megabytes
> >> > > > (IgniteCache.metrics() provides sizes in bytes)
> >> > > >
> >> > > >
> >> > > > I propose to rework this API until it is not too late.
> >> > > >
> >> > > > Thoughts?
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: MemoryMetrics interface inconsistencies

[Vladimir Ozerov]

It seems that all mutator methods should be simply moved to MBean interface
and change MBytes to bytes. In this case metrics interface will be
consistent.

On Mon, Apr 24, 2017 at 7:29 PM, Vladimir Ozerov 
wrote:

> Sergey,
>
> We need to keep API consistent. If we usually return bytes, then these
> metrics should return bytes as well. What is more important, when I look at
> API I understand that this thing is not metrics at all. Metrics in Ignte
> terms is a set of read-only numeric properties. But this interface contains
> strange things like "name", "swapPilePath". What even more strange, I do
> not see how to get these metrics from public API.
>
> All in all, looks like this entity is not "metrics", but "MBean" in usual
> Ignite terms.
>
> Vladimir.
>
> On Mon, Apr 24, 2017 at 7:20 PM, Pavel Tupitsyn 
> wrote:
>
>> Sergey, I disagree.
>>
>> 1) As a user, I would expect MemoryMetrics instance to be
>> read-only and serializable, so I can send it somewhere, store,
>> put into a collection and draw a graph in UI, etc.
>>
>> Other metrics APIs allow this, MemoryMetrics does not.
>>
>> 2) Methods like enableMetrics and rateTimeInterval placed in MemoryMetrics
>> break single responsibility principle and are confusing.
>> Why do I need to Get metrics to Enable them?
>>
>> These methods should be placed somewhere else.
>>
>> I would suggest some thing like this:
>> - introduce Memory class with getMetrics, enableMetrics,
>> setRateTimeInterval, etc
>> - add Ignite.getMemory method
>>
>>
>> On Mon, Apr 24, 2017 at 6:53 PM, Sergey Chugunov <
>> sergey.chugu...@gmail.com>
>> wrote:
>>
>> > I guess the main reason to have IgniteCache returning snapshot metrics
>> was
>> > to collect metrics about distributed cache across the cluster.
>> > At the same time MemoryMetrics were initially designed to be local on
>> each
>> > node, there were no requirements about collecting cluster-wide
>> > MemoryMetrics.
>> >
>> > Collecting MemoryMetrics is considered as an investigation action when
>> > something goes wrong, that's why it was decided to add enable/disable
>> > actions to the interface.
>> > So for me it sounds reasonable.
>> >
>> > The only debatable thing is having MemoryMetrics returning size in
>> > megabytes, however I think about these metrics as designed only for
>> user,
>> > so I think it makes sense to return this metric in human-readable form.
>> >
>> > On Mon, Apr 24, 2017 at 6:41 PM, Vladimir Ozerov 
>> > wrote:
>> >
>> > > Agree, looks inconsistent to me.
>> > >
>> > > Alex G., could you chime in?
>> > >
>> > > On Mon, Apr 24, 2017 at 6:30 PM, Pavel Tupitsyn > >
>> > > wrote:
>> > >
>> > > > Igniters,
>> > > >
>> > > > I have noticed quite a lot of inconsistencies with the rest of our
>> APIs
>> > > in
>> > > > our new MemoryMetrics API:
>> > > >
>> > > > 1) MemoryMetrics is not a snapshot. It is a "live" instance which
>> > returns
>> > > > different values each time you access some property.
>> > > (IgniteCache.metrics()
>> > > > returns a snapshot).
>> > > >
>> > > > 2) MemoryMetrics has methods that modify state, like enableMetrics
>> > > > and rateTimeInterval
>> > > > (IgniteCache.metrics() returns a read-only serializable snapshot)
>> > > >
>> > > > 3) getSize method returns size in megabytes
>> > > > (IgniteCache.metrics() provides sizes in bytes)
>> > > >
>> > > >
>> > > > I propose to rework this API until it is not too late.
>> > > >
>> > > > Thoughts?
>> > > >
>> > >
>> >
>>
>
>

Re: MemoryMetrics interface inconsistencies

[Vladimir Ozerov]

Sergey,

We need to keep API consistent. If we usually return bytes, then these
metrics should return bytes as well. What is more important, when I look at
API I understand that this thing is not metrics at all. Metrics in Ignte
terms is a set of read-only numeric properties. But this interface contains
strange things like "name", "swapPilePath". What even more strange, I do
not see how to get these metrics from public API.

All in all, looks like this entity is not "metrics", but "MBean" in usual
Ignite terms.

Vladimir.

On Mon, Apr 24, 2017 at 7:20 PM, Pavel Tupitsyn 
wrote:

> Sergey, I disagree.
>
> 1) As a user, I would expect MemoryMetrics instance to be
> read-only and serializable, so I can send it somewhere, store,
> put into a collection and draw a graph in UI, etc.
>
> Other metrics APIs allow this, MemoryMetrics does not.
>
> 2) Methods like enableMetrics and rateTimeInterval placed in MemoryMetrics
> break single responsibility principle and are confusing.
> Why do I need to Get metrics to Enable them?
>
> These methods should be placed somewhere else.
>
> I would suggest some thing like this:
> - introduce Memory class with getMetrics, enableMetrics,
> setRateTimeInterval, etc
> - add Ignite.getMemory method
>
>
> On Mon, Apr 24, 2017 at 6:53 PM, Sergey Chugunov <
> sergey.chugu...@gmail.com>
> wrote:
>
> > I guess the main reason to have IgniteCache returning snapshot metrics
> was
> > to collect metrics about distributed cache across the cluster.
> > At the same time MemoryMetrics were initially designed to be local on
> each
> > node, there were no requirements about collecting cluster-wide
> > MemoryMetrics.
> >
> > Collecting MemoryMetrics is considered as an investigation action when
> > something goes wrong, that's why it was decided to add enable/disable
> > actions to the interface.
> > So for me it sounds reasonable.
> >
> > The only debatable thing is having MemoryMetrics returning size in
> > megabytes, however I think about these metrics as designed only for user,
> > so I think it makes sense to return this metric in human-readable form.
> >
> > On Mon, Apr 24, 2017 at 6:41 PM, Vladimir Ozerov 
> > wrote:
> >
> > > Agree, looks inconsistent to me.
> > >
> > > Alex G., could you chime in?
> > >
> > > On Mon, Apr 24, 2017 at 6:30 PM, Pavel Tupitsyn 
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > I have noticed quite a lot of inconsistencies with the rest of our
> APIs
> > > in
> > > > our new MemoryMetrics API:
> > > >
> > > > 1) MemoryMetrics is not a snapshot. It is a "live" instance which
> > returns
> > > > different values each time you access some property.
> > > (IgniteCache.metrics()
> > > > returns a snapshot).
> > > >
> > > > 2) MemoryMetrics has methods that modify state, like enableMetrics
> > > > and rateTimeInterval
> > > > (IgniteCache.metrics() returns a read-only serializable snapshot)
> > > >
> > > > 3) getSize method returns size in megabytes
> > > > (IgniteCache.metrics() provides sizes in bytes)
> > > >
> > > >
> > > > I propose to rework this API until it is not too late.
> > > >
> > > > Thoughts?
> > > >
> > >
> >
>

[GitHub] ignite pull request #1679: IGNITE-4575: Added enum support to SQL queries

[skalashnikov]

Github user skalashnikov closed the pull request at:

https://github.com/apache/ignite/pull/1679


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: MemoryMetrics interface inconsistencies

[Pavel Tupitsyn]

Sergey, I disagree.

1) As a user, I would expect MemoryMetrics instance to be
read-only and serializable, so I can send it somewhere, store,
put into a collection and draw a graph in UI, etc.

Other metrics APIs allow this, MemoryMetrics does not.

2) Methods like enableMetrics and rateTimeInterval placed in MemoryMetrics
break single responsibility principle and are confusing.
Why do I need to Get metrics to Enable them?

These methods should be placed somewhere else.

I would suggest some thing like this:
- introduce Memory class with getMetrics, enableMetrics,
setRateTimeInterval, etc
- add Ignite.getMemory method


On Mon, Apr 24, 2017 at 6:53 PM, Sergey Chugunov 
wrote:

> I guess the main reason to have IgniteCache returning snapshot metrics was
> to collect metrics about distributed cache across the cluster.
> At the same time MemoryMetrics were initially designed to be local on each
> node, there were no requirements about collecting cluster-wide
> MemoryMetrics.
>
> Collecting MemoryMetrics is considered as an investigation action when
> something goes wrong, that's why it was decided to add enable/disable
> actions to the interface.
> So for me it sounds reasonable.
>
> The only debatable thing is having MemoryMetrics returning size in
> megabytes, however I think about these metrics as designed only for user,
> so I think it makes sense to return this metric in human-readable form.
>
> On Mon, Apr 24, 2017 at 6:41 PM, Vladimir Ozerov 
> wrote:
>
> > Agree, looks inconsistent to me.
> >
> > Alex G., could you chime in?
> >
> > On Mon, Apr 24, 2017 at 6:30 PM, Pavel Tupitsyn 
> > wrote:
> >
> > > Igniters,
> > >
> > > I have noticed quite a lot of inconsistencies with the rest of our APIs
> > in
> > > our new MemoryMetrics API:
> > >
> > > 1) MemoryMetrics is not a snapshot. It is a "live" instance which
> returns
> > > different values each time you access some property.
> > (IgniteCache.metrics()
> > > returns a snapshot).
> > >
> > > 2) MemoryMetrics has methods that modify state, like enableMetrics
> > > and rateTimeInterval
> > > (IgniteCache.metrics() returns a read-only serializable snapshot)
> > >
> > > 3) getSize method returns size in megabytes
> > > (IgniteCache.metrics() provides sizes in bytes)
> > >
> > >
> > > I propose to rework this API until it is not too late.
> > >
> > > Thoughts?
> > >
> >
>

[GitHub] ignite pull request #1865: IGNITE-3487: hidden _key and _val columns

[skalashnikov]

GitHub user skalashnikov opened a pull request:

https://github.com/apache/ignite/pull/1865

IGNITE-3487: hidden _key and _val columns



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-3487

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1865.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1865






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1864: ignite-5068

[ilantukh]

GitHub user ilantukh opened a pull request:

https://github.com/apache/ignite/pull/1864

ignite-5068



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5068

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1864.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1864






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: MemoryMetrics interface inconsistencies

[Sergey Chugunov]

I guess the main reason to have IgniteCache returning snapshot metrics was
to collect metrics about distributed cache across the cluster.
At the same time MemoryMetrics were initially designed to be local on each
node, there were no requirements about collecting cluster-wide
MemoryMetrics.

Collecting MemoryMetrics is considered as an investigation action when
something goes wrong, that's why it was decided to add enable/disable
actions to the interface.
So for me it sounds reasonable.

The only debatable thing is having MemoryMetrics returning size in
megabytes, however I think about these metrics as designed only for user,
so I think it makes sense to return this metric in human-readable form.

On Mon, Apr 24, 2017 at 6:41 PM, Vladimir Ozerov 
wrote:

> Agree, looks inconsistent to me.
>
> Alex G., could you chime in?
>
> On Mon, Apr 24, 2017 at 6:30 PM, Pavel Tupitsyn 
> wrote:
>
> > Igniters,
> >
> > I have noticed quite a lot of inconsistencies with the rest of our APIs
> in
> > our new MemoryMetrics API:
> >
> > 1) MemoryMetrics is not a snapshot. It is a "live" instance which returns
> > different values each time you access some property.
> (IgniteCache.metrics()
> > returns a snapshot).
> >
> > 2) MemoryMetrics has methods that modify state, like enableMetrics
> > and rateTimeInterval
> > (IgniteCache.metrics() returns a read-only serializable snapshot)
> >
> > 3) getSize method returns size in megabytes
> > (IgniteCache.metrics() provides sizes in bytes)
> >
> >
> > I propose to rework this API until it is not too late.
> >
> > Thoughts?
> >
>

[GitHub] ignite pull request #1727: IGNITE-3487: _key and _val fields should be exclu...

[skalashnikov]

Github user skalashnikov closed the pull request at:

https://github.com/apache/ignite/pull/1727


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Null cache names

[Roman Shtykh]

Igor,

What are the advantages of such mapping compared to the proposed approach?

1. From Redis perspective, we can have only 16 databases.
2. We will still have to define the cache in xml or programmatically, but make 
things more complicated (a user will have to specify database n in Redis 
configuration, but a real cache name in Ignite settings).

Roman


On Mon, 4/24/17, Seliverstov Igor  wrote:

 Subject: Re: Null cache names
 To: dev@ignite.apache.org, "Roman Shtykh" 
 Received: Monday, April 24, 2017, 10:49 PM
 
 Roman,
 Why
 can't we use just some kind of mapping to bind Redis
 database ID to cache name and define it via xml or
 programmatically?
 In
 this case 0 database will be mapped to a default cache, so
 that we won't break abstraction in Redis
 terms.
 Regards,Igor
 2017-04-24 12:43 GMT+03:00
 Roman Shtykh :
 Vladimir,
 
 Probably we may set the cache name via https://redis.io/commands/
 client-setname, save it and use until the user specifies
 another name.
 
 #That will be the name for the default cache (mainly for
 STRING data). For other data types, like hashes (https://redis.io/topics/data-
 types), I am thinking about putting data into caches
 specified by key.
 
 Or use https://redis.io/commands/
 config-set CONFIG SET DFLT_CACHE cache_name,and save
 cache name somewhere in Ignite cluster (what is the proper
 place to store such info?).
 
 For that, we have to implement one of the above-mentioned
 commands. 
 
 What do you think?
 
 
 
 
 
 
 
     On Monday, April 24, 2017 4:34 PM, Vladimir Ozerov
 
 wrote:
 
 
 
 
 
  Roman,
 
 Is it possible to define a kind of property in Redis
 connection string (or
 
 property map) for this purpose? IMO ideally we should
 "externalize" cache
 
 name somehow, so that it can be changed with no changes to
 user's code.
 
 
 
 Alex,
 
 Not sure if this is a good idea as you will end up with
 PARTITIONED cache
 
 without backups with no ability to change that.
 
 
 
 On Mon, Apr 24, 2017 at 9:35 AM, Alexey Kuznetsov 
 
 wrote:
 
 
 
 > Roman,
 
 >
 
 > Just as idea, how about in case of user does not
 configured "REDIS_CACHE"
 
 >  then create it via ignite.getOrCreateCache(new
 
 > CacheConfiguration("REDIS_ CACHE"))
 
 > and prin warning to log "REDIS_CACHE not
 configured, using default
 
 > partitioned cache".
 
 >
 
 > What do you think?
 
 >
 
 > On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh
 
 
 > wrote:
 
 >
 
 > > Denis, Igor,
 
 > > What can be done now
 
 > > 1. create a default cache name for Redis data,
 e.g. "REDIS_CACHE" that
 
 > has
 
 > > to be configured explicitly in xml file (as it is
 done with other caches)
 
 > > by a user if he/she needs Redis protocol.
 
 > > 2. Force users to specify cache names as prefix to
 their keys, so that we
 
 > > can parse and switch between caches.
 
 > > The 1st one is a very quick fix I can do today.
 This can be extended in
 
 > > future to have a separate cache for each data
 type.
 
 > > Roman
 
 > >
 
 > >
 
 > >    On Monday, April 24, 2017 12:15 AM, Denis
 Magda  >
 
 > > wrote:
 
 > >
 
 > >
 
 > >  Roman, would you suggest a quick solution for
 the redis integration or
 
 > > even
 
 > > implement it in the nearest couple of days? We
 need to change the defaul
 
 > > cache name in 2.0. Otherwise, it can be done only
 in an year or so in
 
 > 3.0.
 
 > >
 
 > > Denis
 
 > >
 
 > > On Sunday, April 23, 2017, Seliverstov Igor 
 
 > wrote:
 
 > >
 
 > > > Hi Roman,
 
 > > >
 
 > > > The ticket number is IGNITE-3488.
 
 > > >
 
 > > > It's planned not to have null-named or
 default caches at all. All the
 
 > > > caches must be defined explicitly or via
 templates and have names. The
 
 > > > current implementation uses a cache with null
 name, so, we need some
 
 > > > configuration property to define which cache
 to use or mapping in case
 
 > of
 
 > > > using Redis databases.
 
 > > >
 
 > > > Regards,
 
 > > > Igor
 
 > > >
 
 > > > 22 апр. 2017 г. 8:47
 пользователь "Roman Shtykh"
 
 > >  > > >
 
 > > > написал:
 
 > > >
 
 > > > > Hi Igor,
 
 > > > > The current implementation supports only
 STRING data type of Redis.
 
 > In
 
 > > > > addition, AFAIK, scaling Redis per
 dataset is normally done via
 
 > running
 
 > > > > separate instances of Redis for each
 dataset. Therefore my choice was
 
 > > > > storing to the default cache. It looks
 fine from Redis' perspective,
 
 > > but
 
 > > > > probably not from Ignite's.For other
 data types like HASH or SORTED
 
 > > SET,
 
 > > > I
 
 > > > > planned to specify cache name by key and
 treat value inside as
 
 > Ignite's
 
 > > > > key-values.# Redis has a notion of
 'database' and you can switch
 
 > > between
 
 > > > > them, but they can be referred only by
 the number, 

Re: MemoryMetrics interface inconsistencies

[Vladimir Ozerov]

Agree, looks inconsistent to me.

Alex G., could you chime in?

On Mon, Apr 24, 2017 at 6:30 PM, Pavel Tupitsyn 
wrote:

> Igniters,
>
> I have noticed quite a lot of inconsistencies with the rest of our APIs in
> our new MemoryMetrics API:
>
> 1) MemoryMetrics is not a snapshot. It is a "live" instance which returns
> different values each time you access some property. (IgniteCache.metrics()
> returns a snapshot).
>
> 2) MemoryMetrics has methods that modify state, like enableMetrics
> and rateTimeInterval
> (IgniteCache.metrics() returns a read-only serializable snapshot)
>
> 3) getSize method returns size in megabytes
> (IgniteCache.metrics() provides sizes in bytes)
>
>
> I propose to rework this API until it is not too late.
>
> Thoughts?
>

[IGNITE-5036] review the changes at the ingite-cassandra-store module

[Taras Ledkov]

Igor,

I see that you are the main contributor of the ignite-cassandra-* modules.

Could you please review the changes at the ingite-cassandra-store module 
for the issue IGNITE-5036.


Please take a look at the issue: 
https://issues.apache.org/jira/browse/IGNITE-5036


--

Taras Ledkov
Mail-To: tled...@gridgain.com

MemoryMetrics interface inconsistencies

[Pavel Tupitsyn]

Igniters,

I have noticed quite a lot of inconsistencies with the rest of our APIs in
our new MemoryMetrics API:

1) MemoryMetrics is not a snapshot. It is a "live" instance which returns
different values each time you access some property. (IgniteCache.metrics()
returns a snapshot).

2) MemoryMetrics has methods that modify state, like enableMetrics
and rateTimeInterval
(IgniteCache.metrics() returns a read-only serializable snapshot)

3) getSize method returns size in megabytes
(IgniteCache.metrics() provides sizes in bytes)


I propose to rework this API until it is not too late.

Thoughts?

[jira] [Created] (IGNITE-5069) QueryWords example fails with exception

[Yakov Zhdanov (JIRA)]

Yakov Zhdanov created IGNITE-5069:
-

 Summary: QueryWords example fails with exception
 Key: IGNITE-5069
 URL: https://issues.apache.org/jira/browse/IGNITE-5069
 Project: Ignite
  Issue Type: Bug
Reporter: Yakov Zhdanov
Priority: Blocker
 Fix For: 2.0


[~sergi.vladykin], please have a look. It may happen so that avg(cnt) in former 
Ignite versions was returned as double for long column. For current master it 
is returned as long and causes exception in formatter. I was able to reproduce 
it on 1.9 also, but did not try other versions.

Steps to reproduce
# start {{ExampleNodeStartup}}
# start {{QueryWords}}
# start {{StreamWords}}

{noformat}
/opt/jdk/jdk1.8.0_121/bin/java...
org.apache.ignite.examples.streaming.wordcount.QueryWords
[18:20:17]__   
[18:20:17]   /  _/ ___/ |/ /  _/_  __/ __/ 
[18:20:17]  _/ // (7 7// /  / / / _/   
[18:20:17] /___/\___/_/|_/___/ /_/ /___/  
[18:20:17] 
[18:20:17] ver. 2.1.0-SNAPSHOT#19700101-sha1:DEV
[18:20:17] 2017 Copyright(C) Apache Software Foundation
[18:20:17] 
[18:20:17] Ignite documentation: http://ignite.apache.org
[18:20:17] 
[18:20:17] Quiet mode.
[18:20:17]   ^-- Logging to file 
'/home/yzhdanov/projects/incubator-ignite/work/log/ignite-dbfdab2d.log'
[18:20:17]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or 
"-v" to ignite.{sh|bat}
[18:20:17] 
[18:20:17] OS: Linux 4.8.0-46-generic amd64
[18:20:17] VM information: Java(TM) SE Runtime Environment 1.8.0_121-b13 Oracle 
Corporation Java HotSpot(TM) 64-Bit Server VM 25.121-b13
[18:20:17] Initial heap size is 246MB (should be no less than 512MB, use 
-Xms512m -Xmx512m).
[18:20:17] Configured plugins:
[18:20:17]   ^-- None
[18:20:17] 
[18:20:17] Message queue limit is set to 0 which may lead to potential OOMEs 
when running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to 
message queues growth on sender and receiver sides.
[18:20:17] Security status [authentication=off, tls/ssl=off]
[18:20:18] REST protocols do not start on client node. To start the protocols 
on client node set '-DIGNITE_REST_START_ON_CLIENT=true' system property.
[18:20:20] Performance suggestions for grid  (fix if possible)
[18:20:20] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
[18:20:20]   ^-- Disable grid events (remove 'includeEventTypes' from 
configuration)
[18:20:20]   ^-- Enable G1 Garbage Collector (add '-XX:+UseG1GC' to JVM options)
[18:20:20]   ^-- Specify JVM heap max size (add '-Xmx[g|G|m|M|k|K]' to 
JVM options)
[18:20:20]   ^-- Set max direct memory size if getting 'OOME: Direct buffer 
memory' (add '-XX:MaxDirectMemorySize=[g|G|m|M|k|K]' to JVM options)
[18:20:20]   ^-- Disable processing of calls to System.gc() (add 
'-XX:+DisableExplicitGC' to JVM options)
[18:20:20] Refer to this page for more performance suggestions: 
https://apacheignite.readme.io/docs/jvm-and-system-tuning
[18:20:20] 
[18:20:20] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
[18:20:20] 
[18:20:20] Ignite node started OK (id=dbfdab2d)
[18:20:20] Topology snapshot [ver=2, servers=1, clients=1, CPUs=4, heap=6.8GB]
Query result set is empty.
Query result set is empty.
[18:20:27] Topology snapshot [ver=3, servers=1, clients=2, CPUs=4, heap=10.0GB]
Query results [avg=[18:20:33] Ignite node stopped OK [uptime=00:00:12:733]
Exception in thread "main" java.util.IllegalFormatConversionException: f != 
java.lang.Long
at 
java.util.Formatter$FormatSpecifier.failConversion(Formatter.java:4302)
at java.util.Formatter$FormatSpecifier.printFloat(Formatter.java:2806)
at java.util.Formatter$FormatSpecifier.print(Formatter.java:2753)
at java.util.Formatter.format(Formatter.java:2520)
at java.io.PrintStream.format(PrintStream.java:970)
at java.io.PrintStream.printf(PrintStream.java:871)
at 
org.apache.ignite.examples.streaming.wordcount.QueryWords.main(QueryWords.java:78)

Process finished with exit code 1

{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1847: IGNITE-1439: Added Futures for C++

[isapego]

Github user isapego closed the pull request at:

https://github.com/apache/ignite/pull/1847


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] ignite pull request #1863: IGNITE-5036 Disallow @QuerySqlField and @QueryTex...

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/1863

IGNITE-5036  Disallow @QuerySqlField and @QueryTextField on methods



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-5036

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/1863.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1863


commit 2d505817cef677c523dc27a521a709d983cc26e5
Author: devozerov 
Date:   2017-04-20T08:31:14Z

WIP.

commit 668be3703cedad20c3440f3b6c050e909ad85e38
Author: tledkov-gridgain 
Date:   2017-04-20T09:50:57Z

Merge branch 'ignite-2.0' into ignite-5036

commit 214c665beec3e50ec78ed1f6aa1f5a8a7f07bd57
Author: tledkov-gridgain 
Date:   2017-04-20T10:32:51Z

IGNITE-5036: save the progress

commit b2a420f04075822682e1101d7776041e62dbf88b
Author: tledkov-gridgain 
Date:   2017-04-20T10:41:33Z

IGNITE-5036: save the progress

commit c92e3090be6aa9a2fff881d71b32df185fcae82d
Author: tledkov-gridgain 
Date:   2017-04-20T15:23:54Z

Merge branch 'ignite-2.0' into ignite-5036

commit 564e59f71482ef0791c4a6d18b50b12cb463d516
Author: tledkov-gridgain 
Date:   2017-04-21T08:34:55Z

IGNITE-5036: save the progress

commit 2861a904d6c8429627cb49e34c247de380fe7b9b
Author: tledkov-gridgain 
Date:   2017-04-21T13:41:10Z

Merge branch 'ignite-2.0' into ignite-5036

commit cc54788c586782110ecf6ed2b673142acef7af40
Author: tledkov-gridgain 
Date:   2017-04-24T09:15:37Z

Merge branch 'ignite-2.0' into ignite-5036

commit 53e07627f1e72dbe3e7594e805a56d9fa3522e42
Author: tledkov-gridgain 
Date:   2017-04-24T12:28:39Z

IGNITE-5036: cassandra: save the progress

commit e27054f9cc430ce8d3541129efffe5b7e0ea523f
Author: tledkov-gridgain 
Date:   2017-04-24T13:28:10Z

IGNITE-5036: cassandra: save the progress

commit 732b6c8850a84a12536ff2b38964010e1c3d20ea
Author: tledkov-gridgain 
Date:   2017-04-24T14:16:11Z

IGNITE-5036: cassandra: save the progress

commit 504890f3182085b304166a88c4c965e1d5a12cbb
Author: tledkov-gridgain 
Date:   2017-04-24T14:37:41Z

IGNITE-5036: the progress




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: SQL usability: catalogs, schemas and tables

[Sergi Vladykin]

Yes, we need to move on making Ignite work as any usual SQL db.

But please avoid mixing all this stuff together, lets have a separate task
(and discussion if needed) for each item in your list.

Sergi

2017-04-24 16:58 GMT+03:00 Vladimir Ozerov :

> Igniters,
>
> [long read, take a cup of coffee]
>
> Historically every SQL in Ignite must be executed against particular cache:
> SqlQuery requires cache, JDBC and ODBC drivers require cache name. This
> works fine until we add CREATE TABLE. Consider an empty cluster - how do
> you connect to it if you have no caches yet? Nohow.
>
> It looks like we cannot have convenient access to cluster unless we
> properly define and implement *schema* abstraction. ANSI SQL'92 defines
> several abstractions: cluster -> catalog -> schema -> table/view/etc..
> Every "catalog" has *INFORMATION_SCHEMA* schema, containing database
> metadata. Almost all vendors support it (notable exclusion - Oracle). Looks
> like we need to introduce similar concept and finally decouple caches from
> schemas.
>
> High-level proposal from my side
>
> 1) Let's consider Ignite cluster as a single database ("catalog" in ANSI
> SQL'92 terms).
>
> 2) It should be possible to connect to the cluster without a single user
> cache. In this case schema is not defined.
>
> 3) We must have a kind of storage for metadata. It could be either another
> system cache, or something analogous to binary metadata cache, which is
> essentially node-local data structure exchanged on node join. It should be
> aligned well with persistence feature, which is expected in AI 2.1.
>
> 4) Content of this storage will be accessible through INFORMATION_SCHEMA
> abstraction.
>
> 5) We must support "CREATE SCHEMA", "DROP SCHEMA" commands which will
> effectively create records in system cache and invoke relevant commands on
> local H2 engines of every node (now it happens implicitly on cache
> start/stop).
>
> 6) From JDBC/ODBC driver perspective schema will be defined either in
> connection string, or in runtime through "SET SCHEMA" command which is
> already supported by H2.
>
> 7) We must finally introduce new native SQL API, which will not use caches
> directly. Something like "IgniteSql sql()". See *IGNITE-4701*.
>
> Once schema is there, usage of "CREATE TABLE" and "DROP TABLE" commands
> will be simple and convenient, and it will fit naturally in user's past
> experience with conventional RDBMS.
>
> Thoughts?
>
> P.S.: CREATE/DROP TABLE feature is not blocked by this problem. It will
> work, but will be inconvenient for users.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-4701
>

Re: Improve binary enum handling

[Sergi Vladykin]

I agree with Dmitriy, it is preferable to have this enum registration
optional. It will be a better user experience.

Why do we "inevitably" need it?

Sergi

2017-04-24 17:02 GMT+03:00 Vladimir Ozerov :

> Dima,
>
> No. It is normal (and inevitably) practice to register enums before they
> are used.
>
> This is how enum is created in MySQL:
>
> CREATE TABLE shirts (
> name VARCHAR(40),
> size ENUM('x-small', 'small', 'medium', 'large', 'x-large')
> );
>
> And in PostgreSQL:
>
> CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
> CREATE TABLE person (
> name text,
> current_mood mood
> );
>
> We will do the same at some point. That is, in future users will register
> enums from SQL, not from native API or configuration.
>
> Vladimir.
>
> On Mon, Apr 24, 2017 at 4:37 PM, Dmitriy Setrakyan 
> wrote:
>
> > Vladimir,
> >
> > I would really like to avoid special registration of Enums. Can you find
> a
> > way to handle it automatically?
> >
> > D.
> >
> > On Mon, Apr 24, 2017 at 6:33 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Sorry, looks like I mismanaged tickets in JIRA. In fact, we implemented
> > H2
> > > part, but Ignite's part is not ready yet and is managed in IGNITE-4575
> > [1].
> > > Ticket you mentioned was an umbrella.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-4575
> > >
> > > On Mon, Apr 24, 2017 at 4:28 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > wrote:
> > >
> > > > Vladimir,
> > > >
> > > > I am very confused. I thought we already had resolved this issue in
> > this
> > > > ticket:
> > > > https://issues.apache.org/jira/browse/IGNITE-3595
> > > >
> > > > Can you clarify?
> > > >
> > > > D.
> > > >
> > > > On Mon, Apr 24, 2017 at 5:58 AM, Vladimir Ozerov <
> voze...@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > Currently we have limited support of binary enums. The main problem
> > is
> > > > that
> > > > > we do not store any metadata about enum names. For this reason it
> is
> > > > > impossible to use enums in SQL even though H2 already supports it
> > [1].
> > > We
> > > > > need to improve enum metadata support and provide some additional
> API
> > > to
> > > > > register new enums in runtime.
> > > > >
> > > > > Proposed API:
> > > > >
> > > > > 1) Enum mappings can be defined statically in
> > BinaryTypeConfiguration:
> > > > >
> > > > > class BinaryTypeConfiguration {
> > > > > boolean isEnum;  // Old method
> > > > > *Map enumValues;* // New method
> > > > > }
> > > > >
> > > > > 2) New enum could be registered through IgniteBinary (e.g. we will
> > use
> > > it
> > > > > if enum is defined in CREATE TABLE statement). Elso it would be
> > > possible
> > > > to
> > > > > build enum using only name.
> > > > >
> > > > > interface IgniteBinary {
> > > > > BinaryObject buildEnum(String typeName, int ordinal);
> > > //
> > > > > Old
> > > > > *BinaryObject buildEnum(String typeName, String name); *
> > > >  //
> > > > > New
> > > > >
> > > > > *BinaryType defineEnum(String typeName, Map
> > > vals);*
> > > > //
> > > > > New
> > > > > }
> > > > >
> > > > > 3) BinaryObject will have new method "enumName":
> > > > >
> > > > > interface BinaryObject {
> > > > > enumOrdinal(); // Old
> > > > > *String enumName();* // New
> > > > > }
> > > > >
> > > > > 4) It would be possible to get the list of known values from
> > > BinaryType:
> > > > >
> > > > > interface BinaryType {
> > > > > boolean isEnum();  // Old
> > > > > *Collection enumValues();* // New
> > > > > }
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > Vladimir.
> > > > >
> > > > > [1] https://github.com/h2database/h2database/pull/487/commits
> > > > >
> > > >
> > >
> >
>

Re: Improve binary enum handling

[Vladimir Ozerov]

Dima,

No. It is normal (and inevitably) practice to register enums before they
are used.

This is how enum is created in MySQL:

CREATE TABLE shirts (
name VARCHAR(40),
size ENUM('x-small', 'small', 'medium', 'large', 'x-large')
);

And in PostgreSQL:

CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
CREATE TABLE person (
name text,
current_mood mood
);

We will do the same at some point. That is, in future users will register
enums from SQL, not from native API or configuration.

Vladimir.

On Mon, Apr 24, 2017 at 4:37 PM, Dmitriy Setrakyan 
wrote:

> Vladimir,
>
> I would really like to avoid special registration of Enums. Can you find a
> way to handle it automatically?
>
> D.
>
> On Mon, Apr 24, 2017 at 6:33 AM, Vladimir Ozerov 
> wrote:
>
> > Sorry, looks like I mismanaged tickets in JIRA. In fact, we implemented
> H2
> > part, but Ignite's part is not ready yet and is managed in IGNITE-4575
> [1].
> > Ticket you mentioned was an umbrella.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-4575
> >
> > On Mon, Apr 24, 2017 at 4:28 PM, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > wrote:
> >
> > > Vladimir,
> > >
> > > I am very confused. I thought we already had resolved this issue in
> this
> > > ticket:
> > > https://issues.apache.org/jira/browse/IGNITE-3595
> > >
> > > Can you clarify?
> > >
> > > D.
> > >
> > > On Mon, Apr 24, 2017 at 5:58 AM, Vladimir Ozerov  >
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > Currently we have limited support of binary enums. The main problem
> is
> > > that
> > > > we do not store any metadata about enum names. For this reason it is
> > > > impossible to use enums in SQL even though H2 already supports it
> [1].
> > We
> > > > need to improve enum metadata support and provide some additional API
> > to
> > > > register new enums in runtime.
> > > >
> > > > Proposed API:
> > > >
> > > > 1) Enum mappings can be defined statically in
> BinaryTypeConfiguration:
> > > >
> > > > class BinaryTypeConfiguration {
> > > > boolean isEnum;  // Old method
> > > > *Map enumValues;* // New method
> > > > }
> > > >
> > > > 2) New enum could be registered through IgniteBinary (e.g. we will
> use
> > it
> > > > if enum is defined in CREATE TABLE statement). Elso it would be
> > possible
> > > to
> > > > build enum using only name.
> > > >
> > > > interface IgniteBinary {
> > > > BinaryObject buildEnum(String typeName, int ordinal);
> > //
> > > > Old
> > > > *BinaryObject buildEnum(String typeName, String name); *
> > >  //
> > > > New
> > > >
> > > > *BinaryType defineEnum(String typeName, Map
> > vals);*
> > > //
> > > > New
> > > > }
> > > >
> > > > 3) BinaryObject will have new method "enumName":
> > > >
> > > > interface BinaryObject {
> > > > enumOrdinal(); // Old
> > > > *String enumName();* // New
> > > > }
> > > >
> > > > 4) It would be possible to get the list of known values from
> > BinaryType:
> > > >
> > > > interface BinaryType {
> > > > boolean isEnum();  // Old
> > > > *Collection enumValues();* // New
> > > > }
> > > >
> > > > Thoughts?
> > > >
> > > > Vladimir.
> > > >
> > > > [1] https://github.com/h2database/h2database/pull/487/commits
> > > >
> > >
> >
>

[jira] [Created] (IGNITE-5068) Redesign usage of GridDhtPartitionTopologyImpl.part2node map to store only diff from affinity assignment

[Ilya Lantukh (JIRA)]

Ilya Lantukh created IGNITE-5068:


 Summary: Redesign usage of GridDhtPartitionTopologyImpl.part2node 
map to store only diff from affinity assignment
 Key: IGNITE-5068
 URL: https://issues.apache.org/jira/browse/IGNITE-5068
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 2.0
Reporter: Ilya Lantukh
Assignee: Ilya Lantukh


This map can become very huge on large topologies, and rebuilding it on each 
update is also costly. Some beneficial changes were made in the scope of 
IGNITE-4626, but further improvement requires complete redesign.
This map always stores affinity nodes + some additional "temporary owners". 
Those owners are only needed to complete rebalancing and they will evict 
partition when rebalancing is finished. It seems that storing only those 
non-affinity owners can greatly reduce memory required by this map (it will be 
empty on stable topology) and effort needed to keep it consistent with 
node2part.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

SQL usability: catalogs, schemas and tables

[Vladimir Ozerov]

Igniters,

[long read, take a cup of coffee]

Historically every SQL in Ignite must be executed against particular cache:
SqlQuery requires cache, JDBC and ODBC drivers require cache name. This
works fine until we add CREATE TABLE. Consider an empty cluster - how do
you connect to it if you have no caches yet? Nohow.

It looks like we cannot have convenient access to cluster unless we
properly define and implement *schema* abstraction. ANSI SQL'92 defines
several abstractions: cluster -> catalog -> schema -> table/view/etc..
Every "catalog" has *INFORMATION_SCHEMA* schema, containing database
metadata. Almost all vendors support it (notable exclusion - Oracle). Looks
like we need to introduce similar concept and finally decouple caches from
schemas.

High-level proposal from my side

1) Let's consider Ignite cluster as a single database ("catalog" in ANSI
SQL'92 terms).

2) It should be possible to connect to the cluster without a single user
cache. In this case schema is not defined.

3) We must have a kind of storage for metadata. It could be either another
system cache, or something analogous to binary metadata cache, which is
essentially node-local data structure exchanged on node join. It should be
aligned well with persistence feature, which is expected in AI 2.1.

4) Content of this storage will be accessible through INFORMATION_SCHEMA
abstraction.

5) We must support "CREATE SCHEMA", "DROP SCHEMA" commands which will
effectively create records in system cache and invoke relevant commands on
local H2 engines of every node (now it happens implicitly on cache
start/stop).

6) From JDBC/ODBC driver perspective schema will be defined either in
connection string, or in runtime through "SET SCHEMA" command which is
already supported by H2.

7) We must finally introduce new native SQL API, which will not use caches
directly. Something like "IgniteSql sql()". See *IGNITE-4701*.

Once schema is there, usage of "CREATE TABLE" and "DROP TABLE" commands
will be simple and convenient, and it will fit naturally in user's past
experience with conventional RDBMS.

Thoughts?

P.S.: CREATE/DROP TABLE feature is not blocked by this problem. It will
work, but will be inconvenient for users.

[1] https://issues.apache.org/jira/browse/IGNITE-4701

[jira] [Created] (IGNITE-5067) Absolute swapFilePath for MemoryPolicy is merged incorrectly with working dir path

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-5067:
---

 Summary: Absolute swapFilePath for MemoryPolicy is merged 
incorrectly with working dir path
 Key: IGNITE-5067
 URL: https://issues.apache.org/jira/browse/IGNITE-5067
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.0
Reporter: Sergey Chugunov
Assignee: Sergey Chugunov
 Fix For: 2.1


h2. Steps to reproduce
* Create *MemoryPolicyConfiguration* with swapFilePath specified to some 
*absolute* path
* Start Ignite node with this configuration.

h2. Expected outcome
Swap file is allocated by absolute path specified by configuration.

h2. Actual outcome
Swap file is allocated by path where current working directory is merged with 
swapFilePath.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Null cache names

[Seliverstov Igor]

Roman,

Why can't we use just some kind of mapping to bind Redis database ID to
cache name and define it via xml or programmatically?

In this case 0 database will be mapped to a default cache, so that we won't
break abstraction in Redis terms.

Regards,
Igor

2017-04-24 12:43 GMT+03:00 Roman Shtykh :

> Vladimir,
> Probably we may set the cache name via https://redis.io/commands/
> client-setname, save it and use until the user specifies another name.
> #That will be the name for the default cache (mainly for STRING data). For
> other data types, like hashes (https://redis.io/topics/data-types), I am
> thinking about putting data into caches specified by key.
> Or use https://redis.io/commands/config-set CONFIG SET DFLT_CACHE
> cache_name,and save cache name somewhere in Ignite cluster (what is the
> proper place to store such info?).
> For that, we have to implement one of the above-mentioned commands.
> What do you think?
>
>
>
> On Monday, April 24, 2017 4:34 PM, Vladimir Ozerov <
> voze...@gridgain.com> wrote:
>
>
>  Roman,
> Is it possible to define a kind of property in Redis connection string (or
> property map) for this purpose? IMO ideally we should "externalize" cache
> name somehow, so that it can be changed with no changes to user's code.
>
> Alex,
> Not sure if this is a good idea as you will end up with PARTITIONED cache
> without backups with no ability to change that.
>
> On Mon, Apr 24, 2017 at 9:35 AM, Alexey Kuznetsov 
> wrote:
>
> > Roman,
> >
> > Just as idea, how about in case of user does not configured "REDIS_CACHE"
> >  then create it via ignite.getOrCreateCache(new
> > CacheConfiguration("REDIS_CACHE"))
> > and prin warning to log "REDIS_CACHE not configured, using default
> > partitioned cache".
> >
> > What do you think?
> >
> > On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh  >
> > wrote:
> >
> > > Denis, Igor,
> > > What can be done now
> > > 1. create a default cache name for Redis data, e.g. "REDIS_CACHE" that
> > has
> > > to be configured explicitly in xml file (as it is done with other
> caches)
> > > by a user if he/she needs Redis protocol.
> > > 2. Force users to specify cache names as prefix to their keys, so that
> we
> > > can parse and switch between caches.
> > > The 1st one is a very quick fix I can do today. This can be extended in
> > > future to have a separate cache for each data type.
> > > Roman
> > >
> > >
> > >On Monday, April 24, 2017 12:15 AM, Denis Magda <
> dma...@gridgain.com
> > >
> > > wrote:
> > >
> > >
> > >  Roman, would you suggest a quick solution for the redis integration or
> > > even
> > > implement it in the nearest couple of days? We need to change the
> defaul
> > > cache name in 2.0. Otherwise, it can be done only in an year or so in
> > 3.0.
> > >
> > > Denis
> > >
> > > On Sunday, April 23, 2017, Seliverstov Igor 
> > wrote:
> > >
> > > > Hi Roman,
> > > >
> > > > The ticket number is IGNITE-3488.
> > > >
> > > > It's planned not to have null-named or default caches at all. All the
> > > > caches must be defined explicitly or via templates and have names.
> The
> > > > current implementation uses a cache with null name, so, we need some
> > > > configuration property to define which cache to use or mapping in
> case
> > of
> > > > using Redis databases.
> > > >
> > > > Regards,
> > > > Igor
> > > >
> > > > 22 апр. 2017 г. 8:47 пользователь "Roman Shtykh"
> > >  > > > >
> > > > написал:
> > > >
> > > > > Hi Igor,
> > > > > The current implementation supports only STRING data type of Redis.
> > In
> > > > > addition, AFAIK, scaling Redis per dataset is normally done via
> > running
> > > > > separate instances of Redis for each dataset. Therefore my choice
> was
> > > > > storing to the default cache. It looks fine from Redis'
> perspective,
> > > but
> > > > > probably not from Ignite's.For other data types like HASH or SORTED
> > > SET,
> > > > I
> > > > > planned to specify cache name by key and treat value inside as
> > Ignite's
> > > > > key-values.# Redis has a notion of 'database' and you can switch
> > > between
> > > > > them, but they can be referred only by the number, and limited to
> 16
> > > > > databases... Not very useful, IMO.
> > > > > If you still have the default cache, the current Redis integration
> > > should
> > > > > work as is (I have to recheck it, what is the JIRA ticket for the
> > null
> > > > > cache issue?)
> > > > > Do you just want to be sure your changes don't affect the Redis
> > > > > integration, or do you want to extend it to switch between caches?
> > > > > Roman
> > > > >
> > > > >On Friday, April 21, 2017 8:17 PM, Seliverstov Igor <
> > > > > gvvinbl...@gmail.com > wrote:
> > > > >
> > > > >
> > > > >  Hi Roman,
> > > > >
> > > > > As far as I inderstand you're the author of the Redis protocol
> > > > > implementation.
> > > > >
> > > > > 

Re: Null cache names

[Dmitriy Setrakyan]

Can we just use the name "default" in such cases?

On Mon, Apr 24, 2017 at 6:33 AM, Seliverstov Igor 
wrote:

> Dear Igniters,
>
> Seems we have almost the same issue with Memcached protocol.
>
> There is an ability to define a cache name via operation extras message
> part (
> https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped#packet-
> structure)
> but it looks a bit complicated from my point of view...
>
> Different client implementations might provide such functionality or not (I
> mean an additional info in an operation extras), so, potential user might
> have some difficultes using Ignite as a Memcached server because of this
> ignite-specific message part becomes mandatory.
>
> An alternative (an the best way, I think) is introducing a configuration
> property to define which cache to use in case it hasn't be defined in a
> message.
>
> I'll appreciate any thoughts on that.
>
> Regards,
> Igor
>
> 2017-04-24 12:43 GMT+03:00 Roman Shtykh :
>
> > Vladimir,
> > Probably we may set the cache name via https://redis.io/commands/
> > client-setname, save it and use until the user specifies another name.
> > #That will be the name for the default cache (mainly for STRING data).
> For
> > other data types, like hashes (https://redis.io/topics/data-types), I am
> > thinking about putting data into caches specified by key.
> > Or use https://redis.io/commands/config-set CONFIG SET DFLT_CACHE
> > cache_name,and save cache name somewhere in Ignite cluster (what is the
> > proper place to store such info?).
> > For that, we have to implement one of the above-mentioned commands.
> > What do you think?
> >
> >
> >
> > On Monday, April 24, 2017 4:34 PM, Vladimir Ozerov <
> > voze...@gridgain.com> wrote:
> >
> >
> >  Roman,
> > Is it possible to define a kind of property in Redis connection string
> (or
> > property map) for this purpose? IMO ideally we should "externalize" cache
> > name somehow, so that it can be changed with no changes to user's code.
> >
> > Alex,
> > Not sure if this is a good idea as you will end up with PARTITIONED cache
> > without backups with no ability to change that.
> >
> > On Mon, Apr 24, 2017 at 9:35 AM, Alexey Kuznetsov  >
> > wrote:
> >
> > > Roman,
> > >
> > > Just as idea, how about in case of user does not configured
> "REDIS_CACHE"
> > >  then create it via ignite.getOrCreateCache(new
> > > CacheConfiguration("REDIS_CACHE"))
> > > and prin warning to log "REDIS_CACHE not configured, using default
> > > partitioned cache".
> > >
> > > What do you think?
> > >
> > > On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh
>  > >
> > > wrote:
> > >
> > > > Denis, Igor,
> > > > What can be done now
> > > > 1. create a default cache name for Redis data, e.g. "REDIS_CACHE"
> that
> > > has
> > > > to be configured explicitly in xml file (as it is done with other
> > caches)
> > > > by a user if he/she needs Redis protocol.
> > > > 2. Force users to specify cache names as prefix to their keys, so
> that
> > we
> > > > can parse and switch between caches.
> > > > The 1st one is a very quick fix I can do today. This can be extended
> in
> > > > future to have a separate cache for each data type.
> > > > Roman
> > > >
> > > >
> > > >On Monday, April 24, 2017 12:15 AM, Denis Magda <
> > dma...@gridgain.com
> > > >
> > > > wrote:
> > > >
> > > >
> > > >  Roman, would you suggest a quick solution for the redis integration
> or
> > > > even
> > > > implement it in the nearest couple of days? We need to change the
> > defaul
> > > > cache name in 2.0. Otherwise, it can be done only in an year or so in
> > > 3.0.
> > > >
> > > > Denis
> > > >
> > > > On Sunday, April 23, 2017, Seliverstov Igor 
> > > wrote:
> > > >
> > > > > Hi Roman,
> > > > >
> > > > > The ticket number is IGNITE-3488.
> > > > >
> > > > > It's planned not to have null-named or default caches at all. All
> the
> > > > > caches must be defined explicitly or via templates and have names.
> > The
> > > > > current implementation uses a cache with null name, so, we need
> some
> > > > > configuration property to define which cache to use or mapping in
> > case
> > > of
> > > > > using Redis databases.
> > > > >
> > > > > Regards,
> > > > > Igor
> > > > >
> > > > > 22 апр. 2017 г. 8:47 пользователь "Roman Shtykh"
> > > >  > > > > >
> > > > > написал:
> > > > >
> > > > > > Hi Igor,
> > > > > > The current implementation supports only STRING data type of
> Redis.
> > > In
> > > > > > addition, AFAIK, scaling Redis per dataset is normally done via
> > > running
> > > > > > separate instances of Redis for each dataset. Therefore my choice
> > was
> > > > > > storing to the default cache. It looks fine from Redis'
> > perspective,
> > > > but
> > > > > > probably not from Ignite's.For other data types like HASH or
> SORTED
> > > > SET,
> > > > > I
> > > > > > planned to 

Re: Improve binary enum handling

[Dmitriy Setrakyan]

Vladimir,

I would really like to avoid special registration of Enums. Can you find a
way to handle it automatically?

D.

On Mon, Apr 24, 2017 at 6:33 AM, Vladimir Ozerov 
wrote:

> Sorry, looks like I mismanaged tickets in JIRA. In fact, we implemented H2
> part, but Ignite's part is not ready yet and is managed in IGNITE-4575 [1].
> Ticket you mentioned was an umbrella.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-4575
>
> On Mon, Apr 24, 2017 at 4:28 PM, Dmitriy Setrakyan 
> wrote:
>
> > Vladimir,
> >
> > I am very confused. I thought we already had resolved this issue in this
> > ticket:
> > https://issues.apache.org/jira/browse/IGNITE-3595
> >
> > Can you clarify?
> >
> > D.
> >
> > On Mon, Apr 24, 2017 at 5:58 AM, Vladimir Ozerov 
> > wrote:
> >
> > > Igniters,
> > >
> > > Currently we have limited support of binary enums. The main problem is
> > that
> > > we do not store any metadata about enum names. For this reason it is
> > > impossible to use enums in SQL even though H2 already supports it [1].
> We
> > > need to improve enum metadata support and provide some additional API
> to
> > > register new enums in runtime.
> > >
> > > Proposed API:
> > >
> > > 1) Enum mappings can be defined statically in BinaryTypeConfiguration:
> > >
> > > class BinaryTypeConfiguration {
> > > boolean isEnum;  // Old method
> > > *Map enumValues;* // New method
> > > }
> > >
> > > 2) New enum could be registered through IgniteBinary (e.g. we will use
> it
> > > if enum is defined in CREATE TABLE statement). Elso it would be
> possible
> > to
> > > build enum using only name.
> > >
> > > interface IgniteBinary {
> > > BinaryObject buildEnum(String typeName, int ordinal);
> //
> > > Old
> > > *BinaryObject buildEnum(String typeName, String name); *
> >  //
> > > New
> > >
> > > *BinaryType defineEnum(String typeName, Map
> vals);*
> > //
> > > New
> > > }
> > >
> > > 3) BinaryObject will have new method "enumName":
> > >
> > > interface BinaryObject {
> > > enumOrdinal(); // Old
> > > *String enumName();* // New
> > > }
> > >
> > > 4) It would be possible to get the list of known values from
> BinaryType:
> > >
> > > interface BinaryType {
> > > boolean isEnum();  // Old
> > > *Collection enumValues();* // New
> > > }
> > >
> > > Thoughts?
> > >
> > > Vladimir.
> > >
> > > [1] https://github.com/h2database/h2database/pull/487/commits
> > >
> >
>

Re: Ignite ML, next steps (IGNITE-5029)

[ArtemM]

Sure. It would be great!



--
View this message in context: 
http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-ML-next-steps-IGNITE-5029-tp17096p17135.html
Sent from the Apache Ignite Developers mailing list archive at Nabble.com.

Re: Null cache names

[Seliverstov Igor]

Dear Igniters,

Seems we have almost the same issue with Memcached protocol.

There is an ability to define a cache name via operation extras message
part (
https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped#packet-structure)
but it looks a bit complicated from my point of view...

Different client implementations might provide such functionality or not (I
mean an additional info in an operation extras), so, potential user might
have some difficultes using Ignite as a Memcached server because of this
ignite-specific message part becomes mandatory.

An alternative (an the best way, I think) is introducing a configuration
property to define which cache to use in case it hasn't be defined in a
message.

I'll appreciate any thoughts on that.

Regards,
Igor

2017-04-24 12:43 GMT+03:00 Roman Shtykh :

> Vladimir,
> Probably we may set the cache name via https://redis.io/commands/
> client-setname, save it and use until the user specifies another name.
> #That will be the name for the default cache (mainly for STRING data). For
> other data types, like hashes (https://redis.io/topics/data-types), I am
> thinking about putting data into caches specified by key.
> Or use https://redis.io/commands/config-set CONFIG SET DFLT_CACHE
> cache_name,and save cache name somewhere in Ignite cluster (what is the
> proper place to store such info?).
> For that, we have to implement one of the above-mentioned commands.
> What do you think?
>
>
>
> On Monday, April 24, 2017 4:34 PM, Vladimir Ozerov <
> voze...@gridgain.com> wrote:
>
>
>  Roman,
> Is it possible to define a kind of property in Redis connection string (or
> property map) for this purpose? IMO ideally we should "externalize" cache
> name somehow, so that it can be changed with no changes to user's code.
>
> Alex,
> Not sure if this is a good idea as you will end up with PARTITIONED cache
> without backups with no ability to change that.
>
> On Mon, Apr 24, 2017 at 9:35 AM, Alexey Kuznetsov 
> wrote:
>
> > Roman,
> >
> > Just as idea, how about in case of user does not configured "REDIS_CACHE"
> >  then create it via ignite.getOrCreateCache(new
> > CacheConfiguration("REDIS_CACHE"))
> > and prin warning to log "REDIS_CACHE not configured, using default
> > partitioned cache".
> >
> > What do you think?
> >
> > On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh  >
> > wrote:
> >
> > > Denis, Igor,
> > > What can be done now
> > > 1. create a default cache name for Redis data, e.g. "REDIS_CACHE" that
> > has
> > > to be configured explicitly in xml file (as it is done with other
> caches)
> > > by a user if he/she needs Redis protocol.
> > > 2. Force users to specify cache names as prefix to their keys, so that
> we
> > > can parse and switch between caches.
> > > The 1st one is a very quick fix I can do today. This can be extended in
> > > future to have a separate cache for each data type.
> > > Roman
> > >
> > >
> > >On Monday, April 24, 2017 12:15 AM, Denis Magda <
> dma...@gridgain.com
> > >
> > > wrote:
> > >
> > >
> > >  Roman, would you suggest a quick solution for the redis integration or
> > > even
> > > implement it in the nearest couple of days? We need to change the
> defaul
> > > cache name in 2.0. Otherwise, it can be done only in an year or so in
> > 3.0.
> > >
> > > Denis
> > >
> > > On Sunday, April 23, 2017, Seliverstov Igor 
> > wrote:
> > >
> > > > Hi Roman,
> > > >
> > > > The ticket number is IGNITE-3488.
> > > >
> > > > It's planned not to have null-named or default caches at all. All the
> > > > caches must be defined explicitly or via templates and have names.
> The
> > > > current implementation uses a cache with null name, so, we need some
> > > > configuration property to define which cache to use or mapping in
> case
> > of
> > > > using Redis databases.
> > > >
> > > > Regards,
> > > > Igor
> > > >
> > > > 22 апр. 2017 г. 8:47 пользователь "Roman Shtykh"
> > >  > > > >
> > > > написал:
> > > >
> > > > > Hi Igor,
> > > > > The current implementation supports only STRING data type of Redis.
> > In
> > > > > addition, AFAIK, scaling Redis per dataset is normally done via
> > running
> > > > > separate instances of Redis for each dataset. Therefore my choice
> was
> > > > > storing to the default cache. It looks fine from Redis'
> perspective,
> > > but
> > > > > probably not from Ignite's.For other data types like HASH or SORTED
> > > SET,
> > > > I
> > > > > planned to specify cache name by key and treat value inside as
> > Ignite's
> > > > > key-values.# Redis has a notion of 'database' and you can switch
> > > between
> > > > > them, but they can be referred only by the number, and limited to
> 16
> > > > > databases... Not very useful, IMO.
> > > > > If you still have the default cache, the current Redis integration
> > > should
> > > > > work as is (I have to recheck it, what is the JIRA 

Re: Improve binary enum handling

[Vladimir Ozerov]

Sorry, looks like I mismanaged tickets in JIRA. In fact, we implemented H2
part, but Ignite's part is not ready yet and is managed in IGNITE-4575 [1].
Ticket you mentioned was an umbrella.

[1] https://issues.apache.org/jira/browse/IGNITE-4575

On Mon, Apr 24, 2017 at 4:28 PM, Dmitriy Setrakyan 
wrote:

> Vladimir,
>
> I am very confused. I thought we already had resolved this issue in this
> ticket:
> https://issues.apache.org/jira/browse/IGNITE-3595
>
> Can you clarify?
>
> D.
>
> On Mon, Apr 24, 2017 at 5:58 AM, Vladimir Ozerov 
> wrote:
>
> > Igniters,
> >
> > Currently we have limited support of binary enums. The main problem is
> that
> > we do not store any metadata about enum names. For this reason it is
> > impossible to use enums in SQL even though H2 already supports it [1]. We
> > need to improve enum metadata support and provide some additional API to
> > register new enums in runtime.
> >
> > Proposed API:
> >
> > 1) Enum mappings can be defined statically in BinaryTypeConfiguration:
> >
> > class BinaryTypeConfiguration {
> > boolean isEnum;  // Old method
> > *Map enumValues;* // New method
> > }
> >
> > 2) New enum could be registered through IgniteBinary (e.g. we will use it
> > if enum is defined in CREATE TABLE statement). Elso it would be possible
> to
> > build enum using only name.
> >
> > interface IgniteBinary {
> > BinaryObject buildEnum(String typeName, int ordinal);  //
> > Old
> > *BinaryObject buildEnum(String typeName, String name); *
>  //
> > New
> >
> > *BinaryType defineEnum(String typeName, Map vals);*
> //
> > New
> > }
> >
> > 3) BinaryObject will have new method "enumName":
> >
> > interface BinaryObject {
> > enumOrdinal(); // Old
> > *String enumName();* // New
> > }
> >
> > 4) It would be possible to get the list of known values from BinaryType:
> >
> > interface BinaryType {
> > boolean isEnum();  // Old
> > *Collection enumValues();* // New
> > }
> >
> > Thoughts?
> >
> > Vladimir.
> >
> > [1] https://github.com/h2database/h2database/pull/487/commits
> >
>

Re: Improve binary enum handling

[Dmitriy Setrakyan]

Vladimir,

I am very confused. I thought we already had resolved this issue in this
ticket:
https://issues.apache.org/jira/browse/IGNITE-3595

Can you clarify?

D.

On Mon, Apr 24, 2017 at 5:58 AM, Vladimir Ozerov 
wrote:

> Igniters,
>
> Currently we have limited support of binary enums. The main problem is that
> we do not store any metadata about enum names. For this reason it is
> impossible to use enums in SQL even though H2 already supports it [1]. We
> need to improve enum metadata support and provide some additional API to
> register new enums in runtime.
>
> Proposed API:
>
> 1) Enum mappings can be defined statically in BinaryTypeConfiguration:
>
> class BinaryTypeConfiguration {
> boolean isEnum;  // Old method
> *Map enumValues;* // New method
> }
>
> 2) New enum could be registered through IgniteBinary (e.g. we will use it
> if enum is defined in CREATE TABLE statement). Elso it would be possible to
> build enum using only name.
>
> interface IgniteBinary {
> BinaryObject buildEnum(String typeName, int ordinal);  //
> Old
> *BinaryObject buildEnum(String typeName, String name); * //
> New
>
> *BinaryType defineEnum(String typeName, Map vals);* //
> New
> }
>
> 3) BinaryObject will have new method "enumName":
>
> interface BinaryObject {
> enumOrdinal(); // Old
> *String enumName();* // New
> }
>
> 4) It would be possible to get the list of known values from BinaryType:
>
> interface BinaryType {
> boolean isEnum();  // Old
> *Collection enumValues();* // New
> }
>
> Thoughts?
>
> Vladimir.
>
> [1] https://github.com/h2database/h2database/pull/487/commits
>

Re: User can't assign ticket to himself.

[Vladimir Ozerov]

Done.

On Mon, Apr 24, 2017 at 3:16 PM, Andrey Mashenkov <
andrey.mashen...@gmail.com> wrote:

> User want to contribute to project, but seems he has no rights in JIRA to
> track a ticket.
> See comments to ticket IGNITE-5046 [1].
>
> Please, would someone handle this?
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-5046
>
> --
> Best regards,
> Andrey V. Mashenkov
>

Improve binary enum handling

[Vladimir Ozerov]

Igniters,

Currently we have limited support of binary enums. The main problem is that
we do not store any metadata about enum names. For this reason it is
impossible to use enums in SQL even though H2 already supports it [1]. We
need to improve enum metadata support and provide some additional API to
register new enums in runtime.

Proposed API:

1) Enum mappings can be defined statically in BinaryTypeConfiguration:

class BinaryTypeConfiguration {
boolean isEnum;  // Old method
*Map enumValues;* // New method
}

2) New enum could be registered through IgniteBinary (e.g. we will use it
if enum is defined in CREATE TABLE statement). Elso it would be possible to
build enum using only name.

interface IgniteBinary {
BinaryObject buildEnum(String typeName, int ordinal);  //
Old
*BinaryObject buildEnum(String typeName, String name); * //
New

*BinaryType defineEnum(String typeName, Map vals);* //
New
}

3) BinaryObject will have new method "enumName":

interface BinaryObject {
enumOrdinal(); // Old
*String enumName();* // New
}

4) It would be possible to get the list of known values from BinaryType:

interface BinaryType {
boolean isEnum();  // Old
*Collection enumValues();* // New
}

Thoughts?

Vladimir.

[1] https://github.com/h2database/h2database/pull/487/commits

User can't assign ticket to himself.

[Andrey Mashenkov]

User want to contribute to project, but seems he has no rights in JIRA to
track a ticket.
See comments to ticket IGNITE-5046 [1].

Please, would someone handle this?


[1] https://issues.apache.org/jira/browse/IGNITE-5046

-- 
Best regards,
Andrey V. Mashenkov

[jira] [Created] (IGNITE-5066) .NET: Continuous query fails with exception on Java side

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-5066:
--

 Summary: .NET: Continuous query fails with exception on Java side
 Key: IGNITE-5066
 URL: https://issues.apache.org/jira/browse/IGNITE-5066
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 1.9
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.0


Reported by the user, simple program being run twice:
{code}
using (IIgnite ignite = Ignition.Start(config))
{
var cache = ignite.GetOrCreateCache(nameof(Data));
cache.QueryContinuous(new ContinuousQuery(new 
Listener()));

// Pressing any key in the console will add a value to the cache
while (true)
{
Console.ReadKey();

var entry = new Data() { Id = Guid.NewGuid(), Value = "a 
value" };
cache.Put(entry.Id, entry);
}
}
{code}

Causes exceptions on Java side:
{code}
Exception in thread "sys-#44%null%" 
javax.cache.event.CacheEntryListenerException: Failed resolve class for ID: 
3076010
at 
org.apache.ignite.internal.processors.platform.utils.PlatformUtils.toCacheEntryListenerException(PlatformUtils.java:593)
at 
org.apache.ignite.internal.processors.platform.utils.PlatformUtils.applyContinuousQueryEvents(PlatformUtils.java:551)
at 
org.apache.ignite.internal.processors.platform.cache.query.PlatformContinuousQueryImpl.onUpdated(PlatformContinuousQueryImpl.java:200)
at 
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.notifyCallback0(CacheContinuousQueryHandler.java:705)
at 
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandler.notifyCallback(CacheContinuousQueryHandler.java:650)
at 
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.processNotification(GridContinuousProcessor.java:1089)
at 
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor.access$2000(GridContinuousProcessor.java:97)
at 
org.apache.ignite.internal.processors.continuous.GridContinuousProcessor$8.onMessage(GridContinuousProcessor.java:741)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1222)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$2000(GridIoManager.java:108)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2443)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1182)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$2300(GridIoManager.java:108)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$8.run(GridIoManager.java:1151)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: class org.apache.ignite.binary.BinaryObjectException: Failed resolve 
class for ID: 3076010
at 
org.apache.ignite.internal.binary.BinaryContext.descriptorForTypeId(BinaryContext.java:699)
at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1491)
at 
org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1450)
at 
org.apache.ignite.internal.binary.BinaryObjectImpl.deserializeValue(BinaryObjectImpl.java:637)
at 
org.apache.ignite.internal.binary.BinaryObjectImpl.value(BinaryObjectImpl.java:142)
at 
org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinary(CacheObjectContext.java:272)
at 
org.apache.ignite.internal.processors.cache.CacheObjectContext.unwrapBinaryIfNeeded(CacheObjectContext.java:160)
at 
org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryEvent.getValue(CacheContinuousQueryEvent.java:73)
at 
org.apache.ignite.internal.processors.platform.utils.PlatformUtils.writeCacheEntryEvent(PlatformUtils.java:606)
at 
org.apache.ignite.internal.processors.platform.utils.PlatformUtils.applyContinuousQueryEvents(PlatformUtils.java:539)
... 15 more
Caused by: class org.apache.ignite.IgniteCheckedException: Class definition was 
not found at marshaller cache and local file. [id=3076010, 
file=C:\Users\USER\AppData\Local\Temp\ignite\work\marshaller\3076010.classname]
at 
org.apache.ignite.internal.MarshallerContextImpl.className(MarshallerContextImpl.java:218)
at 

[jira] [Created] (IGNITE-5065) DSL/scription support

[Yury Babak (JIRA)]

Yury Babak created IGNITE-5065:
--

 Summary: DSL/scription support
 Key: IGNITE-5065
 URL: https://issues.apache.org/jira/browse/IGNITE-5065
 Project: Ignite
  Issue Type: New Feature
  Components: ml
Reporter: Yury Babak
Assignee: Yury Babak


The goal is introduce JS(using Nashorn) support as scripting language. Also we 
should make investigation about using Scala as DSL.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Windows XP support

[Pavel Tupitsyn]

> Pasha, what’s about .NET? Do we still support these ancient Windows
versions or the doc has to be altered as well?

Ignite.NET depends on Ignite C++, so we have to drop XP support there as
well.

On Mon, Apr 24, 2017 at 12:23 PM, Igor Sapego  wrote:

> Done. I'm going to refactor code appropriately then.
>
> Best Regards,
> Igor
>
> On Sat, Apr 22, 2017 at 2:37 AM, Denis Magda  wrote:
>
> > +1 for the discontinuation. Igor, please update C++ readme doc once this
> > feature is implemented. I think it’s still fine to claim that Windows XP,
> > Vista are supported for Ignite Java lib.
> >
> > Pasha, what’s about .NET? Do we still support these ancient Windows
> > versions or the doc has to be altered as well?
> >
> > —
> > Denis
> >
> > > On Apr 21, 2017, at 12:59 PM, Dmitriy Setrakyan  >
> > wrote:
> > >
> > > Agree, let's drop XP support.
> > >
> > > D.
> > >
> > > On Fri, Apr 21, 2017 at 8:03 AM, Pavel Tupitsyn 
> > > wrote:
> > >
> > >> I don't think we should bother with supporting 16 year old
> discontinued
> > OS.
> > >> Vista support has also ended (just 10 days ago).
> > >>
> > >> We should put Windows 7 there as the oldest supported version.
> > >>
> > >> On Fri, Apr 21, 2017 at 5:59 PM, Igor Sapego 
> > wrote:
> > >>
> > >>> Hello Igniters,
> > >>>
> > >>> So, I've been working on implementation of futures support for C++
> [1]
> > >> and
> > >>> It would be convenient for me to use condition variables on Windows
> [2]
> > >> as
> > >>> well as on Linux. The problem here is that they are not supported on
> > >>> Windows XP.
> > >>>
> > >>> So I've checked documentation [3] and found out that we claim Windows
> > XP
> > >>> support. Does anyone really needs it? AFAIK Windows XP is not
> supported
> > >> by
> > >>> MS any more.
> > >>>
> > >>> Would not it be more convenient to drop its support if no one needs
> it
> > so
> > >>> we can use more modern tools?
> > >>>
> > >>> [1] - https://issues.apache.org/jira/browse/IGNITE-1439
> > >>> [2] -
> > >>> https://msdn.microsoft.com/en-us/library/windows/desktop/
> > >>> ms682052(v=vs.85).aspx
> > >>> [3] -
> > >>> https://apacheignite.readme.io/docs/getting-started#
> > >> section-prerequisites
> > >>>
> > >>> Best Regards,
> > >>> Igor
> > >>>
> > >>
> >
> >
>

Re: Null cache names

[Roman Shtykh]

Vladimir,
Probably we may set the cache name via 
https://redis.io/commands/client-setname, save it and use until the user 
specifies another name.
#That will be the name for the default cache (mainly for STRING data). For 
other data types, like hashes (https://redis.io/topics/data-types), I am 
thinking about putting data into caches specified by key.
Or use https://redis.io/commands/config-set CONFIG SET DFLT_CACHE 
cache_name,and save cache name somewhere in Ignite cluster (what is the proper 
place to store such info?).
For that, we have to implement one of the above-mentioned commands. 
What do you think?



On Monday, April 24, 2017 4:34 PM, Vladimir Ozerov  
wrote:
 

 Roman,
Is it possible to define a kind of property in Redis connection string (or
property map) for this purpose? IMO ideally we should "externalize" cache
name somehow, so that it can be changed with no changes to user's code.

Alex,
Not sure if this is a good idea as you will end up with PARTITIONED cache
without backups with no ability to change that.

On Mon, Apr 24, 2017 at 9:35 AM, Alexey Kuznetsov 
wrote:

> Roman,
>
> Just as idea, how about in case of user does not configured "REDIS_CACHE"
>  then create it via ignite.getOrCreateCache(new
> CacheConfiguration("REDIS_CACHE"))
> and prin warning to log "REDIS_CACHE not configured, using default
> partitioned cache".
>
> What do you think?
>
> On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh 
> wrote:
>
> > Denis, Igor,
> > What can be done now
> > 1. create a default cache name for Redis data, e.g. "REDIS_CACHE" that
> has
> > to be configured explicitly in xml file (as it is done with other caches)
> > by a user if he/she needs Redis protocol.
> > 2. Force users to specify cache names as prefix to their keys, so that we
> > can parse and switch between caches.
> > The 1st one is a very quick fix I can do today. This can be extended in
> > future to have a separate cache for each data type.
> > Roman
> >
> >
> >    On Monday, April 24, 2017 12:15 AM, Denis Magda  >
> > wrote:
> >
> >
> >  Roman, would you suggest a quick solution for the redis integration or
> > even
> > implement it in the nearest couple of days? We need to change the defaul
> > cache name in 2.0. Otherwise, it can be done only in an year or so in
> 3.0.
> >
> > Denis
> >
> > On Sunday, April 23, 2017, Seliverstov Igor 
> wrote:
> >
> > > Hi Roman,
> > >
> > > The ticket number is IGNITE-3488.
> > >
> > > It's planned not to have null-named or default caches at all. All the
> > > caches must be defined explicitly or via templates and have names. The
> > > current implementation uses a cache with null name, so, we need some
> > > configuration property to define which cache to use or mapping in case
> of
> > > using Redis databases.
> > >
> > > Regards,
> > > Igor
> > >
> > > 22 апр. 2017 г. 8:47 пользователь "Roman Shtykh"
> >  > > >
> > > написал:
> > >
> > > > Hi Igor,
> > > > The current implementation supports only STRING data type of Redis.
> In
> > > > addition, AFAIK, scaling Redis per dataset is normally done via
> running
> > > > separate instances of Redis for each dataset. Therefore my choice was
> > > > storing to the default cache. It looks fine from Redis' perspective,
> > but
> > > > probably not from Ignite's.For other data types like HASH or SORTED
> > SET,
> > > I
> > > > planned to specify cache name by key and treat value inside as
> Ignite's
> > > > key-values.# Redis has a notion of 'database' and you can switch
> > between
> > > > them, but they can be referred only by the number, and limited to 16
> > > > databases... Not very useful, IMO.
> > > > If you still have the default cache, the current Redis integration
> > should
> > > > work as is (I have to recheck it, what is the JIRA ticket for the
> null
> > > > cache issue?)
> > > > Do you just want to be sure your changes don't affect the Redis
> > > > integration, or do you want to extend it to switch between caches?
> > > > Roman
> > > >
> > > >    On Friday, April 21, 2017 8:17 PM, Seliverstov Igor <
> > > > gvvinbl...@gmail.com > wrote:
> > > >
> > > >
> > > >  Hi Roman,
> > > >
> > > > As far as I inderstand you're the author of the Redis protocol
> > > > implementation.
> > > >
> > > > Currently I'm working on a task to prohibit null names for caches and
> > > I've
> > > > found that current implementation implies using of default (aka null
> > > named)
> > > > cache as a Redis database.
> > > >
> > > > So, I need your assistance to implement Redis databases and mappings
> > them
> > > > to particular caches.
> > > >
> > > > Is it in your plans to do it in near time?
> > > >
> > > > If not I'll appriciate your thoughts on how to do it properly)
> > > >
> > > > Regards,
> > > > Igor
> > > >
> > > >
> > > >
> > >
> >
> >
> >
>
>
>
> --
> Alexey Kuznetsov
>

   

[jira] [Created] (IGNITE-5064) Obsolete EventTypes to be removed

[Sergey Chugunov (JIRA)]

Sergey Chugunov created IGNITE-5064:
---

 Summary: Obsolete EventTypes to be removed
 Key: IGNITE-5064
 URL: https://issues.apache.org/jira/browse/IGNITE-5064
 Project: Ignite
  Issue Type: Task
  Components: general
Reporter: Sergey Chugunov
Assignee: Alexey Kuznetsov


The following list of EventTypes were removed as part of 
[IGNITE-4952|https://issues.apache.org/jira/browse/IGNITE-4952]:

* *EventType#EVT_CACHE_OBJECT_SWAPPED*
* *EventType#EVT_CACHE_OBJECT_UNSWAPPED*
* *EventType#EVT_SWAP_SPACE_DATA_READ*
* *EventType#EVT_SWAP_SPACE_DATA_STORED*
* *EventType#EVT_SWAP_SPACE_DATA_REMOVED*
* *EventType#EVT_SWAP_SPACE_CLEARED*
* *EventType#EVT_SWAP_SPACE_DATA_EVICTED*
* *EventType#EVT_CACHE_OBJECT_TO_OFFHEAP*
* *EventType#EVT_CACHE_OBJECT_FROM_OFFHEAP*

Corresponding array of swap events was also removed: *EventType#EVTS_SWAPSPACE*.

All references to these events must be removed from sources of web-console.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] ignite pull request #1861: IGNITE-5058

[devozerov]

Github user devozerov closed the pull request at:

https://github.com/apache/ignite/pull/1861


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Windows XP support

[Igor Sapego]

Done. I'm going to refactor code appropriately then.

Best Regards,
Igor

On Sat, Apr 22, 2017 at 2:37 AM, Denis Magda  wrote:

> +1 for the discontinuation. Igor, please update C++ readme doc once this
> feature is implemented. I think it’s still fine to claim that Windows XP,
> Vista are supported for Ignite Java lib.
>
> Pasha, what’s about .NET? Do we still support these ancient Windows
> versions or the doc has to be altered as well?
>
> —
> Denis
>
> > On Apr 21, 2017, at 12:59 PM, Dmitriy Setrakyan 
> wrote:
> >
> > Agree, let's drop XP support.
> >
> > D.
> >
> > On Fri, Apr 21, 2017 at 8:03 AM, Pavel Tupitsyn 
> > wrote:
> >
> >> I don't think we should bother with supporting 16 year old discontinued
> OS.
> >> Vista support has also ended (just 10 days ago).
> >>
> >> We should put Windows 7 there as the oldest supported version.
> >>
> >> On Fri, Apr 21, 2017 at 5:59 PM, Igor Sapego 
> wrote:
> >>
> >>> Hello Igniters,
> >>>
> >>> So, I've been working on implementation of futures support for C++ [1]
> >> and
> >>> It would be convenient for me to use condition variables on Windows [2]
> >> as
> >>> well as on Linux. The problem here is that they are not supported on
> >>> Windows XP.
> >>>
> >>> So I've checked documentation [3] and found out that we claim Windows
> XP
> >>> support. Does anyone really needs it? AFAIK Windows XP is not supported
> >> by
> >>> MS any more.
> >>>
> >>> Would not it be more convenient to drop its support if no one needs it
> so
> >>> we can use more modern tools?
> >>>
> >>> [1] - https://issues.apache.org/jira/browse/IGNITE-1439
> >>> [2] -
> >>> https://msdn.microsoft.com/en-us/library/windows/desktop/
> >>> ms682052(v=vs.85).aspx
> >>> [3] -
> >>> https://apacheignite.readme.io/docs/getting-started#
> >> section-prerequisites
> >>>
> >>> Best Regards,
> >>> Igor
> >>>
> >>
>
>

[jira] [Created] (IGNITE-5063) Dynamic cache start hangs in validation fails

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-5063:
---

 Summary: Dynamic cache start hangs in validation fails
 Key: IGNITE-5063
 URL: https://issues.apache.org/jira/browse/IGNITE-5063
 Project: Ignite
  Issue Type: Bug
  Components: cache
Reporter: Vladimir Ozerov
Assignee: Semen Boikov
 Fix For: 2.0


When cache is started dynamically, it's validation occur in exchange thread. If 
exception is thrown at this point, operation hangs forever.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5062) Support new parameters in .Net

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-5062:


 Summary: Support new parameters in .Net
 Key: IGNITE-5062
 URL: https://issues.apache.org/jira/browse/IGNITE-5062
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Belyak
Assignee: Pavel Tupitsyn


Need to support new value and remove old ones:
In TcpDiscoverySpi:
remove maxMissedHeartbeats
remove maxMissedClientHeartbeats
remove heartbeatFrequency
rename hbFreq to metricsUpdateFrequency
In IgniteConfiguration:
add clientFailureDetectionTimeout (long with bounds from metricsUpdateFrequency 
to Integer.MAX_VALUE)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Null cache names

[Roman Shtykh]

Alexey,
Thank you for sharing the idea.
"REDIS_CACHE" can be specified in xml or programmatically (as in your example). 
If not, Ignite will behave in the same way when a cache the user intends to use 
is not created (I think it will produce an error or warning).I.e. it is a 
normal cache a user enables and configures when enabling Redis protocol.
Roman



On Monday, April 24, 2017 3:35 PM, Alexey Kuznetsov  
wrote:
 

 Roman,
Just as idea, how about in case of user does not configured "REDIS_CACHE" then 
create it via ignite.getOrCreateCache(new CacheConfiguration("REDIS_CACHE"))and 
prin warning to log "REDIS_CACHE not configured, using default partitioned 
cache".
What do you think?
On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh  
wrote:

Denis, Igor,
What can be done now
1. create a default cache name for Redis data, e.g. "REDIS_CACHE" that has to 
be configured explicitly in xml file (as it is done with other caches) by a 
user if he/she needs Redis protocol.
2. Force users to specify cache names as prefix to their keys, so that we can 
parse and switch between caches.
The 1st one is a very quick fix I can do today. This can be extended in future 
to have a separate cache for each data type.
Roman


    On Monday, April 24, 2017 12:15 AM, Denis Magda  wrote:


 Roman, would you suggest a quick solution for the redis integration or even
implement it in the nearest couple of days? We need to change the defaul
cache name in 2.0. Otherwise, it can be done only in an year or so in 3.0.

Denis

On Sunday, April 23, 2017, Seliverstov Igor  wrote:

> Hi Roman,
>
> The ticket number is IGNITE-3488.
>
> It's planned not to have null-named or default caches at all. All the
> caches must be defined explicitly or via templates and have names. The
> current implementation uses a cache with null name, so, we need some
> configuration property to define which cache to use or mapping in case of
> using Redis databases.
>
> Regards,
> Igor
>
> 22 апр. 2017 г. 8:47 пользователь "Roman Shtykh"  >
> написал:
>
> > Hi Igor,
> > The current implementation supports only STRING data type of Redis. In
> > addition, AFAIK, scaling Redis per dataset is normally done via running
> > separate instances of Redis for each dataset. Therefore my choice was
> > storing to the default cache. It looks fine from Redis' perspective, but
> > probably not from Ignite's.For other data types like HASH or SORTED SET,
> I
> > planned to specify cache name by key and treat value inside as Ignite's
> > key-values.# Redis has a notion of 'database' and you can switch between
> > them, but they can be referred only by the number, and limited to 16
> > databases... Not very useful, IMO.
> > If you still have the default cache, the current Redis integration should
> > work as is (I have to recheck it, what is the JIRA ticket for the null
> > cache issue?)
> > Do you just want to be sure your changes don't affect the Redis
> > integration, or do you want to extend it to switch between caches?
> > Roman
> >
> >    On Friday, April 21, 2017 8:17 PM, Seliverstov Igor <
> > gvvinbl...@gmail.com > wrote:
> >
> >
> >  Hi Roman,
> >
> > As far as I inderstand you're the author of the Redis protocol
> > implementation.
> >
> > Currently I'm working on a task to prohibit null names for caches and
> I've
> > found that current implementation implies using of default (aka null
> named)
> > cache as a Redis database.
> >
> > So, I need your assistance to implement Redis databases and mappings them
> > to particular caches.
> >
> > Is it in your plans to do it in near time?
> >
> > If not I'll appriciate your thoughts on how to do it properly)
> >
> > Regards,
> > Igor
> >
> >
> >
>

   



-- 
Alexey Kuznetsov


   

Re: Null cache names

[Vladimir Ozerov]

Roman,
Is it possible to define a kind of property in Redis connection string (or
property map) for this purpose? IMO ideally we should "externalize" cache
name somehow, so that it can be changed with no changes to user's code.

Alex,
Not sure if this is a good idea as you will end up with PARTITIONED cache
without backups with no ability to change that.

On Mon, Apr 24, 2017 at 9:35 AM, Alexey Kuznetsov 
wrote:

> Roman,
>
> Just as idea, how about in case of user does not configured "REDIS_CACHE"
>  then create it via ignite.getOrCreateCache(new
> CacheConfiguration("REDIS_CACHE"))
> and prin warning to log "REDIS_CACHE not configured, using default
> partitioned cache".
>
> What do you think?
>
> On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh 
> wrote:
>
> > Denis, Igor,
> > What can be done now
> > 1. create a default cache name for Redis data, e.g. "REDIS_CACHE" that
> has
> > to be configured explicitly in xml file (as it is done with other caches)
> > by a user if he/she needs Redis protocol.
> > 2. Force users to specify cache names as prefix to their keys, so that we
> > can parse and switch between caches.
> > The 1st one is a very quick fix I can do today. This can be extended in
> > future to have a separate cache for each data type.
> > Roman
> >
> >
> > On Monday, April 24, 2017 12:15 AM, Denis Magda  >
> > wrote:
> >
> >
> >  Roman, would you suggest a quick solution for the redis integration or
> > even
> > implement it in the nearest couple of days? We need to change the defaul
> > cache name in 2.0. Otherwise, it can be done only in an year or so in
> 3.0.
> >
> > Denis
> >
> > On Sunday, April 23, 2017, Seliverstov Igor 
> wrote:
> >
> > > Hi Roman,
> > >
> > > The ticket number is IGNITE-3488.
> > >
> > > It's planned not to have null-named or default caches at all. All the
> > > caches must be defined explicitly or via templates and have names. The
> > > current implementation uses a cache with null name, so, we need some
> > > configuration property to define which cache to use or mapping in case
> of
> > > using Redis databases.
> > >
> > > Regards,
> > > Igor
> > >
> > > 22 апр. 2017 г. 8:47 пользователь "Roman Shtykh"
> >  > > >
> > > написал:
> > >
> > > > Hi Igor,
> > > > The current implementation supports only STRING data type of Redis.
> In
> > > > addition, AFAIK, scaling Redis per dataset is normally done via
> running
> > > > separate instances of Redis for each dataset. Therefore my choice was
> > > > storing to the default cache. It looks fine from Redis' perspective,
> > but
> > > > probably not from Ignite's.For other data types like HASH or SORTED
> > SET,
> > > I
> > > > planned to specify cache name by key and treat value inside as
> Ignite's
> > > > key-values.# Redis has a notion of 'database' and you can switch
> > between
> > > > them, but they can be referred only by the number, and limited to 16
> > > > databases... Not very useful, IMO.
> > > > If you still have the default cache, the current Redis integration
> > should
> > > > work as is (I have to recheck it, what is the JIRA ticket for the
> null
> > > > cache issue?)
> > > > Do you just want to be sure your changes don't affect the Redis
> > > > integration, or do you want to extend it to switch between caches?
> > > > Roman
> > > >
> > > >On Friday, April 21, 2017 8:17 PM, Seliverstov Igor <
> > > > gvvinbl...@gmail.com > wrote:
> > > >
> > > >
> > > >  Hi Roman,
> > > >
> > > > As far as I inderstand you're the author of the Redis protocol
> > > > implementation.
> > > >
> > > > Currently I'm working on a task to prohibit null names for caches and
> > > I've
> > > > found that current implementation implies using of default (aka null
> > > named)
> > > > cache as a Redis database.
> > > >
> > > > So, I need your assistance to implement Redis databases and mappings
> > them
> > > > to particular caches.
> > > >
> > > > Is it in your plans to do it in near time?
> > > >
> > > > If not I'll appriciate your thoughts on how to do it properly)
> > > >
> > > > Regards,
> > > > Igor
> > > >
> > > >
> > > >
> > >
> >
> >
> >
>
>
>
> --
> Alexey Kuznetsov
>

[jira] [Created] (IGNITE-5061) Add ability to enable and disable rebalancing per-node

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-5061:


 Summary: Add ability to enable and disable rebalancing per-node
 Key: IGNITE-5061
 URL: https://issues.apache.org/jira/browse/IGNITE-5061
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Reporter: Alexey Goncharuk
 Fix For: 2.1


It would be nice to have an ability to enable and disable rebalancing per node. 
First of all, we need to come up with a simple API to allow this. Corresponding 
methods most likely should be also added to Ignite MBean.
We need to discuss on the dev-list whether it is enough to have this setting 
per-node, or this needs to be done on a per-cache basis.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (IGNITE-5060) Check configuration parameters on the Integer overflowing

[Alexander Belyak (JIRA)]

Alexander Belyak created IGNITE-5060:


 Summary: Check configuration parameters on the Integer overflowing
 Key: IGNITE-5060
 URL: https://issues.apache.org/jira/browse/IGNITE-5060
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Belyak


Time related configuration parameters using long data type (and expect value in 
ms), but standard java.net.Socket class expect integer for soDelay and usually 
long timeouts from configuration cast to ineter with simple (int) method with 
overflow if configuration timeout > Integer.MAX_VALUE.
Need to add configuration check for:
* IgniteConfiguration.failureDetectionTimeout
* IgniteConfiguration.clientFailureDetectionTimeout
* TcpDiscoverySpi.ackTimeout
* TcpDiscoverySpi.netTimeout




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Null cache names

[Alexey Kuznetsov]

Roman,

Just as idea, how about in case of user does not configured "REDIS_CACHE"
 then create it via ignite.getOrCreateCache(new
CacheConfiguration("REDIS_CACHE"))
and prin warning to log "REDIS_CACHE not configured, using default
partitioned cache".

What do you think?

On Mon, Apr 24, 2017 at 10:26 AM, Roman Shtykh 
wrote:

> Denis, Igor,
> What can be done now
> 1. create a default cache name for Redis data, e.g. "REDIS_CACHE" that has
> to be configured explicitly in xml file (as it is done with other caches)
> by a user if he/she needs Redis protocol.
> 2. Force users to specify cache names as prefix to their keys, so that we
> can parse and switch between caches.
> The 1st one is a very quick fix I can do today. This can be extended in
> future to have a separate cache for each data type.
> Roman
>
>
> On Monday, April 24, 2017 12:15 AM, Denis Magda 
> wrote:
>
>
>  Roman, would you suggest a quick solution for the redis integration or
> even
> implement it in the nearest couple of days? We need to change the defaul
> cache name in 2.0. Otherwise, it can be done only in an year or so in 3.0.
>
> Denis
>
> On Sunday, April 23, 2017, Seliverstov Igor  wrote:
>
> > Hi Roman,
> >
> > The ticket number is IGNITE-3488.
> >
> > It's planned not to have null-named or default caches at all. All the
> > caches must be defined explicitly or via templates and have names. The
> > current implementation uses a cache with null name, so, we need some
> > configuration property to define which cache to use or mapping in case of
> > using Redis databases.
> >
> > Regards,
> > Igor
> >
> > 22 апр. 2017 г. 8:47 пользователь "Roman Shtykh"
>  > >
> > написал:
> >
> > > Hi Igor,
> > > The current implementation supports only STRING data type of Redis. In
> > > addition, AFAIK, scaling Redis per dataset is normally done via running
> > > separate instances of Redis for each dataset. Therefore my choice was
> > > storing to the default cache. It looks fine from Redis' perspective,
> but
> > > probably not from Ignite's.For other data types like HASH or SORTED
> SET,
> > I
> > > planned to specify cache name by key and treat value inside as Ignite's
> > > key-values.# Redis has a notion of 'database' and you can switch
> between
> > > them, but they can be referred only by the number, and limited to 16
> > > databases... Not very useful, IMO.
> > > If you still have the default cache, the current Redis integration
> should
> > > work as is (I have to recheck it, what is the JIRA ticket for the null
> > > cache issue?)
> > > Do you just want to be sure your changes don't affect the Redis
> > > integration, or do you want to extend it to switch between caches?
> > > Roman
> > >
> > >On Friday, April 21, 2017 8:17 PM, Seliverstov Igor <
> > > gvvinbl...@gmail.com > wrote:
> > >
> > >
> > >  Hi Roman,
> > >
> > > As far as I inderstand you're the author of the Redis protocol
> > > implementation.
> > >
> > > Currently I'm working on a task to prohibit null names for caches and
> > I've
> > > found that current implementation implies using of default (aka null
> > named)
> > > cache as a Redis database.
> > >
> > > So, I need your assistance to implement Redis databases and mappings
> them
> > > to particular caches.
> > >
> > > Is it in your plans to do it in near time?
> > >
> > > If not I'll appriciate your thoughts on how to do it properly)
> > >
> > > Regards,
> > > Igor
> > >
> > >
> > >
> >
>
>
>



-- 
Alexey Kuznetsov