[jira] [Created] (IGNITE-1450) Excessive exceptions are printed out during cache/node stop

2015-09-11 Thread Alexey Goncharuk (JIRA)
Alexey Goncharuk created IGNITE-1450:


 Summary: Excessive exceptions are printed out during cache/node 
stop
 Key: IGNITE-1450
 URL: https://issues.apache.org/jira/browse/IGNITE-1450
 Project: Ignite
  Issue Type: Bug
Affects Versions: 1.1.4
Reporter: Alexey Goncharuk
Assignee: Yakov Zhdanov
 Fix For: ignite-1.4


In my unit tests, I start and stop caches in quick succession and my logs are 
full of the following exceptions:

6983 [exchange-worker-#114%t1-1%] ERROR GridCachePartitionExchangeManager - 
Runtime error caught during grid runnable execution: GridWorker 
[name=partition-exchanger, gridName=t1-1, finished=false, isCancelled=false, 
hashCode=1934136092, interrupted=false, runner=exchange-worker-#114%t1-1%]
class org.apache.ignite.IgniteException: Failed to wait for affinity ready 
future for topology version: AffinityTopologyVersion [topVer=2, minorTopVer=2]
at 
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.awaitTopologyVersion(GridAffinityAssignmentCache.java:443)
at 
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.cachedAffinity(GridAffinityAssignmentCache.java:398)
at 
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.nodes(GridAffinityAssignmentCache.java:363)
at 
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.nodes(GridCacheAffinityManager.java:201)
at 
org.apache.ignite.internal.processors.cache.GridCacheAffinityManager.localNode(GridCacheAffinityManager.java:303)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemandPool.assign(GridDhtPartitionDemandPool.java:1028)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.assign(GridDhtPreloader.java:260)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1076)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:108)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to wait for 
topology update, cache (or node) is stopping.
at 
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.readyFuture(GridAffinityAssignmentCache.java:316)
at 
org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.awaitTopologyVersion(GridAffinityAssignmentCache.java:437)
... 9 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: SqlQuery.pageSize is ignored in replicated cache

2015-09-11 Thread Sergi Vladykin
Denis,

It appears that in current implementation the page size means number of
rows which will be sent in a single message between nodes but it does not
define how many pages will be loaded. I agree that it is kind of unexpected
behavior but this is not that easy to fix.
We need to add lazy mode to H2 database, because now it executes SQL
queries actively and fetches all the results from remote nodes.

Sergi

2015-09-11 10:44 GMT+03:00 Denis Magda :

> Good,
>
> Sergi I've merged the test to 1.4 branch and assigned IGNITE-613 on you.
>
> When you fix the issue please validate the fix with this test and unmute
> the test by removing 'fail(...)' statement at the beginning of its body.
>
>
> --
> Denis
>
>
> On 9/10/2015 4:00 PM, Sergi Vladykin wrote:
>
>> Something like this could happen because replicated cache is incorrectly
>> expected to have the whole dataset locally.
>> I will take a look .
>>
>> Sergi
>>
>> 2015-09-10 15:38 GMT+03:00 Denis Magda :
>>
>> Hi Sergi,
>>>
>>> I was fixing one of the many SQL query tests (
>>> https://issues.apache.org/jira/browse/IGNITE-613) when found that there
>>> is an issue in how 'pageSize' parameter is processed by Ignite SQL
>>> engine.
>>>
>>> Test does the following.
>>>
>>> At the beginning it starts 3 server nodes and 1 client node. Fills a
>>> replicated cache with 1000 entries.
>>>
>>> Right after that the following query is executed from the client node
>>>
>>> QueryCursor> q =
>>>  cache.query(new SqlQuery(Integer.class,"_key >= 0
>>> order by _key").setPageSize(10));
>>>
>>> Then the test gets the first key in order to pre-load the first page:
>>>
>>> q.iterator().next().getKey()
>>>
>>>
>>> However, the iterator loads all 1000 entries ignoring the page size that
>>> is equal to 10.
>>> I see that the iterator already holds all the entries in a list and
>>> GridMapQueryExecutor.qryRess map on servers side is empty meaning that
>>> the
>>> whole data was transferred to the client.
>>>
>>> Is this a bug or expected behavior?
>>>
>>> --
>>> Denis
>>>
>>>
>


Re: vert.x integration

2015-09-11 Thread Dmitriy Setrakyan
Andrey,

According to IGNITE-1079, the Vert.x work is blocked by 3 other issues:
https://issues.apache.org/jira/browse/IGNITE-1079

Would be nice if folks in the community gave an update on these.

Thanks,
D.

On Fri, Sep 11, 2015 at 9:59 AM, Andrey Gura  wrote:

> Hi,
>
> I want to run Vert.x HA tests with the latest changes in Ignite. After it
> I'll check all related tickets and give answer.
>
> Other good news: bugs that I found in Vert.x test framework was fixed by
> Vert.x team.
>
> On Thu, Sep 10, 2015 at 11:09 PM, Dmitriy Setrakyan  >
> wrote:
>
> > This question goes to Andrey Gura.
> >
> > Andrey, I remember you have been working on Vert.x integration and got
> > stuck on one bug in Ignite that had to be fixed before hand. Was that bug
> > fixed by now? Can you let us know how close we are in your opinion to
> > completing this integration?
> >
> > D.
> >
>
>
>
> --
> Andrey Gura
> GridGain Systems, Inc.
> www.gridgain.com
>


Re: vert.x integration

2015-09-11 Thread Andrey Gura
Hi,

I want to run Vert.x HA tests with the latest changes in Ignite. After it
I'll check all related tickets and give answer.

Other good news: bugs that I found in Vert.x test framework was fixed by
Vert.x team.

On Thu, Sep 10, 2015 at 11:09 PM, Dmitriy Setrakyan 
wrote:

> This question goes to Andrey Gura.
>
> Andrey, I remember you have been working on Vert.x integration and got
> stuck on one bug in Ignite that had to be fixed before hand. Was that bug
> fixed by now? Can you let us know how close we are in your opinion to
> completing this integration?
>
> D.
>



-- 
Andrey Gura
GridGain Systems, Inc.
www.gridgain.com


[jira] [Created] (IGNITE-1449) DiscoverySpi interface should not reference TcpDiscoverySpi implementation

2015-09-11 Thread Kevin (JIRA)
Kevin created IGNITE-1449:
-

 Summary: DiscoverySpi interface should not reference 
TcpDiscoverySpi implementation
 Key: IGNITE-1449
 URL: https://issues.apache.org/jira/browse/IGNITE-1449
 Project: Ignite
  Issue Type: Bug
Reporter: Kevin


In DiscoverySpi, the setDataExchange and setMetricsProvider methods should 
return type DiscoverySpi instead of TcpDiscoverySpi.

It should be possible that implementations of DiscoverySpi do not extend 
TcpDiscoverySpi.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1448) CPP: Implement cache iterators.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1448:
---

 Summary: CPP: Implement cache iterators.
 Key: IGNITE-1448
 URL: https://issues.apache.org/jira/browse/IGNITE-1448
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1447) CPP: Implement cache locks.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1447:
---

 Summary: CPP: Implement cache locks.
 Key: IGNITE-1447
 URL: https://issues.apache.org/jira/browse/IGNITE-1447
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1446) CPP: Implement cache "keep-portable" semantics.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1446:
---

 Summary: CPP: Implement cache "keep-portable" semantics.
 Key: IGNITE-1446
 URL: https://issues.apache.org/jira/browse/IGNITE-1446
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1445) CPP: Implement cache invoke.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1445:
---

 Summary: CPP: Implement cache invoke.
 Key: IGNITE-1445
 URL: https://issues.apache.org/jira/browse/IGNITE-1445
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5


This will require well-defined architecture for generic callback calls from 
Java core to CPP.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1442) CPP: Implement cache async mode.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1442:
---

 Summary: CPP: Implement cache async mode.
 Key: IGNITE-1442
 URL: https://issues.apache.org/jira/browse/IGNITE-1442
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1444) CPP: Implement cache expiry policy.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1444:
---

 Summary: CPP: Implement cache expiry policy.
 Key: IGNITE-1444
 URL: https://issues.apache.org/jira/browse/IGNITE-1444
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1443) CPP: Implement cache continuous queries.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1443:
---

 Summary: CPP: Implement cache continuous queries.
 Key: IGNITE-1443
 URL: https://issues.apache.org/jira/browse/IGNITE-1443
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1441) CPP: Support filter in SCAN queries.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1441:
---

 Summary: CPP: Support filter in SCAN queries.
 Key: IGNITE-1441
 URL: https://issues.apache.org/jira/browse/IGNITE-1441
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1439) CPP: Implement futures.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1439:
---

 Summary: CPP: Implement futures.
 Key: IGNITE-1439
 URL: https://issues.apache.org/jira/browse/IGNITE-1439
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1440) CPP: Implement cache store.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1440:
---

 Summary: CPP: Implement cache store.
 Key: IGNITE-1440
 URL: https://issues.apache.org/jira/browse/IGNITE-1440
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1438) CPP: Implement transactions API.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1438:
---

 Summary: CPP: Implement transactions API.
 Key: IGNITE-1438
 URL: https://issues.apache.org/jira/browse/IGNITE-1438
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1437) CPP: Implement SQL fields query.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1437:
---

 Summary: CPP: Implement SQL fields query.
 Key: IGNITE-1437
 URL: https://issues.apache.org/jira/browse/IGNITE-1437
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1436) C++: Port to MAC OS.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1436:
---

 Summary: C++: Port to MAC OS.
 Key: IGNITE-1436
 URL: https://issues.apache.org/jira/browse/IGNITE-1436
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Blocker
 Fix For: ignite-1.5


It will require minimal porting of "common" and "utils" stuff.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[GitHub] ignite pull request: IGNITE-1435 Platform .Net: Make it possible t...

2015-09-11 Thread ptupitsyn
GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/84

IGNITE-1435 Platform .Net: Make it possible to sign core assembly with 
msbuild



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-1435

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/84.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #84


commit 3e694599b59ca2d941f76c2ffe8b103888c41e9f
Author: ptupitsyn 
Date:   2015-09-11T14:50:07Z

IGNITE-1435 Platform .Net: Make it possible to sign core assembly with 
msbuild




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[jira] [Created] (IGNITE-1435) Platform .Net: Make it possible to sign core assembly with msbuild

2015-09-11 Thread Pavel Tupitsyn (JIRA)
Pavel  Tupitsyn created IGNITE-1435:
---

 Summary: Platform .Net: Make it possible to sign core assembly 
with msbuild
 Key: IGNITE-1435
 URL: https://issues.apache.org/jira/browse/IGNITE-1435
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Pavel  Tupitsyn
Assignee: Pavel  Tupitsyn
 Fix For: ignite-1.4


Currently we use InternalsVisibleTo attribute in Apache.Ignite.Core 
AssemblyInfo, which prevents signing the assembly with msbuild /p:SignAssembly 
file. 

Need to wrap them in a preprocessor check so that msbuild flag 
/p:DefineConstants will exclude that code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1434) Platforms: Ensure that @IgniteInstanceResource is set only on setters, not fields.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1434:
---

 Summary: Platforms: Ensure that @IgniteInstanceResource is set 
only on setters, not fields.
 Key: IGNITE-1434
 URL: https://issues.apache.org/jira/browse/IGNITE-1434
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


Currently @IgniteInstanceResource is set on fields. And then we have to deal 
with complex initiailzation logic. 
Instead, we must set this annotation on setters which will perform 
initialization immediately.
Furthermore, when possible we must avoid injection in favor of direct setter 
calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1433) .Net: Add GridException.JavaStackTrace

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1433:
---

 Summary: .Net: Add GridException.JavaStackTrace
 Key: IGNITE-1433
 URL: https://issues.apache.org/jira/browse/IGNITE-1433
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


Propagate java stack trace as a string in ExceptionUtils.GetException and write 
it to a new field in IgniteException class.

This will simplify debugging for us both locally and when getting error reports 
from clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1432) .Net: Fix InteropCacheEntryProcessor performance on remote nodes

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1432:
---

 Summary: .Net: Fix InteropCacheEntryProcessor performance on 
remote nodes
 Key: IGNITE-1432
 URL: https://issues.apache.org/jira/browse/IGNITE-1432
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5


Considerations:
1) Invoke with single key is expected to be called only once, so no changes is 
needed here - deploy and execute in a single JNI call.
2) If there are several keys, there is a high chance (but still not 100% due to 
partitioning) that processor will be called multiple times.
Proposed solution:
1) Check amout of keys.
2) If cnt == 1, no changes to current logic.
3) If cnt > 1, first deploy (JNI call), then execute (JNI call). Processor 
entry must be put into weak-map located somewhere inside the interop processor. 
Interop processor must constantly listen for corresponding reference queue and 
release .Net entries as soon as processor is weakly reacheable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1431) .Net: Ensure strings are written correctly in BIG/LITTLE endian modes.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1431:
---

 Summary: .Net: Ensure strings are written correctly in BIG/LITTLE 
endian modes.
 Key: IGNITE-1431
 URL: https://issues.apache.org/jira/browse/IGNITE-1431
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Minor
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1430) Platforms: Incorrect behavior of write-behind store on node stop

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1430:
---

 Summary: Platforms: Incorrect behavior of write-behind store on 
node stop
 Key: IGNITE-1430
 URL: https://issues.apache.org/jira/browse/IGNITE-1430
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Blocker
 Fix For: ignite-1.5


We destroy unmanaged native resources on node "stop" phase. But write-behind 
store can still have som unflushed data at this point because cache processor 
is not stoped yet.
To fix this we must add more callbacks to platform processor which are invoked 
before any component is started and after any component is stopped.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1429) .Net: Optimize stream receiver so that it is created only once on remote node.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1429:
---

 Summary: .Net: Optimize stream receiver so that it is created only 
once on remote node.
 Key: IGNITE-1429
 URL: https://issues.apache.org/jira/browse/IGNITE-1429
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5


Looks like we can employ PhantomReference in Java for this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1428) .Net: Use struct with fixed layout for InteropMemory operations.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1428:
---

 Summary: .Net: Use struct with fixed layout for InteropMemory 
operations.
 Key: IGNITE-1428
 URL: https://issues.apache.org/jira/browse/IGNITE-1428
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Minor
 Fix For: ignite-1.5


This will decrease probability of bugs caused by pointer arithmetics.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1427) .Net: Services - add support for non-deserializable AffinityKey.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1427:
---

 Summary: .Net: Services - add support for non-deserializable 
AffinityKey.
 Key: IGNITE-1427
 URL: https://issues.apache.org/jira/browse/IGNITE-1427
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


Currently if affinity key cannot be marshalled for some reason, it will break 
not only affected service descriptor, but all services API.
We should catch marshalling exceptions and pass it to .Net in a wrapped form. 
These exception must be thrown only if user is trying to access problematic key 
explicitly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Heap size in topology snapshot

2015-09-11 Thread Dmitriy Setrakyan
On Fri, Sep 11, 2015 at 4:27 AM, Sergey Kozlov  wrote:

> What's the reason to know total client heap size?
>

Even though the total client heap size is not as meaningful, since we are
showing servers and clients in the log message, for consistency reasons I
think it makes sense to show total allocated servers heap and clients heap.



> On Fri, Sep 11, 2015 at 1:59 PM, Alexey Kuznetsov  >
> wrote:
>
> > It seems that we should write: servers-heap: xxGB, clients-heap: yyGB
> >
> > On Fri, Sep 11, 2015 at 4:32 PM, Sergey Kozlov 
> > wrote:
> >
> > > Hi
> > >
> > > I've found that heap size of client nodes used in total size of grid
> heap
> > > size:
> > >
> > > [11:35:56,444][INFO][main][GridDiscoveryManager] Topology snapshot
> > [ver=4,
> > > servers=4, clients=0, CPUs=4, heap=8.0GB]
> > >
> [11:36:04,556][INFO][disco-event-worker-#45%null%][GridDiscoveryManager]
> > > Added new node to topology: TcpDiscoveryNode
> > > [id=3cc4213b-b4a7-4f45-88b3-07cbdc3488b6, addrs=[0:0:0:0:0:0:0:1,
> > > 127.0.0.1, 192.168.100.10], sockAddrs=[ksm-homepc/169.254.90.143:0,
> > > /0:0:0:0:0:0:0:1:0, /127.0.0.1:0, /192.168.100.10:0], discPort=0,
> > order=5,
> > > intOrder=5, lastExchangeTime=1441960564491, loc=false,
> > > ver=1.4.0#20150910-sha1:071586e2, isClient=true]
> > >
> [11:36:04,558][INFO][disco-event-worker-#45%null%][GridDiscoveryManager]
> > > Topology snapshot [ver=5, servers=4, clients=1, CPUs=4, heap=9.0GB]
> > >
> > > But really only server nodes heap used for a grid and it might confuse
> > the
> > > user for available heap on grid if a topology has a few server nodes
> and
> > > many clients.
> > >
> > > --
> > > Sergey Kozlov
> > >
> >
> >
> >
> > --
> > Alexey Kuznetsov
> > GridGain Systems
> > www.gridgain.com
> >
>
>
>
> --
> Sergey Kozlov
>


[jira] [Created] (IGNITE-1426) .Net: Improve/add forked tests

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1426:
---

 Summary: .Net: Improve/add forked tests
 Key: IGNITE-1426
 URL: https://issues.apache.org/jira/browse/IGNITE-1426
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


Run forked tests from a separate directory to better cover assembly loading 
code and examples scenarios.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1425) .Net: Reflective serializer can cache field IDs.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1425:
---

 Summary: .Net: Reflective serializer can cache field IDs.
 Key: IGNITE-1425
 URL: https://issues.apache.org/jira/browse/IGNITE-1425
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


Instead of calculating field IDs on each run, reflective serializer can 
calculate them once during initialization and then just write directly as ints.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1424) .Net: Add tests for BIG_ENDIANs.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1424:
---

 Summary: .Net: Add tests for BIG_ENDIANs.
 Key: IGNITE-1424
 URL: https://issues.apache.org/jira/browse/IGNITE-1424
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Minor
 Fix For: ignite-1.5


As all our hardware uses LITTLE_ENDIAN byte order, BIG_ENDIAN was never tested 
properly. We do not know, whether it works. We do not know it's performance 
characteristics. 
Need to add ability to override endian final flags in both .Net and Java (e.g. 
through a system property or through "Ex" configuration) and then run unit 
tests with it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1423) Zero expiry policy does not work sometimes

2015-09-11 Thread Pavel Tupitsyn (JIRA)
Pavel  Tupitsyn created IGNITE-1423:
---

 Summary: Zero expiry policy does not work sometimes
 Key: IGNITE-1423
 URL: https://issues.apache.org/jira/browse/IGNITE-1423
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 1.1.4
Reporter: Pavel  Tupitsyn
 Fix For: ignite-1.4


Paste the following test to GridCacheReplicatedMultiNodeFullApiSelfTest.java 
and run.

{code:title=GridCacheReplicatedMultiNodeFullApiSelfTest.java}
public void testExpiredBug() throws Exception {
final ExpiryPolicy plc = new ExpiryPolicy() {
@Override public Duration getExpiryForCreation() {
return Duration.ZERO;
}

@Override public Duration getExpiryForAccess() {
return Duration.ZERO;
}

@Override public Duration getExpiryForUpdate() {
return Duration.ZERO;
}
};

IgniteCache c1 = ignite(0).cache(null);
IgniteCache c2 = ignite(1).cache(null);

String key0 = primaryKeysForCache(c1, 1).get(0);
String key1 = primaryKeysForCache(c2, 1).get(0);

IgniteCache cache0 = c1;
IgniteCache cache = c1.withExpiryPolicy(plc);

for (int i = 0; i < 3000; i++) {
cache.put(key0, key0);
cache.put(key1, key1);
assert !cache0.containsKey(key0);
assert !cache0.containsKey(key1);

cache0.put(key0, key0);
cache0.put(key1, key1);
assert cache0.containsKey(key0);
assert cache0.containsKey(key1);
cache.put(key0, key0 + 1);
cache.put(key1, key1 + 1);
assert !cache0.containsKey(key0);
assert !cache0.containsKey(key1);

cache0.put(key0, key0);
cache0.put(key1, key1);
cache.get(key0);
cache.get(key1);
assert !cache0.containsKey(key0);
assert !cache0.containsKey(key1);
}
}}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1422) .Net: Marshalling can be optimized for fixed-length objects.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1422:
---

 Summary: .Net: Marshalling can be optimized for fixed-length 
objects.
 Key: IGNITE-1422
 URL: https://issues.apache.org/jira/browse/IGNITE-1422
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


Sometimes we can guess user object length in advance:
1) Strict rule: it is "reflective" and contains only fixed-length fields.
2) Speculative rule: It is "marshal aware", contains only fixed-length fields 
and there is only one metadata "path".

In these cases we can do the following:
1) No tracking of length and raw offset.
2) Header can be written as one memcpy() rather than as multiple int/byte/bool 
writes.

We need to have a fallback strategy for a cases when new metadata path appears 
or user write raw data. It is very easy: if expected and actual lengths differ 
at the end, set proper values.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1421) .Net: Optimize writes of ultra-dense structures.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1421:
---

 Summary: .Net: Optimize writes of ultra-dense structures.
 Key: IGNITE-1421
 URL: https://issues.apache.org/jira/browse/IGNITE-1421
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


There is a case when we can dramatically increase marshal/unmarshal performance:
1) Type is a structure;
2) It contains only integer/float/double primitive types with well-defined;
memory layout which is consistent with our serialization protocol;
3) We are sure that there are no gaps in it's memory layout.
4) User writes it as "marshal-aware" and fields write order is consistent with 
memory layout.

In this case we can copy the whole structure into our stream with a single 
memcpy() operation. And we can read it from the stream as easy as [pointer 
dereference + position shift] (provided that target is on the stack).





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1420) .Net: Optimize metadata write.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1420:
---

 Summary: .Net: Optimize metadata write.
 Key: IGNITE-1420
 URL: https://issues.apache.org/jira/browse/IGNITE-1420
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


Currently we perform int dictionary lookup on each field write to track 
metadata updates. This could be less than optimal because metadata updates are 
very rare.

Several optimizations can be applied here:
1) For types where we use reflection we can collect all metadata information 
only once and write it when the first object of a type gets marshalled. Nothing 
more is needed.
2) For "marshal aware" types we never know whether new metadata is expected or 
not. We can introduce tree-like structure. When serialization start we are on 
the top of it. Then we go down as the fields are written. If we cannot find the 
node for the given field ID - we are on the new path. Then we create a copy of 
a tree and continue our path collecting updates. Finally, we flush them to Java 
and update global tree with latest changes.
This can be highly optimized for the most common cases when there is one, at 
most two paths for the given sequence:
- When there is only one known path, data can be stored as array. We simply go 
through it and compare current field ID with expected. So this is just int[].
- When there are more paths in some places, we can have int[] + dispatch map. 
For 2-4 paths its is just a binary switch. For more paths we can fallback to 
normal dictionary.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1419) .Net: Add optional "raw" flag to portable type descriptor.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1419:
---

 Summary: .Net: Add optional "raw" flag to portable type descriptor.
 Key: IGNITE-1419
 URL: https://issues.apache.org/jira/browse/IGNITE-1419
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


By default we write .Net objects with field names included. But in some cases 
it is not needed. E.g. when objects are not going to be used in queries.

Removing field names will speed up serialization/deserialization. We can add 
optional "raw" flag to portable type descriptor to control this behavior.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1418) .Net: Implement distributed count-down latch (event).

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1418:
---

 Summary: .Net: Implement distributed count-down latch (event).
 Key: IGNITE-1418
 URL: https://issues.apache.org/jira/browse/IGNITE-1418
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1417) .Net: Implement desitributed queue.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1417:
---

 Summary: .Net: Implement desitributed queue.
 Key: IGNITE-1417
 URL: https://issues.apache.org/jira/browse/IGNITE-1417
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1416) .Net: Implement "atomic" data structures.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1416:
---

 Summary: .Net: Implement "atomic" data structures.
 Key: IGNITE-1416
 URL: https://issues.apache.org/jira/browse/IGNITE-1416
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5


This includes:
- atomicLong;
- atomicSequence;
- atomicStamped;
- atomicReference.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1415) .Net: Optimize handle registry.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1415:
---

 Summary: .Net: Optimize handle registry.
 Key: IGNITE-1415
 URL: https://issues.apache.org/jira/browse/IGNITE-1415
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Priority: Critical
 Fix For: ignite-1.5


The main problem with handle registry is CAS on a single shared variable. This 
could result in a very bad performance under contention. 
Several techniques can be applied here:
1) Stripes. Assign some ID to a thread and then use it to pick correct stripe 
for the thread. Be careful with false-sharing effects as stripes can be located 
very close to each other.
2) Cleanup with relaxed membars. When we are to remove the handle, no need for 
full HB semantics. We are ok if subsequent calls to the same handle will see 
not-killed object for a while.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1414) .Net: support async/await in the API via extension methods

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1414:
---

 Summary: .Net: support async/await in the API via extension methods
 Key: IGNITE-1414
 URL: https://issues.apache.org/jira/browse/IGNITE-1414
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


Currently to get awaitable target (Task) for async operation, user have to do 
the following (pseudocode):
{code}
target.ExecuteOperation();

IFuture fut = target.CurrentFuture();

Task task = fut.ToTask();
{code}

This is too complex. We need to think how to minimize this boilerplate code. 
Extensions + lambdas appears to be good candidates for this. E.g.:
{code}
Task task = target.DoAsync(t => t.Execute());
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1413) .Net: Get rid of set -> map conversion in PlatformDotNetCacheStore.writeAll

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1413:
---

 Summary: .Net: Get rid of set -> map conversion in 
PlatformDotNetCacheStore.writeAll
 Key: IGNITE-1413
 URL: https://issues.apache.org/jira/browse/IGNITE-1413
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


This is a legacy artifact from a veeery old API which is currently removed. 
Just pass set without unnecessary conversions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1412) .Net: Pass local entry processor pointer to Java.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1412:
---

 Summary: .Net: Pass local entry processor pointer to Java.
 Key: IGNITE-1412
 URL: https://issues.apache.org/jira/browse/IGNITE-1412
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


Currently we always serialize .Net entry processor and pass only serialized 
bytes to Java. 
If processor is to be executed locally, then we pass the same bytes back and 
deserialize them again. 
Instead, we can do the following:
1) Serialize entry processor.
2) Allocate handle for it.
3) Pass bytes and handle to Java.
4) In case of local execution, just return back the handle.
5) Once execution is finished, handle can be safely released in "finally" block.

This can be easily implemented only for synchronous entry processing. 

For async mode there is no try/finally. We could release the handle during 
feature completion, but the problem is that there is no guarantee tha user will 
ever request this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1411) .Net: Understand why CacheAbstractTest.TestCacheEntryGetSet*() tests are commented out.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1411:
---

 Summary: .Net: Understand why 
CacheAbstractTest.TestCacheEntryGetSet*() tests are commented out.
 Key: IGNITE-1411
 URL: https://issues.apache.org/jira/browse/IGNITE-1411
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1410) .Net: Implement IEvents.RecordLocal.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1410:
---

 Summary: .Net: Implement IEvents.RecordLocal.
 Key: IGNITE-1410
 URL: https://issues.apache.org/jira/browse/IGNITE-1410
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Heap size in topology snapshot

2015-09-11 Thread Sergey Kozlov
What's the reason to know total client heap size?

On Fri, Sep 11, 2015 at 1:59 PM, Alexey Kuznetsov 
wrote:

> It seems that we should write: servers-heap: xxGB, clients-heap: yyGB
>
> On Fri, Sep 11, 2015 at 4:32 PM, Sergey Kozlov 
> wrote:
>
> > Hi
> >
> > I've found that heap size of client nodes used in total size of grid heap
> > size:
> >
> > [11:35:56,444][INFO][main][GridDiscoveryManager] Topology snapshot
> [ver=4,
> > servers=4, clients=0, CPUs=4, heap=8.0GB]
> > [11:36:04,556][INFO][disco-event-worker-#45%null%][GridDiscoveryManager]
> > Added new node to topology: TcpDiscoveryNode
> > [id=3cc4213b-b4a7-4f45-88b3-07cbdc3488b6, addrs=[0:0:0:0:0:0:0:1,
> > 127.0.0.1, 192.168.100.10], sockAddrs=[ksm-homepc/169.254.90.143:0,
> > /0:0:0:0:0:0:0:1:0, /127.0.0.1:0, /192.168.100.10:0], discPort=0,
> order=5,
> > intOrder=5, lastExchangeTime=1441960564491, loc=false,
> > ver=1.4.0#20150910-sha1:071586e2, isClient=true]
> > [11:36:04,558][INFO][disco-event-worker-#45%null%][GridDiscoveryManager]
> > Topology snapshot [ver=5, servers=4, clients=1, CPUs=4, heap=9.0GB]
> >
> > But really only server nodes heap used for a grid and it might confuse
> the
> > user for available heap on grid if a topology has a few server nodes and
> > many clients.
> >
> > --
> > Sergey Kozlov
> >
>
>
>
> --
> Alexey Kuznetsov
> GridGain Systems
> www.gridgain.com
>



-- 
Sergey Kozlov


[jira] [Created] (IGNITE-1409) PlatformEventFilterListenerImpl should be externalizable.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1409:
---

 Summary: PlatformEventFilterListenerImpl should be externalizable.
 Key: IGNITE-1409
 URL: https://issues.apache.org/jira/browse/IGNITE-1409
 Project: Ignite
  Issue Type: Task
  Components: interop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
Assignee: Vladimir Ozerov
 Fix For: ignite-1.5






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: Heap size in topology snapshot

2015-09-11 Thread Alexey Kuznetsov
It seems that we should write: servers-heap: xxGB, clients-heap: yyGB

On Fri, Sep 11, 2015 at 4:32 PM, Sergey Kozlov  wrote:

> Hi
>
> I've found that heap size of client nodes used in total size of grid heap
> size:
>
> [11:35:56,444][INFO][main][GridDiscoveryManager] Topology snapshot [ver=4,
> servers=4, clients=0, CPUs=4, heap=8.0GB]
> [11:36:04,556][INFO][disco-event-worker-#45%null%][GridDiscoveryManager]
> Added new node to topology: TcpDiscoveryNode
> [id=3cc4213b-b4a7-4f45-88b3-07cbdc3488b6, addrs=[0:0:0:0:0:0:0:1,
> 127.0.0.1, 192.168.100.10], sockAddrs=[ksm-homepc/169.254.90.143:0,
> /0:0:0:0:0:0:0:1:0, /127.0.0.1:0, /192.168.100.10:0], discPort=0, order=5,
> intOrder=5, lastExchangeTime=1441960564491, loc=false,
> ver=1.4.0#20150910-sha1:071586e2, isClient=true]
> [11:36:04,558][INFO][disco-event-worker-#45%null%][GridDiscoveryManager]
> Topology snapshot [ver=5, servers=4, clients=1, CPUs=4, heap=9.0GB]
>
> But really only server nodes heap used for a grid and it might confuse the
> user for available heap on grid if a topology has a few server nodes and
> many clients.
>
> --
> Sergey Kozlov
>



-- 
Alexey Kuznetsov
GridGain Systems
www.gridgain.com


Heap size in topology snapshot

2015-09-11 Thread Sergey Kozlov
Hi

I've found that heap size of client nodes used in total size of grid heap
size:

[11:35:56,444][INFO][main][GridDiscoveryManager] Topology snapshot [ver=4,
servers=4, clients=0, CPUs=4, heap=8.0GB]
[11:36:04,556][INFO][disco-event-worker-#45%null%][GridDiscoveryManager]
Added new node to topology: TcpDiscoveryNode
[id=3cc4213b-b4a7-4f45-88b3-07cbdc3488b6, addrs=[0:0:0:0:0:0:0:1,
127.0.0.1, 192.168.100.10], sockAddrs=[ksm-homepc/169.254.90.143:0,
/0:0:0:0:0:0:0:1:0, /127.0.0.1:0, /192.168.100.10:0], discPort=0, order=5,
intOrder=5, lastExchangeTime=1441960564491, loc=false,
ver=1.4.0#20150910-sha1:071586e2, isClient=true]
[11:36:04,558][INFO][disco-event-worker-#45%null%][GridDiscoveryManager]
Topology snapshot [ver=5, servers=4, clients=1, CPUs=4, heap=9.0GB]

But really only server nodes heap used for a grid and it might confuse the
user for available heap on grid if a topology has a few server nodes and
many clients.

-- 
Sergey Kozlov


Re: Added missing pages to Ignite documentation

2015-09-11 Thread Anton Vinogradov
Dmitriy,

Thanks for improving javadoc.
I've updated https://apacheignite.readme.io/docs/topology-validation according
to your changes.

On Fri, Sep 11, 2015 at 2:38 AM, Dmitriy Setrakyan 
wrote:

> Thanks Anton,
>
> I made a few edits. Please take a look and see if you agree.
>
> D.
>
> On Thu, Sep 10, 2015 at 3:08 AM, Anton Vinogradov <
> avinogra...@gridgain.com>
> wrote:
>
> > Done, thanks for tips
> >
> > On Thu, Sep 10, 2015 at 12:44 PM, Yakov Zhdanov 
> > wrote:
> >
> > > I think that main docs should be in TopologyValidator interface. Other
> > > places should be briefly documented and contain a link to validator
> > > interface.
> > >
> > > Thanks!
> > >
> > > Yakov
> > > On Sep 10, 2015 11:22, "Anton Vinogradov" 
> > > wrote:
> > >
> > > > Usage explaned at place where TopologyValidator used - at
> > > > CacheConfiguration.setTopologyValidator(tv).
> > > > Is it necessary to dublicate usage javadoc to TopologyValidator
> class?
> > > >
> > > > On Thu, Sep 10, 2015 at 11:00 AM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org
> > > > >
> > > > wrote:
> > > >
> > > > > On Thu, Sep 10, 2015 at 12:35 AM, Anton Vinogradov <
> > > > > avinogra...@gridgain.com
> > > > > > wrote:
> > > > >
> > > > > > Dmitriy,
> > > > > > TopologyValidator's javadoc was improved as a part of these
> > changes.
> > > > > >
> > > > >
> > > > > I just checked in master, and this is the only javadoc I can find
> for
> > > > > TopologyValidator class:
> > > > >
> > > > > /**
> > > > >  * Topology validator.
> > > > >  */
> > > > >
> > > > > A bit too pithy for my taste :)
> > > > >
> > > > >
> > > > > > On Thu, Sep 10, 2015 at 6:41 AM, Dmitriy Setrakyan <
> > > > > dsetrak...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks Anton!
> > > > > > >
> > > > > > > By the way, I just noticed that the TopologyValidator has no
> > > javadoc.
> > > > > > > Perhaps it makes sense to add it as well?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Sat, Sep 5, 2015 at 11:15 PM, Anton Vinogradov <
> > > > > > > avinogra...@gridgain.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > I've created new pages on redme.io.
> > > > > > > > These pages describe usage of Ignite expiry policies and
> > topology
> > > > > > > > validator.
> > > > > > > > Urls:
> > > > > > > > https://apacheignite.readme.io/docs/expiry-policies
> > > > > > > > https://apacheignite.readme.io/docs/topology-validation
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>


Re: NPE

2015-09-11 Thread Denis Magda

Pavel, thanks for reporting!

I'll take a look a bit later.
However, please add the steps to reproduce the issue into the ticket or 
upload already existed code that catches the error. This will help a lot.


--
Denis

On 9/11/2015 10:19 AM, Pavel Konstantinov wrote:

Hi, Igniters,

I'm faced with NPE and ask anybody to take a look at the filed ticket:
https://issues.apache.org/jira/browse/IGNITE-1405

Perhaps we should fix it now in 1.4


Pavel.





[jira] [Created] (IGNITE-1408) Error in schema import code generation

2015-09-11 Thread Pavel Konstantinov (JIRA)
Pavel Konstantinov created IGNITE-1408:
--

 Summary: Error in schema import code generation
 Key: IGNITE-1408
 URL: https://issues.apache.org/jira/browse/IGNITE-1408
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Konstantinov
Assignee: Vasiliy Sisko


Ascending fields defines twice



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1407) IGFS file cannot be removed when underlying HDFS file is removed externally.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1407:
---

 Summary: IGFS file cannot be removed when underlying HDFS file is 
removed externally.
 Key: IGNITE-1407
 URL: https://issues.apache.org/jira/browse/IGNITE-1407
 Project: Ignite
  Issue Type: Task
  Components: hadoop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


In this case we simply check HDFS for file existence, it returns false, and we 
exit immediately without trying to remove a file from IGFS.
Current approach offers better performance, but require all mutable interaction 
with file system to happen through IGFS.
Looks like we have to give user ability to choose between two options:
1) Better performance, but all goes through IGFS;
2) Worse performance, but all external changes do not break IGFS.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (IGNITE-1406) Need to mention "shmem" JAR in Hadoop docs.

2015-09-11 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-1406:
---

 Summary: Need to mention "shmem" JAR in Hadoop docs.
 Key: IGNITE-1406
 URL: https://issues.apache.org/jira/browse/IGNITE-1406
 Project: Ignite
  Issue Type: Improvement
  Components: hadoop
Affects Versions: 1.1.4
Reporter: Vladimir Ozerov
 Fix For: ignite-1.5


We inform users that "core" and "hadoop" modules should be placed to Hadoop 
classpath: https://apacheignite.readme.io/docs/file-system
But it is also necessary to put "shmem" jar in order to enable SHMEM 
communication which is default fo Linux platforms.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Re: SqlQuery.pageSize is ignored in replicated cache

2015-09-11 Thread Denis Magda

Good,

Sergi I've merged the test to 1.4 branch and assigned IGNITE-613 on you.

When you fix the issue please validate the fix with this test and unmute 
the test by removing 'fail(...)' statement at the beginning of its body.



--
Denis

On 9/10/2015 4:00 PM, Sergi Vladykin wrote:

Something like this could happen because replicated cache is incorrectly
expected to have the whole dataset locally.
I will take a look .

Sergi

2015-09-10 15:38 GMT+03:00 Denis Magda :


Hi Sergi,

I was fixing one of the many SQL query tests (
https://issues.apache.org/jira/browse/IGNITE-613) when found that there
is an issue in how 'pageSize' parameter is processed by Ignite SQL engine.

Test does the following.

At the beginning it starts 3 server nodes and 1 client node. Fills a
replicated cache with 1000 entries.

Right after that the following query is executed from the client node

QueryCursor> q =
 cache.query(new SqlQuery(Integer.class,"_key >= 0
order by _key").setPageSize(10));

Then the test gets the first key in order to pre-load the first page:

q.iterator().next().getKey()


However, the iterator loads all 1000 entries ignoring the page size that
is equal to 10.
I see that the iterator already holds all the entries in a list and
GridMapQueryExecutor.qryRess map on servers side is empty meaning that the
whole data was transferred to the client.

Is this a bug or expected behavior?

--
Denis





NPE

2015-09-11 Thread Pavel Konstantinov
Hi, Igniters,

I'm faced with NPE and ask anybody to take a look at the filed ticket:
https://issues.apache.org/jira/browse/IGNITE-1405

Perhaps we should fix it now in 1.4


Pavel.


[jira] [Created] (IGNITE-1405) NPE during node start in coner case

2015-09-11 Thread Pavel Konstantinov (JIRA)
Pavel Konstantinov created IGNITE-1405:
--

 Summary: NPE during node start in coner case
 Key: IGNITE-1405
 URL: https://issues.apache.org/jira/browse/IGNITE-1405
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: ignite-1.4
Reporter: Pavel Konstantinov
Assignee: Yakov Zhdanov
 Fix For: ignite-1.5


I'm tried to establish external connect to node during it starting
{code}
[14:07:29,284][SEVERE][ignite-#45%rest-tester%][GridTcpRestProtocol] Failed to 
process client request: GridClientTopologyRequest [includeMetrics=true, 
includeAttrs=true]
class org.apache.ignite.IgniteCheckedException: null
at 
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:6978)
at 
org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:150)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.allNodes(GridDiscoveryManager.java:1421)
at 
org.apache.ignite.internal.processors.rest.handlers.top.GridTopologyCommandHandler.handleAsync(GridTopologyCommandHandler.java:100)
at 
org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest(GridRestProcessor.java:226)
at 
org.apache.ignite.internal.processors.rest.GridRestProcessor.access$100(GridRestProcessor.java:79)
at 
org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:133)
... 4 more
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)