[jira] [Commented] (IGNITE-2788) Redis API for Ignite to work with data via the Redis protocol

2016-03-18 Thread Roman Shtykh (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198547#comment-15198547
 ] 

Roman Shtykh commented on IGNITE-2788:
--

Yes, I am thinking about taking it after I complete IGNITE-2730, which was 
requested from Kafka users.
This task is pretty big and I also think we have to implement it iteratively.

I will assign it to myself and if anyone else is interested, let's collaborate!

> Redis API for Ignite to work with data via the Redis protocol
> -
>
> Key: IGNITE-2788
> URL: https://issues.apache.org/jira/browse/IGNITE-2788
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Roman Shtykh
>
> Introduce Redis API that works with the Redis protocol but uses Ignite grid.
> Needless to say, Redis is an extremely popular caching solution. Such API 
> will enable smooth migration to Ignite.
> As a first phase we can start with most frequently used commands and enhance 
> gradually.
> Redis commands http://redis.io/commands



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-813) Apache Flink Integration -- data streaming connector

2016-03-18 Thread Saikat Maitra (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15197891#comment-15197891
 ] 

Saikat Maitra commented on IGNITE-813:
--

[~roman_s][~avinogradov]

Hi Roman, Anton

Thank you for reviewing the code and sharing feedback. I will make the changes 
and update on this ticket.

Regards
Saikat

> Apache Flink Integration -- data streaming connector
> 
>
> Key: IGNITE-813
> URL: https://issues.apache.org/jira/browse/IGNITE-813
> Project: Ignite
>  Issue Type: New Feature
>  Components: streaming
>Reporter: Suminda Dharmasena
>Assignee: Saikat Maitra
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2835) BinaryObjectOffHeapImpl leaked to public code

2016-03-18 Thread Artem Shutak (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201916#comment-15201916
 ] 

Artem Shutak commented on IGNITE-2835:
--

Made a potential fix. Waiting for TC results.

> BinaryObjectOffHeapImpl leaked to public code
> -
>
> Key: IGNITE-2835
> URL: https://issues.apache.org/jira/browse/IGNITE-2835
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 1.5.0.final
>Reporter: Denis Magda
>Assignee: Artem Shutak
>Priority: Critical
>  Labels: community, important
> Fix For: 1.6
>
> Attachments: BinaryObjectOffHeapIssue.java
>
>
> To my knowledge {{BinaryObjectOffHeapImpl}} is considered to be used under 
> some internal lock only to prevent possible offheap pointer movement.
> However seems that we made it available to public code. If to start a 
> partitioned cache in {{OFFHEAP_TIRED}} mode, get {{BinaryObject}} from the 
> cache inside of a TX and put the same object back we will get exception like 
> below
> {noformat}
> [15:00:00,892][WARN ][main][GridNearTxLocal] Set transaction invalidation 
> flag to true due to error [tx=GridNearTxLocal [mappings=IgniteTxMappingsImpl 
> [], nearLocallyMapped=false, colocatedLocallyMapped=true, 
> needCheckBackup=null, hasRemoteLocks=false, mappings=IgniteTxMappingsImpl [], 
> super=GridDhtTxLocalAdapter [nearOnOriginatingNode=false, nearNodes=[], 
> dhtNodes=[], explicitLock=false, super=IgniteTxLocalAdapter 
> [completedBase=null, sndTransformedVals=false, depEnabled=false, 
> txState=IgniteTxStateImpl [activeCacheIds=GridLongList [idx=1, 
> arr=[-1206548976]], txMap={IgniteTxKey [key=KeyCacheObjectImpl [val=0, 
> hasValBytes=true], cacheId=-1206548976]=IgniteTxEntry [key=KeyCacheObjectImpl 
> [val=0, hasValBytes=true], cacheId=-1206548976, txKey=IgniteTxKey 
> [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976], 
> val=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, 
> field1=0]], prevVal=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, 
> field2=name_0, field1=0]], entryProcessorsCol=null, ttl=-1, 
> conflictExpireTime=-1, conflictVer=null, explicitVer=null, 
> dhtVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200871, order=1458043167489], filters=[], 
> filtersPassed=false, filtersSet=true, entry=GridDhtColocatedCacheEntry 
> [super=GridDhtCacheEntry [rdrs=[], locPart=GridDhtLocalPartition [id=0, 
> mapPubSize=0, rmvQueue=GridCircularBuffer [sizeMask=255, idxGen=1], cntr=1, 
> state=OWNING, reservations=0, empty=true, createTime=03/15/2016 15:00:00, 
> mapPubSize=0], super=GridDistributedCacheEntry [super=GridCacheMapEntry 
> [key=KeyCacheObjectImpl [val=0, hasValBytes=true], val=null, 
> startVer=1458043167488, ver=GridCacheVersion [topVer=69523200, 
> nodeOrderDrId=1, globalTime=1458043200890, order=1458043167490], 
> hash=-1484017934, extras=GridCacheObsoleteEntryExtras 
> [obsoleteVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200890, order=1458043167491]], flags=7, prepared=false, 
> locked=true, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, locMapped=false, 
> expiryPlc=null, transferExpiryPlc=false, flags=2, partUpdateCntr=0, 
> serReadVer=null, xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200852, order=1458043167487]]}], super=IgniteTxAdapter 
> [xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200852, order=1458043167487], writeVer=GridCacheVersion 
> [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, 
> order=1458043167489], implicit=false, loc=true, threadId=1, 
> startTime=1458043200850, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, 
> startVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200852, order=1458043167487], endVer=null, 
> isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, timeout=0, 
> sysInvalidate=true, sys=false, plc=2, commitVer=GridCacheVersion 
> [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, 
> order=1458043167487], finalizing=NONE, preparing=false, invalidParts=null, 
> state=UNKNOWN, timedOut=false, topVer=AffinityTopologyVersion [topVer=1, 
> minorTopVer=1], duration=40ms, onePhaseCommit=true], size=1]]], err=class 
> o.a.i.i.transactions.IgniteTxHeuristicCheckedException: Failed to locally 
> write to cache (all transaction entries will be invalidated, however there 
> was a window when entries for this transaction were visible to others): 
> GridNearTxLocal [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, 
> colocatedLocallyMapped=true, needCheckBackup=null, hasRemoteLocks=false, 
> mappings=IgniteTxMappingsImpl [], super=GridDhtTxLocalAdapter 
> [nearOnOriginatingNode=false, nearNodes=[], dhtNodes=[],

[jira] [Created] (IGNITE-2858) Refactoring of creation of elements on domains page to using of mixins

2016-03-18 Thread Vasiliy Sisko (JIRA) 
Vasiliy Sisko created IGNITE-2858:
-

 Summary: Refactoring of creation of elements on domains page to 
using of mixins
 Key: IGNITE-2858
 URL: https://issues.apache.org/jira/browse/IGNITE-2858
 Project: Ignite
  Issue Type: Sub-task
  Components: wizards
Affects Versions: 1.6
Reporter: Vasiliy Sisko
Assignee: Vasiliy Sisko






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-2801) Coordinator floods network with partitions' full map messages

2016-03-18 Thread Denis Magda (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Magda updated IGNITE-2801:

Assignee: (was: Vladimir Ozerov)

> Coordinator floods network with partitions' full map messages
> -
>
> Key: IGNITE-2801
> URL: https://issues.apache.org/jira/browse/IGNITE-2801
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Denis Magda
>Priority: Critical
>  Labels: community, important
> Fix For: 1.6
>
> Attachments: basic_node.nps, basic_node.png, coordinator.nps, 
> coordinator.png
>
>
> It is detected that the more machines in the cluster we have and the more 
> caches are started then the more outgoing traffic is produced by a 
> coordinator node.
> As an example in the current deployment
> - 30 nodes;
> - 67 caches;
> - caches are empty and the cluster is not used at all (idle).
> the coordinator constantly uses 300 Mbit/s of outgoing traffic. In contrast 
> each other node shows constant 10 Mbit/s usage of incoming traffic.
> Most likely the reason is that the coordinator indefinitely sends partitions 
> full map for all the caches to all the nodes. This shouldn't happen.
> Need to debug the reason of the issue and fix it.
> Attached snapshots taken from the coordinator and on of cluster's nodes. 
> Probably they would help.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (IGNITE-2835) BinaryObjectOffHeapImpl leaked to public code

2016-03-18 Thread Artem Shutak (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Artem Shutak reassigned IGNITE-2835:


Assignee: Artem Shutak

> BinaryObjectOffHeapImpl leaked to public code
> -
>
> Key: IGNITE-2835
> URL: https://issues.apache.org/jira/browse/IGNITE-2835
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 1.5.0.final
>Reporter: Denis Magda
>Assignee: Artem Shutak
>Priority: Critical
>  Labels: community, important
> Fix For: 1.6
>
> Attachments: BinaryObjectOffHeapIssue.java
>
>
> To my knowledge {{BinaryObjectOffHeapImpl}} is considered to be used under 
> some internal lock only to prevent possible offheap pointer movement.
> However seems that we made it available to public code. If to start a 
> partitioned cache in {{OFFHEAP_TIRED}} mode, get {{BinaryObject}} from the 
> cache inside of a TX and put the same object back we will get exception like 
> below
> {noformat}
> [15:00:00,892][WARN ][main][GridNearTxLocal] Set transaction invalidation 
> flag to true due to error [tx=GridNearTxLocal [mappings=IgniteTxMappingsImpl 
> [], nearLocallyMapped=false, colocatedLocallyMapped=true, 
> needCheckBackup=null, hasRemoteLocks=false, mappings=IgniteTxMappingsImpl [], 
> super=GridDhtTxLocalAdapter [nearOnOriginatingNode=false, nearNodes=[], 
> dhtNodes=[], explicitLock=false, super=IgniteTxLocalAdapter 
> [completedBase=null, sndTransformedVals=false, depEnabled=false, 
> txState=IgniteTxStateImpl [activeCacheIds=GridLongList [idx=1, 
> arr=[-1206548976]], txMap={IgniteTxKey [key=KeyCacheObjectImpl [val=0, 
> hasValBytes=true], cacheId=-1206548976]=IgniteTxEntry [key=KeyCacheObjectImpl 
> [val=0, hasValBytes=true], cacheId=-1206548976, txKey=IgniteTxKey 
> [key=KeyCacheObjectImpl [val=0, hasValBytes=true], cacheId=-1206548976], 
> val=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, field2=name_0, 
> field1=0]], prevVal=[op=UPDATE, val=SomeType [idHash=1337835760, hash=0, 
> field2=name_0, field1=0]], entryProcessorsCol=null, ttl=-1, 
> conflictExpireTime=-1, conflictVer=null, explicitVer=null, 
> dhtVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200871, order=1458043167489], filters=[], 
> filtersPassed=false, filtersSet=true, entry=GridDhtColocatedCacheEntry 
> [super=GridDhtCacheEntry [rdrs=[], locPart=GridDhtLocalPartition [id=0, 
> mapPubSize=0, rmvQueue=GridCircularBuffer [sizeMask=255, idxGen=1], cntr=1, 
> state=OWNING, reservations=0, empty=true, createTime=03/15/2016 15:00:00, 
> mapPubSize=0], super=GridDistributedCacheEntry [super=GridCacheMapEntry 
> [key=KeyCacheObjectImpl [val=0, hasValBytes=true], val=null, 
> startVer=1458043167488, ver=GridCacheVersion [topVer=69523200, 
> nodeOrderDrId=1, globalTime=1458043200890, order=1458043167490], 
> hash=-1484017934, extras=GridCacheObsoleteEntryExtras 
> [obsoleteVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200890, order=1458043167491]], flags=7, prepared=false, 
> locked=true, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, locMapped=false, 
> expiryPlc=null, transferExpiryPlc=false, flags=2, partUpdateCntr=0, 
> serReadVer=null, xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200852, order=1458043167487]]}], super=IgniteTxAdapter 
> [xidVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200852, order=1458043167487], writeVer=GridCacheVersion 
> [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200871, 
> order=1458043167489], implicit=false, loc=true, threadId=1, 
> startTime=1458043200850, nodeId=993f5733-b014-4a5b-a6d1-934aeec9e9f5, 
> startVer=GridCacheVersion [topVer=69523200, nodeOrderDrId=1, 
> globalTime=1458043200852, order=1458043167487], endVer=null, 
> isolation=REPEATABLE_READ, concurrency=PESSIMISTIC, timeout=0, 
> sysInvalidate=true, sys=false, plc=2, commitVer=GridCacheVersion 
> [topVer=69523200, nodeOrderDrId=1, globalTime=1458043200852, 
> order=1458043167487], finalizing=NONE, preparing=false, invalidParts=null, 
> state=UNKNOWN, timedOut=false, topVer=AffinityTopologyVersion [topVer=1, 
> minorTopVer=1], duration=40ms, onePhaseCommit=true], size=1]]], err=class 
> o.a.i.i.transactions.IgniteTxHeuristicCheckedException: Failed to locally 
> write to cache (all transaction entries will be invalidated, however there 
> was a window when entries for this transaction were visible to others): 
> GridNearTxLocal [mappings=IgniteTxMappingsImpl [], nearLocallyMapped=false, 
> colocatedLocallyMapped=true, needCheckBackup=null, hasRemoteLocks=false, 
> mappings=IgniteTxMappingsImpl [], super=GridDhtTxLocalAdapter 
> [nearOnOriginatingNode=false, nearNodes=[], dhtNodes=[], explicitLock=false, 
> super=IgniteTxLocalAdapter

[jira] [Updated] (IGNITE-2747) ODBC: Time cast returns wrong results for Linux.

2016-03-18 Thread Igor Sapego (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Sapego updated IGNITE-2747:

Assignee: Vladimir Ozerov  (was: Igor Sapego)

> ODBC: Time cast returns wrong results for Linux.
> 
>
> Key: IGNITE-2747
> URL: https://issues.apache.org/jira/browse/IGNITE-2747
> Project: Ignite
>  Issue Type: Sub-task
>  Components: odbc
>Affects Versions: 1.5.0.final
>Reporter: Igor Sapego
>Assignee: Vladimir Ozerov
>Priority: Critical
> Fix For: 1.6
>
>
> Current time cast does not work for Linux because {{timezone}} variable does 
> not return right time offset for Linux.
> Also, {{gmtime}} is not thread-safe so it seems that we should use some 
> platform-specific functions for time-conversion operations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2849) BinaryObjectBuilder doesn't properly check metadata

2016-03-18 Thread Denis Magda (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201490#comment-15201490
 ] 

Denis Magda commented on IGNITE-2849:
-

Fixed according to the description. Checked with TC. Ready for review.

> BinaryObjectBuilder doesn't properly check metadata
> ---
>
> Key: IGNITE-2849
> URL: https://issues.apache.org/jira/browse/IGNITE-2849
> Project: Ignite
>  Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Denis Magda
>Priority: Critical
>  Labels: community, important
>
> There are several cases when {{BinaryObjectBuilder}} doesn't properly check 
> fields metadata when {{build}} method is called.
> 1) Set {{builder.setField("name", null).build()}} and 
> {{builder.setField("name", new Date()).build()}} won't check metadata 
> allowing to serialize the object;
> 2) Metadata is not checked at all if new BinaryObject is assembled from the 
> previous one {{binaries.builder(someBinaryObject).setField().build()}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-2809) Optimize IGFS performance.

2016-03-18 Thread Vladimir Ozerov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-2809:

Assignee: (was: Ivan Veselovsky)

> Optimize IGFS performance.
> --
>
> Key: IGNITE-2809
> URL: https://issues.apache.org/jira/browse/IGNITE-2809
> Project: Ignite
>  Issue Type: Task
>  Components: IGFS
>Affects Versions: 1.5.0.final
>Reporter: Vladimir Ozerov
>Priority: Critical
> Fix For: 1.7
>
>
> This is the umbrella ticket to host proposed IGFS performance improvements. 
> Currently IGFS is struggling from some inefficiencies. It moves lots of 
> unnecesary data over network. It has points of high contention - root and 
> trash directories. It has less than efficient default cache properties, etc..
> We need to take care about it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-1837) Rebalancing on a big cluster (30 nodes and more)

2016-03-18 Thread Denis Magda (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denis Magda updated IGNITE-1837:

Summary: Rebalancing on a big cluster (30 nodes and more)  (was: 
Rebalancing on a big cluster (64 nodes and more))

> Rebalancing on a big cluster (30 nodes and more)
> 
>
> Key: IGNITE-1837
> URL: https://issues.apache.org/jira/browse/IGNITE-1837
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: ignite-1.4
>Reporter: Denis Magda
>Assignee: Alexey Goncharuk
> Fix For: 1.6
>
>
> It seems that Ignite has different rebalancing related issues that appear 
> when a big cluster is started.
> Under the big cluster I mean:
> - cluster of 64 server nodes;
> - cluster of 64 server and 64 client nodes.
> The issues can be divided on three main use cases.
> 1) Slow rebalancing on start.
> - If to set partitions number for some cache to value bigger than default one 
> (to 3200 or to 6400, etc.) then rebalancing of such caches may take several 
> minutes. The caches are empty at that time. In addition, as a part of this 
> issue let's document that the number of partitions can't exceed some value.
> - exchange message on NODE_JOINED event that times out for a long time. 
> Discussed there: 
> http://apache-ignite-users.70518.x6.nabble.com/Help-with-tuning-for-larger-clusters-td1692.html#a1813
> 2) Slow rebalancing on client nodes shutdown.
> If to stop a significant number of client nodes at the same time then again 
> by some reason the rebalancing will take serveral minutes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-1743) IGFS: Use async cache put instead of block/ack messages on data write

2016-03-18 Thread Vladimir Ozerov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-1743:

Issue Type: Task  (was: Sub-task)
Parent: (was: IGNITE-1697)

> IGFS: Use async cache put instead of block/ack messages on data write
> -
>
> Key: IGNITE-1743
> URL: https://issues.apache.org/jira/browse/IGNITE-1743
> Project: Ignite
>  Issue Type: Task
>Reporter: Ivan Veselovsky
>Assignee: Ivan Veselovsky
> Fix For: 1.7
>
>
> Item "1)" from IGNITE-1697



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (IGNITE-2820) IGFS: Ensure that all participating IDs are locked right after TX start.

2016-03-18 Thread Vladimir Ozerov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov reassigned IGNITE-2820:
---

Assignee: Vladimir Ozerov

> IGFS: Ensure that all participating IDs are locked right after TX start.
> 
>
> Key: IGNITE-2820
> URL: https://issues.apache.org/jira/browse/IGNITE-2820
> Project: Ignite
>  Issue Type: Task
>  Components: IGFS
>Affects Versions: 1.5.0.final
>Reporter: Vladimir Ozerov
>Assignee: Vladimir Ozerov
>Priority: Critical
> Fix For: 1.7
>
>
> *Problem*
> Sometimes we have to create new entries during some operation on metadata. If 
> we do not lock this ID along with other participating IDs right after TX 
> start, subsequent cache operation on this ID will result in network calls to 
> block this key on other nodes.
> *Solution*
> Create potentially new key before entering TX and lock it with other keys. 
> This should decrease number of network calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2835) BinaryObjectOffHeapImpl leaked to public code

2016-03-18 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201914#comment-15201914
 ] 

ASF GitHub Bot commented on IGNITE-2835:


GitHub user ashutakGG opened a pull request:

https://github.com/apache/ignite/pull/567

IGNITE-2835 Binary object off heap impl leak on public API

Fix.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ashutakGG/incubator-ignite 
ignite-2835-BinaryObjectOffHeapImpl

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/567.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #567


commit 2e64d0d7cc51552fffc231cbc850cd615076fb85
Author: vozerov-gridgain 
Date:   2015-12-29T06:31:58Z

IGNITE-2258: IGFS: now default path modes could be optionally disabled 
using FileSystemConfiguration.isInitializeDefaultPathModes() property.

commit 4cd3b3dc2f1fa0f1a9cceb6bf544dd8fb505d7f5
Author: vozerov-gridgain 
Date:   2015-12-29T09:52:00Z

IGNITE-2258: Fixed type on getter/setter.

commit 5d58fcbf40fdb9114e4cbb32b72dd9bce7fa38ca
Author: iveselovskiy 
Date:   2016-01-04T06:47:28Z

IGNITE-2308: Fixed HadoopClassLoader dependency resolution. This closes 
#391.

commit 83a19179cee2bb15adc36c2265dd0a3c794b60bb
Author: vozerov-gridgain 
Date:   2016-01-04T08:14:58Z

IGNITE-2218: Fixed a problem with native Hadoop libraries load. This closes 
#378.

commit 1d7fb5702fa33cf395e797161f3a86a9600921a7
Author: vozerov-gridgain 
Date:   2016-01-05T06:59:31Z

IGNITE-2206: Hadoop file system creation is now abstracted out using 
factory interface.

commit a12ec7d08573d5396654a5ba05bb7d873e4c2677
Author: Ignite Teamcity 
Date:   2016-01-06T10:50:48Z

1.5.2

commit 090a5de6a930c10a3a57a6e28c486fe5c87e028d
Author: vozerov-gridgain 
Date:   2015-12-29T12:50:39Z

Minor fix.

commit c786820dda7f7cd1849c5593ac24ca9072945887
Author: vozerov-gridgain 
Date:   2016-01-07T13:48:14Z

IgniteHadoopIgfsSecondaryFileSystem.usrName field is renamed to "userName" 
to preserve backward compatibility.

commit 6ab4ce246316442fa4295f9941c372fea70c24ef
Author: vozerov-gridgain 
Date:   2016-01-08T06:23:55Z

IGNITE-2342: Set correct classlader before calling FileSystem.get().

commit 077ab1b3a77fdb1c2c2fd6360fc5b60fda6c50c3
Author: vozerov-gridgain 
Date:   2016-01-08T07:17:45Z

IGNITE-2341: Improved warning message when BinaryMarshaller cannot be used. 
Also it is not shown when "org.apache.ignite" classes is in described situation.

commit 86c4816edfd0e983014f136ffc92cde28ec56cca
Author: vozerov-gridgain 
Date:   2016-01-08T07:26:03Z

IGNITE-2340: Improved error thrown when PROXY mode exists, but secondary 
file system is not IgniteHadoopIgfsSecondaryFileSystem.

commit fc48a8429a84ef6c87ed3225a218d7d3ae617e14
Author: vozerov-gridgain 
Date:   2016-01-08T07:48:42Z

Merge branch 'ignite-1.5.2' into ignite-1.5.3

commit 86740cefe212ed0f506d81056dd8e76de9a31e4f
Author: Ignite Teamcity 
Date:   2016-01-08T09:32:11Z

1.5.3-SNAPSHOT

commit 92229d2a6c6ef86772a62cb52b3aa07a55c99d89
Author: sboikov 
Date:   2016-01-13T05:56:34Z

ignite-2359 Added locking for files used by MarshallerContextImpl. (cherry 
picked from commit 1d8c4e2)

commit 2e4ce585d5f54502c6511d3425b1cd5fbf0a7f4f
Author: Ignite Teamcity 
Date:   2016-01-13T10:37:33Z

1.5.4-SNAPSHOT

commit 6e5f9f0c7d4c86773b1f0cd5c5a673acb58c86c2
Author: Denis Magda 
Date:   2016-01-13T11:42:27Z

Changed year to 2016 in Copyrights

commit 02dbcfd8ed2701a4f415c8871d0b8fd08bfa0583
Author: Alexey Goncharuk 
Date:   2016-01-13T13:47:32Z

IGNITE-2365 - Notify policy if swap or offheap is enabled and rebalanced 
entry was not preloaded.
IGNITE-2099 - Fixing custom collections.
This closes #396

commit 86c2ba2a601e82b824cf17422683e5398a4d8c7d
Author: sboikov 
Date:   2016-01-13T15:40:08Z

ignite-2350 Pass update notifier flag in discovery data (all cluster nodes 
will have the same notifier status as first cluster node)
(cherry picked from commit 7175a42)

commit e1a494df400fc37ca04e8d88d1cf20bca02607b4
Author: sboikov 
Date:   2016-01-14T11:16:33Z

Renamed fields to change fields write order (to preserve backward 
compatibility).
(cherry picked from commit 2a4adf5)

commit 09f978234b6062afa1e1658d5a6439365a856aca
Author: sboikov 
Date:

[jira] [Commented] (IGNITE-2797) Prepare and finish future never time out

2016-03-18 Thread Andrey Gura (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201441#comment-15201441
 ] 

Andrey Gura commented on IGNITE-2797:
-

Found on more place where lock can be created without timeout despite 
transaction has timeout. It is {{IgniteTxLocalAdapter.putAsync0()}} method.

May be we should throw {{IgniteTxTimeoutCheckedException}} from all places 
where {{remainingTime()}} or {{lockTimeout()}} methods invoke.

> Prepare and finish future never time out
> 
>
> Key: IGNITE-2797
> URL: https://issues.apache.org/jira/browse/IGNITE-2797
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Valentin Kulichenko
>Priority: Blocker
>  Labels: community, customer, important
> Fix For: 1.6
>
> Attachments: TxTest2.java
>
>
> Even if transaction timeout is configured, transaction will not timeout if 
> it's already in prepare state. It will be shown in log as pending transaction 
> and can cause the whole cluster hang.
> We need to add a mechanism that will properly timeout prepare and (if 
> possible) finish futures.
> Also we can create an event that will be fired if there is a transaction 
> pending for a long time, showing which nodes we are waiting responses from. 
> This will allow user to recover by stopping only these nodes instead of 
> restarting the whole cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (IGNITE-2816) IGFS: IgfsMetaManager should not use "put" to update parent listing.

2016-03-18 Thread Vladimir Ozerov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-2816.
-
Resolution: Duplicate

> IGFS: IgfsMetaManager should not use "put" to update parent listing.
> 
>
> Key: IGNITE-2816
> URL: https://issues.apache.org/jira/browse/IGNITE-2816
> Project: Ignite
>  Issue Type: Task
>  Components: IGFS
>Affects Versions: 1.5.0.final
>Reporter: Vladimir Ozerov
>Assignee: Vladimir Ozerov
>Priority: Critical
> Fix For: 1.7
>
>
> Lightweight entry processor must be used instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (IGNITE-2797) Prepare and finish future never time out

2016-03-18 Thread Andrey Gura (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201441#comment-15201441
 ] 

Andrey Gura edited comment on IGNITE-2797 at 3/18/16 1:06 PM:
--

Found on more place where lock can be created without timeout despite 
transaction has timeout. It is {{IgniteTxLocalAdapter.putAsync0()}} method.

Fixed. Waiting for TC

May be we should throw {{IgniteTxTimeoutCheckedException}} from all places 
where {{remainingTime()}} or {{lockTimeout()}} methods invoke.


was (Author: agura):
Found on more place where lock can be created without timeout despite 
transaction has timeout. It is {{IgniteTxLocalAdapter.putAsync0()}} method.

May be we should throw {{IgniteTxTimeoutCheckedException}} from all places 
where {{remainingTime()}} or {{lockTimeout()}} methods invoke.

> Prepare and finish future never time out
> 
>
> Key: IGNITE-2797
> URL: https://issues.apache.org/jira/browse/IGNITE-2797
> Project: Ignite
>  Issue Type: Bug
>  Components: cache
>Affects Versions: 1.5.0.final
>Reporter: Valentin Kulichenko
>Priority: Blocker
>  Labels: community, customer, important
> Fix For: 1.6
>
> Attachments: TxTest2.java
>
>
> Even if transaction timeout is configured, transaction will not timeout if 
> it's already in prepare state. It will be shown in log as pending transaction 
> and can cause the whole cluster hang.
> We need to add a mechanism that will properly timeout prepare and (if 
> possible) finish futures.
> Also we can create an event that will be fired if there is a transaction 
> pending for a long time, showing which nodes we are waiting responses from. 
> This will allow user to recover by stopping only these nodes instead of 
> restarting the whole cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (IGNITE-2788) Redis API for Ignite to work with data via the Redis protocol

2016-03-18 Thread Roman Shtykh (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Shtykh reassigned IGNITE-2788:


Assignee: Roman Shtykh

> Redis API for Ignite to work with data via the Redis protocol
> -
>
> Key: IGNITE-2788
> URL: https://issues.apache.org/jira/browse/IGNITE-2788
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Roman Shtykh
>Assignee: Roman Shtykh
>
> Introduce Redis API that works with the Redis protocol but uses Ignite grid.
> Needless to say, Redis is an extremely popular caching solution. Such API 
> will enable smooth migration to Ignite.
> As a first phase we can start with most frequently used commands and enhance 
> gradually.
> Redis commands http://redis.io/commands



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Closed] (IGNITE-2557) ODBC: Add integrity tests.

2016-03-18 Thread Vladimir Ozerov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov closed IGNITE-2557.
---

> ODBC: Add integrity tests.
> --
>
> Key: IGNITE-2557
> URL: https://issues.apache.org/jira/browse/IGNITE-2557
> Project: Ignite
>  Issue Type: Sub-task
>  Components: odbc
>Affects Versions: 1.5.0.final
>Reporter: Igor Sapego
>Assignee: Vladimir Ozerov
> Fix For: 1.6
>
>
> Need to add integrity tests that would work through system API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2847) Failed atomic-offheap-invoke-retry load consistency test

2016-03-18 Thread Ilya Suntsov (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15199287#comment-15199287
 ] 

Ilya Suntsov commented on IGNITE-2847:
--

Today I got the same exceptions during the test with 1 backup.
I attached logs to the ticket.

> Failed atomic-offheap-invoke-retry load consistency test
> 
>
> Key: IGNITE-2847
> URL: https://issues.apache.org/jira/browse/IGNITE-2847
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 1.6
> Environment: Yardstick driver's host: 
> Ubuntu 14.04.3 LTS
> Yardstick server's hosts: Ubuntu 14.04.3 LTS and CentOS release 6.7 (Final)
>Reporter: Ilya Suntsov
>Assignee: Artem Shutak
>Priority: Critical
> Fix For: 1.6
>
> Attachments: logs_configs.zip
>
>
> I ran load test with 2 backups in client mode (1 client, 4 servers) on 3 
> hosts (host1 - client, hosts-2,3 - 2 data nodes on each) and got the 
> following exception after 5h of work of test:
> {noformat}
> <13:49:20> Got 
> exception: 
> org.apache.ignite.yardstick.cache.failover.IgniteConsistencyException: Cache 
> and local map are in inconsistent state [badKeys=[key-62687]]
> org.apache.ignite.yardstick.cache.failover.IgniteConsistencyException: Cache 
> and local map are in inconsistent state 
> [badKeys=[key-62687]]<13:49:20> Full thread 
> dump of the current node below.
> {noformat}
> Logs in attachment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (IGNITE-1433) .Net: Add IgniteException.JavaStackTrace

2016-03-18 Thread Vladimir Ozerov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov updated IGNITE-1433:

Fix Version/s: (was: 1.7)
   1.6

> .Net: Add IgniteException.JavaStackTrace
> 
>
> Key: IGNITE-1433
> URL: https://issues.apache.org/jira/browse/IGNITE-1433
> Project: Ignite
>  Issue Type: Task
>  Components: platforms
>Affects Versions: 1.1.4
>Reporter: Vladimir Ozerov
>Assignee: Vladimir Ozerov
> Fix For: 1.6
>
>
> Propagate java stack trace as a string in ExceptionUtils.GetException and 
> write it to a new field in IgniteException class.
> This will simplify debugging for us both locally and when getting error 
> reports from clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (IGNITE-2820) IGFS: Ensure that all participating IDs are locked right after TX start.

2016-03-18 Thread Vladimir Ozerov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/IGNITE-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Ozerov resolved IGNITE-2820.
-
Resolution: Fixed

Fixed as a part of IGNITE-2860.

> IGFS: Ensure that all participating IDs are locked right after TX start.
> 
>
> Key: IGNITE-2820
> URL: https://issues.apache.org/jira/browse/IGNITE-2820
> Project: Ignite
>  Issue Type: Task
>  Components: IGFS
>Affects Versions: 1.5.0.final
>Reporter: Vladimir Ozerov
>Priority: Critical
> Fix For: 1.7
>
>
> *Problem*
> Sometimes we have to create new entries during some operation on metadata. If 
> we do not lock this ID along with other participating IDs right after TX 
> start, subsequent cache operation on this ID will result in network calls to 
> block this key on other nodes.
> *Solution*
> Create potentially new key before entering TX and lock it with other keys. 
> This should decrease number of network calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (IGNITE-2862) Deployment Ignite in Mesos cluster is failed

2016-03-18 Thread Vasilisa Sidorova (JIRA) 

[ 
https://issues.apache.org/jira/browse/IGNITE-2862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201765#comment-15201765
 ] 

Vasilisa  Sidorova commented on IGNITE-2862:


NB: the screenshot in the step 5 of paragraph "RUN THE FRAMEWORK VIA MARATHON" 
is obsoleted. Please, update it. For example onto attached screenshot 
"mesos_logs.png"

> Deployment Ignite in Mesos cluster is failed
> 
>
> Key: IGNITE-2862
> URL: https://issues.apache.org/jira/browse/IGNITE-2862
> Project: Ignite
>  Issue Type: Bug
>  Components: general
>Affects Versions: 1.5.0.final
>Reporter: Vasilisa  Sidorova
> Attachments: marathon.json, mesos_activetasks.png, mesos_logs.png
>
>
> -
> DESCRIPTION
> -
> Deployment  Ignite in Mesos cluster by this instruction 
> https://apacheignite.readme.io/docs/mesos-deployment is failed 
> -
> STEPS FOR REPRODUCE
> -
> # Do items 1-5 from "RUN THE FRAMEWORK VIA MARATHON"   paragraph 
> -
> ACTUAL RESULT
> -
> # Ignite nodes didn't start. Only ignition task is running. Look at the 
> attached picture "mesos_activetasks.png"
> # Stderr log for ignition task get follow exception:
> {noformat}
> I0318 18:42:08.252142 17138 fetcher.cpp:409] Fetcher Info: 
> {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/20160318-180241-16777343-5050-16554-S0\/root","items":[{"action":"BYPASS_CACHE","uri":{"extract":false,"value":"https:\/\/s3.amazonaws.com\/vasilisk\/m\/ignite-mesos-1.5.0.final.jar"}}],"sandbox_directory":"\/tmp\/mesos\/slaves\/20160318-180241-16777343-5050-16554-S0\/frameworks\/20150603-121744-16842879-5050-6241-\/executors\/ignition.f8fa2435-ed1f-11e5-bea1-0242e00dbbdd\/runs\/776db62f-965f-4e9e-950a-4edb67c44667","user":"root"}
> I0318 18:42:08.253284 17138 fetcher.cpp:364] Fetching URI 
> 'https://s3.amazonaws.com/vasilisk/m/ignite-mesos-1.5.0.final.jar'
> I0318 18:42:08.253298 17138 fetcher.cpp:238] Fetching directly into the 
> sandbox directory
> I0318 18:42:08.253312 17138 fetcher.cpp:176] Fetching URI 
> 'https://s3.amazonaws.com/vasilisk/m/ignite-mesos-1.5.0.final.jar'
> I0318 18:42:08.253325 17138 fetcher.cpp:126] Downloading resource from 
> 'https://s3.amazonaws.com/vasilisk/m/ignite-mesos-1.5.0.final.jar' to 
> '/tmp/mesos/slaves/20160318-180241-16777343-5050-16554-S0/frameworks/20150603-121744-16842879-5050-6241-0000/executors/ignition.f8fa2435-ed1f-11e5-bea1-0242e00dbbdd/runs/776db62f-965f-4e9e-950a-4edb67c44667/ignite-mesos-1.5.0.final.jar'
> I0318 18:42:12.091784 17138 fetcher.cpp:441] Fetched 
> 'https://s3.amazonaws.com/vasilisk/m/ignite-mesos-1.5.0.final.jar' to 
> '/tmp/mesos/slaves/20160318-180241-16777343-5050-16554-S0/frameworks/20150603-121744-16842879-5050-6241-/executors/ignition.f8fa2435-ed1f-11e5-bea1-0242e00dbbdd/runs/776db62f-965f-4e9e-950a-4edb67c44667/ignite-mesos-1.5.0.final.jar'
> I0318 18:42:12.293658 17142 exec.cpp:132] Version: 0.23.0
> I0318 18:42:12.296212 17145 exec.cpp:206] Executor registered on slave 
> 20160318-180241-16777343-5050-16554-S0
> Mar 18, 2016 6:42:12 PM org.apache.ignite.mesos.IgniteFramework main
> INFO: Enabling checkpoint for the framework
> 2016-03-18 18:42:12.495:INFO::main: Logging initialized @151ms
> 2016-03-18 18:42:22.542:INFO:oejs.Server:main: jetty-9.2.z-SNAPSHOT
> 2016-03-18 18:42:22.579:INFO:oejs.ServerConnector:main: Started 
> ServerConnector@1268c278{HTTP/1.1}{172.17.0.1:48610}
> 2016-03-18 18:42:22.580:INFO:oejs.Server:main: Started @10235ms
> Exception in thread "main" java.lang.RuntimeException: Got unexpected 
> response code. Response code: 404
>   at 
> org.apache.ignite.mesos.resource.IgniteProvider.downloadIgnite(IgniteProvider.java:202)
>   at 
> org.apache.ignite.mesos.resource.IgniteProvider.getIgnite(IgniteProvider.java:132)
>   at 
> org.apache.ignite.mesos.resource.ResourceProvider.init(ResourceProvider.java:57)
>   at org.apache.ignite.mesos.IgniteFramework.main(IgniteFramework.java:77)
> {noformat}
> -
> EXPECTED RESULT
> -
> Tasks for all 3 nodes are running with ignition task. 
> -
> ADDITIONAL INFO
> -
> # File "marathon.json" is attached



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)