Re: [NOTICE] TeamCity is down
On Wed, Sep 23, 2015 at 10:20PM, Yakov Zhdanov wrote: > Cos, is it ok that none of the domain names mentioned in ticket responds? > Even those with "dig" output. Looks like our CI master is down as well ;( Will look at it, thanks! > > Created ticket for Ignite - > https://issues.apache.org/jira/browse/INFRA-10484 > > > --Yakov > > 2015-09-22 21:01 GMT+03:00 Konstantin Boudnik : > > > Something like this could be arranged with INFRA > > https://issues.apache.org/jira/browse/INFRA-10092 > > so there's an official CNAME for the CI server. > > > > Cos > > > > On Tue, Sep 22, 2015 at 11:31AM, Raul Kripalani wrote: > > > Hey guys, > > > > > > (Continuing "Maybe review my first commits" thread here). > > > > > > I updated the Wiki in several places to point to .153: > > > > > > - Frontpage > > > - Git process > > > - How to Contribute > > > > > > Are there any plans to create a hostname for TeamCity? Else we can > > create a > > > page at http://ignite.apache.org/teamcity that redirects to the right > > place. > > > > > > Thanks, > > > > > > *Raúl Kripalani* > > > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and > > > Messaging Engineer > > > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani > > > http://blog.raulkr.net | twitter: @raulvk > > > > > > On Tue, Sep 22, 2015 at 6:05 AM, Denis Magda > > wrote: > > > > > > > Hi Raul, > > > > > > > > That server got down almost a month ago. > > > > There is a secondary one - http://204.14.53.153/ > > > > > > > > Are we going to recover .152 server? If we don't then we have to update > > > > our wiki and refer to .153. > > > > > > > > -- > > > > Denis > > > > > > > > > > > > On 9/22/2015 12:25 AM, Raúl Kripalani wrote: > > > > > > > >> I'm trying to access TeamCity via the URL in the Wiki ( > > > >> http://204.14.53.152/) > > > >> but it seems to be down... > > > >> > > > >> http://downforeveryoneorjustme.com/http://204.14.53.152/ > > > >> > > > >> Could someone please check? > > > >> > > > >> Thanks, > > > >> Raúl. > > > >> > > > >> > > > > > >
[jira] [Created] (IGNITE-1547) NPE if using HTTP REST
Pavel Konstantinov created IGNITE-1547:
--
Summary: NPE if using HTTP REST
Key: IGNITE-1547
URL: https://issues.apache.org/jira/browse/IGNITE-1547
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: ignite-1.4
Reporter: Pavel Konstantinov
Assignee: Yakov Zhdanov
Fix For: ignite-1.5
I'm tested Ignite Console SQL feature and got NPE on node start if web agent
trying to get cache information at this moment
{code}
[13:25:56,761][SEVERE][qtp79785210-70][GridJettyRestProtocol] Failed to process
HTTP request [action=/ignite, req=(GET
/ignite?cmd=top&attr=true&mtr=false)@1215478528
org.eclipse.jetty.server.Request@
4872bb00]
class org.apache.ignite.IgniteCheckedException: null
at
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:6979)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.ignite.internal.processors.rest.handlers.top.GridTopologyCommandHandler.createNodeBean(GridTopologyCommandHandler.java:208)
at
org.apache.ignite.internal.processors.rest.handlers.top.GridTopologyCommandHandler.handleAsync(GridTopologyCommandHandler.java:107)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest(GridRestProcessor.java:226)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor.access$100(GridRestProcessor.java:79)
at
org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:133)
... 4 more
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-1546) Rework XML for DB2
Pavel Konstantinov created IGNITE-1546: -- Summary: Rework XML for DB2 Key: IGNITE-1546 URL: https://issues.apache.org/jira/browse/IGNITE-1546 Project: Ignite Issue Type: Sub-task Components: wizards Reporter: Pavel Konstantinov Assignee: Vasiliy Sisko Priority: Trivial Bean which we generates now is incorrect - node cannot start with it I found another notation we should implement: -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (IGNITE-1545) Java8 MessagingExample should be optimized
Vasilisa Sidorova created IGNITE-1545:
--
Summary: Java8 MessagingExample should be optimized
Key: IGNITE-1545
URL: https://issues.apache.org/jira/browse/IGNITE-1545
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: ignite-1.4
Environment: Ubuntu 14.04, community build 1.4.0 #305
Reporter: Vasilisa Sidorova
Priority: Trivial
-
DESCRIPTION
-
When Java8 MessagingExample start without remote nodes it's get "Please start
at least 2 cluster nodes to run example." twice. There is 3 needless
system.out.println as the same print is in checkMinTopologySize method
-
STEPS FOR REPRODUCE
-
1. Run Java8 MessagingExample
-
ACTUAL RESULT
-
{noformat}
>>> Please start at least 2 cluster nodes to run example.
>>> Please start at least 2 cluster nodes to run example.
{noformat}
-
EXPECTED RESULT
-
{noformat}
>>> Please start at least 2 cluster nodes to run example.
{noformat}
like for all anothers Java7 and Java8 Messaging Examples
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-1544) Make sure objects number stored in a cache can be bigger than Integer.MAX_INT
Denis Magda created IGNITE-1544:
---
Summary: Make sure objects number stored in a cache can be bigger
than Integer.MAX_INT
Key: IGNITE-1544
URL: https://issues.apache.org/jira/browse/IGNITE-1544
Project: Ignite
Issue Type: Bug
Components: cache
Affects Versions: ignite-1.4
Reporter: Denis Magda
Assignee: Yakov Zhdanov
Priority: Blocker
Fix For: ignite-1.5
Make sure that a cache (partitioned) can actually hold number of objects bigger
than Integer.MAX_INT. Because Integer.MAX_INT is not enough for some life use
cases and business domains.
At least the following should be done:
- IgniteCache must return cache size as a {{long}} value;
- Make sure (make a test) that a partitioned cache can store number of objects
much bigger than Integer.MAX_INT;
- find and edit info regarding this limitation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-1543) SqlQuery returns empty result set when custome PortableIdMapper
Denis Magda created IGNITE-1543:
---
Summary: SqlQuery returns empty result set when custome
PortableIdMapper
Key: IGNITE-1543
URL: https://issues.apache.org/jira/browse/IGNITE-1543
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: ignite-1.4
Reporter: Denis Magda
Priority: Critical
Fix For: ignite-1.5
Set custom {{PortableIdMapper}} for a type. Let this mapper to return some
predefined id for this type (like 12345).
Then execute SQL to get all the entries of this type. The query will return an
empty result set.
Seems that at some point of execution SQL engine or some other part of the
platform uses default type to id mapping ignoring given {{PotableIdMapper}}.
To reproduce:
- Start a server node with portable marshaller enabled;
- run the code attached (ClientTest.java).
Add a test to our test suites.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (IGNITE-1542) MetaDataEnabled flag is partially ignored when set in PortableTypeConfiguration
Denis Magda created IGNITE-1542:
---
Summary: MetaDataEnabled flag is partially ignored when set in
PortableTypeConfiguration
Key: IGNITE-1542
URL: https://issues.apache.org/jira/browse/IGNITE-1542
Project: Ignite
Issue Type: Bug
Components: general
Affects Versions: ignite-1.4
Reporter: Denis Magda
Assignee: Denis Magda
Priority: Critical
Fix For: ignite-1.5
Define a type using {{PortableTypeConfiguration}} and disable meta data for the
type by setting {{metaDataEnabled}} flag to false.
Then set this portable type configuration in {{PortableMarshaller's}}
configuration list and start a node.
Despite of the fact that the metadata is disable for the type it will be added
to the metadata handler.
The fix is trivial.
Change last line of {{PortableContext.registerUserType()}} to:
{noformat}
if (metadataenabled)
metaHnd.addMeta(id, new PortableMetaDataImpl(typeName, fieldsMeta,
affKeyFieldName));
{noformat}
Add tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] ignite pull request: Ignite 1515 b
GitHub user iveselovskiy opened a pull request: https://github.com/apache/ignite/pull/105 Ignite 1515 b https://issues.apache.org/jira/browse/IGNITE-1515 You can merge this pull request into a Git repository by running: $ git pull https://github.com/iveselovskiy/ignite ignite-1515-b Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/105.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #105 commit 61859f90130c95f4bb9f4bd721f70e9b067068ef Author: iveselovskiy Date: 2015-09-24T13:24:59Z 1515: workable version, needs cleanup. commit 7cad4c35bd301d53544ad35a32e2cc7bd1d473e7 Author: iveselovskiy Date: 2015-09-24T14:14:16Z ignite-1515: cleanup. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (IGNITE-1541) IGFS: mkdirs operation should be truly thread-safe.
Ivan Veselovsky created IGNITE-1541: --- Summary: IGFS: mkdirs operation should be truly thread-safe. Key: IGNITE-1541 URL: https://issues.apache.org/jira/browse/IGNITE-1541 Project: Ignite Issue Type: Bug Reporter: Ivan Veselovsky Assignee: Ivan Veselovsky In IGNITE-1515 we made delete thread-safe. But experience shows that concurrent delete + mkdirs operations nevertheless cause file system structure corruption, see test org.apache.ignite.internal.processors.igfs.IgfsAbstractSelfTest#testDeadlocksDeleteMkdirs . -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: After setting streamer and ignite,getting NULL
Chandresh, As far as understand IBolt implementation should setup all external connections at .prepare() method. So, better way is to get existing Ignite instance or create new at .prepare() method. On Thu, Sep 24, 2015 at 10:55 AM, Gianfranco Murador < murador.gianfra...@gmail.com> wrote: > Chandresh, > I will launch the unit test later this week , if the problem persists. I > think that you should instantiate Ignite through the start() method of > Ignition class > and retrieve the Ignite cache specifying a name for the default cache. You > should provide a name also for data streamer, not passing null. > Could you do this and run again the unit test ? > Thank you, > Regards, > Gianfranco > > > 2015-09-24 5:40 GMT+02:00 chandresh pancholi < > chandreshpancholi...@gmail.com > >: > > > Anton, > > > > Clone this repo https://github.com/chandresh-pancholi/ignite and run > this > > class > > > > > https://github.com/chandresh-pancholi/ignite/blob/master/modules/storm/src/test/java/org/apache/ignite/stream/storm/IgniteStormStreamerSelfTestSuite.java > > > > You will get the Error saying "Oouch,Argument is Null" for getStreamer(). > > > > On Wed, Sep 23, 2015 at 9:46 PM, Anton Vinogradov < > > avinogra...@gridgain.com> > > wrote: > > > > > Chandresh, > > > I'm trying to analize problem. > > > Could you please provide more details and explain step-by-step how can > I > > > reproduce problem? > > > > > > On Tue, Sep 22, 2015 at 11:06 PM, chandresh pancholi < > > > chandreshpancholi...@gmail.com> wrote: > > > > > > > Not yet. > > > > Still doing brainstorming why StormStreamer object is getting NULLL > > > > in-spite setting up ignite and streamer instances. > > > > > > > > On Tue, Sep 22, 2015 at 7:22 PM, Vishal Garg > > wrote: > > > > > > > > > Did you figure it out? Something related to your storm integration? > > > > > Vishal > > > > > > > > > > Sent from my iPhone > > > > > > > > > > > On Sep 22, 2015, at 4:50 AM, chandresh pancholi < > > > > > chandreshpancholi...@gmail.com> wrote: > > > > > > > > > > > > Problem here is with Storm. Its making streamer object null once > it > > > > goes > > > > > > into submit topology. > > > > > > > > > > > > On Mon, Sep 21, 2015 at 7:13 PM, Lalit Kumar Jha < > > > lalitj@gmail.com > > > > > > > > > > > wrote: > > > > > > > > > > > >> Hi Chandresh, > > > > > >> > > > > > >> See test suite class here, its based on annotations > > > > > >> > > > > > >> > > > > > > > > > > > > > > > https://github.com/sylentprayer/ignite/blob/ignite-530/modules/twitter/src/test/java/org/apache/ignite/stream/twitter/IgniteTwitterStreamerTestSuite.java > > > > > >> > > > > > >> No code required in class body. > > > > > >> > > > > > >> On Mon, Sep 21, 2015 at 6:24 PM, chandresh pancholi < > > > > > >> chandreshpancholi...@gmail.com> wrote: > > > > > >> > > > > > >>> Hi Gian/Vishal/Lalit, > > > > > >>> > > > > > >>> You guys have worked on Streamer integration with Ignite. I am > > > > working > > > > > on > > > > > >>> Storm-ignite integration. > > > > > >>> > > > > > >>> I am setting up setStreamer() and setIgnite() in Test class and > > > > sending > > > > > >>> this object to setBolt() method. > > > > > >>> > > > > > >>> When i try to read via gettreamer() and getIgnite() i am > getting > > > > NULL. > > > > > I > > > > > >> am > > > > > >>> trying to find out why is it happening but No Luck. > > > > > >>> > > > > > >>> If any one of you find some time and look into the code and > help > > me > > > > to > > > > > >>> resolve this. It would be great. > > > > > >>> > > > > > >>> Git : https://github.com/chandresh-pancholi/ignite > > > > > >>> Module : storm > > > > > >>> > > > > > >>> -- > > > > > >>> Chandresh Pancholi > > > > > >>> Senior Software Engineer > > > > > >>> Flipkart.com > > > > > >>> Email-id:chandresh.panch...@flipkart.com > > > > > >>> Contact:08951803660 > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Chandresh Pancholi > > > > > > Senior Software Engineer > > > > > > Flipkart.com > > > > > > Email-id:chandresh.panch...@flipkart.com > > > > > > Contact:08951803660 > > > > > > > > > > > > > > > > > > > > > -- > > > > Chandresh Pancholi > > > > Senior Software Engineer > > > > Flipkart.com > > > > Email-id:chandresh.panch...@flipkart.com > > > > Contact:08951803660 > > > > > > > > > > > > > > > -- > > Chandresh Pancholi > > Senior Software Engineer > > Flipkart.com > > Email-id:chandresh.panch...@flipkart.com > > Contact:08951803660 > > >
Re: Pull Request Merge Process
Hello, I've made initial review. Comments was added to issue, please have a look. On Tue, Sep 22, 2015 at 10:15 AM, Lalit Kumar Jha wrote: > I submitted pull request for IGNITE-530 (Twitter streamer). > > What is the proceess for reviewing and merging? >
[jira] [Created] (IGNITE-1540) Unexpected cache value in IgniteCacheSslStartStopSelfTest.testInvokeAll
Semen Boikov created IGNITE-1540:
Summary: Unexpected cache value in
IgniteCacheSslStartStopSelfTest.testInvokeAll
Key: IGNITE-1540
URL: https://issues.apache.org/jira/browse/IGNITE-1540
Project: Ignite
Issue Type: Sub-task
Components: cache
Reporter: Semen Boikov
Assignee: Semen Boikov
Priority: Critical
Fix For: ignite-1.5
Test IgniteCacheSslStartStopSelfTest.testInvokeAll sometimes fails on TC
(cache.get returns old value after update):
{noformat}
unit.framework.AssertionFailedError: expected:<30> but was:<29>
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.failNotEquals(Assert.java:329)
at junit.framework.Assert.assertEquals(Assert.java:78)
at junit.framework.Assert.assertEquals(Assert.java:86)
at junit.framework.TestCase.assertEquals(TestCase.java:253)
at
org.apache.ignite.internal.processors.cache.distributed.dht.IgniteCachePutRetryAbstractSelfTest.checkRetry(IgniteCachePutRetryAbstractSelfTest.java:403)
at
org.apache.ignite.internal.processors.cache.distributed.dht.IgniteCachePutRetryAbstractSelfTest.testInvokeAll(IgniteCachePutRetryAbstractSelfTest.java:214)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: IGFS concurrency issue
This is how we fixed it for now. But it requires paths locking starting from the root. Otherwise they can become parent-child in a moment after sucessfull check. On Thu, Sep 24, 2015 at 11:28 AM, Sergi Vladykin wrote: > May be just check that they are not parent-child within the tx? > > Sergi > Igniters, > > We revealed concurrency problem in IGFS and I would like to discuss > possible solutions to it. > > Consider the following file system structure: > root > |-- A > | |-- B > | | |-- C > | |-- D > > ... two concurrent operations in different threads: > T1: move(/A/B, /A/D); > T2: move(/A/D, /A/B/C); > > ... and how IGFS handles it now: > T1: verify that "/A/B" and "/A/D" exist, they are not child-parent to each > other, etc. -> OK. > T2: do the same for "A/D" and "A/B/C" -> OK. > T1: get IDs of "/A", "/A/B" and "/A/D" to lock them later inside tx. > T2: get IDs of "/A", "/A/D", "/A/B" and "/A/B/C" to lock them later inside > tx. > > T1: Start pessimistic tx, lock IDs of "/A", "/A/B", "/A/D", perform move -> > OK. > root > |-- A > | |-- D > | | |-- B > | | | |-- C > > T2: Start pessimistic tx, lock IDs of "/A", "/A/D", "/A/B" and > "/A/B/C" (*directory > structure already changed at this time!*), perform move -> OK. > root > |-- A > B > |-- D > | |-- C > | | |-- B (loop!) > > File system is corrupted. Folders B, C and D are not reacheable from root. > > To fix this now we additionaly check if directory structure is still > valid *inside > transaction*. It works, no more corruptions. But it requres taking locks on > the whole paths *including root*. So move, delete and mkdirs opeartions > *can > no longer be concurrent*. > > Probably there is a way to relax this while still ensuring consistency, but > I do not see how. One idea is to store real path inside each entry. This > way we will be able to ensure that it is still at a valid location without > blocking parents, so concurrnecy will be restored. But we will have to > propagate strucutral changes to children. E.g. move of a folder with 100 > items will lead to update of >100 cache entries. Not so good. > > Any other ideas? > > Vladimir. >
Re: IGFS concurrency issue
May be just check that they are not parent-child within the tx? Sergi Igniters, We revealed concurrency problem in IGFS and I would like to discuss possible solutions to it. Consider the following file system structure: root |-- A | |-- B | | |-- C | |-- D ... two concurrent operations in different threads: T1: move(/A/B, /A/D); T2: move(/A/D, /A/B/C); ... and how IGFS handles it now: T1: verify that "/A/B" and "/A/D" exist, they are not child-parent to each other, etc. -> OK. T2: do the same for "A/D" and "A/B/C" -> OK. T1: get IDs of "/A", "/A/B" and "/A/D" to lock them later inside tx. T2: get IDs of "/A", "/A/D", "/A/B" and "/A/B/C" to lock them later inside tx. T1: Start pessimistic tx, lock IDs of "/A", "/A/B", "/A/D", perform move -> OK. root |-- A | |-- D | | |-- B | | | |-- C T2: Start pessimistic tx, lock IDs of "/A", "/A/D", "/A/B" and "/A/B/C" (*directory structure already changed at this time!*), perform move -> OK. root |-- A B |-- D | |-- C | | |-- B (loop!) File system is corrupted. Folders B, C and D are not reacheable from root. To fix this now we additionaly check if directory structure is still valid *inside transaction*. It works, no more corruptions. But it requres taking locks on the whole paths *including root*. So move, delete and mkdirs opeartions *can no longer be concurrent*. Probably there is a way to relax this while still ensuring consistency, but I do not see how. One idea is to store real path inside each entry. This way we will be able to ensure that it is still at a valid location without blocking parents, so concurrnecy will be restored. But we will have to propagate strucutral changes to children. E.g. move of a folder with 100 items will lead to update of >100 cache entries. Not so good. Any other ideas? Vladimir.
Re: After setting streamer and ignite,getting NULL
Chandresh, I will launch the unit test later this week , if the problem persists. I think that you should instantiate Ignite through the start() method of Ignition class and retrieve the Ignite cache specifying a name for the default cache. You should provide a name also for data streamer, not passing null. Could you do this and run again the unit test ? Thank you, Regards, Gianfranco 2015-09-24 5:40 GMT+02:00 chandresh pancholi : > Anton, > > Clone this repo https://github.com/chandresh-pancholi/ignite and run this > class > > https://github.com/chandresh-pancholi/ignite/blob/master/modules/storm/src/test/java/org/apache/ignite/stream/storm/IgniteStormStreamerSelfTestSuite.java > > You will get the Error saying "Oouch,Argument is Null" for getStreamer(). > > On Wed, Sep 23, 2015 at 9:46 PM, Anton Vinogradov < > avinogra...@gridgain.com> > wrote: > > > Chandresh, > > I'm trying to analize problem. > > Could you please provide more details and explain step-by-step how can I > > reproduce problem? > > > > On Tue, Sep 22, 2015 at 11:06 PM, chandresh pancholi < > > chandreshpancholi...@gmail.com> wrote: > > > > > Not yet. > > > Still doing brainstorming why StormStreamer object is getting NULLL > > > in-spite setting up ignite and streamer instances. > > > > > > On Tue, Sep 22, 2015 at 7:22 PM, Vishal Garg > wrote: > > > > > > > Did you figure it out? Something related to your storm integration? > > > > Vishal > > > > > > > > Sent from my iPhone > > > > > > > > > On Sep 22, 2015, at 4:50 AM, chandresh pancholi < > > > > chandreshpancholi...@gmail.com> wrote: > > > > > > > > > > Problem here is with Storm. Its making streamer object null once it > > > goes > > > > > into submit topology. > > > > > > > > > > On Mon, Sep 21, 2015 at 7:13 PM, Lalit Kumar Jha < > > lalitj@gmail.com > > > > > > > > > wrote: > > > > > > > > > >> Hi Chandresh, > > > > >> > > > > >> See test suite class here, its based on annotations > > > > >> > > > > >> > > > > > > > > > > https://github.com/sylentprayer/ignite/blob/ignite-530/modules/twitter/src/test/java/org/apache/ignite/stream/twitter/IgniteTwitterStreamerTestSuite.java > > > > >> > > > > >> No code required in class body. > > > > >> > > > > >> On Mon, Sep 21, 2015 at 6:24 PM, chandresh pancholi < > > > > >> chandreshpancholi...@gmail.com> wrote: > > > > >> > > > > >>> Hi Gian/Vishal/Lalit, > > > > >>> > > > > >>> You guys have worked on Streamer integration with Ignite. I am > > > working > > > > on > > > > >>> Storm-ignite integration. > > > > >>> > > > > >>> I am setting up setStreamer() and setIgnite() in Test class and > > > sending > > > > >>> this object to setBolt() method. > > > > >>> > > > > >>> When i try to read via gettreamer() and getIgnite() i am getting > > > NULL. > > > > I > > > > >> am > > > > >>> trying to find out why is it happening but No Luck. > > > > >>> > > > > >>> If any one of you find some time and look into the code and help > me > > > to > > > > >>> resolve this. It would be great. > > > > >>> > > > > >>> Git : https://github.com/chandresh-pancholi/ignite > > > > >>> Module : storm > > > > >>> > > > > >>> -- > > > > >>> Chandresh Pancholi > > > > >>> Senior Software Engineer > > > > >>> Flipkart.com > > > > >>> Email-id:chandresh.panch...@flipkart.com > > > > >>> Contact:08951803660 > > > > > > > > > > > > > > > > > > > > -- > > > > > Chandresh Pancholi > > > > > Senior Software Engineer > > > > > Flipkart.com > > > > > Email-id:chandresh.panch...@flipkart.com > > > > > Contact:08951803660 > > > > > > > > > > > > > > > > -- > > > Chandresh Pancholi > > > Senior Software Engineer > > > Flipkart.com > > > Email-id:chandresh.panch...@flipkart.com > > > Contact:08951803660 > > > > > > > > > -- > Chandresh Pancholi > Senior Software Engineer > Flipkart.com > Email-id:chandresh.panch...@flipkart.com > Contact:08951803660 >
[jira] [Created] (IGNITE-1539) Marshaller performance micro-optimizations.
Vladimir Ozerov created IGNITE-1539: --- Summary: Marshaller performance micro-optimizations. Key: IGNITE-1539 URL: https://issues.apache.org/jira/browse/IGNITE-1539 Project: Ignite Issue Type: Task Components: general Affects Versions: 1.1.4 Reporter: Vladimir Ozerov Fix For: ignite-1.5 The main goal of marshalling is to write an object to byte[]/ByteBuffer, or read from it. Currently each read/write is surrounded by quite a few additional actions which can be avoided either in general, or in some specific cases. Looks like we can apply lots of micro-optimizations to our marshallers: 1) Do not perform HashMap lookups for each field ID. 2) Pre-calculate field IDs for reflection-based types. 3) Pre-calculate metadata for reflection-based types and do not track it. 4) Avoid "raw" mode checks for reflection-based types. 5) Merge small writes into bigger ones where possible. E.g. WRITE_INT_FIELD_ID + WRITE_INT_FIELD_LEN can be replaced with a single WRITE_LONG when field length is constant. 6) Memory bounds checks can be avoided in lots cases if we read/write object part with pre-defined length. 7) For objects having only fixed-length fields, we can pre-calculate a kind of template seriously minimizing writes. 8) Anything else? Applying some of these optimizations in .Net shows very promising results with up to 40% throughput increase. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
IGFS concurrency issue
Igniters, We revealed concurrency problem in IGFS and I would like to discuss possible solutions to it. Consider the following file system structure: root |-- A | |-- B | | |-- C | |-- D ... two concurrent operations in different threads: T1: move(/A/B, /A/D); T2: move(/A/D, /A/B/C); ... and how IGFS handles it now: T1: verify that "/A/B" and "/A/D" exist, they are not child-parent to each other, etc. -> OK. T2: do the same for "A/D" and "A/B/C" -> OK. T1: get IDs of "/A", "/A/B" and "/A/D" to lock them later inside tx. T2: get IDs of "/A", "/A/D", "/A/B" and "/A/B/C" to lock them later inside tx. T1: Start pessimistic tx, lock IDs of "/A", "/A/B", "/A/D", perform move -> OK. root |-- A | |-- D | | |-- B | | | |-- C T2: Start pessimistic tx, lock IDs of "/A", "/A/D", "/A/B" and "/A/B/C" (*directory structure already changed at this time!*), perform move -> OK. root |-- A B |-- D | |-- C | | |-- B (loop!) File system is corrupted. Folders B, C and D are not reacheable from root. To fix this now we additionaly check if directory structure is still valid *inside transaction*. It works, no more corruptions. But it requres taking locks on the whole paths *including root*. So move, delete and mkdirs opeartions *can no longer be concurrent*. Probably there is a way to relax this while still ensuring consistency, but I do not see how. One idea is to store real path inside each entry. This way we will be able to ensure that it is still at a valid location without blocking parents, so concurrnecy will be restored. But we will have to propagate strucutral changes to children. E.g. move of a folder with 100 items will lead to update of >100 cache entries. Not so good. Any other ideas? Vladimir.
