Re: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
Hello! I have filed an usability ticket https://issues.apache.org/jira/browse/IGNITE-11162 Regards, -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
Hello! Index inline is actually a vital parameter that can make up for multiple orders of performance difference. You can find threads about this in users list archive. However, you should definitely not set it to anything comparable to page size. Regards, -- Ilya Kasnacheev ср, 30 янв. 2019 г. в 07:30, mahesh76private : > On further debug, we realised we set index inline option to 2048..while > creating index. This problem (LOCK_RETRIES issue) started only after that. > Before, that we didn't face much of a problem with indexes. > > Also, another notable observation is setting index inline option to 2048 > also made the index huge, ran into 100s of Gig... > > Anyway, we are not using this option anymore Frankly, it seems a > internal detail of ignite, and not sure why this is given as an option to > configure for users. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
On further debug, we realised we set index inline option to 2048..while creating index. This problem (LOCK_RETRIES issue) started only after that. Before, that we didn't face much of a problem with indexes. Also, another notable observation is setting index inline option to 2048 also made the index huge, ran into 100s of Gig... Anyway, we are not using this option anymore Frankly, it seems a internal detail of ignite, and not sure why this is given as an option to configure for users. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
We actually think while building index, the tree is getting corrupted. Increasing LOCK_RETRIES didnt fix it too... Please review the below link as well. Infact, we saw creating index was getting into a infinite loop and throwing out errors at other clients. http://apache-ignite-users.70518.x6.nabble.com/processDynamicIndexChange-exception-at-client-node-while-creating-index-td26770.html My suspicion is this issue happens only when table is partitioned, no affinity keys, and definitely spread across multiple nodes. Please review the logs -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
Hello! This is highly unusual, I guess the default value for this property should be far larger that anything attainable in practice. Dmitriy, Alexey, can you please clarify? Regards. -- Ilya Kasnacheev вс, 27 янв. 2019 г. в 21:16, mahesh76private : > Increasing the size of IGNITE_BPLUS_TREE_LOCK_RETRIES makes the issue go > away. > But the explanation that is occurs due to contention is somewhat not > clear. > > In all cases, in occurred to be when I was creating an index. Feels like it > is somehow related to the height of the tree... and this variable seems to > limit the height. > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
RE: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
Increasing the size of IGNITE_BPLUS_TREE_LOCK_RETRIES makes the issue go away. But the explanation that is occurs due to contention is somewhat not clear. In all cases, in occurred to be when I was creating an index. Feels like it is somehow related to the height of the tree... and this variable seems to limit the height. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
OK. We don't have 1000 users for causing any sort of concurrency related lock ups. So possible explanation is corruption of the tree, below is our scenario -a- We created a table with sql - in the create params KEY_TYPE= ".", VALUE_TYPE="..." are also set. The intention is to retrieve data inserted into SQL table via Key/Value API for some ML code using ignite framework. -b- Once the table is created, we created an index on primary key column. Question: is creating an index column on primary key required? Internally, I assume you would create an index. -c- we started inserting rows into table via SQL insert. In this scenario, we got the LOCK_RETRIES issue Hope this gives you some clue as to why the issue occurred? Can you please address the question in b) regards Mahesh . -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
Yes. You can also use an environment variable instead of the system property: IGNITE_BPLUS_TREE_LOCK_RETRIES=10 ignite.sh … Stan From: mahesh76private Sent: 16 января 2019 г. 11:29 To: user@ignite.apache.org Subject: RE: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES how do I set it ? should I boot ignite node (ignite.sh) with the following switch ? java ... -DIGNITE_BPLUS_TREE_LOCK_RETRIES=10 regards Mahesh -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
how do I set it ? should I boot ignite node (ignite.sh) with the following switch ? java ... -DIGNITE_BPLUS_TREE_LOCK_RETRIES=10 regards Mahesh -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
RE: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES
It means that Ignite couldn’t find the place it needed in a B+ tree in 1000 iterations. It could mean either that there is a high contention on the tree (it changes a lot, and one thread is unlucky and couldn’t keep up with the speed), or that the tree is corrupted. Try to set a larger value to the IGNITE_BPLUS_TREE_LOCK_RETRIES property (e.g. 10). If you see still see the exception then it’s a corruption. If you don’t – it’s a contention. Stan From: mahesh76private Sent: 16 января 2019 г. 7:48 To: user@ignite.apache.org Subject: failure due to IGNITE_BPLUS_TREE_LOCK_RETRIES On 2.7, we are regularly seeing the below message and then the nodes stop. [16:45:04,759][SEVERE][disco-event-worker-#63][] JVM will be halted immediately due to the failure: [failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.IgniteCheckedException: Maximum number of retries 1000 reached for Put operation (the tree may be corrupted). Increase IGNITE_BPLUS_TREE_LOCK_RETRIES system property if you regularly see this message (current value is 1000).]] Can you please through some light on what this error is? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/