RE: [External]Re: removeAll operation on same cache causing deadlock

[Kamlesh Joshi]

Ohh! Anyways thanks for the update !

Regards,
Kamlesh Joshi

From: Alex Plehanov 
Sent: 21 January 2021 13:20
To: user@ignite.apache.org
Subject: [External]Re: removeAll operation on same cache causing deadlock


The e-mail below is from an external source. Please do not open attachments or 
click links from an unknown or suspicious origin.
Hello,

Unfortunately, for atomic caches in 2.9.1 there still java level deadlock 
possible (Ignite system threads can hang) for mass operations (putAll, 
removeAll, invokeAll) if an unordered key set is used. See [1]. This issue will 
be fixed in 2.10 release.

[1]: https://issues.apache.org/jira/browse/IGNITE-12451

чт, 21 янв. 2021 г. в 09:30, Kamlesh Joshi 
mailto:kamlesh.jo...@ril.com>>:
Hi Igniters,

Recently, we upgraded our lower environment to Ignite 2.9.1 from 2.7.0 . 
Earlier, we faced one issue of cluster going into hang state, if there are 
multiple threads performing removeAll() operation on same cache.
Its mentioned in the release notes that “Fixed deadlock on concurrent 
removeAll() on the same cache” this has been fixed. However, we were still able 
to reproduce the issue in this version as well. Can anyone please validate this?

Thanks and Regards,
Kamlesh Joshi


"Confidentiality Warning: This message and any attachments are intended only 
for the use of the intended recipient(s), are confidential and may be 
privileged. If you are not the intended recipient, you are hereby notified that 
any review, re-transmission, conversion to hard copy, copying, circulation or 
other use of this message and any attachments is strictly prohibited. If you 
are not the intended recipient, please notify the sender immediately by return 
email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure 
no viruses are present in this email. The company cannot accept responsibility 
for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only 
for the use of the intended recipient(s). 
are confidential and may be privileged. If you are not the intended recipient. 
you are hereby notified that any 
review. re-transmission. conversion to hard copy. copying. circulation or other 
use of this message and any attachments is 
strictly prohibited. If you are not the intended recipient. please notify the 
sender immediately by return email. 
and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure 
no viruses are present in this email. 
The company cannot accept responsibility for any loss or damage arising from 
the use of this email or attachment."

Re: Column not found using _val and sqlfieldsquery

[Pavel Tupitsyn]

> is there any specific reason why does not working if i use like this
t._val.EntityId

I think the reason here is that SQL (the standardized language) does not
support this kind of syntax.
Think of it as a flat table, _key and _val are just additional columns.


On Fri, Jan 22, 2021 at 9:04 AM siva  wrote:

> Hi ptupitsyn,
>
> yes it is working fine.
>
>  I have one query
> is there any specific reason why does not working if i use like this
> t._val.EntityId
>
> SELECT t._val from (select _val FROM materials ) as t where
> t_val.EntityId='123'
>
> Thanks.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Column not found using _val and sqlfieldsquery

[siva]

Hi ptupitsyn,

yes it is working fine.

 I have one query 
is there any specific reason why does not working if i use like this
t._val.EntityId

SELECT t._val from (select _val FROM materials ) as t where
t_val.EntityId='123'

Thanks.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Column not found using _val and sqlfieldsquery

[Pavel Tupitsyn]

Hi, I guess it should be like this:
SELECT t._val from (select _val, EntityId FROM materials ) as t where
t.EntityId='123'



On Fri, Jan 22, 2021 at 8:10 AM siva  wrote:

> Hi,
> I have .Net Ignite client and server app Ignite v2.9.1.
>
> I am using SqlFieldsQuery using thin client to query ignite.
>
> here is the query what i am trying:
>
> SELECT t._val from (select _val FROM materials ) as t where EntityId='123'
>
> exception msg:
>
> Failed to parse query. Column "ENTITYID" not found; SQL statement:
> SELECT t._val from (select _val FROM materials ) as t where EntityId='123'
> [42122-197]
>
> How to solve above exception.Please Any help?
>
> Thanks.
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Column not found using _val and sqlfieldsquery

[siva]

Hi,
I have .Net Ignite client and server app Ignite v2.9.1.

I am using SqlFieldsQuery using thin client to query ignite.

here is the query what i am trying:

SELECT t._val from (select _val FROM materials ) as t where EntityId='123'

exception msg:

Failed to parse query. Column "ENTITYID" not found; SQL statement:
SELECT t._val from (select _val FROM materials ) as t where EntityId='123'
[42122-197]

How to solve above exception.Please Any help?

Thanks.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

unsubscribe

[Peter Halverson]

> On Jan 21, 2021, at 5:17 PM, akorensh  wrote:
> 
> Hi,
>  Can you actually see what the data distribution of itemid is.
> 
>  AffinityKey will direct Ignite to use that specific column (itemid) for
> partition mapping.
>  If the data in itemid is in a narrow range, then all data will be put into
> fewer partitions.
>  This might affect loading times.
> 
>  more here:
> https://ignite.apache.org/docs/latest/data-modeling/data-partitioning
>  https://ignite.apache.org/docs/latest/data-modeling/affinity-collocation
> 
>  Affinity mapping takes up more memory as various intermediate objects are
> created.
>  Make sure you've allocated sufficient memory
> 
>  Can you collect/analyze GC logs to see whether you are running out of
> heap.
>  https://apacheignite.readme.io/docs/jvm-and-system-tuning
> 
> https://apacheignite.readme.io/docs/jvm-and-system-tuning#debugging-memory-usage-issues-and-gc-pauses
> 
> 
>   If you are sure that enough recourses are allocated, and the system still
> takes a long time,
>   add the Ignite Server log, and the GC logs here. 
> Thanks, Alex
> 
> 
> 
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Slow cache loading with an object key using affinity key

[akorensh]

Hi,
  Can you actually see what the data distribution of itemid is.

  AffinityKey will direct Ignite to use that specific column (itemid) for
partition mapping.
  If the data in itemid is in a narrow range, then all data will be put into
fewer partitions.
  This might affect loading times.

  more here:
https://ignite.apache.org/docs/latest/data-modeling/data-partitioning
  https://ignite.apache.org/docs/latest/data-modeling/affinity-collocation

  Affinity mapping takes up more memory as various intermediate objects are
created.
  Make sure you've allocated sufficient memory
   
  Can you collect/analyze GC logs to see whether you are running out of
heap.
  https://apacheignite.readme.io/docs/jvm-and-system-tuning
 
https://apacheignite.readme.io/docs/jvm-and-system-tuning#debugging-memory-usage-issues-and-gc-pauses


   If you are sure that enough recourses are allocated, and the system still
takes a long time,
   add the Ignite Server log, and the GC logs here. 
Thanks, Alex



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Slow cache loading with an object key using affinity key

[gvaidya]

Hi Alex,

I am simply using GridGain/Ignite web console to generate a project for
testing cacheload of a single table from my persistent storage. For
evaluation/testing purposes, I am running a single node in my cluster on a
2CPU 32Gb Windows machine. 

For testing, I created 2 projects. Test1 is where the table is imported as a
partitioned cache with no (or default) affinity defined. Test2 is where the
same table is imported as a partitioned cache with an affinity key for
colocating data based on a column that is not part of PK. In case of Test1,
I am able to import/load cache with ~537K records in 20 seconds. For Test2,
the same data takes a little over 3 minutes. This is just a sample set of my
overall data for this table. My total data volume is over 42 million
records. I want to partition and colocate my data... but for now trying to
understand the reason for performance difference between cache load times.

I am attaching the two projects as zip files. This was directly generated
from Web console. The only changes I make are correcting maven dependency
for mssql jdbc driver (8.4.1.jre14) in pom.xml and putting my persistent DB
details in secret.properties. I have reviewed the links you provided and the
affinity definition seems to be done correctly in Test2 project as far as I
can see.

Regards,
Gautam

Test1Cluster-project.zip

  
Test2Cluster-project.zip

  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Disappointing SQL update performance - no stored procedures?

[Ryan Trollip]

StephenThanks, the team is going through all of the performance guides for
a second time. We also understand that this operates on a version of the H2
database and are looking at the documentation around that also.
The performance bottleneck is for large transactions which comprise of,
what in a SQL DB would be, a number of select into statements. Selects with
joins, and an insert of a few thousand rows per transaction, with a dozen
of concurrent transactions.
We are running the client in spring boot, in the same JVM but not running
Ignite as compute node. Would that make any difference?
What we are trying tomorrow is bulk insert or "streaming" features, to see
if we can get better performance from that.
After a considerable amount of profiling, we also noticed garbage
collection resource challenges for 2nd and 3rd level cleanup casing large
delays, now and again, and are looking into forcing that in areas and
ensuring all objects are ready for cleanup as soon as possible.

On Thu, Jan 21, 2021 at 2:39 AM Stephen Darlington <
stephen.darling...@gridgain.com> wrote:

> This is the General Performance Tips guide: general-perf-tips
> 
>
> There’s no one-size-fits-all solution, but as a general point, Ignite is
> optimised to work as a cluster. Operating a single server node isn’t where
> you get the best performance relative to other solutions.
>
> The compute grid *is* Ignite’s stored procedure. Using colocated compute
> is one of the key mechanisms for getting the best performance as it avoids
> copying data across the network.
>
> Regards,
> Stephen
>
> On 20 Jan 2021, at 19:23, Ryan Trollip  wrote:
>
> Hey all
>
> For structured data read access is great, but large transactional updates
> to data are very not great, for us. We are working on a single node larger
> machine, so nothing fancy at all.
> In working to trouble shoot this, our dev team has tried a large number of
> different configurations over the past weeks and made good progress turning
> performance, but frankly, it's still nowhere near what I would expect from
> a standard SQL Server database with stored procedures, and it's using far
> more memory.
>
> One major issue is the back and forth with the client application. I've
> done some reading up on using compute grid instead but found this older
> article where they test the performance and for large inserts is many times
> slower than a SQL stored procedure.
> https://cs.ulb.ac.be/public/_media/teaching/ignite_2017.pdf
>
> Is this still the case? or is there a better way to approach this that we
> are missing? Would code written in a compute grid solve these performance
> issues? are compiled stored procedures planned for the future?
>
> Thanks
> Ryan
>
>
>
>

Re: 2.8.1 : INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi [] - Accepted incoming communication connection

[VeenaMithare]

Sure. Let me get back on this.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

[VeenaMithare]

Hi Andrei,
>>From your source code, I can see that the QueryCursor was created inside
startContinuousQuery. Then you return it from that method, but don't store
it in any variable:

>>continuousQueryHelper.startContinuousQuery(ignite,qry,testCache,subscribeParameters.callBackList,outputFieldsList);

I dont need to store the cursor that is returned from startContinuousQuery
because the cursor is read within the startContinuousQuery . 

Please find the code within startContinuousQuery in ContinuousQueryHelper
class . The cursor is stored in a variable and read through in the loop . 

// Create new continuous query.
QueryCursor> cur =
cache.query(qry);
LOGGER.info(
"PROJECT LISTENS: >>> Cache continuous query started
successfully. Start Initial Query...{}",
qry.getInitialQuery());

// Iterate through existing data.
// Please note there is a strange behaviour here. The iterator
returns a iterator of Cache.Entry
// However, each of the element of the iterator is actually not
Cache.Entry .
// Each element of the iterator is List of fields in a row. ( i.e.
it matches what is supposed to be returned by
// SQLFieldsQuery.query("query..").getAll();
List addedOrModifiedList = new ArrayList<>();
List removedList = new ArrayList<>();

int count=0;
for (Object e : cur.getAll()) {

BinaryObject
binaryObject=createBinaryObject((List)e,outputFieldsList,ignite,"TEST");
++count;
addedOrModifiedList.add(binaryObject);
LOGGER.debug("PROJECT LISTENS: Initial entry binaryObject : {}
", binaryObject);


}

I am not sure how it can be garbage collected before it is read. 

regards,
Veena.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: SqlQuery Deprecated Since 2.8, please use SqlFieldsQuery instead....How to get result as passing model Type with SqlFieldsQuery?

[siva]

Hi,

It's working fine in "select _val from mytesttable" query but in case of sub
query or joins it's failing with an error like column not found

It's failing access column with and without table Aliases.
for example

Query::>

SELECT t.* from (select _val FROM materials ) as t INNER JOIN
"F2DEDF6E-393E-42BC-9BB3-E835A1063B30_862C7264-8C8A-4217-AFA4-30CD49AC050F_SECURITY".SecurityGroups
ON t.EntityId = SecurityGroups.SGEntityId  WHERE securityGroupId in
('25c8eff1-1607-4a57-bd88-b4be9a1c3b73') order by t.MaterialName offset 0
rows fetch next 7000 rows only

Exception::>

Failed to parse query. Column "T.MATERIALNAME" not found; SQL statement:
SELECT t.* from (select _val FROM materials ) as t INNER JOIN
"F2DEDF6E-393E-42BC-9BB3-E835A1063B30_862C7264-8C8A-4217-AFA4-30CD49AC050F_SECURITY".SecurityGroups
ON t.EntityId = SecurityGroups.SGEntityId  WHERE securityGroupId in
('25c8eff1-1607-4a57-bd88-b4be9a1c3b73') order by t.MaterialName offset 0
rows fetch next 7000 rows only [42122-197]

Thanks.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

[andrei]

Hi,

From your source code, I can see that the QueryCursor was created 
inside startContinuousQuery. Then you return it from that method, but 
don't store it in any variable:


continuousQueryHelper.startContinuousQuery(ignite,qry,testCache,subscribeParameters.callBackList,outputFieldsList);
 for (int i =0; i <10; i++) {
LOGGER.info("ListenerClient running... i=" + i); try {
Thread.sleep(2); }catch (Exception e) {
//Ignore this block }
}

My guess is that your QueryCursor instance can be collected by the GC 
within those 20 seconds, and I think that in this case, your query can 
be canceled. As the result you see only some part of updates.


Can you please re-create your code according to the example:

https://github.com/gridgain/gridgain/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/CacheContinuousQueryExample.java

And try one more time.

BR,
Andrei

1/20/2021 1:36 PM, VeenaMithare пишет:

Hi Andrei,

Please find a trimmed down version of the code attached.
Have a look at the ContinuousQueryHelper -> setInitialQuery to check how the
initial query has been coded.

regards,
Veena.
InitialQueryProject.zip




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Slow cache loading with an object key using affinity key

[akorensh]

Hi,
   Can you give more context regarding your data load.
   How big is your cluster?* Are the load times comparable if you use only
one node for
   both use cases? *

   If you remove the affinity column in the second use case, does the load
speed up in a cluster env?
   If you lower the amount of data to say, 50K. 100K, etc.. records, are
load times proportionally correlated to a load of your full data set?

   Follow this example to see what you might've missed when setting up cache
loading:
https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/store/jdbc/CacheJdbcPersonStore.java


  Make sure you've properly set up your affinity keys:
https://ignite.apache.org/docs/latest/data-modeling/affinity-collocation#configuring-affinity-key

   You might try data streaming as an alternative to cache loading:
https://ignite.apache.org/docs/latest/data-streaming 

  If you are still stuck:
   Can you send the Ignite config, include new and old cache configs and 
the code that does the loading for both use cases.?

Thanks, Alex



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ever increasing startup times as data grow in persistent storage

[andrei]

Hi,

I don't think there are any other options at the moment other than the 
ones you mentioned.


However, you can also create your own application that will check the 
topology and activate it when all nodes from the baseline are online. 
For example, additional java code when starting a server node.


In case you require any changes to the current Ignite implementation, 
you can create a thread in the Ignite developer list:


http://apache-ignite-developers.2346864.n4.nabble.com/

BR,
Andrei


1/20/2021 9:16 PM, Raymond Wilson пишет:

Hi Andre,

I would like to see Ignite support a graceful shutdown scenario you 
get with deactivation, but which does not need to be manually reactivated.


We run a pretty agile process and it is not uncommon to have multiple 
deploys to production throughout a week. This is a pretty automated 
affair (essentially push-button) and it works well, except for the WAL 
rescan on startup.


Today there are two approaches we can take for a deployment:

1. Stop the nodes (which is what we currently do), leaving the WAL and 
persistent store inconsistent. This requires a rescan of the WAL 
before the grid is auto re-activated on startup. The time to do this 
is increasing with the size of the persistent store - it does not 
appear to be related to the size of the WAL.
2. Deactivate the grid, which leaves the WAL and persistent store in a 
consistent state. This requires manual re-activation on restart, but 
does not incur the increasing WAL restart cost.


Is an option like the one below possible?:

3. Suspend the grid, which performs the same steps deactivation does 
to make the WAL and persistent store consistent, but which leaves the 
grid activated so the manual activation process is not required on 
restart.


Thanks,
Raymond.

On Thu, Jan 21, 2021 at 4:02 AM andrei > wrote:


Hi,

Yes, that was to be expected. The main autoactivation scenario is
cluster restart. If you are using manual deactivation, you should
also manually activate your cluster.

BR,
Andrei

1/20/2021 5:50 AM, Raymond Wilson пишет:

We have been experimenting with using deactivation to shutdown
the grid to reduce the time for the grid to start up again.

It appears there is a downside to this: once deactivated the grid
does not appear to auto-activate once baseline topology is
achieved, which means we will need to run through the
bootstrapping protocol of ensuring the grid has restarted
correctly before activating it once again.

The baseline topology documentation at
https://ignite.apache.org/docs/latest/clustering/baseline-topology

does not cover this condition.

Is this expected?

Thanks,
Raymond.


On Wed, Jan 13, 2021 at 11:49 PM Pavel Tupitsyn
mailto:ptupit...@apache.org>> wrote:

Raymond,

Please use ICluster.SetActive [1] instead, the API linked
above is obsolete


[1]

https://ignite.apache.org/releases/latest/dotnetdoc/api/Apache.Ignite.Core.Cluster.ICluster.html?#Apache_Ignite_Core_Cluster_ICluster_SetActive_System_Boolean_



On Wed, Jan 13, 2021 at 11:54 AM Raymond Wilson
mailto:raymond_wil...@trimble.com>> wrote:

Of course. Obvious! :)

Sent from my iPhone


On 13/01/2021, at 9:15 PM, Zhenya Stanilovsky
mailto:arzamas...@mail.ru>> wrote:





Is there an API version of the cluster deactivation?


https://github.com/apache/ignite/blob/master/modules/platforms/dotnet/Apache.Ignite.Core.Tests/Cache/PersistentStoreTestObsolete.cs#L131



On Wed, Jan 13, 2021 at 8:28 PM Zhenya Stanilovsky
>
wrote:



Hi Zhenya,
Thanks for confirming performing checkpoints
more often will help here.

Hi Raymond !

I have established this configuration so
will experiment with settings little.
On a related note, is there any way to
automatically trigger a checkpoint, for
instance as a pre-shutdown activity?

If you shutdown your cluster gracefully = with
deactivation [1] further start will not
trigger wal readings.
[1]

https://www.gridgain.com/docs/latest/administrators-guide/control-script#deactivating-cluster
 

Re: SqlQuery Deprecated Since 2.8, please use SqlFieldsQuery instead....How to get result as passing model Type with SqlFieldsQuery?

[Alex Plehanov]

Hello,

Try this: "select _val from mytesttable"

чт, 21 янв. 2021 г. в 17:38, siva :

> Hi,
> I am using .net Ignite v2.9.1 and
>
> Since SqlQuery deprecated in documentation suggesting please use
> SqlFieldsQuery.
>
> what is the way to get result as Type using sqlfieldsquery?
>
> for example query
> select * from mytesttable
>
> Thanks.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

SqlQuery Deprecated Since 2.8, please use SqlFieldsQuery instead....How to get result as passing model Type with SqlFieldsQuery?

[siva]

Hi,
I am using .net Ignite v2.9.1 and 

Since SqlQuery deprecated in documentation suggesting please use
SqlFieldsQuery.

what is the way to get result as Type using sqlfieldsquery?

for example query
select * from mytesttable

Thanks.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Disappointing SQL update performance - no stored procedures?

[Stephen Darlington]

This is the General Performance Tips guide: general-perf-tips 


There’s no one-size-fits-all solution, but as a general point, Ignite is 
optimised to work as a cluster. Operating a single server node isn’t where you 
get the best performance relative to other solutions.

The compute grid is Ignite’s stored procedure. Using colocated compute is one 
of the key mechanisms for getting the best performance as it avoids copying 
data across the network.

Regards,
Stephen

> On 20 Jan 2021, at 19:23, Ryan Trollip  wrote:
> 
> Hey all
> 
> For structured data read access is great, but large transactional updates to 
> data are very not great, for us. We are working on a single node larger 
> machine, so nothing fancy at all. 
> In working to trouble shoot this, our dev team has tried a large number of 
> different configurations over the past weeks and made good progress turning 
> performance, but frankly, it's still nowhere near what I would expect from a 
> standard SQL Server database with stored procedures, and it's using far more 
> memory. 
> 
> One major issue is the back and forth with the client application. I've done 
> some reading up on using compute grid instead but found this older article 
> where they test the performance and for large inserts is many times slower 
> than a SQL stored procedure. 
> https://cs.ulb.ac.be/public/_media/teaching/ignite_2017.pdf 
> 
> 
> Is this still the case? or is there a better way to approach this that we are 
> missing? Would code written in a compute grid solve these performance issues? 
> are compiled stored procedures planned for the future? 
> 
> Thanks
> Ryan

Re: 2.8.1 : INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi [] - Accepted incoming communication connection

[alexey.plotnik]

Could you plz attach DEBUG log of the server node running on Linux machine
from the moment server starts until client re-connect after being stopped?
So, full logs basically



-
Alexey Plotnik
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: 2.8.1 : INFO org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi [] - Accepted incoming communication connection

[VeenaMithare]

Hi Alexey,

I am able to reproduce this on my linux env. by doing a kill -9 .  Please
find the logs from my linux env. attached. 
temp.log
  
Also note in our linux env. we have dual ip for our boxes - not sure if that
contributes to this issue. 

Also I do not see this log in the INFO mode in our client logs : 
Sending local node ID (bc6ae633-b643-44ba-afab-321dbf01aaf1 to newly
accepted session ...

I will try shutting down the client with ctrl+c on windows and check if that
helps.

regards,
Veena.





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/