date:20201104

Re: How to get column names for a query in Ignite thin client mode

2020-11-04 Thread Alex Plehanov

Currently, only field names can be obtained, there is no information about
field data types in thin client protocol.

ср, 4 нояб. 2020 г. в 13:58, Shravya Nethula <
shravya.neth...@aline-consulting.com>:

> Ilya and Alex,
>
> Thank you for information.
> Can you please also suggest how to get the datatypes of those columns
> obtained from the query?
>
>
> Regards,
>
> Shravya Nethula,
>
> BigData Developer,
>
>
> Hyderabad.
>
>
> --
> *From:* Alex Plehanov 
> *Sent:* Tuesday, November 3, 2020 12:13 PM
> *To:* user@ignite.apache.org 
> *Subject:* Re: How to get column names for a query in Ignite thin client
> mode
>
> Columns information is read by thin-client only after the first data
> request, so you need to read at least one row to get columns.
>
> вт, 3 нояб. 2020 г. в 09:31, Ilya Kazakov :
>
> Hello, Shravya! It is very interesting! I am trying to reproduce your
> case, and what I see. I can see column names in the thin client only after
> query execution.
>
> For example:
>
> ClientConfiguration clientConfig = new 
> ClientConfiguration().setAddresses("127.0.0.1");
> try(IgniteClient thinClient = Ignition.startClient(clientConfig)){
> SqlFieldsQuery sql = new SqlFieldsQuery("SELECT * FROM T1");
> FieldsQueryCursor cursor = thinClient.query(sql);
> cursor.getAll();
> int count = cursor.getColumnsCount();
> System.out.println(count);
> List columnNames = new ArrayList<>();
> for (int i = 0; i < count; i++) {
> String columnName = cursor.getFieldName(i);
> columnNames.add(columnName);
> }
> System.out.println("columnNames:::"+columnNames);
> }
>
>
> But if this is the correct behavior I do not know yet, I will try to find
> out.
>
> 
> Ilya Kazakov
>
> вт, 3 нояб. 2020 г. в 12:51, Shravya Nethula <
> shravya.neth...@aline-consulting.com>:
>
> Hi,
>
> *For Ignite thick client, the column names for a given sql query are
> coming up as expected with the following code:*
> public class ClientNode {
>
> public static void main(String[] args) {
> IgniteConfiguration igniteCfg = new IgniteConfiguration();
> igniteCfg.setClientMode(true);
>
> Ignite ignite = Ignition.start(igniteCfg);
> *IgniteCache foo **= ignite.getOrCreateCache("foo");*
>
> SqlFieldsQuery sql = new SqlFieldsQuery("SELECT * FROM person");
> *FieldsQueryCursor cursor = foo.query(sql);*
> int count = cursor.getColumnsCount();
> List columnNames = new ArrayList<>();
>
> for (int i = 0; i < count; i++) {
>   String columnName = cursor.getFieldName(i);
>   columnNames.add(columnName);
> }
> System.out.println("columnNames:::"+columnNames);
>
>  } }
>  *Output:*
>  *columnNames:::[ID, NAME, LAST_NAME, AGE, CITY_ID, EMAIL_ID]
> *
> *On the other hand, for thin client, the column names are coming up as empty 
> list.*
> The following is the code:
> public class ClientNode {
>
> public static void main(String[] args) {
> ClientConfiguration clientConfig = new ClientConfiguration();
> cc.setUserName("username");
> cc.setUserPassword("password");
>
> *IgniteClient thinClient = Ignition.startClient(clientConfig);*
>
> SqlFieldsQuery sql = new SqlFieldsQuery("SELECT * FROM person");
> *FieldsQueryCursor cursor = thinClient.query(sql);*
> int count = cursor.getColumnsCount();
> List columnNames = new ArrayList<>();
>
> for (int i = 0; i < count; i++) {
>   String columnName = cursor.getFieldName(i);
>   columnNames.add(columnName);
> }
> System.out.println("columnNames:::"+columnNames);
>
>  } }
>
> *Output:**columnNames:::[ ]*
>
> While using IgniteCache.query(SqlFieldsQuery), the column names are
> coming up. But while using IgniteClient.query(SqlFieldsQuery), the column
> names are not coming up. Are we missing any configurations? Is there
> something wrong in the code? And also is there anyway in which we can
> identify the datatype of columns given in the query! We are looking for
> the datatype of the columns in the query but not the datatype of columns in
> the table!
>
> Any help here will be much appreciated!
> Thanks in advance!
>
>
>
> Regards,
>
> Shravya Nethula,
>
> BigData Developer,
>
>
> Hyderabad.
>
>

Ignite Client Node OOM Issue

2020-11-04 Thread Ravi Makwana

HI,

We are using Apache Ignite 2.7.0 binary and servers are using Linux OS &
app servers are using Windows OS.We are using Apache Ignite .Net APIs.

Recently we have noticed that our application (Client Node) is stopping
with OOM error. App server has 32 GB RAM & we are specifying JVM Heap = 8
GB.

App server utilization at the OOM error was 80%  and we have still 20%
memory left, still we are facing this issue.

1) How to check memory usage for the client node?
2)  Is there any way by looking at logs we can understand, OOM might
occur?
3)  Could you please suggest, how to overcome this issue?

 I am sharing an application crash log.

Thanks & Regards,


OOM-Error.rar
Description: Binary data

Limit ignite-rest-http threads

2020-11-04 Thread ashishb888

Before adding ignite-rest-http to the application my thread count was around
65

ashish@LAPTOP-0CGC34A5:~$ top -H -p 1468
top - 20:41:37 up 19:10,  0 users,  load average: 0.20, 0.16, 0.15
Threads:  64 total,   0 running,  64 sleeping,   0 stopped,   0 zombie


After adding ignite-rest-http to the application it went to around 85
threads

ashish@LAPTOP-0CGC34A5:~$ top -H -p 1545
top - 20:43:32 up 19:12,  0 users,  load average: 0.24, 0.17, 0.15
Threads:  85 total,   0 running,  85 sleeping,   0 stopped,   0 zombie

I tried below but no help
*igniteConfiguration.getConnectorConfiguration().setThreadPoolSize(2);*


BR,
Ashish



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite JDBC connection pooling mechanism

2020-11-04 Thread Sanjaya Kumar Sahoo

The above idea did not work. The ignite setup is as follows/

Ignite 2.8.1
Hikari 3.4.5
Java 1.8
Spring JdbcTemplate
Apache ignite is in Azure K8S cluster, and the service is being exposed as
a Azure internal load balancer.

The api works well for some time, (*till * *1 hour of restart*), after that
we are getting below error, if we restart then it works for another
approximately 1 hour.

is it not advisable to use a connection pooling  mechanism with ignite, if
yes, then what is the best way to serve concurrent requests ? is it kind
creating connection per user request and close once job done ?

Request to help on this we are completely stuck on this use case in
production.


LOGS IN IGNITE
=
^-- System thread pool [active=0, idle=6, qSize=0]
[06:51:33,191][SEVERE][grid-nio-worker-client-listener-2-#32][ClientListenerProcessor]
Failed to process selector key [ses=GridSelectorNioSessionImpl
[worker=ByteBufferNioClientWorker [readBuf=java.nio.HeapByteBuffer[pos=0
lim=8192 cap=8192], super=AbstractNioClientWorker [idx=2, bytesRcvd=0,
bytesSent=0, bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker
[name=grid-nio-worker-client-listener-2, igniteInstanceName=null,
finished=false, heartbeatTs=1604472690975, hashCode=1771197860,
interrupted=false, runner=grid-nio-worker-client-listener-2-#32]]],
writeBuf=null, readBuf=null, inRecovery=null, outRecovery=null,
closeSocket=true, outboundMessagesQueueSizeMetric=null,
super=GridNioSessionImpl [locAddr=/10.188.0.115:10800, rmtAddr=/
10.189.3.42:46262, createTime=1604464791433, closeTime=0, bytesSent=46,
bytesRcvd=51, bytesSent0=0, bytesRcvd0=0, sndSchedTime=1604464791514,
lastSndTime=1604464791514, lastRcvTime=1604464791433, readsPaused=false,
filterChain=FilterChain[filters=[GridNioAsyncNotifyFilter,
GridNioCodecFilter [parser=ClientListenerBufferedParser,
directMode=false]], accepted=true, markedForClose=false]]]

*java.io.IOException: Operation timed out* at
sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
at
org.apache.ignite.internal.util.nio.GridNioServer$ByteBufferNioClientWorker.processRead(GridNioServer.java:1162)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2449)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2216)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1857)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
[06:51:33,191][WARNING][grid-nio-worker-client-listener-2-#32][ClientListenerProcessor]
Client disconnected abruptly due to network connection loss or because the
connection was left open on application shutdown. [cls=class
o.a.i.i.util.nio.GridNioException, msg=Operation timed out]
[06:52:25,552][INFO][db-checkpoint-thread-#68][GridCacheDatabaseSharedManager]
Skipping checkpoint (no pages were modified)
[checkpointBeforeLockTime=17ms, checkpointLockWait=0ms,
checkpointListenersExecuteTime=21ms, checkpointLockHoldTime=23ms,
reason='timeout']
[06:52:25,716][INFO][grid-timeout-worker-#23][IgniteKernal]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=747a4939, uptime=41 days, 13:40:47.769]


LOGS IN APPLICATION SIDE
===
*03-11-2020 23:00:44.027 [http-nio-8080-exec-4] WARN
 com.zaxxer.hikari.pool.ProxyConnection.157 cache-query-service prod v1
cache-query-service-v1-5c5d8cd74d-jgnbb - HikariPool-1 - Connection
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection@62708a92 marked as
broken because of SQLSTATE(08006), ErrorCode(0)*

*java.sql.SQLException: Failed to communicate with Ignite cluster.* at
org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:760)
at
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:212)
at
org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.executeQuery(JdbcThinStatement.java:123)
at
com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:111)
at
com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
at
org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:439)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:376)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:452)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:462)
at
org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:473)
at
org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:480)
at

Re: How to get column names for a query in Ignite thin client mode

2020-11-04 Thread Shravya Nethula

Ilya and Alex,

Thank you for information.
Can you please also suggest how to get the datatypes of those columns obtained 
from the query?



Regards,

Shravya Nethula,

BigData Developer,

[cid:134aaaea-babf-48d0-9084-0e30ff58e8b5]

Hyderabad.


From: Alex Plehanov 
Sent: Tuesday, November 3, 2020 12:13 PM
To: user@ignite.apache.org 
Subject: Re: How to get column names for a query in Ignite thin client mode

Columns information is read by thin-client only after the first data request, 
so you need to read at least one row to get columns.

вт, 3 нояб. 2020 г. в 09:31, Ilya Kazakov 
mailto:kazakov.i...@gmail.com>>:
Hello, Shravya! It is very interesting! I am trying to reproduce your case, and 
what I see. I can see column names in the thin client only after query 
execution.

For example:

ClientConfiguration clientConfig = new 
ClientConfiguration().setAddresses("127.0.0.1");
try(IgniteClient thinClient = Ignition.startClient(clientConfig)){
SqlFieldsQuery sql = new SqlFieldsQuery("SELECT * FROM T1");
FieldsQueryCursor cursor = thinClient.query(sql);
cursor.getAll();
int count = cursor.getColumnsCount();
System.out.println(count);
List columnNames = new ArrayList<>();
for (int i = 0; i < count; i++) {
String columnName = cursor.getFieldName(i);
columnNames.add(columnName);
}
System.out.println("columnNames:::"+columnNames);
}

But if this is the correct behavior I do not know yet, I will try to find out.


Ilya Kazakov

вт, 3 нояб. 2020 г. в 12:51, Shravya Nethula 
mailto:shravya.neth...@aline-consulting.com>>:
Hi,

For Ignite thick client, the column names for a given sql query are coming up 
as expected with the following code:
public class ClientNode {

public static void main(String[] args) {
IgniteConfiguration igniteCfg = new IgniteConfiguration();
igniteCfg.setClientMode(true);

Ignite ignite = Ignition.start(igniteCfg);
IgniteCache foo = ignite.getOrCreateCache("foo");

SqlFieldsQuery sql = new SqlFieldsQuery("SELECT * FROM person");
FieldsQueryCursor cursor = foo.query(sql);
int count = cursor.getColumnsCount();
List columnNames = new ArrayList<>();

for (int i = 0; i < count; i++) {
  String columnName = cursor.getFieldName(i);
  columnNames.add(columnName);
}
System.out.println("columnNames:::"+columnNames);

 } }

 Output:
 columnNames:::[ID, NAME, LAST_NAME, AGE, CITY_ID, EMAIL_ID]



On the other hand, for thin client, the column names are coming up as empty 
list.
The following is the code:

public class ClientNode {

public static void main(String[] args) {
ClientConfiguration clientConfig = new ClientConfiguration();
cc.setUserName("username");
cc.setUserPassword("password");

IgniteClient thinClient = Ignition.startClient(clientConfig);

SqlFieldsQuery sql = new SqlFieldsQuery("SELECT * FROM person");
FieldsQueryCursor cursor = thinClient.query(sql);
int count = cursor.getColumnsCount();
List columnNames = new ArrayList<>();

for (int i = 0; i < count; i++) {
  String columnName = cursor.getFieldName(i);
  columnNames.add(columnName);
}
System.out.println("columnNames:::"+columnNames);

 } }

Output:
columnNames:::[ ]

While using IgniteCache.query(SqlFieldsQuery), the column names are coming up. 
But while using IgniteClient.query(SqlFieldsQuery), the column names are not 
coming up. Are we missing any configurations? Is there something wrong in the 
code? And also is there anyway in which we can identify the datatype of columns 
given in the query! We are looking for the datatype of the columns in the query 
but not the datatype of columns in the table!

Any help here will be much appreciated!
Thanks in advance!



Regards,

Shravya Nethula,

BigData Developer,

[cid:1758cc5f3ffbe2ca0301]

Hyderabad.

Re: Why does CacheBasedDataSet destroy the cache it is given

2020-11-04 Thread zaleslaw

Dear Courtney Robinson, 

please write if you have any cases to update helper cache with paritions to
better understand the situation. 

How and when are you going to clear this helper caches if the alternative
version (as you suggested in the first email) of CacheBased Dataset will be
provided?

Also, could you please answer to akornesh about main data cache behaviour?
Is it really destroyed in your tests or not now?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

[ignite 2.9.0] thin clients cannot access the Ignite Service deployed through UriDeploymentSpi( java.lang.ClassNotFoundException)

2020-11-04 Thread 18624049226


Hi community,

The operation steps are as follows:

1.use ignite.sh  example-deploy.xml start a server node

2.Put the service jar package in the /home/test/deploy directory

3.Deploy services using DeployClient

4.If you use ThickClientTest and ThinClientTest to access the service 
respectively, you will find that the ThickClientTest access is 
successful, but the ThinClientTest access is abnormal. The error is 
java.lang.ClassNotFoundException.


See ticket below for details:

https://issues.apache.org/jira/browse/IGNITE-13633


<>

RE: Ignite Cluster Issue on 2.7.6

2020-11-04 Thread Gurmehar Kalra

Hi,

Below are the logs
BaselineTopology of joining node (e6d542e7-cd73-4e57-90c1-b28da508c2c6) is not 
compatible with BaselineTopology in the cluster. Branching history of cluster 
BlT ([1016056908]) doesn't contain branching point hash of joining node BlT 
(510622971). Consider cleaning persistent storage of the node and adding it to 
the cluster again.
   at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.checkFailedError(TcpDiscoverySpi.java:1946)
 ~[ignite-core-2.7.6.jar:2.7.6]
   at 
org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:969)
 ~[ignite-core-2.7.6.jar:2.7.6]
   at 
org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:391) 
~[ignite-core-2.7.6.jar:2.7.6]
   at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2020)
 ~[ignite-core-2.7.6.jar:2.7.6]
   at 
org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:297)
 ~[ignite-core-2.7.6.jar:2.7.6]
   ... 52 common frames omitted

Regards,
Gurmehar Singh

From: Gurmehar Kalra 
Sent: 04 November 2020 13:34
To: user@ignite.apache.org
Cc: Andrei Aleksandrov 
Subject: RE: Ignite Cluster Issue on 2.7.6

[CAUTION: This Email is from outside the Organization. Unless you trust the 
sender, Don’t click links or open attachments as it may be a Phishing email, 
which can steal your Information and compromise your Computer.]
Hi,
 I did not remove the code ignite.cluster().active(true) , however added 
condition on one application to check if cluster is active or not and the other 
application activates the cluster.

Regards,
Gurmehar Singh

From: Andrei Aleksandrov 
mailto:aealexsand...@gmail.com>>
Sent: 30 October 2020 20:01
To: user@ignite.apache.org
Subject: Re: Ignite Cluster Issue on 2.7.6

[CAUTION: This Email is from outside the Organization. Unless you trust the 
sender, Don’t click links or open attachments as it may be a Phishing email, 
which can steal your Information and compromise your Computer.]

Hi,

Did you remove the code with ignite.cluster().active(true); ?
However, yes, all of your data nodes should be in baseline topology. Could you 
collect logs from your servers?

BR,
Andrei
10/30/2020 2:28 PM, Gurmehar Kalra пишет:
Hi,

I tried changes suggested by you , waited for nodes  and then tried to start 
cluster , but only 1 node is  joins cluster other node  does not participates 
in cluster.
Do I have to add all nodes into BLT ?
Regards,
Gurmehar Singh

From: Andrei Aleksandrov 

Sent: 29 October 2020 20:11
To: user@ignite.apache.org
Subject: Re: Ignite Cluster Issue on 2.7.6

[CAUTION: This Email is from outside the Organization. Unless you trust the 
sender, Don’t click links or open attachments as it may be a Phishing email, 
which can steal your Information and compromise your Computer.]

Hi,

Do you use cluster with persistence? After first actication all your data will 
be located on the first activated node.

In this case, you also should track your baseline.

https://www.gridgain.com/docs/latest/developers-guide/baseline-topology

Baseline topology is a subset of nodes where you cache data located.

The recommendations are the following:

1)you should activate the cluster only when all server nodes were started
2)If the topology changes, you must either restore the failed nodes or reset to 
the base topology to trigger partition reassignment and rebalancing.
3)If some new node should contain the cache data then you should add this node 
to baseline topology:

using java code:

https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCluster.html#setBaselineTopology-java.util.Collection-

using utility tool:

RE: Ignite Cluster Issue on 2.7.6

2020-11-04 Thread Gurmehar Kalra

Hi,
 I did not remove the code ignite.cluster().active(true) , however added 
condition on one application to check if cluster is active or not and the other 
application activates the cluster.

Regards,
Gurmehar Singh

From: Andrei Aleksandrov 
Sent: 30 October 2020 20:01
To: user@ignite.apache.org
Subject: Re: Ignite Cluster Issue on 2.7.6

[CAUTION: This Email is from outside the Organization. Unless you trust the 
sender, Don’t click links or open attachments as it may be a Phishing email, 
which can steal your Information and compromise your Computer.]

Hi,

Did you remove the code with ignite.cluster().active(true); ?
However, yes, all of your data nodes should be in baseline topology. Could you 
collect logs from your servers?

BR,
Andrei
10/30/2020 2:28 PM, Gurmehar Kalra пишет:
Hi,

I tried changes suggested by you , waited for nodes  and then tried to start 
cluster , but only 1 node is  joins cluster other node  does not participates 
in cluster.
Do I have to add all nodes into BLT ?
Regards,
Gurmehar Singh

From: Andrei Aleksandrov 

Sent: 29 October 2020 20:11
To: user@ignite.apache.org
Subject: Re: Ignite Cluster Issue on 2.7.6

[CAUTION: This Email is from outside the Organization. Unless you trust the 
sender, Don’t click links or open attachments as it may be a Phishing email, 
which can steal your Information and compromise your Computer.]

Hi,

Do you use cluster with persistence? After first actication all your data will 
be located on the first activated node.

In this case, you also should track your baseline.

https://www.gridgain.com/docs/latest/developers-guide/baseline-topology

Baseline topology is a subset of nodes where you cache data located.

The recommendations are the following:

1)you should activate the cluster only when all server nodes were started
2)If the topology changes, you must either restore the failed nodes or reset to 
the base topology to trigger partition reassignment and rebalancing.
3)If some new node should contain the cache data then you should add this node 
to baseline topology:

using java code:

https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCluster.html#setBaselineTopology-java.util.Collection-

using utility tool:

https://www.gridgain.com/docs/latest/administrators-guide/control-script#adding-nodes-to-baseline-topology

4)In case if some node from baseline can't be started (e.g because its data on 
the disk was destroyed) it should be removed from baseline:

https://www.gridgain.com/docs/latest/administrators-guide/control-script#removing-nodes-from-baseline-topology

If you are not using persistence, please provide additional information that 
"data is being added to the cache but not available to any of the modules." 
means:

1) How you access data
2) What do you see in the logs

BR,
Andrei
10/29/2020 4:19 PM, Gurmehar Kalra пишет:
Hi,

I have two module(Web and Engine)  and want to share data b/w the modules , but 
when I run  web and engine together , data is added to cache  but is not 
available to either of modules.
below is my ignite config, which is same in

Re: How to get column names for a query in Ignite thin client mode

Ignite Client Node OOM Issue

Limit ignite-rest-http threads

Re: Ignite JDBC connection pooling mechanism

Re: How to get column names for a query in Ignite thin client mode

Re: Why does CacheBasedDataSet destroy the cache it is given

[ignite 2.9.0] thin clients cannot access the Ignite Service deployed through UriDeploymentSpi( java.lang.ClassNotFoundException)

RE: Ignite Cluster Issue on 2.7.6

RE: Ignite Cluster Issue on 2.7.6

9 matches

Site Navigation

Mail list logo

Footer information