Data streamer hangs
Hello.I use SET STREAMING ON and SET STREAMING OFF style to flush data to ignite.Ignite sometimes hangs on SET STREAMING OFF. The thread hangs on client side looks like: daemon prio=5 os_prio=0 tid=0x7fdba0003000 nid=0x3c1 waiting on condition [0x7fdf473f1000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141) at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection$StreamState.executeBatch(JdbcThinConnection.java:1280) at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection$StreamState.close0(JdbcThinConnection.java:1335) at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection$StreamState.close(JdbcThinConnection.java:1325) at org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.executeNative(JdbcThinConnection.java:253) at org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:206) at org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:559) at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:95) at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java) ... at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) The thread hangs on server side looks like: "data-streamer-stripe-21-#150" #178 prio=5 os_prio=0 tid=0x7f29b3737000 nid=0x37c36 waiting on condition [0x7f041d1d] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at org.apache.ignite.internal.util.StripedExecutor$StripeConcurrentQueue.take(StripedExecutor.java:730) at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:541) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Locked ownable synchronizers: - None I do not see any error in ignite log and I do not find any dead locks. Do u have any ideas about such situation?Thanks! -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Data streamer hangs
Hi, Are you seeing pool starvation messages? How long does it stay in this state? Are you not seeing any data saved, even when you close the jdbc connection? Are you able to reproduce this scenario? If so please describe steps/send a reproducer and send full logs. Data Streamer pool config: https://apacheignite.readme.io/docs/thread-pools#data-streamer-pool info on data streamer: https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteDataStreamer.html see: https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteDataStreamer.html#perNodeBufferSize-int- from the doc: https://apacheignite-sql.readme.io/docs/set#description When streaming is enabled, JDBC/ODBC driver will pack your commands in batches and send them to the server (Ignite cluster). On the server side, the batch is converted into a stream of cache update commands which are distributed asynchronously between server nodes. In a JDBC SET STREAMING ON/OFF, this buffer is set internally to, and you need to flush the data by closing the connection. see : https://apacheignite-sql.readme.io/docs/set#example Thanks, Alex -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/