Re: [RESOLVED] JDBC Streaming

Awesome - it worked! Thanks, Ilya. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: [RESOLVED] JDBC Streaming

Thanks Ilya, Will try your suggestions... Where in the XML config do I set the 'failure detection timeout'? Thanks, Jose -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: [RESOLVED] JDBC Streaming

Hello! Can you try to increase 'failure detection timeout' property of your DiscoverySpi to something like 5 minutes (30)? This is failure detection handler which is supposed to catch deadlocks but instead it catches things like WAL copying. Also consider turning off WAL while ingesting data,

Re: [RESOLVED] JDBC Streaming

Hi Ilya, Your recommendation works - thanks. When loading a few million records, I always seem to experience a node failure on one of my two nodes. I have attached the log from a recent run on version 2.7.0, failure occurred after about 10 minutes... Please let me know if you see anything I mus

Re: [RESOLVED] JDBC Streaming

Hello! Yes, thin client has different approach to failover. However, it should not freeze. Please collect thread dumps so that we can see why it froze. The expectation here is that connection will be eventually dropped. I have found a reason for your Thick driver streaming troubles. Turns out, yo

[RESOLVED] JDBC Streaming

Hooray!!! - It works. Thanks, Ilya. Please continue your investigation of the JDBC Client Driver (thick-client), and let me know what you find...? What follows should perhaps be posted separately...but here's something I noticed, which I don't fully understand or know how to deal with: While bu

Re: JDBC Streaming

Hello! Indeed, you will need to reconnect if node that you're connected to fails. It always supported returning cluster-wide data retrieval. Regarda, -- Ilya Kasnacheev пн, 3 дек. 2018 г. в 15:25, joseheitor : > Hi Ilya, > > Thanks for the response. > > My understanding was that Thin JDBC dr

Re: JDBC Streaming

Hi Ilya, Thanks for the response. My understanding was that Thin JDBC driver was only able to connect to a single node (not a cluster), so that if that node failed - it was not able to continue operating on the cluster... It would also only return data residing on that node (not records residing

Re: JDBC Streaming

Hello! Apache Ignite SQL should be accessed by Ignite JDBC Thin driver. This is the preferred way. JDBC Thin driver also has streaming mode in the form of SET STREAMING ON/OFF. Please see attached file where I have introduced "thin" mode. As for client mode streaming not working, I will look in

Re: JDBC Streaming

Hi Ilya, Any update on your investigation of this issue...? Your comments that 'streaming mode' in Client driver and Client driver itself are near-deprecated - are very surprising and concerning! 1. Are you saying that Apache Ignite SQL will seize to be accessible via standard JDBC? 2. If 'stre

Re: JDBC Streaming

Hi Ilya, Please see attached archive with simple application demonstrating the problem. ignite-streaming.gz Once you have extracted the contents of the archive - please refer to the README for instructions... (and

Re: JDBC Streaming

Hello! Can you create a small reproducer project which will exhibit this behavior? Put it on e.g. Github. Note that 'streaming mode' in Client driver and Client driver itself are near-deprecated. So there may be some rough edges. However, there are streaming mode tests in Apache Ignite and they p

Re: JDBC Streaming

Hi Ilya, Yes - I am using JDBC Client driver to INSERT data into the SQL table. It works correctly (but slow) without setting 'streaming=true'. When I set 'streaming=true' in the connection string, as per the Ignite docs ( https://apacheignite-sql.readme.io/docs/jdbc-client-driver#section-streami

Re: JDBC Streaming

Hello! You can use DML on thick client, too, but it will be slower than Data Streamer approach. Regards, -- Ilya Kasnacheev вт, 27 нояб. 2018 г. в 11:22, joseheitor : > Shouldn't I be able to do this through plain JDBC DML operations? The > documentation for the JDBC Client Driver specifies t

Re: JDBC Streaming

Can anyone help with this? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: JDBC Streaming

Shouldn't I be able to do this through plain JDBC DML operations? The documentation for the JDBC Client Driver specifies this as a feature... Don't know whether I am doing something wrong, or if there is a bug? Is there any example showcasing this feature? Thanks, Jose -- Sent from: http://ap

Re: JDBC Streaming

The streaming examples should work fine for you. “Grid caches” and “SQL tables” are not two, different things. They are just two ways of accessing the same underlying structures. You can happily insert data using a Data Streamer and access it later using SQL. Regards, Stephen > On 26 Nov 2018,

Re: JDBC Streaming

Hi Ilya, I realize that Streaming is not supported via the JDBC Client node interface (*thanks for that info). But is it possible to stream-import data via (code) SQL directly and load it into persistent SQL table? (The Ignite code examples for Streaming only show loading data into grid caches...

Re: JDBC Streaming

Hello! Yes, streaming mode is not supported by client node driver. In this case your best bet is using Data Streamer anyway. https://apacheignite.readme.io/v2.6/docs/data-streamers Regards, -- Ilya Kasnacheev вс, 25 нояб. 2018 г. в 11:45, joseheitor : > Is Streaming perhaps not supported with

Re: JDBC Streaming

Is Streaming perhaps not supported with JDBC Client driver...? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: JDBC Streaming

Can anyone spot if I am doing something wrong...? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: JDBC Streaming

Hi Ilya, Tried executing SQL statement - 'SET STREAMING ON' from code, but this produces an exception. Here is my table definition: CREATE TABLE public.test_data ( pk LONG, id LONG, k VARCHAR, v VARCHAR, PRIMARY KEY (pk, id)) WITH "TEMPLATE=PARTITIONED, BACKUPS=1, ATOMICITY=TRANSACTIONAL, W

Re: JDBC Streaming

Hello! Have you tried force-flushing by doing SET STREAMING OFF; SET STREAMING ON; every once in a while? If this does not help, can you post a small reproducer on github? Regards, -- Ilya Kasnacheev вс, 18 нояб. 2018 г. в 10:43, joseheitor : > JDBC Client connection URL with 'streaming=true

JDBC Streaming

JDBC Client connection URL with 'streaming=true' runs (much faster than without) but no data is inserted into table. No errors are reported. I have two nodes with one partitioned table. Without 'streaming=true' it take around 8 hours to load 8,000,000 rows. The insert loop takes only around 8 minu