Re: Batch insert into ignite using jdbc thin driver taking long time
Hello!
That's strange. Can you collect a thread dump after it starts hanging,
share it with us?
Regards,
--
Ilya Kasnacheev
чт, 6 сент. 2018 г. в 20:28, Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:
> Hi Ilya,
>
> It did compiled...but it is not inserting records to ignite at all...
>
> I waited for 20 mins for it to insert and terminated and there are no
> exceptions in the ignite log.
>
> Could you please suggest the right syntax to enable the streaming using
> jdbc thin driver..and my key concern is performance...Inserting 1 million
> records with 90 columns each row should get completed in less than 30 sec
> is what we are looking for...
> Is it possible with ignite?
>
> Thanks & Regards,
> Sriveena
> --
> *From:* Ilya Kasnacheev
> *Sent:* 06 September 2018 17:24:13
> *To:* user@ignite.apache.org
> *Subject:* Re: Batch insert into ignite using jdbc thin driver taking
> long time
>
> Hello!
>
> I'm not sure but it doesn't look right to me. Does it compile?
>
> Note that you also need to SET STREAMING OFF after data load is complete.
>
>
> --
> Ilya Kasnacheev
>
>
> чт, 6 сент. 2018 г. в 14:42, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Thanks Ilya. Will try using together.
>
>
>
> Please correct if the below statement work.
>
> getConnection().createStatement().execute("SET STREAMING ON");
>
>
>
> Regards,
>
> Sriveena
>
>
>
> *From:* Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com]
> *Sent:* Thursday, September 06, 2018 3:28 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: Batch insert into ignite using jdbc thin driver taking
> long time
>
>
>
> Hello!
>
>
>
> I don't see why not. Have you tried? Any difference in run time?
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> чт, 6 сент. 2018 г. в 6:20, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Hi Ilya,
>
>
>
> Can we cobimbe both of them together I mean
>
> 1. SET STREAMING ON;
>
> 2. JDBC thin driver batch insert (pstmt.executeBatch())
>
>
>
> Thanks & Regards,
> Sriveena
>
>
>
> *From:* Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com]
> *Sent:* Wednesday, September 05, 2018 7:02 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: Batch insert into ignite using jdbc thin driver taking
> long time
>
>
>
> Hello!
>
>
>
> Have you tried streaming mode?
>
>
>
> https://apacheignite-sql.readme.io/docs/set
> <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fset&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C7d329f7496a845c707bb08d61333fb1d%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636717511328312021&sdata=WfZ7m34afylAJv0PZ0GRo2o7mIQVxR6f%2BCb2MuQUj3M%3D&reserved=0>
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> ср, 5 сент. 2018 г. в 15:44, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Hi,
>
>
>
> I am trying to batch insert 30 records into ignite cache(on remote
> server) using jdbc thin driver.
>
> It is taking nearly 4mins to complete this operation. Please advise.
>
>
>
> Thanks & Regards,
>
> Sriveena
>
>
>
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
>
Re: Batch insert into ignite using jdbc thin driver taking long time
Hi Ilya,
It did compiled...but it is not inserting records to ignite at all...
I waited for 20 mins for it to insert and terminated and there are no
exceptions in the ignite log.
Could you please suggest the right syntax to enable the streaming using jdbc
thin driver..and my key concern is performance...Inserting 1 million records
with 90 columns each row should get completed in less than 30 sec is what we
are looking for...
Is it possible with ignite?
Thanks & Regards,
Sriveena
From: Ilya Kasnacheev
Sent: 06 September 2018 17:24:13
To: user@ignite.apache.org
Subject: Re: Batch insert into ignite using jdbc thin driver taking long time
Hello!
I'm not sure but it doesn't look right to me. Does it compile?
Note that you also need to SET STREAMING OFF after data load is complete.
--
Ilya Kasnacheev
чт, 6 сент. 2018 г. в 14:42, Sriveena Mattaparthi
mailto:sriveena.mattapar...@ekaplus.com>>:
Thanks Ilya. Will try using together.
Please correct if the below statement work.
getConnection().createStatement().execute("SET STREAMING ON");
Regards,
Sriveena
From: Ilya Kasnacheev
[mailto:ilya.kasnach...@gmail.com<mailto:ilya.kasnach...@gmail.com>]
Sent: Thursday, September 06, 2018 3:28 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Batch insert into ignite using jdbc thin driver taking long time
Hello!
I don't see why not. Have you tried? Any difference in run time?
Regards,
--
Ilya Kasnacheev
чт, 6 сент. 2018 г. в 6:20, Sriveena Mattaparthi
mailto:sriveena.mattapar...@ekaplus.com>>:
Hi Ilya,
Can we cobimbe both of them together I mean
1. SET STREAMING ON;
2. JDBC thin driver batch insert (pstmt.executeBatch())
Thanks & Regards,
Sriveena
From: Ilya Kasnacheev
[mailto:ilya.kasnach...@gmail.com<mailto:ilya.kasnach...@gmail.com>]
Sent: Wednesday, September 05, 2018 7:02 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Batch insert into ignite using jdbc thin driver taking long time
Hello!
Have you tried streaming mode?
https://apacheignite-sql.readme.io/docs/set<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fset&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C7d329f7496a845c707bb08d61333fb1d%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636717511328312021&sdata=WfZ7m34afylAJv0PZ0GRo2o7mIQVxR6f%2BCb2MuQUj3M%3D&reserved=0>
Regards,
--
Ilya Kasnacheev
ср, 5 сент. 2018 г. в 15:44, Sriveena Mattaparthi
mailto:sriveena.mattapar...@ekaplus.com>>:
Hi,
I am trying to batch insert 30 records into ignite cache(on remote server)
using jdbc thin driver.
It is taking nearly 4mins to complete this operation. Please advise.
Thanks & Regards,
Sriveena
“Confidentiality Notice: The contents of this email message and any attachments
are intended solely for the addressee(s) and may contain confidential and/or
privileged information and may be legally protected from disclosure. If you are
not the intended recipient of this message or their agent, or if this message
has been addressed to you in error, please immediately alert the sender by
reply email and then delete this message and any attachments. If you are not
the intended recipient, you are hereby notified that any use, dissemination,
copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments
are intended solely for the addressee(s) and may contain confidential and/or
privileged information and may be legally protected from disclosure. If you are
not the intended recipient of this message or their agent, or if this message
has been addressed to you in error, please immediately alert the sender by
reply email and then delete this message and any attachments. If you are not
the intended recipient, you are hereby notified that any use, dissemination,
copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments
are intended solely for the addressee(s) and may contain confidential and/or
privileged information and may be legally protected from disclosure. If you are
not the intended recipient of this message or their agent, or if this message
has been addressed to you in error, please immediately alert the sender by
reply email and then delete this message and any attachments. If you are not
the intended recipient, you are hereby notified that any use, dissemination,
copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments
are intended solely for the addressee(s) and may contain confidential and/or
privileged information and ma
Re: Batch insert into ignite using jdbc thin driver taking long time
Hello!
I'm not sure but it doesn't look right to me. Does it compile?
Note that you also need to SET STREAMING OFF after data load is complete.
--
Ilya Kasnacheev
чт, 6 сент. 2018 г. в 14:42, Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:
> Thanks Ilya. Will try using together.
>
>
>
> Please correct if the below statement work.
>
> getConnection().createStatement().execute("SET STREAMING ON");
>
>
>
> Regards,
>
> Sriveena
>
>
>
> *From:* Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com]
> *Sent:* Thursday, September 06, 2018 3:28 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: Batch insert into ignite using jdbc thin driver taking
> long time
>
>
>
> Hello!
>
>
>
> I don't see why not. Have you tried? Any difference in run time?
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> чт, 6 сент. 2018 г. в 6:20, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Hi Ilya,
>
>
>
> Can we cobimbe both of them together I mean
>
> 1. SET STREAMING ON;
>
> 2. JDBC thin driver batch insert (pstmt.executeBatch())
>
>
>
> Thanks & Regards,
> Sriveena
>
>
>
> *From:* Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com]
> *Sent:* Wednesday, September 05, 2018 7:02 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: Batch insert into ignite using jdbc thin driver taking
> long time
>
>
>
> Hello!
>
>
>
> Have you tried streaming mode?
>
>
>
> https://apacheignite-sql.readme.io/docs/set
> <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fset&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C7d329f7496a845c707bb08d61333fb1d%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636717511328312021&sdata=WfZ7m34afylAJv0PZ0GRo2o7mIQVxR6f%2BCb2MuQUj3M%3D&reserved=0>
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> ср, 5 сент. 2018 г. в 15:44, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Hi,
>
>
>
> I am trying to batch insert 30 records into ignite cache(on remote
> server) using jdbc thin driver.
>
> It is taking nearly 4mins to complete this operation. Please advise.
>
>
>
> Thanks & Regards,
>
> Sriveena
>
>
>
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
RE: Batch insert into ignite using jdbc thin driver taking long time
Thanks Ilya. Will try using together.
Please correct if the below statement work.
getConnection().createStatement().execute("SET STREAMING ON");
Regards,
Sriveena
From: Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com]
Sent: Thursday, September 06, 2018 3:28 PM
To: user@ignite.apache.org
Subject: Re: Batch insert into ignite using jdbc thin driver taking long time
Hello!
I don't see why not. Have you tried? Any difference in run time?
Regards,
--
Ilya Kasnacheev
чт, 6 сент. 2018 г. в 6:20, Sriveena Mattaparthi
mailto:sriveena.mattapar...@ekaplus.com>>:
Hi Ilya,
Can we cobimbe both of them together I mean
1. SET STREAMING ON;
2. JDBC thin driver batch insert (pstmt.executeBatch())
Thanks & Regards,
Sriveena
From: Ilya Kasnacheev
[mailto:ilya.kasnach...@gmail.com<mailto:ilya.kasnach...@gmail.com>]
Sent: Wednesday, September 05, 2018 7:02 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Batch insert into ignite using jdbc thin driver taking long time
Hello!
Have you tried streaming mode?
https://apacheignite-sql.readme.io/docs/set<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fset&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C7d329f7496a845c707bb08d61333fb1d%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636717511328312021&sdata=WfZ7m34afylAJv0PZ0GRo2o7mIQVxR6f%2BCb2MuQUj3M%3D&reserved=0>
Regards,
--
Ilya Kasnacheev
ср, 5 сент. 2018 г. в 15:44, Sriveena Mattaparthi
mailto:sriveena.mattapar...@ekaplus.com>>:
Hi,
I am trying to batch insert 30 records into ignite cache(on remote server)
using jdbc thin driver.
It is taking nearly 4mins to complete this operation. Please advise.
Thanks & Regards,
Sriveena
“Confidentiality Notice: The contents of this email message and any attachments
are intended solely for the addressee(s) and may contain confidential and/or
privileged information and may be legally protected from disclosure. If you are
not the intended recipient of this message or their agent, or if this message
has been addressed to you in error, please immediately alert the sender by
reply email and then delete this message and any attachments. If you are not
the intended recipient, you are hereby notified that any use, dissemination,
copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments
are intended solely for the addressee(s) and may contain confidential and/or
privileged information and may be legally protected from disclosure. If you are
not the intended recipient of this message or their agent, or if this message
has been addressed to you in error, please immediately alert the sender by
reply email and then delete this message and any attachments. If you are not
the intended recipient, you are hereby notified that any use, dissemination,
copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments
are intended solely for the addressee(s) and may contain confidential and/or
privileged information and may be legally protected from disclosure. If you are
not the intended recipient of this message or their agent, or if this message
has been addressed to you in error, please immediately alert the sender by
reply email and then delete this message and any attachments. If you are not
the intended recipient, you are hereby notified that any use, dissemination,
copying, or storage of this message or its attachments is strictly prohibited.”
Re: Batch insert into ignite using jdbc thin driver taking long time
Hello! I don't see why not. Have you tried? Any difference in run time? Regards, -- Ilya Kasnacheev чт, 6 сент. 2018 г. в 6:20, Sriveena Mattaparthi < sriveena.mattapar...@ekaplus.com>: > Hi Ilya, > > > > Can we cobimbe both of them together I mean > > 1. SET STREAMING ON; > > 2. JDBC thin driver batch insert (pstmt.executeBatch()) > > > > Thanks & Regards, > Sriveena > > > > *From:* Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com] > *Sent:* Wednesday, September 05, 2018 7:02 PM > *To:* user@ignite.apache.org > *Subject:* Re: Batch insert into ignite using jdbc thin driver taking > long time > > > > Hello! > > > > Have you tried streaming mode? > > > > https://apacheignite-sql.readme.io/docs/set > <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fset&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C7d329f7496a845c707bb08d61333fb1d%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636717511328312021&sdata=WfZ7m34afylAJv0PZ0GRo2o7mIQVxR6f%2BCb2MuQUj3M%3D&reserved=0> > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > ср, 5 сент. 2018 г. в 15:44, Sriveena Mattaparthi < > sriveena.mattapar...@ekaplus.com>: > > Hi, > > > > I am trying to batch insert 30 records into ignite cache(on remote > server) using jdbc thin driver. > > It is taking nearly 4mins to complete this operation. Please advise. > > > > Thanks & Regards, > > Sriveena > > > > “Confidentiality Notice: The contents of this email message and any > attachments are intended solely for the addressee(s) and may contain > confidential and/or privileged information and may be legally protected > from disclosure. If you are not the intended recipient of this message or > their agent, or if this message has been addressed to you in error, please > immediately alert the sender by reply email and then delete this message > and any attachments. If you are not the intended recipient, you are hereby > notified that any use, dissemination, copying, or storage of this message > or its attachments is strictly prohibited.” > > “Confidentiality Notice: The contents of this email message and any > attachments are intended solely for the addressee(s) and may contain > confidential and/or privileged information and may be legally protected > from disclosure. If you are not the intended recipient of this message or > their agent, or if this message has been addressed to you in error, please > immediately alert the sender by reply email and then delete this message > and any attachments. If you are not the intended recipient, you are hereby > notified that any use, dissemination, copying, or storage of this message > or its attachments is strictly prohibited.” >
RE: Batch insert into ignite using jdbc thin driver taking long time
Hi Ilya, Can we cobimbe both of them together I mean 1. SET STREAMING ON; 2. JDBC thin driver batch insert (pstmt.executeBatch()) Thanks & Regards, Sriveena From: Ilya Kasnacheev [mailto:ilya.kasnach...@gmail.com] Sent: Wednesday, September 05, 2018 7:02 PM To: user@ignite.apache.org Subject: Re: Batch insert into ignite using jdbc thin driver taking long time Hello! Have you tried streaming mode? https://apacheignite-sql.readme.io/docs/set<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fset&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C7d329f7496a845c707bb08d61333fb1d%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636717511328312021&sdata=WfZ7m34afylAJv0PZ0GRo2o7mIQVxR6f%2BCb2MuQUj3M%3D&reserved=0> Regards, -- Ilya Kasnacheev ср, 5 сент. 2018 г. в 15:44, Sriveena Mattaparthi mailto:sriveena.mattapar...@ekaplus.com>>: Hi, I am trying to batch insert 30 records into ignite cache(on remote server) using jdbc thin driver. It is taking nearly 4mins to complete this operation. Please advise. Thanks & Regards, Sriveena “Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.” “Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.”
Re: Batch insert into ignite using jdbc thin driver taking long time
Hello! Have you tried streaming mode? https://apacheignite-sql.readme.io/docs/set Regards, -- Ilya Kasnacheev ср, 5 сент. 2018 г. в 15:44, Sriveena Mattaparthi < sriveena.mattapar...@ekaplus.com>: > Hi, > > > > I am trying to batch insert 30 records into ignite cache(on remote > server) using jdbc thin driver. > > It is taking nearly 4mins to complete this operation. Please advise. > > > > Thanks & Regards, > > Sriveena > > > “Confidentiality Notice: The contents of this email message and any > attachments are intended solely for the addressee(s) and may contain > confidential and/or privileged information and may be legally protected > from disclosure. If you are not the intended recipient of this message or > their agent, or if this message has been addressed to you in error, please > immediately alert the sender by reply email and then delete this message > and any attachments. If you are not the intended recipient, you are hereby > notified that any use, dissemination, copying, or storage of this message > or its attachments is strictly prohibited.” >
Batch insert into ignite using jdbc thin driver taking long time
Hi, I am trying to batch insert 30 records into ignite cache(on remote server) using jdbc thin driver. It is taking nearly 4mins to complete this operation. Please advise. Thanks & Regards, Sriveena "Confidentiality Notice: The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited."
RE: Batch insert into ignite
Thanks Michael…I have a question here…
1. Will the set/copy options mentioned work for SQL Query fields?
Something like
. Cache.query(new SQLQueryFields(“SET STREAMING
ON”))
2. IgniteDataStreamer streams data and add to cache without using JDBC
driver.
But we would want to add the data stream into insert table. Is this possible?
Regards,
Sriveena
From: Michael Cherkasov [mailto:michael.cherka...@gmail.com]
Sent: Wednesday, July 25, 2018 6:23 PM
To: user@ignite.apache.org
Subject: Re: Batch insert into ignite
Hi again,
One more option as pure sql solution:
https://apacheignite-sql.readme.io/docs/copy<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fcopy&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cb6b82626c30540c84a7208d5f2305410%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636681211752818655&sdata=jn%2BOF9H0ydUVqQ8reY6dkQAFoBbq%2FpOyQWpkPYgWXLc%3D&reserved=0>
https://apacheignite-sql.readme.io/docs/set<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fset&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cb6b82626c30540c84a7208d5f2305410%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636681211752818655&sdata=18Gr7V7L7mB7Mj9dUbl0d4LqdoDVODrrA6w43XkiJVg%3D&reserved=0>
However, IgniteDataStreamer should work faster, because as I know, these SQL
commands have data streamer as underlying implementation.
Thanks,
Mike.
2018-07-25 7:02 GMT+03:00 Sriveena Mattaparthi
mailto:sriveena.mattapar...@ekaplus.com>>:
Thanks for the clarification Michael..will try the streamer option.
Regards,
Sriveena
From: Michael Cherkasov
[mailto:michael.cherka...@gmail.com<mailto:michael.cherka...@gmail.com>]
Sent: Tuesday, July 24, 2018 9:49 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Batch insert into ignite
I can't say for sure which one should be better for sure, it must be
benchmarked.
But if you can use SqlFieldsQuery this means that you can use native Ignite
API, right? So, in this case,
IgniteDataStreamer<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fdata-streamers&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd7a2e5237e04417740a408d5f1814956%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636680459959631292&sdata=Vm%2BmWgmalYFrA6O9fZEWPzW0XRF%2BOYFVaAIwus%2FNU5k%3D&reserved=0>
would be the best choice.
2018-07-24 9:36 GMT+03:00 Sriveena Mattaparthi
mailto:sriveena.mattapar...@ekaplus.com>>:
Hi Mikhail,
Do we have this streaming/batch insert option, if we do a bulk insert using
cache.query(new SqlFieldsQuery("sql");
Also could you please confirm which one would perform better
1. Insert using IgniteJdbcDriver
2. Insert using SqlFieldsQuery
Thanks & Regards,
Sriveena
-Original Message-
From: Mikhail
[mailto:michael.cherka...@gmail.com<mailto:michael.cherka...@gmail.com>]
Sent: Monday, July 23, 2018 10:32 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Batch insert into ignite
Hi Sriveena,
for data load you can try to use streaming mode for jdbc:
https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fjdbc-driver%23section-streaming-mode&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=QGfXouNn0Juxv18j1p0rgnBx%2Fle90Ioo3E6W3S6ohR8%3D&reserved=0
it will provide faster insert operation compare to regular insert, also you
can disable WAL:
https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fwrite-ahead-log%23section-wal-activation-and-deactivation&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=pjwp4tCzN9KnUq8s0cvTbmmaIDVj8O4UPylv7acYEUg%3D&reserved=0
to get better throughput.
Thanks,
Mike.
--
Sent from:
https://apac01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-ignite-users.70518.x6.nabble.com%2F&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=0Z7q%2BpGVuPa2rpB4wjYg4tKt5MPDYi4denqAbskOtZg%3D&reserved=0
Re: Batch insert into ignite
Hi again,
One more option as pure sql solution:
https://apacheignite-sql.readme.io/docs/copy
https://apacheignite-sql.readme.io/docs/set
However, IgniteDataStreamer should work faster, because as I know, these
SQL commands have data streamer as underlying implementation.
Thanks,
Mike.
2018-07-25 7:02 GMT+03:00 Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:
> Thanks for the clarification Michael..will try the streamer option.
>
>
>
> Regards,
>
> Sriveena
>
>
>
> *From:* Michael Cherkasov [mailto:michael.cherka...@gmail.com]
> *Sent:* Tuesday, July 24, 2018 9:49 PM
>
> *To:* user@ignite.apache.org
> *Subject:* Re: Batch insert into ignite
>
>
>
> I can't say for sure which one should be better for sure, it must be
> benchmarked.
>
> But if you can use SqlFieldsQuery this means that you can use native
> Ignite API, right? So, in this case, IgniteDataStreamer
> <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fdata-streamers&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd7a2e5237e04417740a408d5f1814956%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636680459959631292&sdata=Vm%2BmWgmalYFrA6O9fZEWPzW0XRF%2BOYFVaAIwus%2FNU5k%3D&reserved=0>
> would be the best choice.
>
>
>
>
>
> 2018-07-24 9:36 GMT+03:00 Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Hi Mikhail,
>
> Do we have this streaming/batch insert option, if we do a bulk insert
> using
> cache.query(new SqlFieldsQuery("sql");
>
> Also could you please confirm which one would perform better
> 1. Insert using IgniteJdbcDriver
> 2. Insert using SqlFieldsQuery
>
> Thanks & Regards,
> Sriveena
>
> -Original Message-
> From: Mikhail [mailto:michael.cherka...@gmail.com]
> Sent: Monday, July 23, 2018 10:32 PM
> To: user@ignite.apache.org
> Subject: Re: Batch insert into ignite
>
> Hi Sriveena,
>
> for data load you can try to use streaming mode for jdbc:
> https://apac01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fjdbc-
> driver%23section-streaming-mode&data=02%7C01%7CSriveena.Mattaparthi%
> 40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%
> 7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=
> QGfXouNn0Juxv18j1p0rgnBx%2Fle90Ioo3E6W3S6ohR8%3D&reserved=0
> it will provide faster insert operation compare to regular insert, also you
> can disable WAL:
> https://apac01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fwrite-
> ahead-log%23section-wal-activation-and-deactivation&
> amp;data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%
> 7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d
> 373c%7C1%7C0%7C636679621511638203&sdata=pjwp4tCzN9KnUq8s0cvTbmmaIDVj8O
> 4UPylv7acYEUg%3D&reserved=0
> to get better throughput.
>
> Thanks,
> Mike.
>
>
>
> --
>
> Sent from: https://apac01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fapache-ignite-users.70518.x6.nabble.com%2F&
> amp;data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%
> 7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d
> 373c%7C1%7C0%7C636679621511638203&sdata=0Z7q%
> 2BpGVuPa2rpB4wjYg4tKt5MPDYi4denqAbskOtZg%3D&reserved=0
>
>
>
RE: Batch insert into ignite
Thanks for the clarification Michael..will try the streamer option.
Regards,
Sriveena
From: Michael Cherkasov [mailto:michael.cherka...@gmail.com]
Sent: Tuesday, July 24, 2018 9:49 PM
To: user@ignite.apache.org
Subject: Re: Batch insert into ignite
I can't say for sure which one should be better for sure, it must be
benchmarked.
But if you can use SqlFieldsQuery this means that you can use native Ignite
API, right? So, in this case,
IgniteDataStreamer<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fdata-streamers&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd7a2e5237e04417740a408d5f1814956%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636680459959631292&sdata=Vm%2BmWgmalYFrA6O9fZEWPzW0XRF%2BOYFVaAIwus%2FNU5k%3D&reserved=0>
would be the best choice.
2018-07-24 9:36 GMT+03:00 Sriveena Mattaparthi
mailto:sriveena.mattapar...@ekaplus.com>>:
Hi Mikhail,
Do we have this streaming/batch insert option, if we do a bulk insert using
cache.query(new SqlFieldsQuery("sql");
Also could you please confirm which one would perform better
1. Insert using IgniteJdbcDriver
2. Insert using SqlFieldsQuery
Thanks & Regards,
Sriveena
-Original Message-
From: Mikhail
[mailto:michael.cherka...@gmail.com<mailto:michael.cherka...@gmail.com>]
Sent: Monday, July 23, 2018 10:32 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: Re: Batch insert into ignite
Hi Sriveena,
for data load you can try to use streaming mode for jdbc:
https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fjdbc-driver%23section-streaming-mode&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=QGfXouNn0Juxv18j1p0rgnBx%2Fle90Ioo3E6W3S6ohR8%3D&reserved=0
it will provide faster insert operation compare to regular insert, also you
can disable WAL:
https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fwrite-ahead-log%23section-wal-activation-and-deactivation&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=pjwp4tCzN9KnUq8s0cvTbmmaIDVj8O4UPylv7acYEUg%3D&reserved=0
to get better throughput.
Thanks,
Mike.
--
Sent from:
https://apac01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-ignite-users.70518.x6.nabble.com%2F&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=0Z7q%2BpGVuPa2rpB4wjYg4tKt5MPDYi4denqAbskOtZg%3D&reserved=0
Re: Batch insert into ignite
I can't say for sure which one should be better for sure, it must be
benchmarked.
But if you can use SqlFieldsQuery this means that you can use native Ignite
API, right? So, in this case, IgniteDataStreamer
<https://apacheignite.readme.io/docs/data-streamers> would be the best
choice.
2018-07-24 9:36 GMT+03:00 Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:
> Hi Mikhail,
>
> Do we have this streaming/batch insert option, if we do a bulk insert
> using
> cache.query(new SqlFieldsQuery("sql");
>
> Also could you please confirm which one would perform better
> 1. Insert using IgniteJdbcDriver
> 2. Insert using SqlFieldsQuery
>
> Thanks & Regards,
> Sriveena
>
> -Original Message-
> From: Mikhail [mailto:michael.cherka...@gmail.com]
> Sent: Monday, July 23, 2018 10:32 PM
> To: user@ignite.apache.org
> Subject: Re: Batch insert into ignite
>
> Hi Sriveena,
>
> for data load you can try to use streaming mode for jdbc:
> https://apac01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fjdbc-
> driver%23section-streaming-mode&data=02%7C01%7CSriveena.Mattaparthi%
> 40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%
> 7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=
> QGfXouNn0Juxv18j1p0rgnBx%2Fle90Ioo3E6W3S6ohR8%3D&reserved=0
> it will provide faster insert operation compare to regular insert, also you
> can disable WAL:
> https://apac01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fwrite-
> ahead-log%23section-wal-activation-and-deactivation&
> amp;data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%
> 7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d
> 373c%7C1%7C0%7C636679621511638203&sdata=pjwp4tCzN9KnUq8s0cvTbmmaIDVj8O
> 4UPylv7acYEUg%3D&reserved=0
> to get better throughput.
>
> Thanks,
> Mike.
>
>
>
> --
> Sent from: https://apac01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fapache-ignite-users.70518.x6.nabble.com%2F&
> amp;data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%
> 7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d
> 373c%7C1%7C0%7C636679621511638203&sdata=0Z7q%
> 2BpGVuPa2rpB4wjYg4tKt5MPDYi4denqAbskOtZg%3D&reserved=0
>
RE: Batch insert into ignite
Hi Mikhail,
Do we have this streaming/batch insert option, if we do a bulk insert using
cache.query(new SqlFieldsQuery("sql");
Also could you please confirm which one would perform better
1. Insert using IgniteJdbcDriver
2. Insert using SqlFieldsQuery
Thanks & Regards,
Sriveena
-Original Message-
From: Mikhail [mailto:michael.cherka...@gmail.com]
Sent: Monday, July 23, 2018 10:32 PM
To: user@ignite.apache.org
Subject: Re: Batch insert into ignite
Hi Sriveena,
for data load you can try to use streaming mode for jdbc:
https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fjdbc-driver%23section-streaming-mode&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=QGfXouNn0Juxv18j1p0rgnBx%2Fle90Ioo3E6W3S6ohR8%3D&reserved=0
it will provide faster insert operation compare to regular insert, also you
can disable WAL:
https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fwrite-ahead-log%23section-wal-activation-and-deactivation&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=pjwp4tCzN9KnUq8s0cvTbmmaIDVj8O4UPylv7acYEUg%3D&reserved=0
to get better throughput.
Thanks,
Mike.
--
Sent from:
https://apac01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-ignite-users.70518.x6.nabble.com%2F&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=0Z7q%2BpGVuPa2rpB4wjYg4tKt5MPDYi4denqAbskOtZg%3D&reserved=0
RE: Batch insert into ignite
Thanks Mikhail...will try the same and get back to you.. -Original Message- From: Mikhail [mailto:michael.cherka...@gmail.com] Sent: Monday, July 23, 2018 10:32 PM To: user@ignite.apache.org Subject: Re: Batch insert into ignite Hi Sriveena, for data load you can try to use streaming mode for jdbc: https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fjdbc-driver%23section-streaming-mode&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=QGfXouNn0Juxv18j1p0rgnBx%2Fle90Ioo3E6W3S6ohR8%3D&reserved=0 it will provide faster insert operation compare to regular insert, also you can disable WAL: https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2Fwrite-ahead-log%23section-wal-activation-and-deactivation&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=pjwp4tCzN9KnUq8s0cvTbmmaIDVj8O4UPylv7acYEUg%3D&reserved=0 to get better throughput. Thanks, Mike. -- Sent from: https://apac01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fapache-ignite-users.70518.x6.nabble.com%2F&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cd2ff72f68e13451756c008d5f0be11d6%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636679621511638203&sdata=0Z7q%2BpGVuPa2rpB4wjYg4tKt5MPDYi4denqAbskOtZg%3D&reserved=0
Re: Batch insert into ignite
Hi Sriveena, for data load you can try to use streaming mode for jdbc: https://apacheignite.readme.io/docs/jdbc-driver#section-streaming-mode it will provide faster insert operation compare to regular insert, also you can disable WAL: https://apacheignite.readme.io/docs/write-ahead-log#section-wal-activation-and-deactivation to get better throughput. Thanks, Mike. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Batch insert into ignite
Hi, How can we perform batch insert/update into ignite tables (created through sqlquery fields as create table)? Thanks & Regards, Sriveena
