1a. No. DataStreamer doesn't supports transactions. 1b. SQL doesn't support transactions and it is under active development for now [1].
2. DataStreamer updates is not propagated to CacheStore by default, you can force this with allowOverwrite option [2]. DataStreamer uses individual entry updates, but you can change this with setting your own reciever. DataStreamer sends update to primary and backups, and update cache directly, so there is no need to start cluster-wide operation on each update, that may costs. 4. Ignite has powerful invoke() method that allows you can implement your own logic in EntryProcessor. [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-3%3A+Transactional+SQL [2] https://apacheignite.readme.io/v1.0/docs/data-streamers#section-allow-overwrite On Sat, Mar 31, 2018 at 9:03 PM, Prasad Bhalerao < prasadbhalerao1...@gmail.com> wrote: > Can someone please reply? > > Thanks, > Prasad > > > On Sat, Mar 31, 2018, 9:02 AM Prasad Bhalerao < > prasadbhalerao1...@gmail.com> wrote: > >> Hi Andrey, >> >> I have similar requirement and I am using cache.putAll method to update >> existing entries or to insert new ones. >> I will be updating/inserting close to 3 million entries in one go. >> >> I am using wrte through approach to update/insert/delete the data in >> oracle tables. >> I am using cachestores writeAll/ deleteAll method to achieve this. >> >> I am doing this in single ignite distributed transaction. >> >> >> Now the question is, >> 1a) Can I use streamer in ignite transaction? >> 1b) Can I use ignite jdbc bulk update, insert, delete with ignite >> distributed transaction? >> >> 2) if I use streamer will it invoke cache store writeAll method? >> I meant does write through approach work with streamer. >> >> >> 3) If I use Jdbc bulk mode for cache update and insert or delete, will it >> invoke cache store's wrieAll and deleteAll method? >> Does write through approach work with jdbc bulk update/insert/ delete? >> >> >> 4) Does ignite have any apis on cache only for update purpose? Put/putAll >> will insert or overwrite. What if I just want to update existing entries ? >> >> >> Thanks, >> Prasad >> >> On Fri, Mar 30, 2018, 11:12 PM Andrey Mashenkov < >> andrey.mashen...@gmail.com> wrote: >> >>> Hi, >>> >>> Ignite has 2 JDBC drivers. >>> 1. Client driver [1] starts client node (has all failover features) and >>> you have to pass client node config in URL. >>> 2. Thin Driver [2] that connect directly to one of Ignite server node. >>> >>> So, you need a first one to be able to use streaming mode. >>> >>> [1] https://apacheignite-sql.readme.io/docs/jdbc-client-driver >>> [2] https://apacheignite-sql.readme.io/docs/jdbc-driver >>> >>> On Fri, Mar 30, 2018 at 1:16 PM, <linr...@itri.org.tw> wrote: >>> >>>> Hi Andrey, >>>> >>>> >>>> >>>> I am trying to run [2], as: >>>> >>>> // Register JDBC driver. >>>> >>>> Class.forName("org.apache.ignite.IgniteJdbcDriver"); >>>> >>>> // Opening connection in the streaming mode. >>>> >>>> Connection conn = DriverManager.getConnection(" >>>> jdbc:ignite:cfg://streaming=true@file:///etc/config/ignite-jdbc.xml"); >>>> >>>> >>>> >>>> However, I'm a bit confused about that setting in [2] about the >>>> ignite-jdbc.xml. >>>> >>>> >>>> >>>> I do not know how to find or create the xml, and here I run the ignite >>>> node via JVM. >>>> >>>> >>>> >>>> If I can write java code to produce the ignite-jdbc or not? Or only >>>> complete Spring XML configuration? >>>> >>>> >>>> >>>> By the way, I have tried the [1], that worked well. >>>> >>>> >>>> >>>> Finally, I still need to use the SQL as a client node, and quick write >>>> data into cache. >>>> >>>> >>>> >>>> Thank you for helping me >>>> >>>> >>>> >>>> Rick >>>> >>>> >>>> >>>> >>>> >>>> *From:* Andrey Mashenkov [mailto:andrey.mashen...@gmail.com] >>>> *Sent:* Thursday, March 29, 2018 6:20 PM >>>> *To:* user@ignite.apache.org >>>> *Subject:* Re: How to insert multiple rows/data into Cache once >>>> >>>> >>>> >>>> Hi, >>>> >>>> >>>> >>>> Try to use DataStreamer for fast cache load [1]. >>>> >>>> If you need to use SQL, you can try to use bulk mode updated via JDBC >>>> [2] >>>> >>>> >>>> >>>> >>>> >>>> Also a COPY SQL command [3] will be available in next 2.5 release. >>>> >>>> The feature is already in master, you can try to build from it. See >>>> example [4] >>>> >>>> . >>>> >>>> >>>> >>>> [1] https://apacheignite.readme.io/docs/data-streamers >>>> >>>> [2] https://apacheignite.readme.io/v2.0/docs/jdbc- >>>> driver#section-streaming-mode >>>> >>>> [3] https://issues.apache.org/jira/browse/IGNITE-6917 >>>> >>>> [4] https://github.com/apache/ignite/blob/master/examples/ >>>> src/main/java/org/apache/ignite/examples/sql/SqlJdbcCopyExample.java >>>> >>>> >>>> >>>> On Thu, Mar 29, 2018 at 11:30 AM, <linr...@itri.org.tw> wrote: >>>> >>>> Dear all, >>>> >>>> >>>> >>>> I am trying to use the SqlFieldsQuery sdk to insert data to one cache >>>> on Ignite. >>>> >>>> >>>> >>>> I can insert one data into one cache at a time. >>>> >>>> >>>> >>>> However, I have no idea to insert multiple rows/data into the cache >>>> once. >>>> >>>> >>>> >>>> For example, I would like to insert 1000 rows/data into the cache once. >>>> >>>> >>>> >>>> Here, I provide my code to everyone to reproduce my situation. >>>> >>>> *public* *class* IgniteCreateServer { >>>> >>>> *public* *class* Person { >>>> >>>> @QuerySqlField >>>> >>>> *private* String firstName; >>>> >>>> @QuerySqlField >>>> >>>> *private* String lastName; >>>> >>>> *public* Person(String firstName, String lastName) { >>>> >>>> *this*.firstName = firstName; >>>> >>>> *this*.lastName = lastName; >>>> >>>> } >>>> >>>> } >>>> >>>> *public* *static* *void* main(String[] args) { >>>> >>>> cacheConf.setName("igniteCache"); >>>> >>>> *cacheConf**.setIndexedTypes(String.**class**, String.**class**)*; >>>> >>>> cacheConf.setCacheMode(CacheMode.*REPLICATED*); >>>> >>>> cacheConf.setAtomicityMode(CacheAtomicityMode.*ATOMIC*); >>>> >>>> cfg.setCacheConfiguration(cacheConf); >>>> >>>> Ignite igniteNode = Ignition.*getOrStart*(cfg); >>>> >>>> *IgniteCache* cacheKeyvalue = *igniteNode**.getOrCreateCache(* >>>> *cacheConf**)*; >>>> >>>> >>>> >>>> *long* starttime, endtime; >>>> >>>> starttime = System.*currentTimeMillis*(); >>>> >>>> *int* datasize = 100000; >>>> >>>> *for* (*int* i = 0; i < datasize; i++) { >>>> >>>> cacheKeyvalue.put("key " + Integer.*toString*(i), Integer. >>>> *toString*(i)); >>>> >>>> } >>>> >>>> endtime = System.*currentTimeMillis*(); >>>> >>>> System.*out*.println("write " + datasize + " pairkeyvalue data: >>>> spend " + (endtime - starttime) + "milliseconds"); >>>> >>>> //========================================================== >>>> ======================================================= >>>> >>>> >>>> >>>> cacheCfg.setName("personCache"); >>>> >>>> cacheCfg.setIndexedTypes(String.*class*, Person.*class*); >>>> >>>> cacheCfg.setCacheMode(CacheMode.*REPLICATED*); >>>> >>>> cacheCfg.setAtomicityMode(CacheAtomicityMode.*ATOMIC*); >>>> >>>> *IgniteCache* cacheKeyTable = igniteNode.getOrCreateCache(cacheCfg); >>>> >>>> >>>> >>>> *long* starttime1, endtime1; >>>> >>>> starttime1 = System.*currentTimeMillis*(); >>>> >>>> *for* (*int* i = 0; i < datasize; i++) { >>>> >>>> cacheKeyTable.query(*new* SqlFieldsQuery("INSERT INTO >>>> Person(_key,firstName,lastName) VALUES(?,?,?)") >>>> >>>> .setArgs(i, "key " + Integer.*toString*(i), Integer.*toString*(i))); >>>> >>>> } >>>> >>>> endtime1 = System.*currentTimeMillis*(); >>>> >>>> System.*out*.println("write" + datasize + " table data: spend " + ( >>>> endtime1 - starttime1) + "milliseconds"); >>>> >>>> } >>>> >>>> >>>> >>>> The my code show as: >>>> >>>> “write 100000 pairkeyvalue data: spend 4734 milliseconds >>>> >>>> write 100000 table data: spend 2846 milliseconds” >>>> >>>> >>>> >>>> From the above result, I feel that using the SQL to insert data to >>>> cache is faster than using cache.getall(). >>>> >>>> >>>> >>>> I am not sure if this is correct or not? >>>> >>>> >>>> >>>> In addition, that is important for me to insert data into cache via the >>>> use of SQL, >>>> >>>> so I would like to insert multiple rows/data to accelerate it. >>>> >>>> >>>> >>>> if any further information is needed, I am glad to be informed and will >>>> provide to you as soon as possible. >>>> >>>> >>>> >>>> Thanks >>>> >>>> >>>> >>>> Rick >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain >>>> confidential information. Please do not use or disclose it in any way and >>>> delete it if you are not the intended recipient. >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Best regards, >>>> Andrey V. Mashenkov >>>> >>>> >>>> -- >>>> 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain >>>> confidential information. Please do not use or disclose it in any way and >>>> delete it if you are not the intended recipient. >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Andrey V. Mashenkov >>> >> -- Best regards, Andrey V. Mashenkov
