Hi Calvin, Cache.size and SELECT COUNT(*) are not always equal in Ignite. Could you please tell what arguments did you pass to IgniteDataStreamer.addData method?
Sent from my iPhone > On 16 May 2019, at 23:40, Banias H <banias4sp...@gmail.com> wrote: > > Hello Ignite experts, > > I am very new to Ignite. I am trying to ingest 15M rows of data using > DataStreamer into a table in a two-node Ignite cluster (v2.7) but run into > problems of not getting the data through running SQL on DBeaver. > > Here is the list of steps I took: > > 1. Start up two nodes using the following xml. > > <?xml version="1.0" encoding="UTF-8"?> > <beans xmlns="http://www.springframework.org/schema/beans"; > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation=" > http://www.springframework.org/schema/beans > http://www.springframework.org/schema/beans/spring-beans.xsd";> > <bean class="org.apache.ignite.configuration.IgniteConfiguration"> > <!-- Enabling Apache Ignite native persistence. --> > <property name="dataStorageConfiguration"> > <bean class="org.apache.ignite.configuration.DataStorageConfiguration"> > <property name="defaultDataRegionConfiguration"> > <bean > class="org.apache.ignite.configuration.DataRegionConfiguration"> > <property name="persistenceEnabled" value="true"/> > </bean> > </property> > </bean> > </property> > <property name="discoverySpi"> > <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> > <property name="ipFinder"> > <bean > class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder"> > <property name="addresses"> > <list> > <value>IP1</value> > <value>IP2</value> > </list> > </property> > </bean> > </property> > </bean> > </property> > <property name="cacheConfiguration"> > <bean class="org.apache.ignite.configuration.CacheConfiguration"> > <property name="name" value="FOO"/> > <property name="cacheMode" value="PARTITIONED"/> > <property name="backups" value="0"/> > </bean> > </property> > </bean> > </beans> > > 2. Use python thin client to get the cache SQL_PUBLIC_FOO and insert ten row > of data. After this step, both thin client and DBeaver SQL client report the > same count: > > - thin client: > > nodes = [ > (IP1, 10800), > (IP2, 10800), > ] > client = Client() > client.connect(nodes) > cache = client.get_cache("SQL_PUBLIC_FOO") > print(cache.get_size()) > > returns 10 > > - SQL through DBeaver > > SELECT COUNT(*) FROM FOO > > returns 10 > > 3. However when I tried using DataStreamer to ingest 100 rows into the cache > SQL_PUBLIC_FOO, only thin client showed new count value and SQL returned old > count value: > > - ingesting through DataStreamer > //I ran the jar on one of the Ignite nodes > String CONFIG_FILE = <path to the xml file shown above>; > Ignition.setClientMode(true); > Ignite ignite = Ignition.start(CONFIG_FILE); > IgniteDataStreamer<Integer, String> stmr = > ignite.dataStreamer("SQL_PUBLIC_FOO"); > stmr.addData(rowCount, value); > > - thin client: > > nodes = [ > (IP1, 10800), > (IP2, 10800), > ] > client = Client() > client.connect(nodes) > cache = client.get_cache("SQL_PUBLIC_FOO") > cache.get_size() > > returns 110 > > - SQL through DBeaver > > SELECT COUNT(*) FROM FOO > > returns 10 > > Would anyone shed some lights on what I did wrong? I would love to use > DataStreamer to put much more data into the cache so that I would wan to be > able to query them through SQL. > > Thanks for the help. I appreciate it. > > Regards, > Calvin >
