Re: unable to query the cache after restart
It didn't work as expected, tried disable/enable WAL too, still I see the
same behavior.
On Mon, Jun 24, 2019 at 7:32 AM Павлухин Иван wrote:
> Hi Goutham,
>
> I did not get from your last message does it work now as expected? If
> not does it work without a trick with disable/enable WAL?
>
> ср, 19 июн. 2019 г. в 19:18, goutham manchikatla :
> >
> > Hi Denis,
> >
> > I tried removing Default_Region name property from config, still I see
> the same behavior. But when I trigger load_cache process, and query the
> cache(after a complete cluster restart) , I am getting the response.
> >
> > Below is the Load_cache process.
> > try (Ignite ignite = Ignition.start(configFile)) {
> >
> >
> > // Start loading cache on all caching nodes.
> >
> > final IgniteCache cache = ignite.cache(cacheName);
> >
> > long ts = System.currentTimeMillis();
> >
> >
> > IgniteCluster cluster = ignite.cluster();
> >
> > cluster.disableWal(cacheName);
> >
> >
> > LOG.info("Disabling WAL for Initial Data Loading");
> >
> >
> > cache.loadCache(null, ignite, cacheName, sqlquery);
> >
> >
> > LOG.info("Loaded Cache in " + (System.currentTimeMillis() - ts) + "
> millisecs");
> >
> >
> > cluster.enableWal(cacheName);
> >
> >
> > LOG.info("Enabling WAL after the pre-loading is complete");
> >
> > }
> >
> >
> > Thanks,
> >
> > Goutham
> >
> >
> > On Tue, Jun 18, 2019 at 8:45 PM Denis Magda wrote:
> >>
> >> Try to remove Default_Region name property from your config. If to
> follow this example that’s how persistence is enabled for the default
> region:
> >> https://apacheignite.readme.io/docs/distributed-persistent-store
> >>
> >> Denis
> >>
> >>
> >> On Thursday, June 6, 2019, goutham manchikatla
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I didn't change any code between restarts. Below is the configuration.
> >>>
> >>>
> >>> class="org.apache.ignite.configuration.DataStorageConfiguration">
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> class="org.apache.ignite.configuration.DataRegionConfiguration">
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> class="org.apache.ignite.configuration.DataRegionConfiguration">
> >>> value="500MB_Region"/>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> value="true"/>
> >>>
> >>> value="RANDOM_LRU"/>
> >>>
> >>> value="#{1024L * 1024 * 1024}"/>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> class="org.apache.ignite.configuration.CacheConfiguration">
> >>> value="500MB_Region"/>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
> >>> value="com.cachestore.AccountCacheStore">
> >>>
> >>>
> >>>
> >>>
> >>> class="org.apache.ignite.cache.QueryEntity">
> >>> value="java.lang.String">
> >>> value="com.domain.Account">
> >>>
> >>>
> >>> value="java.lang.String">
> >>> value="java.lang.String">
> >>> value="java.lang.String">
> >>> value="java.lang.String">
> >>> value="java.lang.String">
> >>> value="java.lang.String">
> >>> value="java.lang.String">
> >>> value="java.lang.String">
> >>> value="java.lang.String">
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
Evaluating Ignite, is this possible
Hello everyone. We're thinking about using ignite. We have a need to transform some data in a MySQL database. We have a table like Data Table: id val0_cv_id val1_cv_id val2_cv_id 1 2 2 1 2 2 3 null the cv_id columns are foreign keys to this table: Code Values id value_name 1 feature_z 2 feature_a 3 feature_b The tool the data science team needs the data in the database to be in the folowing format: data_table_row_id has_feature_a has_feature_b has_feature_z 1 101 2 110 Essentially, rotating the data 90 degrees about the origin and "binarizing" it. Is it possible for Ignite to do this, and then allow third party tools (Like a SQL client or Metabase or Tableau) to connect to the transformed Ignite dataset? I see in the documentation there is a preprocessor that does binarization, but I'm unsure if the result dataset can be offered as a service. Thank you! -Jonathan -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Canceling a running query on ignite
Hi, How do you execute your SQL queries? Java API SqlFieldsQuery has a setTimeout method. Also there is a QueryCursor.cancel method. Also KILL query SQL command [1] is targeted for 2.8. [1] https://issues.apache.org/jira/browse/IGNITE-11564 пт, 21 июн. 2019 г. в 18:41, Ph Tham : > > Hello All, > > When we run a complex SQL query on ignite cluster from a client, not clear on > what is expected? Does it run forever or can we specify a timeout? Is there a > way it can be canceled so that it does not bring the server down? > > Thanks -- Best regards, Ivan Pavlukhin
Re: Can we avoid the PME when restarts a node in cluster.
It worth noting that IGNITE-9420 seems to be relevant for clusters with enabled persistence. пт, 21 июн. 2019 г. в 12:57, Ilya Kasnacheev : > > Hello! > > 2.7.5 definitely did not include it and there is no set date for 2.8. > > Regards, > > пт, 21 июн. 2019 г., 6:50 Justin Ji : >> >> 2.7.5 may not include the feature~ >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ -- Best regards, Ivan Pavlukhin
Re: Sql Query Based On DateTime in .Net
Hi, I would suggest you to move your business logic out of queryable fields. Put [QuerySqlField] to plain _JoinDate and rewrite your sql to "where _JoinDate = ..." This should work as your *cache.Get() result* example. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: client reconnect working?
I tried non-zero values many times and the result is the same. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: unable to query the cache after restart
Hi Goutham,
I did not get from your last message does it work now as expected? If
not does it work without a trick with disable/enable WAL?
ср, 19 июн. 2019 г. в 19:18, goutham manchikatla :
>
> Hi Denis,
>
> I tried removing Default_Region name property from config, still I see the
> same behavior. But when I trigger load_cache process, and query the
> cache(after a complete cluster restart) , I am getting the response.
>
> Below is the Load_cache process.
> try (Ignite ignite = Ignition.start(configFile)) {
>
>
> // Start loading cache on all caching nodes.
>
> final IgniteCache cache = ignite.cache(cacheName);
>
> long ts = System.currentTimeMillis();
>
>
> IgniteCluster cluster = ignite.cluster();
>
> cluster.disableWal(cacheName);
>
>
> LOG.info("Disabling WAL for Initial Data Loading");
>
>
> cache.loadCache(null, ignite, cacheName, sqlquery);
>
>
> LOG.info("Loaded Cache in " + (System.currentTimeMillis() - ts) + "
> millisecs");
>
>
> cluster.enableWal(cacheName);
>
>
> LOG.info("Enabling WAL after the pre-loading is complete");
>
> }
>
>
> Thanks,
>
> Goutham
>
>
> On Tue, Jun 18, 2019 at 8:45 PM Denis Magda wrote:
>>
>> Try to remove Default_Region name property from your config. If to follow
>> this example that’s how persistence is enabled for the default region:
>> https://apacheignite.readme.io/docs/distributed-persistent-store
>>
>> Denis
>>
>>
>> On Thursday, June 6, 2019, goutham manchikatla wrote:
>>>
>>> Hi,
>>>
>>> I didn't change any code between restarts. Below is the configuration.
>>>
>>>
>>> >> class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>
>>>
>>>
>>>
>>>
>>> >> class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>
>>>
>>>
>>>
>>>
>>>
>>> >> class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>
>>>
>>>
>>>
>>>
>>>
>>> >> value="true"/>
>>>
>>> >> value="RANDOM_LRU"/>
>>>
>>> >> value="#{1024L * 1024 * 1024}"/>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> >> class="org.apache.ignite.configuration.CacheConfiguration">
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> >> class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
>>> >> value="com.cachestore.AccountCacheStore">
>>>
>>>
>>>
>>>
>>> >> class="org.apache.ignite.cache.QueryEntity">
>>> >> value="java.lang.String">
>>> >> value="com.domain.Account">
>>>
>>>
>>> >> value="java.lang.String">
>>> >> value="java.lang.String">
>>> >> value="java.lang.String">
>>> >> value="java.lang.String">
>>> >> value="java.lang.String">
>>> >> value="java.lang.String">
>>> >> value="java.lang.String">
>>> >> value="java.lang.String">
>>> >> value="java.lang.String">
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Jun 6, 2019 at 8:32 AM Ilya Kasnacheev
>>> wrote:
Hello!
This is strange. What's cache configuration? Is there a reproducer? Did
you change the code between restarts, including key/value types, if any?
Regards,
--
Ilya Kasnacheev
чт, 6 июн. 2019 г. в 16:16, goutham manchikatla :
>
> Yes , the query worked before restart.
>
> On Thu, Jun 6,
Re: Failed to run map query remotely.Failed to execute map query on the node
Hi Max, Thanks for your response ! Due to our data policy, I will not able share complete server logs. But, I have checked that logs, there is no exception thrown at the time of the incident. Only ignite sql line client was able to throw exception, which I have put in previous thread. No such specific scenario. I had even checked for server memory outage at that time, but not an issue of it. It has not given such error after that incident till now. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: client reconnect working?
I think the problem could be with communicationSpi config. You specified connectTimeout as 0, which is interpreted as infinite timeout. Try to change it to a non-zero value. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Failed to run map query remotely.Failed to execute map query on the node
Hi, Please share full logs from all server nodes. It has to be an exception with a stacktrace related to the incident. Do you have a scenario to reproduce this issue? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Sudden cluster failure
It recovered, but recovery lasts ~2 hours and customer says that there is no crash on their network, other systems work fine -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Limited set of DataStructure compared to REDIS
I think you’d do operations like that using SQL, i.e., you’d use the SqlFieldsQuery. Regards, Stephen > On 24 Jun 2019, at 11:45, dhiman_nikhil wrote: > > I see many DS not present in Ignite like zrevrangeByScore, sortedSet, etc. > Is there a way I can get all D.S. provided by Redis in Ignite. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Limited set of DataStructure compared to REDIS
I see many DS not present in Ignite like zrevrangeByScore, sortedSet, etc. Is there a way I can get all D.S. provided by Redis in Ignite. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Failed to run map query remotely.Failed to execute map query on the node
Hi, I am using partitioned caches in my ignite 2.6 cluster. Faced this issue, that the ignite cache query (using jdbc ) was unable to retrieve even 'Select count(*) query results'. Though the '!tables command' did display the names of my 3 caches. The ignite server was also running without any exceptions being thrown. While querying, below exception was generated in jdbc:ignite sqlline: java.sql.SQLException: javax.cache.CacheException: Failed to run map query remotely.Failed to execute map query on the node org.apache.ignite.IgniteCheckedException:Failed to parse SQL query: SELECT COUNT(*) __C0_0 This issue did not show up once the ignite cluster was restarted. What could be the cause of such behaviour by Ignite -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: too many dirty pages
Hi, It's not a warning, it's just an INFO message about writing checkpoint to the disk. The reason for this checkpoint was that there were "too many dirty pages". Here is how Ignite persistence works: https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood Best Regards, Evgenii вс, 23 июн. 2019 г. в 13:36, KR Kumar : > Hi - I have ignite persistence enabled cache that has about 100 cache > tables > that i created using Ignite SQL. When I am writing data ( lot of data ) in > these tables, I see these warnings in the logs and writes hang for some > time > and then resume. > > [2019-06-23 10:12:47,647][INFO > ][db-checkpoint-thread-#144][GridCacheDatabaseSharedManager] Checkpoint > started [checkpointId=d067e58d-d62d-4914-9ce8-531369e08c33, > startPtr=FileWALPointer [idx=214, fileOff=115119, len=2123305], > checkpointLockWait=0ms, checkpointLockHoldTime=1727ms, > walCpRecordFsyncDuration=2305ms, pages=304819, reason='too many dirty > pages'] > > Any pointers, please let me know ?? > > Thanx and Regards, > KR Kumar > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
