Re: Loading data from Hive Table into Ignite Cache

[ravi]

Hi,
 Thanks for the reply. Using Automatic RDBMS Configuration, Does ignite web
console supports loading the data from Hive Database(No SQL DB- using Hive
Server2 Driver?).
 Can you share some links if any such a sample is available?

Regards
Ravi




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Loading-data-from-Hive-Table-into-Ignite-Cache-tp16367p16374.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Confused about QueryEntity configuration

[vkulichenko]

Franck,

#2 in Vasiliy's response suggested to use predefined '_key' field name to
reference the key value. This doesn't require additional configuration.

Moreover, this alias is wrong anyway:



Because 'key' here is the name of the field in binary object, while 'value'
is the alias. You have it other way around.

Hope this helps.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Confused-about-QueryEntity-configuration-tp16281p16371.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Cluster segmentation

[Biren Shah]

Hi Val,

Thanks for explaining the difference between events. After some reading I kind 
of figured that I don’t need to check anything once I get segmented event. You 
mentioned that node is the topology gets node fail/left event, then why I get 
node left event on segmented node? Look at the event timeline I have shared in 
the first post.

The application processes huge amount of time series data. we get the data at 
set interval. The data processing has two stages. Once we get data, I 
distribute them for first stage processing based on a key. After the first 
stage, we redistribute the data for second stage based on different key. For 
both stages, we have bunch of other metadata which are also in the distributed 
caches. Some of them are small so I have replicated them on all nodes. Some of 
them are huge which are partitioned. These metadata do not change a lot.

The other issue I faced was that once one of the node gets segmented, the other 
node dies. The reason it dies is because heap usage jumped on the other node 
instantly. It jumped from 4GM to 11 GB. Very strange. Any idea what could cause 
this?

Thanks,
Biren

On 8/21/17, 6:53 PM, "vkulichenko"  wrote:

Hi Biren,

What is the use case and what are you trying to achieve by all this?

First of all, there is a difference between node_left/failed and
node_segmented events. The former is fired on nodes that are still in
topology to notify that one of the nodes left or failed. But the latter
means that *local* node got segmented, and I don't think it makes sense to
do any checks there.

Segmentation can happen for various reasons, but in vast majority of cases
it's a long GC pause. In this case node does not close connections, but
becomes unresponsive, which causes the cluster to remove it from topology
after failure detection timeout. When GC pause finishes, node tries to
continue to operate, but realizes that it was already kicked out. It then
fires node_segmented event locally and stops immediately. This is correct
behavior.

-Val



--
View this message in context: 
https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_Cluster-2Dsegmentation-2Dtp16314p16351.html=DwICAg=Zok6nrOF6Fe0JtVEqKh3FEeUbToa1PtNBZf6G01cvEQ=rbkF1xy5tYmkV8VMdTRVaIVhaXCNGxmyTB5plfGtWuY=EsW9z3oSwgxZCeY4wMYDAG1DZSs6PrI_95QZkz5nrMk=137pm5de4sgSFexWBiXVHz-5keGP5OKYr-q74AlW5To=
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Questions about "ignite-indexing"

[slava.koptilin]

Hi Chris,

It looks like Dmitry already answered your qns:
http://apache-ignite-users.70518.x6.nabble.com/getAsMap-tt16242.html#a16347


> Hi Chris, 
> 
> 1. Indexing may slowdown insertions, but in will not be used if no
> CacheConfiguration.setIndexedTypes() or
> CacheConfiguration.setQueryEntities() set. 
> 
> 2. It depends on conditions. If you have a lot of data, but you need to
> filter out a small set of them, then indexing may greatly help you,
> because it will use index but full scan. In other hand, scan query may be
> greatly optimized if it's used locally and per partition, but again, it
> will check all entries. 
> 
> 3. If you set withKeepBinary(), then you'll be working with BinaryObject
> and it's not require any specific config. 
> 
> Thanks! 
> -Dmitry.





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Questions-about-ignite-indexing-tp16292p16369.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Custom SecurityCredentialsProvider and SecurityCredentials

[slava.koptilin]

Hi Franck,

Yes, Apache Ignite provides SecurityCredentialsBasicProvider out of the box,
which can be used if it is OK to specify credentials directly in
configuration,
and of course you can implement your own provider.
I think the following topic will be helpful
http://apache-ignite-users.70518.x6.nabble.com/Custom-GridSecurityProcessor-plugin-question-td4942.html

Thanks!




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Custom-SecurityCredentialsProvider-and-SecurityCredentials-tp16360p16368.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Loading data from Hive Table into Ignite Cache

[ravi]

Hi,
  We have the OLAP application which aggregates the data on configured
interval(day/week) and loads the data into hive tables(HDFS-SNAPPY-ORC File
format). We have the reporting engine which will query the data from these
tables using Hive/Spark SQL.
   Is it possible to load the data from Hive(Once it reaches to Hive Table)
to Ignite cache in a faster manner(Hive as CacheJDBCStore or using Data
Streamer API's or some other means), so that our reporting engine can run
ignite sql and get the benefit of indexing and other ignite native benefits
and respond faster?.
  We are not in favor of using IGFS and run Hive QL using Tez+LLAP+ORC. We
wanted to run native Ignite SQL.
  So please suggest any other alternate approach
  
Regards
Ravi
  



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Loading-data-from-Hive-Table-into-Ignite-Cache-tp16367.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: persistence data store

[Alexey Kukushkin]

Hi,

Ignite does not support backups but Ignite Persistence exposes API to
implement that. Implement
org.apache.ignite.internal.processors.cache.persistence.snapshot.IgniteCacheSnapshotManager
and add the implementation as Ignite plugin

There are commercial solutions implementing such a plugin.


On Tue, Aug 22, 2017 at 2:31 PM, luqmanahmad  wrote:

> Hi there,
>
> As of 2.0 data can be persisted to SSD I was wondering is there anything
> which can be used to store the data from it, save it somewhere and then
> load
> the data again on some other machine with the same configuration.
>
> Thanks,
> Luqman
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/persistence-data-store-tp16362.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Best regards,
Alexey

Re: ignite.active(true) blocking forever

[slava.koptilin]

Hi,

By default, each Ignite instance grabs 80% of available memory on startup,
that may be a cause the operating system to start swapping and slows
everything down.

As you mentioned, you are trying to start 2 JVMs on same machine,
so, it seems that you need to reduce memory size (it's a
defaultMemoryPolicySize property). 

[1]
https://apacheignite.readme.io/v2.1/docs/memory-configuration#memory-policies

Thanks!



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/ignite-active-true-blocking-forever-tp16346p16364.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Spring Bean Injection To IgniteRunnable Implementation

[sijusuresh]

I was trying to get an External Spring Context. IgniteSpring.start() helped.
Thank You



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Spring-Bean-Injection-To-IgniteRunnable-Implementation-tp16275p16363.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

persistence data store

[luqmanahmad]

Hi there,

As of 2.0 data can be persisted to SSD I was wondering is there anything
which can be used to store the data from it, save it somewhere and then load
the data again on some other machine with the same configuration.

Thanks,
Luqman





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/persistence-data-store-tp16362.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Error while starting service grid with persistent store enabled

[rishi007bansod]

Hi, 
  When error occured I have used following deployment method,

cfg.setName("myService_rishi");
cfg.setService(new codeToDeploy());
cfg.setTotalCount(9);
cfg.setMaxPerNodeCount(1);
ignite.active(true);
ignite.services().deploy(cfg);

but then I changed deployment method to, 

ignite.active(true);
IgniteServices svcs = ignite.services();
svcs.deployNodeSingleton("TestService", new SurveillanceAlert());

Also initially code I wrote in java file and xml file, configuration was
differing. Later I corrected it.

Thanks,
Rishikesh




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Error-while-starting-service-grid-with-persistent-store-enabled-tp15946p16361.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Custom SecurityCredentialsProvider and SecurityCredentials

[franck102]

Hi all,

I can't figure out how to install a custom security credentials provider?
The javadoc says 


... however GridClientConfiguration hardcodes an instance of
SecurityCredentialsBasicProvider, and ServerImpl directly gets credentials
from node.attributes(IgniteNodeAttributes.ATTR_SECURITY_CREDENTIALS).

Am I missing something, is it possible to install a custom provider that
uses a subclass of SecurityContext?

Thanks!
Franck



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Custom-SecurityCredentialsProvider-and-SecurityCredentials-tp16360.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Confused about QueryEntity configuration

[franck102]

Thanks Vasiliy!

#1 works fine, I can go with that

I can't get #2 to work, with this configuration:



I get the exception below:





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Confused-about-QueryEntity-configuration-tp16281p16359.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Error while starting service grid with persistent store enabled

[dkarachentsev]

Hi Rishikesh,

Could you please create a reproducer project with config that fails with
NPE? This will help to find a bug.

Thanks!
-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Error-while-starting-service-grid-with-persistent-store-enabled-tp15946p16358.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Confused about QueryEntity configuration

[Vasiliy Sisko]

I'm sorry. I found that you are subscribed but mail put your letter to spam




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Confused-about-QueryEntity-configuration-tp16281p16357.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Confused about QueryEntity configuration

[Vasiliy Sisko]

Hello @franck102

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.

'empNo' field is a key field and is not included into stored object
(org.apache.ignite.sample.model.Employees) for usage in query

To extract that field you can use one of the following solutions:
1) Configure an own key field alias in QueryEntity instance. F.e.: 

2) Use ‘_key’ alias for the key field in qyery. F.e.: select _key from
Employees
3) Include 'empNo' field into the stored object how you described in message
from Aug 18, 2017; 9:35pm



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Confused-about-QueryEntity-configuration-tp16281p16356.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.