Re: Slow Ignite Cache with multi-nodes

[Roman Guseinov]

Hi Rick,

It seems that you need to increase clients number to achieve better
performance when you increase server nodes number. You can try to put data
from different threads as well.

Also, you can set
CacheConfiguration.setWriteSynchronizationMode(CacheWriteSynchronizationMode.PRIMARY_SYNC).
The default value is FULL_SYNC which means that client node will wait for
write or commit to complete on all participating remote nodes (primary and
backup) [1].

Performance Tips can be found here [2].

Best Regards,
Roman

[1]
https://apacheignite.readme.io/v1.1/docs/primary-and-backup-copies#section-synchronous-and-asynchronous-backups
[2] https://apacheignite.readme.io/docs/performance-tips



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

答复: Hi is ignite have a plan for materialized view

[??′ ??????]

Hi
  Thanks for u answer.
  Actually, I just wanna use ignite as streaming database.
 So if it have materialized view. May helpful.


发送自 Windows 10 版邮件应用


发件人: Andrey Mashenkov 
发送时间: Friday, March 30, 2018 10:19:43 PM
收件人: user@ignite.apache.org
主题: Re: Hi is ignite have a plan for materialized view

Hi,

Ignite has no supported view.
We have no plans to add materialized view support in a meduim term perspective
as it distributed implementation looks too complex and low performant.

On Fri, Mar 30, 2018 at 9:51 AM, ??′ ?? 
> wrote:
Hi
   I wanna to use ignite like stream database。

  I wanna to insert some logs like sale log。

  and then ignite can let me create a materialized view to auto upate the count 
number or sum num number 。

  I insert a salelog ：(time,vmcode,product)value('2018-03-30 
14:00:00','21321','2123')

 a materialized view "vmSaleCountRealTime" like select 
date(time),vmcode,count(1) todaySaleCount from salelog group by 
date(time),vmcode

  ignite can auto update the todaySaleCount value  of record 
dt=2018-03-20,vmcode=21321

 Is thera have a plan for  materialized view base logs？

发送自 Windows 10 版邮件应用




--
Best regards,
Andrey V. Mashenkov

Regarding Ignite Slack Channel

[linrick]

Hi all,



Can someone please add me to the Ignite slack channel?



Thanks



Rick



--
本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain 
confidential information. Please do not use or disclose it in any way and 
delete it if you are not the intended recipient.

Slow Ignite Cache with multi-nodes

[linrick]

Dear all,

Could someone help me to figure out this problem at 
https://issues.apache.org/jira/browse/IGNITE-8117

Thanks

Rick


--
本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain 
confidential information. Please do not use or disclose it in any way and 
delete it if you are not the intended recipient.

Re: Cache access based on the user previleges

[Denis Magda]

Vishwas,

I guess that it's a matter of time when the role-based access gets added to
Ignite.

Version 2.5 already goes with username/password authentication:
https://issues.apache.org/jira/browse/IGNITE-7436

Hope that much more is to come.

--
Denis

On Fri, Apr 6, 2018 at 6:45 AM, Denis Mekhanikov 
wrote:

> Vishwas,
>
> It depends on what you want to achieve by this isolation.
>
> If you want to create separation between data of different applications
> and dedicate some limited volume for each one, then it could be a right
> approach.
> If your applications doesn't have any data dependencies, then you can even
> setup different clusters for them.
>
> Denis
>
> пт, 6 апр. 2018 г. в 4:44, vbm :
>
>> Hi Denis,
>>
>> Thanks for the reply.
>> Is this not needed. How does the isolation between different applications
>> happen ?
>>
>> One thing that comes to my mind is, we can create different data regions
>> and
>> put the cache needed for specific application in these data regions. Is
>> this
>> the right approach ?
>>
>>
>> Regards,
>> Vishwas
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

Re: POC with DataStreamer (Kafka or Ignite - security question)

[Denis Magda]

Built-in encryption facilities are being discussed on the dev list. Check
it up:
http://apache-ignite-developers.2346864.n4.nabble.com/IEP-18-Transparent-Data-Encryption-td29001.html

As for now, use operating system level encryption mechanics for your files.

--
Denis
--
Denis

On Fri, Apr 6, 2018 at 5:39 PM, Gaurav Bajaj  wrote:

> Also I don't think ignite provides any kind encryption for these db files.
>
> Best Regards,
> Gaurav
>
> On 06-Apr-2018 8:23 PM, "David Harvey"  wrote:
>
>> Assuming Ignite Persistence,  you can create a cache in a specific Data
>> Regions, but I'm unclear whether this properties can be set per region.  We
>> are setting them in
>>
>> org.apache.ignite.configuration.DataStorageConfiguration.   What you
>> seem to be asking for is to set these per Data Region.
>>
>>
>>> "/IgnitePersistenceStorage/store"/>
>>
>> 
>>
>>> "/IgnitePersistenceStorage/wal/archive"/>
>>
>>
>>
>>
>> On Fri, Apr 6, 2018 at 1:58 PM, Wilhelm 
>> wrote:
>>
>>> Hello,
>>>
>>> I'm building a POC, right now I have Kafka feeding ignite with the
>>> constraint of having 1 topic per tenant for security reason (historical
>>> data
>>> is persisted to files per topic by Kafka and each "file" container is
>>> encrypted differently per customer)
>>>
>>> If I decide to use only ignite with the DataStream (instead of Kafka),
>>> how
>>> can I make sure the customer data will be separated to disk (like
>>> separate
>>> file or db per customer)? and how can I encrypt this cache file per
>>> customer?
>>>
>>> I hope that make sense. So I guess it come down to: can I have the ignite
>>> cache persisted in different file/db per some rules (like a customer
>>> id)? or
>>> does it needs to be in different ignite memory caches? or it is not
>>> possible?
>>>
>>> Thanks for your help
>>>
>>> w
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>
>>
>>
>> *Disclaimer*
>>
>> The information contained in this communication from the sender is
>> confidential. It is intended solely for use by the recipient and others
>> authorized to receive it. If you are not the recipient, you are hereby
>> notified that any disclosure, copying, distribution or taking action in
>> relation of the contents of this information is strictly prohibited and may
>> be unlawful.
>>
>> This email has been scanned for viruses and malware, and may have been
>> automatically archived by *Mimecast Ltd*, an innovator in Software as a
>> Service (SaaS) for business. Providing a *safer* and *more useful* place
>> for your human generated data. Specializing in; Security, archiving and
>> compliance. To find out more Click Here
>> .
>>
>

Re: Query on Ignite web console

[Denis Magda]

Hi Vishwas,

Ignite Web Console is open sourced:
https://apacheignite-tools.readme.io/docs

At the same time, there is a version built out by GridGain that goes with
extra features and hosted only (can be used free of charge):
https://console.gridgain.com

--
Denis

On Mon, Apr 9, 2018 at 7:02 AM, vbm  wrote:

> Hi,
>
> I wanted to use Ignite web console for my POC.
> Can you please let me know, if it is licensed or open-sourced ?
>
>
> Regards,
> Vishwas
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Is backup copy always stored on different host - is ithost-awarereplication

[Denis Magda]

Hi Naveen,

Taken from CREATE TABLE documentation - see Ignite.addCacheConfiguration
parameter


   - TEMPLATE= - case-sensitive name of a cache
   template registered in Ignite to use as a configuration for the distributed
   cache that is deployed by the CREATE TABLE command. A template is an
   instance of the CacheConfiguration class registered with
   Ignite.addCacheConfiguration in the cluster. Use predefined
   TEMPLATE=PARTITIONED or TEMPLATE=REPLICATEDtemplates to create the cache
   with the corresponding replication mode. The rest of the parameters will be
   those that are defined in the CacheConfiguration object. By default,
   TEMPLATE=PARTITIONED is used if the template is not specified explicitly.

https://apacheignite-sql.readme.io/docs/create-table

On Mon, Apr 9, 2018 at 8:49 AM, Naveen  wrote:

> Hi Stan
>
> I wanted to create a cache template with the below
>
>  class="org.apache.ignite.configuration.CacheConfiguration">
> 
>  value="PARTITIONED" />
> 
> 
>  class="org.apache.ignite.cache.affinity.rendezvous.
> RendezvousAffinityFunction">
>  name="excludeNeighbors" value="true"/>
> 
> 
> 
>
> How do I use this template while creating the cache, what is the property I
> should set
>
>  class="org.apache.ignite.configuration.CacheConfiguration">
> 
>
>
> Thanks
> Naveen
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Readonly nodes

[Denis Magda]

Hi,

Yes, you can tap into NodeFilter interface applying it for your
CacheConfiguration in a way similar to ServiceConfiguration as shown here:
https://apacheignite.readme.io/docs/service-grid#section-node-filter-based-deployment

--
Denis

On Mon, Apr 9, 2018 at 4:23 PM, Alew  wrote:

> Hi!
>  I have a relicated cache. Is there a way to forbid insert and update data
> in the cache for some nodes in a cluster?
>
> Regards
>
>

Readonly nodes

[Alew]

Hi!
 I have a relicated cache. Is there a way to forbid insert and update 
data in the cache for some nodes in a cluster?


Regards

Re: Transaction progress, affinity, locks

[Ariel Tubaltsev]

Hi Pavel

Makes sense

Thank you
Ariel



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Affinity Run Return value

[grvm20]

NVM Got it



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Affinity Run Return value

[grvm20]

Hi,

I understand about Affinity Collocation and Affinity run.
I have a use case where I need to access multiple entries in the cache
related to a certain ID. Affinity Collocation works perfectly. However my
use case if to also return certain value based on all the entries i access.
How can I return value from the affinity run? If its not possible is there
any other way I can use Affinity Collocation and return value based on a
lambda run?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite cache get operation not working, client application stucks at the initializaiton

[Evgenii Zhuravlev]

>class org.apache.ignite.IgniteCheckedException: Log4j configuration path
was not found: log4j.xml
As you see, an application can't find your configuration file log4j.xml.
You need to configure it properly.

First of all, I want to say that you're moving something definitely wrong
between nodes:
>Caused by: java.lang.ClassNotFoundException: Optimized stream class
checksum
>mismatch (is same version of marshalled class present on all nodes?)
>[expected=7739, actual=53, cls=class java.io.FileDescriptor]
I'm pretty sure that this class( java.io.FileDescriptor) shouldn't be
serialized. I'd recommend checking your model - most possible that you have
in your classes fields with the wrong objects.

As for the same files - you need to have the same libs with the same
version for both servers and clients, doesn't matter how you start it.

Evgenii


2018-04-09 10:33 GMT+03:00 Priyanka Shinde :

> Hi,
>
> *You need to find field "ht" in your objects*
>
> I am not quite sure what is field "ht" exactly means and how to find that
> in
> my objects? I directly search for the exact keyword "ht" in my objects but
> did not find anything.
>
> *It's needed to have the same files for both client and server nodes. *
>
> I run the instance of Ignite using ignite.bat which runs the ignite
> instance
> in server mode by default and then I run the client applications using
> ignite.setClient(true) which runs the application in ignite client mode. So
> Both server and client can not have the same files but 2 clients are
> sharing
> the same files.
>
> *
> I've asked in the previous message to provide logs from client nodes, could
> you please share it? *
>
> As I mentioned previously, When I run the application using run
> configuration "-DIGNITE_QUIET=false" Infinite client logs gets printed in
> the console which can not be shared, though I have shared the exact
> exception to you from client logs.
>
> Also I have tried to configure logging by referring below link
> https://apacheignite.readme.io/docs/logging.
> included dependency,
> 
> org.apache.ignite
> ignite-log4j
> 2.3.0
> 
>
> and provided below property in config.xml file,
> 
> 
>   
> 
>   
>
> Also mentioned log4j.xml in the resources folder of the applicaiton. but
> getting below error while running the application.
>
> org.apache.ignite.IgniteException: Failed to instantiate Spring XML
> application context
> [springUrl=file:/D:/cache/apache-ignite-fabric-2.3.0-
> bin/apache-ignite-fabric-2.3.0-bin/config/default-config.xml,
> err=Error creating bean with name 'grid.cfg' defined in URL
> [file:/D:/cache/apache-ignite-fabric-2.3.0-bin/apache-
> ignite-fabric-2.3.0-bin/config/default-config.xml]:
> Cannot create inner bean 'org.apache.ignite.logger.
> log4j.Log4JLogger#48d85e'
> of type [org.apache.ignite.logger.log4j.Log4JLogger] while setting bean
> property 'gridLogger'; nested exception is
> org.springframework.beans.factory.BeanCreationException: Error creating
> bean
> with name 'org.apache.ignite.logger.log4j.Log4JLogger#48d85e' defined in
> URL
> [file:/D:/cache/apache-ignite-fabric-2.3.0-bin/apache-
> ignite-fabric-2.3.0-bin/config/default-config.xml]:
> Bean instantiation via constructor failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to
> instantiate
> [org.apache.ignite.logger.log4j.Log4JLogger]: Constructor threw exception;
> nested exception is class org.apache.ignite.IgniteCheckedException: Log4j
> configuration path was not found: log4j.xml]
>
>
> Appreciate your early response.
> Thanks in advance.
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Transaction progress, affinity, locks

[Pavel Vinokurov]

Hi Arial,

>> But there are also non-transactional reads from one of the caches.
Any operation on a transaction cache executes within a transaction. If the
transaction isn't declared, it is created implicitly with default
concurrency and isolation modes.


>> There will be only 2PC commit for transactions, no 1-phase commit, since
I
>> have 3 nodes, REPLICATED?
Yes 2PC commit will be used.

>>During the transaction, even for get, locks will be taken on backups
>>during the prepare phase, since it's PESSIMISTIC concurrency?
Locks will be taken during a transaction before read/write operations.


>> readFromBackup - looks like it's not going to improve transactions
>> performance since transaction coordinator still going to always send it
to
>> primary node and primary node will instruct backup to take locks?
>>* readFromBackup - still should help in non-transactional read, no? or it
>>always sent to primary node?
readFromBackup enhances performance for cache operations. Locking and
performance depend on the type of cache operation read or write,
concurrency  and isolation levels.
Better performance could be achieved by using deadlock free
optimistic/serializable transactions.

>>* affinity - will it help to configure it for REPLICATED caches, where
>>transaction touches all the node anyway?
I suppose that affinity doesn't affect to transaction performance.

>>* If there is no timeout for transaction and one of the backup nodes is
>>segmented out, transaction seems to be blocked for FailureDetectionTimeout
>>time (worst case). But is it supposed to resume and succeed after topology
>>is recalculated to include 2 nodes only?
Yes transaction resumes after topology is recalculated.

2018-04-06 3:14 GMT+03:00 Ariel Tubaltsev :

> I have 2 different caches C1, C2 on 3 nodes N1, N2, N3. I want to configure
> it in the most possibly reliable way to avoid any inconsistency and data
> loss, so both caches are REPLICATED, TRANSACTIONAL,  FULL_SYNC.
>
> Most of my operations are transactions: PESSIMISTIC, SERIALIZABLE that
> touch
> both caches, for example
> tx {
>   v= get(C1, k1);
>   put(C2, v, x);
> }
>
> But there are also non-transactional reads from one of the caches.
> All operations are performed from client (in client mode).
>
> Some questions to better map the features:
>
> * There will be only 2PC commit for transactions, no 1-phase commit, since
> I
> have 3 nodes, REPLICATED?
> * During the transaction, even for get, locks will be taken on backups
> during the prepare phase, since it's PESSIMISTIC concurrency?
> * readFromBackup - looks like it's not going to improve transactions
> performance since transaction coordinator still going to always send it to
> primary node and primary node will instruct backup to take locks?
> * readFromBackup - still should help in non-transactional read, no? or it
> always sent to primary node?
> * affinity - will it help to configure it for REPLICATED caches, where
> transaction touches all the node anyway?
> * If there is no timeout for transaction and one of the backup nodes is
> segmented out, transaction seems to be blocked for FailureDetectionTimeout
> time (worst case). But is it supposed to resume and succeed after topology
> is recalculated to include 2 nodes only?
>
> Thank you
> Ariel
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 

Regards

Pavel Vinokurov

RE: Is backup copy always stored on different host - is ithost-awarereplication

[Naveen]

Hi Stan

I wanted to create a cache template with the below

 
 
 
 





 

How do I use this template while creating the cache, what is the property I
should set





Thanks
Naveen



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite visor command issue on K8s cluster

[Arseny Kovalchuk]

Hi.

I just shared my experience ), in my opinion it's the most convenient way,
and if somebody decides to update documentation, it would be at least one
of the ways to run tools in k8s.

​
Arseny Kovalchuk

Senior Software Engineer at Synesis
skype: arseny.kovalchuk
mobile: +375 (29) 666-16-16
​LinkedIn Profile ​

On 9 April 2018 at 17:09, vbm  wrote:

> Hi Arseny,
>
> As you mentioned, I created a new pod which basically runs indefinitely.
> Now I logged into that container and was able to connect using ignite visor
> without any issues.
>
> If this is the recommended way, I think it is better to document in the
> Kubernetes ignite guide.
>
>
> Regards,
> Vishwas
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite visor command issue on K8s cluster

[vbm]

Hi Arseny,

As you mentioned, I created a new pod which basically runs indefinitely. 
Now I logged into that container and was able to connect using ignite visor
without any issues.

If this is the recommended way, I think it is better to document in the
Kubernetes ignite guide.


Regards,
Vishwas



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite visor command issue on K8s cluster

[David Harvey]

I've had problems with the visor cache command failing when a client is
connected, which was related to TCP port firewalls.   Visor was trying to
make a connection that was blocked.  I don't remember whether top failed.


On Mon, Apr 9, 2018 at 7:10 AM, vbm  wrote:

> Hi Dave
>
> In the ignite pod logs of the server, I see that the cluster is formed.
> I see the below trace when I run the command,
> [02:53:18] Topology snapshot [ver=4, servers=4, clients=0, CPUs=8,
> heap=6.0GB]
>
> This indicates the cluster of 4 nodes has formed.
>
> When I login to the container and try to run the ignite visor, it says
> connected. But when I tried some commands like top, cache. They were not
> returning any result.
>
>
> I got a link in stack overflow on the usage of visor in K8s
> https://stackoverflow.com/questions/45197520/running-the-
> ignite-visor-in-kubernetes
>
> So as per this, I created a new pod and as part of docker run command I
> gave
> the config file for ignite visor. With this it was able to connect to the
> cluster.
>
> Is this the right approach in K8s environment ? If this is correct, how do
> I
> take care of different input from user to the ignite visor to this pod ?
>
>
> Regards,
> Vishwas
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Disclaimer

The information contained in this communication from the sender is 
confidential. It is intended solely for use by the recipient and others 
authorized to receive it. If you are not the recipient, you are hereby notified 
that any disclosure, copying, distribution or taking action in relation of the 
contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been 
automatically archived by Mimecast Ltd, an innovator in Software as a Service 
(SaaS) for business. Providing a safer and more useful place for your human 
generated data. Specializing in; Security, archiving and compliance. To find 
out more visit the Mimecast website.

Query on Ignite web console

[vbm]

Hi,

I wanted to use Ignite web console for my POC. 
Can you please let me know, if it is licensed or open-sourced ?


Regards,
Vishwas



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: 如何用jdbc 关联查询跨节点Cache数据

[Evgenii Zhuravlev]

Hi,

Unfortunately, it's not clear from this description what kind of problems
you have, could you please share a small reproducer as maven project?

Thanks,
Evgenii

2018-04-09 8:14 GMT+03:00 hulitao198758 :

> Hello, ask, apache ignite using JDBC thin client connection, use the select
> the from the cache. The table where the name in (select the name from
> otherCache. Table) when the returned result is data of the current
> connection node, other nodes can't return of data, how to solve this
> problem
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Cache loading from the web console

[demanfear]

About the loadcache () method, I know it's interesting to know how to
download data besides it



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite visor command issue on K8s cluster

[vbm]

Hi ​Arseny,

I missed you comment. I will try it out and get back.
Thanks for the information.


Regards,
Vishwas



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite visor command issue on K8s cluster

[vbm]

Hi Dave

In the ignite pod logs of the server, I see that the cluster is formed. 
I see the below trace when I run the command,
[02:53:18] Topology snapshot [ver=4, servers=4, clients=0, CPUs=8,
heap=6.0GB]

This indicates the cluster of 4 nodes has formed.

When I login to the container and try to run the ignite visor, it says
connected. But when I tried some commands like top, cache. They were not
returning any result.


I got a link in stack overflow on the usage of visor in K8s 
https://stackoverflow.com/questions/45197520/running-the-ignite-visor-in-kubernetes

So as per this, I created a new pod and as part of docker run command I gave
the config file for ignite visor. With this it was able to connect to the
cluster. 

Is this the right approach in K8s environment ? If this is correct, how do I
take care of different input from user to the ignite visor to this pod ?


Regards,
Vishwas



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: how costly creat a temp cache and drop it?

[Denis Mekhanikov]

Hi!

These requirements don't sound realistic.
You can do an experiment and try inserting your data into a Java TreeMap,
for example, which is many times faster than a distributed cache (but much
less functional, obviously).
It takes almost a second just to insert a few millions of short strings
into a map on my laptop, even if I do in parallel.

So, this is a pretty big amount of data to process, even if no complex
operations are involved.

When we are talking about storing data in Ignite, then you should take into
account the overhead for serialization and maintaining complex data
structures, that let it work in a distributed environment. And if Ignite is
run on a separate server, then network communication will also become a
part of the equation.

Loading the data will probably be the most expensive part. If you eliminate
it, then desired time can be achieved with proper tuning, if your queries
are not too complex.
So, think about keeping the data and modifying the changed pieces instead
of wiping everything out after the processing finishes, if it's possible.

Denis

вс, 8 апр. 2018 г. в 17:53, shawn.du :

> Hi,
> i want to know how costly below case:
> 1 creat a temp tabl/cache dynamically. Table only has 3 or 4 columns.
> 2 insert millions rows data by mapreduce.
> Each row data is small. Like each column less than 20 bytes.
> 3 Then do some simple aggregation query.
> 4 drop the cache.
>
> Can above operation finish in less one second?  Which steps is most costly?
>
> Suppose run on a one-node cluster,8 cores,memory is enough.
>
> Thanks.
>
> shawn.du
> 邮箱：shawn...@neulion.com.cn
>
> 
>
> 签名由 网易邮箱大师  定制
>

Strange node fail

[Ray]

I'm running two cluster on the same 8 nodes, one cluster with native
persistence enabled and the other one is memory only.
The only difference between these two cluster is that the cluster with
native persistence enabled has larger JVM heap than the other one.
The two nodes in the cluster with native persistence enabled failed today, I
looked at the log and GC log and found nothing strange.
I suspect because of the larger JVM heap, the GC takes a lot of time and
then the node is kicked out.
But I found nothing special in the GC log.
Please see the attachment for detailed log information.

node2gc.log
  
node2.log
  
node1gc.log
  
node1.log
  



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite visor command issue on K8s cluster

[Arseny Kovalchuk]

Hi.

In k8s you have to use only one port instead of port range for discovery
and communication between instances [1], because you cannot specify a port
range that POD exposes. Moreover, you have to be sure that instances in the
cluster see each other, so you have to use special discovery SPI. By
default, if you start visor in the POD where you have one instance started,
then visor cannot obtain the same port and uses another from a range, and
as a result it doesn't see other nodes in the cluster or see only one node
in the POD where it has been started.

If this is the case, then I'd recommend to start a separate POD with the
same config but with another CMD, which doesn't start a server node, but
runs a sleep loop instead, in order that k8s won't kill the POD. Then you
can 'exec -- bash' and start the visor/sqlline/control with the same config
that you've provided for other instances.

[1] https://apacheignite.readme.io/docs/network-config

Hope it will help.

​
Arseny Kovalchuk

Senior Software Engineer at Synesis
skype: arseny.kovalchuk
mobile: +375 (29) 666-16-16
​LinkedIn Profile ​

On 8 April 2018 at 16:30, David Harvey  wrote:

> You have 4 nodes running, but are they part of a single cluster?   If you
> look at the log file for one of the server nodes, has it connected to 4
> servers or only one?
>
> PS.  I've found this avoids the need to look up the URL
> "bin/ignitevisor.sh -cfg=$CONFIG_URI".
>
> On Sat, Apr 7, 2018 at 8:40 AM, vbm  wrote:
>
>> Hi,
>>
>> I have deployed 4 pods (ignite server) on a  K8s cluster. I am using
>> ignite
>> 2.3 docker image.
>> I have followed the steps from the  link
>>   .
>>
>> Inside the container, I can see the ignite process running with the config
>> file:
>>
>> root5412  0 11:20 ?00:00:34 /docker-java-home/bin/java
>> -Xms512m -Xmx1536m -XX:+AlwaysPreTouch -XX:+UseG1GC
>> -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -DIGNITE_QUIET=true
>> -DIGNITE_SUCCESS_FILE=/opt/ignite/apache-ignite-fabric-2.3.
>> 0-bin/work/ignite_success_22e88558-2de7-4b54-b0c4-e958859baa57
>> -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=49112
>> -Dcom.sun.management.jmxremote.authenticate=false
>> -Dcom.sun.management.jmxremote.ssl=false
>> -DIGNITE_HOME=/opt/ignite/apache-ignite-fabric-2.3.0-bin
>> -DIGNITE_PROG_NAME=/opt/ignite/apache-ignite-fabric-2.3.0-
>> bin/bin/ignite.sh
>> -cp
>> /opt/ignite/apache-ignite-fabric-2.3.0-bin/libs/*:/opt/ignit
>> e/apache-ignite-fabric-2.3.0-bin/libs/ignite-indexing/*:/
>> opt/ignite/apache-ignite-fabric-2.3.0-bin/libs/ignite-spring
>> /*:/opt/ignite/apache-ignite-fabric-2.3.0-bin/libs/licenses/*
>> org.apache.ignite.startup.cmdline.CommandLineStartup
>> *file:tmp/ignite/ignite-config.xml*
>> root   866   271  0 12:32 ?00:00:00 grep ignite
>>
>>
>> From inside the container, when I try to connect using same config file in
>> the ignite visor console.
>>
>> Command to connect to the cluster:
>> visor> open -cpath=/tmp/ignite/ignite-config.xml
>>
>> +--+
>> | Status   | Connected |
>> | Ignite instance name |  |
>> | Config path  | /tmp/ignite/ignite-config.xml |
>> | Uptime   | 00:00:00  |
>> +--+
>> visor>
>> *visor> node
>> (wrn) : Topology is empty.*
>>
>> When I do a curl request using below command, I get the list of topology
>> as
>> expected.
>> curl 'http://10.254.61.219:8080/ignite?cmd=top' | jq
>>
>> I think it is not able to discover the other pods. How to use the ignite
>> visor script in a K8s environment ?
>>
>> Regards,
>> Vishwas
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>
>
> *Disclaimer*
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and others
> authorized to receive it. If you are not the recipient, you are hereby
> notified that any disclosure, copying, distribution or taking action in
> relation of the contents of this information is strictly prohibited and may
> be unlawful.
>
> This email has been scanned for viruses and malware, and may have been
> automatically archived by *Mimecast Ltd*, an innovator in Software as a
> Service (SaaS) for business. Providing a *safer* and *more useful* place
> for your human generated data. Specializing in; Security, archiving and
> compliance. To find out more Click Here
> .
>

Re: Using Apache Ignite with Zookeeper

[Denis Mekhanikov]

You have to open discovery and communication ports at least.
By default they are 47500 and 47100.

If you want to use JDBC, REST or JMX, then you should open the
corresponding ports as well.

Denis

сб, 7 апр. 2018 г. в 10:08, monal sinha :

> Thanks, so I don't need to open any port on my ignite cluster node? My zk
> cluster is running on separate node.
>
> On Thu, Apr 5, 2018, 4:39 PM Roman Shtykh  wrote:
>
>> You can specify the port your zk is running on as shown in [1], and it
>> should be opened
>>
>>
>> https://apacheignite.readme.io/docs/cluster-config#zookeeper-based-discovery
>>
>> Roman
>>
>>
>> On Thursday, April 5, 2018, 4:48:05 p.m. GMT+9, monalsinha <
>> monalsi...@gmail.com> wrote:
>>
>>
>> I am using Apche Ignite with zookeeper based discovery using curator
>> framework.
>> Do I need to open any port on my ignite node like TCP based discovery, in
>> order for this to work?
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

Re: TcpDiscoverSpi error connection refused

[Neeraj Vaidya]

Thanks Denis.

Sent from my iPhone

> On 9 Apr 2018, at 18:22, Denis Mekhanikov  wrote:
> 
> There are the following lines in the TcpDiscoverySpi implementation:
> 
> if (log.isDebugEnabled())
> log.error("Exception on direct send: " + e.getMessage(), e);
> 
> So, these exceptions are shown only when debug logging is enabled, and they 
> are logged as errors.
> 
> Neeraj, you probably just didn't enable debug log on the second node, so you 
> don't see these exceptions on it.
> 
> Denis
> 
> пн, 9 апр. 2018 г. в 7:00, Neeraj Vaidya :
>> Hi Denis,
>> But this issue does not occur on node2 even if it is started before node1. I 
>> feel even otherwise this message related to Discovery of level SEVERE is a 
>> bit misleading. Wouldn’t it be visible when log level is anything higher 
>> than DEBUG  as well ? I agree this is not a functional issue as it only 
>> occurs at startup when performing discovery and gets cleared when a peer is 
>> discovered.
>> 
>> Regards,
>> Neeraj
>> 
>> Sent from my iPhone
>> 
>>> On 8 Apr 2018, at 01:58, Amir Akhmedov  wrote:
>>> 
>>> Hi Denis,
>>> 
>>> I had a similar logs before and I can confirm I got them when enabled 
>>> "debug" level but the log level "severe" was really confusing.
>>> 
 On Fri, Apr 6, 2018, 8:52 AM Denis Mekhanikov  
 wrote:
 Neeraj,
 
 You probably enabled debug logging on the following package: 
 org.apache.ignite.spi.discovery.tcp
 
 There is nothing wrong in these exceptions actually.
 The first connection refused exception happens because there are two hosts 
 in the IP finder configuration, but when you start the first node, nobody 
 is bound to the discovery port on the second host. So, you get "Connection 
 refused" exception, when discovery SPI is trying to connect to the second 
 host.
 
 If you see a message like this, then it means, that discovery finished its 
 work successfully: 
 Topology snapshot [ver=1, servers=1, ...]
 
 You can disable debug logging not to see these exceptions.
 
 The second exception is also doesn't mean anything bad.
 It only means, that some object was transferred to this node, and 
 connection was suddenly closed.
 
 So, none of these exceptions mean anything bad.
 
 Denis
 
 пт, 6 апр. 2018 г. в 2:34, Neeraj Vaidya :
> Hi,
> I have 2 virtualbox guest OS'es (CentOS7 64-bit) , each having firewalld 
> stopped and disabled.
> The nodes are axlrate-node-1 and axlrate-node-2. I have updated 
> default-config.xml in both servers
> My default-config.xml looks like below :
> -default-config.xml
>  @ axlrate-node-1-
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="
>http://www.springframework.org/schema/beans
>http://www.springframework.org/schema/beans/spring-beans.xsd;>
> 
>  class="org.apache.ignite.configuration.IgniteConfiguration">
> 
> 
> 
>   
>  class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>   
> 
>   
>   axlrate-node-1
>   axlrate-node-2
>   
> 
>   
> 
>   
> 
>   
> 
> 
> ---
> 
> -default-config.xml
>  @ axlrate-node-2-
> http://www.springframework.org/schema/beans;
>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>xsi:schemaLocation="
>http://www.springframework.org/schema/beans
>http://www.springframework.org/schema/beans/spring-beans.xsd;>
> 
>  class="org.apache.ignite.configuration.IgniteConfiguration">
> 
> 
> 
>   
>  class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>   
> 
>   
>   axlrate-node-1
>   axlrate-node-2
>   
> 
>   
> 
>   
> 
>   
> 
> 
> 

Re: TcpDiscoverSpi error connection refused

[Denis Mekhanikov]

There are the following lines in the TcpDiscoverySpi implementation:

if (log.isDebugEnabled())
log.error("Exception on direct send: " + e.getMessage(), e);

So, these exceptions are shown only when debug logging is enabled, and they
are logged as errors.

Neeraj, you probably just didn't enable debug log on the second node, so
you don't see these exceptions on it.

Denis

пн, 9 апр. 2018 г. в 7:00, Neeraj Vaidya :

> Hi Denis,
> But this issue does not occur on node2 even if it is started before node1.
> I feel even otherwise this message related to Discovery of level SEVERE is
> a bit misleading. Wouldn’t it be visible when log level is anything higher
> than DEBUG  as well ? I agree this is not a functional issue as it only
> occurs at startup when performing discovery and gets cleared when a peer is
> discovered.
>
> Regards,
> Neeraj
>
> Sent from my iPhone
>
> On 8 Apr 2018, at 01:58, Amir Akhmedov  wrote:
>
> Hi Denis,
>
> I had a similar logs before and I can confirm I got them when enabled
> "debug" level but the log level "severe" was really confusing.
>
> On Fri, Apr 6, 2018, 8:52 AM Denis Mekhanikov 
> wrote:
>
>> Neeraj,
>>
>> You probably enabled debug logging on the following package:
>> *org.apache.ignite.spi.discovery.tcp*
>>
>> There is nothing wrong in these exceptions actually.
>> The first connection refused exception happens because there are two
>> hosts in the IP finder configuration, but when you start the first node,
>> nobody is bound to the discovery port on the second host. So, you get 
>> *"Connection
>> refused" *exception, when discovery SPI is trying to connect to the
>> second host.
>>
>> If you see a message like this, then it means, that discovery finished
>> its work successfully:
>> *Topology snapshot [ver=1, servers=1, ...]*
>>
>> You can disable debug logging not to see these exceptions.
>>
>> The second exception is also doesn't mean anything bad.
>> It only means, that some object was transferred to this node, and
>> connection was suddenly closed.
>>
>> So, none of these exceptions mean anything bad.
>>
>> Denis
>>
>> пт, 6 апр. 2018 г. в 2:34, Neeraj Vaidya :
>>
>>> Hi,
>>> I have 2 virtualbox guest OS'es (CentOS7 64-bit) , each having firewalld
>>> stopped and disabled.
>>> The nodes are axlrate-node-1 and axlrate-node-2. I have updated
>>> default-config.xml in both servers
>>> My default-config.xml looks like below :
>>> -default-config.xml
>>> @ axlrate-node-1-
>>> http://www.springframework.org/schema/beans;
>>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>>>xsi:schemaLocation="
>>>http://www.springframework.org/schema/beans
>>>http://www.springframework.org/schema/beans/spring-beans.xsd;>
>>> 
>>> >> class="org.apache.ignite.configuration.IgniteConfiguration">
>>> 
>>> 
>>> 
>>>   
>>> >> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>>   
>>> 
>>>   
>>>   axlrate-node-1
>>>   axlrate-node-2
>>>   
>>> 
>>>   
>>> 
>>>   
>>> 
>>>   
>>> 
>>> 
>>>
>>> ---
>>>
>>> -default-config.xml
>>> @ axlrate-node-2-
>>> http://www.springframework.org/schema/beans;
>>>xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
>>>xsi:schemaLocation="
>>>http://www.springframework.org/schema/beans
>>>http://www.springframework.org/schema/beans/spring-beans.xsd;>
>>> 
>>> >> class="org.apache.ignite.configuration.IgniteConfiguration">
>>> 
>>> 
>>> 
>>>   
>>> >> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>>>   
>>> 
>>>   
>>>   axlrate-node-1
>>>   axlrate-node-2
>>>   
>>> 
>>>   
>>> 
>>>   
>>> 
>>>   
>>> 
>>> 
>>>
>>> ---
>>>
>>>
>>> When I try to start ignite on axlrate-node-1, I see the following error
>>> at startup.
>>>
>>> ---log
>>> on
>>> axlrate-node-1---
>>> [09:14:41,747][SEVERE][main][TcpDiscoverySpi] Exception on direct send:

Re: Ignite cache get operation not working, client application stucks at the initializaiton

[Priyanka Shinde]

Hi,

*You need to find field "ht" in your objects*

I am not quite sure what is field "ht" exactly means and how to find that in
my objects? I directly search for the exact keyword "ht" in my objects but
did not find anything.

*It's needed to have the same files for both client and server nodes. *

I run the instance of Ignite using ignite.bat which runs the ignite instance
in server mode by default and then I run the client applications using
ignite.setClient(true) which runs the application in ignite client mode. So
Both server and client can not have the same files but 2 clients are sharing
the same files. 

*
I've asked in the previous message to provide logs from client nodes, could 
you please share it? *

As I mentioned previously, When I run the application using run
configuration "-DIGNITE_QUIET=false" Infinite client logs gets printed in
the console which can not be shared, though I have shared the exact
exception to you from client logs.

Also I have tried to configure logging by referring below link
https://apacheignite.readme.io/docs/logging.
included dependency,

org.apache.ignite
ignite-log4j
2.3.0


and provided below property in config.xml file,


  

  

Also mentioned log4j.xml in the resources folder of the applicaiton. but
getting below error while running the application.

org.apache.ignite.IgniteException: Failed to instantiate Spring XML
application context
[springUrl=file:/D:/cache/apache-ignite-fabric-2.3.0-bin/apache-ignite-fabric-2.3.0-bin/config/default-config.xml,
err=Error creating bean with name 'grid.cfg' defined in URL
[file:/D:/cache/apache-ignite-fabric-2.3.0-bin/apache-ignite-fabric-2.3.0-bin/config/default-config.xml]:
Cannot create inner bean 'org.apache.ignite.logger.log4j.Log4JLogger#48d85e'
of type [org.apache.ignite.logger.log4j.Log4JLogger] while setting bean
property 'gridLogger'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating bean
with name 'org.apache.ignite.logger.log4j.Log4JLogger#48d85e' defined in URL
[file:/D:/cache/apache-ignite-fabric-2.3.0-bin/apache-ignite-fabric-2.3.0-bin/config/default-config.xml]:
Bean instantiation via constructor failed; nested exception is
org.springframework.beans.BeanInstantiationException: Failed to instantiate
[org.apache.ignite.logger.log4j.Log4JLogger]: Constructor threw exception;
nested exception is class org.apache.ignite.IgniteCheckedException: Log4j
configuration path was not found: log4j.xml]


Appreciate your early response.
Thanks in advance.
 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/