Re: Limiting the number of files in WalArchive folder

[aMark]

Using Ignite-2.7 solved the issue.
Thanks a lot.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Full GC in client after cluster become unstable with "Unable to await partitions release latch within timeout"

[aMark]

I will send you the logs soon.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Failed to send partition supply message to node: 5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class org.apache.ignite.IgniteCheckedException: Could not find start pointer for partition

[aMark]

Hi, 

We are using Ignite 2.6 as persistent store in Partitioned Mode having 12
server node running in cluster, each node is running on different machine.

There are around 48 client JVM as well which connect to cluster to fetch the
data.

Recently we have started getting following exception on server nodes (Though
clients are still able to read/write data): 

2018-12-25 02:59:48,423 ERROR
[sys-#22846%a738c793-6e94-48cc-b6cf-d53ccab5f0fe%] {}
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier
- Failed to send partition supply message to node:
5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class
org.apache.ignite.IgniteCheckedException: Could not find start pointer for
partition [part=9, partCntrSince=484857]
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.historicalIterator(GridCacheOffheapManager.java:792)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.historicalIterator(GridCacheOffheapManager.java:90)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.rebalanceIterator(IgniteCacheOffheapManagerImpl.java:893)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier.handleDemandMessage(GridDhtPartitionSupplier.java:283)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleDemandMessage(GridDhtPreloader.java:364)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:379)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:364)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:99)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1603)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
at
org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752)
at
org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125)
at
org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)


Does someone has any idea about the exception and possible resolution as
well ? 




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failed to send partition supply message to node: 5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class org.apache.ignite.IgniteCheckedException: Could not find start pointer for partition

[Pavel Kovalenko]

Hello,

It means that node with id "5423e6b5-c9be-4eb8-8f68-e643357ec2b3" has
outdated data (possibly due to restart) and started to rebalance missed
updates from a node with up-to-date data (where you have exception) using
WAL.
WAL rebalance is used when the number of entries in some partition exceeds
threshold controlled by system property IGNITE_PDS_WAL_REBALANCE_THRESHOLD
, default value of that is 500k entries. WAL rebalance is very efficient
when node has a lot of data and was in short period of down-time.
Unfortunately this mechanism is currently unstable and may lead to such
errors you noticed. A very few users have such amount of data in
persistence in 1 partition. There are a couple of tickets [1], [2], [3]
which should be fixed in 2.8 release and make it more robust.

To avoid such problem you should set JVM system property
IGNITE_PDS_WAL_REBALANCE_THRESHOLD value to some very high threshold (e.g.
2kk) in all Ignite instances and perform rolling restart. In this case
default full rebalance will be used. It's slower but durable approach.

[1] https://issues.apache.org/jira/browse/IGNITE-8459
[2] https://issues.apache.org/jira/browse/IGNITE-8391
[3] https://issues.apache.org/jira/browse/IGNITE-10078

ср, 26 дек. 2018 г. в 11:19, aMark :

> Hi,
>
> We are using Ignite 2.6 as persistent store in Partitioned Mode having 12
> server node running in cluster, each node is running on different machine.
>
> There are around 48 client JVM as well which connect to cluster to fetch
> the
> data.
>
> Recently we have started getting following exception on server nodes
> (Though
> clients are still able to read/write data):
>
> 2018-12-25 02:59:48,423 ERROR
> [sys-#22846%a738c793-6e94-48cc-b6cf-d53ccab5f0fe%] {}
>
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier
> - Failed to send partition supply message to node:
> 5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class
> org.apache.ignite.IgniteCheckedException: Could not find start pointer for
> partition [part=9, partCntrSince=484857]
> at
>
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.historicalIterator(GridCacheOffheapManager.java:792)
> at
>
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.historicalIterator(GridCacheOffheapManager.java:90)
> at
>
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.rebalanceIterator(IgniteCacheOffheapManagerImpl.java:893)
> at
>
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionSupplier.handleDemandMessage(GridDhtPartitionSupplier.java:283)
> at
>
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleDemandMessage(GridDhtPreloader.java:364)
> at
>
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:379)
> at
>
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:364)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1054)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:99)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1603)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.java:2752)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.unwindMessageSet(GridIoManager.java:1516)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4400(GridIoManager.java:125)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager$10.run(GridIoManager.java:1485)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
>
> Does someone has any idea about the exception and possible resolution as
> well ?
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

error in ignite-spark

[mehdi sey]

hi. i want to execute a RDD example in spark from example folder of ignite
2.7, but i have encounter and error just like an attached picture. in import
section you see an underlined line. i have added dependency but still is
remained why?


 

 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

continous query

[Som Som]

hello.

i would like to use labda expression for initial and remote filters of
continousquery. simething like this:

[Serializable]

public class ScanQueryFilter3 :ICacheEntryFilter

{

private readonly Expression, bool>>
_pdct;



public ScanQueryFilter3(Expression,
bool>> pdct)

{

_pdct = pdct;

}



public bool Invoke(ICacheEntry entry)

{

return _pdct.Compile().Invoke(entry);

}

}
is it possible? do you have examples?

Re: error in ignite-spark

[Ilya Kasnacheev]

Hello!

You have to enable 'scala' profile in Maven to be able to build Spark
examples. However, even after I rebuild with scala profile, netbeans will
not show them propetly. You will still be able to run them all right, I
think.

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 12:27, mehdi sey :

> hi. i want to execute a RDD example in spark from example folder of ignite
> 2.7, but i have encounter and error just like an attached picture. in
> import
> section you see an underlined line. i have added dependency but still is
> remained why?
>
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-31-16.png>
>
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-34-19.png>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Threads and their purposes

[ashishb888]

I have an application, which uses data streamers (4 streamers) to load data
in caches (4 caches).
I can see different threads and their count:
1. tcp-comm-worker 1
2. pub 16
3. sys 16
4. sys-stripe 17
5. utility 16
6. upd-ver-checker 1
7. ttl-cleanup-worker 1
8. tcp-disco-srvr 1
9. tcp-disco-sock-reader 1
10. tcp-disco-msg-worker 1
11. grid-nio-worker-tcp-rest 4
12. grid-nio-worker-tcp-comm 16
13. grid-nio-worker-client-listener 4
14. data-streamer-stripe 32

Can somebody explain threads in Ignite?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Threads and their purposes

[Ilya Kasnacheev]

Hello!

Here is description of most of thread pools:
https://apacheignite.readme.io/docs/thread-pools

tcp-disco is discovery: https://apacheignite.readme.io/docs/tcpip-discovery
tcp-comm is communication:
https://apacheignite.readme.io/docs/network-config#tcpcommunicationspi
upd-ver-checker checks if new Ignite version is out to remind you to
upgrade every now and when.

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 13:17, ashishb888 :

> I have an application, which uses data streamers (4 streamers) to load data
> in caches (4 caches).
> I can see different threads and their count:
> 1. tcp-comm-worker 1
> 2. pub 16
> 3. sys 16
> 4. sys-stripe 17
> 5. utility 16
> 6. upd-ver-checker 1
> 7. ttl-cleanup-worker 1
> 8. tcp-disco-srvr 1
> 9. tcp-disco-sock-reader 1
> 10. tcp-disco-msg-worker 1
> 11. grid-nio-worker-tcp-rest 4
> 12. grid-nio-worker-tcp-comm 16
> 13. grid-nio-worker-client-listener 4
> 14. data-streamer-stripe 32
>
> Can somebody explain threads in Ignite?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: [E] Re: Updating Apache Ignite WebSession attributes

[Ilya Kasnacheev]

Hello!

Are you sure that your servers actually form an Ignite topology? Can you
provide logs?

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 10:07, Rout, Biswajeet :

> ++ Ilya
>
> On Wed, Dec 26, 2018 at 12:35 PM Rout, Biswajeet <
> biswajeet.r...@verizon.com> wrote:
>
>> Hi  Ilya,
>>
>> I have two servers under a load balancer. I want my requests to be
>> processed in any of these servers.
>> When I keep on proceeding on my page, the requests may get processed in
>> any of the servers.
>>
>> Answer to your questions: No I do not have any node loss and I have 1
>> backup node configured.
>>
>> Regards,
>> Biswajeet
>>
>> On Mon, Dec 24, 2018 at 8:13 PM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> Do you have a reproducer for this behavior? What do you mean by
>>> "different server" here? Is there node loss? Do you have backups configured?
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пн, 24 дек. 2018 г. в 15:57, Rout, Biswajeet >> >:
>>>
 Hi,

 We are currently integrating Apache Ignite in our application to share
 sessions in a cluster. At this point, we can successfully share sessions
 between two servers, but there's one use case, which (seems) is not taken
 care by Ignite.

 In our application, we have a session object which is called
 *'SessiondataObject'*. This contains all the user information and all
 kinds of attributes are added to this object while the user browses through
 the application.

 We update the *'Sessiondataobject'* in various places.

 Without Apache Ignite, our session will contain the correct value. But
 when using Ignite, the Ignite *'WebSession'* object seems to work a
 bit different: when the control goes to a different server, it looks like
 the *'SessionDataObject'* does not have that updated information.

 how to make sure that the update to the *'SessionDataObject'* updates
 in the WebSession cache as well?

 --

 

 Biswajeet
 Rout
 DTIX , DELPHI

 O +18136176308 | M 9703349977
 biswajeet.r...@verizon.com

>>>
>>
>> --
>>
>> 
>>
>> Biswajeet
>> Rout
>> DTIX , DELPHI
>>
>> O +18136176308 | M 9703349977
>> biswajeet.r...@verizon.com
>>
>
>
> --
>
> 
>
> Biswajeet
> Rout
> DTIX , DELPHI
>
> O +18136176308 | M 9703349977
> biswajeet.r...@verizon.com
>

Re: error in ignite-spark

[Mehdi Seydali]

spark example have been written with java not scala. i think this error
relate to spark version. i have another question. for running this example
is it  necessary to intsall spark before running?.

On Wed, Dec 26, 2018 at 1:44 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> You have to enable 'scala' profile in Maven to be able to build Spark
> examples. However, even after I rebuild with scala profile, netbeans will
> not show them propetly. You will still be able to run them all right, I
> think.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 26 дек. 2018 г. в 12:27, mehdi sey :
>
>> hi. i want to execute a RDD example in spark from example folder of ignite
>> 2.7, but i have encounter and error just like an attached picture. in
>> import
>> section you see an underlined line. i have added dependency but still is
>> remained why?
>>
>> <
>> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-31-16.png>
>>
>> <
>> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-34-19.png>
>>
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

Re: Failed to send partition supply message to node: 5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class org.apache.ignite.IgniteCheckedException: Could not find start pointer for partition

[aMark]

Thanks Pavel for prompt response.

I could confirm that node "5423e6b5-c9be-4eb8-8f68-e643357ec2b3" (and no
other node in the cluster) did not go down, not sure how did stale data
cropped up on few nodes.  And this type of exception is coming from every
server node in the cluster.

What happens if re-balancing did not happen properly due to this exception,
could it lead to data loss ? 
does data get corrupted on the part*.bin files (in persistent store) in the
Ignite cache due to this exception ?

Thanks,






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: [E] Re: Updating Apache Ignite WebSession attributes

[Rout, Biswajeet]

Hi,
I did not get your question.

I have two independent app servers under one load balancer which is in AWS.
the Ignite nodes are getting discovered using the S3 discovery mechanism.

I am pretty much sure that the nodes are getting discovered as well as the
session are getting shared across the servers.
But my problem is like, when I do some modification to the session, those
changes information is not cached.

I have two questions in general:


   - At which point of time the web session cache the session?
   - Do ignite updates the web session cache, if I perform an update to
   session?

Regards,
Biswajeet Rout

On Wed, Dec 26, 2018 at 4:02 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> Are you sure that your servers actually form an Ignite topology? Can you
> provide logs?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 26 дек. 2018 г. в 10:07, Rout, Biswajeet :
>
>> ++ Ilya
>>
>> On Wed, Dec 26, 2018 at 12:35 PM Rout, Biswajeet <
>> biswajeet.r...@verizon.com> wrote:
>>
>>> Hi  Ilya,
>>>
>>> I have two servers under a load balancer. I want my requests to be
>>> processed in any of these servers.
>>> When I keep on proceeding on my page, the requests may get processed in
>>> any of the servers.
>>>
>>> Answer to your questions: No I do not have any node loss and I have 1
>>> backup node configured.
>>>
>>> Regards,
>>> Biswajeet
>>>
>>> On Mon, Dec 24, 2018 at 8:13 PM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 Do you have a reproducer for this behavior? What do you mean by
 "different server" here? Is there node loss? Do you have backups 
 configured?

 Regards,
 --
 Ilya Kasnacheev


 пн, 24 дек. 2018 г. в 15:57, Rout, Biswajeet <
 biswajeet.r...@verizon.com>:

> Hi,
>
> We are currently integrating Apache Ignite in our application to share
> sessions in a cluster. At this point, we can successfully share sessions
> between two servers, but there's one use case, which (seems) is not taken
> care by Ignite.
>
> In our application, we have a session object which is called
> *'SessiondataObject'*. This contains all the user information and all
> kinds of attributes are added to this object while the user browses 
> through
> the application.
>
> We update the *'Sessiondataobject'* in various places.
>
> Without Apache Ignite, our session will contain the correct value. But
> when using Ignite, the Ignite *'WebSession'* object seems to work a
> bit different: when the control goes to a different server, it looks like
> the *'SessionDataObject'* does not have that updated information.
>
> how to make sure that the update to the *'SessionDataObject'* updates
> in the WebSession cache as well?
>
> --
>
> 
>
> Biswajeet
> Rout
> DTIX , DELPHI
>
> O +18136176308 | M 9703349977
> biswajeet.r...@verizon.com
>

>>>
>>> --
>>>
>>> 
>>>
>>> Biswajeet
>>> Rout
>>> DTIX , DELPHI
>>>
>>> O +18136176308 | M 9703349977
>>> biswajeet.r...@verizon.com
>>>
>>
>>
>> --
>>
>> 
>>
>> Biswajeet
>> Rout
>> DTIX , DELPHI
>>
>> O +18136176308 | M 9703349977
>> biswajeet.r...@verizon.com
>>
>

-- 



Biswajeet
Rout
DTIX , DELPHI

O +18136176308 | M 9703349977
biswajeet.r...@verizon.com

Re: error in importing ignite 2.7 to netbeans

[aealexsandrov]

Hi,

Depends on how exactly you start the SharedRDDExample?

1)In case if you use the new maven project then possible you forgot to add
dependencies for Ignite spark:


org.apache.ignite
ignite-core
2.7.0



org.apache.ignite
ignite-spring
2.7.0



org.apache.ignite
ignite-indexing
2.7.0



org.apache.ignite
ignite-spark
2.7.0


2)In case if you are going to build from the source then check that you set
the IGNITE_HOME.

If everything fine then provide the full build log error.

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: error in importing ignite 2.7 to netbeans

[Mehdi Seydali]

this is pom file in folder of example.




http://maven.apache.org/POM/4.0.0"; xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance";
 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
4.0.0


org.apache.ignite
ignite-parent
1
../parent


ignite-examples
2.7.0



javax.cache
cache-api
${javax.cache.version}



org.apache.ignite
ignite-core
${project.version}



org.apache.ignite
ignite-spring
${project.version}



org.springframework
spring-core


org.springframework
spring-beans





org.apache.ignite
ignite-log4j
${project.version}



org.apache.ignite
ignite-indexing
${project.version}



org.apache.ignite
ignite-spring-data_2.0
${project.version}



org.apache.ignite
ignite-ml
${project.version}



commons-cli
commons-cli
1.2



com.google.code.simple-spring-memcached
spymemcached
2.8.4



org.apache.ignite
ignite-core
${project.version}
test-jar
test



org.gridgain
ignite-shmem
1.0.0
test



org.javassist
javassist
${javassist.version}
test





src/main/java
src/test/java
src/main/java
src/test/java




scala


src/main/spark
src/test/spark




org.apache.ignite
ignite-scalar
${project.version}



org.scalatest
scalatest_2.11
2.2.4
test


org.scala-lang
scala-library





org.apache.ignite
ignite-spark
${project.version}






net.alchim31.maven
scala-maven-plugin






lgpl


src/main/java-lgpl
src/test/java-lgpl




org.apache.ignite
ignite-hibernate_5.1
${project.version}



org.apache.ignite
ignite-schedule
${project.version}



org.apache.ignite
ignite-geospatial
${project.version}








src/main/java

**/*.java



config





org.codehaus.mojo
build-helper-maven-plugin


add-sources
generate-sources

add-source



${lgpl.folder}
${spark.folder}





add-tests
generate-test-sources

add-test-source



${lgpl.test.folder}
${spark.test.folder}







org.apache.maven.plugins
maven-deploy-plugin

true






 is anything 

Re: Failed to send partition supply message to node: 5423e6b5-c9be-4eb8-8f68-e643357ec2b3 class org.apache.ignite.IgniteCheckedException: Could not find start pointer for partition

[Pavel Kovalenko]

This sounds strange. There definetely should be a cause of such behaviour.
Rebalancing is happened only after an topology change (node join/leave,
deactivation/activation).
Could you please share logs from node with exception you mentioned in
message, node with id "5423e6b5-c9be-4eb8-8f68-e643357ec2b3", and
coordinator (oldest) node (you can find this node grepping "crd=true" in
logs) to find the root cause of such behaviour?
Cache configurations / Data storage configurations would be also very
useful to debug.

1) If rebalancing didn't happen you should notice MOVING partitions in your
cache groups (from metrics MxBeans or Visor). It's possible to write data
to such partitions and read (it depends on configured PartitionLossPolicy
in your caches). If you have at least 1 owner (OWNING state) for each of
such replicated partition there is no data loss. Such MOVING partitions
will be properly rebalanced after node restart and data become consistent
in primary-backups partitions.
2) If part*.bin files are corrupted you may notice it only during node
restart or subsequent cluster deactivation/activation or if you have less
RAM than your data size and node do pages swapping (replacing) to/from
disk. In usual cluster life this is undetectable since all data placed in
RAM.


ср, 26 дек. 2018 г. в 13:44, aMark :

> Thanks Pavel for prompt response.
>
> I could confirm that node "5423e6b5-c9be-4eb8-8f68-e643357ec2b3" (and no
> other node in the cluster) did not go down, not sure how did stale data
> cropped up on few nodes.  And this type of exception is coming from every
> server node in the cluster.
>
> What happens if re-balancing did not happen properly due to this exception,
> could it lead to data loss ?
> does data get corrupted on the part*.bin files (in persistent store) in the
> Ignite cache due to this exception ?
>
> Thanks,
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: error in importing ignite 2.7 to netbeans

[aealexsandrov]

Please check that you set the scale profile because of without it the spark
dependencies will not be added:

...


scala


src/main/spark




org.apache.ignite
ignite-scalar
2.7.0



org.apache.ignite
ignite-spark
2.7.0


...

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: diferrences between IgniteRdd and SparkRdd

[aealexsandrov]

Hi,

The main difference between native Spark RDD and IgniteRDD is that Ignite
RDD provides a shared in-memory view on data across different Spark jobs,
workers, or applications, while native Spark RDD cannot be seen by other
Spark jobs or applications.

You can see the attached image that describes it.

spark-ignite-rdd.png

  

According to your second question:

The provided code do next:

1)Create Spark context
2)Creates Ignite context with provided configuration.
3)Creates an Ignite Shared RDD of Type (Int,Int)
4)Fill the Ignite Shared RDD in with Int pairs.

To get the full example please take a look at ScalarSharedRDDExample.java
that located in source files:

https://github.com/apache/ignite/blob/master/examples/src/main/scala/org/apache/ignite/scalar/examples/spark/ScalarSharedRDDExample.scala

BR,
Andrei




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: error in ignite-spark

[Ilya Kasnacheev]

Hello!

No you don't.

You can use the following maven commands from examples directory to run
example:

mvn clean install -Pscala -DskipTests
mvn -Pscala exec:java
-Dexec.mainClass=org.apache.ignite.examples.spark.SharedRDDExample

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 13:42, Mehdi Seydali :

> spark example have been written with java not scala. i think this error
> relate to spark version. i have another question. for running this example
> is it  necessary to intsall spark before running?.
>
> On Wed, Dec 26, 2018 at 1:44 PM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> You have to enable 'scala' profile in Maven to be able to build Spark
>> examples. However, even after I rebuild with scala profile, netbeans will
>> not show them propetly. You will still be able to run them all right, I
>> think.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 26 дек. 2018 г. в 12:27, mehdi sey :
>>
>>> hi. i want to execute a RDD example in spark from example folder of
>>> ignite
>>> 2.7, but i have encounter and error just like an attached picture. in
>>> import
>>> section you see an underlined line. i have added dependency but still is
>>> remained why?
>>>
>>> <
>>> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-31-16.png>
>>>
>>> <
>>> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-34-19.png>
>>>
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>

Re: [E] Re: Updating Apache Ignite WebSession attributes

[Ilya Kasnacheev]

Hello!

When you do getAttribute/setAttribute/removeAttribute on web session, it
should be updated at the end of requedt.

Do you have reproducer for this kind of behavior?

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 13:46, Rout, Biswajeet :

> Hi,
> I did not get your question.
>
> I have two independent app servers under one load balancer which is in
> AWS. the Ignite nodes are getting discovered using the S3 discovery
> mechanism.
>
> I am pretty much sure that the nodes are getting discovered as well as the
> session are getting shared across the servers.
> But my problem is like, when I do some modification to the session, those
> changes information is not cached.
>
> I have two questions in general:
>
>
>- At which point of time the web session cache the session?
>- Do ignite updates the web session cache, if I perform an update to
>session?
>
> Regards,
> Biswajeet Rout
>
> On Wed, Dec 26, 2018 at 4:02 PM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> Are you sure that your servers actually form an Ignite topology? Can you
>> provide logs?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 26 дек. 2018 г. в 10:07, Rout, Biswajeet > >:
>>
>>> ++ Ilya
>>>
>>> On Wed, Dec 26, 2018 at 12:35 PM Rout, Biswajeet <
>>> biswajeet.r...@verizon.com> wrote:
>>>
 Hi  Ilya,

 I have two servers under a load balancer. I want my requests to be
 processed in any of these servers.
 When I keep on proceeding on my page, the requests may get processed in
 any of the servers.

 Answer to your questions: No I do not have any node loss and I have 1
 backup node configured.

 Regards,
 Biswajeet

 On Mon, Dec 24, 2018 at 8:13 PM Ilya Kasnacheev <
 ilya.kasnach...@gmail.com> wrote:

> Hello!
>
> Do you have a reproducer for this behavior? What do you mean by
> "different server" here? Is there node loss? Do you have backups 
> configured?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 24 дек. 2018 г. в 15:57, Rout, Biswajeet <
> biswajeet.r...@verizon.com>:
>
>> Hi,
>>
>> We are currently integrating Apache Ignite in our application to
>> share sessions in a cluster. At this point, we can successfully share
>> sessions between two servers, but there's one use case, which (seems) is
>> not taken care by Ignite.
>>
>> In our application, we have a session object which is called
>> *'SessiondataObject'*. This contains all the user information and
>> all kinds of attributes are added to this object while the user browses
>> through the application.
>>
>> We update the *'Sessiondataobject'* in various places.
>>
>> Without Apache Ignite, our session will contain the correct value.
>> But when using Ignite, the Ignite *'WebSession'* object seems to
>> work a bit different: when the control goes to a different server, it 
>> looks
>> like the *'SessionDataObject'* does not have that updated
>> information.
>>
>> how to make sure that the update to the *'SessionDataObject'* updates
>> in the WebSession cache as well?
>>
>> --
>>
>> 
>>
>> Biswajeet
>> Rout
>> DTIX , DELPHI
>>
>> O +18136176308 | M 9703349977
>> biswajeet.r...@verizon.com
>>
>

 --

 

 Biswajeet
 Rout
 DTIX , DELPHI

 O +18136176308 | M 9703349977
 biswajeet.r...@verizon.com

>>>
>>>
>>> --
>>>
>>> 
>>>
>>> Biswajeet
>>> Rout
>>> DTIX , DELPHI
>>>
>>> O +18136176308 | M 9703349977
>>> biswajeet.r...@verizon.com
>>>
>>
>
> --
>
> 
>
> Biswajeet
> Rout
> DTIX , DELPHI
>
> O +18136176308 | M 9703349977
> biswajeet.r...@verizon.com
>

Re: [E] Re: Updating Apache Ignite WebSession attributes

[Rout, Biswajeet]

Hi Ilya,

This is a real enterprise project, so I can not share any kind of code or
logs.

In my project, I am doing a lot of update to an Object which is inside the
session.
I can see the websession is present in the Cache, but the updates done by
me at later point of time inside the code is not persisted into the cache.

How do I persist the updated web session?
Should Ignite take care of it automatically? or Do I need to perform
something in order to save it in web session cache?

Regards,
Biswajeet

On Wed, Dec 26, 2018 at 6:13 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> When you do getAttribute/setAttribute/removeAttribute on web session, it
> should be updated at the end of requedt.
>
> Do you have reproducer for this kind of behavior?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 26 дек. 2018 г. в 13:46, Rout, Biswajeet :
>
>> Hi,
>> I did not get your question.
>>
>> I have two independent app servers under one load balancer which is in
>> AWS. the Ignite nodes are getting discovered using the S3 discovery
>> mechanism.
>>
>> I am pretty much sure that the nodes are getting discovered as well as
>> the session are getting shared across the servers.
>> But my problem is like, when I do some modification to the session, those
>> changes information is not cached.
>>
>> I have two questions in general:
>>
>>
>>- At which point of time the web session cache the session?
>>- Do ignite updates the web session cache, if I perform an update to
>>session?
>>
>> Regards,
>> Biswajeet Rout
>>
>> On Wed, Dec 26, 2018 at 4:02 PM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> Are you sure that your servers actually form an Ignite topology? Can you
>>> provide logs?
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> ср, 26 дек. 2018 г. в 10:07, Rout, Biswajeet >> >:
>>>
 ++ Ilya

 On Wed, Dec 26, 2018 at 12:35 PM Rout, Biswajeet <
 biswajeet.r...@verizon.com> wrote:

> Hi  Ilya,
>
> I have two servers under a load balancer. I want my requests to be
> processed in any of these servers.
> When I keep on proceeding on my page, the requests may get processed
> in any of the servers.
>
> Answer to your questions: No I do not have any node loss and I have 1
> backup node configured.
>
> Regards,
> Biswajeet
>
> On Mon, Dec 24, 2018 at 8:13 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> Do you have a reproducer for this behavior? What do you mean by
>> "different server" here? Is there node loss? Do you have backups 
>> configured?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пн, 24 дек. 2018 г. в 15:57, Rout, Biswajeet <
>> biswajeet.r...@verizon.com>:
>>
>>> Hi,
>>>
>>> We are currently integrating Apache Ignite in our application to
>>> share sessions in a cluster. At this point, we can successfully share
>>> sessions between two servers, but there's one use case, which (seems) is
>>> not taken care by Ignite.
>>>
>>> In our application, we have a session object which is called
>>> *'SessiondataObject'*. This contains all the user information and
>>> all kinds of attributes are added to this object while the user browses
>>> through the application.
>>>
>>> We update the *'Sessiondataobject'* in various places.
>>>
>>> Without Apache Ignite, our session will contain the correct value.
>>> But when using Ignite, the Ignite *'WebSession'* object seems to
>>> work a bit different: when the control goes to a different server, it 
>>> looks
>>> like the *'SessionDataObject'* does not have that updated
>>> information.
>>>
>>> how to make sure that the update to the *'SessionDataObject'* updates
>>> in the WebSession cache as well?
>>>
>>> --
>>>
>>> 
>>>
>>> Biswajeet
>>> Rout
>>> DTIX , DELPHI
>>>
>>> O +18136176308 | M 9703349977
>>> biswajeet.r...@verizon.com
>>>
>>
>
> --
>
> 
>
> Biswajeet
> Rout
> DTIX , DELPHI
>
> O +18136176308 | M 9703349977
> biswajeet.r...@verizon.com
>


 --

 

 Biswajeet
 Rout
 DTIX , DELPHI

 O +18136176308 | M 9703349977
 biswajeet.r...@verizon.com

>>>
>>
>> --
>>
>> 
>>
>> Biswajeet
>> Rout
>> DTIX , DELPHI
>>
>> O +18136176308 | M 9703349977
>> biswajeet.r...@verizon.com
>>
>

-- 



Biswajeet
Rout
DTIX , DELPHI

O +18136176308 | M 9703349977
biswajeet.r...@verizon.com

Re: Use of GridFutureAdapter$Node objects ?

[aealexsandrov]

Hi,

Looks like it's a part of the current implementation. I will try to
investigate it.

However, as a workaround you can use the next code to decrease the possible
memory allocation in case if you are going to load a lot of data:

int chunk_size = 1000;

try (IgniteDataStreamer streamer =
 ignite.dataStreamer("Cache1")) {
streamer.allowOverwrite(true);

Map entryMap = new HashMap<>();   

// your logic where you fill the map. As example:

for (int i = 0; i < chunk_size; i++)
entryMap.put(i, i);

futures.add(streamer.addData(entryMap));
}

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Write Synchronization Mode per client

[aealexsandrov]

Hi,

Synchronization mode is a part of the cache configuration. It will be set up
at the cache starting. After that, the cache configuration can't be changed
in runtime.

The only case when cache configuration could be changed is when you destroy
old cache and create a new one with the new configuration.

Could you please provide the more details of how exactly your clients are
affected by cache synchronization mode?

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: If I want to know how long it takes for data reading from the server side, where should I add logs?

[aealexsandrov]

Hi,

Can you share your benchmark and explain what results you are expecting? Are
you sure that there is no GC pauses at that moment?

Ignite already have its own benchmarks that used Yardstick:

https://apacheignite.readme.io/docs/perfomance-benchmarking

Are you use them? If not possible you can take a look.

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Loading data from Spark Cluster to Ignite Cache to perform Ignite ML

[zaleslaw]

Hi, Igniters!

I am looking for a possibility to load data from Spark RDD or DataFrame to
Ignite cache with next declaration IgniteCache dataCache
to perform Ignite ML algorithms.

As I understand the current mechanism of Ignite-Spark integration helps to
store RDD/DF from Spark in Ignite to improve performance of Spark Jobs and
this implementation couldn't help me, am I correct?

Dou  you know how to make this small ETL more effectively? Without
collecting data on one node like in example below?

 IgniteCache cache = getCache(ignite);

SparkSession spark = SparkSession
.builder()
.appName("SparkForIgnite")
.master("local")
.config("spark.executor.instances", "2")
.getOrCreate();

Dataset ds = ;

ds.show();

List data = ds.collectAsList(); // stupid solution

Object[] parsedRow = new Object[14];
for (int i = 0; i < data.size(); i++) {
for (int j = 0; j < 14; j++)
parsedRow[j] = data.get(i).get(j);
cache.put(i, parsedRow);
}

spark.stop();



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Use of GridFutureAdapter$Node objects ?

[userx]

Thanks Andrei, will try what you have suggested.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Write Synchronization Mode per client

[rom_we]

Hi Andrei,
Thanks for the response.

I'm trying to understand how can i use ignite as a *global cross
applications* cache for our system entities.
This cache could be used by different parties with different needs. Some
care more about data consistency and some requires low latency.

I just wanted to understand how flexible the cache configuration is and
better understand why this specific configuration is per cache and not
determined by the client.

BR,
Rom 




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Can we put a timeout on Ignite.close ?

[userx]

Hi

Is there a way we can put a timeout on close operation of Ignite. For an
example I have a java program which does some computation and then starts
the Ignite in client mode so that the computed numbers can be written to
cache on Ignite Servers in persistent mode. Say if there is an execution
exception, I would want the java program to call close on the Ignite started
in the client mode and return to its normal processing with in a specified
time say 5 seconds or so ?
Which means that if the ignite in the client mode is not getting closed in 5
seconds, all the ignite threads started on the client side should be
interrupted and should be closed/killed/stopped.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: continous query

[Ilya Kasnacheev]

Hello!

Have you tried? Got any relevant errors? Can you share a sample project?

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 12:39, Som Som <2av10...@gmail.com>:

> hello.
>
> i would like to use labda expression for initial and remote filters of
> continousquery. simething like this:
>
> [Serializable]
>
> public class ScanQueryFilter3 :ICacheEntryFilter
>
> {
>
> private readonly Expression, bool>>
> _pdct;
>
>
>
> public ScanQueryFilter3(Expression >, bool>> pdct)
>
> {
>
> _pdct = pdct;
>
> }
>
>
>
> public bool Invoke(ICacheEntry entry)
>
> {
>
> return _pdct.Compile().Invoke(entry);
>
> }
>
> }
> is it possible? do you have examples?
>
>
>

Re: [E] Re: Updating Apache Ignite WebSession attributes

[Ilya Kasnacheev]

Hello!

- Have you tried to explicitly do setAttribute(key, changedObject) during
request handling?
- Do you have binary or POJO session store? This is controlled by
org.apache.ignite.cache.websession.WebSessionFilter#WEB_SES_KEEP_BINARY_PARAM
= true|false. Can you try both?
- Can you enable DEBUG logging, see if you observe messages like "Session
binary attributes updated" or "Session attributes updated"?

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 15:52, Rout, Biswajeet :

> Hi Ilya,
>
> This is a real enterprise project, so I can not share any kind of code or
> logs.
>
> In my project, I am doing a lot of update to an Object which is inside the
> session.
> I can see the websession is present in the Cache, but the updates done by
> me at later point of time inside the code is not persisted into the cache.
>
> How do I persist the updated web session?
> Should Ignite take care of it automatically? or Do I need to perform
> something in order to save it in web session cache?
>
> Regards,
> Biswajeet
>
> On Wed, Dec 26, 2018 at 6:13 PM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> When you do getAttribute/setAttribute/removeAttribute on web session, it
>> should be updated at the end of requedt.
>>
>> Do you have reproducer for this kind of behavior?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 26 дек. 2018 г. в 13:46, Rout, Biswajeet > >:
>>
>>> Hi,
>>> I did not get your question.
>>>
>>> I have two independent app servers under one load balancer which is in
>>> AWS. the Ignite nodes are getting discovered using the S3 discovery
>>> mechanism.
>>>
>>> I am pretty much sure that the nodes are getting discovered as well as
>>> the session are getting shared across the servers.
>>> But my problem is like, when I do some modification to the session,
>>> those changes information is not cached.
>>>
>>> I have two questions in general:
>>>
>>>
>>>- At which point of time the web session cache the session?
>>>- Do ignite updates the web session cache, if I perform an update to
>>>session?
>>>
>>> Regards,
>>> Biswajeet Rout
>>>
>>> On Wed, Dec 26, 2018 at 4:02 PM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 Are you sure that your servers actually form an Ignite topology? Can
 you provide logs?

 Regards,
 --
 Ilya Kasnacheev


 ср, 26 дек. 2018 г. в 10:07, Rout, Biswajeet <
 biswajeet.r...@verizon.com>:

> ++ Ilya
>
> On Wed, Dec 26, 2018 at 12:35 PM Rout, Biswajeet <
> biswajeet.r...@verizon.com> wrote:
>
>> Hi  Ilya,
>>
>> I have two servers under a load balancer. I want my requests to be
>> processed in any of these servers.
>> When I keep on proceeding on my page, the requests may get processed
>> in any of the servers.
>>
>> Answer to your questions: No I do not have any node loss and I have 1
>> backup node configured.
>>
>> Regards,
>> Biswajeet
>>
>> On Mon, Dec 24, 2018 at 8:13 PM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> Do you have a reproducer for this behavior? What do you mean by
>>> "different server" here? Is there node loss? Do you have backups 
>>> configured?
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пн, 24 дек. 2018 г. в 15:57, Rout, Biswajeet <
>>> biswajeet.r...@verizon.com>:
>>>
 Hi,

 We are currently integrating Apache Ignite in our application to
 share sessions in a cluster. At this point, we can successfully share
 sessions between two servers, but there's one use case, which (seems) 
 is
 not taken care by Ignite.

 In our application, we have a session object which is called
 *'SessiondataObject'*. This contains all the user information and
 all kinds of attributes are added to this object while the user browses
 through the application.

 We update the *'Sessiondataobject'* in various places.

 Without Apache Ignite, our session will contain the correct value.
 But when using Ignite, the Ignite *'WebSession'* object seems to
 work a bit different: when the control goes to a different server, it 
 looks
 like the *'SessionDataObject'* does not have that updated
 information.

 how to make sure that the update to the *'SessionDataObject'* updates
 in the WebSession cache as well?

 --

 

 Biswajeet
 Rout
 DTIX , DELPHI

 O +18136176308 | M 9703349977
 biswajeet.r...@verizon.com

>>>
>>
>> --
>>
>> 
>>
>> Biswajeet
>> Rout
>> DTIX , DELPHI
>>
>> O +18136176308 | M 9703349977
>> biswajeet.r...@verizon.com
>>
>

Re: Write Synchronization Mode per client

[aealexsandrov]

Hi,

Cache in Ignite is a cluster wide storage for data. This data will be
available around all clients. And it should work similarly for each of them.
That's why the configuration can't be changed during runtime on the already
started cache.

I don't think that all your clients will use the same data but have
different consistency requirements for it. So in your case, I suggest create
different caches for every goal/user with required configurations (every
cache could have different synchronization mode).

BR,
Andrei



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: [E] Re: Updating Apache Ignite WebSession attributes

[Rout, Biswajeet]

Hi Ilya,

- No, I have not used explicitly setting the setAttribute() method. I will
definitely try that.
- What is binary or POJO session store? is it a feature in Ignite? how to
use this? Where to to add  *WEB_SES_KEEP_BINARY_PARAM = true|false*
*- *Yes I have enabled DEBUG logging for Ignite, I can not see any
messages  like "Session binary attributes updated" or "Session attributes
updated"

Regards,
Biswajeet

On Wed, Dec 26, 2018 at 7:36 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> - Have you tried to explicitly do setAttribute(key, changedObject) during
> request handling?
> - Do you have binary or POJO session store? This is controlled by
> org.apache.ignite.cache.websession.WebSessionFilter#WEB_SES_KEEP_BINARY_PARAM
> = true|false. Can you try both?
> - Can you enable DEBUG logging, see if you observe messages like "Session
> binary attributes updated" or "Session attributes updated"?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 26 дек. 2018 г. в 15:52, Rout, Biswajeet :
>
>> Hi Ilya,
>>
>> This is a real enterprise project, so I can not share any kind of code or
>> logs.
>>
>> In my project, I am doing a lot of update to an Object which is inside
>> the session.
>> I can see the websession is present in the Cache, but the updates done by
>> me at later point of time inside the code is not persisted into the cache.
>>
>> How do I persist the updated web session?
>> Should Ignite take care of it automatically? or Do I need to perform
>> something in order to save it in web session cache?
>>
>> Regards,
>> Biswajeet
>>
>> On Wed, Dec 26, 2018 at 6:13 PM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> When you do getAttribute/setAttribute/removeAttribute on web session, it
>>> should be updated at the end of requedt.
>>>
>>> Do you have reproducer for this kind of behavior?
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> ср, 26 дек. 2018 г. в 13:46, Rout, Biswajeet >> >:
>>>
 Hi,
 I did not get your question.

 I have two independent app servers under one load balancer which is in
 AWS. the Ignite nodes are getting discovered using the S3 discovery
 mechanism.

 I am pretty much sure that the nodes are getting discovered as well as
 the session are getting shared across the servers.
 But my problem is like, when I do some modification to the session,
 those changes information is not cached.

 I have two questions in general:


- At which point of time the web session cache the session?
- Do ignite updates the web session cache, if I perform an update
to session?

 Regards,
 Biswajeet Rout

 On Wed, Dec 26, 2018 at 4:02 PM Ilya Kasnacheev <
 ilya.kasnach...@gmail.com> wrote:

> Hello!
>
> Are you sure that your servers actually form an Ignite topology? Can
> you provide logs?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 26 дек. 2018 г. в 10:07, Rout, Biswajeet <
> biswajeet.r...@verizon.com>:
>
>> ++ Ilya
>>
>> On Wed, Dec 26, 2018 at 12:35 PM Rout, Biswajeet <
>> biswajeet.r...@verizon.com> wrote:
>>
>>> Hi  Ilya,
>>>
>>> I have two servers under a load balancer. I want my requests to be
>>> processed in any of these servers.
>>> When I keep on proceeding on my page, the requests may get processed
>>> in any of the servers.
>>>
>>> Answer to your questions: No I do not have any node loss and I have
>>> 1 backup node configured.
>>>
>>> Regards,
>>> Biswajeet
>>>
>>> On Mon, Dec 24, 2018 at 8:13 PM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 Do you have a reproducer for this behavior? What do you mean by
 "different server" here? Is there node loss? Do you have backups 
 configured?

 Regards,
 --
 Ilya Kasnacheev


 пн, 24 дек. 2018 г. в 15:57, Rout, Biswajeet <
 biswajeet.r...@verizon.com>:

> Hi,
>
> We are currently integrating Apache Ignite in our application to
> share sessions in a cluster. At this point, we can successfully share
> sessions between two servers, but there's one use case, which (seems) 
> is
> not taken care by Ignite.
>
> In our application, we have a session object which is called
> *'SessiondataObject'*. This contains all the user information and
> all kinds of attributes are added to this object while the user 
> browses
> through the application.
>
> We update the *'Sessiondataobject'* in various places.
>
> Without Apache Ignite, our session will contain the correct value.
> But when using Ignite, the Ignite *'WebSession'* object seems to
> work a bit different: when the control goes to a different server, it 

Re: [E] Re: Updating Apache Ignite WebSession attributes

[Ilya Kasnacheev]

Hello!

You can add *WEB_SES_KEEP_BINARY_PARAM* as true or false as an init
parameter to WebSessionFilter.

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 17:30, Rout, Biswajeet :

> Hi Ilya,
>
> - No, I have not used explicitly setting the setAttribute() method. I will
> definitely try that.
> - What is binary or POJO session store? is it a feature in Ignite? how to
> use this? Where to to add  *WEB_SES_KEEP_BINARY_PARAM = true|false*
> *- *Yes I have enabled DEBUG logging for Ignite, I can not see any
> messages  like "Session binary attributes updated" or "Session attributes
> updated"
>
> Regards,
> Biswajeet
>
> On Wed, Dec 26, 2018 at 7:36 PM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> - Have you tried to explicitly do setAttribute(key, changedObject) during
>> request handling?
>> - Do you have binary or POJO session store? This is controlled by
>> org.apache.ignite.cache.websession.WebSessionFilter#WEB_SES_KEEP_BINARY_PARAM
>> = true|false. Can you try both?
>> - Can you enable DEBUG logging, see if you observe messages like "Session
>> binary attributes updated" or "Session attributes updated"?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 26 дек. 2018 г. в 15:52, Rout, Biswajeet > >:
>>
>>> Hi Ilya,
>>>
>>> This is a real enterprise project, so I can not share any kind of code
>>> or logs.
>>>
>>> In my project, I am doing a lot of update to an Object which is inside
>>> the session.
>>> I can see the websession is present in the Cache, but the updates done
>>> by me at later point of time inside the code is not persisted into the
>>> cache.
>>>
>>> How do I persist the updated web session?
>>> Should Ignite take care of it automatically? or Do I need to perform
>>> something in order to save it in web session cache?
>>>
>>> Regards,
>>> Biswajeet
>>>
>>> On Wed, Dec 26, 2018 at 6:13 PM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 When you do getAttribute/setAttribute/removeAttribute on web session,
 it should be updated at the end of requedt.

 Do you have reproducer for this kind of behavior?

 Regards,
 --
 Ilya Kasnacheev


 ср, 26 дек. 2018 г. в 13:46, Rout, Biswajeet <
 biswajeet.r...@verizon.com>:

> Hi,
> I did not get your question.
>
> I have two independent app servers under one load balancer which is in
> AWS. the Ignite nodes are getting discovered using the S3 discovery
> mechanism.
>
> I am pretty much sure that the nodes are getting discovered as well as
> the session are getting shared across the servers.
> But my problem is like, when I do some modification to the session,
> those changes information is not cached.
>
> I have two questions in general:
>
>
>- At which point of time the web session cache the session?
>- Do ignite updates the web session cache, if I perform an update
>to session?
>
> Regards,
> Biswajeet Rout
>
> On Wed, Dec 26, 2018 at 4:02 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> Are you sure that your servers actually form an Ignite topology? Can
>> you provide logs?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 26 дек. 2018 г. в 10:07, Rout, Biswajeet <
>> biswajeet.r...@verizon.com>:
>>
>>> ++ Ilya
>>>
>>> On Wed, Dec 26, 2018 at 12:35 PM Rout, Biswajeet <
>>> biswajeet.r...@verizon.com> wrote:
>>>
 Hi  Ilya,

 I have two servers under a load balancer. I want my requests to be
 processed in any of these servers.
 When I keep on proceeding on my page, the requests may get
 processed in any of the servers.

 Answer to your questions: No I do not have any node loss and I have
 1 backup node configured.

 Regards,
 Biswajeet

 On Mon, Dec 24, 2018 at 8:13 PM Ilya Kasnacheev <
 ilya.kasnach...@gmail.com> wrote:

> Hello!
>
> Do you have a reproducer for this behavior? What do you mean by
> "different server" here? Is there node loss? Do you have backups 
> configured?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 24 дек. 2018 г. в 15:57, Rout, Biswajeet <
> biswajeet.r...@verizon.com>:
>
>> Hi,
>>
>> We are currently integrating Apache Ignite in our application to
>> share sessions in a cluster. At this point, we can successfully share
>> sessions between two servers, but there's one use case, which 
>> (seems) is
>> not taken care by Ignite.
>>
>> In our application, we have a session object which is called
>> *'SessiondataObject'*. This contains all the user information
>> and all kinds of attributes are added to this object

Index inline size

[Prasad Bhalerao]

Hi,

I have set sqlIndexMaxInline size in cache configuration level as follows.

cacheCfg.setSqlIndexMaxInlineSize(100);

But still  I am getting following warning message in log.

WARN  o.a.i.i.p.q.h2.database.H2TreeIndex -  Indexed
columns of a row cannot be fully inlined into index what may lead to
slowdown due to additional data page reads, increase index inline size if
needed (use INLINE_SIZE option for CREATE INDEX command,
QuerySqlField.inlineSize for annotated classes, or QueryIndex.inlineSize
for explicit QueryEntity configuration) [cacheName=USERACCOUNTDATA,
tableName=USER_ACCOUNT_CACHE, idxName=USER_ACCOUNT_IDX4,
idxCols=(SUBSCRIPTIONID, UNITID, _KEY, AFFINITYID), idxType=SECONDARY,
curSize=10, recommendedInlineSize=83]

1) Is it necessary to set inline size using @QuerySqlField annotation?

2) How do I set inline size in case of group index in following case?
Do I need to set inline property inside each @QuerySqlField annotation?

public class Person implements Serializable {
  /** Indexed in a group index with "salary". */
  @QuerySqlField(orderedGroups={@QuerySqlField.Group(
name = "age_salary_idx", order = 0, descending = true)})
  private int age;

  /** Indexed separately and in a group index with "age". */
  @QuerySqlField(index = true, orderedGroups={@QuerySqlField.Group(
name = "age_salary_idx", order = 3)})
  private double salary;
}







Thanks,
Prasad

Re: Query Execution very slow

[Ilya Kasnacheev]

Hello!

Can you try pushing temp table to 1st position and setting
enforceJoinOrder=true?

SELECT ipv4agd.id,
  ipv4agd.assetGroupId,
  ipv4agd.ipStart,
  ipv4agd.ipEnd
FROM TABLE (assetGroupId bigint = ? ) temp
JOIN IpV4AssetGroupData ipv4agd
ON ipv4agd.assetGroupId = temp.assetGroupId
WHERE subscriptionId= ?
AND (ipStart <= ? AND ipEnd >= ?)
ORDER BY ipv4agd.assetGroupId

See https://apacheignite.readme.io/docs/configuration-parameters for
enforceJoinOrder.

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 19:01, Prasad Bhalerao :

> I am executing following SQL on ignite cache. This cache has 37 million
> records and this data is distributed across 4 nodes.
> *SQL:*
> SELECT ipv4agd.id,
>   ipv4agd.assetGroupId,
>   ipv4agd.ipStart,
>   ipv4agd.ipEnd
> FROM IpV4AssetGroupData ipv4agd
> JOIN TABLE (assetGroupId bigint = ? ) temp
> ON ipv4agd.assetGroupId = temp.assetGroupId
> WHERE subscriptionId= ?
> AND (ipStart <= ? AND ipEnd >= ?)
> ORDER BY ipv4agd.assetGroupId
>
>
> As per the execution plan show below, ignite is using index
> "IPV4_ASSET_GROUP_DATA_IDX2 " and execution plan attached below.
>
> This sql is taking around 23 seconds. I have set the max index inline size
> as 65 bytes.
> The number of assetGroupIds set in join clause are 50.
>
> *Is there anything I can do to improve the performance of this SQL?*
>
> I have checked the memory and cpu utilization and it it is very low. I
> also tried to profile it using jprofiler to find out the issue, but could
> not find solution. I have also attached profiler snapshot at the end.
> Please check.
>
>
> .
>
> *Indexes:*
>
> public class IpV4AssetGroupData implements 
> UpdatableData {
>
>   @QuerySqlField
>   private long id;
>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
> "ipv4_asset_group_data_idx2", order = 2)})
>   private long assetGroupId;
>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
> "ipv4_asset_group_data_idx1", order = 1),
>   @QuerySqlField.Group(name = "ipv4_asset_group_data_idx2", order = 1)})
>   private long subscriptionId;
>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
> "ipv4_asset_group_data_idx1", order = 2),
>   @QuerySqlField.Group(name = "ipv4_asset_group_data_idx2", order = 3)})
>   private int ipStart;
>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
> "ipv4_asset_group_data_idx1", order = 3),
>   @QuerySqlField.Group(name = "ipv4_asset_group_data_idx2", order = 4)})
>   private int ipEnd;
>
> }
>
> *Execution plan:*
>
> Query execution is too long [time=15788 ms, sql='SELECT
> IPV4AGD__Z0.ID __C0_0,
> IPV4AGD__Z0.ASSETGROUPID __C0_1,
> IPV4AGD__Z0.IPSTART __C0_2,
> IPV4AGD__Z0.IPEND __C0_3
> FROM IPV4_ASSET_GROUP_DETAIL_CACHE.IPV4ASSETGROUPDATA IPV4AGD__Z0
>  INNER JOIN TABLE(ASSETGROUPID BIGINT=?1) TEMP__Z1
>  ON TRUE
> WHERE (IPV4AGD__Z0.ASSETGROUPID = TEMP__Z1.ASSETGROUPID) AND
> ((IPV4AGD__Z0.SUBSCRIPTIONID = ?2) AND ((IPV4AGD__Z0.IPSTART <= ?3) AND
> (IPV4AGD__Z0.IPEND >= ?4)))
> ORDER BY 2',
>
> plan=
>
> SELECT
> IPV4AGD__Z0.ID AS __C0_0,
> IPV4AGD__Z0.ASSETGROUPID AS __C0_1,
> IPV4AGD__Z0.IPSTART AS __C0_2,
> IPV4AGD__Z0.IPEND AS __C0_3
> FROM IPV4_ASSET_GROUP_DETAIL_CACHE.IPV4ASSETGROUPDATA IPV4AGD__Z0
> /* IPV4_ASSET_GROUP_DETAIL_CACHE.IPV4_ASSET_GROUP_DATA_IDX2:
> SUBSCRIPTIONID = ?2
> AND IPSTART <= ?3
> AND IPEND >= ?4
>  */
> /* WHERE (IPV4AGD__Z0.IPEND >= ?4)
> AND ((IPV4AGD__Z0.SUBSCRIPTIONID = ?2)
> AND (IPV4AGD__Z0.IPSTART <= ?3))
> */
> INNER JOIN TABLE(ASSETGROUPID BIGINT=?1) TEMP__Z1
> /* function: ASSETGROUPID = IPV4AGD__Z0.ASSETGROUPID */
> ON 1=1
> WHERE (IPV4AGD__Z0.ASSETGROUPID = TEMP__Z1.ASSETGROUPID)
> AND ((IPV4AGD__Z0.SUBSCRIPTIONID = ?2)
> AND ((IPV4AGD__Z0.IPSTART <= ?3)
> AND (IPV4AGD__Z0.IPEND >= ?4)))
> ORDER BY 2
> , parameters=[[3483555, 3180458, 3250090, 3483563, 3182509, 3213230,
> 3245998, 3487661, 3215281, 3444657, 3182515, 3372974, 3483573, 3372981,
> 3200951, 3485624, 3295161, 3485626, 3379125, 3211196, 3213242, 3381181,
> 3194805, 3213247, 3258299, 3379123, 3377070, 3315637, 3352502, 3295174,
> 3485618, 3438530, 3483592, 3352516, 3155914, 3424204, 3192775, 3485643,
> 3317711, 3246026, 3209159, 3485584, 3485645, 3483594, 3248085, 3321799,
> 3248086, 3190744, 3211222, 3379162], 164307, 1084754675, -2094919442]]
>
>
> Profiler snapshot:
>
> [image: image.png]
>

Re: error in ignite-spark

[Mehdi Seydali]

How could i enable scala profile??

On Wed, 26 Dec 2018, 13:44 Ilya Kasnacheev, 
wrote:

> Hello!
>
> You have to enable 'scala' profile in Maven to be able to build Spark
> examples. However, even after I rebuild with scala profile, netbeans will
> not show them propetly. You will still be able to run them all right, I
> think.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 26 дек. 2018 г. в 12:27, mehdi sey :
>
>> hi. i want to execute a RDD example in spark from example folder of ignite
>> 2.7, but i have encounter and error just like an attached picture. in
>> import
>> section you see an underlined line. i have added dependency but still is
>> remained why?
>>
>> <
>> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-31-16.png>
>>
>> <
>> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-34-19.png>
>>
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

Re: error in ignite-spark

[Ilya Kasnacheev]

Hello!

With -Pscala command or via netbeans dialogs.

Regards,
-- 
Ilya Kasnacheev


ср, 26 дек. 2018 г. в 19:54, Mehdi Seydali :

> How could i enable scala profile??
>
> On Wed, 26 Dec 2018, 13:44 Ilya Kasnacheev, 
> wrote:
>
>> Hello!
>>
>> You have to enable 'scala' profile in Maven to be able to build Spark
>> examples. However, even after I rebuild with scala profile, netbeans will
>> not show them propetly. You will still be able to run them all right, I
>> think.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 26 дек. 2018 г. в 12:27, mehdi sey :
>>
>>> hi. i want to execute a RDD example in spark from example folder of
>>> ignite
>>> 2.7, but i have encounter and error just like an attached picture. in
>>> import
>>> section you see an underlined line. i have added dependency but still is
>>> remained why?
>>>
>>> <
>>> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-31-16.png>
>>>
>>> <
>>> http://apache-ignite-users.70518.x6.nabble.com/file/t2160/Screenshot_from_2018-12-26_10-34-19.png>
>>>
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>

Re: Query Execution very slow

[Prasad Bhalerao]

How to push the ids to temp table, can you please give any example?
Is it a in memory temp table created by ignite?

Can you please explain how enforceJoinOrder will help in this case?
Thanks ,
Prasad

On Wed 26 Dec, 2018, 9:37 PM Ilya Kasnacheev  Hello!
>
> Can you try pushing temp table to 1st position and setting
> enforceJoinOrder=true?
>
> SELECT ipv4agd.id,
>   ipv4agd.assetGroupId,
>   ipv4agd.ipStart,
>   ipv4agd.ipEnd
> FROM TABLE (assetGroupId bigint = ? ) temp
> JOIN IpV4AssetGroupData ipv4agd
> ON ipv4agd.assetGroupId = temp.assetGroupId
> WHERE subscriptionId= ?
> AND (ipStart <= ? AND ipEnd >= ?)
> ORDER BY ipv4agd.assetGroupId
>
> See https://apacheignite.readme.io/docs/configuration-parameters for
> enforceJoinOrder.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 26 дек. 2018 г. в 19:01, Prasad Bhalerao  >:
>
>> I am executing following SQL on ignite cache. This cache has 37 million
>> records and this data is distributed across 4 nodes.
>> *SQL:*
>> SELECT ipv4agd.id,
>>   ipv4agd.assetGroupId,
>>   ipv4agd.ipStart,
>>   ipv4agd.ipEnd
>> FROM IpV4AssetGroupData ipv4agd
>> JOIN TABLE (assetGroupId bigint = ? ) temp
>> ON ipv4agd.assetGroupId = temp.assetGroupId
>> WHERE subscriptionId= ?
>> AND (ipStart <= ? AND ipEnd >= ?)
>> ORDER BY ipv4agd.assetGroupId
>>
>>
>> As per the execution plan show below, ignite is using index
>> "IPV4_ASSET_GROUP_DATA_IDX2 " and execution plan attached below.
>>
>> This sql is taking around 23 seconds. I have set the max index inline
>> size as 65 bytes.
>> The number of assetGroupIds set in join clause are 50.
>>
>> *Is there anything I can do to improve the performance of this SQL?*
>>
>> I have checked the memory and cpu utilization and it it is very low. I
>> also tried to profile it using jprofiler to find out the issue, but could
>> not find solution. I have also attached profiler snapshot at the end.
>> Please check.
>>
>>
>> .
>>
>> *Indexes:*
>>
>> public class IpV4AssetGroupData implements 
>> UpdatableData {
>>
>>   @QuerySqlField
>>   private long id;
>>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
>> "ipv4_asset_group_data_idx2", order = 2)})
>>   private long assetGroupId;
>>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
>> "ipv4_asset_group_data_idx1", order = 1),
>>   @QuerySqlField.Group(name = "ipv4_asset_group_data_idx2", order = 1)})
>>   private long subscriptionId;
>>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
>> "ipv4_asset_group_data_idx1", order = 2),
>>   @QuerySqlField.Group(name = "ipv4_asset_group_data_idx2", order = 3)})
>>   private int ipStart;
>>   @QuerySqlField(orderedGroups = {@QuerySqlField.Group(name = 
>> "ipv4_asset_group_data_idx1", order = 3),
>>   @QuerySqlField.Group(name = "ipv4_asset_group_data_idx2", order = 4)})
>>   private int ipEnd;
>>
>> }
>>
>> *Execution plan:*
>>
>> Query execution is too long [time=15788 ms, sql='SELECT
>> IPV4AGD__Z0.ID __C0_0,
>> IPV4AGD__Z0.ASSETGROUPID __C0_1,
>> IPV4AGD__Z0.IPSTART __C0_2,
>> IPV4AGD__Z0.IPEND __C0_3
>> FROM IPV4_ASSET_GROUP_DETAIL_CACHE.IPV4ASSETGROUPDATA IPV4AGD__Z0
>>  INNER JOIN TABLE(ASSETGROUPID BIGINT=?1) TEMP__Z1
>>  ON TRUE
>> WHERE (IPV4AGD__Z0.ASSETGROUPID = TEMP__Z1.ASSETGROUPID) AND
>> ((IPV4AGD__Z0.SUBSCRIPTIONID = ?2) AND ((IPV4AGD__Z0.IPSTART <= ?3) AND
>> (IPV4AGD__Z0.IPEND >= ?4)))
>> ORDER BY 2',
>>
>> plan=
>>
>> SELECT
>> IPV4AGD__Z0.ID AS __C0_0,
>> IPV4AGD__Z0.ASSETGROUPID AS __C0_1,
>> IPV4AGD__Z0.IPSTART AS __C0_2,
>> IPV4AGD__Z0.IPEND AS __C0_3
>> FROM IPV4_ASSET_GROUP_DETAIL_CACHE.IPV4ASSETGROUPDATA IPV4AGD__Z0
>> /* IPV4_ASSET_GROUP_DETAIL_CACHE.IPV4_ASSET_GROUP_DATA_IDX2:
>> SUBSCRIPTIONID = ?2
>> AND IPSTART <= ?3
>> AND IPEND >= ?4
>>  */
>> /* WHERE (IPV4AGD__Z0.IPEND >= ?4)
>> AND ((IPV4AGD__Z0.SUBSCRIPTIONID = ?2)
>> AND (IPV4AGD__Z0.IPSTART <= ?3))
>> */
>> INNER JOIN TABLE(ASSETGROUPID BIGINT=?1) TEMP__Z1
>> /* function: ASSETGROUPID = IPV4AGD__Z0.ASSETGROUPID */
>> ON 1=1
>> WHERE (IPV4AGD__Z0.ASSETGROUPID = TEMP__Z1.ASSETGROUPID)
>> AND ((IPV4AGD__Z0.SUBSCRIPTIONID = ?2)
>> AND ((IPV4AGD__Z0.IPSTART <= ?3)
>> AND (IPV4AGD__Z0.IPEND >= ?4)))
>> ORDER BY 2
>> , parameters=[[3483555, 3180458, 3250090, 3483563, 3182509, 3213230,
>> 3245998, 3487661, 3215281, 3444657, 3182515, 3372974, 3483573, 3372981,
>> 3200951, 3485624, 3295161, 3485626, 3379125, 3211196, 3213242, 3381181,
>> 3194805, 3213247, 3258299, 3379123, 3377070, 3315637, 3352502, 3295174,
>> 3485618, 3438530, 3483592, 3352516, 3155914, 3424204, 3192775, 3485643,
>> 3317711, 3246026, 3209159, 3485584, 3485645, 3483594, 3248085, 3321799,
>> 3248086, 3190744, 3211222, 3379162], 164307, 1084754675, -2094919442]]
>>
>>
>> Profiler snapshot:
>>
>> [image: image.png]
>>
>

Re: ignite stops expiring caches after a period of time

[Scott Feldstein]

hi Ilya,
Why wouldn't many people use Expiry with 2.x?  Is there another feature
that I should be using that accomplishes the same thing?

I've reproduced the issue for 2.6.0, but with 2.7.0 it looks like it is
working.  Here is the repro if anyone is interested -

https://github.com/scottmf/ignite-expiry-repro

On Tue, Dec 25, 2018 at 5:30 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> Not many people are using Expiry with 2.x.
>
> If you can share a reproducer, please create a ticket right away and I
> hope that somebody will look into it.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 25 дек. 2018 г. в 01:36, Scott Feldstein :
>
>> Thanks Ilya, I’ll upgrade and see if that works soon. Right now we’ve
>> switched to only using partitioned caches to avoid this issue. If it’s
>> something that isn’t fixed with 2.7 then I’ll create a bug and attach code
>> to reproduce it.
>>
>> I’m really surprised that no one else is complaining about it. The repro
>> is very straight forward..
>>
>>
>> On Mon, Dec 24, 2018 at 06:36 Ilya Kasnacheev 
>> wrote:
>>
>>> Hello!
>>>
>>> There could be fixes for expiration between 2.6 and 2.7.
>>>
>>> However, if it will still be the case, we would be glad if you could
>>> post reproducer for this behavior.
>>>
>>> Note that you can almost emulate replicated cache by creating
>>> partitioned cache with large number of backups, maybe you should be using
>>> that in the meantime.
>>>
>>> Regards,
>>>
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пт, 21 дек. 2018 г. в 23:28, scottmf :
>>>
 I forgot to mention this is on Ignite 2.6.0 and i'm running with a 3
 node
 cluster.

 I've been experimenting some more and I'm finding that using PARTITIONED
 caches works fine, but replicated caches stop expiring usually within 10
 minutes.  The behavior is very consistent.

 I plan on trying this with Ignite 2.7.0...



 --
 Sent from: http://apache-ignite-users.70518.x6.nabble.com/

>>>

Re: If I want to know how long it takes for data reading from the server side, where should I add logs?

[Justin Ji]

Andrei - 

Thank for your reply, sorry for missing a sentence in this post, it should
be: there are almost 0.013% read requests take more than 20ms, and 0.0336%
more than 200ms. 
When the traffic is very large, the number of requests over 200ms will be
very large. So I want to know where the time is spent, client or server.

I think GC is not the root cause since most of GC time is less than 10ms and
GC is not frequently.

My benchmark code is simple, read and write cache with 8 threads, about 3300
getAll operations and 3300 putAll operations every second, and every
getAll/putAll contains three entries.

The ignite cluster has three nodes(8core 16g, 3g JVM heap).

*Here is my server configuration:*
































































*Here is my cache configuration:*

private static CacheConfiguration
getCacheConfiguration(IgniteConfiguration cfg) {
CacheConfiguration cacheCfg =
DefaultIgniteConfiguration.getCacheConfiguration(IgniteCacheKey.DATA_POINT.getCode());
cacheCfg.setBackups(1);
cacheCfg.setDataRegionName(Constants.THREE_GB_REGION);
   
cacheCfg.setCacheStoreFactory(FactoryBuilder.factoryOf(DataPointCacheStore.class));
cacheCfg.setWriteThrough(true);
cacheCfg.setWriteBehindEnabled(true);
cacheCfg.setWriteBehindFlushThreadCount(2);
//every 15 seconds
cacheCfg.setWriteBehindFlushFrequency(15 * 1000);
cacheCfg.setWriteBehindFlushSize(409600);
cacheCfg.setWriteBehindBatchSize(512);
cacheCfg.setStoreKeepBinary(true);
cacheCfg.setQueryParallelism(4);
//2M
cacheCfg.setRebalanceBatchSize(2 * 1024 * 1024);

List entities = getQueryEntities();
cacheCfg.setQueryEntities(entities);

cfg.setCacheConfiguration(cacheCfg);
return cacheCfg;
}

private static List getQueryEntities() {
List entities = Lists.newArrayList();

QueryEntity entity = new QueryEntity(String.class.getName(),
DpCache.class.getName());
entity.setTableName(IgniteTableKey.T_DATA_POINT.getCode());

LinkedHashMap map = new LinkedHashMap<>();
map.put("id", "java.lang.String");
map.put("gmtCreate", "java.lang.Long");
map.put("gmtModified", "java.lang.Long");
map.put("devId", "java.lang.String");
map.put("dpId", "java.lang.Integer");
map.put("code", "java.lang.String");
map.put("name", "java.lang.String");
map.put("customName", "java.lang.String");
map.put("mode", "java.lang.String");
map.put("type", "java.lang.String");
map.put("value", "java.lang.String");
map.put("rawValue", byte[].class.getName());
map.put("time", "java.lang.Long");
map.put("status", "java.lang.Boolean");
map.put("uuid", "java.lang.String");

entity.setFields(map);

QueryIndex devIdIdx = new QueryIndex("devId");
devIdIdx.setName("idx_devId");
QueryIndex dpIdIdx = new QueryIndex("dpId");
dpIdIdx.setName("idx_dpId");
List indexes = Lists.newArrayList(devIdIdx, dpIdIdx);
entity.setIndexes(indexes);

entities.add(entity);

return entities;
}



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Data Replication Between Ignite Clusters Through Kafka

[Prashant Murthy]

Hi, 

I am referring the below link to create data replication between two
clusters using kafka. 
https://dzone.com/articles/real-time-data-replication-between-ignite-clusters

I started two clusters and started kafka/zeekeeper service and created a
topic - test2 

But once I start to connect, I get some crash and error as shown in attached
file. 
kafka-crash.txt
  

./bin/connect-standalone.sh ./config/connect-standalone.properties
./config/ignite-connector-source.properties 

After this I dont see a 'node' getting connected to ignite DB. 

I think this crash is causing the issue. How to check that the kafka is
connected properly to ignite? How to resolve this crash. 

Thanks, 
Prashant



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Can Ignite code throw any RuntimeExceptions ?

[userx]

Hi

Let say we have a java program which starts Ignite in a client mode. Let us
say there is a RuntimeException which happens in the Ignite code (on the
client side only), would it be wrapped as an IgniteException and passed to
the Java client or would it be thrown as a RuntimeException to the java
program under any circumstances ?

I have seen most of the methods declaring throws IgniteException but just
thinking about any scenario where in a RuntimeException occurs and get
passed to the client.

Regards




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/