Re: Ignite Cluster Config Issue

2021-11-25 Thread andrei
instance of the service across three nodes. And your service was deployed as: ig.services(ig.cluster().forServers()); Regards, Andrei 11/25/2021 2:01 PM, Gurmehar Kalra пишет: Hey, Let me explain Architecture I am running 3 Ignite server nodes  ,each have some services  which read and write data

Re: Ignite Cluster Config Issue

2021-11-25 Thread andrei
Hey, Sorry, but I don't understand what the issue you mentioned? You are using a transaction cache and most likely using transactions. Perhaps you are just having design problems. Tell us more about what you are going to achieve. Sincerely, Andrey 11/25/2021 11:24 AM, Gurmehar Kalra пишет:

Re:

2021-11-23 Thread andrei
Hi, You can set it as part of CacheConfiguration: https://www.javadoc.io/doc/org.apache.ignite/ignite-core/1.6.0/org/apache/ignite/configuration/CacheConfiguration.html#setAffinity(org.apache.ignite.cache.affinity.AffinityFunction) Regards, Andei 11/23/2021 1:39 PM,

Re: Unable to join using thick client when authentication is enabled

2021-11-18 Thread andrei
Hello, Could you please provide a full exception for this node? If you can provide complete logs, it will be even better. I see locCls=null, but as I can see from the code it should be locCls=processor.getClass().getName(). I'm not sure how it can be null. Regards, Andrei 11/17/2021 8:56

Re: APACHE HADOOP PERFORMANCE ACCELERATION and APACHE SPARK PERFORMANCE ACCELERATION

2021-11-11 Thread andrei
://ignite.apache.org/docs/latest/extensions-and-integrations/ignite-for-spark/overview Regards, Andrei 11/11/2021 10:04 AM, Siddharth Jain пишет: Hi, I need help in how to configure acceleration for Hadoop and Spark. Can I be pointed for any step by step documentation for setup of these? Thanks, Sidd

Re: Dynamically configuring data region

2021-10-07 Thread andrei
your data region to the configuration of all nodes. 2) Stop the cluster and start it again. 3) Add a new cache config that will use this region. (it can be done with step 2) However, do not create a cache that will use this new area before it is created on all data nodes. BR, Andrei 10/4/2021

Re: Ignite not generating backup partition

2021-09-28 Thread andrei
Hi, How do you check that partitions were created or not? BR, Andrei 9/28/2021 4:37 PM, satyajit.man...@barclays.com пишет: Hi  Andrei, Could  it  be possible  that  multiple  processes  running  in  each individual  host  can  result  in  backup  partitions  not getting  generated

Re: Ignite not generating backup partition

2021-09-28 Thread andrei
, Andrei 9/28/2021 2:44 PM, satyajit.man...@barclays.com пишет: Hi  Team, I  have  a  cache  configuration  setup as  below with backups  =1 . But  when I  check  the  cache statistics  using  visor  I  don’t see  any  backup  partition  getting  generated.  Can  someone  explain  under  what

Re: Heap dump is not getting generated even after specifying HeapDumpOnOutOfMemoryError

2021-06-10 Thread andrei
Hi, I guess you should remove -J-XX:+ExitOnOutOfMemoryError. BR, Andrei 6/8/2021 3:42 PM, Naveen пишет: HI We are using Ignite 2.8.1 we start Ignite node with below command and whenever node gets crashed with OOM it is not generating the heap dump in spite of having the below settings

Re: Cannot disable system view JMX exporter via configuration

2021-06-10 Thread andrei
Hi, As an option you can just add all SQL GridGain system views to blacklist: blacklistObjectNames: ["group=views,*"] It should help too. BR, Andrei 6/2/2021 6:50 PM, tsipporah22 пишет: An update on the workaround. I'm now using below whitelist in my prometheus.yaml as previous

Re: Nodes taking 5 minutes to reattempt joining grid when failed to join

2021-06-10 Thread andrei
case is very similar to the situation described above. It is possible that some client was running with a newer version of JAVA, but at the same time your server node was started as needed. BR, Andrei 5/21/2021 3:47 PM, tarunk пишет: Hi Team, We are seeing quite a long delay in node reattempting

Re: CacheEntryProcessor ClassNotFoundException after 2.7.6 -> 2.10.0 Upgrade

2021-06-10 Thread andrei
Hi, The question was answered here (JIRA ticket was created) - http://apache-ignite-users.70518.x6.nabble.com/Exception-on-CacheEntryProcessor-invoke-2-10-0-possible-bug-tt36042.html#a36090 BR, Andrei 5/18/2021 6:16 PM, ihalilaltun пишет: what we expect here is that related

Re: Right Handling of UnRegisteredBinaryObjectTypeException

2021-06-10 Thread andrei
Hi, Answer was provided here - http://apache-ignite-users.70518.x6.nabble.com/2-8-1-CacheEntryProcessor-for-insert-update-within-Transaction-supported-tt35918.html BR, Andrei 4/20/2021 3:50 PM, VeenaMithare пишет: Hello all, Please guide me on the right way to use cacheentryprocessor

Webinar: How to Use Spark With Apache Ignite for Big Data Processing

2021-04-26 Thread andrei
ite-big-data-processing> If you'd like to learn more about the existing API, take a look at some use cases, or just ask a few questions, feel free to join the webinar. BR, Andrei

Re: Ignite AWS library to support spark 3.0.1 and scala 2.12.12 version

2021-04-14 Thread andrei
Hi, Spark 3.0 is not supported for Apache Ignite, nor in the GridGain plugin. Supported Spark versions: 2.3 and 2.4. BR, Andrei 3/31/2021 9:49 PM, ChandanS пишет: Hi All, Can anyone please let me know which apache or grid-gain ignite library for AWS EC2 instance deployment will support AWS

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-21 Thread andrei
And try one more time. BR, Andrei 1/20/2021 1:36 PM, VeenaMithare пишет: Hi Andrei, Please find a trimmed down version of the code attached. Have a look at the ContinuousQueryHelper -> setInitialQuery to check how the initial query has been coded. regards, Veena. InitialQueryProject.zip <h

Re: Ever increasing startup times as data grow in persistent storage

2021-01-21 Thread andrei
node. In case you require any changes to the current Ignite implementation, you can create a thread in the Ignite developer list: http://apache-ignite-developers.2346864.n4.nabble.com/ BR, Andrei 1/20/2021 9:16 PM, Raymond Wilson пишет: Hi Andre, I would like to see Ignite support

Re: Ever increasing startup times as data grow in persistent storage

2021-01-20 Thread andrei
Hi, Yes, that was to be expected. The main autoactivation scenario is cluster restart. If you are using manual deactivation, you should also manually activate your cluster. BR, Andrei 1/20/2021 5:50 AM, Raymond Wilson пишет: We have been experimenting with using deactivation to shutdown

Re: 2.8.1 : Continuous Query Initial query not returning any result sometimes

2021-01-19 Thread andrei
Hi, Could you please provide your client's code where you are listening to continuous query? Probably something is wrong there. BR, Andrei 1/19/2021 4:13 PM, VeenaMithare пишет: Hi , It happened a couple of times on Friday ( 15th ) . Today I started a dummy client with a continuous query

Re: Issue with BaselineTopology Branching History

2020-12-14 Thread andrei
Just try not to change the baseline after each server node restart to avoid this problem. The base topology will wait for this node. BR, Andrei 12/12/2020 4:44 PM, rakshita04 пишет: If by any chance, someone messes up this sequence, sometimes ignite is throwing error which is great on which

Re: BaselineTopology branching error

2020-12-14 Thread andrei
Hi, When the baseline of your nodes is different, they will not be able to connect to each other. Before attaching to such a node, you must remove all data from the working directory (db, wal, wal archive, etc). BR, Andrei 12/10/2020 3:53 PM, rakshita04 пишет: What if i delete "metast

Re: Failed to wait for affinity ready future for topology version: AffinityTopologyVersion

2020-12-11 Thread andrei
-/SnapshotDiscoveryMessage/ * Global WAL enable / disable -/WalStateAbstractMessage/ * Late affinity assignment -/CacheAffinityChangeMessage/ BR, Andrei 12/10/2020 3:54 AM, 38797715 пишет: Hi Andrei, Another question is, what is the starting point of PME? PME starts when a new node join request

Re: BaselineTopology branching error

2020-12-09 Thread andrei
As I said, if your "new" node was previously part of a cluster, then we can get baseline information from that cluster stored on the disk. If you are going to start a "new" node without the old baseline, delete all PDS data for that node. 12/9/2020 12:19 PM, rakshita04 пишет: is there a way

Re: Issue with BaselineTopology Branching History

2020-12-08 Thread andrei
No, you cannot start a new node first, because it will have a new baseline that will be different from the old nodes. Please start old nodes first and then add new node using the API mentioned above. 12/8/2020 3:59 PM, rakshita04 пишет: If i want to add a fresh node to cluster. Is it possible

Re: BaselineTopology branching error

2020-12-08 Thread andrei
-from-code https://apacheignite.readme.io/docs/baseline-topology#setting-the-topology-from-command-line BR, Andrei 12/8/2020 7:49 AM, rakshita04 пишет: Hi Team, We are using baseline topology concept for our two nodes, running on separate machines. We are getting below error when both

Re: Query on implementing GridSecurityProcessor

2020-11-27 Thread andrei
Hello, I think you should post your question on the Ignite Developer List: http://apache-ignite-developers.2346864.n4.nabble.com/ IEP41 was developed by Denis Garus and discussed in the following thread:

Re: Inserting date into ignite with spark jdbc

2020-11-03 Thread Andrei Aleksandrov
Hi, It will be great if you share the reproducer. BR, Andrei 11/3/2020 10:17 AM, Humphrey пишет: Let me have a summarize here. Working with | IgniteDataFrameSettings.OPTION_CONFIG_FILE(), configPath | seems to work fine. But when I'm using the JDBC thin client connection, (like connecting

Re: Too long JVM pause out of nowhere leading into shutdowns of ignite-servers

2020-11-02 Thread Andrei Aleksandrov
for the pause of your virtual machine. BR, Andrei 10/30/2020 5:56 PM, VincentCE пишет: Hello! In our project we are currently using ignite 2.81 and using zookeeper. During the last couple of days we were facing shutdowns of some of our ignite-server nodes. Please find the logs below: 1) Why ca

Re: Execution of local SqlFieldsQuery on client node disallowed

2020-11-02 Thread Andrei Aleksandrov
Hello, You can use events: https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/events/CacheRebalancingEvent.html But in this case, you should wait until the rebalancing of all caches is completed. Also you can use the web console tool:

Re: Inserting date into ignite with spark jdbc

2020-10-30 Thread Andrei Aleksandrov
Denis, I can check it out soon. The mentioned problem can probably only be related to JDBC data frames. In this case, I will create a JIRA ticket. But as I know using OPTION_CREATE_TABLE_PRIMARY_KEY_FIELDS should be the same as I showed in my example. BR, Andrei 10/30/2020 6:01 PM, Denis

Re: Ignite Cluster Issue on 2.7.6

2020-10-30 Thread Andrei Aleksandrov
Hi, Did you remove the code with ignite.cluster().active(*true*); ? However, yes, all of your data nodes should be in baseline topology. Could you collect logs from your servers? BR, Andrei 10/30/2020 2:28 PM, Gurmehar Kalra пишет: Hi, I tried changes suggested by you , waited for nodes

Re: IgniteSpiOperationTimeoutException: Operation timed out [timeoutStrategy= ExponentialBackoffTimeoutStrategy

2020-10-30 Thread Andrei Aleksandrov
operties.html#IGNITE_ENABLE_FORCIBLE_NODE_KILL If these recommendations do not help, then yes, as Ilya said, we require a loudspeaker on your part. BR, Andrei 10/30/2020 2:20 PM, Ilya Kasnacheev пишет: Hello! Do you have a reproducer for this behaviour that I could run and see it failing? Regards, -- Il

Re: Ignite instances frequently failing - BUG: soft lockup - CPU#1 stuck

2020-10-30 Thread Andrei Aleksandrov
Hello, Too little information has been provided on your part: 1) Could you provide the screenshot from the web console at this time? 2) Could you collect Ignite logs during this period? 3) What tool shows that the processors are frozen? Have you checked other tools? BR, Andrew 10/30/2020

Re: Inserting date into ignite with spark jdbc

2020-10-30 Thread Andrei Aleksandrov
quot;Person") .option(IgniteDataFrameSettings.OPTION_CREATE_TABLE_PRIMARY_KEY_FIELDS(), "id, city_id") .option(IgniteDataFrameSettings.OPTION_CREATE_TABLE_PARAMETERS(), "template=partitioned,backups=1") .mode(Append); | Please let me know if something is wrong here. BR, Andrei

Re: Ignite Cluster Issue on 2.7.6

2020-10-29 Thread Andrei Aleksandrov
-from-baseline-topology If you are not using persistence, please provide additional information that "data is being added to the cache but not available to any of the modules." means: 1) How you access data 2) What do you see in the logs BR, Andrei 10/29/2020 4:19 PM, Gurmehar K

Re: Large Heap with lots of BinaryMetaDataHolders

2020-10-29 Thread Andrei Aleksandrov
Hello, Let's start from the very beginning. 1) Could you please share the server and client config? 2) Java code of what you have in your client launcher application I will try to investigate your case. BR, Andrew 10/28/2020 7:19 PM, ssansoy пишет: Hi, could anyone please help understand

Re: High number of Exception Unexpected response ID with setTimeout

2020-09-28 Thread Andrei Aleksandrov
Hi, It will be great to take a look at full logs that contain these errors. BR, Andrei 9/28/2020 12:35 PM, AravindJP пишет: Thank you stepan for your reply , problem is the parent thread calling ignite cache get has a timeout of 120 ms , hence we allow a max of 100 ms for every cache read

Re: OutOfMemoryException with Persistence and Eviction Enabled

2020-09-25 Thread Andrei Aleksandrov
simple - use bigger data region. BR, Andrei 9/24/2020 11:44 PM, Mitchell Rathbun (BLOOMBERG/ 731 LEX) пишет: I tried doubling it from 200 MB to 400 MB. My initial test worked, but as I increased the number of fields that I was writing per entry, the same issue occurred again. So it seems

Re: IgniteCache.size() is hanging

2020-09-24 Thread Andrei Aleksandrov
Hi, Highly likely some of the nodes go offline and try to connect again. Probably you had some network issues. I think I will see this and other information in the logs. Can you provide them? BR, Andrei 9/24/2020 6:54 PM, Alan Ward пишет: The only log I see is from one of the server nodes

Re: OutOfMemoryException with Persistence and Eviction Enabled

2020-09-24 Thread Andrei Aleksandrov
. If you still will see the issue then I guess we should create the ticket: 1)Collect the logs 2)Provide the java code example 3)Provide the configuration of the nodes After that, we can take a look more deeply into it and if it's an issue then file JIRA. BR, Andrei 9/23/2020 7:36 PM, Mitchell

Re: Performance on Windows vs Linux

2020-07-23 Thread Andrei Aleksandrov
Got it. If everything is the same then it looks like I can't suggest something to you here but as I know most users run their solutions on Linux OS. Probably it can be related to some performance benefits of Linux OS but I don't think that this difference should be significant. 7/23/2020 7:34

Re: Performance on Windows vs Linux

2020-07-23 Thread Andrei Aleksandrov
the used operation system. However, can you describe your POC details? Probably it can help us to advise you something else. BR, Andrei 7/23/2020 12:45 PM, njcstreet пишет: Hi, I am about to start a proof of concept on Ignite and we have the option of deploying either on Windows Server 2016 or Red

Re: 2.8.1 : Ignite Security : Cache_Put event generated from a remote_client user action has subject uuid of Node that executes the request

2020-07-23 Thread Andrei Aleksandrov
that somebody on the user mail list can help here. You can try to ask one more time on the developer mail list. Also, you can try to investigate some third party security plugins in case if it's important to you. BR, Andrei 7/22/2020 4:17 PM, VeenaMithare пишет: Hi Team, 1. I noticed that this issue

Re: third-party persistance and junction table

2020-07-23 Thread Andrei Aleksandrov
Hi, Unfortunately, Ignite doesn't support such kind of relations out of the box. Ignite just translates it to third party data storage that used as cache-store. It's expected that inserts and updates will be rejected in case if they break some rules. BR, Andrei 7/21/2020 11:16 AM, Bastien

Re: Cache query exception when using generic type: class java.util.ArrayList cannot be cast to

2020-07-23 Thread Andrei Aleksandrov
StationDto. Please note that in case if you are going to change the ValueType of the cache then you must destroy it and then create with a new configuration. I guess values of different types in the same cache are a reason of your issue. BR, Andrei 7/16/2020 4:41 PM, xingjl6280 пишет: hi

Re: cache.getAsync() blocks if cluster is not activated.

2020-07-23 Thread Andrei Aleksandrov
)Can you please share a server and cache configuration? BR, Andrei 7/15/2020 9:57 PM, John Smith пишет: Hi, testing some failover scenarios etc... When we call cache.getAsync() and the state of the cluster is not active. It seems to block. I implemented a cache repository as follows and using

Re: Ignite node log file setup

2020-06-10 Thread Andrei Aleksandrov
Hi, Can you please attach the whole log file? BR, Andrei 6/9/2020 5:14 AM, kay пишет: Hello! I start up sh ./ignite.sh -J-DgridName=testGridName -v ./config/config-cache.xml and in config-cache.xml but server start failed. Is that not proper to set igniteInstanceName?? log is here

Re: Cache_Put event generated from a remote_client user action has subject uuid of Node that executes the request sometimes

2020-06-10 Thread Andrei Aleksandrov
Hi, Your approach is correct as for me. IgniteCache can help to hold all the information about started transactions and users. Using EventStorageSpi is a good way to handle the events. BR, Andrei 6/9/2020 10:06 AM, VeenaMithare пишет: Cache_Put event generated from a remote_client user

Re: Ignite.cache.loadcache.Does this method do Increamental Load?

2020-03-26 Thread Andrei Aleksandrov
should do the same for Ignite. However, you can use read-through and write-through properties. They can help you to make your updates in 3-rd party store using Ignite API: https://apacheignite.readme.io/docs/3rd-party-store#section-read-through-and-write-through BR, Andrei 3/23/2020 9:34 PM

Re: How to access IGFS file written one node from other node in cluster ??

2020-02-24 Thread Andrei Aleksandrov
files (not tables) then please use HDFS or Spark API directly. Ignite provides good Spark integration: https://apacheignite-fs.readme.io/docs/ignite-data-frame BR, Andrei 2/21/2020 6:52 PM, Preet пишет: Then how to create shared file system or is there any way to access/modify file written

Re: Slow cache updates with indexing module enabled

2020-02-14 Thread Andrei Aleksandrov
to highlite it on the development mail list. BR, Andrei 2/13/2020 5:50 PM, xero пишет: Hi Andrei, thanks for taking the time to answer my question. I will consider your suggestion if we decide to switch to a multiple tables approach that will require those JOIN considerations. But, in this case we have

Re: Scheduling Cache Refresh using Ignite

2020-02-14 Thread Andrei Aleksandrov
for reconnect. // Can proceed and use the same IgniteCompute instance. }| I can't say why your client was disconnected. Highly likely it's because of some network issues. You can try to take a look at server logs and find there *NODE_LEFT *or *NODE_FAILED *messages. BR, Andrei 2/14/2020 8:08 AM

Re: Ignite yarn resources keep on increasing

2020-02-13 Thread Andrei Aleksandrov
the binaries or forget to copy binaries to some node? BR, Andrei 2/13/2020 1:12 PM, ChandanS пишет: I am using ignite 2.7 version for ignite yarn deployment. I have my own spark application that start ignite yarn cluster and load data to ignite. It works fine in positive scenarios, but whenever

Re: Scheduling Cache Refresh using Ignite

2020-02-13 Thread Andrei Aleksandrov
Hi, Can you please attach the full logs with the mentioned exception? BTW I don't see any attaches in the previous message (probably user list can't do it). BR, Andrei 2/13/2020 3:44 PM, nithin91 пишет: Attached the bean file used -- Sent from: http://apache-ignite-users.70518.x6

Re: Slow cache updates with indexing module enabled

2020-02-13 Thread Andrei Aleksandrov
LLTABLE.id = BIGTABLE.id - incorrect Check the join order using the EXPLAIN command. BR, Andrei 2/12/2020 11:24 PM, xero пишет: Hi, We are experiencing slow updates to a cache with multiple indexed fields (around 25 indexes during testing but we expect to have many more) for updates that are onl

Re: Scheduling Cache Refresh using Ignite

2020-02-13 Thread Andrei Aleksandrov
or Kafka: https://apacheignite-mix.readme.io/docs/getting-started BR, Andrei 2/12/2020 9:11 PM, nithin91 пишет: Hi We are doing a a POC on exploring the Ignite in memory capabilities and building a rest api on top of it using node express. Currently as a part of POC, installed

Re: Ignite on yarn doesn't started

2020-02-13 Thread Andrei Aleksandrov
/filebrowser/view=/tmp/ignite/apache-ignite-2.7.6-bin.zip So it looks like you don't provide the Ignite binaries to your YARN deployment. BR, Andrei <http://ambari1.dmz.loc:/filebrowser/view=/tmp/ignite/apache-ignite-2.7.6-bin.zip> 2/11/2020 9:19 PM, v.shinkevich пишет: aealexsandrov

Re: Using EntryProcessor arguments recommendations

2020-02-13 Thread Andrei Aleksandrov
external keys inside EntryProcessor 2)Use unordered maps for invokeAll. TreeMap is suggested. BR, Andrei 2/12/2020 12:18 AM, Григорий Доможиров пишет: I see two options of using EntryProcessor: 1. Pass arguments like this: cache.invoke(key, new CustomProcessor(), someValue) 2. Pass stateful

Re: JDBC thin client incorrect security context

2020-02-13 Thread Andrei Aleksandrov
piException { /* No-op. */ } }| In case if this approach doesn't work for your implementation because of some issues then you can try to start the thread on Ignite developer mail list. BR, Andrei 2/12/2020 6:54 PM, VeenaMithare пишет: Hi , We have built a security and audit plugin for secur

Re: Issue with large number of Continuous Queries

2020-02-11 Thread Andrei Aleksandrov
suggest to don't set so more QE without reason. BR, Andrei 2/11/2020 9:42 AM, Dorah пишет: *Topology*: Server-1 --> Cache myCache, holds continuously updating data like market data(prices, status, tradetime etc) for instruments. InstrumentId is the key for this cache. Ser

Re: Ignite on yarn doesn't started

2020-02-11 Thread Andrei Aleksandrov
Duplicate of http://apache-ignite-users.70518.x6.nabble.com/Ignite-on-yarn-doesn-t-started-td31325.html 2/10/2020 11:00 PM, v.shinkevich пишет: Hi, All! I tried to run Ignite on Yarn My cluster properties: I also tried to add hdfs:// before paths, but result is the same. I created dir

Re: Ignite on yarn doesn't started

2020-02-11 Thread Andrei Aleksandrov
. Probably you should avoid using a temporary folder. However, the real reason we can see only from Ignite logs. BR, Andrei 2/11/2020 8:49 AM, v.shinkevich пишет: Hi, All! I tried to run Ignite on Yarn My cluster properties: #IGNITE_HOME = /tmp/ignite # The HDFS path to the Apache Ignite config

Re: Exception when joining a data node

2020-01-31 Thread Andrei Aleksandrov
ese steps, they will not be able to join each other. The solution to the current situation was also described in the message. You should clean the persistence store of the node that can't join to the cluster. Also I suggest to read next article: https://apacheignite.readme.io/docs/baseline-topology

Re: Cache.replace-K-V-V- performing slow

2020-01-31 Thread Andrei Aleksandrov
Hi, I have also described the possible problem here - https://stackoverflow.com/questions/59950157/why-apache-ignite-cache-replace-k-v-v-api-call-performing-slow BR, Andrei 1/31/2020 12:33 PM, Ilya Kasnacheev пишет: Hello! I'm not sure the benchmarking is relevant, since most of work

Re: "Adding entry to partition that is concurrently evicted" error

2020-01-31 Thread Andrei Aleksandrov
-failure-handling I am not sure about possible workarounds here (probably you can set the NoOpFailureHandler). You also can try to create the thread on developer user list: http://apache-ignite-developers.2346864.n4.nabble.com/Apache-Ignite-2-7-release-td34076i40.html BR, Andrei<http://apa

Re: Distinct Query too slow

2020-01-24 Thread Andrei Aleksandrov
next from any SQL tool: explain select distinct ipStart,ipEnd from IpContainerIpV4Data where subscriptionId = some_value; BR, Andrei 1/24/2020 3:23 PM, Prasad Bhalerao пишет: Hi, I am using Ignite 2.6 version. I have around total 6 million entries in my cache. Following sql is taking too

Re: How to set timeout while use data streamer in sql mode.

2020-01-07 Thread Andrei Aleksandrov
*timeout: Timeout, in milliseconds, that data streamer should use to flush data. By default, the data is flushed on connection close. Thin clients looks like don't have it yet. Probably you should ask about it on developers user list. BR, Andrei 1/2/2020 9:25 AM, yangjiajun пишет: Hello

Re: CEP and Sliding Window Documentation

2020-01-07 Thread Andrei Aleksandrov
dme.io/v1.0/docs/sliding-windows section contains information about expire policies and SQL operations. Expire policies sections: https://apacheignite.readme.io/docs/expiry-policies SQL documentation: https://apacheignite-sql.readme.io/docs BR, Andrei 1/3/2020 3:57 PM, narges saleh пишет:

Re: Streamer and data loss

2020-01-07 Thread Andrei Aleksandrov
, catch the exceptions, check the cache sizes, etc) and use another client for data loading in case if previous one was failed. BR, Andrei 1/6/2020 2:37 AM, narges saleh пишет: Hi All, Another question regarding ignite's streamer. What happens to the data if the streamer node crashes before

Re: Ignite Persistence: Baseline Topology

2020-01-06 Thread Andrei Aleksandrov
DataRegion is not configured: 1GB_Region_Eviction     at org.apache.ignite.internal.processors.cache.persistence.IgniteCacheDatabaseSharedManager.dataRegion(IgniteCacheDatabaseSharedManager.java:729) BR, Andrei 1/6/2020 2:52 PM, djm132 пишет: You can also look to this topic, probably related

Re: Ignite with Spark Intergration

2019-12-12 Thread Andrei Aleksandrov
on driver side using Ignition.start(). BR, Andrei 12/12/2019 12:17 PM, datta пишет: Stopped client node in the machine where spark worker node was running and started spark shell. It started a ignite client node from within. I have only 1 problem that how to specify ignite jvm options from spark

Re: Manage offset of KafkaStreamer

2019-12-10 Thread Andrei Aleksandrov
(org.apache.kafka.common.TopicPartition,%20long) but can't be configured. You can create the JIRA ticket for it. However, you can try to configure it using Kafka scripts like it described here: https://stackoverflow.com/questions/29791268/how-to-change-start-offset-for-topic BR, Andrei 12/3/2019 1:07 PM, ashishb888 пишет: I

Re: Ignite on-heap & off-heap caches

2019-12-02 Thread Andrei Aleksandrov
* memory uses *initial *and *max *properties for region size that should be set in the data region configuration. This memory will be used for data storage. Java GC will not work with current memory. You can read more here: https://apacheignite.readme.io/docs/memory-architecture BR, Andrei 11/28

Re: Ignite AI learning resources

2019-12-02 Thread Andrei Aleksandrov
)Ignite documentation portal - https://apacheignite.readme.io/docs/getting-started Also exist a lot of different articles in the internet. BR, Andrei 12/2/2019 11:47 AM, joseheitor пишет: Hi, Can anyone recommend some resources to learn the fundamentals of ML and DL, and how to use these techniques

Re: Cache data not being stored on server nodes

2019-12-02 Thread Andrei Aleksandrov
. BR, Andrei 12/2/2019 7:39 AM, swattal пишет: I have recently started using Ap= ache Ignite for my application and had questions about where the data gets stored in the cache. I have two nodes which act as Cached servers and another node which acts as a client. I am using the Zookeeper discovery

Re: Blocked Worker Thread

2019-11-08 Thread Andrei Aleksandrov
Hi Conrad, The reasons can be different. Could you please share the logs? BR, Andrei 11/7/2019 10:35 PM, Conrad Mukai (cmukai) пишет: We are running a cache in a 4 node cluster with atomicityMode set to ATOMIC and have persistence enabled. We repeatedly get a SYSTEM_WORKER_BLOCKED error

Re: Kubernetes- Failed to retrieve Ignite pods IP addresses.

2019-11-07 Thread Andrei Aleksandrov
Hi, Using the search I found that current issue was resolved in next thread: http://apache-ignite-users.70518.x6.nabble.com/Ignite-on-RBAC-enabled-K8s-cluster-td22165.html I guess that you should also read next article: https://apacheignite.readme.io/docs/rbac-authorization BR, Andrei 11/7

Re: apache ignite installation fails in kubernetes

2019-11-06 Thread Andrei Aleksandrov
Hi, As I know it should be fixed in Apache Ignite 2.7.6. Also as an option you can try to modify your current Docker file as next (master version): https://github.com/apache/ignite/blob/master/docker/apache-ignite/Dockerfile BR, Andrei 11/6/2019 10:33 AM, Gokulnath Chidambaram пишет

Re: Ignite YARN deployment - how to use TCP IP Discovery?

2019-11-06 Thread Andrei Aleksandrov
to connect each other. BR, Andrei 11/5/2019 9:15 PM, Seshan, Manoj N. (TR Tech, Content & Ops) пишет: We are using Ignite as a Distributed In-Memory cache, deployed using YARN on a Hadoop Cluster.  We have configured Zookeeper Discovery, and this is working fine. Given this is a small 20

Re: Not able to start second server node due to authentication failure

2019-11-06 Thread Andrei Aleksandrov
Hi, It's correct that SecurityContext is null in your case:     SecurityContext subj = spi.nodeAuth.authenticateNode(node, cred);     if (subj == null) {         // Node has not pass authentication.         LT.warn(log, "Authentication failed [nodeId=" + node.id() +             ", addrs=" +

Re: Get key or cache's updation time?

2019-10-11 Thread Andrei Aleksandrov
(but here you can get performance drop): https://apacheignite.readme.io/docs/events BR, Andrei 10/11/2019 4:08 PM, SidP пишет: Is there a way to know if key and/or cache is updation time? I want to check if key or cache is updated in last 10 sec or not? -- Sent from: http://apache-ignite

Re: Authorizing thin clients

2019-10-02 Thread Andrei Aleksandrov
Hi Kurt, Unfortunately from the box Ignite provide only simple username/password authentication for thin clients. You can read more about it here: https://apacheignite.readme.io/docs/advanced-security BR, Andrei 10/1/2019 3:08 PM, Kurt Semba пишет: Hi all, is there a way to define which

Re: Spark setup

2019-09-23 Thread Andrei Aleksandrov
$JARS ; do if [ "$eachjarinlib" != "ABCDEFGHIJKLMNOPQRSTUVWXYZ.JAR" ]; then     EXECUTOR_PATH=file:$eachjarinlib:$EXECUTOR_PATH fi done spark-submit --deploy-mode client --master spark://andrei-ThinkPad-P51s:7077 --conf "spark.driver.extraClassPath=$EXECUTOR_PATH"

Re: Authentication

2019-09-17 Thread Andrei Aleksandrov
s://apacheignite-sql.readme.io/docs/jdbc-driver#section-parameters 2)Thick java nodes: Implement your own plugin for security I see the answer from Evgenii here about it: https://stackoverflow.com/questions/46150920/custom-security-plugin-for-apache-ignite BR, Andrei 9/16/2019 1:26 PM, Kurt Semba п

Re: Authentication

2019-09-16 Thread Andrei Aleksandrov
/advanced-security#section-enable-authentication I created ticket on documentation: https://issues.apache.org/jira/browse/IGNITE-12170 BR, Andrei 9/16/2019 10:43 AM, Kurt Semba пишет: Hi all, I used the web-console to auto-generate some code and then extended the ServerNodeCodeStartup.java class

Re: Slowness in initial data load using COPY command

2019-09-12 Thread Andrei Aleksandrov
-sparktm-integration-using-ignite-rdds 2)Using java code (some logic to read CSV) + IgniteDataStreamer https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteDataStreamer.html 3)Using kafka integration https://apacheignite-mix.readme.io/docs/kafka-streamer BR, Andrei 9/12/2019 3:20

Re: Users named "ignite" cannot change their password?

2019-09-11 Thread Andrei Aleksandrov
Hi, In addition to Ilia answer, please read next page: https://apacheignite-sql.readme.io/docs/create-user#section-description BR, Andrei 9/4/2019 11:47 AM, Ilya Kasnacheev пишет: Hello! User names are case sensitive and bound by SQL rules, i.e.: ALTER USER "ignite" WITH PASSWOR

Re: Altered sql table (adding new columns) does not reflect in Spark shell

2019-09-10 Thread Andrei Aleksandrov
Hi, Yes, I can confirm that this is the issue. I filed next ticket for it: https://issues.apache.org/jira/browse/IGNITE-12159 BR, Andrei 9/7/2019 10:00 PM, Shravya Nethula пишет: Hi, *I created and altered the table using the following queries: * a. CREATE TABLE person (id LONG, name

Re: IgniteCache.invoke deadlock example

2019-09-09 Thread Andrei Aleksandrov
for provided one. Otherwise, entry processor will wait for these *other keys*. It's typical deadlock. Sorry, I will not provide the example but hopes that my explanation is clear. BR, Andrei 9/7/2019 6:31 PM, Evangelos Morakis пишет: Dear igniters, I would like to elicit your expert advice

Re: Cache expiry policy not deleting records from disk(native persistence)

2019-09-09 Thread Andrei Aleksandrov
Hello, I guess that generated WAL will take this disk space. Please read about WAL here: https://apacheignite.readme.io/docs/write-ahead-log Please provide the size of every folder under /opt/ignite/persistence. BR, Andrei 9/6/2019 9:45 PM, Shiva Kumar пишет: Hi all, I have set cache

Re: Ignite ignores cache config when putting entries through near cache

2019-09-09 Thread Andrei Aleksandrov
. BR, Andrei 9/6/2019 12:36 PM, Bartłomiej Stefański пишет: Hi, I have a problem with putting entries to partitioned or replicated cache through near cache on client node. Even when I configured cache on server to put values to off-heap space they are stored on heap. I already descibed it o

Re: Concurrent threads updating the same cache item

2019-09-05 Thread Andrei Aleksandrov
ocs/distributed-locks> BR, Andrei 9/5/2019 5:41 PM, Ari Erev пишет: Hello, This question is related to the question by “humenius”,  with subject: “Race condition and conflicts during cache modifications?” – but I believe it is a simpler case… When code is run on an Ignite serve

Re: Common table expressions preserved as view.

2019-09-05 Thread Andrei Aleksandrov
Hi, At the moment SQL views and temporary tables (CTE) don't fully supported in Ignite. The CTE syntax will just set the sub-query inside your SQL request. BR, Andrei 9/5/2019 4:36 PM, kresimir.horvat пишет: Hi, I noticed that some common table expressions defined in queries are left

Re: Can Ignite transaction manage cached and non-cached data writes?

2019-09-03 Thread Andrei Aleksandrov
Hi, By default Ignite will handle only updates that were done using Ignite cache API. I guess that you have already read next article: https://www.gridgain.com/resources/blog/apache-ignite-transactions-architecture-transaction-handling-level-3rd-party BR, Andrei 9/3/2019 2:03 PM, bijunathg

Re: Job Stealing node not stealing jobs

2019-09-03 Thread Andrei Aleksandrov
try to do. Possible job stealing didn't work because of your weak node began executions of some tasks in public pool but just do it longer then faster one. Could you please share your full reproducer for investigation? BR, Andrei 9/3/2019 1:43 PM, Pascoe Scholle пишет: HI there, I have

Re: Cache spreading to new nodes

2019-08-12 Thread Andrei Aleksandrov
Hi, Could you share the whole reproducer with all configurations and required methods? BR, Andrei 8/12/2019 4:48 PM, Marco Bernagozzi пишет: I have a set of nodes, and I want to be able to set a cache in specific nodes. It works, but whenever I turn on a new node the cache is automatically

Re: TimeoutException not wrapped in CacheException

2019-08-09 Thread Andrei Aleksandrov
, Andrei 8/9/2019 6:21 PM, Andrey Davydov пишет: As I see In javadocs for org.apache.ignite.transactions.Transaction     /**      * Commits this transaction by initiating {@code two-phase-commit} process.      *      * @throws IgniteException If commit failed.      * @throws

Re: TimeoutException not wrapped in CacheException

2019-08-09 Thread Andrei Aleksandrov
tException;     } So could you please provide the reproducer for your issue? We will check it and create the JIRA for it. BR, Andrei 8/9/2019 5:16 PM, Andrey Davydov пишет: On ignite 2.7.5 I got TransactionTimeoutException not wrapped in CacheException. If it is normal behaviour and I should

Re: Ignite Spark Example Question

2019-08-09 Thread Andrei Aleksandrov
id is the primary key In Ignite you will see only {id=1, name=Nikita, age=26}. Also here you can see the code sample for you and other information about SaveModes: https://apacheignite-fs.readme.io/docs/ignite-data-frame#section-saving-dataframes BR, Andrei On 2019/08/08 17:33:39, sri hari

Re: IgniteCache.lock behaviour on a key that doesn't exist in the cache (yet)

2019-08-07 Thread Andrei Aleksandrov
the default value 3)Run getAndPutIfAbsent(key, defaultValue) If another client was able to put the value during current prepare the default then you will get that value, otherwise default value will be returned. BR, Andrei 8/7/2019 12:08 PM, Yohan Fernando пишет: I'm trying to write a transaction

  1   2   >