Re: Read Write through cache

2023-07-18 Thread Arunima Barik
How does write through work? I mean if I add a row in Ignite dataframe, how does it reflect to spark? I have 50 rows in Ignite and all 100 rows in Spark. If I perform a union all, wont the performance degrade? I mean if will get slower than just querying spark On Tue, 18 Jul, 2023, 10:43 pm

Re: Ignite SQL

2023-07-18 Thread Arunima Barik
I have a huge dataset and I am keeping few (say 100) rows in Ignite and the entire dataset remains in Spark When I query Ignite I want to write an Sql query to perform the same. Does option 1 still hold good? On Tue, 18 Jul, 2023, 10:40 pm Stephen Darlington, < stephen.darling...@gridgain.com>

Cache write synchronization mode

2023-07-18 Thread Raymond Wilson
I have a query regarding the CacheWriteSynchronizationMode in CacheConfiguration. This enum is defined like this in the .Net client: public enum CacheWriteSynchronizationMode { /// /// Mode indicating that Ignite should wait for write or commit replies from all nodes. /// This

Re: Possible WAL corruption on running system during K8s update

2023-07-18 Thread Raymond Wilson
Hi Alex, Here is the log from the Ignite startup. It's fairly short but shows everything I think: 2023-07-17 22:38:55,061 [1] DBG [ImmutableCacheComputeServer] Starting Ignite.NET 2.15.0.23172 2023-07-17 22:38:55,065 [1] DBG [ImmutableCacheComputeServer] 2023-07-17 22:38:55,068 [1] DBG

Re: Possible WAL corruption on running system during K8s update

2023-07-18 Thread Raymond Wilson
Hi Alex, We are using Ignite v2.15. I will track down the additional log information and reply on this thread. Raymond. On Wed, Jul 19, 2023 at 2:55 AM Alex Plehanov wrote: > Hello, > > Which Ignite version do you use? > Please share exception details after "Exception during start

Re: Read Write through cache

2023-07-18 Thread Stephen Darlington
Write through works regardless of how you insert data into Ignite. I’m not clear what you mean by federated query. Are the records in Spark a subset of those in the cache? Assuming not, create a data frame with a SQL query against Ignite. Create a data frame with a SQL query against your Spark

Re: Ignite SQL

2023-07-18 Thread Stephen Darlington
“Correct” is hard to quantify without knowing your use case, but option 1 is probably what you want. Spark pushes down SQL execution to Ignite, so you get all the distribution, use of indexes, etc. > On 14 Jul 2023, at 16:12, Arunima Barik wrote: > > Hello team > > What is the correct way

Re: Possible WAL corruption on running system during K8s update

2023-07-18 Thread Alex Plehanov
Hello, Which Ignite version do you use? Please share exception details after "Exception during start processors, node will be stopped and close connections" (there should be a reason in the log, why the page delta can't be applied). вт, 18 июл. 2023 г. в 05:05, Raymond Wilson : > Hi, > > We run