Re: Ignite H2 to Calcite issues
Hi Alex,
Also looks like the query does work with this error/warning. But not sure
what's the implication on the result or overall performance.
Thanks,
Amit Jolly
On Fri, Aug 16, 2024 at 10:03 AM Amit Jolly wrote:
> Hi Alex,
>
> Find below the details you requested.
>
> CacheConfiguration cacheConfiguration = new
> CacheConfiguration<>();
> cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
> cacheConfiguration.setBackups(2);
> cacheConfiguration.setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(new
> Duration(MINUTES, 30)));
> cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
> cacheConfiguration.setName("ORDERSTOMATCH");
> cacheConfiguration.setSqlSchema("PUBLIC");
> *cacheConfiguration.setIndexedTypes(OrderId.class, OrdersToMatch.class);*
> cacheConfiguration.setOnheapCacheEnabled(true);
> cacheConfiguration.setEvictionPolicyFactory(new
> LruEvictionPolicyFactory<>());
>
> explain plan for
>
> SELECT _key FROM PUBLIC.ORDERSTOMATCH
>
>
> IgniteExchange(distribution=[single]): rowcount = 1.0, cumulative cost =
> IgniteCost [rowCount=2.0, cpu=2.0, memory=1.0, io=1.0, network=5.0], id =
> 1098
> IgniteIndexScan(table=[[PUBLIC, ORDERSTOMATCH]], index=[_key_PK],
> requiredColumns=[{0}], inlineScan=[false], collation=[[0
> ASC-nulls-first]]): rowcount = 1.0, cumulative cost = IgniteCost
> [rowCount=1.0, cpu=1.0, memory=1.0, io=1.0, network=1.0], id = 1087
>
> It is using Index scan.
>
> Thanks,
>
> Amit Jolly
>
>
> On Fri, Aug 16, 2024 at 3:03 AM Alex Plehanov
> wrote:
>
>> Amit,
>>
>> Can you please show the output of "EXPLAIN PLAN FOR "?
>> I've found the bug in index scan on binary object field (ticket [1]),
>> but I can't reproduce it with select without "order by", or without
>> forcing index scan.
>>
>> [1]: https://issues.apache.org/jira/browse/IGNITE-23004
>>
>> ср, 14 авг. 2024 г. в 18:07, Amit Jolly :
>> >
>> > HI, Jeremy & Alex,
>> >
>> > First of all thanks for the quick response.
>> >
>> > Sorry for the confusion here, We are trying to switch from using H2 as
>> a default query engine to Calcite due to above mentioned CVE's.
>> > I have read the ticket and understood that those CVE's do not have any
>> impact on how Ignite uses H2.
>> > We are trying to explain the same to our internal security audit team
>> and at the same time trying to evaluate Calcite.
>> >
>> > Below is the query we are using
>> >
>> > SELECT _key FROM OrdersToMatchCache
>> >
>> > OrdersToMatchCache has OrderId.java as key and OrdersToMatch.java as
>> value
>> >
>> > OrderId.java
>> >
>> > import lombok.Data;
>> > import org.apache.ignite.cache.query.annotations.QuerySqlField;
>> > import java.io.Serializable;
>> >
>> > @Data
>> > public class OrderId implements Serializable {
>> > @QuerySqlField
>> > private String orderId;
>> > @QuerySqlField
>> > private String regionId;
>> > @QuerySqlField
>> > private Integer date;
>> > }
>> >
>> >
>> > OrdersToMatch.java
>> >
>> > import lombok.Data;
>> > import org.apache.ignite.cache.query.annotations.QuerySqlField;
>> > import java.io.Serializable;
>> >
>> > @Data
>> > public class OrdersToMatch implements Serializable {
>> > @QuerySqlField
>> > private List buyOrders = new ArrayList<>();
>> > @QuerySqlField
>> > private List sellOrders = new ArrayList<>();
>> > }
>> >
>> >
>> > Order.java
>> >
>> > import lombok.Data;
>> > import java.io.Serializable;
>> >
>> > @Data
>> > public class Order implements Serializable {
>> > private String direction; // BUY or SELL
>> > private Integer qty;
>> > }
>> >
>> > Thanks,
>> >
>> > Amit Jolly
>> >
>> > On Wed, Aug 14, 2024 at 10:27 AM Jeremy McMillan
>> wrote:
>> >>
>> >> Amit:
>> >>
>> >> I'm concerned that you may be misreading the CVE details in the ticket
>> you cited, since you indicated you are moving TO H2.. Also the stack trace
>> is a Calcite stack trace. This is ambiguous whether this is the before
>> (persistence config changes) depicted or after changing persiste
Re: Ignite H2 to Calcite issues
Hi Alex,
Find below the details you requested.
CacheConfiguration cacheConfiguration = new
CacheConfiguration<>();
cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
cacheConfiguration.setBackups(2);
cacheConfiguration.setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(new
Duration(MINUTES, 30)));
cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cacheConfiguration.setName("ORDERSTOMATCH");
cacheConfiguration.setSqlSchema("PUBLIC");
*cacheConfiguration.setIndexedTypes(OrderId.class, OrdersToMatch.class);*
cacheConfiguration.setOnheapCacheEnabled(true);
cacheConfiguration.setEvictionPolicyFactory(new
LruEvictionPolicyFactory<>());
explain plan for
SELECT _key FROM PUBLIC.ORDERSTOMATCH
IgniteExchange(distribution=[single]): rowcount = 1.0, cumulative cost =
IgniteCost [rowCount=2.0, cpu=2.0, memory=1.0, io=1.0, network=5.0], id =
1098
IgniteIndexScan(table=[[PUBLIC, ORDERSTOMATCH]], index=[_key_PK],
requiredColumns=[{0}], inlineScan=[false], collation=[[0
ASC-nulls-first]]): rowcount = 1.0, cumulative cost = IgniteCost
[rowCount=1.0, cpu=1.0, memory=1.0, io=1.0, network=1.0], id = 1087
It is using Index scan.
Thanks,
Amit Jolly
On Fri, Aug 16, 2024 at 3:03 AM Alex Plehanov
wrote:
> Amit,
>
> Can you please show the output of "EXPLAIN PLAN FOR "?
> I've found the bug in index scan on binary object field (ticket [1]),
> but I can't reproduce it with select without "order by", or without
> forcing index scan.
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-23004
>
> ср, 14 авг. 2024 г. в 18:07, Amit Jolly :
> >
> > HI, Jeremy & Alex,
> >
> > First of all thanks for the quick response.
> >
> > Sorry for the confusion here, We are trying to switch from using H2 as a
> default query engine to Calcite due to above mentioned CVE's.
> > I have read the ticket and understood that those CVE's do not have any
> impact on how Ignite uses H2.
> > We are trying to explain the same to our internal security audit team
> and at the same time trying to evaluate Calcite.
> >
> > Below is the query we are using
> >
> > SELECT _key FROM OrdersToMatchCache
> >
> > OrdersToMatchCache has OrderId.java as key and OrdersToMatch.java as
> value
> >
> > OrderId.java
> >
> > import lombok.Data;
> > import org.apache.ignite.cache.query.annotations.QuerySqlField;
> > import java.io.Serializable;
> >
> > @Data
> > public class OrderId implements Serializable {
> > @QuerySqlField
> > private String orderId;
> > @QuerySqlField
> > private String regionId;
> > @QuerySqlField
> > private Integer date;
> > }
> >
> >
> > OrdersToMatch.java
> >
> > import lombok.Data;
> > import org.apache.ignite.cache.query.annotations.QuerySqlField;
> > import java.io.Serializable;
> >
> > @Data
> > public class OrdersToMatch implements Serializable {
> > @QuerySqlField
> > private List buyOrders = new ArrayList<>();
> > @QuerySqlField
> > private List sellOrders = new ArrayList<>();
> > }
> >
> >
> > Order.java
> >
> > import lombok.Data;
> > import java.io.Serializable;
> >
> > @Data
> > public class Order implements Serializable {
> > private String direction; // BUY or SELL
> > private Integer qty;
> > }
> >
> > Thanks,
> >
> > Amit Jolly
> >
> > On Wed, Aug 14, 2024 at 10:27 AM Jeremy McMillan
> wrote:
> >>
> >> Amit:
> >>
> >> I'm concerned that you may be misreading the CVE details in the ticket
> you cited, since you indicated you are moving TO H2.. Also the stack trace
> is a Calcite stack trace. This is ambiguous whether this is the before
> (persistence config changes) depicted or after changing persistence
> depicted.
> >>
> >> A) The CVEs cited in the ticket
> https://issues.apache.org/jira/browse/IGNITE-15241 are all H2
> vulnerabilities.
> >> B) The H2 vulnerabilities cited all involve behaviors of H2 that Ignite
> does not use, therefore Ignite is affected neither by Calcite nor H2
> persistence involvement.
> >>
> >> I don't want to discourage you from moving from H2 to Calcite, but
> maybe this isn't as urgent as it appears, so please proceed carefully. As
> Alex requested, it will be helpful for the community to see which queries
> produce exceptions and which ones do not. H2 and Calcite have different SQL
> p
Re: Ignite H2 to Calcite issues
HI, Jeremy & Alex,
First of all thanks for the quick response.
Sorry for the confusion here, We are trying to switch from using H2 as a
default query engine to Calcite due to above mentioned CVE's.
I have read the ticket and understood that those CVE's do not have any
impact on how Ignite uses H2.
We are trying to explain the same to our internal security audit team and
at the same time trying to evaluate Calcite.
Below is the query we are using
SELECT _key FROM OrdersToMatchCache
OrdersToMatchCache has OrderId.java as key and OrdersToMatch.java as value
OrderId.java
import lombok.Data;
import org.apache.ignite.cache.query.annotations.QuerySqlField;
import java.io.Serializable;
@Data
public class OrderId implements Serializable {
@QuerySqlField
private String orderId;
@QuerySqlField
private String regionId;
@QuerySqlField
private Integer date;
}
OrdersToMatch.java
import lombok.Data;
import org.apache.ignite.cache.query.annotations.QuerySqlField;
import java.io.Serializable;
@Data
public class OrdersToMatch implements Serializable {
@QuerySqlField
private List buyOrders = new ArrayList<>();
@QuerySqlField
private List sellOrders = new ArrayList<>();
}
Order.java
import lombok.Data;
import java.io.Serializable;
@Data
public class Order implements Serializable {
private String direction; // BUY or SELL
private Integer qty;
}
Thanks,
Amit Jolly
On Wed, Aug 14, 2024 at 10:27 AM Jeremy McMillan wrote:
> Amit:
>
> I'm concerned that you may be misreading the CVE details in the ticket you
> cited, since you indicated you are moving TO H2.. Also the stack trace is a
> Calcite stack trace. This is ambiguous whether this is the before
> (persistence config changes) depicted or after changing persistence
> depicted.
>
> A) The CVEs cited in the ticket
> https://issues.apache.org/jira/browse/IGNITE-15241 are all H2
> vulnerabilities.
> B) The H2 vulnerabilities cited all involve behaviors of H2 that Ignite
> does not use, therefore Ignite is affected neither by Calcite nor H2
> persistence involvement.
>
> I don't want to discourage you from moving from H2 to Calcite, but maybe
> this isn't as urgent as it appears, so please proceed carefully. As Alex
> requested, it will be helpful for the community to see which queries
> produce exceptions and which ones do not. H2 and Calcite have different SQL
> parsers and query planners and underlying implementations, so it should not
> be surprising that queries might need rework in the course of switching.
> You should expect to encounter issues like this one, and others like it.
> It's a migration effort.
>
>
> On Tue, Aug 13, 2024 at 9:17 AM Amit Jolly wrote:
>
>> Hi,
>>
>> We are trying to switch to H2 due to Security Vulnerabilities as listed
>> in JIRA https://issues.apache.org/jira/browse/IGNITE-15241
>>
>> We are seeing below errors post switching. We are just running select *
>> From table query.
>>
>> Caused by: java.lang.ClassCastException: class
>> org.apache.ignite.internal.binary.BinaryObjectImpl cannot be cast to class
>> java.lang.Comparable (org.apache.ignite.internal.binary.BinaryObjectImpl is
>> in unnamed module of loader 'app'; java.lang.Comparable is in module
>> java.base of loader 'bootstrap')
>> at
>> org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:223)
>> ~[ignite-calcite-2.16.0.jar:2.16.0]
>> at
>> org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.access$100(ExpressionFactoryImpl.java:85)
>> ~[ignite-calcite-2.16.0.jar:2.16.0]
>> at
>> org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl$1.compare(ExpressionFactoryImpl.java:157)
>> ~[ignite-calcite-2.16.0.jar:2.16.0]
>> at
>> java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:560)
>> ~[?:?]
>> at
>> java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:660)
>> ~[?:?]
>> at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:637) ~[?:?]
>> at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:330) ~[?:?]
>> at
>> org.apache.ignite.internal.processors.query.calcite.exec.rel.Inbox.pushOrdered(Inbox.java:239)
>> ~[ignite-calcite-2.16.0.jar:2.16.0]
>> at
>> org.apache.ignite.internal.processors.query.calcite.exec.rel.Inbox.push(Inbox.java:201)
>> ~[ignite-calcite-2.16.0.jar:2.16.0]
>> at
>> org.apache.ignite.internal.processors.query.calcite.exec.rel.Inbox.onBatchReceived(Inbox.java:177)
>
Ignite H2 to Calcite issues
Hi, We are trying to switch to H2 due to Security Vulnerabilities as listed in JIRA https://issues.apache.org/jira/browse/IGNITE-15241 We are seeing below errors post switching. We are just running select * >From table query. Caused by: java.lang.ClassCastException: class org.apache.ignite.internal.binary.BinaryObjectImpl cannot be cast to class java.lang.Comparable (org.apache.ignite.internal.binary.BinaryObjectImpl is in unnamed module of loader 'app'; java.lang.Comparable is in module java.base of loader 'bootstrap') at org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:223) ~[ignite-calcite-2.16.0.jar:2.16.0] at org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl.access$100(ExpressionFactoryImpl.java:85) ~[ignite-calcite-2.16.0.jar:2.16.0] at org.apache.ignite.internal.processors.query.calcite.exec.exp.ExpressionFactoryImpl$1.compare(ExpressionFactoryImpl.java:157) ~[ignite-calcite-2.16.0.jar:2.16.0] at java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:560) ~[?:?] at java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:660) ~[?:?] at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:637) ~[?:?] at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:330) ~[?:?] at org.apache.ignite.internal.processors.query.calcite.exec.rel.Inbox.pushOrdered(Inbox.java:239) ~[ignite-calcite-2.16.0.jar:2.16.0] at org.apache.ignite.internal.processors.query.calcite.exec.rel.Inbox.push(Inbox.java:201) ~[ignite-calcite-2.16.0.jar:2.16.0] at org.apache.ignite.internal.processors.query.calcite.exec.rel.Inbox.onBatchReceived(Inbox.java:177) ~[ignite-calcite-2.16.0.jar:2.16.0] at org.apache.ignite.internal.processors.query.calcite.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:324) ~[ignite-calcite-2.16.0.jar:2.16.0] at org.apache.ignite.internal.processors.query.calcite.exec.ExchangeServiceImpl.lambda$init$2(ExchangeServiceImpl.java:195) ~[ignite-calcite-2.16.0.jar:2.16.0] at org.apache.ignite.internal.processors.query.calcite.message.MessageServiceImpl.onMessageInternal(MessageServiceImpl.java:276) ~[ignite-calcite-2.16.0.jar:2.16.0] at org.apache.ignite.internal.processors.query.calcite.message.MessageServiceImpl.lambda$onMessage$0(MessageServiceImpl.java:254) ~[ignite-calcite-2.16.0.jar:2.16.0] at org.apache.ignite.internal.processors.query.calcite.exec.QueryTaskExecutorImpl.lambda$execute$0(QueryTaskExecutorImpl.java:66) ~[ignite-calcite-2.16.0.jar:2.16.0] Any idea what might be causing this? Regards, Amit Jolly
Re: Will Apache Ignite 3.0 be compliant with JSR107 sepc?
Hi Pavel, Thanks for the quick response. We are currently evaluating Apache Ignite for our project and are planning to use features like *CacheEntryProcessor*, *Thick clients with Near Cache *(Looks like 3.0 will only have thin clients and near cache is only supported in thick clients) and *Continues query cache*, while going through the code of 3.0, I could not find/validate whether these features will be supported in 3.0 or not. Is there any matrix which explains/shows feature to feature comparison between 2.X and 3.0? Thanks, Amit Jolly On Mon, Jun 3, 2024 at 1:41 AM Pavel Tupitsyn wrote: > Amit, unfortunately, I don't have answers at the moment. > > I think a JSR107 wrapper can be developed on top of existing Ignite 3 APIs > (Table + Compute), including CacheEntryProcessor support, but we don't have > specific plans for now. > > On Fri, May 31, 2024 at 4:34 PM Amit Jolly wrote: > >> Hi Pavel, >> >> Thanks for the quick response. >> >> I had looked at the ignite-3 github repo and could not find any reference >> to JSR 107, hence asked this question. >> >> Since Ignite 2.X is fully JSR 107 compliant, now the question is if >> ignite-3 is going to be the successor of Ignite 2.X and is going to replace >> Ignite 2.X in future, will Ignite 3 be JSR 107 compliant as well? If yes, >> do we have timelines when ignite-3 will be JSR 107 compliant, If no, what >> will be the migration strategy for the current Ignite 2.X user using >> features listed in JSR 107. >> >> Thanks, >> >> Amit Jolly >> >> >> >> >> >> On Fri, May 31, 2024 at 12:08 AM Pavel Tupitsyn >> wrote: >> >>> For now it does not have any of that. KeyValueView [1] is a table access >>> interface in Ignite 3 that is most similar to a "cache". >>> >>> >>> https://github.com/apache/ignite-3/blob/main/modules/api/src/main/java/org/apache/ignite/table/KeyValueView.java >>> >>> On Thu, May 30, 2024 at 6:19 PM Amit Jolly wrote: >>> >>>> HI, >>>> >>>> Will Apache Ignite 3.0 be compliant with JSR107 sepc? >>>> >>>> In particular, I am looking at the feature CacheEntryProcessor support >>>> in Ignite 3.0 >>>> >>>> >>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#invoke-K-org.apache.ignite.cache.CacheEntryProcessor-java.lang.Object...- >>>> >>>> >>>> https://www.javadoc.io/doc/javax.cache/cache-api/latest/javax/cache/processor/EntryProcessor.html >>>> >>>> Thanks, >>>> >>>> Amit Jolly >>>> >>>
Re: Will Apache Ignite 3.0 be compliant with JSR107 sepc?
Hi Pavel, Thanks for the quick response. I had looked at the ignite-3 github repo and could not find any reference to JSR 107, hence asked this question. Since Ignite 2.X is fully JSR 107 compliant, now the question is if ignite-3 is going to be the successor of Ignite 2.X and is going to replace Ignite 2.X in future, will Ignite 3 be JSR 107 compliant as well? If yes, do we have timelines when ignite-3 will be JSR 107 compliant, If no, what will be the migration strategy for the current Ignite 2.X user using features listed in JSR 107. Thanks, Amit Jolly On Fri, May 31, 2024 at 12:08 AM Pavel Tupitsyn wrote: > For now it does not have any of that. KeyValueView [1] is a table access > interface in Ignite 3 that is most similar to a "cache". > > > https://github.com/apache/ignite-3/blob/main/modules/api/src/main/java/org/apache/ignite/table/KeyValueView.java > > On Thu, May 30, 2024 at 6:19 PM Amit Jolly wrote: > >> HI, >> >> Will Apache Ignite 3.0 be compliant with JSR107 sepc? >> >> In particular, I am looking at the feature CacheEntryProcessor support in >> Ignite 3.0 >> >> >> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#invoke-K-org.apache.ignite.cache.CacheEntryProcessor-java.lang.Object...- >> >> >> https://www.javadoc.io/doc/javax.cache/cache-api/latest/javax/cache/processor/EntryProcessor.html >> >> Thanks, >> >> Amit Jolly >> >
Will Apache Ignite 3.0 be compliant with JSR107 sepc?
HI, Will Apache Ignite 3.0 be compliant with JSR107 sepc? In particular, I am looking at the feature CacheEntryProcessor support in Ignite 3.0 https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#invoke-K-org.apache.ignite.cache.CacheEntryProcessor-java.lang.Object...- https://www.javadoc.io/doc/javax.cache/cache-api/latest/javax/cache/processor/EntryProcessor.html Thanks, Amit Jolly
