[jira] [Created] (IGNITE-12902) Concurrent modification in time to iterate by events
Kirill Tkalenko created IGNITE-12902:
Summary: Concurrent modification in time to iterate by events
Key: IGNITE-12902
URL: https://issues.apache.org/jira/browse/IGNITE-12902
Project: Ignite
Issue Type: Improvement
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
{noformat}[10:20:37]W: [org.apache.ignite:ignite-core] [2020-02-20
10:20:37,324][ERROR][main][CacheExchangeMergeTest9] Failed to pre-stop
processor: GridProcessorAdapter []
[10:20:37]W: [org.apache.ignite:ignite-core]
java.util.ConcurrentModificationException
[10:20:37]W: [org.apache.ignite:ignite-core] at
java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
[10:20:37]W: [org.apache.ignite:ignite-core] at
java.util.ArrayList$Itr.next(ArrayList.java:859)
[10:20:37]W: [org.apache.ignite:ignite-core] at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:2302)
[10:20:37]W: [org.apache.ignite:ignite-core] at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onDone(GridDhtPartitionsExchangeFuture.java:142)
[10:20:37]W: [org.apache.ignite:ignite-core] at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:464)
[10:20:37]W: [org.apache.ignite:ignite-core] at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStop0(GridCachePartitionExchangeManager.java:821)
[10:20:37]W: [org.apache.ignite:ignite-core] at
org.apache.ignite.internal.processors.cache.GridCacheSharedManagerAdapter.onKernalStop(GridCacheSharedManagerAdapter.java:120)
[10:20:37]W: [org.apache.ignite:ignite-core] at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStop(GridCacheProcessor.java:972)\{noformat}
In that place we are going over event’s collection, looking for server leave’s
event:
{code}
for (DiscoveryEvent evt : evts.events()) {
if (serverLeftEvent(evt)) {
for (CacheGroupContext grp : cctx.cache().cacheGroups())
grp.affinityFunction().removeNode(evt.eventNode().id());
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12901) SQL: Uncorrelated subquery should run only once.
Alexey Kukushkin created IGNITE-12901:
-
Summary: SQL: Uncorrelated subquery should run only once.
Key: IGNITE-12901
URL: https://issues.apache.org/jira/browse/IGNITE-12901
Project: Ignite
Issue Type: Bug
Affects Versions: 2.8
Reporter: Alexey Kukushkin
Currently uncorrelated subqueries (where subquery is not depends on the outer
query) are executed on each nested loop iteration in the
org.h2.command.dml.Select#isConditionMet method.
We may avoid this, for example, using results caching.
h2. Reproducer
{code:java}
public class SubQueryTest extends AbstractIndexingCommonTest {
/** Keys counts at the RIGHT table. */
private static final int RIGHT_CNT = 10;
/** Keys counts at the LEFT table. */
private static final int LEFT_CNT = 50;
/** {@inheritDoc} */
@SuppressWarnings("unchecked")
@Override protected void beforeTest() throws Exception {
super.beforeTest();
startGrids(1);
IgniteCache cacheA = grid(0).createCache(new CacheConfiguration()
.setName("A")
.setSqlSchema("TEST")
.setQueryEntities(Collections.singleton(new
QueryEntity(Long.class.getTypeName(), "A_VAL")
.setTableName("A")
.addQueryField("ID", Long.class.getName(), null)
.addQueryField("JID", Long.class.getName(), null)
.addQueryField("VAL", Long.class.getName(), null)
.setKeyFieldName("ID")
)));
IgniteCache cacheB = grid(0).createCache(new CacheConfiguration()
.setCacheMode(CacheMode.REPLICATED)
.setName("B")
.setSqlSchema("TEST")
.setQueryEntities(Collections.singleton(new
QueryEntity(Long.class.getName(), "B_VAL")
.setTableName("B")
.addQueryField("ID", Long.class.getName(), null)
.addQueryField("A_JID", Long.class.getName(), null)
.addQueryField("VAL0", String.class.getName(), null)
.setKeyFieldName("ID")
)));
Map batch = new HashMap<>();
for (long i = 0; i < LEFT_CNT; ++i) {
batch.put(i, grid(0).binary().builder("A_VAL")
.setField("JID", i % RIGHT_CNT)
.setField("VAL", i)
.build());
if (batch.size() > 1000) {
cacheA.putAll(batch);
batch.clear();
}
}
if (batch.size() > 0) {
cacheA.putAll(batch);
batch.clear();
}
for (long i = 0; i < RIGHT_CNT; ++i)
cacheB.put(i, grid(0).binary().builder("B_VAL")
.setField("A_JID", i)
.setField("VAL0", String.format("val%03d", i))
.build());
}
/** {@inheritDoc} */
@Override protected void afterTest() throws Exception {
stopAllGrids();
super.afterTest();
}
/**
* Test local query execution.
*/
@Test
public void test() {
sql(true, "SELECT * FROM A WHERE A.JID IN (SELECT A_JID FROM
B)").getAll();
}
/**
* @param enforceJoinOrder Enforce join order mode.
* @param sql SQL query.
* @param args Query parameters.
* @return Results cursor.
*/
private FieldsQueryCursor> sql(boolean enforceJoinOrder, String
sql, Object... args) {
return grid(0).context().query().querySqlFields(new SqlFieldsQuery(sql)
.setSchema("TEST")
.setLazy(true)
.setEnforceJoinOrder(enforceJoinOrder)
.setArgs(args), false);
}
}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-12900) Calcite integration. Use RowHandler to access fields.
Igor Seliverstov created IGNITE-12900: - Summary: Calcite integration. Use RowHandler to access fields. Key: IGNITE-12900 URL: https://issues.apache.org/jira/browse/IGNITE-12900 Project: Ignite Issue Type: Task Reporter: Igor Seliverstov Currently only Object[] can be used as a row because most of execution nodes require Object[] as a row type. Let's use generic row types with appropriate RowHandler in execution nodes and compiled expressions to get more flexibility. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSSION] Major changes in Ignite in 2020
Hello, Igniters. I am going to contribute a new feature - profiling tool and performance report. This is part of IEP-35. [1] The tool will be able to collect performance statistics and create a human-readable report. It will help to analyze workload and to tune configuration and applications. Example of report [2, 3, 4]. [1] https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool [2] https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png [3] https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png [4] https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png сб, 11 апр. 2020 г. в 13:54, Alexei Scherbakov : > > Folks, > > I keep working on tasks related to data consistency. > > This includes: > > Lost partitions handling overhaul (almost done) and tombstones support > aforementioned by Ivan Rakov > Atomic protocol overhaul (see [1]) > > The ultimate goal of the year is to prepare Ignite for passing Jepsen tests. > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/IEP-12+Make+ATOMIC+Caches+Consistent+Again > > > пт, 10 апр. 2020 г. в 18:49, Denis Magda : > > > Steven, > > > > Please start a dedicated discussion for the Golang support. At the moment, > > I'm not aware if anybody from the community planned to provide support > > out-of-the-box. However, that's not a tricky task thanks to Ignite's binary > > protocol that lets enable any programming language support easily. > > > > - > > Denis > > > > > > On Fri, Apr 10, 2020 at 8:43 AM smeadows-abb > > wrote: > > > > > First thanks for your quick response. > > > > > > I looked at https://github.com/amsokol/ignite-go-client and its NOT > > > completed and nothing has been for last 16 months. Initial test with > > > package > > > failed, so trying to determine your project roadmap with regards to > > Golang > > > and maybe Rust support. > > > > > > I'm NOT sure of 'AFAIK' ? > > > > > > We may need to implement your Restful API to provide support for Golang > > and > > > Rust, provided it's complete? > > > > > > Thanks, > > > Steve > > > > > > > > > > > > > > > > > > -- > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > -- > > Best regards, > Alexei Scherbakov -- Best wishes, Amelchev Nikita
Re: [DISCUSSION] Major changes in Ignite in 2020
Hello, I plan to implement cache key rotation for TDE (phase 3) [1][2]. (At the moment, the design is very draft and will be changed soon). [1] https://issues.apache.org/jira/browse/IGNITE-12843 [2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652384 сб, 11 апр. 2020 г. в 13:54, Alexei Scherbakov : > > Folks, > > I keep working on tasks related to data consistency. > > This includes: > > Lost partitions handling overhaul (almost done) and tombstones support > aforementioned by Ivan Rakov > Atomic protocol overhaul (see [1]) > > The ultimate goal of the year is to prepare Ignite for passing Jepsen tests. > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/IEP-12+Make+ATOMIC+Caches+Consistent+Again > > > пт, 10 апр. 2020 г. в 18:49, Denis Magda : > > > Steven, > > > > Please start a dedicated discussion for the Golang support. At the moment, > > I'm not aware if anybody from the community planned to provide support > > out-of-the-box. However, that's not a tricky task thanks to Ignite's binary > > protocol that lets enable any programming language support easily. > > > > - > > Denis > > > > > > On Fri, Apr 10, 2020 at 8:43 AM smeadows-abb > > wrote: > > > > > First thanks for your quick response. > > > > > > I looked at https://github.com/amsokol/ignite-go-client and its NOT > > > completed and nothing has been for last 16 months. Initial test with > > > package > > > failed, so trying to determine your project roadmap with regards to > > Golang > > > and maybe Rust support. > > > > > > I'm NOT sure of 'AFAIK' ? > > > > > > We may need to implement your Restful API to provide support for Golang > > and > > > Rust, provided it's complete? > > > > > > Thanks, > > > Steve > > > > > > > > > > > > > > > > > > -- > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > -- > > Best regards, > Alexei Scherbakov
[jira] [Created] (IGNITE-12899) Calcite integration. Distribution multitrait
Igor Seliverstov created IGNITE-12899: - Summary: Calcite integration. Distribution multitrait Key: IGNITE-12899 URL: https://issues.apache.org/jira/browse/IGNITE-12899 Project: Ignite Issue Type: Task Reporter: Igor Seliverstov Currently Ignite nodes have single distribution value which isn't true in several cases like: # a partitioned table with a key alias should have 2 distribution traits: by _key and by key alias (id for example) - without distribution by _key column almost impossible to implement IGNITE-12692 # After inner / cross join the result should have several hash distributions (by left and by right keys) - it may be important on join transpose optimization -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Remove "This operating system has been tested less rigorously" diagnostic
Thumbs up! Best regards, Ivan Pavlukhin пн, 13 апр. 2020 г. в 15:08, Pavel Tupitsyn : > > Since there are no objections, I'll go ahead with removal. > > Thanks > > On Wed, Apr 8, 2020 at 10:35 PM Pavel Tupitsyn wrote: > > > Igniters, > > > > Let's remove "This operating system has been tested less rigorously" > > diagnostic [1] [2]. > > It does not make sense: > > * All Linux and macOS versions are considered the same > > * Windows versions are differentiated > > * Windows 10 and all Windows Servers are considered badly tested > > > > None of that is correct. We barely test on macOS. We don't test all the > > different Linux distros, old kernels, and so on. > > > > It is hardly possible to make this diagnostic useful. Let's remove it. > > Any objections? > > > > [1] > > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/GridDiagnostic.java#L94 > > [2] > > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/util/IgniteUtils.java#L6864 > >
