Re: New Monitoring System: items to complete before GA

[Denis Magda]

Nikolay, thanks for joining the discussion.

Please give me a few days to respond and create tickets. A bit hectic week.

-
Denis


On Sat, Apr 18, 2020 at 6:10 AM Nikolay Izhikov  wrote:

> > 1. Add ability to enable\disable subset of the metrics for collection:
> > https://issues.apache.org/jira/browse/IGNITE-11927
> > Am I right that the task will let us perform a desired configuration
> MetricExporterSPI and its specific implementation?
>
> This improvement should allow Ignite users to disable or enable
> `MetricsRegistry` globally on the node.
> When some `MetricRegistry` disabled then registry:
> * Do not consume resources(CPU, RAM) to keep metrics values up to
> date(don’t perform increment, decrement operations,  etc)
> * Can’t be viewed in any exporter.
>
> > Frankly, I couldn't figure out how to do that with
> "setExportFilter(Predicate filter);"
>
> With filter user can exclude arbitrary registry from arbitrary exporter.
> Example of filtering out all cache metrics from JMX:
>
> ```
> JmxMetricExporterSpi jmxSpi = new JmxMetricExporterSpi();
> jmxSpi.setExportFilter(mreg -> !mreg.name().startsWith(«cache»));
>
> cfg.setMetricExporterSpi(jmxSpi);
> ```
>
> > 2. Extension of IgniteSpiContext in order to allow Ignite developers to
> implement their own exporters
>
> Please, clarify, your idea.
> Exporters can be implemented without usage of the internal API.
> You can look into JmxMetricExporterSpi as an example.
>
> > 3. Adding/extending/removing public metrics related to the API facade.
>
> I think we can start with a discussion and finishing Java API to read
> metrics.
> It implemented in my PR [1].
>
> > 4. After registering an exporter with a node only a subset of metrics
> gets updated. This is extremely confusing. Any way we can remove this
> ambiguity?
>
> Yes, it a bad thing.
> And it’s known restriction of the current implementation that was
> introduced to preserve backward compatibility.
> It seems we can remove this restriction after  IGNITE-11927 would be
> implemented.
>
> > 5. The system view that should track compute tasks (views.tasks) is
> always empty
>
> TASKS reflects the running task that was started on the local node.
> You can find an example of using this view in tests [2]
> It seems we should add examples for the views.
>
> The view to observe compute JOBS(parts of the task that are executed on the
> remote node) was added by me in [3]
> It’s not a part of 2.8.0 release but should be available in 2.8.1
>
> > 6. Ignite MBeans are listed under "org.apache.{node_id}" but should be
> placed under "org.apache.ignite.{node_id}".
>
> Please, create a ticket and I will try to fix it.
>
> > we can also add descriptions to MXBeans explaining what a metric is used
> for.
>
> Great idea! Let’s do it, for sure!
> Please, create a ticket for it
>
> > 8. Failed to use LogExporterSpi:
>
> On my plate, I will fix it.
>
> [1] https://github.com/apache/ignite/pull/7283
> [2]
>
> https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/metric/SystemViewSelfTest.java#L236
> [3] https://issues.apache.org/jira/browse/IGNITE-12745
>
>
> ср, 8 апр. 2020 г. в 23:10, Denis Magda :
>
> > Igniters,
> >
> > There was a private discussion between Nikolay Izhikov, Andrey Gura, Alex
> > Goncharuk and I some time ago where we attempted to outline a list of
> tasks
> > to complete before announcing the feature in the GA state (removing the
> > experimental flag from the new APIs and deprecating the legacy ones).
> > Folks, with your permission, let me share the list first and next we can
> > talk out each item in detail separately:
> >
> > 1. Add ability to enable\disable subset of the metrics for collection:
> > https://issues.apache.org/jira/browse/IGNITE-11927
> > Am I right that the task will let us perform a desired configuration
> > MetricExporterSPI and its specific implementation? Frankly, I couldn't
> > figure out how to do that
> > with "setExportFilter(Predicate filter);" method
> --
> > probably, that's just the matter of documentation.
> >
> > 2. Extension of IgniteSpiContext in order to allow Ignite developers to
> > implement their own exporters (with no usage of the internal API).
> >
> > 3. Adding/extending/removing public metrics related to the API
> > facade. Andrey, please elaborate on this item if needed.
> >
> > Also, I had a chance to experience the API in wearing a hat of an
> > application developer while preparing for the Ignite 2.8 webinar. Please
> > take into account my feedback below. Overall, I was impressed with the
> way
> > the new system is designed and operates. All I had to do is just to
> > register exporters and connect my favorite tool to the cluster. Kudos to
> > Nikolay, Andrey and everyone else involved. So, that's my list and I
> ready
> > to create JIRA tickets whenever is relevant:
> >
> > 4. After registering an exporter with a node only a subset of metrics
> gets
> > updated. For instance, "GetTime_X" set of metrics 

Re: [DISCUSSION] Major changes in Ignite in 2020

[Denis Magda]

Igniters,

Here is a draft of our very first roadmap. I decided to make it damp simple
but descriptive:
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Roadmap

What we need to do next is to:

   - Fill in the "Readiness Estimated Time" column with your best guess of
   when an improvement is to be ready for a release.
   - Add references to JIRAs or IEPs to the first column.
   - Add the names of those contributors who will be cooperating with you
   during development. Goes to the "Contributors" column.

Once this step is complete, we'll see how many features converge around the
same date/months and we'll plan through 2.9, 2.10, etc. releases
accordingly.

Please don't put this aside for a while, let's move on quicker. If the
roadmap misses any contributions that you are going to add, then edit the
wiki page.

-
Denis


On Wed, Apr 15, 2020 at 3:35 AM Nikita Amelchev 
wrote:

> Hello, Igniters.
>
> I am going to contribute a new feature - profiling tool and
> performance report. This is part of IEP-35. [1]
>
> The tool will be able to collect performance statistics and create a
> human-readable report. It will help to analyze workload and to tune
> configuration and applications.
>
> Example of report [2, 3, 4].
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool
> [2]
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647581/p1.png
> [3]
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647582/p2.png
> [4]
> https://cwiki.apache.org/confluence/display/IGNITE/Cluster+performance+profiling+tool?preview=/145723859/148647583/p3.png
>
> сб, 11 апр. 2020 г. в 13:54, Alexei Scherbakov <
> alexey.scherbak...@gmail.com>:
> >
> > Folks,
> >
> > I keep working on tasks related to data consistency.
> >
> > This includes:
> >
> > Lost partitions handling overhaul (almost done) and tombstones support
> > aforementioned by Ivan Rakov
> > Atomic protocol overhaul (see [1])
> >
> > The ultimate goal of the year is to prepare Ignite for passing Jepsen
> tests.
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-12+Make+ATOMIC+Caches+Consistent+Again
> >
> >
> > пт, 10 апр. 2020 г. в 18:49, Denis Magda :
> >
> > > Steven,
> > >
> > > Please start a dedicated discussion for the Golang support. At the
> moment,
> > > I'm not aware if anybody from the community planned to provide support
> > > out-of-the-box. However, that's not a tricky task thanks to Ignite's
> binary
> > > protocol that lets enable any programming language support easily.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Fri, Apr 10, 2020 at 8:43 AM smeadows-abb <
> steven.mead...@us.abb.com>
> > > wrote:
> > >
> > > > First thanks for your quick response.
> > > >
> > > > I looked at  https://github.com/amsokol/ignite-go-client and its NOT
> > > > completed and nothing has been for last 16 months. Initial test with
> > > > package
> > > > failed, so trying to determine your project roadmap with regards to
> > > Golang
> > > > and maybe Rust support.
> > > >
> > > > I'm NOT sure of 'AFAIK' ?
> > > >
> > > > We may need to implement your Restful API to provide support for
> Golang
> > > and
> > > > Rust, provided it's complete?
> > > >
> > > > Thanks,
> > > >   Steve
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > > >
> > >
> >
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
>
>
>
> --
> Best wishes,
> Amelchev Nikita
>

Re: contributors permissions

[Pavel Tupitsyn]

Hello Johnny,

Added you to contributors group in JIRA.
Welcome to Ignite community!

Pavel

On Wed, Apr 22, 2020 at 7:58 PM Johnny Galatikitis <
johnny.galatiki...@gmail.com> wrote:

> Hello,
>
> I am working on IGNITE-12905 and forgot to ask for contributors
> permissions.
>
> regards, J.
>

contributors permissions

[Johnny Galatikitis]

Hello,

I am working on IGNITE-12905 and forgot to ask for contributors
permissions.

regards, J.

Re: Moving python, php and node.js in separate repos and release cycles

[Pavel Tupitsyn]

Ok, thanks for the clarification.
I retract my "-1": no strong objections.

On Wed, Apr 22, 2020 at 3:15 PM Dmitriy Pavlov  wrote:

> Hi Folks,
>
> There is nothing wrong with releasing system by it's modules, one-by-one.
> Also it is perfectly OK to have each module in a separate git repo. Each of
> these module release-candidate is voted separately. At the foundation
> level, release cycle of the project/modules is not defined, it is up to the
> community to decide. It is just general common sense: often  releases are
> preferred.
>
> The only one rule, community should vote on release with 3 +1 votes from
> the PMC members. And there can be the only issue if module release would
> stall without required number of binding votes.
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 22 апр. 2020 г. в 15:03, Igor Sapego :
>
> > Pavel,
> >
> > 1. We can conduct separate votes for every client, do you see any issues
> > here?
> > 2. This is true, but we have backward compatibility in our protocol, so
> > everything
> > should work just fine.
> >
> > Best Regards,
> > Igor
> >
> >
> > On Tue, Apr 21, 2020 at 9:22 PM Pavel Tupitsyn 
> > wrote:
> >
> > > -1
> > >
> > > - Ignite is a single Apache project, it follows Apache release
> > guidelines,
> > > with voting and so on. Not sure how are we going to follow that with a
> > > separate repo.
> > > - Thin client features are often tied to server-side changes
> > >
> > > > What about dotnet and cpp thin clients?
> > > Those reuse some code with thick counterparts - same way as Java thin
> > does.
> > >
> > >
> > > On Tue, Apr 21, 2020 at 4:22 PM Nikolay Izhikov 
> > > wrote:
> > >
> > > > What about dotnet and cpp thin clients?
> > > >
> > > > > 21 апр. 2020 г., в 16:19, Dmitriy Pavlov 
> > > > написал(а):
> > > > >
> > > > > +1 since
> > > > > - Simpler release may allow us to release more often
> > > > > - Often releases - users will get updates faster, more chances to
> > grow
> > > > and
> > > > > keep our user base
> > > > > - Faster updates and easy to get next update may have positive
> effect
> > > on
> > > > > community growth. Since newcomer may want to fix a bug and later
> use
> > > > result
> > > > > in his/her production environment.
> > > > >
> > > > > вт, 21 апр. 2020 г. в 13:27, Alexey Zinoviev <
> zaleslaw@gmail.com
> > >:
> > > > >
> > > > >> Agree with these non-JVM languages.
> > > > >> Especially for Python:)
> > > > >>
> > > > >> вт, 21 апр. 2020 г. в 12:58, Igor Sapego :
> > > > >>
> > > > >>> Guys,
> > > > >>>
> > > > >>> It was discussed on the dev list a few times that it would be a
> > good
> > > > >>> idea to move Python, Node.js and PHP thin clients to separate
> repos
> > > > >>> and separate release cycles.
> > > > >>>
> > > > >>> In short there are several arguments for that:
> > > > >>>
> > > > >>> 1. There are no dependencies on the core functionality so there
> is
> > > > simply
> > > > >>> no need for them to be in the main repo.
> > > > >>>
> > > > >>> 2. Those thin clients often do not get updates from release to
> > > release,
> > > > >> but
> > > > >>> still
> > > > >>> we "release" them, as they are a part of the main release.
> > > > >>>
> > > > >>> 3. Moving them to a separate release cycle will allow us to
> release
> > > > some
> > > > >>> hot
> > > > >>> fixes for those clients faster and easier.
> > > > >>>
> > > > >>> 4. Composer needs a PHP packet that is released to be in a
> separate
> > > > >>> repository.
> > > > >>>
> > > > >>> Thoughts?
> > > > >>>
> > > > >>> Best Regards,
> > > > >>> Igor
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>

[jira] [Created] (IGNITE-12934) Change test start|stop log format for correct TC and build.log visibility.

[Ilya Kasnacheev (Jira)]

Ilya Kasnacheev created IGNITE-12934:


 Summary: Change test start|stop log format for correct TC and 
build.log visibility.
 Key: IGNITE-12934
 URL: https://issues.apache.org/jira/browse/IGNITE-12934
 Project: Ignite
  Issue Type: Improvement
  Components: build
Reporter: Ilya Kasnacheev
Assignee: Ilya Kasnacheev


For correct TC and builds log visibility need to repeat log format, from : 
">>> Stopping test: "

as 
"##teamcity[testFinished name='"

additional info:
https://www.jetbrains.com/help/teamcity/build-script-interaction-with-teamcity.html#BuildScriptInteractionwithTeamCity-ReportingTests

Also, make “Starting test”, “Stopping test” messages visible in Quiet mode!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12933) Node failed after put incorrect key class for indexed type to transactional cache

[Aleksey Plekhanov (Jira)]

Aleksey Plekhanov created IGNITE-12933:
--

 Summary: Node failed after put incorrect key class for indexed 
type to transactional cache
 Key: IGNITE-12933
 URL: https://issues.apache.org/jira/browse/IGNITE-12933
 Project: Ignite
  Issue Type: Bug
Reporter: Aleksey Plekhanov


Reproducer:
{code:java}
public class IndexedTypesTest extends GridCommonAbstractTest {
private boolean failed;

@Override protected IgniteConfiguration getConfiguration(String 
igniteInstanceName) throws Exception {
return super.getConfiguration(igniteInstanceName)
.setFailureHandler((ignite, ctx) -> failed = true)
.setCacheConfiguration(new CacheConfiguration<>(DEFAULT_CACHE_NAME)
.setAtomicityMode(TRANSACTIONAL)
.setIndexedTypes(String.class, String.class));
}

@Test
public void testPutIndexedType() throws Exception {
Ignite ignite = startGrids(2);

for (int i = 0; i < 10; i++) {
try {
ignite.cache(DEFAULT_CACHE_NAME).put(i, "val" + i);
}
catch (Exception ignore) {
}
}

assertFalse(failed);
}
}
{code}
Node failed with exception:
{noformat}
[2020-04-22 
17:05:34,524][ERROR][sys-stripe-11-#76%cache.IndexedTypesTest1%][IgniteTestResources]
 Critical system error detected. Will be handled accordingly to configured 
handler 
[hnd=o.a.i.i.processors.cache.IndexedTypesTest$$Lambda$115/0x00080024d040@147237db,
 failureCtx=FailureContext [type=CRITICAL_ERROR, err=class 
o.a.i.i.transactions.IgniteTxHeuristicCheckedException: Committing a 
transaction has produced runtime exception]]
class 
org.apache.ignite.internal.transactions.IgniteTxHeuristicCheckedException: 
Committing a transaction has produced runtime exception
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.heuristicException(IgniteTxAdapter.java:800)
at 
org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxRemoteAdapter.commitIfLocked(GridDistributedTxRemoteAdapter.java:838)
at 
org.apache.ignite.internal.processors.cache.distributed.GridDistributedTxRemoteAdapter.commitRemoteTx(GridDistributedTxRemoteAdapter.java:893)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:1502)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processDhtTxPrepareRequest(IgniteTxHandler.java:1233)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:229)
at 
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:227)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1142)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
at 
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1847)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1472)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1367)
at 
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:565)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: class org.apache.ignite.IgniteCheckedException: Failed to update 
index, incorrect key class [expCls=java.lang.String, 
actualCls=java.lang.Integer]
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.typeByValue(GridQueryProcessor.java:2223)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:2092)
at 
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:410)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:2627)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1713)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1688)
at 

Re: [jira] [Created] (IGNITE-12911) B+Tree Corrupted exception when using a key extracted from a value object --- and SQL enabled.

[Dmitriy Pavlov]

Folks,  could you please take a look at this issue?

It seems quite serious if we can potentially get B+Tree corrupted.

Sincerely,
Dmitriy Pavlov

пт, 17 апр. 2020 г. в 19:25, Alexander Korenshteyn (Jira) :

> Alexander Korenshteyn created IGNITE-12911:
> --
>
>  Summary: B+Tree Corrupted exception when using a key
> extracted from a value object --- and SQL enabled.
>  Key: IGNITE-12911
>  URL: https://issues.apache.org/jira/browse/IGNITE-12911
>  Project: Ignite
>   Issue Type: Bug
>   Components: cache, sql
> Reporter: Alexander Korenshteyn
>  Attachments: Employee.java, EmployeeId.java, ServerNode.java
>
> If a key is part of the value like so:
> class key \{ ... }
>
> class value {
> private Key key
> getKey()
> }
>
> cache = ignite.cache()*.withKeepBinary()*
>
> the following scenario yields a B+ Tree exception:
>
> 1. *Enable SQL via annotations or QueryEntities.*
>
> key1= new Key()
> value1 = new Value(key1)
>
> cache.put(key1, value1)
>
> BinaryObject val2 = cache.get(key1)
> BinaryObject key2 = val2.field("key")
>
> *cache.put(key2, emp1); // CRASH!!! CorruptedTreeException: B+Tree
> iscorrupted ...*
>
> user list:
> [
> http://apache-ignite-users.70518.x6.nabble.com/Ignite-crashes-with-CorruptedTreeException-quot-B-Tree-is-corrupted-quot-on-a-composite-BinaryObjecto-tc32032.html
> ]
>
> *Reproducer – attached*
>
>
>
> Exception:
>
>  [2020-04-17 11:24:33,475][ERROR][main][root] Critical system error
> detected. Will be handled accordingly to configured handler
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0,
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]],
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
> o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is
> corrupted [pages(groupId, pageId)=[IgniteBiTuple [val1=1258113742,
> val2=844420635164673]], cacheId=1976096430, cacheName=EMPLOYEE,
> indexName=_key_PK, msg=Runtime failure on row: Row@2233cac0[ key:
> model.EmployeeId [idHash=1744523301, hash=674030145,
> employeeNumber=65348765, departmentNumber=123], val: model.Employee
> [idHash=382762227, hash=1627745429, firstName=John, lastName=Smith,
> id=model.EmployeeId [idHash=2021540695, hash=674030145,
> employeeNumber=65348765, departmentNumber=123]] ][ John, Smith 
>
> class
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple
> [val1=1258113742, val2=844420635164673]], cacheId=1976096430,
> cacheName=EMPLOYEE, indexName=_key_PK, msg=Runtime failure on row:
> Row@2233cac0[ key: model.EmployeeId [idHash=1744523301, hash=674030145,
> employeeNumber=65348765, departmentNumber=123], val: model.Employee
> [idHash=382762227, hash=1627745429, firstName=John, lastName=Smith,
> id=model.EmployeeId [idHash=2021540695, hash=674030145,
> employeeNumber=65348765, departmentNumber=123]] ][ John, Smith ]]
>
>  at
> org.apache.ignite.internal.processors.query.h2.database.H2Tree.corruptedTreeException(H2Tree.java:836)
>
>  at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doPut(BPlusTree.java:2447)
>
>  at
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putx(BPlusTree.java:2394)
>
>  at
> org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:437)
>
>  at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.update(GridH2Table.java:756)
>
>  at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.store(IgniteH2Indexing.java:363)
>
>  at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:2016)
>
>  at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:401)
>
>  at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:2555)
>
>  at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1664)
>
>  at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1639)
>
>  at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:436)
>
>  at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:2311)
>
>  at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2593)
>
>  at
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:2053)
>
>  at
> 

Re: Moving python, php and node.js in separate repos and release cycles

[Dmitriy Pavlov]

Hi Folks,

There is nothing wrong with releasing system by it's modules, one-by-one.
Also it is perfectly OK to have each module in a separate git repo. Each of
these module release-candidate is voted separately. At the foundation
level, release cycle of the project/modules is not defined, it is up to the
community to decide. It is just general common sense: often  releases are
preferred.

The only one rule, community should vote on release with 3 +1 votes from
the PMC members. And there can be the only issue if module release would
stall without required number of binding votes.

Sincerely,
Dmitriy Pavlov

ср, 22 апр. 2020 г. в 15:03, Igor Sapego :

> Pavel,
>
> 1. We can conduct separate votes for every client, do you see any issues
> here?
> 2. This is true, but we have backward compatibility in our protocol, so
> everything
> should work just fine.
>
> Best Regards,
> Igor
>
>
> On Tue, Apr 21, 2020 at 9:22 PM Pavel Tupitsyn 
> wrote:
>
> > -1
> >
> > - Ignite is a single Apache project, it follows Apache release
> guidelines,
> > with voting and so on. Not sure how are we going to follow that with a
> > separate repo.
> > - Thin client features are often tied to server-side changes
> >
> > > What about dotnet and cpp thin clients?
> > Those reuse some code with thick counterparts - same way as Java thin
> does.
> >
> >
> > On Tue, Apr 21, 2020 at 4:22 PM Nikolay Izhikov 
> > wrote:
> >
> > > What about dotnet and cpp thin clients?
> > >
> > > > 21 апр. 2020 г., в 16:19, Dmitriy Pavlov 
> > > написал(а):
> > > >
> > > > +1 since
> > > > - Simpler release may allow us to release more often
> > > > - Often releases - users will get updates faster, more chances to
> grow
> > > and
> > > > keep our user base
> > > > - Faster updates and easy to get next update may have positive effect
> > on
> > > > community growth. Since newcomer may want to fix a bug and later use
> > > result
> > > > in his/her production environment.
> > > >
> > > > вт, 21 апр. 2020 г. в 13:27, Alexey Zinoviev  >:
> > > >
> > > >> Agree with these non-JVM languages.
> > > >> Especially for Python:)
> > > >>
> > > >> вт, 21 апр. 2020 г. в 12:58, Igor Sapego :
> > > >>
> > > >>> Guys,
> > > >>>
> > > >>> It was discussed on the dev list a few times that it would be a
> good
> > > >>> idea to move Python, Node.js and PHP thin clients to separate repos
> > > >>> and separate release cycles.
> > > >>>
> > > >>> In short there are several arguments for that:
> > > >>>
> > > >>> 1. There are no dependencies on the core functionality so there is
> > > simply
> > > >>> no need for them to be in the main repo.
> > > >>>
> > > >>> 2. Those thin clients often do not get updates from release to
> > release,
> > > >> but
> > > >>> still
> > > >>> we "release" them, as they are a part of the main release.
> > > >>>
> > > >>> 3. Moving them to a separate release cycle will allow us to release
> > > some
> > > >>> hot
> > > >>> fixes for those clients faster and easier.
> > > >>>
> > > >>> 4. Composer needs a PHP packet that is released to be in a separate
> > > >>> repository.
> > > >>>
> > > >>> Thoughts?
> > > >>>
> > > >>> Best Regards,
> > > >>> Igor
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: Moving python, php and node.js in separate repos and release cycles

[Igor Sapego]

Pavel,

1. We can conduct separate votes for every client, do you see any issues
here?
2. This is true, but we have backward compatibility in our protocol, so
everything
should work just fine.

Best Regards,
Igor


On Tue, Apr 21, 2020 at 9:22 PM Pavel Tupitsyn  wrote:

> -1
>
> - Ignite is a single Apache project, it follows Apache release guidelines,
> with voting and so on. Not sure how are we going to follow that with a
> separate repo.
> - Thin client features are often tied to server-side changes
>
> > What about dotnet and cpp thin clients?
> Those reuse some code with thick counterparts - same way as Java thin does.
>
>
> On Tue, Apr 21, 2020 at 4:22 PM Nikolay Izhikov 
> wrote:
>
> > What about dotnet and cpp thin clients?
> >
> > > 21 апр. 2020 г., в 16:19, Dmitriy Pavlov 
> > написал(а):
> > >
> > > +1 since
> > > - Simpler release may allow us to release more often
> > > - Often releases - users will get updates faster, more chances to grow
> > and
> > > keep our user base
> > > - Faster updates and easy to get next update may have positive effect
> on
> > > community growth. Since newcomer may want to fix a bug and later use
> > result
> > > in his/her production environment.
> > >
> > > вт, 21 апр. 2020 г. в 13:27, Alexey Zinoviev :
> > >
> > >> Agree with these non-JVM languages.
> > >> Especially for Python:)
> > >>
> > >> вт, 21 апр. 2020 г. в 12:58, Igor Sapego :
> > >>
> > >>> Guys,
> > >>>
> > >>> It was discussed on the dev list a few times that it would be a good
> > >>> idea to move Python, Node.js and PHP thin clients to separate repos
> > >>> and separate release cycles.
> > >>>
> > >>> In short there are several arguments for that:
> > >>>
> > >>> 1. There are no dependencies on the core functionality so there is
> > simply
> > >>> no need for them to be in the main repo.
> > >>>
> > >>> 2. Those thin clients often do not get updates from release to
> release,
> > >> but
> > >>> still
> > >>> we "release" them, as they are a part of the main release.
> > >>>
> > >>> 3. Moving them to a separate release cycle will allow us to release
> > some
> > >>> hot
> > >>> fixes for those clients faster and easier.
> > >>>
> > >>> 4. Composer needs a PHP packet that is released to be in a separate
> > >>> repository.
> > >>>
> > >>> Thoughts?
> > >>>
> > >>> Best Regards,
> > >>> Igor
> > >>>
> > >>
> >
> >
>

Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

[Nikolay Izhikov]

Hello, Ivan.

I think we can include this improvements.
Please, go ahead.

> 22 апр. 2020 г., в 10:21, Ivan Bessonov  написал(а):
> 
> Hi Igniters,
> 
> I'm continuing with IGNITE-12756 PR creation. It turned out that we need 3
> more cherry-picks
> to avoid massive code changes. Tests started failing after my initial
> attempt to create that PR.
> 
> So, in total I have PR with 6 commits in it. Some of them fix components
> and tests for them,
> others are required for new tests to compile and run without changes in
> 2.8.1 branch.
> RunAll is in progress now and I'll reply with news if I have any.
> 
> Andrey Gura, Nikolay Izhikov, are you OK with 3 more commits in the scope?
> If you don't,
> then I won't be able to port IGNITE-12756 to 2.8.1 properly by myself, I
> guess it would be
> easier to reimplement it from the scratch.
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-12735
> [2] https://issues.apache.org/jira/browse/IGNITE-12568
> [3] https://issues.apache.org/jira/browse/IGNITE-12756
> 
> [4] https://issues.apache.org/jira/browse/IGNITE-12285
> [5] https://issues.apache.org/jira/browse/IGNITE-12668
> [6] https://issues.apache.org/jira/browse/IGNITE-12682
> [7] https://github.com/apache/ignite/pull/7708
> 
> 
> вт, 21 апр. 2020 г. в 19:36, Pavel Pereslegin :
> 
>> Hello, Nikolay.
>> 
>> This has been fixed by increasing the test timeout in IGNITE-12683 [1][2].
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-12683
>> [2]
>> https://github.com/apache/ignite/commit/bf394a77e1de6432e493eb2818243a82b577f11e
>> 
>> вт, 21 апр. 2020 г. в 18:28, Nikolay Izhikov :
>>> 
>>> Hello, Igniters.
>>> 
>>> While cherry-picking to 2.8.1 I found flaky test regarding BPlusTree
>> [1], [2]
>>> 
>>> All failures of this tests relates to PR’s based on 2.8.
>>> It seems we hav fix in master but doesn’t have it in 2.8 and 2.8.1
>>> 
>>> Can someone suggest, what commit fixes this tests?
>>> 
>>> [1]
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8==testDetails=-7895536196794411367=TEST_STATUS_DESC_IgniteTests24Java8=__all_branches__=50
>>> 
>>> [2]
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8==testDetails=-3485591522651012009=TEST_STATUS_DESC_IgniteTests24Java8=__all_branches__=50
>>> 
>>> 
 21 апр. 2020 г., в 17:49, Ivan Bessonov 
>> написал(а):
 
 Sure, here's PR with 3 cherry-picked commits that I mentioned:
 https://github.com/apache/ignite/pull/7708
 
 вт, 21 апр. 2020 г. в 17:17, Nikolay Izhikov :
 
> OK then, let’s include it in the 2.8.1
> 
> Ivan, can you, please, prepare PR in the ignite-2.8.1 branch that
>> contain
> cherry-pick for all required commits?
> 
>> 21 апр. 2020 г., в 17:01, Andrey Gura  написал(а):
>> 
>> Hi
>> 
>>> IGNITE-12735 - Metric exporter implementation could lead to
> NullPointerException from gauge which invoke communication
>>> IGNITE-12568 - MessageFactory implementations refactoring
>> 
>>> Personally, I’m against any refactoring improvements in bug fix
>> release.
>>> So, I propose to exclude IGNITE-12756 from 2.8.1
>> 
>>> Andrey, what do you think as a committer of this improvements?
>> 
>> Mainly IGNITE-12756 brings some improvements related with TCP
>> communication metrics (performance, memory footprint,
>> IgniteSpiContext
>> improved in order to provide ability to implement metrics related
>> SPI's without using internal API's, code improvements)
>> 
>> But! It also fixes potential NPE's which can be thrown on node start.
>> So it would be great to include this fix to 2.8.1 release.
>> 
>> On Tue, Apr 21, 2020 at 11:12 AM Nikolay Izhikov <
>> nizhi...@apache.org>
> wrote:
>>> 
 I've cherry-picked IGNITE-12734 to 2.8.1 branch.
>>> 
>>> 
>>> Thanks!
>>> 
 considering commit "683f22e64f IGNITE-12756 TcpCommunication SPI
> metrics
> improvement" - it depends
> on https://issues.apache.org/jira/browse/IGNITE-12735 and
> https://issues.apache.org/jira/browse/IGNITE-12568,
> both were targeted to 2.9, but this has to be changed probably.
>>> 
>>> IGNITE-12735 - Metric exporter implementation could lead to
> NullPointerException from gauge which invoke communication
>>> IGNITE-12568 - MessageFactory implementations refactoring
>>> 
>>> Ivan,
>>> Personally, I’m against any refactoring improvements in bug fix
>> release.
>>> So, I propose to exclude IGNITE-12756 from 2.8.1
>>> 
>>> Andrey, what do you think as a committer of this improvements?
>>> 
>>> 
 21 апр. 2020 г., в 10:44, Alex Plehanov 
> написал(а):
 
 Nikolay,
 
 I've cherry-picked IGNITE-12734 to 2.8.1 branch.
 
 вт, 21 апр. 2020 г. в 10:02, Ivan Bessonov >> :
 
> Hello, 

[jira] [Created] (IGNITE-12932) Thin client cluster discovery

[Pavel Tupitsyn (Jira)]

Pavel Tupitsyn created IGNITE-12932:
---

 Summary: Thin client cluster discovery
 Key: IGNITE-12932
 URL: https://issues.apache.org/jira/browse/IGNITE-12932
 Project: Ignite
  Issue Type: New Feature
  Components: thin client
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.9


Thin clients should be able to discover all server nodes automatically when 
connected to any of them, and maintain an up to date list of servers at all 
times.

See linked IEP-44 for more details.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (IGNITE-12931) General error: "java.lang.ArrayIndexOutOfBound

[icode (Jira)]

icode created IGNITE-12931:
--

 Summary: General error: "java.lang.ArrayIndexOutOfBound
 Key: IGNITE-12931
 URL: https://issues.apache.org/jira/browse/IGNITE-12931
 Project: Ignite
  Issue Type: Bug
Reporter: icode
 Attachments: console.log

Hi guys!

I'm use ignite v2.8.0. I get the some error, please help me. Thanks.

I find like this error in mailling list, but it different.

Please see attachment.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Why OPTIMIZE_REUSE_RESULT is set to 0 explicitly in o.a.i.i.p.query.h2.ConnectionManager#DEFAULT_DB_OPTIONS

[Ivan Daschinsky]

Thank you for your response, Konstantin

I think we cannot just turn this optimization
>
Of course not, that is the reason why I started this thread.

org.apache.ignite.internal.processors.query.h2.opt.GridH2Table don't change
> it neither on remove or update


 Yes, but we can increment this counter and find possible workarounds.

 It should be benchmarked first.
>
Absolutelly agree.

I started this thread to discuss possible caveats and traps and decide
whether to implement this optimization or not.
If there is too many traps or moreover this is impossible or SQL
maintainers are against this, then it's probably not worth to do it.



ср, 22 апр. 2020 г. в 09:52, Konstantin Orlov :

> Hello, Ivan
>
> I think we cannot just turn this optimization on because a result
> invalidation counts on org.h2.engine.Database#modificationDataId (see
> org.h2.command.dml.Query#sameResultAsLast(Session, Value[], Value[],
> long)), but org.apache.ignite.internal.processors.query.h2.opt.GridH2Table
> don't change it neither on remove or update. And incrementing
> org.h2.engine.Database#modificationDataId on each data update may lead to
> some performance issues. It should be benchmarked first.
>
> --
> Regards,
> Konstantin Orlov
>
>
> > On 21 Apr 2020, at 14:51, Ivan Daschinskiy  wrote:
> >
> > Hello folks.
> >
> > Recently I trying to investigate why when the query, i.e. «SELECT * FROM
> T1 WHERE ID IN (SELECT T1_ID FROM T2), is executed locally,
> > subquery evaluated, even if result is the same. I noticed, that in
> mentioned in subj (.ConnectionManager#DEFAULT_DB_OPTIONS),
> > parameter OPTIMIZE_REUSE_RESULT explicitly set to 0. This value disable
> internal H2 optimization (see details here
> org.h2.command.dml.Query#query(int, org.h2.result.ResultTarget) and lead
> the mentioned above query to be ineffective.
> >
> > I cannot understand why this decision was made. Also I don’t find any
> evidence that setting OPTIMIZE_REUSE_RESULT to 1 could break something.
> Unfortunatelly, it is impossible to change this behavior per Session
> without forking H2, because this flag is set in org.h2.engine.Database.
> >
> > Let’s discuss possible caveats in enabling this optimization by default
> in DEFAULT_DB_OPTIONS.
> >
>
>

-- 
Sincerely yours, Ivan Daschinskiy

Re: Apache Ignite 2.8.1 RELEASE [Time, Scope, Manager]

[Ivan Bessonov]

Hi Igniters,

I'm continuing with IGNITE-12756 PR creation. It turned out that we need 3
more cherry-picks
to avoid massive code changes. Tests started failing after my initial
attempt to create that PR.

So, in total I have PR with 6 commits in it. Some of them fix components
and tests for them,
others are required for new tests to compile and run without changes in
2.8.1 branch.
RunAll is in progress now and I'll reply with news if I have any.

Andrey Gura, Nikolay Izhikov, are you OK with 3 more commits in the scope?
If you don't,
then I won't be able to port IGNITE-12756 to 2.8.1 properly by myself, I
guess it would be
easier to reimplement it from the scratch.

[1] https://issues.apache.org/jira/browse/IGNITE-12735
[2] https://issues.apache.org/jira/browse/IGNITE-12568
[3] https://issues.apache.org/jira/browse/IGNITE-12756

[4] https://issues.apache.org/jira/browse/IGNITE-12285
[5] https://issues.apache.org/jira/browse/IGNITE-12668
[6] https://issues.apache.org/jira/browse/IGNITE-12682
[7] https://github.com/apache/ignite/pull/7708


вт, 21 апр. 2020 г. в 19:36, Pavel Pereslegin :

> Hello, Nikolay.
>
> This has been fixed by increasing the test timeout in IGNITE-12683 [1][2].
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12683
> [2]
> https://github.com/apache/ignite/commit/bf394a77e1de6432e493eb2818243a82b577f11e
>
> вт, 21 апр. 2020 г. в 18:28, Nikolay Izhikov :
> >
> > Hello, Igniters.
> >
> > While cherry-picking to 2.8.1 I found flaky test regarding BPlusTree
> [1], [2]
> >
> > All failures of this tests relates to PR’s based on 2.8.
> > It seems we hav fix in master but doesn’t have it in 2.8 and 2.8.1
> >
> > Can someone suggest, what commit fixes this tests?
> >
> > [1]
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8==testDetails=-7895536196794411367=TEST_STATUS_DESC_IgniteTests24Java8=__all_branches__=50
> >
> > [2]
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8==testDetails=-3485591522651012009=TEST_STATUS_DESC_IgniteTests24Java8=__all_branches__=50
> >
> >
> > > 21 апр. 2020 г., в 17:49, Ivan Bessonov 
> написал(а):
> > >
> > > Sure, here's PR with 3 cherry-picked commits that I mentioned:
> > > https://github.com/apache/ignite/pull/7708
> > >
> > > вт, 21 апр. 2020 г. в 17:17, Nikolay Izhikov :
> > >
> > >> OK then, let’s include it in the 2.8.1
> > >>
> > >> Ivan, can you, please, prepare PR in the ignite-2.8.1 branch that
> contain
> > >> cherry-pick for all required commits?
> > >>
> > >>> 21 апр. 2020 г., в 17:01, Andrey Gura  написал(а):
> > >>>
> > >>> Hi
> > >>>
> >  IGNITE-12735 - Metric exporter implementation could lead to
> > >> NullPointerException from gauge which invoke communication
> >  IGNITE-12568 - MessageFactory implementations refactoring
> > >>>
> >  Personally, I’m against any refactoring improvements in bug fix
> release.
> >  So, I propose to exclude IGNITE-12756 from 2.8.1
> > >>>
> >  Andrey, what do you think as a committer of this improvements?
> > >>>
> > >>> Mainly IGNITE-12756 brings some improvements related with TCP
> > >>> communication metrics (performance, memory footprint,
> IgniteSpiContext
> > >>> improved in order to provide ability to implement metrics related
> > >>> SPI's without using internal API's, code improvements)
> > >>>
> > >>> But! It also fixes potential NPE's which can be thrown on node start.
> > >>> So it would be great to include this fix to 2.8.1 release.
> > >>>
> > >>> On Tue, Apr 21, 2020 at 11:12 AM Nikolay Izhikov <
> nizhi...@apache.org>
> > >> wrote:
> > 
> > > I've cherry-picked IGNITE-12734 to 2.8.1 branch.
> > 
> > 
> >  Thanks!
> > 
> > > considering commit "683f22e64f IGNITE-12756 TcpCommunication SPI
> > >> metrics
> > >> improvement" - it depends
> > >> on https://issues.apache.org/jira/browse/IGNITE-12735 and
> > >> https://issues.apache.org/jira/browse/IGNITE-12568,
> > >> both were targeted to 2.9, but this has to be changed probably.
> > 
> >  IGNITE-12735 - Metric exporter implementation could lead to
> > >> NullPointerException from gauge which invoke communication
> >  IGNITE-12568 - MessageFactory implementations refactoring
> > 
> >  Ivan,
> >  Personally, I’m against any refactoring improvements in bug fix
> release.
> >  So, I propose to exclude IGNITE-12756 from 2.8.1
> > 
> >  Andrey, what do you think as a committer of this improvements?
> > 
> > 
> > > 21 апр. 2020 г., в 10:44, Alex Plehanov 
> > >> написал(а):
> > >
> > > Nikolay,
> > >
> > > I've cherry-picked IGNITE-12734 to 2.8.1 branch.
> > >
> > > вт, 21 апр. 2020 г. в 10:02, Ivan Bessonov  >:
> > >
> > >> Hello, Igniters,
> > >>
> > >> considering commit "683f22e64f IGNITE-12756 TcpCommunication SPI
> > >> metrics
> > >> improvement" - it depends
> > >> on 

Re: Why OPTIMIZE_REUSE_RESULT is set to 0 explicitly in o.a.i.i.p.query.h2.ConnectionManager#DEFAULT_DB_OPTIONS

[Konstantin Orlov]

Hello, Ivan

I think we cannot just turn this optimization on because a result invalidation 
counts on org.h2.engine.Database#modificationDataId (see 
org.h2.command.dml.Query#sameResultAsLast(Session, Value[], Value[], long)), 
but org.apache.ignite.internal.processors.query.h2.opt.GridH2Table don't change 
it neither on remove or update. And incrementing 
org.h2.engine.Database#modificationDataId on each data update may lead to some 
performance issues. It should be benchmarked first.

-- 
Regards,
Konstantin Orlov


> On 21 Apr 2020, at 14:51, Ivan Daschinskiy  wrote:
> 
> Hello folks.
> 
> Recently I trying to investigate why when the query, i.e. «SELECT * FROM T1 
> WHERE ID IN (SELECT T1_ID FROM T2), is executed locally,
> subquery evaluated, even if result is the same. I noticed, that in mentioned 
> in subj (.ConnectionManager#DEFAULT_DB_OPTIONS),
> parameter OPTIMIZE_REUSE_RESULT explicitly set to 0. This value disable 
> internal H2 optimization (see details here 
> org.h2.command.dml.Query#query(int, org.h2.result.ResultTarget) and lead the 
> mentioned above query to be ineffective.
> 
> I cannot understand why this decision was made. Also I don’t find any 
> evidence that setting OPTIMIZE_REUSE_RESULT to 1 could break something. 
> Unfortunatelly, it is impossible to change this behavior per Session without 
> forking H2, because this flag is set in org.h2.engine.Database. 
> 
> Let’s discuss possible caveats in enabling this optimization by default in 
> DEFAULT_DB_OPTIONS.
>