Re: [Discussion] Race on heartbeat update in checkpoint thread

[Ivan Daschinsky]

+1 For current fix. Code is clean and understandable. I suppose that the
current fix is a correct variant to update heartbeatTs.

вт, 20 июл. 2021 г. в 16:13, Alex Plehanov :

> Hello, Ilya
>
> > But anyway, I propose to remove the update of the heartbeat from other
> threads altogether and wrap the call to listeners in a blockingSection.
> I don't quite understand your proposal. Which call to listeners do you
> mean? If we wrap the listener into the blocking section the result will be
> the same.
> Alternatively, I think we can wrap awaitPendingTasksFinished into the
> blocking section, this will also solve the problem, but in this case we can
> never detect lack of progress by async executor threads and checkpointer
> thread hangs due to this reason.
> What's wrong with the current fix? It solves the current problem and I hope
> will protect us from problems like this in the future.
>
> вт, 20 июл. 2021 г. в 15:28, Ilya Kazakov :
>
> > Hi Igniters, hi Alexey.
> >
> > I want to discuss this issue:
> > https://issues.apache.org/jira/browse/IGNITE-15099. I have caught it
> too.
> >
> > I was able to determine where there is a race.
> >
> > The update of the heartbeat happens asynchronously into the listener
> code.
> > But we always wait in the checkpoint thread for all pending async
> > tasks. And this is reasonable.
> >
> > for (CheckpointListener lsnr : dbLsnrs)
> >   lsnr.beforeCheckpointBegin(ctx0);
> >
> > ctx0.awaitPendingTasksFinished();
> >
> > The race was because of inappropriate order of future registration. In
> > CheckpointContextImpl.executor () (inside listeners execution)
> >
> > GridFutureAdapter res = new GridFutureAdapter<>();
> > res.listen(fut -> heartbeatUpdater.updateHeartbeat());
> > asyncRunner.execute(U.wrapIgniteFuture(cmd, res));
> > pendingTaskFuture.add(res);
> >
> > Here we create a task, submit a task to the executor, and only after this
> > do we register the task. Thus we got a situation where checkpointer
> thread
> > was moving on after ctx0.awaitPendingTasksFinished(); and still,
> > the unregistered asyncRunner task was moving on in parallel.
> >
> > But anyway, I propose to remove the update of the heartbeat from other
> > threads altogether and wrap the call to listeners in a blockingSection.
> >
> > As I understand heartbeat was designed just to indicate self-progress by
> a
> > worker. If a worker can not indicate self-progress we should wrap such
> code
> > into blockingSections. In case of listeners, worker can not indicate
> > self-progress, thus let's wrap it into blockingSection.
> >
> > Guys, what do you think about this?
> >
> > -
> > Ilya Kazakov
> >
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: [DISCUSS] Confuse default inspections.

[Ivan Daschinsky]

Hi!

Firstly, lets talk about interfaces.

1. First of all, do we have an automatic inspection for it? AFAIK we don't
2. I am for consistency. At least for production code. Nothing worse is
when someone mixes both approaches.

About a prohibition of curly brackets around one line -- I am strongly
against this rule, it should be removed. There were a few bugs that this
rule caused.

вт, 20 июл. 2021 г. в 09:23, Zhenya Stanilovsky :

>
> Igniters, i understand that this is very long and fundamental story. but
> … still want to rise up this discussion, we have 2 very strange inspections:
> *  «public» modifier in interface methods.
> *  Illegal ‘{}’ for one line statement. — i found it harmful.
> I don`t want to link additional discussion about pos. 2 i think that harm
> is obvious.
> I suggest to get rid of them.
>
> what do you think ?
>
>
>



-- 
Sincerely yours, Ivan Daschinskiy

[ANNOUNCE] Release pyignite-0.5.1

[Ivan Daschinsky]

Igniters!

I suppose it is time to release pyignite 0.5.1. There are few good features
available and some users are eager to try it.

1. Added ability to record clients' events by event listeners. This is a
quite agile mechanism, and it allows writing custom metrics, tracing and
debugging.
2. Added debug logging

When implementing these features, I have found and fixed a few bugs thanks
for excessive logging and events. (see release notes of [1]).
Also I want to implement one minor feature that can be useful (ability to
set handshake timeout).
I am going to implement it in a few days. [2]

Full list of tickets for release is here [3]

I think that code freeze will be at 15:00 UTC 07/23. And on monday, 07/26
we will publish a release candidate for voting.

WDYT?


[1] -- https://issues.apache.org/jira/browse/IGNITE-15102
[2] -- https://issues.apache.org/jira/browse/IGNITE-15118
[3] --
https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%20python-0.5.1

Client events and monitoring in pyignite

[Ivan Daschinsky]

Hi, igniters!

Recently, one user asked me to add this functionality to pyignite.
Many users want:
1. To expose client metrics of app to prometheus (for example)
2. To write custom logic on specific client events (connect, disconnect,
reconnect)
3. To perform tracing

I suggests to introduce client side events listener API to pyignite.

In few words, a proposed API looks like this.
from pyignite.monitoring import QueryEventListener, ConnectionEventListener

class MyQueryEventListener(QueryEventListener):
   def on_query_success(self, event):
  ...

class MyQueryEventListener(QueryEventListener):
   def on_handshake_success (self, event):
  ...

client = Client(event_listeners=[MyQueryEventListener(),
MyConnectionEventListener()]

More details are here in PR[1]

What do you think about this feature? I suppose that it is of crucial
importance and
may be introduced to others thin clients.

[1] --- https://github.com/apache/ignite-python-thin-client/pull/46/files

Re: IEP-61 Transaction API desing for Ignite 3

[Ivan Daschinsky]

> yes, it is stated in the javadoc in the PR.
Ah, I see.

ср, 14 июл. 2021 г. в 12:16, Alexei Scherbakov :

> Ivan,
>
> And what if I have already committed transaction? Is it safe rollback
> already committed transaction? Rollback will silently return and do
> nothing? - yes, it is stated in the javadoc in the PR.
>
> Andrey,
>
> Then using "runInTransaction", lack of commit will cause a transaction to
> rollback automatically.
>
> There is no need for a "close" method, it just adds confusion.
>
>
> ср, 14 июл. 2021 г. в 11:37, Andrey Mashenkov  >:
>
> > Agree with Ivan.
> >
> > Method runInTransaction() should try to finish the transaction if the
> user
> > forgot to commit one.
> > I guess it might be a common mistake among new users.
> >
> > Also, I suggest to extent all table projections for better UX.
> > Let's allow
> > table.kvView().withTx(tx)
> > to user may cache kvVew instance and do
> > kvView.withTx(tx)
> > rather than
> > table.withTx(tx).kvVew()
> >
> >
> >
> > On Wed, Jul 14, 2021 at 10:13 AM Ivan Daschinsky 
> > wrote:
> >
> > > Alexey, and is there any analogue to close() of transaction? When you
> > start
> > > transaction, you should somehow to close it, if you don't catch
> exception
> > > or forget to commit.
> > >
> > > I suggest to add method closeAsync() to Transaction, so user can call
> it
> > in
> > > handle or whenComplete, i.e.
> > >
> > > So code will looks like
> > >
> > > CacheApi cache = CacheApi.getCache("testCache");
> > >
> > > Transactions
> > > .beginTransaction()
> > > .thenCompose(tx -> {
> > > CacheApi txCache = cache.withTx(tx);
> > > CompletableFuture result = txCache.getAsync("key")
> > > .thenCompose(val -> {
> > > if (val == "test") {
> > > return txCache.putAsync("key", "test1");
> > > }
> > > else
> > > return CompletableFuture.completedFuture(null);
> > > })
> > > .thenCompose(v -> tx.commitAsync())
> > > .handle((v, ex) -> null);
> > > return result.thenCompose(v -> tx.closeAsync());
> > > });
> > >
> > > I also suggests to add method something like this
> > >
> > > static CompletableFuture inTxAsync(Function > > CompletableFuture> action) {
> > > return Transactions
> > > .beginTransaction()
> > > .thenCompose(tx -> {
> > > CompletableFuture result = action.apply(tx)
> > > .handle((v, ex) -> null);
> > > return result.thenCompose(v -> tx.closeAsync());
> > > });
> > > }
> > >
> > > Async api is not very readable, but this method can help user write
> code,
> > > this is rewritten first example:
> > >
> > > Transactions.inTxAsync(tx -> {
> > > CacheApi txCache = cache.withTx(tx);
> > > return txCache.getAsync("key")
> > > .thenCompose(val -> {
> > > if (val == "test") {
> > > return txCache.putAsync("key", "test1");
> > > }
> > > else
> > > return CompletableFuture.completedFuture(null);
> > > })
> > > .thenCompose(v -> tx.commitAsync());
> > > });
> > >
> > > ср, 14 июл. 2021 г. в 10:03, Alexei Scherbakov <
> > > alexey.scherbak...@gmail.com
> > > >:
> > >
> > > > Andrey,
> > > >
> > > > I suggest you look at the PR [1], if you haven't.
> > > >
> > > > A transaction [2]
> > > > Transactions facade [3]
> > > > Examples [4]
> > > >
> > > > [1] https://github.com/apache/ignite-3/pull/214/files
> > > > [2]
> > > >
> > > >
> > >
> >
> https://github.com/apache/ignite-3/blob/d2122ce8c15de020e121f53509bd5a097aac9cf2/modules/api/src/main/java/org/apache/ignite/tx/Transaction.java
> > > > [3]
> > > >
> > > >
> > >
> >
> https://github.com/apache/ignite-3/blob/d2122ce8c15de020e121f53509bd5a097aac9cf2/module

Re: IEP-61 Transaction API desing for Ignite 3

[Ivan Daschinsky]

>>  Semantically "close" is the same as "rollback".
And what if I have already committed transaction? Is it safe rollback
already committed transaction? Rollback will silently return and do nothing?

ср, 14 июл. 2021 г. в 11:25, Alexei Scherbakov :

> Ivan,
>
> We don't need the "close" method in the proposed approach, because it's
> enough having "commit(Async)" and "rollback(Async)" to finish a
> transaction. Semantically "close" is the same as "rollback".
>
> If you are using "runInTransaction" API, you can't forget to call "close",
> it will be done automatically (rollback will be called at the end of a
> transaction closure).
>
> For async API it's required semantically to call commit or rollback.
>
> As for utility methods, I would keep tx API as small as possible for now.
> Anyway, these methods can be added later, if the need arises.
>
>
>
>
>
> ср, 14 июл. 2021 г. в 10:13, Ivan Daschinsky :
>
> > Alexey, and is there any analogue to close() of transaction? When you
> start
> > transaction, you should somehow to close it, if you don't catch exception
> > or forget to commit.
> >
> > I suggest to add method closeAsync() to Transaction, so user can call it
> in
> > handle or whenComplete, i.e.
> >
> > So code will looks like
> >
> > CacheApi cache = CacheApi.getCache("testCache");
> >
> > Transactions
> > .beginTransaction()
> > .thenCompose(tx -> {
> > CacheApi txCache = cache.withTx(tx);
> > CompletableFuture result = txCache.getAsync("key")
> > .thenCompose(val -> {
> > if (val == "test") {
> > return txCache.putAsync("key", "test1");
> > }
> > else
> > return CompletableFuture.completedFuture(null);
> > })
> > .thenCompose(v -> tx.commitAsync())
> > .handle((v, ex) -> null);
> > return result.thenCompose(v -> tx.closeAsync());
> > });
> >
> > I also suggests to add method something like this
> >
> > static CompletableFuture inTxAsync(Function > CompletableFuture> action) {
> > return Transactions
> > .beginTransaction()
> > .thenCompose(tx -> {
> > CompletableFuture result = action.apply(tx)
> > .handle((v, ex) -> null);
> > return result.thenCompose(v -> tx.closeAsync());
> > });
> > }
> >
> > Async api is not very readable, but this method can help user write code,
> > this is rewritten first example:
> >
> > Transactions.inTxAsync(tx -> {
> > CacheApi txCache = cache.withTx(tx);
> > return txCache.getAsync("key")
> > .thenCompose(val -> {
> > if (val == "test") {
> > return txCache.putAsync("key", "test1");
> > }
> > else
> > return CompletableFuture.completedFuture(null);
> > })
> > .thenCompose(v -> tx.commitAsync());
> > });
> >
> > ср, 14 июл. 2021 г. в 10:03, Alexei Scherbakov <
> > alexey.scherbak...@gmail.com
> > >:
> >
> > > Andrey,
> > >
> > > I suggest you look at the PR [1], if you haven't.
> > >
> > > A transaction [2]
> > > Transactions facade [3]
> > > Examples [4]
> > >
> > > [1] https://github.com/apache/ignite-3/pull/214/files
> > > [2]
> > >
> > >
> >
> https://github.com/apache/ignite-3/blob/d2122ce8c15de020e121f53509bd5a097aac9cf2/modules/api/src/main/java/org/apache/ignite/tx/Transaction.java
> > > [3]
> > >
> > >
> >
> https://github.com/apache/ignite-3/blob/d2122ce8c15de020e121f53509bd5a097aac9cf2/modules/api/src/main/java/org/apache/ignite/tx/IgniteTransactions.java
> > > [4]
> > >
> > >
> >
> https://github.com/apache/ignite-3/blob/d2122ce8c15de020e121f53509bd5a097aac9cf2/modules/table/src/test/java/org/apache/ignite/internal/table/TxTest.java
> > >
> > >
> > > вт, 13 июл. 2021 г. в 19:41, Andrey Gura :
> > >
> > > > Alexey,
> > > >
> > > > could you please describe Transaction interface?
> > > >
> > > > Also it would be great to have a couple examples of using the
> proposed
> > > API.
> > > >
> > > > On Tue, Jul 13, 2021 at 4:43 PM Alexei Scherbakov
> > > >  wrote:
> > > > >
> > > > > Folks,
> > > > >
> > > > > I've prepared a PR implementing my vision of public transactions
> API.
> > > > >
> > > > > API is very simple and similar to Ignite 2, but has some
> differences.
> > > > >
> > > > > More details can be found here [1]
> > > > >
> > > > > Share your thoughts.
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-15086
> > > > >
> > > > > --
> > > > >
> > > > > Best regards,
> > > > > Alexei Scherbakov
> > > >
> > >
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> > >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: IEP-61 Transaction API desing for Ignite 3

[Ivan Daschinsky]

Alexey, and is there any analogue to close() of transaction? When you start
transaction, you should somehow to close it, if you don't catch exception
or forget to commit.

I suggest to add method closeAsync() to Transaction, so user can call it in
handle or whenComplete, i.e.

So code will looks like

CacheApi cache = CacheApi.getCache("testCache");

Transactions
.beginTransaction()
.thenCompose(tx -> {
CacheApi txCache = cache.withTx(tx);
CompletableFuture result = txCache.getAsync("key")
.thenCompose(val -> {
if (val == "test") {
return txCache.putAsync("key", "test1");
}
else
return CompletableFuture.completedFuture(null);
})
.thenCompose(v -> tx.commitAsync())
.handle((v, ex) -> null);
return result.thenCompose(v -> tx.closeAsync());
});

I also suggests to add method something like this

static CompletableFuture inTxAsync(Function> action) {
return Transactions
.beginTransaction()
.thenCompose(tx -> {
CompletableFuture result = action.apply(tx)
.handle((v, ex) -> null);
return result.thenCompose(v -> tx.closeAsync());
});
}

Async api is not very readable, but this method can help user write code,
this is rewritten first example:

Transactions.inTxAsync(tx -> {
CacheApi txCache = cache.withTx(tx);
return txCache.getAsync("key")
.thenCompose(val -> {
if (val == "test") {
return txCache.putAsync("key", "test1");
}
else
return CompletableFuture.completedFuture(null);
})
.thenCompose(v -> tx.commitAsync());
});

ср, 14 июл. 2021 г. в 10:03, Alexei Scherbakov :

> Andrey,
>
> I suggest you look at the PR [1], if you haven't.
>
> A transaction [2]
> Transactions facade [3]
> Examples [4]
>
> [1] https://github.com/apache/ignite-3/pull/214/files
> [2]
>
> https://github.com/apache/ignite-3/blob/d2122ce8c15de020e121f53509bd5a097aac9cf2/modules/api/src/main/java/org/apache/ignite/tx/Transaction.java
> [3]
>
> https://github.com/apache/ignite-3/blob/d2122ce8c15de020e121f53509bd5a097aac9cf2/modules/api/src/main/java/org/apache/ignite/tx/IgniteTransactions.java
> [4]
>
> https://github.com/apache/ignite-3/blob/d2122ce8c15de020e121f53509bd5a097aac9cf2/modules/table/src/test/java/org/apache/ignite/internal/table/TxTest.java
>
>
> вт, 13 июл. 2021 г. в 19:41, Andrey Gura :
>
> > Alexey,
> >
> > could you please describe Transaction interface?
> >
> > Also it would be great to have a couple examples of using the proposed
> API.
> >
> > On Tue, Jul 13, 2021 at 4:43 PM Alexei Scherbakov
> >  wrote:
> > >
> > > Folks,
> > >
> > > I've prepared a PR implementing my vision of public transactions API.
> > >
> > > API is very simple and similar to Ignite 2, but has some differences.
> > >
> > > More details can be found here [1]
> > >
> > > Share your thoughts.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-15086
> > >
> > > --
> > >
> > > Best regards,
> > > Alexei Scherbakov
> >
>
>
> --
>
> Best regards,
> Alexei Scherbakov
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

Pavel, actually, setters in builder is not ok. But as for me, the better
approach is something like this:

IgniteClient client = IgniteClient.start("127.0.0.1:10800")

or

CompletableFuture client =
IgniteClient.startAsync(IgniteClientConfiguration.builder()
  .addresses("127.0.0.1:10800)
  .timeout(5000)
  .build())

I suppose that should be a few simple shortcuts to start client as is and
immutable IgniteClientConfiguration.

My point is:
1. It is easier to understand (creation is within IgniteClient, easy to
find and very predictable, Ignition is not intuitive, but may be fun and
eastern egg like)
2. Immutable configuration -- harder to make a mistake when extending
configuration (you must always copy config if it is mutable), less syntaxic
garbage ('set' prefix is not needed)



вт, 13 июл. 2021 г. в 09:55, Pavel Tupitsyn :

> Val,
>
> Ivan D has convinced me above to try the builder pattern, which is used by
> many other drivers (Redis, Mongo, etc).
> With IgniteClient class in ignite-client module:
>
> Ignite client = IgniteClient.builder()
> .setAddresses("127.0.0.1:10800")
> .setTimeout(5000)
> .build();
>
> What do you think?
>
> On Mon, Jul 12, 2021 at 10:57 PM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Pavel,
> >
> > If Ignition is the single entry point, it needs to have access to the
> > server node instance, so it needs to depend on ignite-runner. Therefore,
> > including Ignition in ignite-client means that ignite-client depends on
> > ignite-runner, which we cannot have.
> >
> > -Val
> >
> > On Sat, Jul 10, 2021 at 4:17 AM Pavel Tupitsyn 
> > wrote:
> >
> > > Val,
> > >
> > > My suggestion is to have Ignition class in ignite-client module.
> > >
> > > On Fri, Jul 9, 2021 at 11:01 PM Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Pavel,
> > > >
> > > > Ivan actually brings a good point. While the client is in a separate
> > > > module, Ignition (if we make it static) will have to depend on both
> > > > ignite-client and ignite-runner, and we will have to ship it along
> with
> > > the
> > > > client. This indeed creates an uber-jar, so we can't really have a
> > single
> > > > entry point, unfortunately.
> > > >
> > > > I'm not sure what is the best way to proceed here. Let's think it
> over
> > > and
> > > > see if there are any suggestions.
> > > >
> > > > -Val
> > > >
> > > > On Fri, Jul 9, 2021 at 6:31 AM Pavel Tupitsyn 
> > > > wrote:
> > > >
> > > > > > why thin client should be in core module
> > > > >
> > > > > It will be in a separate module (ignite-client).
> > > > > I was talking about "core library" as a primary set of modules that
> > we
> > > > > ship.
> > > > > Integrations with 3rd party libraries and frameworks can be shipped
> > as
> > > > > extensions.
> > > > >
> > > > > Anyway, let's postpone the discussion of Rx and Kotlin.
> > > > > The main goal right now is to implement the most basic Java thin
> > > client.
> > > > > CompletableFuture is the primary way to deliver async APIs in
> Ignite
> > > 3.0,
> > > > > other things can be added later.
> > > > >
> > > > > On Fri, Jul 9, 2021 at 3:37 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > > I don't think we need an explicit reactive API in the core
> > library.
> > > > > >
> > > > > > Have you ever thought about why thin client should be in core
> > module?
> > > > Why
> > > > > > we do the same thing as we did in ignite 2.x? In the days of
> cloud
> > > > native
> > > > > > we still think about large uber-jar with everything?
> > > > > >
> > > > > > > Same story with Kotlin, it works with CompletableFuture.
> > > > > > Users don't want to code by theyselves, they want to use ready
> and
> > > > > complete
> > > > > > clients. Please, don't underestimate kotlin, kotlin coroutines
> and
> > > > > reactive
> > > > > > streams. They are all the first class citizens in spring 5 for 3

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

The main reason is that immutable configurations nowadays is more
preferable variant than mutable. For example, when you initialize client
you must copy configuration in order to be consistent. Nobody nowadays read
configuration from files as is, moreover -- the most microservices
retrieves just simple string variables from env or from consul/etcd.

вс, 11 июл. 2021 г., 17:48 Pavel Tupitsyn :

> Serialization is just one example, and it does not have to be XML.
> JSON, YAML, HOCON configs are widely used.
>
> Anyway, I see no reason for it NOT to be a POJO.
> POJOs are ergonomic and work everywhere.
>
>
> On Sun, Jul 11, 2021 at 8:24 AM Ivan Daschinsky 
> wrote:
>
> > > But configuration should be a POJO - a class with setters and getters,
> > nothing else.
> >  Why it should? Why ClientConfiguration should be serializable? Who needs
> > that? Xml configuration even in spring are not widely used for years :)
> >
> > вс, 11 июл. 2021 г., 00:57 Pavel Tupitsyn :
> >
> > > Ivan D,
> > >
> > > Let's compare WebClient approach with Ignite 2.x approach:
> > >
> > > // Ignite 2.x
> > > var cfg = new ClientConfiguration()
> > > .setAddresses("127.0.0.1:10800", "127.0.0.1:10801")
> > > .setTimeout(4000);
> > >
> > > Ignite client = Ignition.startClient(cfg);
> > >
> > > // Ignite 3.x inspired by WebClient
> > >
> > > Ignite client = IgniteClient.builder()
> > > .setAddresses("127.0.0.1:10800", "127.0.0.1:10801")
> > > .setTimeout(4000)
> > > .build();
> > >
> > >
> > > To me, there is virtually no difference. Both are simple and
> > > straightforward.
> > > "specific static method within strange class with strange name"
> (whatever
> > > that means) can be applied equally to both.
> > >
> > > However, ClientConfiguration is a POJO with all the corresponding
> > benefits,
> > > like easy serialization.
> > > WebClient.Builder is an interface that does not even provide a way to
> get
> > > the property value back.
> > >
> > >
> > > To be clear, I don't have strong opinions on naming (it can be
> > > Ignition.start or IgniteClient.connect or whatever).
> > > But configuration should be a POJO - a class with setters and getters,
> > > nothing else.
> > >
> > > On Sat, Jul 10, 2021 at 9:49 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Pavel, I and Ivan P.  have already get examples of lettuce.io
> > > >
> > > > Another example is spring 5  reactive WebClient
> > > > https://www.baeldung.com/spring-5-webclient
> > > >
> > > > сб, 10 июл. 2021 г., 19:50 Pavel Tupitsyn :
> > > >
> > > > > Ivan D.,
> > > > >
> > > > > > simple and straightforward client builder
> > > > > Can you please provide an example of that, a piece of code with
> > > suggested
> > > > > API usage? Ideally compared to the current 2.x approach.
> > > > >
> > > > > On Sat, Jul 10, 2021 at 3:20 PM Ivan Daschinsky <
> ivanda...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > It is a quite questionable decision , as for me,  to have
> specific
> > > > static
> > > > > > method within strange class with strange name (and it is a well
> > known
> > > > > > antipatter ) with chained twisted configuration class instead of
> > just
> > > > > > simple and straightforward client builder, isn't it? Name
> > 'Ignition'
> > > is
> > > > > may
> > > > > > be fun, but I cannot understand how it name connects to
> > bootstraping
> > > > > > ignite's client.
> > > > > >
> > > > > > сб, 10 июл. 2021 г., 14:17 Pavel Tupitsyn  >:
> > > > > >
> > > > > > > Val,
> > > > > > >
> > > > > > > My suggestion is to have Ignition class in ignite-client
> module.
> > > > > > >
> > > > > > > On Fri, Jul 9, 2021 at 11:01 PM Valentin Kulichenko <
> > > > > > > valentin.kuliche...@gmail.com> wrote:
> > > > > > >
> > > > > > > > Pavel,
&g

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

But if you looked at other competitors, non  of them doesn't use mutable
configurations. POJO or even more, JavaBeans are not used in the mos
products.

I just asked if you looked at others. Today it looks like you suggest use
just the same interfaces as in 2.x. Things changed a lot.

вс, 11 июл. 2021 г., 17:48 Pavel Tupitsyn :

> Serialization is just one example, and it does not have to be XML.
> JSON, YAML, HOCON configs are widely used.
>
> Anyway, I see no reason for it NOT to be a POJO.
> POJOs are ergonomic and work everywhere.
>
>
> On Sun, Jul 11, 2021 at 8:24 AM Ivan Daschinsky 
> wrote:
>
> > > But configuration should be a POJO - a class with setters and getters,
> > nothing else.
> >  Why it should? Why ClientConfiguration should be serializable? Who needs
> > that? Xml configuration even in spring are not widely used for years :)
> >
> > вс, 11 июл. 2021 г., 00:57 Pavel Tupitsyn :
> >
> > > Ivan D,
> > >
> > > Let's compare WebClient approach with Ignite 2.x approach:
> > >
> > > // Ignite 2.x
> > > var cfg = new ClientConfiguration()
> > > .setAddresses("127.0.0.1:10800", "127.0.0.1:10801")
> > > .setTimeout(4000);
> > >
> > > Ignite client = Ignition.startClient(cfg);
> > >
> > > // Ignite 3.x inspired by WebClient
> > >
> > > Ignite client = IgniteClient.builder()
> > > .setAddresses("127.0.0.1:10800", "127.0.0.1:10801")
> > > .setTimeout(4000)
> > > .build();
> > >
> > >
> > > To me, there is virtually no difference. Both are simple and
> > > straightforward.
> > > "specific static method within strange class with strange name"
> (whatever
> > > that means) can be applied equally to both.
> > >
> > > However, ClientConfiguration is a POJO with all the corresponding
> > benefits,
> > > like easy serialization.
> > > WebClient.Builder is an interface that does not even provide a way to
> get
> > > the property value back.
> > >
> > >
> > > To be clear, I don't have strong opinions on naming (it can be
> > > Ignition.start or IgniteClient.connect or whatever).
> > > But configuration should be a POJO - a class with setters and getters,
> > > nothing else.
> > >
> > > On Sat, Jul 10, 2021 at 9:49 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Pavel, I and Ivan P.  have already get examples of lettuce.io
> > > >
> > > > Another example is spring 5  reactive WebClient
> > > > https://www.baeldung.com/spring-5-webclient
> > > >
> > > > сб, 10 июл. 2021 г., 19:50 Pavel Tupitsyn :
> > > >
> > > > > Ivan D.,
> > > > >
> > > > > > simple and straightforward client builder
> > > > > Can you please provide an example of that, a piece of code with
> > > suggested
> > > > > API usage? Ideally compared to the current 2.x approach.
> > > > >
> > > > > On Sat, Jul 10, 2021 at 3:20 PM Ivan Daschinsky <
> ivanda...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > It is a quite questionable decision , as for me,  to have
> specific
> > > > static
> > > > > > method within strange class with strange name (and it is a well
> > known
> > > > > > antipatter ) with chained twisted configuration class instead of
> > just
> > > > > > simple and straightforward client builder, isn't it? Name
> > 'Ignition'
> > > is
> > > > > may
> > > > > > be fun, but I cannot understand how it name connects to
> > bootstraping
> > > > > > ignite's client.
> > > > > >
> > > > > > сб, 10 июл. 2021 г., 14:17 Pavel Tupitsyn  >:
> > > > > >
> > > > > > > Val,
> > > > > > >
> > > > > > > My suggestion is to have Ignition class in ignite-client
> module.
> > > > > > >
> > > > > > > On Fri, Jul 9, 2021 at 11:01 PM Valentin Kulichenko <
> > > > > > > valentin.kuliche...@gmail.com> wrote:
> > > > > > >
> > > > > > > > Pavel,
> > > > > > > >
> > > > 

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

> But configuration should be a POJO - a class with setters and getters,
nothing else.
 Why it should? Why ClientConfiguration should be serializable? Who needs
that? Xml configuration even in spring are not widely used for years :)

вс, 11 июл. 2021 г., 00:57 Pavel Tupitsyn :

> Ivan D,
>
> Let's compare WebClient approach with Ignite 2.x approach:
>
> // Ignite 2.x
> var cfg = new ClientConfiguration()
> .setAddresses("127.0.0.1:10800", "127.0.0.1:10801")
> .setTimeout(4000);
>
> Ignite client = Ignition.startClient(cfg);
>
> // Ignite 3.x inspired by WebClient
>
> Ignite client = IgniteClient.builder()
> .setAddresses("127.0.0.1:10800", "127.0.0.1:10801")
> .setTimeout(4000)
> .build();
>
>
> To me, there is virtually no difference. Both are simple and
> straightforward.
> "specific static method within strange class with strange name" (whatever
> that means) can be applied equally to both.
>
> However, ClientConfiguration is a POJO with all the corresponding benefits,
> like easy serialization.
> WebClient.Builder is an interface that does not even provide a way to get
> the property value back.
>
>
> To be clear, I don't have strong opinions on naming (it can be
> Ignition.start or IgniteClient.connect or whatever).
> But configuration should be a POJO - a class with setters and getters,
> nothing else.
>
> On Sat, Jul 10, 2021 at 9:49 PM Ivan Daschinsky 
> wrote:
>
> > Pavel, I and Ivan P.  have already get examples of lettuce.io
> >
> > Another example is spring 5  reactive WebClient
> > https://www.baeldung.com/spring-5-webclient
> >
> > сб, 10 июл. 2021 г., 19:50 Pavel Tupitsyn :
> >
> > > Ivan D.,
> > >
> > > > simple and straightforward client builder
> > > Can you please provide an example of that, a piece of code with
> suggested
> > > API usage? Ideally compared to the current 2.x approach.
> > >
> > > On Sat, Jul 10, 2021 at 3:20 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > It is a quite questionable decision , as for me,  to have specific
> > static
> > > > method within strange class with strange name (and it is a well known
> > > > antipatter ) with chained twisted configuration class instead of just
> > > > simple and straightforward client builder, isn't it? Name 'Ignition'
> is
> > > may
> > > > be fun, but I cannot understand how it name connects to bootstraping
> > > > ignite's client.
> > > >
> > > > сб, 10 июл. 2021 г., 14:17 Pavel Tupitsyn :
> > > >
> > > > > Val,
> > > > >
> > > > > My suggestion is to have Ignition class in ignite-client module.
> > > > >
> > > > > On Fri, Jul 9, 2021 at 11:01 PM Valentin Kulichenko <
> > > > > valentin.kuliche...@gmail.com> wrote:
> > > > >
> > > > > > Pavel,
> > > > > >
> > > > > > Ivan actually brings a good point. While the client is in a
> > separate
> > > > > > module, Ignition (if we make it static) will have to depend on
> both
> > > > > > ignite-client and ignite-runner, and we will have to ship it
> along
> > > with
> > > > > the
> > > > > > client. This indeed creates an uber-jar, so we can't really have
> a
> > > > single
> > > > > > entry point, unfortunately.
> > > > > >
> > > > > > I'm not sure what is the best way to proceed here. Let's think it
> > > over
> > > > > and
> > > > > > see if there are any suggestions.
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Fri, Jul 9, 2021 at 6:31 AM Pavel Tupitsyn <
> > ptupit...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > > why thin client should be in core module
> > > > > > >
> > > > > > > It will be in a separate module (ignite-client).
> > > > > > > I was talking about "core library" as a primary set of modules
> > that
> > > > we
> > > > > > > ship.
> > > > > > > Integrations with 3rd party libraries and frameworks can be
> > shipped
> > > > as
>

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

Pavel, I and Ivan P.  have already get examples of lettuce.io

Another example is spring 5  reactive WebClient
https://www.baeldung.com/spring-5-webclient

сб, 10 июл. 2021 г., 19:50 Pavel Tupitsyn :

> Ivan D.,
>
> > simple and straightforward client builder
> Can you please provide an example of that, a piece of code with suggested
> API usage? Ideally compared to the current 2.x approach.
>
> On Sat, Jul 10, 2021 at 3:20 PM Ivan Daschinsky 
> wrote:
>
> > It is a quite questionable decision , as for me,  to have specific static
> > method within strange class with strange name (and it is a well known
> > antipatter ) with chained twisted configuration class instead of just
> > simple and straightforward client builder, isn't it? Name 'Ignition' is
> may
> > be fun, but I cannot understand how it name connects to bootstraping
> > ignite's client.
> >
> > сб, 10 июл. 2021 г., 14:17 Pavel Tupitsyn :
> >
> > > Val,
> > >
> > > My suggestion is to have Ignition class in ignite-client module.
> > >
> > > On Fri, Jul 9, 2021 at 11:01 PM Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Pavel,
> > > >
> > > > Ivan actually brings a good point. While the client is in a separate
> > > > module, Ignition (if we make it static) will have to depend on both
> > > > ignite-client and ignite-runner, and we will have to ship it along
> with
> > > the
> > > > client. This indeed creates an uber-jar, so we can't really have a
> > single
> > > > entry point, unfortunately.
> > > >
> > > > I'm not sure what is the best way to proceed here. Let's think it
> over
> > > and
> > > > see if there are any suggestions.
> > > >
> > > > -Val
> > > >
> > > > On Fri, Jul 9, 2021 at 6:31 AM Pavel Tupitsyn 
> > > > wrote:
> > > >
> > > > > > why thin client should be in core module
> > > > >
> > > > > It will be in a separate module (ignite-client).
> > > > > I was talking about "core library" as a primary set of modules that
> > we
> > > > > ship.
> > > > > Integrations with 3rd party libraries and frameworks can be shipped
> > as
> > > > > extensions.
> > > > >
> > > > > Anyway, let's postpone the discussion of Rx and Kotlin.
> > > > > The main goal right now is to implement the most basic Java thin
> > > client.
> > > > > CompletableFuture is the primary way to deliver async APIs in
> Ignite
> > > 3.0,
> > > > > other things can be added later.
> > > > >
> > > > > On Fri, Jul 9, 2021 at 3:37 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > > I don't think we need an explicit reactive API in the core
> > library.
> > > > > >
> > > > > > Have you ever thought about why thin client should be in core
> > module?
> > > > Why
> > > > > > we do the same thing as we did in ignite 2.x? In the days of
> cloud
> > > > native
> > > > > > we still think about large uber-jar with everything?
> > > > > >
> > > > > > > Same story with Kotlin, it works with CompletableFuture.
> > > > > > Users don't want to code by theyselves, they want to use ready
> and
> > > > > complete
> > > > > > clients. Please, don't underestimate kotlin, kotlin coroutines
> and
> > > > > reactive
> > > > > > streams. They are all the first class citizens in spring 5 for 3
> > > years
> > > > > >
> > > > > > пт, 9 июл. 2021 г., 14:43 Pavel Tupitsyn :
> > > > > >
> > > > > > > > You forget about reactive api
> > > > > > >
> > > > > > > I don't think we need an explicit reactive API in the core
> > library.
> > > > > > > Observable.fromFuture bridges async to Rx easily:
> > > > > > >
> > > > > > > Observable.fromFuture(client.putAsync(k, v)).flatMap(...)
> > > > > > >
> > > > > > > Same story with Kotlin, it works with CompletableFuture.
> > > > > > >
> > >

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

It is a quite questionable decision , as for me,  to have specific static
method within strange class with strange name (and it is a well known
antipatter ) with chained twisted configuration class instead of just
simple and straightforward client builder, isn't it? Name 'Ignition' is may
be fun, but I cannot understand how it name connects to bootstraping
ignite's client.

сб, 10 июл. 2021 г., 14:17 Pavel Tupitsyn :

> Val,
>
> My suggestion is to have Ignition class in ignite-client module.
>
> On Fri, Jul 9, 2021 at 11:01 PM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Pavel,
> >
> > Ivan actually brings a good point. While the client is in a separate
> > module, Ignition (if we make it static) will have to depend on both
> > ignite-client and ignite-runner, and we will have to ship it along with
> the
> > client. This indeed creates an uber-jar, so we can't really have a single
> > entry point, unfortunately.
> >
> > I'm not sure what is the best way to proceed here. Let's think it over
> and
> > see if there are any suggestions.
> >
> > -Val
> >
> > On Fri, Jul 9, 2021 at 6:31 AM Pavel Tupitsyn 
> > wrote:
> >
> > > > why thin client should be in core module
> > >
> > > It will be in a separate module (ignite-client).
> > > I was talking about "core library" as a primary set of modules that we
> > > ship.
> > > Integrations with 3rd party libraries and frameworks can be shipped as
> > > extensions.
> > >
> > > Anyway, let's postpone the discussion of Rx and Kotlin.
> > > The main goal right now is to implement the most basic Java thin
> client.
> > > CompletableFuture is the primary way to deliver async APIs in Ignite
> 3.0,
> > > other things can be added later.
> > >
> > > On Fri, Jul 9, 2021 at 3:37 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > > I don't think we need an explicit reactive API in the core library.
> > > >
> > > > Have you ever thought about why thin client should be in core module?
> > Why
> > > > we do the same thing as we did in ignite 2.x? In the days of cloud
> > native
> > > > we still think about large uber-jar with everything?
> > > >
> > > > > Same story with Kotlin, it works with CompletableFuture.
> > > > Users don't want to code by theyselves, they want to use ready and
> > > complete
> > > > clients. Please, don't underestimate kotlin, kotlin coroutines and
> > > reactive
> > > > streams. They are all the first class citizens in spring 5 for 3
> years
> > > >
> > > > пт, 9 июл. 2021 г., 14:43 Pavel Tupitsyn :
> > > >
> > > > > > You forget about reactive api
> > > > >
> > > > > I don't think we need an explicit reactive API in the core library.
> > > > > Observable.fromFuture bridges async to Rx easily:
> > > > >
> > > > > Observable.fromFuture(client.putAsync(k, v)).flatMap(...)
> > > > >
> > > > > Same story with Kotlin, it works with CompletableFuture.
> > > > >
> > > > > On Fri, Jul 9, 2021 at 1:31 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > You forget about reactive api :)
> > > > > >
> > > > > > And whats a problem with discocerability?
> > > > > >
> > > > > > var syncApi = client.sync();
> > > > > > syncApi.put(k, v);
> > > > > >
> > > > > > var rxApi = client.reactive();
> > > > > > rxApi.put(k,v).flatMap(res -> );
> > > > > >
> > > > > > And sync, async and reactive is not enough, it is good idea to
> > > support
> > > > > > kotlin coroutines also :)
> > > > > >
> > > > > > пт, 9 июл. 2021 г., 13:26 Pavel Tupitsyn :
> > > > > >
> > > > > > > Ivan D.,
> > > > > > >
> > > > > > > > container of properties
> > > > > > >
> > > > > > > What is a container of properties?
> > > > > > > As a user, I want a simple way to start a client and perform
> > > > > operations.
> > > > > > >
> > > > > > > I don't wan

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

> I don't think we need an explicit reactive API in the core library.

Have you ever thought about why thin client should be in core module? Why
we do the same thing as we did in ignite 2.x? In the days of cloud native
we still think about large uber-jar with everything?

> Same story with Kotlin, it works with CompletableFuture.
Users don't want to code by theyselves, they want to use ready and complete
clients. Please, don't underestimate kotlin, kotlin coroutines and reactive
streams. They are all the first class citizens in spring 5 for 3 years

пт, 9 июл. 2021 г., 14:43 Pavel Tupitsyn :

> > You forget about reactive api
>
> I don't think we need an explicit reactive API in the core library.
> Observable.fromFuture bridges async to Rx easily:
>
> Observable.fromFuture(client.putAsync(k, v)).flatMap(...)
>
> Same story with Kotlin, it works with CompletableFuture.
>
> On Fri, Jul 9, 2021 at 1:31 PM Ivan Daschinsky 
> wrote:
>
> > You forget about reactive api :)
> >
> > And whats a problem with discocerability?
> >
> > var syncApi = client.sync();
> > syncApi.put(k, v);
> >
> > var rxApi = client.reactive();
> > rxApi.put(k,v).flatMap(res -> );
> >
> > And sync, async and reactive is not enough, it is good idea to support
> > kotlin coroutines also :)
> >
> > пт, 9 июл. 2021 г., 13:26 Pavel Tupitsyn :
> >
> > > Ivan D.,
> > >
> > > > container of properties
> > >
> > > What is a container of properties?
> > > As a user, I want a simple way to start a client and perform
> operations.
> > >
> > > I don't want anything confusing and complicated like Netty Bootstrap.
> > There
> > > might be a reason for Netty to be this way - it is a low-level library.
> > But
> > > Ignite is not.
> > >
> > >
> > > > separate facades for sync, async
> > >
> > > Strongly disagree with this idea. It hurts API discoverability.
> > > As a user, in my IDE I type "igniteTable.get" and see a list of
> > suggestions
> > > like get, getAsync, getAndPut, getAndPutAsync.
> > > I don't want to have a separate interface and a separate variable to
> deal
> > > with sync and async methods.
> > >
> > > Not sure what's the problem with documentation - can you elaborate
> > please?
> > >
> > > On Fri, Jul 9, 2021 at 12:51 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Pavel, actually I suggests to separate container of properties(client
> > in
> > > > lettuce) and actual connection or connections (stateful connection in
> > > > lettuce). Actual connection initialization could be sync or async, it
> > > > doesn't matter. It can be Ignition#startClient or
> > > > Ignition#startClientAsync, but I'd prefer lettuce approach
> > > >
> > > >
> > > > Also, it would be great to have separate facades for sync, async and
> > > > reactive api. Mixing all of them in one interface is a documentation
> > > > nightmare.
> > > >
> > > > пт, 9 июл. 2021 г., 11:55 Pavel Tupitsyn :
> > > >
> > > > > Ivan P., Ivan D.,
> > > > >
> > > > > I don't think it makes sense to separate IgniteConnection and
> > > > IgniteClient
> > > > > like Lettuce does,
> > > > > because IgniteClient will maintain connections to multiple server
> > nodes
> > > > > automatically,
> > > > > and the number of connections can grow and shrink dynamically.
> > > > >
> > > > > This is required to support dynamic clusters together with
> partition
> > > > > awareness.
> > > > >
> > > > >
> > > > > > Why not to make async variant of connection
> > > > >
> > > > > Ignite API will (eventually) have both sync and async variants of
> > every
> > > > > method, where applicable,
> > > > > including the method that connects the client to the cluster.
> > > > >
> > > > > On Fri, Jul 9, 2021 at 9:55 AM Ivan Pavlukhin  >
> > > > wrote:
> > > > >
> > > > > > Val,
> > > > > >
> > > > > > > Ignition IS the entry point to Ignite, so I'm not sure I got
> your
> > > > point
> > > > > > :)
> > > > > > > Either way, please feel free to give y

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

You forget about reactive api :)

And whats a problem with discocerability?

var syncApi = client.sync();
syncApi.put(k, v);

var rxApi = client.reactive();
rxApi.put(k,v).flatMap(res -> );

And sync, async and reactive is not enough, it is good idea to support
kotlin coroutines also :)

пт, 9 июл. 2021 г., 13:26 Pavel Tupitsyn :

> Ivan D.,
>
> > container of properties
>
> What is a container of properties?
> As a user, I want a simple way to start a client and perform operations.
>
> I don't want anything confusing and complicated like Netty Bootstrap. There
> might be a reason for Netty to be this way - it is a low-level library. But
> Ignite is not.
>
>
> > separate facades for sync, async
>
> Strongly disagree with this idea. It hurts API discoverability.
> As a user, in my IDE I type "igniteTable.get" and see a list of suggestions
> like get, getAsync, getAndPut, getAndPutAsync.
> I don't want to have a separate interface and a separate variable to deal
> with sync and async methods.
>
> Not sure what's the problem with documentation - can you elaborate please?
>
> On Fri, Jul 9, 2021 at 12:51 PM Ivan Daschinsky 
> wrote:
>
> > Pavel, actually I suggests to separate container of properties(client in
> > lettuce) and actual connection or connections (stateful connection in
> > lettuce). Actual connection initialization could be sync or async, it
> > doesn't matter. It can be Ignition#startClient or
> > Ignition#startClientAsync, but I'd prefer lettuce approach
> >
> >
> > Also, it would be great to have separate facades for sync, async and
> > reactive api. Mixing all of them in one interface is a documentation
> > nightmare.
> >
> > пт, 9 июл. 2021 г., 11:55 Pavel Tupitsyn :
> >
> > > Ivan P., Ivan D.,
> > >
> > > I don't think it makes sense to separate IgniteConnection and
> > IgniteClient
> > > like Lettuce does,
> > > because IgniteClient will maintain connections to multiple server nodes
> > > automatically,
> > > and the number of connections can grow and shrink dynamically.
> > >
> > > This is required to support dynamic clusters together with partition
> > > awareness.
> > >
> > >
> > > > Why not to make async variant of connection
> > >
> > > Ignite API will (eventually) have both sync and async variants of every
> > > method, where applicable,
> > > including the method that connects the client to the cluster.
> > >
> > > On Fri, Jul 9, 2021 at 9:55 AM Ivan Pavlukhin 
> > wrote:
> > >
> > > > Val,
> > > >
> > > > > Ignition IS the entry point to Ignite, so I'm not sure I got your
> > point
> > > > :)
> > > > > Either way, please feel free to give your suggestions for an
> > > alternative
> > > > name if you have any.
> > > >
> > > > Well, it is not only about naming but it is also about code
> > > > organization. Ivan D. already referenced to alternative API styles (I
> > > > suppose [1] describes the idea).
> > > >
> > > > My main points are:
> > > > 1. Ignite 3 is a great opportunity to make things better.
> > > > 2. Using (or reusing) confusing names and entities should be avoided.
> > > >
> > > > Another rather straightforward startup/bootstrap approach is used in
> > > > Netty [2]. For Ignite I would spell it like IgniteServer.Bootstrap
> and
> > > > IgniteClient.Bootstrap.
> > > >
> > > > Also I suppose that thin client API is more important because much
> > > > more users will use it. I hope that a lot of Community members will
> > > > share their ideas.
> > > >
> > > > [1] https://www.baeldung.com/java-redis-lettuce
> > > > [2] https://netty.io/4.0/api/io/netty/bootstrap/ServerBootstrap.html
> > > >
> > > > 2021-07-09 1:41 GMT+03:00, Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com>:
> > > > > Ivan,
> > > > >
> > > > > I've seen the link, but I still don't understand what exactly you
> > > propose
> > > > > to change in the current API, and what is your concern. Could you
> > > please
> > > > > clarify? How you think Ignite API should look like?
> > > > >
> > > > > -Val
> > > > >
> > > > > On Thu, Jul 8, 2021 at 2:18 PM Ivan Daschinsky <
> ivanda.

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

Pavel, actually I suggests to separate container of properties(client in
lettuce) and actual connection or connections (stateful connection in
lettuce). Actual connection initialization could be sync or async, it
doesn't matter. It can be Ignition#startClient or
Ignition#startClientAsync, but I'd prefer lettuce approach


Also, it would be great to have separate facades for sync, async and
reactive api. Mixing all of them in one interface is a documentation
nightmare.

пт, 9 июл. 2021 г., 11:55 Pavel Tupitsyn :

> Ivan P., Ivan D.,
>
> I don't think it makes sense to separate IgniteConnection and IgniteClient
> like Lettuce does,
> because IgniteClient will maintain connections to multiple server nodes
> automatically,
> and the number of connections can grow and shrink dynamically.
>
> This is required to support dynamic clusters together with partition
> awareness.
>
>
> > Why not to make async variant of connection
>
> Ignite API will (eventually) have both sync and async variants of every
> method, where applicable,
> including the method that connects the client to the cluster.
>
> On Fri, Jul 9, 2021 at 9:55 AM Ivan Pavlukhin  wrote:
>
> > Val,
> >
> > > Ignition IS the entry point to Ignite, so I'm not sure I got your point
> > :)
> > > Either way, please feel free to give your suggestions for an
> alternative
> > name if you have any.
> >
> > Well, it is not only about naming but it is also about code
> > organization. Ivan D. already referenced to alternative API styles (I
> > suppose [1] describes the idea).
> >
> > My main points are:
> > 1. Ignite 3 is a great opportunity to make things better.
> > 2. Using (or reusing) confusing names and entities should be avoided.
> >
> > Another rather straightforward startup/bootstrap approach is used in
> > Netty [2]. For Ignite I would spell it like IgniteServer.Bootstrap and
> > IgniteClient.Bootstrap.
> >
> > Also I suppose that thin client API is more important because much
> > more users will use it. I hope that a lot of Community members will
> > share their ideas.
> >
> > [1] https://www.baeldung.com/java-redis-lettuce
> > [2] https://netty.io/4.0/api/io/netty/bootstrap/ServerBootstrap.html
> >
> > 2021-07-09 1:41 GMT+03:00, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> > > Ivan,
> > >
> > > I've seen the link, but I still don't understand what exactly you
> propose
> > > to change in the current API, and what is your concern. Could you
> please
> > > clarify? How you think Ignite API should look like?
> > >
> > > -Val
> > >
> > > On Thu, Jul 8, 2021 at 2:18 PM Ivan Daschinsky 
> > wrote:
> > >
> > >> Val, I have already gave examples -- lettuce, a very performant and
> > >> modern
> > >> redis java client
> > >>
> > >> I can duplicate links again
> > >> https://lettuce.io/core/release/api/io/lettuce/core/RedisClient.html
> > >>
> > >>
> >
> https://lettuce.io/core/release/api/io/lettuce/core/api/StatefulRedisConnection.html
> > >> https://www.baeldung.com/java-redis-lettuce
> > >>
> > >> чт, 8 июл. 2021 г., 23:47 Valentin Kulichenko <
> > >> valentin.kuliche...@gmail.com
> > >> >:
> > >>
> > >> > Ivan,
> > >> >
> > >> > Can you please clarify what you mean by "separate creation of client
> > >> > and
> > >> > connection"? Can you give an example?
> > >> >
> > >> > -Val
> > >> >
> > >> > On Thu, Jul 8, 2021 at 12:53 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > I'm sorry, but why we didn't consider to separate creation of
> Client
> > >> and
> > >> > > connection? Why not to make async variant of connection? See for
> > >> example
> > >> > > [1]
> > >> > > [1] --- https://lettuce.io/core/release/api/index.html
> > >> > >
> > >> > >
> > >> > > чт, 8 июл. 2021 г., 09:50 Pavel Tupitsyn :
> > >> > >
> > >> > > > Val,
> > >> > > >
> > >> > > > So the plan is:
> > >> > > >
> > >> > > > - Remove Ignition#start from the public API
> > >> > > > - Make Ignition a class, no

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

Val, I have already gave examples -- lettuce, a very performant and modern
redis java client

I can duplicate links again
https://lettuce.io/core/release/api/io/lettuce/core/RedisClient.html
https://lettuce.io/core/release/api/io/lettuce/core/api/StatefulRedisConnection.html
https://www.baeldung.com/java-redis-lettuce

чт, 8 июл. 2021 г., 23:47 Valentin Kulichenko :

> Ivan,
>
> Can you please clarify what you mean by "separate creation of client and
> connection"? Can you give an example?
>
> -Val
>
> On Thu, Jul 8, 2021 at 12:53 PM Ivan Daschinsky 
> wrote:
>
> > I'm sorry, but why we didn't consider to separate creation of Client and
> > connection? Why not to make async variant of connection? See for example
> > [1]
> > [1] --- https://lettuce.io/core/release/api/index.html
> >
> >
> > чт, 8 июл. 2021 г., 09:50 Pavel Tupitsyn :
> >
> > > Val,
> > >
> > > So the plan is:
> > >
> > > - Remove Ignition#start from the public API
> > > - Make Ignition a class, not an interface
> > > - Add static Ignition#startClient
> > >
> > > Sounds good?
> > >
> > > On Thu, Jul 8, 2021 at 6:13 AM Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Hi Ivan,
> > > >
> > > > Ignition IS the entry point to Ignite, so I'm not sure I got your
> point
> > > :)
> > > > Where is the contradiction?
> > > >
> > > > Either way, please feel free to give your suggestions for an
> > alternative
> > > > name if you have any.
> > > >
> > > > -Val
> > > >
> > > > On Wed, Jul 7, 2021 at 7:56 PM Ivan Pavlukhina 
> > > > wrote:
> > > >
> > > > > A side note. Actually “Ignition” naming always confused me. I think
> > > about
> > > > > it as some fancy named API entry point for Ignite. Perhaps it is a
> > good
> > > > > moment to revisit naming.
> > > > >
> > > > > > On 8 Jul 2021, at 07:09, Valentin Kulichenko <
> > > > > valentin.kuliche...@gmail.com> wrote:
> > > > > >
> > > > > > Hi Pavel,
> > > > > >
> > > > > > I don't think we will need the pure embedded mode, but we still
> > need
> > > to
> > > > > be
> > > > > > able to access the API from compute and services. That said,
> there
> > > are
> > > > > two
> > > > > > usages of the 'Ignite' API:
> > > > > >
> > > > > >   1. Remote, via the binary protocol.
> > > > > >   2. Local - needed for compute and services. (This is how it
> works
> > > > now.)
> > > > > >
> > > > > > I believe that the API should be the same, and there should be a
> > > > unified
> > > > > > access point. Ignition seems to be a good candidate for this.
> > > > > >
> > > > > > Ignition#start should eventually be removed from the public API.
> It
> > > is
> > > > > > currently there only because we don't have the thin client yet.
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > >> On Wed, Jul 7, 2021 at 5:47 AM Pavel Tupitsyn <
> > ptupit...@apache.org
> > > >
> > > > > wrote:
> > > > > >>
> > > > > >> Igniters,
> > > > > >>
> > > > > >> I have a few questions regarding server node startup and thin
> > > clients.
> > > > > >>
> > > > > >> State of things:
> > > > > >> - Server nodes will be started with 'ignite run' from CLI [1]
> > > > > >> - ignite-api module represents our public API
> > > > > >> - ignite-api has Ignition interface to start server nodes
> > > > > >>
> > > > > >> Questions:
> > > > > >> - What's the idea behind Ignition interface in the public API?
> Are
> > > we
> > > > > going
> > > > > >> to have an "embedded mode" where servers can be started from
> > code? I
> > > > > >> thought this was not planned.
> > > > > >> - How are users supposed to retrieve an instance of the Ignition
> > > > > interface?
> > > > > >> - Are there any plans to start thin clients from Ignition
> > interface,
> > > > or
> > > > > >> should we have a separate way of doing this?
> > > > > >>
> > > > > >>
> > > > > >> [1]
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958
> > > > > >>
> > > > >
> > > >
> > >
> >
>

Re: Ignite 3.0 Ignition API, node startup, and thin client startup

[Ivan Daschinsky]

I'm sorry, but why we didn't consider to separate creation of Client and
connection? Why not to make async variant of connection? See for example [1]
[1] --- https://lettuce.io/core/release/api/index.html


чт, 8 июл. 2021 г., 09:50 Pavel Tupitsyn :

> Val,
>
> So the plan is:
>
> - Remove Ignition#start from the public API
> - Make Ignition a class, not an interface
> - Add static Ignition#startClient
>
> Sounds good?
>
> On Thu, Jul 8, 2021 at 6:13 AM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Hi Ivan,
> >
> > Ignition IS the entry point to Ignite, so I'm not sure I got your point
> :)
> > Where is the contradiction?
> >
> > Either way, please feel free to give your suggestions for an alternative
> > name if you have any.
> >
> > -Val
> >
> > On Wed, Jul 7, 2021 at 7:56 PM Ivan Pavlukhina 
> > wrote:
> >
> > > A side note. Actually “Ignition” naming always confused me. I think
> about
> > > it as some fancy named API entry point for Ignite. Perhaps it is a good
> > > moment to revisit naming.
> > >
> > > > On 8 Jul 2021, at 07:09, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > > >
> > > > Hi Pavel,
> > > >
> > > > I don't think we will need the pure embedded mode, but we still need
> to
> > > be
> > > > able to access the API from compute and services. That said, there
> are
> > > two
> > > > usages of the 'Ignite' API:
> > > >
> > > >   1. Remote, via the binary protocol.
> > > >   2. Local - needed for compute and services. (This is how it works
> > now.)
> > > >
> > > > I believe that the API should be the same, and there should be a
> > unified
> > > > access point. Ignition seems to be a good candidate for this.
> > > >
> > > > Ignition#start should eventually be removed from the public API. It
> is
> > > > currently there only because we don't have the thin client yet.
> > > >
> > > > -Val
> > > >
> > > >> On Wed, Jul 7, 2021 at 5:47 AM Pavel Tupitsyn  >
> > > wrote:
> > > >>
> > > >> Igniters,
> > > >>
> > > >> I have a few questions regarding server node startup and thin
> clients.
> > > >>
> > > >> State of things:
> > > >> - Server nodes will be started with 'ignite run' from CLI [1]
> > > >> - ignite-api module represents our public API
> > > >> - ignite-api has Ignition interface to start server nodes
> > > >>
> > > >> Questions:
> > > >> - What's the idea behind Ignition interface in the public API? Are
> we
> > > going
> > > >> to have an "embedded mode" where servers can be started from code? I
> > > >> thought this was not planned.
> > > >> - How are users supposed to retrieve an instance of the Ignition
> > > interface?
> > > >> - Are there any plans to start thin clients from Ignition interface,
> > or
> > > >> should we have a separate way of doing this?
> > > >>
> > > >>
> > > >> [1]
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158873958
> > > >>
> > >
> >
>

Re: Ignite 3.0 Tuple API: how to check if value is null?

[Ivan Daschinsky]

Function basically returns two values. if value is null, it returns smth
like false, NaN, otherwise ,smth like true, 4.5. Syntax is a bit weird as
for me, but it is better than nothing.


In golang it looks like this:

if isValid, val:= getVal; isValid {

}



ср, 7 июл. 2021 г., 00:28 Valentin Kulichenko :

> So what happens if the value is NULL? Exception?
>
> -Val
>
> On Tue, Jul 6, 2021 at 1:52 PM Ivan Daschinsky 
> wrote:
>
> > > Out of curiosity, what would this code do if the value is NULL? What is
> > the
> > type of the 'weight' variable?
> >
> > float of course.
> > https://www.c-sharpcorner.com/article/out-parameter-in-c-sharp-7/
> >
> >
> > вт, 6 июл. 2021 г., 22:30 Valentin Kulichenko <
> > valentin.kuliche...@gmail.com
> > >:
> >
> > > Pavel,
> > >
> > > Optionals are available in Java and we can use them. This is still
> boxing
> > > though, and I don't know what the performance impact would be. In
> > addition,
> > > optional API is redundant for non-nullable fields. Perhaps, we can
> > provide
> > > both options (e.g. having intValue() and intValueOptional() methods).
> > >
> > > Out of curiosity, what would this code do if the value is NULL? What is
> > the
> > > type of the 'weight' variable?
> > >
> > > if (tuple.TryGetFloatValue("weight", out var weight))
> > > doSomething(weight)
> > >
> > > -Val
> > >
> > > On Tue, Jul 6, 2021 at 2:13 AM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Ah, I see, you meant Optionals family. Yep, it is worth to think
> about.
> > > >
> > > > вт, 6 июл. 2021 г., 10:06 Pavel Tupitsyn :
> > > >
> > > > > Ivan,
> > > > >
> > > > > Nothing wrong except for performance concerns.
> > > > > The following code looks up the column by name twice:
> > > > >
> > > > > if (!tuple.isNull("weight"))
> > > > >doSomething(tuple.floatValue("weight"))
> > > > >
> > > > > Whereas in other languages you could do it in one shot:
> > > > >
> > > > > if (tuple.TryGetFloatValue("weight", out var weight))
> > > > > doSomething(weight)
> > > > >
> > > > > or Option weight = tuple.floatValue("weight") and so on.
> > > > >
> > > > > On Tue, Jul 6, 2021 at 9:58 AM Ivan Daschinsky <
> ivanda...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Sorry, but what is wrong with simple method isNull()
> > > > > >
> > > > > > вт, 6 июл. 2021 г., 09:55 Pavel Tupitsyn :
> > > > > >
> > > > > > > Val,
> > > > > > >
> > > > > > > > I don't think there is a significantly better way
> > > > > > > > of doing this in Java.
> > > > > > >
> > > > > > > Yep looks like there is no way to return two values without
> > boxing.
> > > > > > > No ref, no out, no value types.
> > > > > > >
> > > > > > > > Schema already provides this information, doesn't it?
> > > > > > >
> > > > > > > It does, though we don't have an agreement on how to expose
> this
> > on
> > > > > > public
> > > > > > > API yet,
> > > > > > > or do we?
> > > > > > >
> > > > > > > On Tue, Jul 6, 2021 at 12:44 AM Valentin Kulichenko <
> > > > > > > valentin.kuliche...@gmail.com> wrote:
> > > > > > >
> > > > > > > > Pavel,
> > > > > > > >
> > > > > > > > That's a good point, but I don't think there is a
> significantly
> > > > > better
> > > > > > > way
> > > > > > > > of doing this in Java.
> > > > > > > >
> > > > > > > > There should be a way to check if a field is nullable or not
> > > > though.
> > > > > > > Schema
> > > > > > > > already provides this information, doesn't it?
> > > > > > > >
> > > > > > > > -Val
> > > > > > > >
> > > > > > > > On Mon, Jul 5, 2021 at 11:03 AM Pavel Tupitsyn <
> > > > ptupit...@apache.org
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Igniters,
> > > > > > > > >
> > > > > > > > > Looks like Tuple API has no efficient way to tell if a
> value
> > > for
> > > > a
> > > > > > > > nullable
> > > > > > > > > column of primitive type is null.
> > > > > > > > >
> > > > > > > > > - Tuple#intValue() will return 0 when the actual value is
> > null
> > > =>
> > > > > not
> > > > > > > > clear
> > > > > > > > > if 0 is 0 or null.
> > > > > > > > > - Tuple#value() works, but is more expensive due to boxing
> > and
> > > > type
> > > > > > > > lookup.
> > > > > > > > >
> > > > > > > > > Any ideas on how to improve this?
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Ignite 3.0 Tuple API: how to check if value is null?

[Ivan Daschinsky]

> Out of curiosity, what would this code do if the value is NULL? What is
the
type of the 'weight' variable?

float of course.
https://www.c-sharpcorner.com/article/out-parameter-in-c-sharp-7/


вт, 6 июл. 2021 г., 22:30 Valentin Kulichenko :

> Pavel,
>
> Optionals are available in Java and we can use them. This is still boxing
> though, and I don't know what the performance impact would be. In addition,
> optional API is redundant for non-nullable fields. Perhaps, we can provide
> both options (e.g. having intValue() and intValueOptional() methods).
>
> Out of curiosity, what would this code do if the value is NULL? What is the
> type of the 'weight' variable?
>
> if (tuple.TryGetFloatValue("weight", out var weight))
> doSomething(weight)
>
> -Val
>
> On Tue, Jul 6, 2021 at 2:13 AM Ivan Daschinsky 
> wrote:
>
> > Ah, I see, you meant Optionals family. Yep, it is worth to think about.
> >
> > вт, 6 июл. 2021 г., 10:06 Pavel Tupitsyn :
> >
> > > Ivan,
> > >
> > > Nothing wrong except for performance concerns.
> > > The following code looks up the column by name twice:
> > >
> > > if (!tuple.isNull("weight"))
> > >doSomething(tuple.floatValue("weight"))
> > >
> > > Whereas in other languages you could do it in one shot:
> > >
> > > if (tuple.TryGetFloatValue("weight", out var weight))
> > > doSomething(weight)
> > >
> > > or Option weight = tuple.floatValue("weight") and so on.
> > >
> > > On Tue, Jul 6, 2021 at 9:58 AM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Sorry, but what is wrong with simple method isNull()
> > > >
> > > > вт, 6 июл. 2021 г., 09:55 Pavel Tupitsyn :
> > > >
> > > > > Val,
> > > > >
> > > > > > I don't think there is a significantly better way
> > > > > > of doing this in Java.
> > > > >
> > > > > Yep looks like there is no way to return two values without boxing.
> > > > > No ref, no out, no value types.
> > > > >
> > > > > > Schema already provides this information, doesn't it?
> > > > >
> > > > > It does, though we don't have an agreement on how to expose this on
> > > > public
> > > > > API yet,
> > > > > or do we?
> > > > >
> > > > > On Tue, Jul 6, 2021 at 12:44 AM Valentin Kulichenko <
> > > > > valentin.kuliche...@gmail.com> wrote:
> > > > >
> > > > > > Pavel,
> > > > > >
> > > > > > That's a good point, but I don't think there is a significantly
> > > better
> > > > > way
> > > > > > of doing this in Java.
> > > > > >
> > > > > > There should be a way to check if a field is nullable or not
> > though.
> > > > > Schema
> > > > > > already provides this information, doesn't it?
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > > On Mon, Jul 5, 2021 at 11:03 AM Pavel Tupitsyn <
> > ptupit...@apache.org
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > Looks like Tuple API has no efficient way to tell if a value
> for
> > a
> > > > > > nullable
> > > > > > > column of primitive type is null.
> > > > > > >
> > > > > > > - Tuple#intValue() will return 0 when the actual value is null
> =>
> > > not
> > > > > > clear
> > > > > > > if 0 is 0 or null.
> > > > > > > - Tuple#value() works, but is more expensive due to boxing and
> > type
> > > > > > lookup.
> > > > > > >
> > > > > > > Any ideas on how to improve this?
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

I suppose, that general idea is great. Some details are missing, but I
suppose during implementation of clients we will add more details and
redefine some parts.

вт, 6 июл. 2021 г., 09:59 Pavel Tupitsyn :

> Ivan, Val, and others - are there any open objections or questions?
> Can we accept the proposal?
>
> On Mon, Jul 5, 2021 at 1:28 PM Pavel Tupitsyn 
> wrote:
>
> > Igniters,
> >
> > I've updated the IEP to support "Live Schema" [1] from IEP-54.
> > Some operations now have schemaless variants, where tuples are serialized
> > as maps (String -> val).
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach#IEP54:SchemafirstApproach-Dynamicschemaexpansion(Live-schema)
> >
> > On Thu, Jul 1, 2021 at 8:31 PM Ivan Daschinsky 
> > wrote:
> >
> >> Val, my understanding about it was exactly the same as yours, but,
> again,
> >> I
> >> heard a different opinion.
> >>
> >> But nevertheless, binary protocol should not be about objects, records
> aka
> >> tuples are the best varii, simple and powerful.
> >>
> >> As for me, I want to take part in implementing python and golang thin
> >> clients for ignite 3, so consider my remarks using this info. I am not
> >> just
> >> a rude critic,
> >> I am just interested in consice and universal binary prorocol
> >> чт, 1 июл. 2021 г., 20:23 Valentin Kulichenko <
> >> valentin.kuliche...@gmail.com
> >> >:
> >>
> >> > Ivan,
> >> >
> >> > KV view does work over the tuples. Nested objects and arbitrary
> >> structures
> >> > can be stored as blobs. So if you need a basic KV cache, you can
> always
> >> > create a table with two blob fields - one for key and one for value -
> >> and
> >> > store anything there.
> >> >
> >> > -Val
> >> >
> >> > On Thu, Jul 1, 2021 at 9:55 AM Ivan Daschinsky 
> >> > wrote:
> >> >
> >> > > Val, am I right, that kv view over the tuples is just simple mapping
> >> from
> >> > > POJO to tuple? No collections, no nested objects? I have discussed
> >> this
> >> > in
> >> > > private with Igor and Pavel and they told me different info.
> >> > >
> >> > > чт, 1 июл. 2021 г., 19:43 Valentin Kulichenko <
> >> > > valentin.kuliche...@gmail.com
> >> > > >:
> >> > >
> >> > > > Ivan,
> >> > > >
> >> > > > I was answering your question about the KV API. The API I provided
> >> has
> >> > > been
> >> > > > discussed and agreed upon. One of the goals of the protocol is to
> >> > > implement
> >> > > > this API, so it should give you a clear idea of what we're looking
> >> for.
> >> > > >
> >> > > > Of course, I agree with you that the protocol should be simple and
> >> > > flexible
> >> > > > enough to allow other implementations for different languages and
> >> > > > platforms.
> >> > > >
> >> > > > -Val
> >> > > >
> >> > > > On Thu, Jul 1, 2021 at 9:38 AM Ivan Daschinsky <
> ivanda...@gmail.com
> >> >
> >> > > > wrote:
> >> > > >
> >> > > > > Andrey, yep, you are right. This was just a quick idea. As for
> >> me, I
> >> > > just
> >> > > > > don't want to repeat the same problem with compactFooter in thin
> >> > client
> >> > > > api
> >> > > > > of ignite 2.x.
> >> > > > >
> >> > > > > чт, 1 июл. 2021 г., 19:22 Andrey Mashenkov <
> >> > andrey.mashen...@gmail.com
> >> > > >:
> >> > > > >
> >> > > > > > >
> >> > > > > > > I suppose that we should describe this more verbose and
> >> > explicit. I
> >> > > > > > > nevertheless suggest to also consider writing values this
> way:
> >> > > > > > > - arr of fields names (if name is missed, corresponding
> field
> >> is
> >> > > nil)
> >> > > > > > > - arr of rows (row as array, length equal to fields array)
> >> > > > > >
> >> 

Re: Ignite 3.0 Tuple API: how to check if value is null?

[Ivan Daschinsky]

Ah, I see, you meant Optionals family. Yep, it is worth to think about.

вт, 6 июл. 2021 г., 10:06 Pavel Tupitsyn :

> Ivan,
>
> Nothing wrong except for performance concerns.
> The following code looks up the column by name twice:
>
> if (!tuple.isNull("weight"))
>doSomething(tuple.floatValue("weight"))
>
> Whereas in other languages you could do it in one shot:
>
> if (tuple.TryGetFloatValue("weight", out var weight))
> doSomething(weight)
>
> or Option weight = tuple.floatValue("weight") and so on.
>
> On Tue, Jul 6, 2021 at 9:58 AM Ivan Daschinsky 
> wrote:
>
> > Sorry, but what is wrong with simple method isNull()
> >
> > вт, 6 июл. 2021 г., 09:55 Pavel Tupitsyn :
> >
> > > Val,
> > >
> > > > I don't think there is a significantly better way
> > > > of doing this in Java.
> > >
> > > Yep looks like there is no way to return two values without boxing.
> > > No ref, no out, no value types.
> > >
> > > > Schema already provides this information, doesn't it?
> > >
> > > It does, though we don't have an agreement on how to expose this on
> > public
> > > API yet,
> > > or do we?
> > >
> > > On Tue, Jul 6, 2021 at 12:44 AM Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Pavel,
> > > >
> > > > That's a good point, but I don't think there is a significantly
> better
> > > way
> > > > of doing this in Java.
> > > >
> > > > There should be a way to check if a field is nullable or not though.
> > > Schema
> > > > already provides this information, doesn't it?
> > > >
> > > > -Val
> > > >
> > > > On Mon, Jul 5, 2021 at 11:03 AM Pavel Tupitsyn  >
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > Looks like Tuple API has no efficient way to tell if a value for a
> > > > nullable
> > > > > column of primitive type is null.
> > > > >
> > > > > - Tuple#intValue() will return 0 when the actual value is null =>
> not
> > > > clear
> > > > > if 0 is 0 or null.
> > > > > - Tuple#value() works, but is more expensive due to boxing and type
> > > > lookup.
> > > > >
> > > > > Any ideas on how to improve this?
> > > > >
> > > >
> > >
> >
>

Re: Ignite 3.0 Tuple API: how to check if value is null?

[Ivan Daschinsky]

Sorry, but what is wrong with simple method isNull()

вт, 6 июл. 2021 г., 09:55 Pavel Tupitsyn :

> Val,
>
> > I don't think there is a significantly better way
> > of doing this in Java.
>
> Yep looks like there is no way to return two values without boxing.
> No ref, no out, no value types.
>
> > Schema already provides this information, doesn't it?
>
> It does, though we don't have an agreement on how to expose this on public
> API yet,
> or do we?
>
> On Tue, Jul 6, 2021 at 12:44 AM Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Pavel,
> >
> > That's a good point, but I don't think there is a significantly better
> way
> > of doing this in Java.
> >
> > There should be a way to check if a field is nullable or not though.
> Schema
> > already provides this information, doesn't it?
> >
> > -Val
> >
> > On Mon, Jul 5, 2021 at 11:03 AM Pavel Tupitsyn 
> > wrote:
> >
> > > Igniters,
> > >
> > > Looks like Tuple API has no efficient way to tell if a value for a
> > nullable
> > > column of primitive type is null.
> > >
> > > - Tuple#intValue() will return 0 when the actual value is null => not
> > clear
> > > if 0 is 0 or null.
> > > - Tuple#value() works, but is more expensive due to boxing and type
> > lookup.
> > >
> > > Any ideas on how to improve this?
> > >
> >
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

Val, my understanding about it was exactly the same as yours, but, again, I
heard a different opinion.

But nevertheless, binary protocol should not be about objects, records aka
tuples are the best varii, simple and powerful.

As for me, I want to take part in implementing python and golang thin
clients for ignite 3, so consider my remarks using this info. I am not just
a rude critic,
I am just interested in consice and universal binary prorocol
чт, 1 июл. 2021 г., 20:23 Valentin Kulichenko :

> Ivan,
>
> KV view does work over the tuples. Nested objects and arbitrary structures
> can be stored as blobs. So if you need a basic KV cache, you can always
> create a table with two blob fields - one for key and one for value - and
> store anything there.
>
> -Val
>
> On Thu, Jul 1, 2021 at 9:55 AM Ivan Daschinsky 
> wrote:
>
> > Val, am I right, that kv view over the tuples is just simple mapping from
> > POJO to tuple? No collections, no nested objects? I have discussed this
> in
> > private with Igor and Pavel and they told me different info.
> >
> > чт, 1 июл. 2021 г., 19:43 Valentin Kulichenko <
> > valentin.kuliche...@gmail.com
> > >:
> >
> > > Ivan,
> > >
> > > I was answering your question about the KV API. The API I provided has
> > been
> > > discussed and agreed upon. One of the goals of the protocol is to
> > implement
> > > this API, so it should give you a clear idea of what we're looking for.
> > >
> > > Of course, I agree with you that the protocol should be simple and
> > flexible
> > > enough to allow other implementations for different languages and
> > > platforms.
> > >
> > > -Val
> > >
> > > On Thu, Jul 1, 2021 at 9:38 AM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Andrey, yep, you are right. This was just a quick idea. As for me, I
> > just
> > > > don't want to repeat the same problem with compactFooter in thin
> client
> > > api
> > > > of ignite 2.x.
> > > >
> > > > чт, 1 июл. 2021 г., 19:22 Andrey Mashenkov <
> andrey.mashen...@gmail.com
> > >:
> > > >
> > > > > >
> > > > > > I suppose that we should describe this more verbose and
> explicit. I
> > > > > > nevertheless suggest to also consider writing values this way:
> > > > > > - arr of fields names (if name is missed, corresponding field is
> > nil)
> > > > > > - arr of rows (row as array, length equal to fields array)
> > > > >
> > > > >
> > > > > Ivan,
> > > > > I think GET and PUT operation parameters should be consistent.
> > > > > With PUT operation this way may be tricky.
> > > > >
> > > > > SQL INSERT operation (which is similar PUT operation) semantic
> allows
> > > > > skipping columns that have a default value.
> > > > > Assume we have smth like this:
> > > > >
> > > > > CREATE TABLE t1 (
> > > > >'id' INT;
> > > > >'colname' VARCHAR DEFAULT "abc";
> > > > > )
> > > > > INSERT INTO t1 VALUES(1)
> > > > >
> > > > > Actually, this will add a row (1, "abc")
> > > > >
> > > > > Your suggestion related to missed fields will not work this way as
> it
> > > is
> > > > > impossible to distinct
> > > > > case with 'null' value from the case with a default value.
> > > > >
> > > > >
> > > > > On Thu, Jul 1, 2021 at 5:51 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > > Here is the description of TUPLE_GET_ALL:
> > > > > > - UUID: table ID
> > > > > > - int: schema ID
> > > > > > - arr of arr: array of rows with values for all columns in given
> > > schema
> > > > > >
> > > > > > I suppose that we should describe this more verbose and
> explicit. I
> > > > > > nevertheless suggest to also consider writing values this way:
> > > > > > - arr of fields names (if name is missed, corresponding field is
> > nil)
> > > > > > - arr of rows (row as array, length equal to fields array)
> > > > > >
> > > > > > It is quite simple and if we use str8 (it is more than enough fo

Re: Setting IGNITE_TO_STRING_INCLUDE_SENSITIVE=false prevents VM Arguments output

[Ivan Daschinsky]

Ok, this can be excluded using blocklist-jvm-params.properties or just by
providing and extra arg to annotation, as I have just suggested

чт, 1 июл. 2021 г., 19:51 Valentin Kulichenko :

> Ivan,
>
> IP addresses (e.g. IGNITE_TCP_DISCOVERY_ADDRESSES) and file paths
> (e.g. IGNITE_CONFIG_URL) are often considered sensitive information. Data
> related to authentication (e.g. IGNITE_SSH_USER_NAME) is very likely to be
> sensitive.
>
> Once again - I would exclude any property that can contain user-specific
> information. Only our internal settings (stuff
> like IGNITE_SQL_MERGE_TABLE_MAX_SIZE) are OK to appear in the logs.
>
> -Val
>
> On Thu, Jul 1, 2021 at 9:47 AM Ivan Daschinsky 
> wrote:
>
> > We can add add an extra param in annotation, that blocks param to be
> > printed, just set it to false by default and block it wheb set to true
> >
> > чт, 1 июл. 2021 г., 19:45 Atri Sharma :
> >
> > > What if we allowed a blocklist of parameters that are never printed?
> > >
> > > On Thu, 1 Jul 2021, 22:06 Valentin Kulichenko, <
> > > valentin.kuliche...@gmail.com> wrote:
> > >
> > > > Not all of them are OK to be printed out. At the very least, we
> should
> > > have
> > > > a mechanism to exclude some of them. I would still go with opt-in
> > rather
> > > > than opt-out though, but I guess that is up to a discussion.
> > > >
> > > > -Val
> > > >
> > > > On Thu, Jul 1, 2021 at 9:29 AM Ivan Daschinsky 
> > > > wrote:
> > > >
> > > > > This is security through obscurity, an obvious and a well-known
> anti
> > > > > pattern. I suppose that printing jvm options, that is registered by
> > > > > @IgniteSystemProperty annotation is an ideal variant
> > > > >
> > > > > чт, 1 июл. 2021 г., 19:25 Valentin Kulichenko <
> > > > > valentin.kuliche...@gmail.com
> > > > > >:
> > > > >
> > > > > > Folks,
> > > > > >
> > > > > > *Anything* that a user provides to the system can potentially be
> > > > > considered
> > > > > > sensitive information. This includes the VM arguments. We can't
> > > predict
> > > > > > what exactly one can put there, so let's not make assumptions.
> > > > > >
> > > > > > When dealing with security, we should be as conservative as
> > possible.
> > > > > That
> > > > > > said, I do not even agree with the pattern approach - there might
> > be
> > > a
> > > > > > user's system property named IGNITE_xxx. It is also possible for
> > our
> > > > > > internal properties to contain sensitive information (not all of
> > them
> > > > are
> > > > > > boolean flags).
> > > > > >
> > > > > > The only option I see is to print out specific properties for
> which
> > > we
> > > > > > agree that they are safe. For example, we can introduce an
> > annotation
> > > > > that
> > > > > > would mark "safe" properties in IgniteSystemProperties; we will
> > then
> > > > > print
> > > > > > out only those that are marked with the annotation.
> > > > > >
> > > > > > -Val
> > > > > >
> > > > > >
> > > > > > On Thu, Jul 1, 2021 at 7:07 AM Вячеслав Коптилин <
> > > > > slava.kopti...@gmail.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hello Ivan,
> > > > > > >
> > > > > > > > At least, we could just hide params that match a specific
> > pattern
> > > > > > > Yes, we can filter out all vm options that do not relate to
> > Ignite,
> > > > for
> > > > > > > instance.
> > > > > > >
> > > > > > > > Ilya, go ahead, file ticket and prepare a PR.
> > > > > > > Please do not rush. Let's listen to other community members.
> This
> > > > > > question
> > > > > > > is about security and it should not be discussed in a hurry
> (even
> > > > > though
> > > > > > it
> > > > > > > looks like an obvious thing).
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

Val, am I right, that kv view over the tuples is just simple mapping from
POJO to tuple? No collections, no nested objects? I have discussed this in
private with Igor and Pavel and they told me different info.

чт, 1 июл. 2021 г., 19:43 Valentin Kulichenko :

> Ivan,
>
> I was answering your question about the KV API. The API I provided has been
> discussed and agreed upon. One of the goals of the protocol is to implement
> this API, so it should give you a clear idea of what we're looking for.
>
> Of course, I agree with you that the protocol should be simple and flexible
> enough to allow other implementations for different languages and
> platforms.
>
> -Val
>
> On Thu, Jul 1, 2021 at 9:38 AM Ivan Daschinsky 
> wrote:
>
> > Andrey, yep, you are right. This was just a quick idea. As for me, I just
> > don't want to repeat the same problem with compactFooter in thin client
> api
> > of ignite 2.x.
> >
> > чт, 1 июл. 2021 г., 19:22 Andrey Mashenkov :
> >
> > > >
> > > > I suppose that we should describe this more verbose and explicit. I
> > > > nevertheless suggest to also consider writing values this way:
> > > > - arr of fields names (if name is missed, corresponding field is nil)
> > > > - arr of rows (row as array, length equal to fields array)
> > >
> > >
> > > Ivan,
> > > I think GET and PUT operation parameters should be consistent.
> > > With PUT operation this way may be tricky.
> > >
> > > SQL INSERT operation (which is similar PUT operation) semantic allows
> > > skipping columns that have a default value.
> > > Assume we have smth like this:
> > >
> > > CREATE TABLE t1 (
> > >'id' INT;
> > >'colname' VARCHAR DEFAULT "abc";
> > > )
> > > INSERT INTO t1 VALUES(1)
> > >
> > > Actually, this will add a row (1, "abc")
> > >
> > > Your suggestion related to missed fields will not work this way as it
> is
> > > impossible to distinct
> > > case with 'null' value from the case with a default value.
> > >
> > >
> > > On Thu, Jul 1, 2021 at 5:51 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > > Here is the description of TUPLE_GET_ALL:
> > > > - UUID: table ID
> > > > - int: schema ID
> > > > - arr of arr: array of rows with values for all columns in given
> schema
> > > >
> > > > I suppose that we should describe this more verbose and explicit. I
> > > > nevertheless suggest to also consider writing values this way:
> > > > - arr of fields names (if name is missed, corresponding field is nil)
> > > > - arr of rows (row as array, length equal to fields array)
> > > >
> > > > It is quite simple and if we use str8 (it is more than enough for any
> > > utf-8
> > > > reasonable field name), overhead will be negligible, but realization
> > of a
> > > > client will be way simpler
> > > >
> > > > чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn :
> > > >
> > > > > > No it isn't, I have carefully read code and IEP, in your code you
> > > write
> > > > > > schema id in each tuple.
> > > > >
> > > > > There is no code for batch operations yet.
> > > > >
> > > > > Here is the description of TUPLE_GET_ALL:
> > > > > - UUID: table ID
> > > > > - int: schema ID
> > > > > - arr of arr: array of rows with values for all columns in given
> > schema
> > > > > (nil when value is missing for a column)
> > > > >
> > > > > As you can see, schema ID is written once for all rows.
> > > > > A row is just a set of values according to the schema.
> > > > >
> > > > >
> > > > > > Also, my biggest concern -- extra serde step. I suppose we should
> > > pass
> > > > > > bytearray to internal api, and use msgpack throughout all wire
> > > > protocols,
> > > > > > as tarantool does.
> > > > >
> > > > > I agree. But this was decided before in IEP-54, and is out of scope
> > for
> > > > > current IEP.
> > > > > Would you like to start a separate thread to discuss this? Or I can
> > do
> > > > this
> > > > > a bit later.
> > > > >
> > > > > On Thu, Ju

Re: Setting IGNITE_TO_STRING_INCLUDE_SENSITIVE=false prevents VM Arguments output

[Ivan Daschinsky]

We can add add an extra param in annotation, that blocks param to be
printed, just set it to false by default and block it wheb set to true

чт, 1 июл. 2021 г., 19:45 Atri Sharma :

> What if we allowed a blocklist of parameters that are never printed?
>
> On Thu, 1 Jul 2021, 22:06 Valentin Kulichenko, <
> valentin.kuliche...@gmail.com> wrote:
>
> > Not all of them are OK to be printed out. At the very least, we should
> have
> > a mechanism to exclude some of them. I would still go with opt-in rather
> > than opt-out though, but I guess that is up to a discussion.
> >
> > -Val
> >
> > On Thu, Jul 1, 2021 at 9:29 AM Ivan Daschinsky 
> > wrote:
> >
> > > This is security through obscurity, an obvious and a well-known anti
> > > pattern. I suppose that printing jvm options, that is registered by
> > > @IgniteSystemProperty annotation is an ideal variant
> > >
> > > чт, 1 июл. 2021 г., 19:25 Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com
> > > >:
> > >
> > > > Folks,
> > > >
> > > > *Anything* that a user provides to the system can potentially be
> > > considered
> > > > sensitive information. This includes the VM arguments. We can't
> predict
> > > > what exactly one can put there, so let's not make assumptions.
> > > >
> > > > When dealing with security, we should be as conservative as possible.
> > > That
> > > > said, I do not even agree with the pattern approach - there might be
> a
> > > > user's system property named IGNITE_xxx. It is also possible for our
> > > > internal properties to contain sensitive information (not all of them
> > are
> > > > boolean flags).
> > > >
> > > > The only option I see is to print out specific properties for which
> we
> > > > agree that they are safe. For example, we can introduce an annotation
> > > that
> > > > would mark "safe" properties in IgniteSystemProperties; we will then
> > > print
> > > > out only those that are marked with the annotation.
> > > >
> > > > -Val
> > > >
> > > >
> > > > On Thu, Jul 1, 2021 at 7:07 AM Вячеслав Коптилин <
> > > slava.kopti...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hello Ivan,
> > > > >
> > > > > > At least, we could just hide params that match a specific pattern
> > > > > Yes, we can filter out all vm options that do not relate to Ignite,
> > for
> > > > > instance.
> > > > >
> > > > > > Ilya, go ahead, file ticket and prepare a PR.
> > > > > Please do not rush. Let's listen to other community members. This
> > > > question
> > > > > is about security and it should not be discussed in a hurry (even
> > > though
> > > > it
> > > > > looks like an obvious thing).
> > > > >
> > > > > Thanks,
> > > > > S.
> > > > >
> > > > > чт, 1 июл. 2021 г. в 16:55, Ivan Daschinsky :
> > > > >
> > > > > > I suppose, that all normal users should not suffer from this
> > > > > restrictions.
> > > > > > Nobody will pass password using jvm options. It is absolutely
> > insane,
> > > > > > normal users pass passwords using environment variables.
> > > > > >
> > > > > > At least, we could just hide params that match specific pattern
> > > > > >
> > > > > > Ilya, go ahead, file ticket and prepare a PR.
> > > > > >
> > > > > > чт, 1 июл. 2021 г., 16:45 Вячеслав Коптилин <
> > > slava.kopti...@gmail.com
> > > > >:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > Unfortunately, the user can pass its own system properties via
> > JVM
> > > > > > options
> > > > > > > as follows: -DMY_SECRET_PASSWORD=123
> > > > > > > It does not seem, this approach is the best one. However, the
> > user
> > > > > should
> > > > > > > have a "kostyl" in order to hide these properties and values in
> > the
> > > > log
> > > > > > > file, IMHO.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > S.
> > > > > > >
> > > > > > > ср, 30 июн. 2021 г. в 22:52, Shishkov Ilya <
> > shishkovi...@gmail.com
> > > >:
> > > > > > >
> > > > > > > > Hi Igniters,
> > > > > > > >
> > > > > > > > This feature [1, 2] prevents logging of the VM arguments when
> > > > > > > > IGNITE_TO_STRING_INCLUDE_SENSITIVE option is set to false.
> Till
> > > > now,
> > > > > > > method
> > > > > > > > IgniteKernal#ackVmArguments remains mostly the same [3].
> > > > > > > >
> > > > > > > > Is this behaviour actual now? Often, we should be able to get
> > > from
> > > > > logs
> > > > > > > the
> > > > > > > > actual VM options used at startup even if output of sensitive
> > > data
> > > > is
> > > > > > > > restricted.
> > > > > > > >
> > > > > > > > 1. https://issues.apache.org/jira/browse/IGNITE-4991
> > > > > > > > 2.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/ignite/pull/2428/commits/4f90b6fd77bd23fa818620f0757b792ba388ef93
> > > > > > > > 3.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java#L3002
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Setting IGNITE_TO_STRING_INCLUDE_SENSITIVE=false prevents VM Arguments output

[Ivan Daschinsky]

> Not all of them are OK to be printed out
Could you give an example please? As for me, all of them are pretty harmless

чт, 1 июл. 2021 г., 19:36 Valentin Kulichenko :

> Not all of them are OK to be printed out. At the very least, we should have
> a mechanism to exclude some of them. I would still go with opt-in rather
> than opt-out though, but I guess that is up to a discussion.
>
> -Val
>
> On Thu, Jul 1, 2021 at 9:29 AM Ivan Daschinsky 
> wrote:
>
> > This is security through obscurity, an obvious and a well-known anti
> > pattern. I suppose that printing jvm options, that is registered by
> > @IgniteSystemProperty annotation is an ideal variant
> >
> > чт, 1 июл. 2021 г., 19:25 Valentin Kulichenko <
> > valentin.kuliche...@gmail.com
> > >:
> >
> > > Folks,
> > >
> > > *Anything* that a user provides to the system can potentially be
> > considered
> > > sensitive information. This includes the VM arguments. We can't predict
> > > what exactly one can put there, so let's not make assumptions.
> > >
> > > When dealing with security, we should be as conservative as possible.
> > That
> > > said, I do not even agree with the pattern approach - there might be a
> > > user's system property named IGNITE_xxx. It is also possible for our
> > > internal properties to contain sensitive information (not all of them
> are
> > > boolean flags).
> > >
> > > The only option I see is to print out specific properties for which we
> > > agree that they are safe. For example, we can introduce an annotation
> > that
> > > would mark "safe" properties in IgniteSystemProperties; we will then
> > print
> > > out only those that are marked with the annotation.
> > >
> > > -Val
> > >
> > >
> > > On Thu, Jul 1, 2021 at 7:07 AM Вячеслав Коптилин <
> > slava.kopti...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hello Ivan,
> > > >
> > > > > At least, we could just hide params that match a specific pattern
> > > > Yes, we can filter out all vm options that do not relate to Ignite,
> for
> > > > instance.
> > > >
> > > > > Ilya, go ahead, file ticket and prepare a PR.
> > > > Please do not rush. Let's listen to other community members. This
> > > question
> > > > is about security and it should not be discussed in a hurry (even
> > though
> > > it
> > > > looks like an obvious thing).
> > > >
> > > > Thanks,
> > > > S.
> > > >
> > > > чт, 1 июл. 2021 г. в 16:55, Ivan Daschinsky :
> > > >
> > > > > I suppose, that all normal users should not suffer from this
> > > > restrictions.
> > > > > Nobody will pass password using jvm options. It is absolutely
> insane,
> > > > > normal users pass passwords using environment variables.
> > > > >
> > > > > At least, we could just hide params that match specific pattern
> > > > >
> > > > > Ilya, go ahead, file ticket and prepare a PR.
> > > > >
> > > > > чт, 1 июл. 2021 г., 16:45 Вячеслав Коптилин <
> > slava.kopti...@gmail.com
> > > >:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > Unfortunately, the user can pass its own system properties via
> JVM
> > > > > options
> > > > > > as follows: -DMY_SECRET_PASSWORD=123
> > > > > > It does not seem, this approach is the best one. However, the
> user
> > > > should
> > > > > > have a "kostyl" in order to hide these properties and values in
> the
> > > log
> > > > > > file, IMHO.
> > > > > >
> > > > > > Thanks,
> > > > > > S.
> > > > > >
> > > > > > ср, 30 июн. 2021 г. в 22:52, Shishkov Ilya <
> shishkovi...@gmail.com
> > >:
> > > > > >
> > > > > > > Hi Igniters,
> > > > > > >
> > > > > > > This feature [1, 2] prevents logging of the VM arguments when
> > > > > > > IGNITE_TO_STRING_INCLUDE_SENSITIVE option is set to false. Till
> > > now,
> > > > > > method
> > > > > > > IgniteKernal#ackVmArguments remains mostly the same [3].
> > > > > > >
> > > > > > > Is this behaviour actual now? Often, we should be able to get
> > from
> > > > logs
> > > > > > the
> > > > > > > actual VM options used at startup even if output of sensitive
> > data
> > > is
> > > > > > > restricted.
> > > > > > >
> > > > > > > 1. https://issues.apache.org/jira/browse/IGNITE-4991
> > > > > > > 2.
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/ignite/pull/2428/commits/4f90b6fd77bd23fa818620f0757b792ba388ef93
> > > > > > > 3.
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java#L3002
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

Andrey, yep, you are right. This was just a quick idea. As for me, I just
don't want to repeat the same problem with compactFooter in thin client api
of ignite 2.x.

чт, 1 июл. 2021 г., 19:22 Andrey Mashenkov :

> >
> > I suppose that we should describe this more verbose and explicit. I
> > nevertheless suggest to also consider writing values this way:
> > - arr of fields names (if name is missed, corresponding field is nil)
> > - arr of rows (row as array, length equal to fields array)
>
>
> Ivan,
> I think GET and PUT operation parameters should be consistent.
> With PUT operation this way may be tricky.
>
> SQL INSERT operation (which is similar PUT operation) semantic allows
> skipping columns that have a default value.
> Assume we have smth like this:
>
> CREATE TABLE t1 (
>'id' INT;
>'colname' VARCHAR DEFAULT "abc";
> )
> INSERT INTO t1 VALUES(1)
>
> Actually, this will add a row (1, "abc")
>
> Your suggestion related to missed fields will not work this way as it is
> impossible to distinct
> case with 'null' value from the case with a default value.
>
>
> On Thu, Jul 1, 2021 at 5:51 PM Ivan Daschinsky 
> wrote:
>
> > > Here is the description of TUPLE_GET_ALL:
> > - UUID: table ID
> > - int: schema ID
> > - arr of arr: array of rows with values for all columns in given schema
> >
> > I suppose that we should describe this more verbose and explicit. I
> > nevertheless suggest to also consider writing values this way:
> > - arr of fields names (if name is missed, corresponding field is nil)
> > - arr of rows (row as array, length equal to fields array)
> >
> > It is quite simple and if we use str8 (it is more than enough for any
> utf-8
> > reasonable field name), overhead will be negligible, but realization of a
> > client will be way simpler
> >
> > чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn :
> >
> > > > No it isn't, I have carefully read code and IEP, in your code you
> write
> > > > schema id in each tuple.
> > >
> > > There is no code for batch operations yet.
> > >
> > > Here is the description of TUPLE_GET_ALL:
> > > - UUID: table ID
> > > - int: schema ID
> > > - arr of arr: array of rows with values for all columns in given schema
> > > (nil when value is missing for a column)
> > >
> > > As you can see, schema ID is written once for all rows.
> > > A row is just a set of values according to the schema.
> > >
> > >
> > > > Also, my biggest concern -- extra serde step. I suppose we should
> pass
> > > > bytearray to internal api, and use msgpack throughout all wire
> > protocols,
> > > > as tarantool does.
> > >
> > > I agree. But this was decided before in IEP-54, and is out of scope for
> > > current IEP.
> > > Would you like to start a separate thread to discuss this? Or I can do
> > this
> > > a bit later.
> > >
> > > On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > > This is described in all operations that include multiple tuples.
> > > > No it isn't, I have carefully read code and IEP, in your code you
> write
> > > > schema id in each tuple.
> > > >
> > > > Also, my biggest concern -- extra serde step. I suppose we should
> pass
> > > > bytearray to internal api, and use msgpack throughout all wire
> > protocols,
> > > > as tarantool does.
> > > >
> > > > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn :
> > > >
> > > > > Ivan,
> > > > >
> > > > > >  that there is not neccesary to write schema versions in each row
> > > > > > in collectionof tuples
> > > > >
> > > > > This is described in all operations that include multiple tuples.
> > > > >
> > > > >
> > > > > > it is not clear from your code (probably
> > > > > > mistake?) how differ key tuples and value tuples from each other
> > > > >
> > > > > Key tuples include only key columns. Key columns come first in the
> > > > schema.
> > > > > Value tuples include all columns, key and value. Added "Key tuples"
> > > > > section.
> > > > >
> > > > >
> > > > > > As for me, these excercises with schema's doesn't worth

Re: Setting IGNITE_TO_STRING_INCLUDE_SENSITIVE=false prevents VM Arguments output

[Ivan Daschinsky]

This is security through obscurity, an obvious and a well-known anti
pattern. I suppose that printing jvm options, that is registered by
@IgniteSystemProperty annotation is an ideal variant

чт, 1 июл. 2021 г., 19:25 Valentin Kulichenko :

> Folks,
>
> *Anything* that a user provides to the system can potentially be considered
> sensitive information. This includes the VM arguments. We can't predict
> what exactly one can put there, so let's not make assumptions.
>
> When dealing with security, we should be as conservative as possible. That
> said, I do not even agree with the pattern approach - there might be a
> user's system property named IGNITE_xxx. It is also possible for our
> internal properties to contain sensitive information (not all of them are
> boolean flags).
>
> The only option I see is to print out specific properties for which we
> agree that they are safe. For example, we can introduce an annotation that
> would mark "safe" properties in IgniteSystemProperties; we will then print
> out only those that are marked with the annotation.
>
> -Val
>
>
> On Thu, Jul 1, 2021 at 7:07 AM Вячеслав Коптилин  >
> wrote:
>
> > Hello Ivan,
> >
> > > At least, we could just hide params that match a specific pattern
> > Yes, we can filter out all vm options that do not relate to Ignite, for
> > instance.
> >
> > > Ilya, go ahead, file ticket and prepare a PR.
> > Please do not rush. Let's listen to other community members. This
> question
> > is about security and it should not be discussed in a hurry (even though
> it
> > looks like an obvious thing).
> >
> > Thanks,
> > S.
> >
> > чт, 1 июл. 2021 г. в 16:55, Ivan Daschinsky :
> >
> > > I suppose, that all normal users should not suffer from this
> > restrictions.
> > > Nobody will pass password using jvm options. It is absolutely insane,
> > > normal users pass passwords using environment variables.
> > >
> > > At least, we could just hide params that match specific pattern
> > >
> > > Ilya, go ahead, file ticket and prepare a PR.
> > >
> > > чт, 1 июл. 2021 г., 16:45 Вячеслав Коптилин  >:
> > >
> > > > Hello,
> > > >
> > > > Unfortunately, the user can pass its own system properties via JVM
> > > options
> > > > as follows: -DMY_SECRET_PASSWORD=123
> > > > It does not seem, this approach is the best one. However, the user
> > should
> > > > have a "kostyl" in order to hide these properties and values in the
> log
> > > > file, IMHO.
> > > >
> > > > Thanks,
> > > > S.
> > > >
> > > > ср, 30 июн. 2021 г. в 22:52, Shishkov Ilya :
> > > >
> > > > > Hi Igniters,
> > > > >
> > > > > This feature [1, 2] prevents logging of the VM arguments when
> > > > > IGNITE_TO_STRING_INCLUDE_SENSITIVE option is set to false. Till
> now,
> > > > method
> > > > > IgniteKernal#ackVmArguments remains mostly the same [3].
> > > > >
> > > > > Is this behaviour actual now? Often, we should be able to get from
> > logs
> > > > the
> > > > > actual VM options used at startup even if output of sensitive data
> is
> > > > > restricted.
> > > > >
> > > > > 1. https://issues.apache.org/jira/browse/IGNITE-4991
> > > > > 2.
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/ignite/pull/2428/commits/4f90b6fd77bd23fa818620f0757b792ba388ef93
> > > > > 3.
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java#L3002
> > > > >
> > > >
> > >
> >
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

Val, when we think about binary protocol, using java interface as an
example is a not good idea. Please, see a tarantool binary api as an
example. Implementor of the client can write it immediately (1)
[1] --
 https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/


чт, 1 июл. 2021 г., 19:15 Valentin Kulichenko :

> Ivan,
>
> Regarding the API, please take a look at this package:
>
> https://github.com/apache/ignite-3/tree/main/modules/api/src/main/java/org/apache/ignite/table
>
> 'Table' is the primary API, which works with raw tuples. There are also
> multiple views on top of it, including KeyValueView and KeyValueBinaryView
> - these provide the key-value API.
>
> The ultimate goal is for the thin client to implement all these APIs.
>
> -Val
>
> On Thu, Jul 1, 2021 at 8:42 AM Igor Sapego  wrote:
>
> > Ivan, what are extra serde steps you are talking about?
> >
> > Best Regards,
> > Igor
> >
> >
> > On Thu, Jul 1, 2021 at 5:52 PM Ivan Daschinsky 
> > wrote:
> >
> > > > I agree. But this was decided before in IEP-54, and is out of scope
> for
> > > current IEP.
> > > Would you like to start a separate thread to discuss this? Or I can do
> > this
> > > a bit later.
> > >
> > > Great idea, let's discuss it. I suppose this will simplify many aspects
> > of
> > > realization and improve performance a lot
> > >
> > > чт, 1 июл. 2021 г., 17:50 Ivan Daschinsky :
> > >
> > > > > Here is the description of TUPLE_GET_ALL:
> > > > - UUID: table ID
> > > > - int: schema ID
> > > > - arr of arr: array of rows with values for all columns in given
> schema
> > > >
> > > > I suppose that we should describe this more verbose and explicit. I
> > > > nevertheless suggest to also consider writing values this way:
> > > > - arr of fields names (if name is missed, corresponding field is nil)
> > > > - arr of rows (row as array, length equal to fields array)
> > > >
> > > > It is quite simple and if we use str8 (it is more than enough for any
> > > > utf-8 reasonable field name), overhead will be negligible, but
> > > realization
> > > > of a client will be way simpler
> > > >
> > > > чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn :
> > > >
> > > >> > No it isn't, I have carefully read code and IEP, in your code you
> > > write
> > > >> > schema id in each tuple.
> > > >>
> > > >> There is no code for batch operations yet.
> > > >>
> > > >> Here is the description of TUPLE_GET_ALL:
> > > >> - UUID: table ID
> > > >> - int: schema ID
> > > >> - arr of arr: array of rows with values for all columns in given
> > schema
> > > >> (nil when value is missing for a column)
> > > >>
> > > >> As you can see, schema ID is written once for all rows.
> > > >> A row is just a set of values according to the schema.
> > > >>
> > > >>
> > > >> > Also, my biggest concern -- extra serde step. I suppose we should
> > pass
> > > >> > bytearray to internal api, and use msgpack throughout all wire
> > > >> protocols,
> > > >> > as tarantool does.
> > > >>
> > > >> I agree. But this was decided before in IEP-54, and is out of scope
> > for
> > > >> current IEP.
> > > >> Would you like to start a separate thread to discuss this? Or I can
> do
> > > >> this
> > > >> a bit later.
> > > >>
> > > >> On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky  >
> > > >> wrote:
> > > >>
> > > >> > > This is described in all operations that include multiple
> tuples.
> > > >> > No it isn't, I have carefully read code and IEP, in your code you
> > > write
> > > >> > schema id in each tuple.
> > > >> >
> > > >> > Also, my biggest concern -- extra serde step. I suppose we should
> > pass
> > > >> > bytearray to internal api, and use msgpack throughout all wire
> > > >> protocols,
> > > >> > as tarantool does.
> > > >> >
> > > >> > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn :
> > > >> >
> > > >> > > Ivan,
> > > >

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

Igor, as you can see from the code, we read data from msgpack and build the
tuple from scratch. Then we serialize this tuple again when send it to
backups. I presume this is absolutely unnecessary

чт, 1 июл. 2021 г., 18:42 Igor Sapego :

> Ivan, what are extra serde steps you are talking about?
>
> Best Regards,
> Igor
>
>
> On Thu, Jul 1, 2021 at 5:52 PM Ivan Daschinsky 
> wrote:
>
> > > I agree. But this was decided before in IEP-54, and is out of scope for
> > current IEP.
> > Would you like to start a separate thread to discuss this? Or I can do
> this
> > a bit later.
> >
> > Great idea, let's discuss it. I suppose this will simplify many aspects
> of
> > realization and improve performance a lot
> >
> > чт, 1 июл. 2021 г., 17:50 Ivan Daschinsky :
> >
> > > > Here is the description of TUPLE_GET_ALL:
> > > - UUID: table ID
> > > - int: schema ID
> > > - arr of arr: array of rows with values for all columns in given schema
> > >
> > > I suppose that we should describe this more verbose and explicit. I
> > > nevertheless suggest to also consider writing values this way:
> > > - arr of fields names (if name is missed, corresponding field is nil)
> > > - arr of rows (row as array, length equal to fields array)
> > >
> > > It is quite simple and if we use str8 (it is more than enough for any
> > > utf-8 reasonable field name), overhead will be negligible, but
> > realization
> > > of a client will be way simpler
> > >
> > > чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn :
> > >
> > >> > No it isn't, I have carefully read code and IEP, in your code you
> > write
> > >> > schema id in each tuple.
> > >>
> > >> There is no code for batch operations yet.
> > >>
> > >> Here is the description of TUPLE_GET_ALL:
> > >> - UUID: table ID
> > >> - int: schema ID
> > >> - arr of arr: array of rows with values for all columns in given
> schema
> > >> (nil when value is missing for a column)
> > >>
> > >> As you can see, schema ID is written once for all rows.
> > >> A row is just a set of values according to the schema.
> > >>
> > >>
> > >> > Also, my biggest concern -- extra serde step. I suppose we should
> pass
> > >> > bytearray to internal api, and use msgpack throughout all wire
> > >> protocols,
> > >> > as tarantool does.
> > >>
> > >> I agree. But this was decided before in IEP-54, and is out of scope
> for
> > >> current IEP.
> > >> Would you like to start a separate thread to discuss this? Or I can do
> > >> this
> > >> a bit later.
> > >>
> > >> On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky 
> > >> wrote:
> > >>
> > >> > > This is described in all operations that include multiple tuples.
> > >> > No it isn't, I have carefully read code and IEP, in your code you
> > write
> > >> > schema id in each tuple.
> > >> >
> > >> > Also, my biggest concern -- extra serde step. I suppose we should
> pass
> > >> > bytearray to internal api, and use msgpack throughout all wire
> > >> protocols,
> > >> > as tarantool does.
> > >> >
> > >> > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn :
> > >> >
> > >> > > Ivan,
> > >> > >
> > >> > > >  that there is not neccesary to write schema versions in each
> row
> > >> > > > in collectionof tuples
> > >> > >
> > >> > > This is described in all operations that include multiple tuples.
> > >> > >
> > >> > >
> > >> > > > it is not clear from your code (probably
> > >> > > > mistake?) how differ key tuples and value tuples from each other
> > >> > >
> > >> > > Key tuples include only key columns. Key columns come first in the
> > >> > schema.
> > >> > > Value tuples include all columns, key and value. Added "Key
> tuples"
> > >> > > section.
> > >> > >
> > >> > >
> > >> > > > As for me, these excercises with schema's doesn't worth a lot
> > >> > >
> > >> > > I'll add a benchmark and we'll 

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

> I agree. But this was decided before in IEP-54, and is out of scope for
current IEP.
Would you like to start a separate thread to discuss this? Or I can do this
a bit later.

Great idea, let's discuss it. I suppose this will simplify many aspects of
realization and improve performance a lot

чт, 1 июл. 2021 г., 17:50 Ivan Daschinsky :

> > Here is the description of TUPLE_GET_ALL:
> - UUID: table ID
> - int: schema ID
> - arr of arr: array of rows with values for all columns in given schema
>
> I suppose that we should describe this more verbose and explicit. I
> nevertheless suggest to also consider writing values this way:
> - arr of fields names (if name is missed, corresponding field is nil)
> - arr of rows (row as array, length equal to fields array)
>
> It is quite simple and if we use str8 (it is more than enough for any
> utf-8 reasonable field name), overhead will be negligible, but realization
> of a client will be way simpler
>
> чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn :
>
>> > No it isn't, I have carefully read code and IEP, in your code you write
>> > schema id in each tuple.
>>
>> There is no code for batch operations yet.
>>
>> Here is the description of TUPLE_GET_ALL:
>> - UUID: table ID
>> - int: schema ID
>> - arr of arr: array of rows with values for all columns in given schema
>> (nil when value is missing for a column)
>>
>> As you can see, schema ID is written once for all rows.
>> A row is just a set of values according to the schema.
>>
>>
>> > Also, my biggest concern -- extra serde step. I suppose we should pass
>> > bytearray to internal api, and use msgpack throughout all wire
>> protocols,
>> > as tarantool does.
>>
>> I agree. But this was decided before in IEP-54, and is out of scope for
>> current IEP.
>> Would you like to start a separate thread to discuss this? Or I can do
>> this
>> a bit later.
>>
>> On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky 
>> wrote:
>>
>> > > This is described in all operations that include multiple tuples.
>> > No it isn't, I have carefully read code and IEP, in your code you write
>> > schema id in each tuple.
>> >
>> > Also, my biggest concern -- extra serde step. I suppose we should pass
>> > bytearray to internal api, and use msgpack throughout all wire
>> protocols,
>> > as tarantool does.
>> >
>> > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn :
>> >
>> > > Ivan,
>> > >
>> > > >  that there is not neccesary to write schema versions in each row
>> > > > in collectionof tuples
>> > >
>> > > This is described in all operations that include multiple tuples.
>> > >
>> > >
>> > > > it is not clear from your code (probably
>> > > > mistake?) how differ key tuples and value tuples from each other
>> > >
>> > > Key tuples include only key columns. Key columns come first in the
>> > schema.
>> > > Value tuples include all columns, key and value. Added "Key tuples"
>> > > section.
>> > >
>> > >
>> > > > As for me, these excercises with schema's doesn't worth a lot
>> > >
>> > > I'll add a benchmark and we'll see.
>> > >
>> > >
>> > > On Thu, Jul 1, 2021 at 3:17 PM Ivan Daschinsky 
>> > > wrote:
>> > >
>> > > > I suppose, that there is not neccesary to write schema versions in
>> each
>> > > row
>> > > > in collectionof tuples. Also it is not clear from your code
>> (probably
>> > > > mistake?) how differ key tuples and value tuples from each other. In
>> > > > readTuple you always read full schema and check for full length. As
>> for
>> > > me,
>> > > > these excercises with schema's doesn't worth a lot. I.e. postgres
>> just
>> > > > writes field names and then simpy rows with data. Saving few bytes
>> > > doesn't
>> > > > make much deal. Btw, msgpack has special types for short strings
>> (i.e.
>> > > > str8). It is much easier use it and write field name as is.
>> > > >
>> > > > чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn :
>> > > >
>> > > > > Ivan, tuple serialization section added to the IEP, let me know
>> if it
>> > > is
>> > > > > clear enough.
>>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

> Here is the description of TUPLE_GET_ALL:
- UUID: table ID
- int: schema ID
- arr of arr: array of rows with values for all columns in given schema

I suppose that we should describe this more verbose and explicit. I
nevertheless suggest to also consider writing values this way:
- arr of fields names (if name is missed, corresponding field is nil)
- arr of rows (row as array, length equal to fields array)

It is quite simple and if we use str8 (it is more than enough for any utf-8
reasonable field name), overhead will be negligible, but realization of a
client will be way simpler

чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn :

> > No it isn't, I have carefully read code and IEP, in your code you write
> > schema id in each tuple.
>
> There is no code for batch operations yet.
>
> Here is the description of TUPLE_GET_ALL:
> - UUID: table ID
> - int: schema ID
> - arr of arr: array of rows with values for all columns in given schema
> (nil when value is missing for a column)
>
> As you can see, schema ID is written once for all rows.
> A row is just a set of values according to the schema.
>
>
> > Also, my biggest concern -- extra serde step. I suppose we should pass
> > bytearray to internal api, and use msgpack throughout all wire protocols,
> > as tarantool does.
>
> I agree. But this was decided before in IEP-54, and is out of scope for
> current IEP.
> Would you like to start a separate thread to discuss this? Or I can do this
> a bit later.
>
> On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky 
> wrote:
>
> > > This is described in all operations that include multiple tuples.
> > No it isn't, I have carefully read code and IEP, in your code you write
> > schema id in each tuple.
> >
> > Also, my biggest concern -- extra serde step. I suppose we should pass
> > bytearray to internal api, and use msgpack throughout all wire protocols,
> > as tarantool does.
> >
> > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn :
> >
> > > Ivan,
> > >
> > > >  that there is not neccesary to write schema versions in each row
> > > > in collectionof tuples
> > >
> > > This is described in all operations that include multiple tuples.
> > >
> > >
> > > > it is not clear from your code (probably
> > > > mistake?) how differ key tuples and value tuples from each other
> > >
> > > Key tuples include only key columns. Key columns come first in the
> > schema.
> > > Value tuples include all columns, key and value. Added "Key tuples"
> > > section.
> > >
> > >
> > > > As for me, these excercises with schema's doesn't worth a lot
> > >
> > > I'll add a benchmark and we'll see.
> > >
> > >
> > > On Thu, Jul 1, 2021 at 3:17 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > I suppose, that there is not neccesary to write schema versions in
> each
> > > row
> > > > in collectionof tuples. Also it is not clear from your code (probably
> > > > mistake?) how differ key tuples and value tuples from each other. In
> > > > readTuple you always read full schema and check for full length. As
> for
> > > me,
> > > > these excercises with schema's doesn't worth a lot. I.e. postgres
> just
> > > > writes field names and then simpy rows with data. Saving few bytes
> > > doesn't
> > > > make much deal. Btw, msgpack has special types for short strings
> (i.e.
> > > > str8). It is much easier use it and write field name as is.
> > > >
> > > > чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn :
> > > >
> > > > > Ivan, tuple serialization section added to the IEP, let me know if
> it
> > > is
> > > > > clear enough.
> > > > >
> > > > > Thanks!
> > > > >
> > > > > On Thu, Jul 1, 2021 at 2:06 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I can't find any description of tuple serialization in IEP, only
> in
> > > > code
> > > > > >
> > > > > > чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn :
> > > > > >
> > > > > > > Ivan,
> > > > > > >
> > > > > > > 0. The IEP is not in progress, it is ready for review and
> > > discussion.
> > > > > > > 1. Tuple serialization is described in the IEP and de

Re: Setting IGNITE_TO_STRING_INCLUDE_SENSITIVE=false prevents VM Arguments output

[Ivan Daschinsky]

I suppose, that all normal users should not suffer from this restrictions.
Nobody will pass password using jvm options. It is absolutely insane,
normal users pass passwords using environment variables.

At least, we could just hide params that match specific pattern

Ilya, go ahead, file ticket and prepare a PR.

чт, 1 июл. 2021 г., 16:45 Вячеслав Коптилин :

> Hello,
>
> Unfortunately, the user can pass its own system properties via JVM options
> as follows: -DMY_SECRET_PASSWORD=123
> It does not seem, this approach is the best one. However, the user should
> have a "kostyl" in order to hide these properties and values in the log
> file, IMHO.
>
> Thanks,
> S.
>
> ср, 30 июн. 2021 г. в 22:52, Shishkov Ilya :
>
> > Hi Igniters,
> >
> > This feature [1, 2] prevents logging of the VM arguments when
> > IGNITE_TO_STRING_INCLUDE_SENSITIVE option is set to false. Till now,
> method
> > IgniteKernal#ackVmArguments remains mostly the same [3].
> >
> > Is this behaviour actual now? Often, we should be able to get from logs
> the
> > actual VM options used at startup even if output of sensitive data is
> > restricted.
> >
> > 1. https://issues.apache.org/jira/browse/IGNITE-4991
> > 2.
> >
> >
> https://github.com/apache/ignite/pull/2428/commits/4f90b6fd77bd23fa818620f0757b792ba388ef93
> > 3.
> >
> >
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java#L3002
> >
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

> This is described in all operations that include multiple tuples.
No it isn't, I have carefully read code and IEP, in your code you write
schema id in each tuple.

Also, my biggest concern -- extra serde step. I suppose we should pass
bytearray to internal api, and use msgpack throughout all wire protocols,
as tarantool does.

чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn :

> Ivan,
>
> >  that there is not neccesary to write schema versions in each row
> > in collectionof tuples
>
> This is described in all operations that include multiple tuples.
>
>
> > it is not clear from your code (probably
> > mistake?) how differ key tuples and value tuples from each other
>
> Key tuples include only key columns. Key columns come first in the schema.
> Value tuples include all columns, key and value. Added "Key tuples"
> section.
>
>
> > As for me, these excercises with schema's doesn't worth a lot
>
> I'll add a benchmark and we'll see.
>
>
> On Thu, Jul 1, 2021 at 3:17 PM Ivan Daschinsky 
> wrote:
>
> > I suppose, that there is not neccesary to write schema versions in each
> row
> > in collectionof tuples. Also it is not clear from your code (probably
> > mistake?) how differ key tuples and value tuples from each other. In
> > readTuple you always read full schema and check for full length. As for
> me,
> > these excercises with schema's doesn't worth a lot. I.e. postgres just
> > writes field names and then simpy rows with data. Saving few bytes
> doesn't
> > make much deal. Btw, msgpack has special types for short strings (i.e.
> > str8). It is much easier use it and write field name as is.
> >
> > чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn :
> >
> > > Ivan, tuple serialization section added to the IEP, let me know if it
> is
> > > clear enough.
> > >
> > > Thanks!
> > >
> > > On Thu, Jul 1, 2021 at 2:06 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > I can't find any description of tuple serialization in IEP, only in
> > code
> > > >
> > > > чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn :
> > > >
> > > > > Ivan,
> > > > >
> > > > > 0. The IEP is not in progress, it is ready for review and
> discussion.
> > > > > 1. Tuple serialization is described in the IEP and demonstrated in
> > the
> > > > PoC
> > > > > (see ClientMessageHandler#readTuple), let me know if more details
> are
> > > > > required
> > > > > 2. Tuple schema serialization is described in SCHEMAS_GET section.
> > > Table
> > > > > schema (configuration) needs more details, you are right - I'll add
> > > them.
> > > > > 3. This IEP is about tables (tuple-based) API only, since it is the
> > > only
> > > > > API that we have right now, as noted in Risks and Assumptions.
> > > > >
> > > > > On Thu, Jul 1, 2021 at 1:53 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Also, is there any clear information about KV api? Is there any
> > plan
> > > to
> > > > > > implement it? Or is there any proposal about it?
> > > > > >
> > > > > > чт, 1 июл. 2021 г., 13:51 Ivan Daschinsky :
> > > > > >
> > > > > > > Pavel, but IEP is in progress, isn't it?
> > > > > > >
> > > > > > > 1. There is not any information about tuple serialization. And
> > > there
> > > > > > isn't
> > > > > > > a clear consensus about it.
> > > > > > > 2. There is not any information about schrma serialization
> > format.
> > > > And
> > > > > > > AFAIK, there isn't a clear consensus also.
> > > > > > >
> > > > > > > чт, 1 июл. 2021 г., 13:26 Pavel Tupitsyn  >:
> > > > > > >
> > > > > > >> Igniters,
> > > > > > >>
> > > > > > >> Please review the IEP for thin client protocol in 3.0 [1].
> > > > > > >> PoC is in progress [2]
> > > > > > >>
> > > > > > >> [1]
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0
> > > > > > >>
> > > > > > >> [2] https://github.com/apache/ignite-3/pull/191
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Apache Ignite 2.11

[Ivan Daschinsky]

Binary prorocol description should stand and improved (there are some
mistakes in this article)

чт, 1 июл. 2021 г., 16:05 Ivan Daschinsky :

> https://apache-ignite-binary-protocol-client.readthedocs.io/en/stable/
>
> AFAIK, thin clients have been already removed since 2.10
>
> чт, 1 июл. 2021 г., 15:57 Nikita Safonov :
>
>> Hi guys!
>>
>> Sorry, totally missed this email.
>>
>> I can do this right away.
>>
>> Just a couple of questions first:
>>
>> [1] On the website, under the Thin Clients tab, we have the following
>> articles:
>>
>> Thin Clients
>> Thin Clients Overview
>> Java Thin Client
>> .NET Thin Client
>> C++ Thin Client
>> Python Thin Client
>> PHP Thin Client
>> Node.js Thin Client
>> Binary Client Protocol
>>
>> I do believe that the introduction ( "Thin Clients" - the first one)
>> should
>> remain, while we can remove the rest (by the way, what about Binary Client
>> Protocol? Should it remain too?).
>> What do you think about this?
>>
>> [2] Could you please indicate the place in the docs where the link to the
>> actual readthedocs.io documentation is? I didn't manage to find it.
>> If we don't have it, I would place it at the beginning of the Thin
>> clients'
>> introduction article.
>>
>> Regards,
>> Nikita
>>
>>
>> ср, 30 июн. 2021 г. в 09:36, Alexey Gidaspov :
>>
>> > Hi, Nikita!
>> >
>> > Your help with documentation is really needed. Can you describe how we
>> can
>> > start with it?
>> >
>> > On 2021/06/09 10:06:09, Nikita Safonov 
>> wrote:
>> > > Hello Igniters!
>> > >
>> > > I guess that AI 2.11 will include documentation work.
>> > > I'll be glad to help with this.
>> > > And I'll be happy to know the scope of features that we need to
>> document.
>> > >
>> > > Regards,
>> > > Nikita
>> > >
>> > > пт, 4 июн. 2021 г. в 03:50, 18624049226 <18624049...@163.com>:
>> > >
>> > > > hello,
>> > > >
>> > > > can someone can help to merge the following patch into master?
>> > > >
>> > > > https://issues.apache.org/jira/browse/IGNITE-12852
>> > > >
>> > > > This patch has been completed for a long time, and I think it is a
>> > > > valuable one.
>> > > >
>> > > > 在 2021/6/3 下午11:12, Dmitry Pavlov 写道:
>> > > > > Sure, thanks for your reply
>> > > > >
>> > > > > Folks, could you please advice how to setup JIRA account
>> integration
>> > for
>> > > > non-committers?
>> > > > >
>> > > > > For the page
>> > > > >
>> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.11
>> > > > > I can see all issues, but Alexey's login seems to be not
>> sufficient.
>> > Can
>> > > > wiki admins set up smth? Or Alexey should do some setup on his own?
>> > > > >
>> > > > > On 2021/06/03 14:55:25, Maksim Timonin 
>> > wrote:
>> > > > >> I mean I'm OK to release it with 2.12. So, I am glad to hear
>> there
>> > are
>> > > > >> chances for 2.11, but I guess we can postpone it.
>> > > > >>
>> > > > >>
>> > > > >> On Thu, Jun 3, 2021 at 5:39 PM Dmitry Pavlov > >
>> > > > wrote:
>> > > > >>
>> > > > >>> ok, Maksim, keep us posted.
>> > > > >>>
>> > > > >>> We're in the middle of paperwork rigth now, so there is a
>> chance ))
>> > > > >>>
>> > > > >>> On 2021/06/03 14:30:55, Maksim Timonin > >
>> > > > wrote:
>> > > > >>>>> Regarding non-SQL index API it really depends on readiness. I
>> > guess
>> > > > if
>> > > > >>> it
>> > > > >>>> is not in the master, we can skip and release it next time.
>> > > > >>>>
>> > > > >>>> Hi, Dmitry! Yes, currently it is on review. I'm OK to release
>> it
>> > > > within
>> > > > >>> the
>> > > > >>>> next release.
>> > > >

Re: Apache Ignite 2.11

[Ivan Daschinsky]

https://apache-ignite-binary-protocol-client.readthedocs.io/en/stable/

AFAIK, thin clients have been already removed since 2.10

чт, 1 июл. 2021 г., 15:57 Nikita Safonov :

> Hi guys!
>
> Sorry, totally missed this email.
>
> I can do this right away.
>
> Just a couple of questions first:
>
> [1] On the website, under the Thin Clients tab, we have the following
> articles:
>
> Thin Clients
> Thin Clients Overview
> Java Thin Client
> .NET Thin Client
> C++ Thin Client
> Python Thin Client
> PHP Thin Client
> Node.js Thin Client
> Binary Client Protocol
>
> I do believe that the introduction ( "Thin Clients" - the first one) should
> remain, while we can remove the rest (by the way, what about Binary Client
> Protocol? Should it remain too?).
> What do you think about this?
>
> [2] Could you please indicate the place in the docs where the link to the
> actual readthedocs.io documentation is? I didn't manage to find it.
> If we don't have it, I would place it at the beginning of the Thin clients'
> introduction article.
>
> Regards,
> Nikita
>
>
> ср, 30 июн. 2021 г. в 09:36, Alexey Gidaspov :
>
> > Hi, Nikita!
> >
> > Your help with documentation is really needed. Can you describe how we
> can
> > start with it?
> >
> > On 2021/06/09 10:06:09, Nikita Safonov 
> wrote:
> > > Hello Igniters!
> > >
> > > I guess that AI 2.11 will include documentation work.
> > > I'll be glad to help with this.
> > > And I'll be happy to know the scope of features that we need to
> document.
> > >
> > > Regards,
> > > Nikita
> > >
> > > пт, 4 июн. 2021 г. в 03:50, 18624049226 <18624049...@163.com>:
> > >
> > > > hello,
> > > >
> > > > can someone can help to merge the following patch into master?
> > > >
> > > > https://issues.apache.org/jira/browse/IGNITE-12852
> > > >
> > > > This patch has been completed for a long time, and I think it is a
> > > > valuable one.
> > > >
> > > > 在 2021/6/3 下午11:12, Dmitry Pavlov 写道:
> > > > > Sure, thanks for your reply
> > > > >
> > > > > Folks, could you please advice how to setup JIRA account
> integration
> > for
> > > > non-committers?
> > > > >
> > > > > For the page
> > > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.11
> > > > > I can see all issues, but Alexey's login seems to be not
> sufficient.
> > Can
> > > > wiki admins set up smth? Or Alexey should do some setup on his own?
> > > > >
> > > > > On 2021/06/03 14:55:25, Maksim Timonin 
> > wrote:
> > > > >> I mean I'm OK to release it with 2.12. So, I am glad to hear there
> > are
> > > > >> chances for 2.11, but I guess we can postpone it.
> > > > >>
> > > > >>
> > > > >> On Thu, Jun 3, 2021 at 5:39 PM Dmitry Pavlov 
> > > > wrote:
> > > > >>
> > > > >>> ok, Maksim, keep us posted.
> > > > >>>
> > > > >>> We're in the middle of paperwork rigth now, so there is a chance
> ))
> > > > >>>
> > > > >>> On 2021/06/03 14:30:55, Maksim Timonin 
> > > > wrote:
> > > > > Regarding non-SQL index API it really depends on readiness. I
> > guess
> > > > if
> > > > >>> it
> > > >  is not in the master, we can skip and release it next time.
> > > > 
> > > >  Hi, Dmitry! Yes, currently it is on review. I'm OK to release it
> > > > within
> > > > >>> the
> > > >  next release.
> > > > 
> > > >  On Thu, Jun 3, 2021 at 5:26 PM Dmitry Pavlov <
> dpav...@apache.org>
> > > > wrote:
> > > > 
> > > > > Hi Alexey,
> > > > >
> > > > > Releasing master as-is makes sence for me.
> > > > >
> > > > > Regarding non-SQL index API it really depends on readiness. I
> > guess
> > > > if
> > > > >>> it
> > > > > is not in the master, we can skip and release it next time.
> > > > >
> > > > > We've a bit overdue initial scope freeze date. It's not a big
> > deal,
> > > > but
> > > > > still makes sense to start discussion, create and share wiki
> > page and
> > > > > create branch.
> > > > >
> > > > > Since committer privileges required to create branch, could you
> > > > please
> > > > > notify me in advance.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > On 2021/06/03 14:08:34, ��  <
> > > > > olive.c...@gmail.com> wrote:
> > > > >> :scope:
> > > > >>
> > > > >> Here is a list [1] of 195 tickets, i picked for Apache Ignite
> > 2.11
> > > > > release
> > > > >> Let's start discussion
> > > > >>
> > > > >> [1]
> > > > >>>
> > > >
> >
> https://issues.apache.org/jira/browse/IGNITE-14781?jql=project%20%3D%20IGNITE%20%20AND%20status%20%20%3D%20Resolved%20and%20fixVersion%20%3D%202.11%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
> > > > >> On 2021/05/13 14:27:33, Dmitry Pavlov 
> > wrote:
> > > > >>> Hi Alexey,
> > > > >>>
> > > > >>> :pmc to sign distribution:
> > > > >>> Formally, only PMC member can be a release manager, so part
> of
> > > > > activities should be picked up by committer and PMC member. If
> > noone
> > 

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

I suppose, that there is not neccesary to write schema versions in each row
in collectionof tuples. Also it is not clear from your code (probably
mistake?) how differ key tuples and value tuples from each other. In
readTuple you always read full schema and check for full length. As for me,
these excercises with schema's doesn't worth a lot. I.e. postgres just
writes field names and then simpy rows with data. Saving few bytes doesn't
make much deal. Btw, msgpack has special types for short strings (i.e.
str8). It is much easier use it and write field name as is.

чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn :

> Ivan, tuple serialization section added to the IEP, let me know if it is
> clear enough.
>
> Thanks!
>
> On Thu, Jul 1, 2021 at 2:06 PM Ivan Daschinsky 
> wrote:
>
> > I can't find any description of tuple serialization in IEP, only in code
> >
> > чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn :
> >
> > > Ivan,
> > >
> > > 0. The IEP is not in progress, it is ready for review and discussion.
> > > 1. Tuple serialization is described in the IEP and demonstrated in the
> > PoC
> > > (see ClientMessageHandler#readTuple), let me know if more details are
> > > required
> > > 2. Tuple schema serialization is described in SCHEMAS_GET section.
> Table
> > > schema (configuration) needs more details, you are right - I'll add
> them.
> > > 3. This IEP is about tables (tuple-based) API only, since it is the
> only
> > > API that we have right now, as noted in Risks and Assumptions.
> > >
> > > On Thu, Jul 1, 2021 at 1:53 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Also, is there any clear information about KV api? Is there any plan
> to
> > > > implement it? Or is there any proposal about it?
> > > >
> > > > чт, 1 июл. 2021 г., 13:51 Ivan Daschinsky :
> > > >
> > > > > Pavel, but IEP is in progress, isn't it?
> > > > >
> > > > > 1. There is not any information about tuple serialization. And
> there
> > > > isn't
> > > > > a clear consensus about it.
> > > > > 2. There is not any information about schrma serialization format.
> > And
> > > > > AFAIK, there isn't a clear consensus also.
> > > > >
> > > > > чт, 1 июл. 2021 г., 13:26 Pavel Tupitsyn :
> > > > >
> > > > >> Igniters,
> > > > >>
> > > > >> Please review the IEP for thin client protocol in 3.0 [1].
> > > > >> PoC is in progress [2]
> > > > >>
> > > > >> [1]
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0
> > > > >>
> > > > >> [2] https://github.com/apache/ignite-3/pull/191
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

I can't find any description of tuple serialization in IEP, only in code

чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn :

> Ivan,
>
> 0. The IEP is not in progress, it is ready for review and discussion.
> 1. Tuple serialization is described in the IEP and demonstrated in the PoC
> (see ClientMessageHandler#readTuple), let me know if more details are
> required
> 2. Tuple schema serialization is described in SCHEMAS_GET section. Table
> schema (configuration) needs more details, you are right - I'll add them.
> 3. This IEP is about tables (tuple-based) API only, since it is the only
> API that we have right now, as noted in Risks and Assumptions.
>
> On Thu, Jul 1, 2021 at 1:53 PM Ivan Daschinsky 
> wrote:
>
> > Also, is there any clear information about KV api? Is there any plan to
> > implement it? Or is there any proposal about it?
> >
> > чт, 1 июл. 2021 г., 13:51 Ivan Daschinsky :
> >
> > > Pavel, but IEP is in progress, isn't it?
> > >
> > > 1. There is not any information about tuple serialization. And there
> > isn't
> > > a clear consensus about it.
> > > 2. There is not any information about schrma serialization format. And
> > > AFAIK, there isn't a clear consensus also.
> > >
> > > чт, 1 июл. 2021 г., 13:26 Pavel Tupitsyn :
> > >
> > >> Igniters,
> > >>
> > >> Please review the IEP for thin client protocol in 3.0 [1].
> > >> PoC is in progress [2]
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0
> > >>
> > >> [2] https://github.com/apache/ignite-3/pull/191
> > >>
> > >
> >
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

Also, is there any clear information about KV api? Is there any plan to
implement it? Or is there any proposal about it?

чт, 1 июл. 2021 г., 13:51 Ivan Daschinsky :

> Pavel, but IEP is in progress, isn't it?
>
> 1. There is not any information about tuple serialization. And there isn't
> a clear consensus about it.
> 2. There is not any information about schrma serialization format. And
> AFAIK, there isn't a clear consensus also.
>
> чт, 1 июл. 2021 г., 13:26 Pavel Tupitsyn :
>
>> Igniters,
>>
>> Please review the IEP for thin client protocol in 3.0 [1].
>> PoC is in progress [2]
>>
>> [1]
>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0
>>
>> [2] https://github.com/apache/ignite-3/pull/191
>>
>

Re: IEP-76 Thin Client Protocol for Ignite 3.0

[Ivan Daschinsky]

Pavel, but IEP is in progress, isn't it?

1. There is not any information about tuple serialization. And there isn't
a clear consensus about it.
2. There is not any information about schrma serialization format. And
AFAIK, there isn't a clear consensus also.

чт, 1 июл. 2021 г., 13:26 Pavel Tupitsyn :

> Igniters,
>
> Please review the IEP for thin client protocol in 3.0 [1].
> PoC is in progress [2]
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0
>
> [2] https://github.com/apache/ignite-3/pull/191
>

Re: Setting IGNITE_TO_STRING_INCLUDE_SENSITIVE=false prevents VM Arguments output

[Ivan Daschinsky]

I suppose, that we should revert this particular line. I don't understand
who ever considers vm options as sensitive info.

ср, 30 июн. 2021 г., 22:52 Shishkov Ilya :

> Hi Igniters,
>
> This feature [1, 2] prevents logging of the VM arguments when
> IGNITE_TO_STRING_INCLUDE_SENSITIVE option is set to false. Till now, method
> IgniteKernal#ackVmArguments remains mostly the same [3].
>
> Is this behaviour actual now? Often, we should be able to get from logs the
> actual VM options used at startup even if output of sensitive data is
> restricted.
>
> 1. https://issues.apache.org/jira/browse/IGNITE-4991
> 2.
>
> https://github.com/apache/ignite/pull/2428/commits/4f90b6fd77bd23fa818620f0757b792ba388ef93
> 3.
>
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/IgniteKernal.java#L3002
>

Re: IGNITE-14812: Statistics

[Ivan Daschinsky]

AFAIK, it is enough to include mention of this library in NOTICE, please
see here [1][2][3]

[1] -- https://www.apache.org/legal/src-headers.html#3party
[2] -- https://www.apache.org/legal/src-headers.html#notice
[3] -- https://www.apache.org/legal/resolved.html

ср, 23 июн. 2021 г. в 17:36, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Taras,
>
> It is true that we try to minimize dependencies, but there are some anyway.
> I think it's perfectly fine to add this library as a dependency.
>
> Will you be able to do this asap? It is surely better than reverting the
> commit :)
>
> -Val
>
> On Wed, Jun 23, 2021 at 12:32 AM Taras Ledkov 
> wrote:
>
> > Hi,
> >
> >  > Taras, can you, please, describe the features that was implemented in
> > this merge?
> >  > How users supposed to use them?
> >  > Do we have plans to document?
> >
> > Sure. Alexander Belyak will describe and file ticket to documentation.
> >
> > On 23.06.2021 9:27, Nikolay Izhikov wrote:
> > > Hello, Taras.
> > >
> > > Thanks for feedback.
> > >
> > >> AFAIK and as long as I can remember Ignite project try to minimize
> > external dependencies usage and adds new external dependency only when
> > there is no other way out.
> > > Does it mean we have to incapsulate every external library we want to
> > use?
> > >
> > > Taras, can you, please, describe the features that was implemented in
> > this merge?
> > > How users supposed to use them?
> > > Do we have plans to document?
> > >
> > >
> > >> 23 июня 2021 г., в 09:21, Taras Ledkov 
> > написал(а):
> > >>
> > >> Hi,
> > >>
> > >> We have discussed BCrypt include/add dependency here [1].
> > >> AFAIK and as long as I can remember Ignite project try to minimize
> > external dependencies usage
> > >> and adds new external dependency only when there is no other way out.
> > >>
> > >> [1].
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Username-password-authentication-for-thin-clients-tp26058p26954.html
> > >>
> > >> On 23.06.2021 3:08, Valentin Kulichenko wrote:
> > >>> Dmitry,
> > >>>
> > >>> As the PMC chair, would you mind contacting legal regarding the
> matter?
> > >>> This is not the only example of such code (e.g. [1]), so we should
> look
> > >>> into this asap.
> > >>>
> > >>> [1]
> > >>>
> >
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/mindrot/BCrypt.java
> > >>>
> > >>> As for this particular commit, can HLL be added as a dependency
> > instead? Is
> > >>> there any particular reason to include the source code? @Sasha Belyak
> > >>>  , can you please chime in?
> > >>>
> > >>> -Val
> > >>>
> > >>> On Tue, Jun 22, 2021 at 8:10 AM Dmitry Pavlov 
> > wrote:
> > >>>
> >  Hi Nikolai,
> > 
> >  thank you for noticing. I guess it's not about license, but about
> >  Intellectual Property (IP) ownership.
> > 
> >  AFAIK, Apache License 2.0 is here and AL 2.0 is definetely allowed
> to
> > be
> >  used in the codebase for an Apache project (
> >  https://www.apache.org/legal/resolved.html)
> > 
> >  But licenese and IP owner are 2 sligthly different things. E.g at
> the
> > end
> >  of any website you can find:
> >  Copyright © 2021 The Apache Software Foundation, Licensed under the
> > Apache
> >  License, Version 2.0.
> > 
> >  Incubated projects are mandated to transfer IP to the ASF. And I'm
> not
> >  aware of any exceptions.
> > 
> >  In this PR there are 5 classes which licenses with AL 2.0, but IP
> > owner is
> >  3rd party company.
> > 
> >  I'm a bit concerned about having such code in the project. I'd
> rather
> >  reverted it until we have approval from experts at mailing list:
> >  legal-disc...@apache.org
> > 
> >  Sincerely,
> >  Dmitriy Pavlov
> > 
> >  On 2021/06/22 14:56:49, Nikolay Izhikov 
> wrote:
> > > Hello, Igniters.
> > >
> > > Recently huge commit was merged [1].
> > >
> > > Taras, Alexander, can you, please, explain what is purpose of the
> > commit?
> > > What feature it implemented?
> > >
> > > Looked inside the ticket and found no explanation.
> > > Description is "Add statistics collection and usage.»
> > >
> > > Do we have plans to document this new feature?
> > >
> > > Also, I see very strange license in added files [2]
> > >
> > > ```
> > >   * Copyright 2013 Aggregate Knowledge, Inc.
> > >   *
> > >   * Licensed under the Apache License, Version 2.0 (the "License");
> > > ```
> > >
> > > Is it OK to have those copyright inside ASF codebase?
> > > Is it some kind of external dependency we adopted as part of the
> > code?
> > > Why do we need it?
> > >
> > > [1]
> > 
> >
> https://github.com/apache/ignite/commit/503a98495433e1d0cf84f8be8c1e2adc57034fbb
> > > [2]
> > 
> >
> https://github.com/apache/ignite/blob/master/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query

Re: IEP-75 Thin Client MsgPack Serialization for 3.0

[Ivan Daschinsky]

Pavel, let me clarify one thing.

1. If this proposal is about binary protocol, then there is no objection I
suppose.
2. If this proposal about serialization of key-value, there are some
uncertainties, especially about complex objects. In this case this proposal
needs more work.

ср, 23 июн. 2021 г. в 14:07, Pavel Tupitsyn :

> Igniters,
>
> Looks like there are no objections and we can accept the proposal.
> I will close it tomorrow and move on to the thin client protocol itself.
>
> On Fri, Jun 18, 2021 at 12:10 PM Ivan Daschinsky 
> wrote:
>
> > >> To make it fair. Ignite uses thread-local reusable buffers, see [1].
> > I know, but PooledMessageBufferOutput is not about thread-local, isn't
> it?
> >
> > I'm not against about MsgPack, I'm for fair and not biased comparison.
> >
> > I suppose that MsgPack is an ideal candidate for thin client binary
> > protocol, not only for serializing some user data.
> > I am analyzed some tarantool connectors [1] [2] [3] and found msgpack
> based
> > protocol is a really good idea.
> > Also there is realy super fast and just 1 header library with liberal
> BSD-2
> > licence for C -- msgpuck [4]. It used in tarantool itself
> > and in [1] and is stable and bullet proof.
> >
> > [1] -- https://github.com/igorcoding/asynctnt
> > [2] -- https://github.com/tarantool/tarantool-python/
> > [3] -- https://github.com/tarantool/go-tarantool
> > [4] -- https://github.com/rtsisyk/msgpuck
> >
> > пт, 18 июн. 2021 г. в 11:44, Pavel Tupitsyn :
> >
> > > Ivan,
> > >
> > > > why do you use  PooledMessageBufferOutput in benchmarks?
> > >
> > > To make it fair. Ignite uses thread-local reusable buffers, see [1].
> > >
> > >
> > > > why packer from msgpack-core show better performance than
> > > > BinaryWriter. And I suppose that benchmark is not quite fair.
> > >
> > > MsgPack writes and reads less bytes, so it should be faster.
> > > Benchmark is not 100% fair, there are some small extra things that
> > > BinaryWriter does.
> > >
> > > However:
> > > 1. I don't think we care about super-precise benchmarks here, just the
> > > ballpark.
> > > 2. We are discussing the format, not the implementation.
> > >
> > > Important takeaway is:
> > > The format does not prevent someone from implementing it efficiently.
> > >
> > >
> > >
> > > [1]
> > >
> > >
> >
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryWriterExImpl.java#L101
> > >
> > > On Thu, Jun 17, 2021 at 10:40 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Andrey, here we discuss serialization format, as far as I understand.
> > > > Current implementation of ignite binary object serialization can be
> > > > rewritten.
> > > > If we do not care about fast (O(1)) field lookup, about schema
> > validation
> > > > and so on, msgpack is a really good option. It is also good for
> client
> > > > binary protocol, i.e.
> > > > tarantool uses it.
> > > >
> > > > >> Binarilizable interface forces user to write serialization code
> > > > I am talking about speed comparison. You can see from Pavel's data,
> > > > jackson-msgpack shows a pathetic performance comparing with a
> ignite's
> > > > default binary marshaller. If you want really fast serialization --
> the
> > > > only option is to write code by yourself or use code generation.
> > Default
> > > > packer from msgpack-core java package is similar to BinaryWriter. So
> I
> > am
> > > > wondering why packer from msgpack-core show better performance than
> > > > BinaryWriter. And I suppose that benchmark is not quite fair.
> > > >
> > > >
> > > > чт, 17 июн. 2021 г. в 22:19, Andrey Mashenkov <
> > > andrey.mashen...@gmail.com
> > > > >:
> > > >
> > > > > Ivan, thankd for clarification.
> > > > >
> > > > > Binarilizable interface forces user to write serialization code. We
> > can
> > > > > support this or similar interface.
> > > > > But I'd like Ignite has some default serializer in addition. It can
> > be
> > > > also
> > > > > useful e.g. in compute for param and result serialization.
> > > > >
> > 

[ANNOUNCE] Apache IGNITE python thin client (pyignite) 0.5.0 released

[Ivan Daschinsky]

The Apache Ignite Community is pleased to announce the release of
Apache IGNITE python thin client (pyignite) 0.5.0.

This new release of python thin client contains new cool features:
Namely:
1. Transactions support
2. Expiry policy support.
3. Performance improvement of asyncio version
4. Few bug-fixes

For the full list of changes, you can refer to the RELEASE_NOTES.
https://ignite.apache.org/releases/pyignite/0.5.0/release_notes.html

You can install this version using pip
>> pip install pyignite==0.5.0

Alternatively, you can download sources and binary packages (wheels) from
here:
https://dist.apache.org/repos/dist/release/ignite/pyignite/0.5.0/

Please let us know if you have any problems
https://ignite.apache.org/community/resources.html#ask

Regards,
Ivan Daschinsky on behalf of the Apache Ignite community.

[RESULT] [VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

Hello, Igniters!

Release pyignite-0.5.0-rc1 has been accepted.

The votes received:
3 "+1" binding votes
4 "+1" votes

There are no "+0" or "-1"

Here the votes received:
- Igor Sapego (binding) +1
- Pavel Tupitsyn (binding) +1
- Nickolay Izhikov (binding) +1
- Ivan Daschinsky +1
- Alexander Surkov +1
- Stephen Darlington +1
- Ilya Shishkov +1

New release will be announced soon

Re: [VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

I suppose that we should not cancel this release, despite the fact that
this is not obvious behaviour. This is not a regression, this behaviour is
documented and this behaviour lasts for few years. Lets remove it, if the
majority are against it, but in the next release.

пт, 18 июн. 2021 г. в 13:08, Ivan Daschinsky :

> >>  we explicitly set use_ssl=True.
> Sorry, typo -- implicitly
>
> пт, 18 июн. 2021 г. в 12:59, Ivan Daschinsky :
>
>> AHA! I see, this is not a bug -- this is a feature. If you pass username
>> and password, we explicitly set use_ssl=True. So if your cluster is
>> configured without ssl but with authentication,
>> you should explicitly pass use_ssl=False.
>>
>> This behaviour is from old version and I suppose it is correct. Who wants
>> authentication that sent without encryption?
>>
>> пт, 18 июн. 2021 г. в 12:54, Ivan Daschinsky :
>>
>>> Just rechecked test on release branch, add extra check with cluster
>>> activation and putting some data -- everything works ok. Authentication
>>> enabled, persistence enabled,
>>> with and without ssl. Could you please provide you ignite config and
>>> your code.
>>>
>>> пт, 18 июн. 2021 г. в 12:46, Ivan Daschinsky :
>>>
>>>> There is a test for it.
>>>>
>>>> пт, 18 июн. 2021 г. в 12:30, Stephen Darlington <
>>>> stephen.darling...@gridgain.com>:
>>>>
>>>>> Oh… can someone else check this: it appears that authenticated
>>>>> connections fail.
>>>>>
>>>>> With Ignite 2.10 the connection times-out:
>>>>>
>>>>> [10:28:58,015][WARNING][grid-timeout-worker-#22][ClientListenerNioListener]
>>>>> Unable to perform handshake within timeout [timeout=1, remoteAddr=/
>>>>> 127.0.0.1:54044]
>>>>>
>>>>> Didn’t try this with 0.4.0 so not sure if it’s a regression, but it’s
>>>>> not great.
>>>>>
>>>>> > On 18 Jun 2021, at 09:36, Stephen Darlington <
>>>>> stephen.darling...@gridgain.com> wrote:
>>>>> >
>>>>> > +1
>>>>> >
>>>>> > Checked on macOS, played with the new expiry APIs and a bunch of
>>>>> thefundamentals.
>>>>> >
>>>>> >> On 17 Jun 2021, at 12:46, Pavel Tupitsyn 
>>>>> wrote:
>>>>> >>
>>>>> >> +1
>>>>> >>
>>>>> >> Checked pip install from tar.gz on Python 3.8 on Ubuntu 20.04, ran
>>>>> some of
>>>>> >> the examples.
>>>>> >>
>>>>> >> On Thu, Jun 17, 2021 at 2:32 PM Igor Sapego 
>>>>> wrote:
>>>>> >>
>>>>> >>> +1 from me
>>>>> >>>
>>>>> >>> Best Regards,
>>>>> >>> Igor
>>>>> >>>
>>>>> >>>
>>>>> >>> On Thu, Jun 17, 2021 at 12:10 PM Ivan Daschinsky <
>>>>> ivanda...@gmail.com>
>>>>> >>> wrote:
>>>>> >>>
>>>>> >>>> +1 From me
>>>>> >>>> Checked on Ubuntu 20.04 and windows 10
>>>>> >>>> 1. Installation from wheels for pythons 3.6 3.7 3.8 3.9
>>>>> >>>> 2. Native module work
>>>>> >>>> 3. Examples
>>>>> >>>>
>>>>> >>>> Checked on Ubuntu 20.04 building from source package and correct
>>>>> work of
>>>>> >>>> result package.
>>>>> >>>>
>>>>> >>>> Checked all sha256 checksums and gpg signatures.
>>>>> >>>>
>>>>> >>>> Let's extend voting period till June 18, 15:00 UTC
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> ср, 16 июн. 2021 г. в 17:34, Ivan Daschinsky <
>>>>> ivanda...@apache.org>:
>>>>> >>>>
>>>>> >>>>> The vote will end at June, 17 15:00 UTC.
>>>>> >>>>>
>>>>> >>>>> ср, 16 июн. 2021 г. в 17:33, Ivan Daschinsky <
>>>>> ivanda...@apache.org>:
>>>>> >>>>>>
>>>>> >>>>>> Dear Igniters!
>>>

Re: [VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

>>  we explicitly set use_ssl=True.
Sorry, typo -- implicitly

пт, 18 июн. 2021 г. в 12:59, Ivan Daschinsky :

> AHA! I see, this is not a bug -- this is a feature. If you pass username
> and password, we explicitly set use_ssl=True. So if your cluster is
> configured without ssl but with authentication,
> you should explicitly pass use_ssl=False.
>
> This behaviour is from old version and I suppose it is correct. Who wants
> authentication that sent without encryption?
>
> пт, 18 июн. 2021 г. в 12:54, Ivan Daschinsky :
>
>> Just rechecked test on release branch, add extra check with cluster
>> activation and putting some data -- everything works ok. Authentication
>> enabled, persistence enabled,
>> with and without ssl. Could you please provide you ignite config and your
>> code.
>>
>> пт, 18 июн. 2021 г. в 12:46, Ivan Daschinsky :
>>
>>> There is a test for it.
>>>
>>> пт, 18 июн. 2021 г. в 12:30, Stephen Darlington <
>>> stephen.darling...@gridgain.com>:
>>>
>>>> Oh… can someone else check this: it appears that authenticated
>>>> connections fail.
>>>>
>>>> With Ignite 2.10 the connection times-out:
>>>>
>>>> [10:28:58,015][WARNING][grid-timeout-worker-#22][ClientListenerNioListener]
>>>> Unable to perform handshake within timeout [timeout=1, remoteAddr=/
>>>> 127.0.0.1:54044]
>>>>
>>>> Didn’t try this with 0.4.0 so not sure if it’s a regression, but it’s
>>>> not great.
>>>>
>>>> > On 18 Jun 2021, at 09:36, Stephen Darlington <
>>>> stephen.darling...@gridgain.com> wrote:
>>>> >
>>>> > +1
>>>> >
>>>> > Checked on macOS, played with the new expiry APIs and a bunch of
>>>> thefundamentals.
>>>> >
>>>> >> On 17 Jun 2021, at 12:46, Pavel Tupitsyn 
>>>> wrote:
>>>> >>
>>>> >> +1
>>>> >>
>>>> >> Checked pip install from tar.gz on Python 3.8 on Ubuntu 20.04, ran
>>>> some of
>>>> >> the examples.
>>>> >>
>>>> >> On Thu, Jun 17, 2021 at 2:32 PM Igor Sapego 
>>>> wrote:
>>>> >>
>>>> >>> +1 from me
>>>> >>>
>>>> >>> Best Regards,
>>>> >>> Igor
>>>> >>>
>>>> >>>
>>>> >>> On Thu, Jun 17, 2021 at 12:10 PM Ivan Daschinsky <
>>>> ivanda...@gmail.com>
>>>> >>> wrote:
>>>> >>>
>>>> >>>> +1 From me
>>>> >>>> Checked on Ubuntu 20.04 and windows 10
>>>> >>>> 1. Installation from wheels for pythons 3.6 3.7 3.8 3.9
>>>> >>>> 2. Native module work
>>>> >>>> 3. Examples
>>>> >>>>
>>>> >>>> Checked on Ubuntu 20.04 building from source package and correct
>>>> work of
>>>> >>>> result package.
>>>> >>>>
>>>> >>>> Checked all sha256 checksums and gpg signatures.
>>>> >>>>
>>>> >>>> Let's extend voting period till June 18, 15:00 UTC
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> ср, 16 июн. 2021 г. в 17:34, Ivan Daschinsky >>> >:
>>>> >>>>
>>>> >>>>> The vote will end at June, 17 15:00 UTC.
>>>> >>>>>
>>>> >>>>> ср, 16 июн. 2021 г. в 17:33, Ivan Daschinsky <
>>>> ivanda...@apache.org>:
>>>> >>>>>>
>>>> >>>>>> Dear Igniters!
>>>> >>>>>>
>>>> >>>>>> Release candidate binaries for subj are uploaded and ready for
>>>> vote
>>>> >>>>>> You can find them here:
>>>> >>>>>> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc1
>>>> >>>>>>
>>>> >>>>>> If you follow the link above, you will find source package
>>>> (*.tar.gz
>>>> >>>> and
>>>> >>>>> *.zip)
>>>> >>>>>> and binary packages (wheels) for windows (amd64) and linux
>>>> (x86_64)
>>>> >&

Re: [VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

And this behavior is thoroughly described in docs
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/examples.html#password-authentication

пт, 18 июн. 2021 г. в 12:59, Ivan Daschinsky :

> AHA! I see, this is not a bug -- this is a feature. If you pass username
> and password, we explicitly set use_ssl=True. So if your cluster is
> configured without ssl but with authentication,
> you should explicitly pass use_ssl=False.
>
> This behaviour is from old version and I suppose it is correct. Who wants
> authentication that sent without encryption?
>
> пт, 18 июн. 2021 г. в 12:54, Ivan Daschinsky :
>
>> Just rechecked test on release branch, add extra check with cluster
>> activation and putting some data -- everything works ok. Authentication
>> enabled, persistence enabled,
>> with and without ssl. Could you please provide you ignite config and your
>> code.
>>
>> пт, 18 июн. 2021 г. в 12:46, Ivan Daschinsky :
>>
>>> There is a test for it.
>>>
>>> пт, 18 июн. 2021 г. в 12:30, Stephen Darlington <
>>> stephen.darling...@gridgain.com>:
>>>
>>>> Oh… can someone else check this: it appears that authenticated
>>>> connections fail.
>>>>
>>>> With Ignite 2.10 the connection times-out:
>>>>
>>>> [10:28:58,015][WARNING][grid-timeout-worker-#22][ClientListenerNioListener]
>>>> Unable to perform handshake within timeout [timeout=1, remoteAddr=/
>>>> 127.0.0.1:54044]
>>>>
>>>> Didn’t try this with 0.4.0 so not sure if it’s a regression, but it’s
>>>> not great.
>>>>
>>>> > On 18 Jun 2021, at 09:36, Stephen Darlington <
>>>> stephen.darling...@gridgain.com> wrote:
>>>> >
>>>> > +1
>>>> >
>>>> > Checked on macOS, played with the new expiry APIs and a bunch of
>>>> thefundamentals.
>>>> >
>>>> >> On 17 Jun 2021, at 12:46, Pavel Tupitsyn 
>>>> wrote:
>>>> >>
>>>> >> +1
>>>> >>
>>>> >> Checked pip install from tar.gz on Python 3.8 on Ubuntu 20.04, ran
>>>> some of
>>>> >> the examples.
>>>> >>
>>>> >> On Thu, Jun 17, 2021 at 2:32 PM Igor Sapego 
>>>> wrote:
>>>> >>
>>>> >>> +1 from me
>>>> >>>
>>>> >>> Best Regards,
>>>> >>> Igor
>>>> >>>
>>>> >>>
>>>> >>> On Thu, Jun 17, 2021 at 12:10 PM Ivan Daschinsky <
>>>> ivanda...@gmail.com>
>>>> >>> wrote:
>>>> >>>
>>>> >>>> +1 From me
>>>> >>>> Checked on Ubuntu 20.04 and windows 10
>>>> >>>> 1. Installation from wheels for pythons 3.6 3.7 3.8 3.9
>>>> >>>> 2. Native module work
>>>> >>>> 3. Examples
>>>> >>>>
>>>> >>>> Checked on Ubuntu 20.04 building from source package and correct
>>>> work of
>>>> >>>> result package.
>>>> >>>>
>>>> >>>> Checked all sha256 checksums and gpg signatures.
>>>> >>>>
>>>> >>>> Let's extend voting period till June 18, 15:00 UTC
>>>> >>>>
>>>> >>>>
>>>> >>>>
>>>> >>>> ср, 16 июн. 2021 г. в 17:34, Ivan Daschinsky >>> >:
>>>> >>>>
>>>> >>>>> The vote will end at June, 17 15:00 UTC.
>>>> >>>>>
>>>> >>>>> ср, 16 июн. 2021 г. в 17:33, Ivan Daschinsky <
>>>> ivanda...@apache.org>:
>>>> >>>>>>
>>>> >>>>>> Dear Igniters!
>>>> >>>>>>
>>>> >>>>>> Release candidate binaries for subj are uploaded and ready for
>>>> vote
>>>> >>>>>> You can find them here:
>>>> >>>>>> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc1
>>>> >>>>>>
>>>> >>>>>> If you follow the link above, you will find source package
>>>> (*.tar.gz
>>>> >>>> and
>>>> >>>>> *.zip)
>>>> >>>>>> and binary pack

Re: [VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

AHA! I see, this is not a bug -- this is a feature. If you pass username
and password, we explicitly set use_ssl=True. So if your cluster is
configured without ssl but with authentication,
you should explicitly pass use_ssl=False.

This behaviour is from old version and I suppose it is correct. Who wants
authentication that sent without encryption?

пт, 18 июн. 2021 г. в 12:54, Ivan Daschinsky :

> Just rechecked test on release branch, add extra check with cluster
> activation and putting some data -- everything works ok. Authentication
> enabled, persistence enabled,
> with and without ssl. Could you please provide you ignite config and your
> code.
>
> пт, 18 июн. 2021 г. в 12:46, Ivan Daschinsky :
>
>> There is a test for it.
>>
>> пт, 18 июн. 2021 г. в 12:30, Stephen Darlington <
>> stephen.darling...@gridgain.com>:
>>
>>> Oh… can someone else check this: it appears that authenticated
>>> connections fail.
>>>
>>> With Ignite 2.10 the connection times-out:
>>>
>>> [10:28:58,015][WARNING][grid-timeout-worker-#22][ClientListenerNioListener]
>>> Unable to perform handshake within timeout [timeout=1, remoteAddr=/
>>> 127.0.0.1:54044]
>>>
>>> Didn’t try this with 0.4.0 so not sure if it’s a regression, but it’s
>>> not great.
>>>
>>> > On 18 Jun 2021, at 09:36, Stephen Darlington <
>>> stephen.darling...@gridgain.com> wrote:
>>> >
>>> > +1
>>> >
>>> > Checked on macOS, played with the new expiry APIs and a bunch of
>>> thefundamentals.
>>> >
>>> >> On 17 Jun 2021, at 12:46, Pavel Tupitsyn 
>>> wrote:
>>> >>
>>> >> +1
>>> >>
>>> >> Checked pip install from tar.gz on Python 3.8 on Ubuntu 20.04, ran
>>> some of
>>> >> the examples.
>>> >>
>>> >> On Thu, Jun 17, 2021 at 2:32 PM Igor Sapego 
>>> wrote:
>>> >>
>>> >>> +1 from me
>>> >>>
>>> >>> Best Regards,
>>> >>> Igor
>>> >>>
>>> >>>
>>> >>> On Thu, Jun 17, 2021 at 12:10 PM Ivan Daschinsky <
>>> ivanda...@gmail.com>
>>> >>> wrote:
>>> >>>
>>> >>>> +1 From me
>>> >>>> Checked on Ubuntu 20.04 and windows 10
>>> >>>> 1. Installation from wheels for pythons 3.6 3.7 3.8 3.9
>>> >>>> 2. Native module work
>>> >>>> 3. Examples
>>> >>>>
>>> >>>> Checked on Ubuntu 20.04 building from source package and correct
>>> work of
>>> >>>> result package.
>>> >>>>
>>> >>>> Checked all sha256 checksums and gpg signatures.
>>> >>>>
>>> >>>> Let's extend voting period till June 18, 15:00 UTC
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> ср, 16 июн. 2021 г. в 17:34, Ivan Daschinsky >> >:
>>> >>>>
>>> >>>>> The vote will end at June, 17 15:00 UTC.
>>> >>>>>
>>> >>>>> ср, 16 июн. 2021 г. в 17:33, Ivan Daschinsky >> >:
>>> >>>>>>
>>> >>>>>> Dear Igniters!
>>> >>>>>>
>>> >>>>>> Release candidate binaries for subj are uploaded and ready for
>>> vote
>>> >>>>>> You can find them here:
>>> >>>>>> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc1
>>> >>>>>>
>>> >>>>>> If you follow the link above, you will find source package
>>> (*.tar.gz
>>> >>>> and
>>> >>>>> *.zip)
>>> >>>>>> and binary packages (wheels) for windows (amd64) and linux
>>> (x86_64)
>>> >>>>>> for pythons 36, 37, 38, 39. Also, there are sha512 and gpg
>>> >>> signatures.
>>> >>>>>> Code signing keys can be found here --
>>> >>>>> https://downloads.apache.org/ignite/KEYS
>>> >>>>>> Here you can find instructions how to verify packages
>>> >>>>>> https://www.apache.org/info/verification.html
>>> >>>>>>
>>> >>>>>> You can install binary

Re: [VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

Just rechecked test on release branch, add extra check with cluster
activation and putting some data -- everything works ok. Authentication
enabled, persistence enabled,
with and without ssl. Could you please provide you ignite config and your
code.

пт, 18 июн. 2021 г. в 12:46, Ivan Daschinsky :

> There is a test for it.
>
> пт, 18 июн. 2021 г. в 12:30, Stephen Darlington <
> stephen.darling...@gridgain.com>:
>
>> Oh… can someone else check this: it appears that authenticated
>> connections fail.
>>
>> With Ignite 2.10 the connection times-out:
>>
>> [10:28:58,015][WARNING][grid-timeout-worker-#22][ClientListenerNioListener]
>> Unable to perform handshake within timeout [timeout=1, remoteAddr=/
>> 127.0.0.1:54044]
>>
>> Didn’t try this with 0.4.0 so not sure if it’s a regression, but it’s not
>> great.
>>
>> > On 18 Jun 2021, at 09:36, Stephen Darlington <
>> stephen.darling...@gridgain.com> wrote:
>> >
>> > +1
>> >
>> > Checked on macOS, played with the new expiry APIs and a bunch of
>> thefundamentals.
>> >
>> >> On 17 Jun 2021, at 12:46, Pavel Tupitsyn  wrote:
>> >>
>> >> +1
>> >>
>> >> Checked pip install from tar.gz on Python 3.8 on Ubuntu 20.04, ran
>> some of
>> >> the examples.
>> >>
>> >> On Thu, Jun 17, 2021 at 2:32 PM Igor Sapego 
>> wrote:
>> >>
>> >>> +1 from me
>> >>>
>> >>> Best Regards,
>> >>> Igor
>> >>>
>> >>>
>> >>> On Thu, Jun 17, 2021 at 12:10 PM Ivan Daschinsky > >
>> >>> wrote:
>> >>>
>> >>>> +1 From me
>> >>>> Checked on Ubuntu 20.04 and windows 10
>> >>>> 1. Installation from wheels for pythons 3.6 3.7 3.8 3.9
>> >>>> 2. Native module work
>> >>>> 3. Examples
>> >>>>
>> >>>> Checked on Ubuntu 20.04 building from source package and correct
>> work of
>> >>>> result package.
>> >>>>
>> >>>> Checked all sha256 checksums and gpg signatures.
>> >>>>
>> >>>> Let's extend voting period till June 18, 15:00 UTC
>> >>>>
>> >>>>
>> >>>>
>> >>>> ср, 16 июн. 2021 г. в 17:34, Ivan Daschinsky :
>> >>>>
>> >>>>> The vote will end at June, 17 15:00 UTC.
>> >>>>>
>> >>>>> ср, 16 июн. 2021 г. в 17:33, Ivan Daschinsky > >:
>> >>>>>>
>> >>>>>> Dear Igniters!
>> >>>>>>
>> >>>>>> Release candidate binaries for subj are uploaded and ready for vote
>> >>>>>> You can find them here:
>> >>>>>> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc1
>> >>>>>>
>> >>>>>> If you follow the link above, you will find source package
>> (*.tar.gz
>> >>>> and
>> >>>>> *.zip)
>> >>>>>> and binary packages (wheels) for windows (amd64) and linux (x86_64)
>> >>>>>> for pythons 36, 37, 38, 39. Also, there are sha512 and gpg
>> >>> signatures.
>> >>>>>> Code signing keys can be found here --
>> >>>>> https://downloads.apache.org/ignite/KEYS
>> >>>>>> Here you can find instructions how to verify packages
>> >>>>>> https://www.apache.org/info/verification.html
>> >>>>>>
>> >>>>>> You can install binary package for specific version of python using
>> >>> pip
>> >>>>>> For example do this on linux for python 3.8
>> >>>>>>>> pip install pyignite-0.5.0-cp38-cp38-manylinux1_x86_64.whl
>> >>>>>>
>> >>>>>> You can build and install package from source using this command:
>> >>>>>>>> pip install pyignite-0.5.0.tar.gz
>> >>>>>> You can build wheel on your platform using this command:
>> >>>>>>>> pip wheel --no-deps pyignite-0.5.0.tar.gz
>> >>>>>>
>> >>>>>> For building C module, you should have python headers and C
>> compiler
>> >>>>> installed.
>> >>>>>> (i.e. for ubuntu sudo apt install b

Re: [VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

There is a test for it.

пт, 18 июн. 2021 г. в 12:30, Stephen Darlington <
stephen.darling...@gridgain.com>:

> Oh… can someone else check this: it appears that authenticated connections
> fail.
>
> With Ignite 2.10 the connection times-out:
>
> [10:28:58,015][WARNING][grid-timeout-worker-#22][ClientListenerNioListener]
> Unable to perform handshake within timeout [timeout=1, remoteAddr=/
> 127.0.0.1:54044]
>
> Didn’t try this with 0.4.0 so not sure if it’s a regression, but it’s not
> great.
>
> > On 18 Jun 2021, at 09:36, Stephen Darlington <
> stephen.darling...@gridgain.com> wrote:
> >
> > +1
> >
> > Checked on macOS, played with the new expiry APIs and a bunch of
> thefundamentals.
> >
> >> On 17 Jun 2021, at 12:46, Pavel Tupitsyn  wrote:
> >>
> >> +1
> >>
> >> Checked pip install from tar.gz on Python 3.8 on Ubuntu 20.04, ran some
> of
> >> the examples.
> >>
> >> On Thu, Jun 17, 2021 at 2:32 PM Igor Sapego  wrote:
> >>
> >>> +1 from me
> >>>
> >>> Best Regards,
> >>> Igor
> >>>
> >>>
> >>> On Thu, Jun 17, 2021 at 12:10 PM Ivan Daschinsky 
> >>> wrote:
> >>>
> >>>> +1 From me
> >>>> Checked on Ubuntu 20.04 and windows 10
> >>>> 1. Installation from wheels for pythons 3.6 3.7 3.8 3.9
> >>>> 2. Native module work
> >>>> 3. Examples
> >>>>
> >>>> Checked on Ubuntu 20.04 building from source package and correct work
> of
> >>>> result package.
> >>>>
> >>>> Checked all sha256 checksums and gpg signatures.
> >>>>
> >>>> Let's extend voting period till June 18, 15:00 UTC
> >>>>
> >>>>
> >>>>
> >>>> ср, 16 июн. 2021 г. в 17:34, Ivan Daschinsky :
> >>>>
> >>>>> The vote will end at June, 17 15:00 UTC.
> >>>>>
> >>>>> ср, 16 июн. 2021 г. в 17:33, Ivan Daschinsky :
> >>>>>>
> >>>>>> Dear Igniters!
> >>>>>>
> >>>>>> Release candidate binaries for subj are uploaded and ready for vote
> >>>>>> You can find them here:
> >>>>>> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc1
> >>>>>>
> >>>>>> If you follow the link above, you will find source package (*.tar.gz
> >>>> and
> >>>>> *.zip)
> >>>>>> and binary packages (wheels) for windows (amd64) and linux (x86_64)
> >>>>>> for pythons 36, 37, 38, 39. Also, there are sha512 and gpg
> >>> signatures.
> >>>>>> Code signing keys can be found here --
> >>>>> https://downloads.apache.org/ignite/KEYS
> >>>>>> Here you can find instructions how to verify packages
> >>>>>> https://www.apache.org/info/verification.html
> >>>>>>
> >>>>>> You can install binary package for specific version of python using
> >>> pip
> >>>>>> For example do this on linux for python 3.8
> >>>>>>>> pip install pyignite-0.5.0-cp38-cp38-manylinux1_x86_64.whl
> >>>>>>
> >>>>>> You can build and install package from source using this command:
> >>>>>>>> pip install pyignite-0.5.0.tar.gz
> >>>>>> You can build wheel on your platform using this command:
> >>>>>>>> pip wheel --no-deps pyignite-0.5.0.tar.gz
> >>>>>>
> >>>>>> For building C module, you should have python headers and C compiler
> >>>>> installed.
> >>>>>> (i.e. for ubuntu sudo apt install build-essential python3-dev)
> >>>>>> In Mac OS X xcode-tools and python from homebrew are the best
> option.
> >>>>>>
> >>>>>> In order to check whether C module works, use following:
> >>>>>>>> from pyignite import _cutils
> >>>>>>>> print(_cutils.hashcode('test'))
> >>>>>>>> 3556498
> >>>>>>
> >>>>>> You can find documentation here:
> >>>>>>
> >>>>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1
> >>>>>>
> >>>>>> You can find examples here (to check them, you should start ignite
> >>>>> locally):
> >>>>>>
> >>>>>
> >>>>
> >>>
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1/examples.html
> >>>>>> Also, examples can be found in source archive in examples subfolder.
> >>>>>> docker-compose.yml is supplied in order to start ignite quickly.
> (Use
> >>>>>> `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> >>>>>> down` to shut down it)
> >>>>>>
> >>>>>> Release notes:
> >>>>>>
> >>>>>
> >>>>
> >>>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f
> >>>>>>
> >>>>>> Git release tag was created:
> >>>>>>
> >>>>>
> >>>>
> >>>
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc1
> >>>>>>
> >>>>>> The vote is formal, see voting guidelines
> >>>>>> https://www.apache.org/foundation/voting.html
> >>>>>>
> >>>>>> +1 - to accept pyignite-0.5.0-rc1
> >>>>>> 0 - don't care either way
> >>>>>> -1 - DO NOT accept pyignite-0.5.0-rc1
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Sincerely yours, Ivan Daschinskiy
> >>>>
> >>>
> >
> >
>
>
>

-- 
Sincerely yours, Ivan Daschinskiy

Re: IEP-75 Thin Client MsgPack Serialization for 3.0

[Ivan Daschinsky]

>> To make it fair. Ignite uses thread-local reusable buffers, see [1].
I know, but PooledMessageBufferOutput is not about thread-local, isn't it?

I'm not against about MsgPack, I'm for fair and not biased comparison.

I suppose that MsgPack is an ideal candidate for thin client binary
protocol, not only for serializing some user data.
I am analyzed some tarantool connectors [1] [2] [3] and found msgpack based
protocol is a really good idea.
Also there is realy super fast and just 1 header library with liberal BSD-2
licence for C -- msgpuck [4]. It used in tarantool itself
and in [1] and is stable and bullet proof.

[1] -- https://github.com/igorcoding/asynctnt
[2] -- https://github.com/tarantool/tarantool-python/
[3] -- https://github.com/tarantool/go-tarantool
[4] -- https://github.com/rtsisyk/msgpuck

пт, 18 июн. 2021 г. в 11:44, Pavel Tupitsyn :

> Ivan,
>
> > why do you use  PooledMessageBufferOutput in benchmarks?
>
> To make it fair. Ignite uses thread-local reusable buffers, see [1].
>
>
> > why packer from msgpack-core show better performance than
> > BinaryWriter. And I suppose that benchmark is not quite fair.
>
> MsgPack writes and reads less bytes, so it should be faster.
> Benchmark is not 100% fair, there are some small extra things that
> BinaryWriter does.
>
> However:
> 1. I don't think we care about super-precise benchmarks here, just the
> ballpark.
> 2. We are discussing the format, not the implementation.
>
> Important takeaway is:
> The format does not prevent someone from implementing it efficiently.
>
>
>
> [1]
>
> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/binary/BinaryWriterExImpl.java#L101
>
> On Thu, Jun 17, 2021 at 10:40 PM Ivan Daschinsky 
> wrote:
>
> > Andrey, here we discuss serialization format, as far as I understand.
> > Current implementation of ignite binary object serialization can be
> > rewritten.
> > If we do not care about fast (O(1)) field lookup, about schema validation
> > and so on, msgpack is a really good option. It is also good for client
> > binary protocol, i.e.
> > tarantool uses it.
> >
> > >> Binarilizable interface forces user to write serialization code
> > I am talking about speed comparison. You can see from Pavel's data,
> > jackson-msgpack shows a pathetic performance comparing with a ignite's
> > default binary marshaller. If you want really fast serialization -- the
> > only option is to write code by yourself or use code generation. Default
> > packer from msgpack-core java package is similar to BinaryWriter. So I am
> > wondering why packer from msgpack-core show better performance than
> > BinaryWriter. And I suppose that benchmark is not quite fair.
> >
> >
> > чт, 17 июн. 2021 г. в 22:19, Andrey Mashenkov <
> andrey.mashen...@gmail.com
> > >:
> >
> > > Ivan, thankd for clarification.
> > >
> > > Binarilizable interface forces user to write serialization code. We can
> > > support this or similar interface.
> > > But I'd like Ignite has some default serializer in addition. It can be
> > also
> > > useful e.g. in compute for param and result serialization.
> > >
> > > BinaryObjectBuider requires an Ignite node for object construction, but
> > we
> > > are looking for a detached builder and won't care about schemas.
> > >
> > > AFAIR, BinaryObject creates an objectReader on every single field read
> > > operation.
> > > So, BO solution produces a lot of garbage and BO has noticable overhead
> > > which affects the object footprint.
> > >
> > > чт, 17 июн. 2021 г., 21:41 Ivan Daschinsky :
> > >
> > > > >> Double checked -- there is not any links to PR either in IEP or in
> > > jira
> > > > issue
> > > > Sorry, there is a link in IEP, but not in jira ticket.
> > > >
> > > > чт, 17 июн. 2021 г. в 21:39, Ivan Daschinsky :
> > > >
> > > > > Andrey,
> > > > > >> arbitrary object graph
> > > > > Also, that is not true, msgpack format doesn't handle circular
> > graphs.
> > > > > Think about msgpack as binary json. You couldn't understand full
> > > > structure
> > > > > of message if you didn't deserialize it fully before, maps and
> arrays
> > > are
> > > > > serialized just as contiguos chunks
> > > > >  of values/kv-pairs. Msgpack is a really dumb and simple format.
> > > > >
> >

Re: IEP-75 Thin Client MsgPack Serialization for 3.0

[Ivan Daschinsky]

Andrey, here we discuss serialization format, as far as I understand.
Current implementation of ignite binary object serialization can be
rewritten.
If we do not care about fast (O(1)) field lookup, about schema validation
and so on, msgpack is a really good option. It is also good for client
binary protocol, i.e.
tarantool uses it.

>> Binarilizable interface forces user to write serialization code
I am talking about speed comparison. You can see from Pavel's data,
jackson-msgpack shows a pathetic performance comparing with a ignite's
default binary marshaller. If you want really fast serialization -- the
only option is to write code by yourself or use code generation. Default
packer from msgpack-core java package is similar to BinaryWriter. So I am
wondering why packer from msgpack-core show better performance than
BinaryWriter. And I suppose that benchmark is not quite fair.


чт, 17 июн. 2021 г. в 22:19, Andrey Mashenkov :

> Ivan, thankd for clarification.
>
> Binarilizable interface forces user to write serialization code. We can
> support this or similar interface.
> But I'd like Ignite has some default serializer in addition. It can be also
> useful e.g. in compute for param and result serialization.
>
> BinaryObjectBuider requires an Ignite node for object construction, but we
> are looking for a detached builder and won't care about schemas.
>
> AFAIR, BinaryObject creates an objectReader on every single field read
> operation.
> So, BO solution produces a lot of garbage and BO has noticable overhead
> which affects the object footprint.
>
> чт, 17 июн. 2021 г., 21:41 Ivan Daschinsky :
>
> > >> Double checked -- there is not any links to PR either in IEP or in
> jira
> > issue
> > Sorry, there is a link in IEP, but not in jira ticket.
> >
> > чт, 17 июн. 2021 г. в 21:39, Ivan Daschinsky :
> >
> > > Andrey,
> > > >> arbitrary object graph
> > > Also, that is not true, msgpack format doesn't handle circular graphs.
> > > Think about msgpack as binary json. You couldn't understand full
> > structure
> > > of message if you didn't deserialize it fully before, maps and arrays
> are
> > > serialized just as contiguos chunks
> > >  of values/kv-pairs. Msgpack is a really dumb and simple format.
> > >
> > > Also, as for me, I cannot understand why current ignite serialization
> > > (BinaryObjectBuilder or Binarilizable) is slower than raw message pack
> > > serializer.
> > > I suppose that this is an issue and we should investigate it.
> > >
> > > Pavel,  why do you use  PooledMessageBufferOutput in benchmarks? I'm
> > > sorry, but is it fair to use it?
> > >
> > > >> The code is linked in the IEP [2]
> > > Double checked -- there is not any links to PR either in IEP or in jira
> > > issue
> > >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: IEP-75 Thin Client MsgPack Serialization for 3.0

[Ivan Daschinsky]

>> Double checked -- there is not any links to PR either in IEP or in jira
issue
Sorry, there is a link in IEP, but not in jira ticket.

чт, 17 июн. 2021 г. в 21:39, Ivan Daschinsky :

> Andrey,
> >> arbitrary object graph
> Also, that is not true, msgpack format doesn't handle circular graphs.
> Think about msgpack as binary json. You couldn't understand full structure
> of message if you didn't deserialize it fully before, maps and arrays are
> serialized just as contiguos chunks
>  of values/kv-pairs. Msgpack is a really dumb and simple format.
>
> Also, as for me, I cannot understand why current ignite serialization
> (BinaryObjectBuilder or Binarilizable) is slower than raw message pack
> serializer.
> I suppose that this is an issue and we should investigate it.
>
> Pavel,  why do you use  PooledMessageBufferOutput in benchmarks? I'm
> sorry, but is it fair to use it?
>
> >> The code is linked in the IEP [2]
> Double checked -- there is not any links to PR either in IEP or in jira
> issue
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: IEP-75 Thin Client MsgPack Serialization for 3.0

[Ivan Daschinsky]

Andrey,
>> arbitrary object graph
Also, that is not true, msgpack format doesn't handle circular graphs.
Think about msgpack as binary json. You couldn't understand full structure
of message if you didn't deserialize it fully before, maps and arrays are
serialized just as contiguos chunks
 of values/kv-pairs. Msgpack is a really dumb and simple format.

Also, as for me, I cannot understand why current ignite serialization
(BinaryObjectBuilder or Binarilizable) is slower than raw message pack
serializer.
I suppose that this is an issue and we should investigate it.

Pavel,  why do you use  PooledMessageBufferOutput in benchmarks? I'm sorry,
but is it fair to use it?

>> The code is linked in the IEP [2]
Double checked -- there is not any links to PR either in IEP or in jira
issue

Re: IEP-75 Thin Client MsgPack Serialization for 3.0

[Ivan Daschinsky]

Andrey, i'm sorry but what do you mean as additional code harness? Usually,
POJO is serialized simply as map.

чт, 17 июн. 2021 г., 19:55 Andrey Mashenkov :

> Hi Pavel,
>
> What you suggest looks promising: arbitrary object graph and platform
> independence aspects in particular.
>
> In IEP-54 we support only flat objects and only some standard types and
> assume inner objects of custom types will be serialized to byte[] somehow
> and their schema will not be managed by Ignite.
> And, we want to offer some default serializers to make it easier for
> end-user.
>
> AFAIU, MsgPack is suitable for the purpose as we don't want to invent yet
> another effective binary format.
> With an additional code harness, we can write object schema (field names)
> within the object itself.
> Is it right?
>
> I am just confused with "a schemaful way" and "field names are not
> included" in the same sentence.
>
>
> On Thu, Jun 17, 2021 at 4:46 PM Ivan Daschinsky 
> wrote:
>
> > Also, it's well known use case of msgpack in the world of memory grids --
> > tarantool.io uses msgpack for clients binary protocol [1]
> > So writing connectors to tarantool is quite easy task.
> >
> > [1] --
> > https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/
> >
> > чт, 17 июн. 2021 г. в 16:15, Pavel Tupitsyn :
> >
> > > Ivan,
> > >
> > > > Have you considered format with schema?
> > >
> > > 1. We should be able to serialize arbitrary user data on the client
> side.
> > > I think we don't want to require extra steps from the user.
> > >
> > > 2. MsgPack can be also used in a schemaful way, when user objects are
> > > written as arrays, not as maps - so that field names are not included.
> > >
> > >
> > > > we should benchmark formats thoroughly
> > >
> > > Strictly speaking, the IEP is about the format (the spec), not about
> the
> > > implementation.
> > > The format itself is simple and efficient, there is nothing to make it
> > > slower than anything else.
> > > C# impl proves this by beating every competitor [1].
> > >
> > >
> > > > Could you please share your code for benchmarks?
> > >
> > > The code is linked in the IEP [2]
> > >
> > >
> > > [1]
> > >
> > >
> >
> https://aloiskraus.wordpress.com/2019/09/29/net-serialization-benchmark-2019-roundup/
> > > [2] https://github.com/apache/ignite/pull/9178
> > >
> > > On Thu, Jun 17, 2021 at 4:02 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Could you please share your code for benchmarks?
> > > >
> > > > чт, 17 июн. 2021 г. в 15:56, Ivan Daschinsky :
> > > >
> > > > > Hi, Pavel. Have you considered format with schema? Or schemaless
> of a
> > > > > candidate format was  a prerequisite?
> > > > > As for me, msgpack is great, but I suppose that we should benchmark
> > > > > formats thoroughly. And not only for Java.
> > > > >
> > > > > чт, 17 июн. 2021 г. в 15:29, Pavel Tupitsyn  >:
> > > > >
> > > > >> Igniters,
> > > > >>
> > > > >> I have drafted an IEP on thin client serialization format [1],
> > > > >> please review and let me know what you think.
> > > > >>
> > > > >> [1]
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-75+Thin+Client+Serialization
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Sincerely yours, Ivan Daschinskiy
> > > > >
> > > >
> > > >
> > > > --
> > > > Sincerely yours, Ivan Daschinskiy
> > > >
> > >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>

Re: IEP-75 Thin Client MsgPack Serialization for 3.0

[Ivan Daschinsky]

Also, it's well known use case of msgpack in the world of memory grids --
tarantool.io uses msgpack for clients binary protocol [1]
So writing connectors to tarantool is quite easy task.

[1] --
https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/

чт, 17 июн. 2021 г. в 16:15, Pavel Tupitsyn :

> Ivan,
>
> > Have you considered format with schema?
>
> 1. We should be able to serialize arbitrary user data on the client side.
> I think we don't want to require extra steps from the user.
>
> 2. MsgPack can be also used in a schemaful way, when user objects are
> written as arrays, not as maps - so that field names are not included.
>
>
> > we should benchmark formats thoroughly
>
> Strictly speaking, the IEP is about the format (the spec), not about the
> implementation.
> The format itself is simple and efficient, there is nothing to make it
> slower than anything else.
> C# impl proves this by beating every competitor [1].
>
>
> > Could you please share your code for benchmarks?
>
> The code is linked in the IEP [2]
>
>
> [1]
>
> https://aloiskraus.wordpress.com/2019/09/29/net-serialization-benchmark-2019-roundup/
> [2] https://github.com/apache/ignite/pull/9178
>
> On Thu, Jun 17, 2021 at 4:02 PM Ivan Daschinsky 
> wrote:
>
> > Could you please share your code for benchmarks?
> >
> > чт, 17 июн. 2021 г. в 15:56, Ivan Daschinsky :
> >
> > > Hi, Pavel. Have you considered format with schema? Or schemaless of a
> > > candidate format was  a prerequisite?
> > > As for me, msgpack is great, but I suppose that we should benchmark
> > > formats thoroughly. And not only for Java.
> > >
> > > чт, 17 июн. 2021 г. в 15:29, Pavel Tupitsyn :
> > >
> > >> Igniters,
> > >>
> > >> I have drafted an IEP on thin client serialization format [1],
> > >> please review and let me know what you think.
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-75+Thin+Client+Serialization
> > >>
> > >
> > >
> > > --
> > > Sincerely yours, Ivan Daschinskiy
> > >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: IEP-75 Thin Client MsgPack Serialization for 3.0

[Ivan Daschinsky]

Could you please share your code for benchmarks?

чт, 17 июн. 2021 г. в 15:56, Ivan Daschinsky :

> Hi, Pavel. Have you considered format with schema? Or schemaless of a
> candidate format was  a prerequisite?
> As for me, msgpack is great, but I suppose that we should benchmark
> formats thoroughly. And not only for Java.
>
> чт, 17 июн. 2021 г. в 15:29, Pavel Tupitsyn :
>
>> Igniters,
>>
>> I have drafted an IEP on thin client serialization format [1],
>> please review and let me know what you think.
>>
>> [1]
>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-75+Thin+Client+Serialization
>>
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: IEP-75 Thin Client MsgPack Serialization for 3.0

[Ivan Daschinsky]

Hi, Pavel. Have you considered format with schema? Or schemaless of a
candidate format was  a prerequisite?
As for me, msgpack is great, but I suppose that we should benchmark formats
thoroughly. And not only for Java.

чт, 17 июн. 2021 г. в 15:29, Pavel Tupitsyn :

> Igniters,
>
> I have drafted an IEP on thin client serialization format [1],
> please review and let me know what you think.
>
> [1]
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-75+Thin+Client+Serialization
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: [VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

+1 From me
Checked on Ubuntu 20.04 and windows 10
1. Installation from wheels for pythons 3.6 3.7 3.8 3.9
2. Native module work
3. Examples

Checked on Ubuntu 20.04 building from source package and correct work of
result package.

Checked all sha256 checksums and gpg signatures.

Let's extend voting period till June 18, 15:00 UTC



ср, 16 июн. 2021 г. в 17:34, Ivan Daschinsky :

> The vote will end at June, 17 15:00 UTC.
>
> ср, 16 июн. 2021 г. в 17:33, Ivan Daschinsky :
> >
> > Dear Igniters!
> >
> > Release candidate binaries for subj are uploaded and ready for vote
> > You can find them here:
> > https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc1
> >
> > If you follow the link above, you will find source package (*.tar.gz and
> *.zip)
> > and binary packages (wheels) for windows (amd64) and linux (x86_64)
> > for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
> > Code signing keys can be found here --
> https://downloads.apache.org/ignite/KEYS
> > Here you can find instructions how to verify packages
> > https://www.apache.org/info/verification.html
> >
> > You can install binary package for specific version of python using pip
> > For example do this on linux for python 3.8
> > >> pip install pyignite-0.5.0-cp38-cp38-manylinux1_x86_64.whl
> >
> > You can build and install package from source using this command:
> > >> pip install pyignite-0.5.0.tar.gz
> > You can build wheel on your platform using this command:
> > >> pip wheel --no-deps pyignite-0.5.0.tar.gz
> >
> > For building C module, you should have python headers and C compiler
> installed.
> > (i.e. for ubuntu sudo apt install build-essential python3-dev)
> > In Mac OS X xcode-tools and python from homebrew are the best option.
> >
> > In order to check whether C module works, use following:
> > >> from pyignite import _cutils
> > >> print(_cutils.hashcode('test'))
> > >> 3556498
> >
> > You can find documentation here:
> > https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1
> >
> > You can find examples here (to check them, you should start ignite
> locally):
> >
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1/examples.html
> > Also, examples can be found in source archive in examples subfolder.
> > docker-compose.yml is supplied in order to start ignite quickly. (Use
> > `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> > down` to shut down it)
> >
> > Release notes:
> >
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f
> >
> > Git release tag was created:
> >
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc1
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept pyignite-0.5.0-rc1
> > 0 - don't care either way
> > -1 - DO NOT accept pyignite-0.5.0-rc1
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: [DISCUSSION] Code style. Variable abbrevations

[Ivan Daschinsky]

I'm sorry, but the rule is not strict and, moreover, not clear enough for
constans. See here [1]
```
Type and method names are usually not abbreviated (except for the
well-accepted abbreviations such as EOF, Impl, Fifo, etc.).

Abbreviation applies to local variables, method parameters, class fields
and in **some cases public static fileds** (constants) of the class.
```
But there is not any list that contains these cases. I suppose, that
constant naming should not be abbreviated.
As I see, the most cases of violations, described in initial post, are
about constant's namings.

I suppose that we should define strict and clear rules about constants
naming.

[1] -- https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules


чт, 17 июн. 2021 г. в 10:10, Konstantin Orlov :

> Enforced validation doesn't guarantee code consistency. No validation in
> the world prevent me from typing manually "mess" instead of "msg"/"message"
> or "svc" instead of "srvc"/"service" (btw "svc" is more common imo).
>
> And the fact that someone name a variable "count" instead of "cnt" doesn't
> make the whole project immature.
>
> --
> Regards,
> Konstantin Orlov
>
>
>
>
> > On 17 Jun 2021, at 08:34, Ivan Pavlukhin  wrote:
> >
> > Hi Nikolay,
> >
> > Thanks, it is rather interesting.
> >
> > Do we all agree that using different conventions for different code
> > packages does not break "consistency"? Or did I get something wrong?
> >
> > 2021-06-17 7:12 GMT+03:00, Николай Ижиков :
> >> Hello, Ivan.
> >>
> >> We can create checkstyle rule to enforce usage of abbreviations.
> >> Internal/public code differs by package.
> >>
> >> PoC of rule [1]
> >>
> >> [1] https://github.com/apache/ignite/pull/9153
> >>
> >>> 17 июня 2021 г., в 07:01, Ivan Pavlukhin 
> >>> написал(а):
> >>>
> >>> Nikita,
> >>>
> >>> Do you have a plan in your mind how to make Ignite codebase consistent?
> >>>
> >>> AFAIR, we had it intentionally inconsistent for a long time at least
> >>> for one sake: for internal code we used special conventions (e.g.
> >>> abbreviations, shorten getters) and common Java conventions for public
> >>> API and examples (e.g. no abbreviations and usual getters).
> >>>
> >>> 2021-06-16 23:37 GMT+03:00, Nikita Ivanov :
>  Consistency is what makes it easier to contribute to the project and
>  attract new members. Consistency implies strong, well defined and
>  universally enforced rules. Just because we have some individuals who
>  are lazy or inexperienced - it does not mean that the entire project
>  should relax the basic-level engineering discipline.
> 
>  On a personal node - nothing screams "immaturity" louder that a code
>  that uses inconsistent naming, commenting, code style & organization,
>  etc.
>  --
>  Nikita Ivanov
> 
> > On Wed, Jun 16, 2021 at 5:56 AM Andrey Gura 
> wrote:
> >
> > Hi,
> >
> > I understand that this rule seems too strict or may be weird. But
> > removing the rule could lead to review comments like "use future
> > instead of fut". So my proposal is to change rule from "required" to
> > "recommended".
> >
> > On Wed, Jun 16, 2021 at 2:49 PM Valentin Kulichenko
> >  wrote:
> >>
> >> Konstantin, thanks for chiming in.
> >>
> >> That's exactly my concern. Overformalization is generally not a good
> >> thing.
> >> Someone used "mess" to abbreviate "message"? That is surely not a
> good
> >> coding style, but that's what code reviews are for. I believe that
> our
> >> committers are more than capable of identifying such issues.
> >>
> >> At the same time, with the current rules, we are *forced* to use
> >> abbreviations like "locAddrGrpMgr", whether we like it or not. All
> it
> >> does
> >> is makes it harder to contribute to Ignite, without providing any
> >> clear
> >> value.
> >>
> >> -Val
> >>
> >> On Thu, Jun 10, 2021 at 9:46 AM Konstantin Orlov <
> kor...@gridgain.com>
> >> wrote:
> >>
> >>> +1 for making this optional
> >>>
> >>> Common abbreviation rules is good for sure, but sometimes I getting
> >>> sick
> >>> of all those locAddrGrpMgr.
> >>>
> >>>
> >>> --
> >>> Regards,
> >>> Konstantin Orlov
> >>>
> >>>
> >>>
> >>>
>  On 7 Jun 2021, at 14:31, Nikolay Izhikov 
>  wrote:
> 
>  Hello, Anton, Alexei.
> 
>  Thanks for the feedback.
> 
>  Personally, I’m pretty happy current abbreviation rules too.
>  Let see what we can do to make our codebase even more consistent.
> 
> > 7 июня 2021 г., в 13:23, Alexei Scherbakov <
> >>> alexey.scherbak...@gmail.com> написал(а):
> >
> > -1
> > Common abbrevs add quality to the code.
> >
> > пн, 7 июн. 2021 г. в 12:38, Anton Vinogradov :
> >
> >> -1 here.
> >> We can fix the

Re: [VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

The vote will end at June, 17 15:00 UTC.

ср, 16 июн. 2021 г. в 17:33, Ivan Daschinsky :
>
> Dear Igniters!
>
> Release candidate binaries for subj are uploaded and ready for vote
> You can find them here:
> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc1
>
> If you follow the link above, you will find source package (*.tar.gz and 
> *.zip)
> and binary packages (wheels) for windows (amd64) and linux (x86_64)
> for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
> Code signing keys can be found here -- 
> https://downloads.apache.org/ignite/KEYS
> Here you can find instructions how to verify packages
> https://www.apache.org/info/verification.html
>
> You can install binary package for specific version of python using pip
> For example do this on linux for python 3.8
> >> pip install pyignite-0.5.0-cp38-cp38-manylinux1_x86_64.whl
>
> You can build and install package from source using this command:
> >> pip install pyignite-0.5.0.tar.gz
> You can build wheel on your platform using this command:
> >> pip wheel --no-deps pyignite-0.5.0.tar.gz
>
> For building C module, you should have python headers and C compiler 
> installed.
> (i.e. for ubuntu sudo apt install build-essential python3-dev)
> In Mac OS X xcode-tools and python from homebrew are the best option.
>
> In order to check whether C module works, use following:
> >> from pyignite import _cutils
> >> print(_cutils.hashcode('test'))
> >> 3556498
>
> You can find documentation here:
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1
>
> You can find examples here (to check them, you should start ignite locally):
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1/examples.html
> Also, examples can be found in source archive in examples subfolder.
> docker-compose.yml is supplied in order to start ignite quickly. (Use
> `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> down` to shut down it)
>
> Release notes:
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f
>
> Git release tag was created:
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc1
>
> The vote is formal, see voting guidelines
> https://www.apache.org/foundation/voting.html
>
> +1 - to accept pyignite-0.5.0-rc1
> 0 - don't care either way
> -1 - DO NOT accept pyignite-0.5.0-rc1

[VOTE] Release pyignite 0.5.0-rc1

[Ivan Daschinsky]

Dear Igniters!

Release candidate binaries for subj are uploaded and ready for vote
You can find them here:
https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc1

If you follow the link above, you will find source package (*.tar.gz and *.zip)
and binary packages (wheels) for windows (amd64) and linux (x86_64)
for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
Code signing keys can be found here -- https://downloads.apache.org/ignite/KEYS
Here you can find instructions how to verify packages
https://www.apache.org/info/verification.html

You can install binary package for specific version of python using pip
For example do this on linux for python 3.8
>> pip install pyignite-0.5.0-cp38-cp38-manylinux1_x86_64.whl

You can build and install package from source using this command:
>> pip install pyignite-0.5.0.tar.gz
You can build wheel on your platform using this command:
>> pip wheel --no-deps pyignite-0.5.0.tar.gz

For building C module, you should have python headers and C compiler installed.
(i.e. for ubuntu sudo apt install build-essential python3-dev)
In Mac OS X xcode-tools and python from homebrew are the best option.

In order to check whether C module works, use following:
>> from pyignite import _cutils
>> print(_cutils.hashcode('test'))
>> 3556498

You can find documentation here:
https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1

You can find examples here (to check them, you should start ignite locally):
https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc1/examples.html
Also, examples can be found in source archive in examples subfolder.
docker-compose.yml is supplied in order to start ignite quickly. (Use
`docker-compose up -d` to start 3 nodes cluster and `docker-compose
down` to shut down it)

Release notes:
https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f

Git release tag was created:
https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc1

The vote is formal, see voting guidelines
https://www.apache.org/foundation/voting.html

+1 - to accept pyignite-0.5.0-rc1
0 - don't care either way
-1 - DO NOT accept pyignite-0.5.0-rc1

[CANCEL][VOTE] Release pyignite 0.5.0-rc0

[Ivan Daschinsky]

This vote was canceled since it was decided to add [1] to the release.
[1] has been already merged to master branch and cherrypicked to
release branch. New vote will be created soon.

[1]  -- https://issues.apache.org/jira/browse/IGNITE-14911

Re: [VOTE] Release pyignite 0.5.0-rc0

[Ivan Daschinsky]

Yes, I agree, after merging
https://issues.apache.org/jira/browse/IGNITE-14911 that resolves this
issue, I suggests to start a new vote.

ср, 16 июн. 2021 г. в 15:54, Igor Sapego :
>
> I propose to cancel this release and fix the issue which was highlighted in
> the
> "Seconds and milliseconds confusion in python thin client" thread.
>
> WDYT?
>
> Best Regards,
> Igor
>
>
> On Wed, Jun 16, 2021 at 12:10 AM Igor Sapego  wrote:
>
> > +1 from me
> >
> > Uploaded to test.pipy.org: https://test.pypi.org/project/pyignite/0.5.0/
> >
> > Everything looks good.
> >
> > Best Regards,
> > Igor
> >
> >
> > On Tue, Jun 15, 2021 at 10:09 PM Ivan Daschinsky 
> > wrote:
> >
> >> Also checked hash sums and signature. Packages are verified and
> >> signature is OK, signed by Igor Sapego with key
> >> 5C10A0722D947727923C98B5AF35DBD958FE8DC5
> >> Keys can be found here -- https://downloads.apache.org/ignite/KEYS
> >> Here you can find instructions how to verify packages
> >> https://www.apache.org/info/verification.html
> >>
> >> вт, 15 июн. 2021 г. в 22:01, Ivan Daschinsky :
> >> >
> >> > +1 From me
> >> > Checked building from source on Ubuntu 20.04 amd64 for pythons 3.6.12
> >> > 3.7.9 3.8.6 3.9.1.
> >> > Checked installing binary packages on Ubuntu 20.04 amd64 and Windows
> >> > 10 amd 64 for pythons 3.6.12 3.7.9 3.8.6 3.9.1.
> >> > Run examples for both platforms
> >> > Check that native module works
> >> > Checked documentation for tag 0.5.0.rc0 on readthedocs.io
> >> >
> >> > вт, 15 июн. 2021 г. в 21:58, Ivan Daschinsky :
> >> > >
> >> > > Dear Igniters!
> >> > >
> >> > > Release candidate binaries for subj are uploaded and ready for vote
> >> > > You can find them here:
> >> > > https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc0
> >> > >
> >> > > If you follow the link above, you will find source package (*.tar.gz
> >> and *.zip)
> >> > > and binary packages (wheels) for windows (amd64) and linux (x86_64)
> >> > > for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
> >> > >
> >> > > You can install binary package for specific version of python using
> >> pip
> >> > > For example do this on linux for python 3.8
> >> > > >> pip install pyignite-0.4.0-cp38-cp38-manylinux1_x86_64.whl
> >> > >
> >> > > You can build and install package from source using this command:
> >> > > >> pip install pyignite-0.4.0.tar.gz
> >> > > You can build wheel on your platform using this command:
> >> > > >> pip wheel --no-deps pyignite-0.4.0.tar.gz
> >> > >
> >> > > For building C module, you should have python headers and C compiler
> >> installed.
> >> > > (i.e. for ubuntu sudo apt install build-essential python3-dev)
> >> > > In Mac OS X xcode-tools and python from homebrew are the best option.
> >> > >
> >> > > In order to check whether C module works, use following:
> >> > > >> from pyignite import _cutils
> >> > > >> print(_cutils.hashcode('test'))
> >> > > >> 3556498
> >> > >
> >> > > You can find documentation here:
> >> > >
> >> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0
> >> > >
> >> > > You can find examples here (to check them, you should start ignite
> >> locally):
> >> > >
> >> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0/examples.html
> >> > > Also, examples can be found in source archive in examples subfolder.
> >> > > docker-compose.yml is supplied in order to start ignite quickly. (Use
> >> > > `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> >> > > down` to shut down it)
> >> > >
> >> > > Release notes:
> >> > >
> >> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f
> >> > >
> >> > > Git release tag was created:
> >> > >
> >> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc0
> >> > >
> >> > > The vote is formal, see voting guidelines
> >> > > https://www.apache.org/foundation/voting.html
> >> > >
> >> > > +1 - to accept pyignite-0.5.0-rc0
> >> > > 0 - don't care either way
> >> > > -1 - DO NOT accept pyignite-0.5.0-rc0
> >> > >
> >> > > The vote will end at June, 17 15:00 UTC.
> >> >
> >> >
> >> >
> >> > --
> >> > Sincerely yours, Ivan Daschinskiy
> >>
> >>
> >>
> >> --
> >> Sincerely yours, Ivan Daschinskiy
> >>
> >



-- 
Sincerely yours, Ivan Daschinskiy

Re: Seconds and milliseconds confusion in python thin client

[Ivan Daschinsky]

I have a compromise variant.

1. Large timeouts are set usually only for expire_policy. I suggest to
support datetime.timedelta here and int as milliseconds
2. All others timeouts should accept only ints as milliseconds.
3. Only timeout in Connection as sockettimeout should remain float in
seconds, because it corresponds to socket.settimeout method in python
(accepts floating point number)

This is a compromise between usability and backward compatibility

WDYT?

ср, 16 июн. 2021 г. в 11:09, Ivan Daschinsky :
>
> I've created ticket for it https://issues.apache.org/jira/browse/IGNITE-14911
>
> ср, 16 июн. 2021 г. в 08:37, Ivan Daschinsky :
> >
> > Ops, i don't even know about it. I believe that this is so rarely used, i 
> > don't even noticed it. I am talking about transactions and expiry policy. I 
> > suppose that in the case of sql we can simply change it.
> >
> > ср, 16 июн. 2021 г., 00:46 Igor Sapego :
> >>
> >> Why is it not released?
> >>
> >> I can see client.sql(timeout) in 0.4.0 for example, which is int number of
> >> ms.
> >>
> >> Best Regards,
> >> Igor
> >>
> >>
> >> On Tue, Jun 15, 2021 at 11:52 PM Ivan Daschinsky 
> >> wrote:
> >>
> >> > BTW, common approach is to treat both ints and floats as seconds. Floats
> >> > are used to set timeout with millisecods precision. I.e. 
> >> > asyncio.sleep(1.0)
> >> > and asyncio.sleep(1) pauses coroutine for 1 sec. Lets create ticket for 
> >> > it,
> >> > stop voting for 0.5.0.rc0 and schedule next vote.
> >> >
> >> > вт, 15 июн. 2021 г., 23:49 Ivan Daschinsky :
> >> >
> >> > > Igor, I suppose that you are probably right. But there is no need to
> >> > > notice or deprecate something. This functionality is not released yet
> >> > >
> >> > > вт, 15 июн. 2021 г., 23:41 Igor Sapego :
> >> > >
> >> > >> Hi Igniters,
> >> > >>
> >> > >> I've noticed a weird behaviour of python thin client. In those places
> >> > >> where
> >> > >> we have
> >> > >> timeouts or any other parameters that take time in some places we 
> >> > >> treat
> >> > it
> >> > >> like integer
> >> > >> number of milliseconds, in others it can take both floats (as a number
> >> > of
> >> > >> seconds)
> >> > >> and ints (number of milliseconds). This approach looks very confusing 
> >> > >> to
> >> > >> me
> >> > >> as
> >> > >> it leads to things where tx_start(1) and tx_start(1.0) are not 
> >> > >> actually
> >> > >> the
> >> > >> same thing.
> >> > >>
> >> > >> AFAIK in python the most common way to pass time to such functions is 
> >> > >> to
> >> > >> use floats
> >> > >> as a number of seconds. This is the approach I propose to use in our 
> >> > >> API
> >> > >> as
> >> > >> well. Let's
> >> > >> deprecate usage of ints in those functions with the appropriate 
> >> > >> warning
> >> > >> before getting rid
> >> > >> of it.
> >> > >>
> >> > >> What do you think?
> >> > >>
> >> > >> Best Regards,
> >> > >> Igor
> >> > >>
> >> > >
> >> >
>
>
>
> --
> Sincerely yours, Ivan Daschinskiy



-- 
Sincerely yours, Ivan Daschinskiy

Re: Seconds and milliseconds confusion in python thin client

[Ivan Daschinsky]

I've created ticket for it https://issues.apache.org/jira/browse/IGNITE-14911

ср, 16 июн. 2021 г. в 08:37, Ivan Daschinsky :
>
> Ops, i don't even know about it. I believe that this is so rarely used, i 
> don't even noticed it. I am talking about transactions and expiry policy. I 
> suppose that in the case of sql we can simply change it.
>
> ср, 16 июн. 2021 г., 00:46 Igor Sapego :
>>
>> Why is it not released?
>>
>> I can see client.sql(timeout) in 0.4.0 for example, which is int number of
>> ms.
>>
>> Best Regards,
>> Igor
>>
>>
>> On Tue, Jun 15, 2021 at 11:52 PM Ivan Daschinsky 
>> wrote:
>>
>> > BTW, common approach is to treat both ints and floats as seconds. Floats
>> > are used to set timeout with millisecods precision. I.e. asyncio.sleep(1.0)
>> > and asyncio.sleep(1) pauses coroutine for 1 sec. Lets create ticket for it,
>> > stop voting for 0.5.0.rc0 and schedule next vote.
>> >
>> > вт, 15 июн. 2021 г., 23:49 Ivan Daschinsky :
>> >
>> > > Igor, I suppose that you are probably right. But there is no need to
>> > > notice or deprecate something. This functionality is not released yet
>> > >
>> > > вт, 15 июн. 2021 г., 23:41 Igor Sapego :
>> > >
>> > >> Hi Igniters,
>> > >>
>> > >> I've noticed a weird behaviour of python thin client. In those places
>> > >> where
>> > >> we have
>> > >> timeouts or any other parameters that take time in some places we treat
>> > it
>> > >> like integer
>> > >> number of milliseconds, in others it can take both floats (as a number
>> > of
>> > >> seconds)
>> > >> and ints (number of milliseconds). This approach looks very confusing to
>> > >> me
>> > >> as
>> > >> it leads to things where tx_start(1) and tx_start(1.0) are not actually
>> > >> the
>> > >> same thing.
>> > >>
>> > >> AFAIK in python the most common way to pass time to such functions is to
>> > >> use floats
>> > >> as a number of seconds. This is the approach I propose to use in our API
>> > >> as
>> > >> well. Let's
>> > >> deprecate usage of ints in those functions with the appropriate warning
>> > >> before getting rid
>> > >> of it.
>> > >>
>> > >> What do you think?
>> > >>
>> > >> Best Regards,
>> > >> Igor
>> > >>
>> > >
>> >



-- 
Sincerely yours, Ivan Daschinskiy

Re: Seconds and milliseconds confusion in python thin client

[Ivan Daschinsky]

Ops, i don't even know about it. I believe that this is so rarely used, i
don't even noticed it. I am talking about transactions and expiry policy. I
suppose that in the case of sql we can simply change it.

ср, 16 июн. 2021 г., 00:46 Igor Sapego :

> Why is it not released?
>
> I can see client.sql(timeout) in 0.4.0 for example, which is int number of
> ms.
>
> Best Regards,
> Igor
>
>
> On Tue, Jun 15, 2021 at 11:52 PM Ivan Daschinsky 
> wrote:
>
> > BTW, common approach is to treat both ints and floats as seconds. Floats
> > are used to set timeout with millisecods precision. I.e.
> asyncio.sleep(1.0)
> > and asyncio.sleep(1) pauses coroutine for 1 sec. Lets create ticket for
> it,
> > stop voting for 0.5.0.rc0 and schedule next vote.
> >
> > вт, 15 июн. 2021 г., 23:49 Ivan Daschinsky :
> >
> > > Igor, I suppose that you are probably right. But there is no need to
> > > notice or deprecate something. This functionality is not released yet
> > >
> > > вт, 15 июн. 2021 г., 23:41 Igor Sapego :
> > >
> > >> Hi Igniters,
> > >>
> > >> I've noticed a weird behaviour of python thin client. In those places
> > >> where
> > >> we have
> > >> timeouts or any other parameters that take time in some places we
> treat
> > it
> > >> like integer
> > >> number of milliseconds, in others it can take both floats (as a number
> > of
> > >> seconds)
> > >> and ints (number of milliseconds). This approach looks very confusing
> to
> > >> me
> > >> as
> > >> it leads to things where tx_start(1) and tx_start(1.0) are not
> actually
> > >> the
> > >> same thing.
> > >>
> > >> AFAIK in python the most common way to pass time to such functions is
> to
> > >> use floats
> > >> as a number of seconds. This is the approach I propose to use in our
> API
> > >> as
> > >> well. Let's
> > >> deprecate usage of ints in those functions with the appropriate
> warning
> > >> before getting rid
> > >> of it.
> > >>
> > >> What do you think?
> > >>
> > >> Best Regards,
> > >> Igor
> > >>
> > >
> >
>

Re: Seconds and milliseconds confusion in python thin client

[Ivan Daschinsky]

BTW, common approach is to treat both ints and floats as seconds. Floats
are used to set timeout with millisecods precision. I.e. asyncio.sleep(1.0)
and asyncio.sleep(1) pauses coroutine for 1 sec. Lets create ticket for it,
stop voting for 0.5.0.rc0 and schedule next vote.

вт, 15 июн. 2021 г., 23:49 Ivan Daschinsky :

> Igor, I suppose that you are probably right. But there is no need to
> notice or deprecate something. This functionality is not released yet
>
> вт, 15 июн. 2021 г., 23:41 Igor Sapego :
>
>> Hi Igniters,
>>
>> I've noticed a weird behaviour of python thin client. In those places
>> where
>> we have
>> timeouts or any other parameters that take time in some places we treat it
>> like integer
>> number of milliseconds, in others it can take both floats (as a number of
>> seconds)
>> and ints (number of milliseconds). This approach looks very confusing to
>> me
>> as
>> it leads to things where tx_start(1) and tx_start(1.0) are not actually
>> the
>> same thing.
>>
>> AFAIK in python the most common way to pass time to such functions is to
>> use floats
>> as a number of seconds. This is the approach I propose to use in our API
>> as
>> well. Let's
>> deprecate usage of ints in those functions with the appropriate warning
>> before getting rid
>> of it.
>>
>> What do you think?
>>
>> Best Regards,
>> Igor
>>
>

Re: Seconds and milliseconds confusion in python thin client

[Ivan Daschinsky]

Igor, I suppose that you are probably right. But there is no need to notice
or deprecate something. This functionality is not released yet

вт, 15 июн. 2021 г., 23:41 Igor Sapego :

> Hi Igniters,
>
> I've noticed a weird behaviour of python thin client. In those places where
> we have
> timeouts or any other parameters that take time in some places we treat it
> like integer
> number of milliseconds, in others it can take both floats (as a number of
> seconds)
> and ints (number of milliseconds). This approach looks very confusing to me
> as
> it leads to things where tx_start(1) and tx_start(1.0) are not actually the
> same thing.
>
> AFAIK in python the most common way to pass time to such functions is to
> use floats
> as a number of seconds. This is the approach I propose to use in our API as
> well. Let's
> deprecate usage of ints in those functions with the appropriate warning
> before getting rid
> of it.
>
> What do you think?
>
> Best Regards,
> Igor
>

Re: [VOTE] Release pyignite 0.5.0-rc0

[Ivan Daschinsky]

Also checked hash sums and signature. Packages are verified and
signature is OK, signed by Igor Sapego with key
5C10A0722D947727923C98B5AF35DBD958FE8DC5
Keys can be found here -- https://downloads.apache.org/ignite/KEYS
Here you can find instructions how to verify packages
https://www.apache.org/info/verification.html

вт, 15 июн. 2021 г. в 22:01, Ivan Daschinsky :
>
> +1 From me
> Checked building from source on Ubuntu 20.04 amd64 for pythons 3.6.12
> 3.7.9 3.8.6 3.9.1.
> Checked installing binary packages on Ubuntu 20.04 amd64 and Windows
> 10 amd 64 for pythons 3.6.12 3.7.9 3.8.6 3.9.1.
> Run examples for both platforms
> Check that native module works
> Checked documentation for tag 0.5.0.rc0 on readthedocs.io
>
> вт, 15 июн. 2021 г. в 21:58, Ivan Daschinsky :
> >
> > Dear Igniters!
> >
> > Release candidate binaries for subj are uploaded and ready for vote
> > You can find them here:
> > https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc0
> >
> > If you follow the link above, you will find source package (*.tar.gz and 
> > *.zip)
> > and binary packages (wheels) for windows (amd64) and linux (x86_64)
> > for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
> >
> > You can install binary package for specific version of python using pip
> > For example do this on linux for python 3.8
> > >> pip install pyignite-0.4.0-cp38-cp38-manylinux1_x86_64.whl
> >
> > You can build and install package from source using this command:
> > >> pip install pyignite-0.4.0.tar.gz
> > You can build wheel on your platform using this command:
> > >> pip wheel --no-deps pyignite-0.4.0.tar.gz
> >
> > For building C module, you should have python headers and C compiler 
> > installed.
> > (i.e. for ubuntu sudo apt install build-essential python3-dev)
> > In Mac OS X xcode-tools and python from homebrew are the best option.
> >
> > In order to check whether C module works, use following:
> > >> from pyignite import _cutils
> > >> print(_cutils.hashcode('test'))
> > >> 3556498
> >
> > You can find documentation here:
> > https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0
> >
> > You can find examples here (to check them, you should start ignite locally):
> > https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0/examples.html
> > Also, examples can be found in source archive in examples subfolder.
> > docker-compose.yml is supplied in order to start ignite quickly. (Use
> > `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> > down` to shut down it)
> >
> > Release notes:
> > https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f
> >
> > Git release tag was created:
> > https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc0
> >
> > The vote is formal, see voting guidelines
> > https://www.apache.org/foundation/voting.html
> >
> > +1 - to accept pyignite-0.5.0-rc0
> > 0 - don't care either way
> > -1 - DO NOT accept pyignite-0.5.0-rc0
> >
> > The vote will end at June, 17 15:00 UTC.
>
>
>
> --
> Sincerely yours, Ivan Daschinskiy



-- 
Sincerely yours, Ivan Daschinskiy

Re: [VOTE] Release pyignite 0.5.0-rc0

[Ivan Daschinsky]

+1 From me
Checked building from source on Ubuntu 20.04 amd64 for pythons 3.6.12
3.7.9 3.8.6 3.9.1.
Checked installing binary packages on Ubuntu 20.04 amd64 and Windows
10 amd 64 for pythons 3.6.12 3.7.9 3.8.6 3.9.1.
Run examples for both platforms
Check that native module works
Checked documentation for tag 0.5.0.rc0 on readthedocs.io

вт, 15 июн. 2021 г. в 21:58, Ivan Daschinsky :
>
> Dear Igniters!
>
> Release candidate binaries for subj are uploaded and ready for vote
> You can find them here:
> https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc0
>
> If you follow the link above, you will find source package (*.tar.gz and 
> *.zip)
> and binary packages (wheels) for windows (amd64) and linux (x86_64)
> for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.
>
> You can install binary package for specific version of python using pip
> For example do this on linux for python 3.8
> >> pip install pyignite-0.4.0-cp38-cp38-manylinux1_x86_64.whl
>
> You can build and install package from source using this command:
> >> pip install pyignite-0.4.0.tar.gz
> You can build wheel on your platform using this command:
> >> pip wheel --no-deps pyignite-0.4.0.tar.gz
>
> For building C module, you should have python headers and C compiler 
> installed.
> (i.e. for ubuntu sudo apt install build-essential python3-dev)
> In Mac OS X xcode-tools and python from homebrew are the best option.
>
> In order to check whether C module works, use following:
> >> from pyignite import _cutils
> >> print(_cutils.hashcode('test'))
> >> 3556498
>
> You can find documentation here:
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0
>
> You can find examples here (to check them, you should start ignite locally):
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0/examples.html
> Also, examples can be found in source archive in examples subfolder.
> docker-compose.yml is supplied in order to start ignite quickly. (Use
> `docker-compose up -d` to start 3 nodes cluster and `docker-compose
> down` to shut down it)
>
> Release notes:
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f
>
> Git release tag was created:
> https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc0
>
> The vote is formal, see voting guidelines
> https://www.apache.org/foundation/voting.html
>
> +1 - to accept pyignite-0.5.0-rc0
> 0 - don't care either way
> -1 - DO NOT accept pyignite-0.5.0-rc0
>
> The vote will end at June, 17 15:00 UTC.



-- 
Sincerely yours, Ivan Daschinskiy

[VOTE] Release pyignite 0.5.0-rc0

[Ivan Daschinsky]

Dear Igniters!

Release candidate binaries for subj are uploaded and ready for vote
You can find them here:
https://dist.apache.org/repos/dist/dev/ignite/pyignite/0.5.0-rc0

If you follow the link above, you will find source package (*.tar.gz and *.zip)
and binary packages (wheels) for windows (amd64) and linux (x86_64)
for pythons 36, 37, 38, 39. Also, there are sha512 and gpg signatures.

You can install binary package for specific version of python using pip
For example do this on linux for python 3.8
>> pip install pyignite-0.4.0-cp38-cp38-manylinux1_x86_64.whl

You can build and install package from source using this command:
>> pip install pyignite-0.4.0.tar.gz
You can build wheel on your platform using this command:
>> pip wheel --no-deps pyignite-0.4.0.tar.gz

For building C module, you should have python headers and C compiler installed.
(i.e. for ubuntu sudo apt install build-essential python3-dev)
In Mac OS X xcode-tools and python from homebrew are the best option.

In order to check whether C module works, use following:
>> from pyignite import _cutils
>> print(_cutils.hashcode('test'))
>> 3556498

You can find documentation here:
https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0

You can find examples here (to check them, you should start ignite locally):
https://apache-ignite-binary-protocol-client.readthedocs.io/en/0.5.0.rc0/examples.html
Also, examples can be found in source archive in examples subfolder.
docker-compose.yml is supplied in order to start ignite quickly. (Use
`docker-compose up -d` to start 3 nodes cluster and `docker-compose
down` to shut down it)

Release notes:
https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=blob;f=RELEASE_NOTES.txt;h=9d2ae81af2de22ce9e8c9d3b7ece14dd9e75ca0e;hb=61c83cb0ab6752f019518b4a2cb0724bd027755f

Git release tag was created:
https://gitbox.apache.org/repos/asf?p=ignite-python-thin-client.git;a=tag;h=refs/tags/0.5.0.rc0

The vote is formal, see voting guidelines
https://www.apache.org/foundation/voting.html

+1 - to accept pyignite-0.5.0-rc0
0 - don't care either way
-1 - DO NOT accept pyignite-0.5.0-rc0

The vote will end at June, 17 15:00 UTC.

Re: [DISCUSSION] Release python-0.5.0

[Ivan Daschinsky]

Ok, guys, code is freezed. RC is almost ready, the voting is going to
start soon. Stay tuned.

вт, 15 июн. 2021 г. в 10:35, Ivan Daschinsky :
>
> I've created release branch [1].
>
> I suppose that since there is not any open issue related to python,
> code freeze will be at 06/15/2021 12:00 UTC.
>
>
> [1] -- 
> https://github.com/apache/ignite-python-thin-client/commits/pyignite-0.5.0
>
> пт, 11 июн. 2021 г. в 17:29, Ivan Daschinsky :
> >
> > Since nobody has any contradiction and all planned issues are merged
> > to master, I am going to cut release branch next tuesday and to
> > prepare release binaries for voting.
> > Stay tuned.
> >
> > чт, 10 июн. 2021 г. в 09:54, Nikolay Izhikov :
> > >
> > > +1
> > >
> > > > 9 июня 2021 г., в 09:48, Ivan Daschinsky  
> > > > написал(а):
> > > >
> > > > Hi, folks!
> > > >
> > > > I suppose, that it's time to release the next version of python thin
> > > > client, because we are ready to ship new cool features.
> > > >
> > > > 1. Expiry policy (TTL) support (merged to master) [1]
> > > > 2. Transactions (patch available) [2]
> > > > 3. As part of tx api, huge performance improvement of asyncio [3]
> > > > 4. Enable partition awareness by default
> > > >
> > > > Also, few bugs have been fixed.
> > > >
> > > > There is a full list of tickets for this release [4]. Most of them are
> > > > resolved, transactions [2] are in PA state, unresolved tickets are
> > > > trivial (documentation and enabling PA by default). I suppose
> > > > that we can freeze scope at the end of next week and prepare RC for 
> > > > voting.
> > > >
> > > > WDYT?
> > > >
> > > > [1] -- https://issues.apache.org/jira/browse/IGNITE-14595
> > > > [2] -- https://issues.apache.org/jira/browse/IGNITE-12467
> > > > [3] -- 
> > > > https://issues.apache.org/jira/browse/IGNITE-12467?focusedCommentId=17359293&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17359293
> > > > [4] -- 
> > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%20python-0.5.0
> > >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
>
>
>
> --
> Sincerely yours, Ivan Daschinskiy

Re: [DISCUSSION] Release python-0.5.0

[Ivan Daschinsky]

I've created release branch [1].

I suppose that since there is not any open issue related to python,
code freeze will be at 06/15/2021 12:00 UTC.


[1] -- 
https://github.com/apache/ignite-python-thin-client/commits/pyignite-0.5.0

пт, 11 июн. 2021 г. в 17:29, Ivan Daschinsky :
>
> Since nobody has any contradiction and all planned issues are merged
> to master, I am going to cut release branch next tuesday and to
> prepare release binaries for voting.
> Stay tuned.
>
> чт, 10 июн. 2021 г. в 09:54, Nikolay Izhikov :
> >
> > +1
> >
> > > 9 июня 2021 г., в 09:48, Ivan Daschinsky  
> > > написал(а):
> > >
> > > Hi, folks!
> > >
> > > I suppose, that it's time to release the next version of python thin
> > > client, because we are ready to ship new cool features.
> > >
> > > 1. Expiry policy (TTL) support (merged to master) [1]
> > > 2. Transactions (patch available) [2]
> > > 3. As part of tx api, huge performance improvement of asyncio [3]
> > > 4. Enable partition awareness by default
> > >
> > > Also, few bugs have been fixed.
> > >
> > > There is a full list of tickets for this release [4]. Most of them are
> > > resolved, transactions [2] are in PA state, unresolved tickets are
> > > trivial (documentation and enabling PA by default). I suppose
> > > that we can freeze scope at the end of next week and prepare RC for 
> > > voting.
> > >
> > > WDYT?
> > >
> > > [1] -- https://issues.apache.org/jira/browse/IGNITE-14595
> > > [2] -- https://issues.apache.org/jira/browse/IGNITE-12467
> > > [3] -- 
> > > https://issues.apache.org/jira/browse/IGNITE-12467?focusedCommentId=17359293&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17359293
> > > [4] -- 
> > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%20python-0.5.0
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy



-- 
Sincerely yours, Ivan Daschinskiy

Re: Apache Ignite 2.11

[Ivan Daschinsky]

I suppose that python thin client documentation should be removed, only
link to actual readthedocs.io documentation should remain

пт, 11 июн. 2021 г., 14:01 Игорь Гусев :

>
> Hello all.
>
> I’d also join in on Nikita’s question n what docs are required for
> release. Currently we only have one doc issue for over 200 tickets resolved
> in the released, and it looks like we might be missing something.
>
>
> >Hello Igniters!
> >
> >I guess that AI 2.11 will include documentation work.
> >I'll be glad to help with this.
> >And I'll be happy to know the scope of features that we need to document.
> >
> >Regards,
> >Nikita
> >пт, 4 июн. 2021 г. в 03:50, 18624049226 < 18624049...@163.com >:
> >>hello,
> >>
> >>can someone can help to merge the following patch into master?
> >>
> >>https://issues.apache.org/jira/browse/IGNITE-12852
> >>
> >>This patch has been completed for a long time, and I think it is a
> >>valuable one.
> >>
> >>在 2021/6/3 下午11:12, Dmitry Pavlov 写道:
> >>> Sure, thanks for your reply
> >>>
> >>> Folks, could you please advice how to setup JIRA account integration
> for non-committers?
> >>>
> >>> For the page
> >>>  https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.11
> >>> I can see all issues, but Alexey's login seems to be not sufficient.
> Can wiki admins set up smth? Or Alexey should do some setup on his own?
> >>>
> >>> On 2021/06/03 14:55:25, Maksim Timonin < timonin.ma...@gmail.com >
> wrote:
>  I mean I'm OK to release it with 2.12. So, I am glad to hear there are
>  chances for 2.11, but I guess we can postpone it.
> 
> 
>  On Thu, Jun 3, 2021 at 5:39 PM Dmitry Pavlov < dpav...@apache.org >
> wrote:
> 
> > ok, Maksim, keep us posted.
> >
> > We're in the middle of paperwork rigth now, so there is a chance ))
> >
> > On 2021/06/03 14:30:55, Maksim Timonin < timonin.ma...@gmail.com >
> wrote:
> >>> Regarding non-SQL index API it really depends on readiness. I
> guess if
> > it
> >> is not in the master, we can skip and release it next time.
> >>
> >> Hi, Dmitry! Yes, currently it is on review. I'm OK to release it
> within
> > the
> >> next release.
> >>
> >> On Thu, Jun 3, 2021 at 5:26 PM Dmitry Pavlov < dpav...@apache.org
> > wrote:
> >>
> >>> Hi Alexey,
> >>>
> >>> Releasing master as-is makes sence for me.
> >>>
> >>> Regarding non-SQL index API it really depends on readiness. I
> guess if
> > it
> >>> is not in the master, we can skip and release it next time.
> >>>
> >>> We've a bit overdue initial scope freeze date. It's not a big
> deal, but
> >>> still makes sense to start discussion, create and share wiki page
> and
> >>> create branch.
> >>>
> >>> Since committer privileges required to create branch, could you
> please
> >>> notify me in advance.
> >>>
> >>> Sincerely,
> >>> Dmitriy Pavlov
> >>>
> >>> On 2021/06/03 14:08:34, <
> >>>  olive.c...@gmail.com > wrote:
>  :scope:
> 
>  Here is a list [1] of 195 tickets, i picked for Apache Ignite 2.11
> >>> release
>  Let's start discussion
> 
>  [1]
> >
> https://issues.apache.org/jira/browse/IGNITE-14781?jql=project%20%3D%20IGNITE%20%20AND%20status%20%20%3D%20Resolved%20and%20fixVersion%20%3D%202.11%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC
>  On 2021/05/13 14:27:33, Dmitry Pavlov < dpav...@apache.org >
> wrote:
> > Hi Alexey,
> >
> > :pmc to sign distribution:
> > Formally, only PMC member can be a release manager, so part of
> >>> activities should be picked up by committer and PMC member. If
> noone
> > else
> >>> want to help you here, I would.
> > :timeline:
> > And, could you estimate all phases of the release.
> >
> > :scope:
> > I guess we can just pick up master current state and release it.
> > But
> >>> if someone has some ideas if we should wait for particular feature
> to
> > be
> >>> completed before branch divergence/release candidate build, please
> let
> > know.
> > Sincerely,
> > Dmitriy Pavlov
> >
> > On 2021/05/13 09:45:02, Алексей Гидаспов < olive.c...@gmail.com
> >
> >>> wrote:
> >> Dear Ignite Community!
> >>
> >> I suggest starting Apache Ignite 2.11 release activities. Also
> >>> suggest
> >> myself to be a release manager for this release. I plan release
> > at
> >>> the end
> >> of june 2021.
> >>
> >> ___
> >>
> >> Best Regards,
> >> Alexey Gidaspov
> >>
>
>
> --
> Igor Gusev
>

Re: [DISCUSSION] Release python-0.5.0

[Ivan Daschinsky]

Since nobody has any contradiction and all planned issues are merged
to master, I am going to cut release branch next tuesday and to
prepare release binaries for voting.
Stay tuned.

чт, 10 июн. 2021 г. в 09:54, Nikolay Izhikov :
>
> +1
>
> > 9 июня 2021 г., в 09:48, Ivan Daschinsky  написал(а):
> >
> > Hi, folks!
> >
> > I suppose, that it's time to release the next version of python thin
> > client, because we are ready to ship new cool features.
> >
> > 1. Expiry policy (TTL) support (merged to master) [1]
> > 2. Transactions (patch available) [2]
> > 3. As part of tx api, huge performance improvement of asyncio [3]
> > 4. Enable partition awareness by default
> >
> > Also, few bugs have been fixed.
> >
> > There is a full list of tickets for this release [4]. Most of them are
> > resolved, transactions [2] are in PA state, unresolved tickets are
> > trivial (documentation and enabling PA by default). I suppose
> > that we can freeze scope at the end of next week and prepare RC for voting.
> >
> > WDYT?
> >
> > [1] -- https://issues.apache.org/jira/browse/IGNITE-14595
> > [2] -- https://issues.apache.org/jira/browse/IGNITE-12467
> > [3] -- 
> > https://issues.apache.org/jira/browse/IGNITE-12467?focusedCommentId=17359293&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17359293
> > [4] -- 
> > https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%20python-0.5.0
>


-- 
Sincerely yours, Ivan Daschinskiy

[DISCUSSION] Release python-0.5.0

[Ivan Daschinsky]

Hi, folks!

I suppose, that it's time to release the next version of python thin
client, because we are ready to ship new cool features.

1. Expiry policy (TTL) support (merged to master) [1]
2. Transactions (patch available) [2]
3. As part of tx api, huge performance improvement of asyncio [3]
4. Enable partition awareness by default

Also, few bugs have been fixed.

There is a full list of tickets for this release [4]. Most of them are
resolved, transactions [2] are in PA state, unresolved tickets are
trivial (documentation and enabling PA by default). I suppose
that we can freeze scope at the end of next week and prepare RC for voting.

WDYT?

[1] -- https://issues.apache.org/jira/browse/IGNITE-14595
[2] -- https://issues.apache.org/jira/browse/IGNITE-12467
[3] -- 
https://issues.apache.org/jira/browse/IGNITE-12467?focusedCommentId=17359293&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17359293
[4] -- 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20fixVersion%20%3D%20python-0.5.0

Re: [MTCGA]: new failures in builds [6024739] needs to be handled

[Ivan Daschinsky]

Hi, Pavel, could you please look at this [1]? It seems after updating
TC, few new inspections arrived.


]1] -- 
https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PlatformNetInspections/6024739?buildTab=Inspection

пт, 28 мая 2021 г. в 04:55, :
>
> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than 
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were a 
> volunteer to make the contribution to this project, but things change and you 
> may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and fix 
> test failures or step down and some committer may revert you commit.
>
>  *New Critical Failure in master Platform .NET (Inspections)* 
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_PlatformNetInspections?branch=%3Cdefault%3E
>  Changes may lead to failure were done by
>  - sergei ryzhov  
> https://ci.ignite.apache.org/viewModification.html?modId=926866
>
>  - Here's a reminder of what contributors were agreed to do 
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 04:55:20 28-05-2021

Re: [DISCUSSION] Transactional cache could be in inconsistent state after node recovery.

[Ivan Daschinsky]

As for me, it is logical to remove this flag after merging
IGNITE-6324. I suppose that the slowdown is negligible. BTW, yardstick
reports contains no information about confidence interval. I suppose
that another run could show not drop but improvement :)

пн, 24 мая 2021 г. в 12:06, Zhenya Stanilovsky :
>
>
> Igniters, as previously was found [1] in some cases transactional cache can 
> contain unexpected data after node crash and further recovery. Short 
> explanation: it`s all due to ignite does not save transactional records into 
> the WAL.
> The simplest example: 1 node cluster and transactional cache, if crash has 
> occurred during transaction processing data records will be partially stored 
> into the wal and further recovery procedure will apply them, as is, thus if 
> transaction contain keys [k1, k2, k3] after recovery we can obtain only [k1, 
> k2], of course this is unexpected and erroneous behavior. There are numerous 
> variants with multi nodes on how they can obtain inconsistent data after 
> recovery [2]. PR [1] (already merged into master) has changed this situation 
> and i suggest to turn on by default transactional records storing and remove 
> flag  IGNITE_WAL_LOG_TX_RECORDS at all.
> I benchmarked (attached in issue) and found no potential slowdowns here.
> Any comments ? Review is appreciated too. Thanks!
>
> [1]  https://issues.apache.org/jira/browse/IGNITE-6324
> [2]  
> https://github.com/apache/ignite/pull/8987/files#diff-96ba20321a66ec20d0df55abcdeb0808ff6dd1d4b87b782c892496369321a593R75
> [3]  https://issues.apache.org/jira/browse/IGNITE-14739
>
>
>
>



-- 
Sincerely yours, Ivan Daschinskiy

Re: Building with maven 3.8.1

[Ivan Daschinsky]

And so what? There are no changes in pom's (in this PR) that break
build on earlier maven versions. Why we should trust this patch
(moreover, it breaks even some travis ci checks)

пн, 24 мая 2021 г. в 10:09, Petr Ivanov :
>
> Our TeamCity currently does not support 3.8.1 maven build runner.
> I think it will be available with 2021.1 version that is going to be 
> delivered soon.
>
>
> > On 21 May 2021, at 12:28, Ivan Daschinsky  wrote:
> >
> > Hi. But where is TC run? And I suppose, that
> > https://travis-ci.com/github/apache/ignite/jobs/506675544 should be at
> > least fixed
> >
> > пт, 21 мая 2021 г. в 10:22, Petr Ivanov :
> >>
> >> Hi, Ilya.
> >>
> >>
> >> Left small comment on formatting issue.
> >> Otherwise looks good!
> >>
> >>
> >> Considering 3.8.1 maven support — we will be migrating builds there after 
> >> TC 2021.1 will be delivered.
> >>
> >>
> >>> On 20 May 2021, at 19:22, Ilya Korol  wrote:
> >>>
> >>> Hi, all.
> >>>
> >>> Maybe someone has already faced the issue with Ignite and latest Maven 
> >>> release 3.8.1?
> >>>
> >>> https://issues.apache.org/jira/browse/IGNITE-14753
> >>>
> >>> From 3.8.1 maven supplied with config that will block any http 
> >>> repository/mirror. (See details here 
> >>> https://maven.apache.org/docs/3.8.1/release-notes.html#cve-2021-26291)
> >>>
> >>> Attempt to perform a build produces several errors:
> >>>
> >>>
> >>> 1. Third party dependencies
> >>>
> >>> 1.1) jta, hibernate-4.2, hibernate-5.1, hibernate-5.3 (btw hibernate-* 
> >>> modules aren't built during mvn install)
> >>>
> >>> org.ow2.jotm:jotm-core:jar:2.2.3
> >>> -> org.ow2.carol:carol:jar:3.0.8
> >>> -> org.jacorb:jacorb:jar:2.2.3-jonas-patch-20071018
> >>>
> >>> jotm is a test dependency. Switch to latest available version 2.3.1-M1 
> >>> did the trick. I didn't find any changelog for latest jotm release (their 
> >>> site jotm.ow2.org seems a bit abandoned). I checked a little the diff 
> >>> between 2.2.3 and 2.3.1-M1 source jars. Seems that there was some changes 
> >>> in RMI related facilities, but i don't have enough expertise make a 
> >>> conclusion that switch to 2.3.1-M1 would be safe (even if tests would be 
> >>> green). Due to state of JOTM project maybe we should consider using 
> >>> another JTA implementation with ongoing support like Atomicos or Narayana 
> >>> (this implementation is also from the JBoss family like Hibernate)?
> >>>
> >>> 1.2) spark
> >>>
> >>> [ERROR] Failed to execute goal on project ignite-spark: Could not resolve 
> >>> dependencies for project 
> >>> org.apache.ignite:ignite-spark:jar:2.11.0-SNAPSHOT: Failed to collect 
> >>> dependencies at org.apache.spark:spark-core_2.11:jar:2.3.0 -> 
> >>> net.java.dev.jets3t:jets3t:jar:0.9.4 -> 
> >>> commons-codec:commons-codec:jar:1.15-SNAPSHOT: Failed to read artifact 
> >>> descriptor for commons-codec:commons-codec:jar:1.15-SNAPSHOT: Could not 
> >>> transfer artifact commons-codec:commons-codec:pom:1.15-SNAPSHOT from/to 
> >>> maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for 
> >>> repositories: [apache.snapshots (http://repository.apache.org/snapshots, 
> >>> default, snapshots)] -> [Help 1]
> >>>
> >>> Updating to latest spark-core_2.11 maintenance version (2.3.0 -> 2.3.4) 
> >>> did the job.
> >>>
> >>>
> >>> 2. Broken plugins configuration
> >>>
> >>> Currently ignite-parent uses org.apache:apache:16 as parent. Up to 18 
> >>> release apache used http schema in different places of its configuration 
> >>> (e.g. snapshot repository). So i guess its a good reason to update apache 
> >>> parent at least to 18 release or maybe even to latest 23. This upgrade 
> >>> will break builds for several modules:
> >>>
> >>> 2.1) maven-jar-plugin */useDefaultManifestFile/* option was removed, so 
> >>> usage of this option (true for ignite) will break the build. I guess we 
> >>> can safely remove it from parent pom.
> >>>
> >>> 2.2) Classifiers in ignite-exdata-uri. Building ignite-exdata-uri with 
> >>> latest jar plugin produces errors like:
> >>>
> >>>ignite-extdata-uri: You have to use a classifier to attach 
> >>> supplemental artifacts to the project instead of replacing them
> >>>
> >>> Seems that jar plugin doesn't like when build produces multiple jars even 
> >>> if they finalName's are different. Reworking build configuration with 
> >>> classifiers fixed the problem.
> >>>
> >>>
> >>> I've created a PR with proposed changes: 
> >>> https://github.com/apache/ignite/pull/9116, comments are welcome
> >>>
> >>
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: Building with maven 3.8.1

[Ivan Daschinsky]

Hi. But where is TC run? And I suppose, that
https://travis-ci.com/github/apache/ignite/jobs/506675544 should be at
least fixed

пт, 21 мая 2021 г. в 10:22, Petr Ivanov :
>
> Hi, Ilya.
>
>
> Left small comment on formatting issue.
> Otherwise looks good!
>
>
> Considering 3.8.1 maven support — we will be migrating builds there after TC 
> 2021.1 will be delivered.
>
>
> > On 20 May 2021, at 19:22, Ilya Korol  wrote:
> >
> > Hi, all.
> >
> > Maybe someone has already faced the issue with Ignite and latest Maven 
> > release 3.8.1?
> >
> > https://issues.apache.org/jira/browse/IGNITE-14753
> >
> > From 3.8.1 maven supplied with config that will block any http 
> > repository/mirror. (See details here 
> > https://maven.apache.org/docs/3.8.1/release-notes.html#cve-2021-26291)
> >
> > Attempt to perform a build produces several errors:
> >
> >
> > 1. Third party dependencies
> >
> > 1.1) jta, hibernate-4.2, hibernate-5.1, hibernate-5.3 (btw hibernate-* 
> > modules aren't built during mvn install)
> >
> > org.ow2.jotm:jotm-core:jar:2.2.3
> > -> org.ow2.carol:carol:jar:3.0.8
> > -> org.jacorb:jacorb:jar:2.2.3-jonas-patch-20071018
> >
> > jotm is a test dependency. Switch to latest available version 2.3.1-M1 did 
> > the trick. I didn't find any changelog for latest jotm release (their site 
> > jotm.ow2.org seems a bit abandoned). I checked a little the diff between 
> > 2.2.3 and 2.3.1-M1 source jars. Seems that there was some changes in RMI 
> > related facilities, but i don't have enough expertise make a conclusion 
> > that switch to 2.3.1-M1 would be safe (even if tests would be green). Due 
> > to state of JOTM project maybe we should consider using another JTA 
> > implementation with ongoing support like Atomicos or Narayana (this 
> > implementation is also from the JBoss family like Hibernate)?
> >
> > 1.2) spark
> >
> > [ERROR] Failed to execute goal on project ignite-spark: Could not resolve 
> > dependencies for project 
> > org.apache.ignite:ignite-spark:jar:2.11.0-SNAPSHOT: Failed to collect 
> > dependencies at org.apache.spark:spark-core_2.11:jar:2.3.0 -> 
> > net.java.dev.jets3t:jets3t:jar:0.9.4 -> 
> > commons-codec:commons-codec:jar:1.15-SNAPSHOT: Failed to read artifact 
> > descriptor for commons-codec:commons-codec:jar:1.15-SNAPSHOT: Could not 
> > transfer artifact commons-codec:commons-codec:pom:1.15-SNAPSHOT from/to 
> > maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for 
> > repositories: [apache.snapshots (http://repository.apache.org/snapshots, 
> > default, snapshots)] -> [Help 1]
> >
> > Updating to latest spark-core_2.11 maintenance version (2.3.0 -> 2.3.4) did 
> > the job.
> >
> >
> > 2. Broken plugins configuration
> >
> > Currently ignite-parent uses org.apache:apache:16 as parent. Up to 18 
> > release apache used http schema in different places of its configuration 
> > (e.g. snapshot repository). So i guess its a good reason to update apache 
> > parent at least to 18 release or maybe even to latest 23. This upgrade will 
> > break builds for several modules:
> >
> > 2.1) maven-jar-plugin */useDefaultManifestFile/* option was removed, so 
> > usage of this option (true for ignite) will break the build. I guess we can 
> > safely remove it from parent pom.
> >
> > 2.2) Classifiers in ignite-exdata-uri. Building ignite-exdata-uri with 
> > latest jar plugin produces errors like:
> >
> > ignite-extdata-uri: You have to use a classifier to attach supplemental 
> > artifacts to the project instead of replacing them
> >
> > Seems that jar plugin doesn't like when build produces multiple jars even 
> > if they finalName's are different. Reworking build configuration with 
> > classifiers fixed the problem.
> >
> >
> > I've created a PR with proposed changes: 
> > https://github.com/apache/ignite/pull/9116, comments are welcome
> >
>


-- 
Sincerely yours, Ivan Daschinskiy

Re: [DISCUSS] Python thin client development approach.

[Ivan Daschinsky]

Igniters. Almost half of a year passed since this thread begun. We
released 0.4.0, we adopted travis-ci and use it as primary source for
test results but nothing changed in TC

1. We use deprecated pytest-runner (even here
(https://pypi.org/project/pytest-runner) you can see deprecation
notice and advice to use tox instead) in TC.
This dependency is absolutely unnecessary and cause a bunch of
problems when pyignite is installed as source from pypi.org local
mirror
We just need at least tox in linux agents on TC.
2. We build release 0.4.0 manually on my and Igor Sapego's laptops.
There are no release build on TC, despite the fact that all needed
scripts are included in git repo and comprehensive instruction is
available.
For running these scripts, we just need docker available on linux
agents and some additional packages on windows agents.

I, Igor and Petr Ivanov had personal talk about this few months ago
but nothing changed.

пт, 22 янв. 2021 г. в 16:12, Ivan Daschinsky :
>
> Igor, I've never talked about complete removal of TC builds.
> I just suggested to add TC jobs for different python versions and use travis 
> heavily.
>
> Currently, we have done most of the tasks, except of run TC tests on 
> different python's versions.
>
>
>
> пт, 22 янв. 2021 г. в 15:16, Igor Sapego :
>>
>> Ivan,
>>
>> Though I generally agree with the approach you've suggested, I can see
>> a problem here. Since we now have a separate repos for thin clients, for
>> some features we may need to introduce changes to Ignite and python-thin
>> repos in a single ticket and we should have an ability to run tests on with
>> changes on both python client and server nodes. Current TC suites provide
>> such ability, Travis does not. So I believe, it's too soon to abandon TC
>> for thin
>> clients, at least until we could solve the issue I've described.
>>
>> Best Regards,
>> Igor
>>
>>
>> On Fri, Dec 25, 2020 at 1:49 PM Nikolay Izhikov  wrote:
>>
>> > Hello, Ivan.
>> >
>> > I’m +1 for your proposal.
>> >
>> > > 25 дек. 2020 г., в 13:14, Ivan Daschinsky 
>> > написал(а):
>> > >
>> > > Hi folks!
>> > >
>> > > Since we already have a separate repo for thin-clients [1], [2]
>> > > I'd like to propose some improvements in development process/
>> > >
>> > > 1. We should simplify and automate unit tests run for different versions
>> > of
>> > > python
>> > > 2. We should add travis integration per commit and pr. Tests could be run
>> > > against latest dockered image of ignite
>> > > 3. There should be ability to run tests against multiple pythons on TC
>> > > 4. For thin client development process, travis should be the first
>> > option.
>> > > TC suite should be used only to check that compatibility is not broken
>> > > and when new functionality is developed (rare case).
>> > >
>> > > I've prepared fix [3], you can see successful builds for travis. It uses
>> > > tox [5], the most common tool to run tests in multiple environments.
>> > > There are few environments set up in tox.ini -- with and without docker,
>> > > with or without ssl, etc. This helped a lot
>> > > to setup travis CI build (you can see in commits list of PR) and simplify
>> > > run tests for developers. Also docker-compose was introduced
>> > > to help python thin client developers.
>> > >
>> > > But  I need some assistance for TC:
>> > > 1. There is outdated python setuptools on TC agents, so tests cannot be
>> > run
>> > > with updated pytest etc.
>> > > 2. Multiple pythons should be installed on TC agents (via
>> > > https://github.com/pyenv/pyenv), latest minor versions
>> > > for 3.6, 3.7 and 3.8
>> > > 3. After that, TC job should be changed to utilize tox
>> > >
>> > > WDYT about this initiative?
>> > >
>> > >
>> > > [1] -- https://issues.apache.org/jira/browse/IGNITE-13767
>> > > [2] -- https://github.com/apache/ignite-python-thin-client
>> > > [3] -- https://issues.apache.org/jira/browse/IGNITE-13903
>> > > [4] --
>> > https://github.com/apache/ignite-python-thin-client/pull/1/commits
>> > > [5] -- https://tox.readthedocs.io/en/latest/
>> > >
>> > > --
>> > > Sincerely yours, Ivan Daschinskiy
>> >
>> >
>
>
>
> --
> Sincerely yours, Ivan Daschinskiy



-- 
Sincerely yours, Ivan Daschinskiy

Re: Extensions section on the downloads page

[Ivan Daschinsky]

Do be honest, python (and other clients) section should be reworked
(i.e. popup with simple copy-paster command like `pip install`) and
optional manual downloads (may be popup for
packages for different platforms). My web design skills are not very
good and I'm afraid I don't have enough time for it. If it is ok that
links to thin clients cannot be found on download page, let it be.

пт, 14 мая 2021 г. в 20:07, Denis Magda :
>
> Yes, this should work on the production server only. Just test on
> production, it looks like simple logic.
>
> -
> Denis
>
>
> On Fri, May 14, 2021 at 12:37 PM Ivan Daschinsky 
> wrote:
>
> > Denis, there are screenshots from locally hosted site.
> >
> > https://imgbox.com/Zeqb0DVD
> > https://imgbox.com/MgsTOj3M
> >
> > You can see, that links are broken. I hosted this site locally using
> > instructions.
> >
> > пт, 14 мая 2021 г. в 19:02, Denis Magda :
> > >
> > > What exactly are you trying to do?
> > >
> > > The thing is that Apache INFRA doesn't allow executing the server-side
> > > logic anyway (the CGI code snippets that you can find on the downloads
> > page
> > > are added by INFRA). But the client-side JS works and can be tested
> > locally.
> > >
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Fri, May 14, 2021 at 10:16 AM Ivan Daschinsky 
> > > wrote:
> > >
> > > > But if you wanted to create page with custom logic (as I did), it
> > > > would be quite difficult.
> > > >
> > > > пт, 14 мая 2021 г. в 16:38, Denis Magda :
> > > > >
> > > > > But does it render the page? Some of the links that require CGI won’t
> > > > work.
> > > > > I do this just to confirm that there no issues with HTML/CSS and then
> > > > merge
> > > > > to the prod where scripts can be checked.
> > > > >
> > > > >
> > > > > On Friday, May 14, 2021, Ivan Daschinsky 
> > wrote:
> > > > >
> > > > > > Denis, have you tried this? I tried, and see broken links and js
> > code
> > > > > > doesn't work.
> > > > > >
> > > > > > пт, 14 мая 2021 г. в 15:08, Denis Magda :
> > > > > > >
> > > > > > > Hi Ivan,
> > > > > > >
> > > > > > > The download page goes with CGI scripts that are not handled
> > > > properly in
> > > > > > a
> > > > > > > local set up. Please change "download.cgi" to "download.html"
> > > > locally.
> > > > > > >
> > > > > > > -
> > > > > > > Denis
> > > > > > >
> > > > > > >
> > > > > > > On Fri, May 14, 2021 at 2:40 AM Ivan Daschinsky <
> > > > ivanda...@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi! And what about creating "download" page for thin clients? I
> > > > > > > > created issue for this
> > > > > > > > https://issues.apache.org/jira/browse/IGNITE-14628
> > > > > > > >
> > > > > > > > Btw, to be honest, I was unable to set up website with working
> > > > > > > > download page locally.
> > > > > > > > May be instruction is incomplete, but locally everything works
> > fine
> > > > > > > > except download page functionality.
> > > > > > > > I've tried to develop this section by myself but failed
> > because of
> > > > > > this.
> > > > > > > >
> > > > > > > > чт, 13 мая 2021 г. в 23:23, Igor Gusev :
> > > > > > > > >
> > > > > > > > > I am getting a bit too much recognition for this, but I will
> > work
> > > > > > with
> > > > > > > > > Mauricio on implementing openable chapter pages for Ignite
> > > > website.
> > > > > > > > >
> > > > > > > > > On Thu, May 13, 2021 at 3:40 PM Denis Magda <
> > dma...@apache.org>
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > @Igor Gusev ,
> > > > >

Re: Proposal to remove explicit "GC disable" startup suggestion

[Ivan Daschinsky]

Hi, Val! Please take a look to java.nio.Bits.reserveMemory (jdk8). In this
method you can see explicit call of System.gc()

вс, 16 мая 2021 г., 01:22 Valentin Kulichenko :

> Hi Ilya,
>
> Out of curiosity - what are the potential problems with memory buffers that
> you're referring to?
>
> -Val
>
> On Fri, May 14, 2021 at 5:06 AM Ilya Korol  wrote:
>
> > Hi, everyone.
> >
> > There is a proposal to remove suggestion that user should disable
> > explicit GC calls in https://issues.apache.org/jira/browse/IGNITE-14720.
> >
> > Nowadays people usually don't use this facility directly without huge
> > need (at least we hope so), so this suggestion doesn't bring much
> > profit, but instead could lead to potential problems with reclaiming
> > memory used by direct memory buffers.
> >
> > I'm going to fix this.
> >
> > Is there any objections (suggestions)?
> >
> >
>

Re: Extensions section on the downloads page

[Ivan Daschinsky]

>> What exactly are you trying to do?
Denis, what I wanted I have described here --
https://issues.apache.org/jira/browse/IGNITE-14628

пт, 14 мая 2021 г. в 19:37, Ivan Daschinsky :
>
> Denis, there are screenshots from locally hosted site.
>
> https://imgbox.com/Zeqb0DVD
> https://imgbox.com/MgsTOj3M
>
> You can see, that links are broken. I hosted this site locally using
> instructions.
>
> пт, 14 мая 2021 г. в 19:02, Denis Magda :
> >
> > What exactly are you trying to do?
> >
> > The thing is that Apache INFRA doesn't allow executing the server-side
> > logic anyway (the CGI code snippets that you can find on the downloads page
> > are added by INFRA). But the client-side JS works and can be tested locally.
> >
> >
> > -
> > Denis
> >
> >
> > On Fri, May 14, 2021 at 10:16 AM Ivan Daschinsky 
> > wrote:
> >
> > > But if you wanted to create page with custom logic (as I did), it
> > > would be quite difficult.
> > >
> > > пт, 14 мая 2021 г. в 16:38, Denis Magda :
> > > >
> > > > But does it render the page? Some of the links that require CGI won’t
> > > work.
> > > > I do this just to confirm that there no issues with HTML/CSS and then
> > > merge
> > > > to the prod where scripts can be checked.
> > > >
> > > >
> > > > On Friday, May 14, 2021, Ivan Daschinsky  wrote:
> > > >
> > > > > Denis, have you tried this? I tried, and see broken links and js code
> > > > > doesn't work.
> > > > >
> > > > > пт, 14 мая 2021 г. в 15:08, Denis Magda :
> > > > > >
> > > > > > Hi Ivan,
> > > > > >
> > > > > > The download page goes with CGI scripts that are not handled
> > > properly in
> > > > > a
> > > > > > local set up. Please change "download.cgi" to "download.html"
> > > locally.
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Fri, May 14, 2021 at 2:40 AM Ivan Daschinsky <
> > > ivanda...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi! And what about creating "download" page for thin clients? I
> > > > > > > created issue for this
> > > > > > > https://issues.apache.org/jira/browse/IGNITE-14628
> > > > > > >
> > > > > > > Btw, to be honest, I was unable to set up website with working
> > > > > > > download page locally.
> > > > > > > May be instruction is incomplete, but locally everything works 
> > > > > > > fine
> > > > > > > except download page functionality.
> > > > > > > I've tried to develop this section by myself but failed because of
> > > > > this.
> > > > > > >
> > > > > > > чт, 13 мая 2021 г. в 23:23, Igor Gusev :
> > > > > > > >
> > > > > > > > I am getting a bit too much recognition for this, but I will 
> > > > > > > > work
> > > > > with
> > > > > > > > Mauricio on implementing openable chapter pages for Ignite
> > > website.
> > > > > > > >
> > > > > > > > On Thu, May 13, 2021 at 3:40 PM Denis Magda 
> > > > > wrote:
> > > > > > > >
> > > > > > > > > @Igor Gusev ,
> > > > > > > > >
> > > > > > > > > Remember you added an ability to select and share links of
> > > > > > > documentation
> > > > > > > > > chapters? Could you please add a similar capability to Ignite?
> > > For
> > > > > > > > > instance, we have the "Extensions and Integrations" section
> > > and to
> > > > > fix
> > > > > > > a
> > > > > > > > > problem here we can simply:
> > > > > > > > > 1. Update the download's page Extensions section by adding 
> > > > > > > > > some
> > > > > > > standard
> > > > > > > > > paragraph describing the extensions
> > > > > > > > > 2. Provide a reference to t

Re: Extensions section on the downloads page

[Ivan Daschinsky]

Denis, there are screenshots from locally hosted site.

https://imgbox.com/Zeqb0DVD
https://imgbox.com/MgsTOj3M

You can see, that links are broken. I hosted this site locally using
instructions.

пт, 14 мая 2021 г. в 19:02, Denis Magda :
>
> What exactly are you trying to do?
>
> The thing is that Apache INFRA doesn't allow executing the server-side
> logic anyway (the CGI code snippets that you can find on the downloads page
> are added by INFRA). But the client-side JS works and can be tested locally.
>
>
> -
> Denis
>
>
> On Fri, May 14, 2021 at 10:16 AM Ivan Daschinsky 
> wrote:
>
> > But if you wanted to create page with custom logic (as I did), it
> > would be quite difficult.
> >
> > пт, 14 мая 2021 г. в 16:38, Denis Magda :
> > >
> > > But does it render the page? Some of the links that require CGI won’t
> > work.
> > > I do this just to confirm that there no issues with HTML/CSS and then
> > merge
> > > to the prod where scripts can be checked.
> > >
> > >
> > > On Friday, May 14, 2021, Ivan Daschinsky  wrote:
> > >
> > > > Denis, have you tried this? I tried, and see broken links and js code
> > > > doesn't work.
> > > >
> > > > пт, 14 мая 2021 г. в 15:08, Denis Magda :
> > > > >
> > > > > Hi Ivan,
> > > > >
> > > > > The download page goes with CGI scripts that are not handled
> > properly in
> > > > a
> > > > > local set up. Please change "download.cgi" to "download.html"
> > locally.
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Fri, May 14, 2021 at 2:40 AM Ivan Daschinsky <
> > ivanda...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Hi! And what about creating "download" page for thin clients? I
> > > > > > created issue for this
> > > > > > https://issues.apache.org/jira/browse/IGNITE-14628
> > > > > >
> > > > > > Btw, to be honest, I was unable to set up website with working
> > > > > > download page locally.
> > > > > > May be instruction is incomplete, but locally everything works fine
> > > > > > except download page functionality.
> > > > > > I've tried to develop this section by myself but failed because of
> > > > this.
> > > > > >
> > > > > > чт, 13 мая 2021 г. в 23:23, Igor Gusev :
> > > > > > >
> > > > > > > I am getting a bit too much recognition for this, but I will work
> > > > with
> > > > > > > Mauricio on implementing openable chapter pages for Ignite
> > website.
> > > > > > >
> > > > > > > On Thu, May 13, 2021 at 3:40 PM Denis Magda 
> > > > wrote:
> > > > > > >
> > > > > > > > @Igor Gusev ,
> > > > > > > >
> > > > > > > > Remember you added an ability to select and share links of
> > > > > > documentation
> > > > > > > > chapters? Could you please add a similar capability to Ignite?
> > For
> > > > > > > > instance, we have the "Extensions and Integrations" section
> > and to
> > > > fix
> > > > > > a
> > > > > > > > problem here we can simply:
> > > > > > > > 1. Update the download's page Extensions section by adding some
> > > > > > standard
> > > > > > > > paragraph describing the extensions
> > > > > > > > 2. Provide a reference to the "Extensions and Integrations"
> > > > section in
> > > > > > the
> > > > > > > > docs where one can find all they need
> > > > > > > >
> > > > > > > >
> > > > > > > > -
> > > > > > > > Denis
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, May 12, 2021 at 4:35 PM Valentin Kulichenko <
> > > > > > > > valentin.kuliche...@gmail.com> wrote:
> > > > > > > >
> > > > > > > >> Agree. I brought the section back because there was a broken
> > link,
> &

Re: Extensions section on the downloads page

[Ivan Daschinsky]

But if you wanted to create page with custom logic (as I did), it
would be quite difficult.

пт, 14 мая 2021 г. в 16:38, Denis Magda :
>
> But does it render the page? Some of the links that require CGI won’t work.
> I do this just to confirm that there no issues with HTML/CSS and then merge
> to the prod where scripts can be checked.
>
>
> On Friday, May 14, 2021, Ivan Daschinsky  wrote:
>
> > Denis, have you tried this? I tried, and see broken links and js code
> > doesn't work.
> >
> > пт, 14 мая 2021 г. в 15:08, Denis Magda :
> > >
> > > Hi Ivan,
> > >
> > > The download page goes with CGI scripts that are not handled properly in
> > a
> > > local set up. Please change "download.cgi" to "download.html" locally.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Fri, May 14, 2021 at 2:40 AM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Hi! And what about creating "download" page for thin clients? I
> > > > created issue for this
> > > > https://issues.apache.org/jira/browse/IGNITE-14628
> > > >
> > > > Btw, to be honest, I was unable to set up website with working
> > > > download page locally.
> > > > May be instruction is incomplete, but locally everything works fine
> > > > except download page functionality.
> > > > I've tried to develop this section by myself but failed because of
> > this.
> > > >
> > > > чт, 13 мая 2021 г. в 23:23, Igor Gusev :
> > > > >
> > > > > I am getting a bit too much recognition for this, but I will work
> > with
> > > > > Mauricio on implementing openable chapter pages for Ignite website.
> > > > >
> > > > > On Thu, May 13, 2021 at 3:40 PM Denis Magda 
> > wrote:
> > > > >
> > > > > > @Igor Gusev ,
> > > > > >
> > > > > > Remember you added an ability to select and share links of
> > > > documentation
> > > > > > chapters? Could you please add a similar capability to Ignite? For
> > > > > > instance, we have the "Extensions and Integrations" section and to
> > fix
> > > > a
> > > > > > problem here we can simply:
> > > > > > 1. Update the download's page Extensions section by adding some
> > > > standard
> > > > > > paragraph describing the extensions
> > > > > > 2. Provide a reference to the "Extensions and Integrations"
> > section in
> > > > the
> > > > > > docs where one can find all they need
> > > > > >
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Wed, May 12, 2021 at 4:35 PM Valentin Kulichenko <
> > > > > > valentin.kuliche...@gmail.com> wrote:
> > > > > >
> > > > > >> Agree. I brought the section back because there was a broken link,
> > > > and I
> > > > > >> believe I removed it earlier by mistake. We definitely should add
> > more
> > > > > >> details though.
> > > > > >>
> > > > > >> -Val
> > > > > >>
> > > > > >> On Wed, May 12, 2021 at 9:20 AM Dmitry Pavlov  > >
> > > > wrote:
> > > > > >>
> > > > > >> > Hi Denis,
> > > > > >> >
> > > > > >> > +1 from my side.
> > > > > >> >
> > > > > >> > Now extensions in the download page looks like a bit incomplete
> > > > snippet
> > > > > >> of
> > > > > >> > code.
> > > > > >> >
> > > > > >> > It is better than nothing, but completed table would look
> > better.
> > > > > >> >
> > > > > >> > Sincerely,
> > > > > >> > Dmitriy Pavlov
> > > > > >> >
> > > > > >> > On 2021/05/12 14:02:04, Denis Magda  wrote:
> > > > > >> > > Igniters, Val,
> > > > > >> > >
> > > > > >> > > I came across the Extensions section that was restored on the
> > > > > >> downloads
> > > > > >> > > page. It looks clumsy, broken, and incomplete. We're listing
> > > > spring
> > > > > >> boot
> > > > > >> > > artifacts without any reference to other modules:
> > > > > >> > > https://ignite.apache.org/download.cgi#extensions
> > > > > >> > >
> > > > > >> > > How about:
> > > > > >> > >
> > > > > >> > >- Removing the code snippets of the Spring Boot extensions
> > > > > >> > >- Add a paragraph and a list with references to all the
> > > > extensions
> > > > > >> we
> > > > > >> > >have documented on our pages. See extensions and
> > integrations
> > > > > >> section:
> > > > > >> > >https://ignite.apache.org/docs/latest/
> > > > > >> > >
> > > > > >> > > @Igor Gusev , @Nikita Safonov
> > > > > >> > >  could
> > > > > >> > > you also join this thread?
> > > > > >> > >
> > > > > >> > > -
> > > > > >> > > Denis
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > >
> >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>
>
> --
> -
> Denis



-- 
Sincerely yours, Ivan Daschinskiy

Re: Extensions section on the downloads page

[Ivan Daschinsky]

Denis, have you tried this? I tried, and see broken links and js code
doesn't work.

пт, 14 мая 2021 г. в 15:08, Denis Magda :
>
> Hi Ivan,
>
> The download page goes with CGI scripts that are not handled properly in a
> local set up. Please change "download.cgi" to "download.html" locally.
>
> -
> Denis
>
>
> On Fri, May 14, 2021 at 2:40 AM Ivan Daschinsky 
> wrote:
>
> > Hi! And what about creating "download" page for thin clients? I
> > created issue for this
> > https://issues.apache.org/jira/browse/IGNITE-14628
> >
> > Btw, to be honest, I was unable to set up website with working
> > download page locally.
> > May be instruction is incomplete, but locally everything works fine
> > except download page functionality.
> > I've tried to develop this section by myself but failed because of this.
> >
> > чт, 13 мая 2021 г. в 23:23, Igor Gusev :
> > >
> > > I am getting a bit too much recognition for this, but I will work with
> > > Mauricio on implementing openable chapter pages for Ignite website.
> > >
> > > On Thu, May 13, 2021 at 3:40 PM Denis Magda  wrote:
> > >
> > > > @Igor Gusev ,
> > > >
> > > > Remember you added an ability to select and share links of
> > documentation
> > > > chapters? Could you please add a similar capability to Ignite? For
> > > > instance, we have the "Extensions and Integrations" section and to fix
> > a
> > > > problem here we can simply:
> > > > 1. Update the download's page Extensions section by adding some
> > standard
> > > > paragraph describing the extensions
> > > > 2. Provide a reference to the "Extensions and Integrations" section in
> > the
> > > > docs where one can find all they need
> > > >
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Wed, May 12, 2021 at 4:35 PM Valentin Kulichenko <
> > > > valentin.kuliche...@gmail.com> wrote:
> > > >
> > > >> Agree. I brought the section back because there was a broken link,
> > and I
> > > >> believe I removed it earlier by mistake. We definitely should add more
> > > >> details though.
> > > >>
> > > >> -Val
> > > >>
> > > >> On Wed, May 12, 2021 at 9:20 AM Dmitry Pavlov 
> > wrote:
> > > >>
> > > >> > Hi Denis,
> > > >> >
> > > >> > +1 from my side.
> > > >> >
> > > >> > Now extensions in the download page looks like a bit incomplete
> > snippet
> > > >> of
> > > >> > code.
> > > >> >
> > > >> > It is better than nothing, but completed table would look better.
> > > >> >
> > > >> > Sincerely,
> > > >> > Dmitriy Pavlov
> > > >> >
> > > >> > On 2021/05/12 14:02:04, Denis Magda  wrote:
> > > >> > > Igniters, Val,
> > > >> > >
> > > >> > > I came across the Extensions section that was restored on the
> > > >> downloads
> > > >> > > page. It looks clumsy, broken, and incomplete. We're listing
> > spring
> > > >> boot
> > > >> > > artifacts without any reference to other modules:
> > > >> > > https://ignite.apache.org/download.cgi#extensions
> > > >> > >
> > > >> > > How about:
> > > >> > >
> > > >> > >- Removing the code snippets of the Spring Boot extensions
> > > >> > >- Add a paragraph and a list with references to all the
> > extensions
> > > >> we
> > > >> > >have documented on our pages. See extensions and integrations
> > > >> section:
> > > >> > >https://ignite.apache.org/docs/latest/
> > > >> > >
> > > >> > > @Igor Gusev , @Nikita Safonov
> > > >> > >  could
> > > >> > > you also join this thread?
> > > >> > >
> > > >> > > -
> > > >> > > Denis
> > > >> > >
> > > >> >
> > > >>
> > > >
> >



-- 
Sincerely yours, Ivan Daschinskiy

Re: Thin Clients: enable partition awareness by default

[Ivan Daschinsky]

Pavel, sorry, but where is the link to ticket?

пт, 14 мая 2021 г. в 14:13, Pavel Tupitsyn :
>
> Ticket created [1], I'll make the changes, thanks for the input everyone.
>
> On Wed, May 12, 2021 at 7:14 PM Dmitry Pavlov  wrote:
>
> > +1
> >
> > Without awareneess, on the contrary, we can get huge memory utilization on
> > one server node.
> > With awareness we're more scalable on that matter.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > On 2021/05/12 15:39:53, Igor Sapego  wrote:
> > > +1 from me. There were no major issues with this feature and it gives
> > > good performance boost for many cases.
> > >
> > > Best Regards,
> > > Igor
> > >
> > >
> > > On Wed, May 12, 2021 at 5:18 PM Ivan Daschinsky 
> > wrote:
> > >
> > > > Huge +1 from me. PA should be enabled by default.
> > > >
> > > > ср, 12 мая 2021 г. в 13:33, Pavel Tupitsyn :
> > > > >
> > > > > Igniters,
> > > > >
> > > > > Partition Awareness (PA) is implemented in 5 out of 6 thin clients
> > [1].
> > > > >
> > > > > However, this feature is disabled by default in most clients for
> > > > > compatibility reasons:
> > > > > initially we only used one connection to the cluster, but with PA
> > enabled
> > > > > we establish
> > > > > connections to every server node, which may be not desirable in some
> > use
> > > > > cases.
> > > > >
> > > > > I expect those scenarios to be rare, and in most cases it makes
> > sense to
> > > > > enable PA by default.
> > > > > Thoughts, objections?
> > > > >
> > > > >
> > > > > [1]
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Thin+clients+features
> > > >
> > > >
> > > >
> > > > --
> > > > Sincerely yours, Ivan Daschinskiy
> > > >
> > >
> >



-- 
Sincerely yours, Ivan Daschinskiy

Re: Extensions section on the downloads page

[Ivan Daschinsky]

Hi! And what about creating "download" page for thin clients? I
created issue for this
https://issues.apache.org/jira/browse/IGNITE-14628

Btw, to be honest, I was unable to set up website with working
download page locally.
May be instruction is incomplete, but locally everything works fine
except download page functionality.
I've tried to develop this section by myself but failed because of this.

чт, 13 мая 2021 г. в 23:23, Igor Gusev :
>
> I am getting a bit too much recognition for this, but I will work with
> Mauricio on implementing openable chapter pages for Ignite website.
>
> On Thu, May 13, 2021 at 3:40 PM Denis Magda  wrote:
>
> > @Igor Gusev ,
> >
> > Remember you added an ability to select and share links of documentation
> > chapters? Could you please add a similar capability to Ignite? For
> > instance, we have the "Extensions and Integrations" section and to fix a
> > problem here we can simply:
> > 1. Update the download's page Extensions section by adding some standard
> > paragraph describing the extensions
> > 2. Provide a reference to the "Extensions and Integrations" section in the
> > docs where one can find all they need
> >
> >
> > -
> > Denis
> >
> >
> > On Wed, May 12, 2021 at 4:35 PM Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> >> Agree. I brought the section back because there was a broken link, and I
> >> believe I removed it earlier by mistake. We definitely should add more
> >> details though.
> >>
> >> -Val
> >>
> >> On Wed, May 12, 2021 at 9:20 AM Dmitry Pavlov  wrote:
> >>
> >> > Hi Denis,
> >> >
> >> > +1 from my side.
> >> >
> >> > Now extensions in the download page looks like a bit incomplete snippet
> >> of
> >> > code.
> >> >
> >> > It is better than nothing, but completed table would look better.
> >> >
> >> > Sincerely,
> >> > Dmitriy Pavlov
> >> >
> >> > On 2021/05/12 14:02:04, Denis Magda  wrote:
> >> > > Igniters, Val,
> >> > >
> >> > > I came across the Extensions section that was restored on the
> >> downloads
> >> > > page. It looks clumsy, broken, and incomplete. We're listing spring
> >> boot
> >> > > artifacts without any reference to other modules:
> >> > > https://ignite.apache.org/download.cgi#extensions
> >> > >
> >> > > How about:
> >> > >
> >> > >- Removing the code snippets of the Spring Boot extensions
> >> > >- Add a paragraph and a list with references to all the extensions
> >> we
> >> > >have documented on our pages. See extensions and integrations
> >> section:
> >> > >https://ignite.apache.org/docs/latest/
> >> > >
> >> > > @Igor Gusev , @Nikita Safonov
> >> > >  could
> >> > > you also join this thread?
> >> > >
> >> > > -
> >> > > Denis
> >> > >
> >> >
> >>
> >

Re: Thin Clients: enable partition awareness by default

[Ivan Daschinsky]

Huge +1 from me. PA should be enabled by default.

ср, 12 мая 2021 г. в 13:33, Pavel Tupitsyn :
>
> Igniters,
>
> Partition Awareness (PA) is implemented in 5 out of 6 thin clients [1].
>
> However, this feature is disabled by default in most clients for
> compatibility reasons:
> initially we only used one connection to the cluster, but with PA enabled
> we establish
> connections to every server node, which may be not desirable in some use
> cases.
>
> I expect those scenarios to be rare, and in most cases it makes sense to
> enable PA by default.
> Thoughts, objections?
>
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/Thin+clients+features



-- 
Sincerely yours, Ivan Daschinskiy

Re: [DISCUSSION] Array to BinaryObject serialization

[Ivan Daschinsky]

Hi!
First of all, when array is serialized, marshaller actually DO
PRESERVE type of element (seel
org.apache.ignite.internal.binary.BinaryUtils#doReadObjectArray and
org.apache.ignite.internal.binary.BinaryWriterExImpl#doWriteObjectArray).
AFAIK, the motivation of Nickolay proposal, is the fact, that during
call of PlatformService we do additional marshal/unmarshall step and
during this step we loose array type info
See org.apache.ignite.internal.binary.BinaryReaderExImpl#readObjectDetached
and org.apache.ignite.internal.binary.BinaryUtils#doReadObjectArray)

In order to handle this problem, we can just add some wrapper that
contains element type info and use it in our INTERNAL API.
I just don't understand why we should change IgniteBinary API.

>>  It was designed as a data storage format, and the fact
that Ignite uses it elsewhere is a fundamental design flaw. BTW, this is
something we're trying to fix in 3.0.

Please, do not. There are many cases when this can really improve
performance. Reflection is always slower than low level api and many
users are happy with low level API.

сб, 1 мая 2021 г. в 00:51, Valentin Kulichenko :
>
> Hi Nikolay,
>
> Is there a specific motivation behind your proposal? I do acknowledge that
> the semantics of the toBinary method is a little weird, but my concern is
> that the way it works with arrays is just an example. Are you suggesting
> changing collections, primitives, and other "first citizen" data types as
> well? To me, that looks like a huge risky change without a clear value.
>
> I actually believe that binary format *should not* be used as a universal
> serde mechanism. It was designed as a data storage format, and the fact
> that Ignite uses it elsewhere is a fundamental design flaw. BTW, this is
> something we're trying to fix in 3.0.
>
> That said, I'm not necessarily against the change (especially if we do not
> change the default behavior). But I would like to better understand its
> scope (e.g., arrays only or beyond?), as well as get some examples of use
> cases that would benefit from the change.
>
> -Val
>
>
> On Fri, Apr 30, 2021 at 7:35 AM Nikolay Izhikov  wrote:
>
> > Hello, Ilya.
> >
> > Thanks for the feedback!
> >
> > > For me it sounds like something we would like to do in 3.0
> >
> > Ignite 3 is a very long way to go, so I prefer to target this fix in
> > Ignite 2.x.
> >
> > > I think making it default "true" is a breaking change and is not
> > possible in a minor release
> >
> > Yes, you are correct it is a breaking change.
> > It seems for me, we all agreed that breaking changes are possible in minor
> > releases.
> >
> > Anyway, if we will decide do not to enable this feature by default it’s OK
> > for me.
> > We still can implement it and improve the binary SerDe mechanism.
> >
> >
> > > 30 апр. 2021 г., в 17:23, Ilya Kasnacheev 
> > написал(а):
> > >
> > > Hello!
> > >
> > > For me it sounds like something we would like to do in 3.0 (if indeed it
> > > will have arrays as possible value (or key) type), but doing it in 2.x
> > > raises concerns whether it has enough time left to stabilize.
> > >
> > > Also, I think making it default "true" is a breaking change and is not
> > > possible in a minor release, case in point,
> > > IGNITE_BINARY_SORT_OBJECT_FIELDS is still waiting for 3.0 and it is less
> > > destructive.
> > >
> > > Of course I would also like to hear what other community members think.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пт, 30 апр. 2021 г. в 17:16, Nikolay Izhikov :
> > >
> > >> Igniters,
> > >>
> > >> Want to clarify my proposal about new array store format.
> > >> I think we should store array in special binary wrapper that will keep
> > >> original component type
> > >>
> > >> ```
> > >> public class BinaryArrayWrapper implements BinaryObjectEx,
> > Externalizable {
> > >>/** Type ID. */
> > >>private int compTypeId;
> > >>
> > >>/** Raw data. */
> > >>private String compClsName;
> > >>
> > >>/** Value. */
> > >>private Object[] arr;
> > >>
> > >>// Further implementation.
> > >> }
> > >> ```
> > >>
> > >>
> > >>> 30 апр. 2021 г., в 16:31, Nikolay Izhikov 
> > >> написал(а):
> > >>>
> > >>> Hello, Igniters.
> > >>>
> > >>> Currently, binary marshaller works as follows(Say, we have a class
> > >> `User` then):
> > >>>
> > >>> IgniteBinary#toBinary(User)` -> BinaryObject
> > >>> IgniteBinary#toBinary(User[])` -> Object[]
> > >>> IgniteBinary#toBinary(Object[])` -> Object[]
> > >>>
> > >>> This means, that we lose array component type information during binary
> > >> serialization.
> > >>> AFAIK, it’s a design choice made during binary infrastructure
> > >> development.
> > >>>
> > >>> This lead to the following disadvantages:
> > >>>
> > >>> 1. `IgniteBinary` can’t be used as a universal SerDe mechanism.
> > >>> 2. Ignite internals(service grid, .Net calls) contains many tweaks and
> > >> hacks to deal with custom user array and still has many issues [1]
> > >>>
> > 

Re: [DISCUSSION] Documentation of thin clients (python, php, nodejs)

[Ivan Daschinsky]

Hi, Nikita! Thanks for support.

As developers of thin client, we also would try to update
documentation as soon as possible.
For example, recently I've merged to master expiry policy support for
python thin client and documentation was updated simultaneously [2][3]

I am planning to update documentation frequently and I would
appreciate any feedback about our current documentation on
readthedocs.io


[1] -- https://issues.apache.org/jira/browse/IGNITE-14595
[2] -- 
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/examples.html#expirypolicy
[3] -- 
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/async_examples.html#expirypolicy



пт, 23 апр. 2021 г. в 13:29, Никита Сафонов :
>
> Hi Ivan,
>
> Thank you for the suggestion.
>
> I agree that Ignite main docs on thin clients probably need a clean-up to
> be kept up-to-date.
> And migrating the key information to readthedocs.io is really an option.
>
> Let's see what other Igniters think about this idea.
>
> I'd be glad if Igor Gusev could share his opinion on this.
>
> Regards,
> Nikita
>
>
>
> чт, 22 апр. 2021 г. в 22:15, Ivan Daschinsky :
>
> > Igniters, there are some questions regarding the documentation state
> > of thin clients.
> >
> > Recently, we have released pyignite 0.4.0. Traditionally,
> > documentation for python thin client is autogenerated from source and
> > contains in the same git repository, as the client itself.
> > Documentation is autogenerated and hosted in familiar for python
> > developers manner -- on readthedocs.io (Namely,
> > https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/)
> >
> > I suppose the same statement is true for other separately developed
> > clients (nodejs, php).
> >
> > So I'd like to discuss a current documentation approach for thin clients.
> > 1. I strongly believe that the main documentation site for at least
> > python thin client should be readthedocs.io.
> > 2. Documentation should be maintained in the same repository as the
> > thin client itself.
> > 3. As the main documentation's version is tightly coupled with ignite
> > release cycle, it is by default outdated and doesn't resemble the
> > latest version of thin client.
> >
> > I suggest just remove all documentation from the main docs except
> > simple installation instruction (i.e. pip install pyignite) and link
> > to readthedocs.io
> >
> > Regards, Ivan Daschinsky
> >



-- 
Sincerely yours, Ivan Daschinskiy

[DISCUSSION] Documentation of thin clients (python, php, nodejs)

[Ivan Daschinsky]

Igniters, there are some questions regarding the documentation state
of thin clients.

Recently, we have released pyignite 0.4.0. Traditionally,
documentation for python thin client is autogenerated from source and
contains in the same git repository, as the client itself.
Documentation is autogenerated and hosted in familiar for python
developers manner -- on readthedocs.io (Namely,
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/)

I suppose the same statement is true for other separately developed
clients (nodejs, php).

So I'd like to discuss a current documentation approach for thin clients.
1. I strongly believe that the main documentation site for at least
python thin client should be readthedocs.io.
2. Documentation should be maintained in the same repository as the
thin client itself.
3. As the main documentation's version is tightly coupled with ignite
release cycle, it is by default outdated and doesn't resemble the
latest version of thin client.

I suggest just remove all documentation from the main docs except
simple installation instruction (i.e. pip install pyignite) and link
to readthedocs.io

Regards, Ivan Daschinsky

[ANNOUNCE] Apache IGNITE python thin client (pyignite) 0.4.0 released

[Ivan Daschinsky]

The Apache Ignite Community is pleased to announce the release of
Apache IGNITE python thin client (pyignite) 0.4.0.

This new release of python thin client contains a lot of changes.
Namely:
1. Partition awareness support.
2. Asyncio support.
3. Numerous performance fixes.
4. Numerous bug fixes.

For the full list of changes, you can refer to the RELEASE_NOTES.
https://ignite.apache.org/releases/pyignite/0.4.0/release_notes.html

You can install this version using pip
>> pip install pyignite==0.4.0

Alternatively, you can download sources and binary packages (wheels) from
here:
https://archive.apache.org/dist/ignite/pyignite/0.4.0/

Please let us know if you have any problems
https://ignite.apache.org/community/resources.html#ask

Regards,
Ivan Daschinsky on behalf of the Apache Ignite community.

[RESULT][VOTE] Release pyignite-0.4.0-rc1

[Ivan Daschinsky]

Hello, Igniters!

Release pyignite-0.4.0-rc1 has been accepted.

The votes received:
3 "+1" binding votes
2 "+1" votes

There are no "+0" or "-1"

Here the votes received:
- Denis Magda (binding)
- Igor Sapego (binding)
- Anton Vinogradov (binding)
- Ivan Daschinsky
- Alexander Surkov

Link to the thread:
http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Release-pyignite-0-4-0-rc1-td52364.html

Thank you!

[jira] [Created] (IGNITE-14595) ExpiryPolicy support to python thin client

[Ivan Daschinsky (Jira)]

Ivan Daschinsky created IGNITE-14595:


 Summary: ExpiryPolicy support to python thin client
 Key: IGNITE-14595
 URL: https://issues.apache.org/jira/browse/IGNITE-14595
 Project: Ignite
  Issue Type: New Feature
  Components: python, thin client
Reporter: Ivan Daschinsky


It's of a crucial importance to add support for expiration policy in python 
client.
This is on of the most used feature of any cache.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)