Re: Custom string encoding

[Dmitriy Setrakyan]

On Sat, Jul 1, 2017 at 2:24 AM, Sergi Vladykin 
wrote:

> In SQL indexes we may store partial strings and assume them to be in UTF-8,
> I don't think this can be abstracted away. But may be this is not a big
> deal if in indexes we still will use UTF-8.
>

Sergi, why does it matter if it is UTF8 or custom encoding? Why can't we
use our own compact encoding in indexes?


>
> 2017-07-01 10:13 GMT+03:00 Dmitriy Setrakyan :
>
> > Val, do you know how we compare strings in SQL queries? Will we be able
> to
> > use this encoder?
> >
> > Additionally, I think that the encoder is a bit too abstract. Why not go
> > even further and allow users create their own ASCII table for encoding?
> >
> > D.
> >
> > On Fri, Jun 30, 2017 at 6:49 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> >
> > > Andrey,
> > >
> > > Can you elaborate more on this? What is your concern?
> > >
> > > -Val
> > >
> > > On Fri, Jun 30, 2017 at 6:17 PM Andrey Mashenkov <
> > > andrey.mashen...@gmail.com>
> > > wrote:
> > >
> > > > Val,
> > > >
> > > > Looks like make sense.
> > > >
> > > > This will not affect FullText index, as Lucene has own format for
> > storing
> > > > data.
> > > >
> > > > But.. would it be compatible with H2 indexing ? I doubt.
> > > >
> > > > 1 июля 2017 г. 2:27 пользователь "Valentin Kulichenko" <
> > > > valentin.kuliche...@gmail.com> написал:
> > > >
> > > > > Folks,
> > > > >
> > > > > Currently binary marshaller always encodes strings in UTF-8.
> However,
> > > > > sometimes it can be useful to customize this. For example, if data
> > > > contains
> > > > > a lot of Cyrillic, Chinese or other symbols, but not so many Latin
> > > > symbols,
> > > > > memory is used very inefficiently. In this case it would be great
> to
> > > > encode
> > > > > most frequently used symbols in one byte instead of two or three.
> > > > >
> > > > > I propose to introduce BinaryStringEncoder interface that will
> > convert
> > > > > strings to byte arrays and back, and make it pluggable via
> > > > > BinaryConfiguration. This will allow users to plug in any encoding
> > > > > algorithms based on their requirements.
> > > > >
> > > > > Thoughts?
> > > > >
> > > > > https://issues.apache.org/jira/browse/IGNITE-5655
> > > > >
> > > > > -Val
> > > > >
> > > >
> > >
> >
>

Re: Request for contributor permission

[Aleksander Meterko]

Thanks. I've assigned ticket to myself and started working on it, but could
your answer some questions which have arisen?

1. IgniteCache#localEvict() internally delegates to
IgniteInternalCache#evictAll(). Should evict() / evictAll() operations be
left alone or deleted as well?
2. Tests heavily rely on IgniteCache#localEvict() operation. What should be
used instead of it?

2017-06-30 17:51 GMT+04:00 Denis Magda :

> Hi,
>
> Added you to the contributors list. Please go ahead and assign the tickets
> on yourself.
>
> —
> Denis
>
> > On Jun 30, 2017, at 12:10 AM, Александр Метерко <
> aleksander.mete...@gmail.com> wrote:
> >
> > Dear Ignite team,
> >
> > I would like to start contributing to your project starting with ticket
> > https://issues.apache.org/jira/browse/IGNITE-5592 . Could you grant me
> > permissions in Jira to assign this ticket to me? My login is ameterko.
> >
> > Thanks in advance.
>
>

Re: Custom string encoding

[Sergi Vladykin]

In SQL indexes we may store partial strings and assume them to be in UTF-8,
I don't think this can be abstracted away. But may be this is not a big
deal if in indexes we still will use UTF-8.

Sergi

2017-07-01 10:13 GMT+03:00 Dmitriy Setrakyan :

> Val, do you know how we compare strings in SQL queries? Will we be able to
> use this encoder?
>
> Additionally, I think that the encoder is a bit too abstract. Why not go
> even further and allow users create their own ASCII table for encoding?
>
> D.
>
> On Fri, Jun 30, 2017 at 6:49 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
>
> > Andrey,
> >
> > Can you elaborate more on this? What is your concern?
> >
> > -Val
> >
> > On Fri, Jun 30, 2017 at 6:17 PM Andrey Mashenkov <
> > andrey.mashen...@gmail.com>
> > wrote:
> >
> > > Val,
> > >
> > > Looks like make sense.
> > >
> > > This will not affect FullText index, as Lucene has own format for
> storing
> > > data.
> > >
> > > But.. would it be compatible with H2 indexing ? I doubt.
> > >
> > > 1 июля 2017 г. 2:27 пользователь "Valentin Kulichenko" <
> > > valentin.kuliche...@gmail.com> написал:
> > >
> > > > Folks,
> > > >
> > > > Currently binary marshaller always encodes strings in UTF-8. However,
> > > > sometimes it can be useful to customize this. For example, if data
> > > contains
> > > > a lot of Cyrillic, Chinese or other symbols, but not so many Latin
> > > symbols,
> > > > memory is used very inefficiently. In this case it would be great to
> > > encode
> > > > most frequently used symbols in one byte instead of two or three.
> > > >
> > > > I propose to introduce BinaryStringEncoder interface that will
> convert
> > > > strings to byte arrays and back, and make it pluggable via
> > > > BinaryConfiguration. This will allow users to plug in any encoding
> > > > algorithms based on their requirements.
> > > >
> > > > Thoughts?
> > > >
> > > > https://issues.apache.org/jira/browse/IGNITE-5655
> > > >
> > > > -Val
> > > >
> > >
> >
>

Re: Distributed scheduling

[Dmitriy Setrakyan]

Val,

In this case, we should have a notion of a named scheduler and ensure that
we don't schedule the same task more than once. This is beginning to look
more like a durable cluster singleton service, no?

D.

On Fri, Jun 30, 2017 at 1:39 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> I think this functionality should provide durable way of scheduled task or
> closure execution on the cluster. Job descriptors should be persisted on
> server side and executed there.
>
> As for API, I believe this should be part of Compute Grid. I suggest to
> introduce IgniteCompute#withSchedulingPolicy(SchedulingPolicy policy)
> method, where SchedulingPolicy is smth like this:
>
> public interface SchedulingPolicy {
> /**
>  * @return Timestamp of next execution.
>  */
> public Date nextTime();
> }
>
> This will enable scheduling for all compute features (tasks, callables,
> closures, etc.) and also very flexible. Policy implementation can provide
> simple periodic scheduling, scheduling based on Cron or anything else.
>
> Thoughts?
>
> -Val
>
> On Fri, Jun 30, 2017 at 7:55 AM, Dmitriy Setrakyan 
> wrote:
>
> > On Fri, Jun 30, 2017 at 12:29 AM, Alexey Kuznetsov <
> akuznet...@apache.org>
> > wrote:
> >
> > > Dmitriy,
> > >
> > > >> Can you provide a simple example of API calls that will make this
> > > possible?
> > > API could be like this:
> > > 1) via scheduler:
> > > Ignite ignite = Ignition.start();
> > >
> > > ignite.scheduler().schedulel(job, "0 0 * * *"); // This will execute
> job
> > > every day at 00:00
> > >
> > > 2) via compute
> > >
> > > Ignite ignite = Ignition.start();
> > >
> > > ignite.compute().schedulel(task, "0 0 * * *"); // This will execute
> > > compute
> > > task every day at 00:00
> > >
> > > Make sense?
> > >
> > >
> > Yes, it does, but I am failing to see how is this a *distributed*
> > scheduling. Are we persisting the scheduler somewhere in the cluster or
> is
> > it only triggered on the client side?
> >
>

Re: Custom string encoding

[Dmitriy Setrakyan]

Val, do you know how we compare strings in SQL queries? Will we be able to
use this encoder?

Additionally, I think that the encoder is a bit too abstract. Why not go
even further and allow users create their own ASCII table for encoding?

D.

On Fri, Jun 30, 2017 at 6:49 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Andrey,
>
> Can you elaborate more on this? What is your concern?
>
> -Val
>
> On Fri, Jun 30, 2017 at 6:17 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com>
> wrote:
>
> > Val,
> >
> > Looks like make sense.
> >
> > This will not affect FullText index, as Lucene has own format for storing
> > data.
> >
> > But.. would it be compatible with H2 indexing ? I doubt.
> >
> > 1 июля 2017 г. 2:27 пользователь "Valentin Kulichenko" <
> > valentin.kuliche...@gmail.com> написал:
> >
> > > Folks,
> > >
> > > Currently binary marshaller always encodes strings in UTF-8. However,
> > > sometimes it can be useful to customize this. For example, if data
> > contains
> > > a lot of Cyrillic, Chinese or other symbols, but not so many Latin
> > symbols,
> > > memory is used very inefficiently. In this case it would be great to
> > encode
> > > most frequently used symbols in one byte instead of two or three.
> > >
> > > I propose to introduce BinaryStringEncoder interface that will convert
> > > strings to byte arrays and back, and make it pluggable via
> > > BinaryConfiguration. This will allow users to plug in any encoding
> > > algorithms based on their requirements.
> > >
> > > Thoughts?
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-5655
> > >
> > > -Val
> > >
> >
>