[GitHub] ignite pull request #4765: Ignite 2.3

2018-09-16 Thread zjw4267127
GitHub user zjw4267127 opened a pull request:

https://github.com/apache/ignite/pull/4765

Ignite 2.3

asdf

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/ignite ignite-2.3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4765.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4765


commit c5d6decfaf4c8be907975531f7d7bd41dda2da4e
Author: dpavlov 
Date:   2017-10-04T06:46:16Z

IGNITE-6285 Enhance persistent store paths handling - Fixes #2775.

Signed-off-by: Alexey Goncharuk 

commit 2e013fbe85da059580e97d57c15e9cbc92c0aa54
Author: dpavlov 
Date:   2017-10-04T12:05:48Z

IGNITE-6554 Atomic cache remove operations are not logged to WAL - Fixes 
#2800.

Signed-off-by: Alexey Goncharuk 

commit 959bf853b1a77fa674ac8115fcf57c31feb0e6c0
Author: oleg-ostanin 
Date:   2017-10-04T14:35:50Z

Removed excluding ML from examples/src

(cherry picked from commit 78f77b1)

commit d4e5d4fd3e989b69f7bfa49986ee8f8d04cac62f
Author: tledkov-gridgain 
Date:   2017-10-05T08:55:26Z

IGNITE-6529 JDBC: support column metadata 'nullable' property. This closes 
#2793.

commit 81d20410205086ff29ac2e66b7cd17909173d642
Author: tledkov-gridgain 
Date:   2017-10-05T13:32:33Z

IGNITE-6358: JDBC thick: support multiple statements. This closes #2777.

commit 2e046d61078874c12e4fd33bba0826f5453b055d
Author: tledkov-gridgain 
Date:   2017-10-05T13:45:31Z

IGNITE-6556: JDBC thin: fixed setSchema() case sensitivity handling. This 
closes #2805.

commit 62b4b5311109af1f4ac0ae84ac47ad885eaa3533
Author: Alexey Goncharuk 
Date:   2017-10-05T14:37:04Z

IGNITE-5739 Fixed Ignite node crash on deactivation

commit b33be44164d4dc4948ab5c27857d53b75ac4a332
Author: Vasiliy Sisko 
Date:   2017-10-06T07:25:42Z

IGNITE-6287 Web Console: Improved DDL support.
(cherry picked from commit 2410f07)

commit 6c4c41d4ce6f2066a399756f60f798507e0553ad
Author: Alexey Kuznetsov 
Date:   2017-10-06T10:00:39Z

IGNITE-6570 Web Console: Move parsing of JSON to pool of workers.
(cherry picked from commit 74f0400)

commit 6efb0cee753eac1289efd21cc773ca372775aa68
Author: Pavel Tupitsyn 
Date:   2017-10-03T15:33:57Z

.NET: Fix TestRecordLocal

commit fca198bf1ecd4eee8bb72233cd59d0ea2e427c7c
Author: Alexander Paschenko 
Date:   2017-10-06T15:04:44Z

IGNITE-6054: SQL: implemented "WRAP_KEY" and "WRAP_VALUE" modes for CREATE 
TABLE. This closes #2784.

commit 73f092df4792bee918083322ee93026dfa95e3b8
Author: devozerov 
Date:   2017-10-09T07:48:06Z

Fixed JavaDoc.

commit 81b16ada108dc497d38f702b7f678de539345705
Author: devozerov 
Date:   2017-10-09T08:34:51Z

Fixed JavaDoc again.

commit d9f0f4e1bec08ab9f52c6c1ba842524f25a4ba19
Author: devozerov 
Date:   2017-10-09T08:57:06Z

IGNITE-6054: Fixed tests.

commit 9358a88538189e48de68ad2d157c2c1ebf9a219e
Author: tledkov-gridgain 
Date:   2017-10-09T12:14:23Z

IGNITE-6529: JDBC thin: fixed driver protocol compatibility. This closes 
#2819.

commit c4acf54413a0e8cb9cedcb7201084f29ee554ee7
Author: Vasiliy Sisko 
Date:   2017-10-09T12:23:23Z

IGNITE-6287 Web Console: Improved DDL support: added checkbox "Use selected 
cache as default schema name".
(cherry picked from commit a45677c)

commit 831c4d9635c911fde3781987f79a28523d6cf15b
Author: Pavel Tupitsyn 
Date:   2017-10-09T14:33:46Z

IGNITE-6397 .NET thin client: basic cache operations. This closes #2725.

commit 1a6bb2bbf513c8a50f81e9f8def1b3f479c08f4b
Author: Andrey Gura 
Date:   2017-09-27T10:50:26Z

ignite-6305 Ignite update checker enabled

commit 72b409cc233919462b3df1547afdec082d8273e1
Author: Alexander Paschenko 
Date:   2017-10-09T19:17:28Z

IGNITE-6569: Fixed hang when trying to execute "DROP TABLE" on the cache 
this table belongs to. This closes #2823.

commit ab384d4b1692538eca6de81c04ec9b720a73308e
Author: Alexander Paschenko 
Date:   2017-10-10T07:05:12Z

IGNITE-6568: Fixed cache configuration persistence logic. This closes #2815.

commit f06927d13303b780bcb9865a3eea9c95b926a25e
Author: Sergey Chugunov 
Date:   2017-10-09T15:35:11Z

IGNITE-6583 Proper getters for rebalance metrics were added; ignite-style 
getters (without get) were deprecated

Signed-off-by: Andrey Gura 

commit 4a782fb8b8f10917addecd70af77049719f908bd
Author: Alexander Belyak 
Date:   2017-10-10T12:09:30Z

IGNITE-5733: Fixed failures in JerryRestProcessorAbstractSelfTest.

commit 1f6ab52d94de05b248b8677f07e162a87f6a1889
Author: devozerov 
Date:   2017-10-10T12:14:37Z

Merge branch 'ignite-2.3' into ignite-2.3.1

commit f0c64e7288ef94c84984a5e7a70f7038a539c6cd
Author: devozerov 
Date:   2017-10-10T14:54:50Z

IGNITE-6588: SQL: optimized index segment resolution. This closes #2825.

commit 0df4bf00e52ace07c4f3adf8b4e463e2e63167b2
Aut

[jira] [Created] (IGNITE-9607) Service Grid redesign - phase 1

2018-09-16 Thread Vyacheslav Daradur (JIRA)
Vyacheslav Daradur created IGNITE-9607:
--

 Summary: Service Grid redesign - phase 1
 Key: IGNITE-9607
 URL: https://issues.apache.org/jira/browse/IGNITE-9607
 Project: Ignite
  Issue Type: Improvement
  Components: managed services
Reporter: Vyacheslav Daradur
Assignee: Vyacheslav Daradur
 Fix For: 2.7


This is an umbrella ticket for tasks which should be implemented atomically in 
phase #1 of Service Grid redesign.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] ignite pull request #4766: PR

2018-09-16 Thread akuznetsov-gridgain
GitHub user akuznetsov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/4766

PR



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-14196

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4766.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4766


commit 8709785814a432f981c30274a55e2ef730667421
Author: Pavel Tupitsyn 
Date:   2018-02-18T20:27:29Z

.NET: Fix SslConfigurationTest

commit d2a1136fd725b0e09e1e655152509aeae755c368
Author: Pavel Tupitsyn 
Date:   2018-02-26T22:15:11Z

IGNITE-7329 .NET: Thin client: SSL connection implemented

This closes #3447

commit c2a5c5df71197418bd126e87f62e66681ac25b76
Author: Pavel Tupitsin 
Date:   2018-03-05T16:07:32Z

IGNITE-7878 .NET: Ignore TestStartDefault

commit eec0a6714dd9f59edb637c8c068205843f5b9af6
Author: Alexey Popov 
Date:   2018-03-06T17:47:18Z

IGNITE-7889: .NET: LINQ: Fix GroupBy with Where

This closes #3608

commit 74fbd41f7f4e0b129797297c603c471ea558f2f6
Author: Pavel Tupitsyn 
Date:   2018-03-07T08:10:08Z

IGNITE-7566 .NET: Build scripts: stop build when one of the steps fails

commit 4ae42e6d525e04c182c854785b5526617f72ade6
Author: Pavel Tupitsyn 
Date:   2018-03-07T08:25:07Z

IGNITE-7566 .NET: Build scripts: stop build when one of the steps fails

commit 3e334ec43f0090d9feb6574d23955c4ca7a90c14
Author: Sergey Stronchinskiy 
Date:   2018-03-07T08:44:51Z

IGNITE-5298 .NET: DML update via LINQ

This closes #3599

commit 5c90bc77e014da526d64ab9f76b813685c52d6f3
Author: Pavel Tupitsyn 
Date:   2018-03-12T13:21:41Z

IGNITE-7878 .NET: Un-Ignore TestStartDefault

This reverts commit 18c6b4a5d079874e9c156fa39564970ad3c31914.

commit 35ef3bcb4d2ae6ea39203fbceb9e9eb5026eb213
Author: Pavel Tupitsyn 
Date:   2018-03-12T17:12:13Z

.NET: Update DEVNOTES - better .NET Core test commands

commit e9ea2da7d7f6afa609475f16b0250de21e6c139e
Author: Pavel Tupitsyn 
Date:   2018-03-14T21:05:32Z

.NET: Fix code inspection errors

commit 99d022bb3257a2a39e121dba47787e81356a318f
Author: Pavel Tupitsyn 
Date:   2018-03-15T13:48:46Z

.NET: Suppress incorrect code inspection errors

commit e093e51f88cf304e64fcf5535279cc176937fbea
Author: Igor Sapego 
Date:   2018-03-26T10:33:40Z

IGNITE-7852: ODBC: Implemented username/password authentication. This 
closes #3671.

commit 6a4fa6cc87b7d78aed7e5c6a2531d9cfa657bda0
Author: devozerov 
Date:   2018-03-26T10:50:36Z

Merge branch 'ignite-2.4.4-.net' into ignite-2.4.4

commit b630e8fd8d39bed21e044ce55eb6986da0d1f02e
Author: Anton Kalashnikov 
Date:   2018-03-26T11:57:11Z

GG-13638 move setting compatibilityMode to head of collectGridNodeData - 
Fixes #3665.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit df8c823)

commit cdbf963efc6bad7520fa174a47207fd0b34fe9f2
Author: Anton Kalashnikov 
Date:   2018-03-26T11:57:11Z

GG-13638 move setting compatibilityMode to head of collectGridNodeData - 
Fixes #3665.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit df8c823)

commit 2280e55a228d37ad324d5d2fc7248f2b0c1b46f8
Author: Alexey Popov 
Date:   2018-03-26T14:00:07Z

IGNITE-7928: .NET: Fixed hang caused by mishandled exception during custom 
cache store deserialization. This closes #3690.

commit e0a24a1014a19fb3bd698b83d9a1664a3b9a06b9
Author: Pavel Tupitsyn 
Date:   2018-01-26T08:48:14Z

IGNITE-7530 .NET: Fix GetAll memory usage and performance in binary mode

This closes #3436

commit e721130d85cf31775d94a004164a4287e3c2f68f
Author: Alexey Popov 
Date:   2018-03-06T17:47:18Z

IGNITE-7889: .NET: LINQ: Fix GroupBy with Where

This closes #3608

commit 0534edaedcc84473c8c26c9d19ad09ef66210246
Author: Pavel Tupitsyn 
Date:   2018-03-27T07:51:49Z

IGNITE-8034 .NET: Add IgniteConfiguration.AuthenticationEnabled

This closes #3696

commit 9478c5a7bd2d97252ab0de72c8b96e7eadc651de
Author: devozerov 
Date:   2018-03-27T10:58:54Z

Merge branch 'ignite-2.4.4' into ignite-2.4-master

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cluster/GridClusterStateProcessor.java

commit 3a0dc509fa44c6b5d5204a06ffa8485e530ca933
Author: Valentin Kulichenko 
Date:   2018-03-20T02:20:20Z

Added new test to suite

Signed-off-by: Valentin Kulichenko 
(cherry picked from commit baf08bc9501676e418dab2431211a659ae421d25)

commit f1fec0e87b5d37850636c04ddf72655fc8ee951f
Author: Alexey Popov 
Date:   2018-03-26T14:00:07Z

IGNITE-7928: .NET: Fixed hang caused by mishandled exception during custom 
cache store deserialization. This closes #3690.

commit 926a2ee11873edc090c1fc06781d369ffae46d44
Author: Denis Mekhanikov 
Date:   2018-03

[GitHub] ignite pull request #4316: IGNITE-8938 Failure handling for file-decompresso...

2018-09-16 Thread agura
Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/4316


---


Cache scan efficiency

2018-09-16 Thread Alexei Scherbakov
Igniters,

My use case involves scenario where it's necessary to iterate over
large(many TBs) persistent cache doing some calculation on read data.

The basic solution is to iterate cache using ScanQuery.

This turns out to be slow because iteration over cache involves a lot of
random disk access for reading data pages referenced from leaf pages by
links.

This is especially true when data is stored on disks with slow random
access, like SAS disks. In my case on modern SAS disks array reading speed
was like several MB/sec while sequential read speed in perf test was about
GB/sec.

I was able to fix the issue by using ScanQuery with explicit partition set
and running simple warmup code before each partition scan.

The code pins cold pages in memory in sequential order thus eliminating
random disk access. Speedup was like x100 magnitude.

I suggest adding the improvement to the product's core  by always
sequentially preloading pages for all internal partition iterations (cache
iterators, scan queries, sql queries with scan plan) if partition is cold
(low number of pinned pages).

This also should speed up rebalancing from cold partitions.

Ignite JIRA ticket [1]

Thoughts ?

[1] https://issues.apache.org/jira/browse/IGNITE-8873

-- 

Best regards,
Alexei Scherbakov


Re: Cache scan efficiency

2018-09-16 Thread Dmitriy Setrakyan
Alexey, this is a great feature. Can you explain what you meant by
"warm-up" when iterating through pages? Do you have this feature already
implemented?

D.

On Sun, Sep 16, 2018 at 6:44 PM Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> Igniters,
>
> My use case involves scenario where it's necessary to iterate over
> large(many TBs) persistent cache doing some calculation on read data.
>
> The basic solution is to iterate cache using ScanQuery.
>
> This turns out to be slow because iteration over cache involves a lot of
> random disk access for reading data pages referenced from leaf pages by
> links.
>
> This is especially true when data is stored on disks with slow random
> access, like SAS disks. In my case on modern SAS disks array reading speed
> was like several MB/sec while sequential read speed in perf test was about
> GB/sec.
>
> I was able to fix the issue by using ScanQuery with explicit partition set
> and running simple warmup code before each partition scan.
>
> The code pins cold pages in memory in sequential order thus eliminating
> random disk access. Speedup was like x100 magnitude.
>
> I suggest adding the improvement to the product's core  by always
> sequentially preloading pages for all internal partition iterations (cache
> iterators, scan queries, sql queries with scan plan) if partition is cold
> (low number of pinned pages).
>
> This also should speed up rebalancing from cold partitions.
>
> Ignite JIRA ticket [1]
>
> Thoughts ?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-8873
>
> --
>
> Best regards,
> Alexei Scherbakov
>


Re: Request for review : IGNITE-3303 Apache Flink Integration - Flink source

2018-09-16 Thread Saikat Maitra
Hi Andrew,

I have updated the tests and also added java docs.

Please review and share feedback.

Regards
Saikat


On Sat, Sep 8, 2018 at 2:09 PM, Saikat Maitra 
wrote:

> Hi Andrew, Alexey
>
> I have incorporated the review changes.
>
> I have also refactored the CacheEventSerializer class and moved it to test
> folder because it is used only in the FlinkIgniteSourceSelfExample and not
> required for IgniteSource.
>
> Build links https://ci.ignite.apache.org/viewLog.html?buildId=1821778&;
>
> https://ci.ignite.apache.org/viewLog.html?buildId=1821774&;
>
> Please review and share feedback.
>
> Regards
> Saikat
>
> On Tue, Sep 4, 2018 at 9:57 PM, Saikat Maitra 
> wrote:
>
>> Hi Alexey,
>>
>> Thank you for reviewing the changes and sharing feedback, I am updating
>> the PR. I will share the changes shortly.
>>
>> Regards,
>> Saikat
>>
>> On Tue, Sep 4, 2018 at 10:59 AM, Alexey Goncharuk <
>> alexey.goncha...@gmail.com> wrote:
>>
>>> Hello Saikat,
>>>
>>> I see a few fellow Igniters added some comments to your PR (including
>>> me).
>>> I believe the PR can be merged after you address them.
>>>
>>> Thanks,
>>> AG
>>>
>>> пт, 31 авг. 2018 г. в 3:11, Saikat Maitra :
>>>
>>> > Thank you, Denis
>>> >
>>> > Regards,
>>> > Saikat
>>> >
>>> > On Thu, Aug 30, 2018 at 7:01 PM, Denis Magda 
>>> wrote:
>>> >
>>> > > Hello Saikat,
>>> > >
>>> > > Hopefully, someone from the community will review the changes in the
>>> > > nearest time.
>>> > >
>>> > > --
>>> > > Denis
>>> > >
>>> > > On Thu, Aug 30, 2018 at 4:37 PM Saikat Maitra <
>>> saikat.mai...@gmail.com>
>>> > > wrote:
>>> > >
>>> > > > Hello,
>>> > > >
>>> > > > The changes for IGNITE-3303 for IgniteSource is complete. This will
>>> > help
>>> > > is
>>> > > > streaming data from Ignite cluster and process, filter, transform
>>> and
>>> > > > publish it back to Ignite using IgniteSink or in any other data
>>> sink.
>>> > > >
>>> > > > I was hoping if the changes can be approved I can go ahead merge
>>> the
>>> > > > changes.
>>> > > >
>>> > > >
>>> > > > Regards,
>>> > > > Saikat
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Tue, Aug 28, 2018 at 12:56 AM, Saikat Maitra <
>>> > saikat.mai...@gmail.com
>>> > > >
>>> > > > wrote:
>>> > > >
>>> > > > > Hi Andrew,
>>> > > > >
>>> > > > > As discussed I have incorporated the changes. Please review and
>>> let
>>> > me
>>> > > > > know if any changes required.
>>> > > > >
>>> > > > > Regards,
>>> > > > > Saikat
>>> > > > >
>>> > > > > On Mon, Aug 27, 2018 at 1:45 AM, Saikat Maitra <
>>> > > saikat.mai...@gmail.com>
>>> > > > > wrote:
>>> > > > >
>>> > > > >> Hi,
>>> > > > >>
>>> > > > >> I have updated the PR with additional tests.
>>> > > > >>
>>> > > > >> Please review and share feedback.
>>> > > > >>
>>> > > > >> This PR is related to IgniteSink but allows to stream data from
>>> > > Ignite.
>>> > > > >>
>>> > > > >> PR https://github.com/apache/ignite/pull/870/files
>>> > > > >>
>>> > > > >> Review https://reviews.ignite.apache.
>>> org/ignite/review/IGNT-CR-135
>>> > > > >>
>>> > > > >> Regards,
>>> > > > >> Saikat
>>> > > > >>
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>


Re: Cache scan efficiency

2018-09-16 Thread Vladimir Ozerov
HI Alex,

This is good that you observed speedup. But I do not think this solution
works for the product in general case. Amount of RAM is limited, and even a
single partition may need more space than RAM available. Moving a lot of
pages to page memory for scan means that you evict a lot of other pages,
what will ultimately lead to bad performance of subsequent queries and
defeat LRU algorithms, which are of great improtance for good database
performance.

Database vendors choose another approach - skip BTrees, iterate direclty
over data pages, read them in multi-block fashion, use separate scan buffer
to avoid excessive evictions of other hot pages. Corresponding ticket for
SQL exists [1], but idea is common for all parts of the system, requiring
scans.

As far as proposed solution, it might be good idea to add special API to
"warmup" partition with clear explanation of pros (fast scan after warmup)
and cons (slowdown of any other operations). But I think we should not make
this approach part of normal scans.

Vladimir.

[1] https://issues.apache.org/jira/browse/IGNITE-6057


On Sun, Sep 16, 2018 at 6:44 PM Alexei Scherbakov <
alexey.scherbak...@gmail.com> wrote:

> Igniters,
>
> My use case involves scenario where it's necessary to iterate over
> large(many TBs) persistent cache doing some calculation on read data.
>
> The basic solution is to iterate cache using ScanQuery.
>
> This turns out to be slow because iteration over cache involves a lot of
> random disk access for reading data pages referenced from leaf pages by
> links.
>
> This is especially true when data is stored on disks with slow random
> access, like SAS disks. In my case on modern SAS disks array reading speed
> was like several MB/sec while sequential read speed in perf test was about
> GB/sec.
>
> I was able to fix the issue by using ScanQuery with explicit partition set
> and running simple warmup code before each partition scan.
>
> The code pins cold pages in memory in sequential order thus eliminating
> random disk access. Speedup was like x100 magnitude.
>
> I suggest adding the improvement to the product's core  by always
> sequentially preloading pages for all internal partition iterations (cache
> iterators, scan queries, sql queries with scan plan) if partition is cold
> (low number of pinned pages).
>
> This also should speed up rebalancing from cold partitions.
>
> Ignite JIRA ticket [1]
>
> Thoughts ?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-8873
>
> --
>
> Best regards,
> Alexei Scherbakov
>


[GitHub] ignite pull request #4767: IGNITE-7953: MVCC TX: continuous queriesю

2018-09-16 Thread rkondakov
GitHub user rkondakov opened a pull request:

https://github.com/apache/ignite/pull/4767

IGNITE-7953: MVCC TX: continuous queriesю



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7953

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4767.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4767


commit d256cf8db85ca84c39f1c5b78829979c9d14d3fb
Author: rkondakov 
Date:   2018-09-16T18:27:02Z

IGNITE-7953: MVCC CQ. Prototype WIP.




---


Re: Cache scan efficiency

2018-09-16 Thread Dmitriy Pavlov
Hi Alexei,

I did not find any PRs associated with the ticket for check code changes
behind this idea. Are there any PRs?

If we create some forwards scan of pages, it should be a very intellectual
algorithm including a lot of parameters (how much RAM is free, how probably
we will need next page, etc). We had the private talk about such idea some
time ago.

By my experience, Linux systems already do such forward reading of file
data (for corresponding sequential flagged file descriptors), but some
prefetching of data at the level of application may be useful for O_DIRECT
file descriptors.

And one more concern from me is about selecting a right place in the system
to do such prefetch.

Sincerely,
Dmitriy Pavlov

вс, 16 сент. 2018 г. в 19:54, Vladimir Ozerov :

> HI Alex,
>
> This is good that you observed speedup. But I do not think this solution
> works for the product in general case. Amount of RAM is limited, and even a
> single partition may need more space than RAM available. Moving a lot of
> pages to page memory for scan means that you evict a lot of other pages,
> what will ultimately lead to bad performance of subsequent queries and
> defeat LRU algorithms, which are of great improtance for good database
> performance.
>
> Database vendors choose another approach - skip BTrees, iterate direclty
> over data pages, read them in multi-block fashion, use separate scan buffer
> to avoid excessive evictions of other hot pages. Corresponding ticket for
> SQL exists [1], but idea is common for all parts of the system, requiring
> scans.
>
> As far as proposed solution, it might be good idea to add special API to
> "warmup" partition with clear explanation of pros (fast scan after warmup)
> and cons (slowdown of any other operations). But I think we should not make
> this approach part of normal scans.
>
> Vladimir.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-6057
>
>
> On Sun, Sep 16, 2018 at 6:44 PM Alexei Scherbakov <
> alexey.scherbak...@gmail.com> wrote:
>
> > Igniters,
> >
> > My use case involves scenario where it's necessary to iterate over
> > large(many TBs) persistent cache doing some calculation on read data.
> >
> > The basic solution is to iterate cache using ScanQuery.
> >
> > This turns out to be slow because iteration over cache involves a lot of
> > random disk access for reading data pages referenced from leaf pages by
> > links.
> >
> > This is especially true when data is stored on disks with slow random
> > access, like SAS disks. In my case on modern SAS disks array reading
> speed
> > was like several MB/sec while sequential read speed in perf test was
> about
> > GB/sec.
> >
> > I was able to fix the issue by using ScanQuery with explicit partition
> set
> > and running simple warmup code before each partition scan.
> >
> > The code pins cold pages in memory in sequential order thus eliminating
> > random disk access. Speedup was like x100 magnitude.
> >
> > I suggest adding the improvement to the product's core  by always
> > sequentially preloading pages for all internal partition iterations
> (cache
> > iterators, scan queries, sql queries with scan plan) if partition is cold
> > (low number of pinned pages).
> >
> > This also should speed up rebalancing from cold partitions.
> >
> > Ignite JIRA ticket [1]
> >
> > Thoughts ?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-8873
> >
> > --
> >
> > Best regards,
> > Alexei Scherbakov
> >
>


[GitHub] ignite pull request #3134: Ignite-602

2018-09-16 Thread agura
Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/3134


---


[GitHub] ignite pull request #4707: IGNITE-9384 Transaction status can be set incorre...

2018-09-16 Thread agura
Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/4707


---


[GitHub] ignite pull request #3159: Ignite-602-raw

2018-09-16 Thread agura
Github user agura closed the pull request at:

https://github.com/apache/ignite/pull/3159


---


how to use kafka 2.0 in ignite-kafka

2018-09-16 Thread kcheng.mvp
Is there any plan to migrate kafka 2.0 in ignite-kafka?



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/


[jira] [Created] (IGNITE-9608) Fix buttons on start demo

2018-09-16 Thread Alexander Kalinin (JIRA)
Alexander Kalinin created IGNITE-9608:
-

 Summary: Fix buttons on start demo
 Key: IGNITE-9608
 URL: https://issues.apache.org/jira/browse/IGNITE-9608
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Reporter: Alexander Kalinin
Assignee: Alexander Kalinin


!https://snag.gy/HrOW1Y.jpg!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (IGNITE-9609) Web console: update to AngularJS 1.7.4

2018-09-16 Thread Ilya Borisov (JIRA)
Ilya Borisov created IGNITE-9609:


 Summary: Web console: update to AngularJS 1.7.4
 Key: IGNITE-9609
 URL: https://issues.apache.org/jira/browse/IGNITE-9609
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Ilya Borisov
Assignee: Ilya Borisov


Let's update package-.json to use AngularJS 1.7.4, the 1.7.3 release introduced 
some interesting new feature we might use (like extra form methods and 
arbitrary event/property bindings).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: how to use kafka 2.0 in ignite-kafka

2018-09-16 Thread Dmitriy Pavlov
Hi, Actually there is no such plan for 2.7. In 2.7 we will probably switch
to 1.1.1.

Sincerely,
Dmitriy Pavlov

пн, 17 сент. 2018 г. в 5:22, kcheng.mvp :

> Is there any plan to migrate kafka 2.0 in ignite-kafka?
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>