[GitHub] ignite pull request #3439: Ignite 7192 :: JDBC support FQDN to multiple IPs ...

[gromtech]

GitHub user gromtech opened a pull request:

https://github.com/apache/ignite/pull/3439

Ignite 7192 :: JDBC support FQDN to multiple IPs during connection 
establishment

* Implemented connection to host's IP-addresses one by one.
* Added unit tests.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGNITE-7192

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3439.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3439


commit 4c93f854e0b1d02f154eaf24b0033b8470cb78d8
Author: Roman Guseinov 
Date:   2018-01-25T16:22:54Z

IGNITE-7192 :: JDBC support FQDN to multiple IPs

commit eb7184f223ab24a3ea3140e4973099c70baa8950
Author: Roman Guseinov 
Date:   2018-01-26T07:19:37Z

IGNITE-7192 :: Updated doc comments




---

Re: Ignite Semaphore Bug 7090

[Tim Onyschak]

It does not look like i have permissions to assign tickets. I'll gladly
take it, if somebody can assign it to me.

Also, Vlad i updated the ticket with my comment as well.


On Thu, Jan 25, 2018 at 7:44 PM, Yakov Zhdanov  wrote:

> Vlad and Tim, thanks for working on this!
>
> Tim, please assign ticket to yourself to follow community process.
> Currently I see it is unassigned.
>
> --Yakov
>
> 2018-01-25 8:53 GMT-08:00 Vladisav Jelisavcic :
>
>> Hi Tim,
>>
>> thank you for delving deeper into the problem,
>> I left you a comment/question on the JIRA.
>>
>> Best regards,
>> Vladisav
>>
>> On Tue, Jan 23, 2018 at 3:00 PM, Vladisav Jelisavcic > >
>> wrote:
>>
>> > Hi Tim,
>> >
>> > thanks for the update! I left you a comment on Jira.
>> >
>> > Best regards,
>> > Vladisav
>> >
>> > On Mon, Jan 22, 2018 at 6:17 PM, Tim Onyschak 
>> wrote:
>> >
>> >> Hey Vladisav,
>> >>
>> >> I implemented your requests. Take a look, specifically, i created an
>> >> interface to encapsulate the NodeUpdates and let
>> >> the DataStructuresProcessor handle the execution by checking for one
>> type
>> >> as opposed to multiple if checks. In this case it checks for
>> GridCacheNodeUpdate
>> >> instance and executes onNodeRemoved. Let me know what you think.
>> >>
>> >> Thanks,
>> >> Tim
>> >>
>> >>
>> >>
>> >> On Sat, Jan 20, 2018 at 8:10 AM, Vladisav Jelisavcic <
>> vladis...@gmail.com
>> >> > wrote:
>> >>
>> >>> Hi Tim,
>> >>>
>> >>> I reviewed your contribution and left you some comments on the pr.
>> >>> Thanks!
>> >>>
>> >>> Vladisav
>> >>>
>> >>> On Wed, Jan 17, 2018 at 10:14 PM, Vladisav Jelisavcic <
>> >>> vladis...@gmail.com> wrote:
>> >>>
>>  Hi Tim,
>> 
>>  thank you for the contribution!
>>  I'll do the review soon and let you know.
>> 
>> 
>> 
>>  On Wed, Jan 17, 2018 at 8:56 AM, Yakov Zhdanov 
>>  wrote:
>> 
>> > Thanks Tim! I hope Vlad can review your patch. If this does not
>> happen
>> > in
>> > 2-3 days I will take a look. Can you please let me know on weekend
>> if I
>> > need to?
>> >
>> > --Yakov
>> >
>> > 2018-01-16 23:36 GMT+03:00 Tim Onyschak :
>> >
>> > > Hey all,
>> > >
>> > > I created a patch and posted to user group, was told feed back
>> would
>> > be
>> > > left on the jira. I wanted to see if we could get a fix in with
>> 2.4,
>> > could
>> > > somebody please review.
>> > >
>> > > http://apache-ignite-users.70518.x6.nabble.com/Semaphore-
>> > > Stuck-when-no-acquirers-to-assign-permit-td18639.html
>> > >
>> > > https://issues.apache.org/jira/browse/IGNITE-7090
>> > >
>> > > https://github.com/apache/ignite/pull/3138
>> > >
>> > > Thanks,
>> > > Tim
>> > >
>> >
>> 
>> 
>> >>>
>> >>
>> >
>>
>
>

Re: Ignite Semaphore Bug 7090

[Yakov Zhdanov]

Vlad and Tim, thanks for working on this!

Tim, please assign ticket to yourself to follow community process.
Currently I see it is unassigned.

--Yakov

2018-01-25 8:53 GMT-08:00 Vladisav Jelisavcic :

> Hi Tim,
>
> thank you for delving deeper into the problem,
> I left you a comment/question on the JIRA.
>
> Best regards,
> Vladisav
>
> On Tue, Jan 23, 2018 at 3:00 PM, Vladisav Jelisavcic 
> wrote:
>
> > Hi Tim,
> >
> > thanks for the update! I left you a comment on Jira.
> >
> > Best regards,
> > Vladisav
> >
> > On Mon, Jan 22, 2018 at 6:17 PM, Tim Onyschak 
> wrote:
> >
> >> Hey Vladisav,
> >>
> >> I implemented your requests. Take a look, specifically, i created an
> >> interface to encapsulate the NodeUpdates and let
> >> the DataStructuresProcessor handle the execution by checking for one
> type
> >> as opposed to multiple if checks. In this case it checks for
> GridCacheNodeUpdate
> >> instance and executes onNodeRemoved. Let me know what you think.
> >>
> >> Thanks,
> >> Tim
> >>
> >>
> >>
> >> On Sat, Jan 20, 2018 at 8:10 AM, Vladisav Jelisavcic <
> vladis...@gmail.com
> >> > wrote:
> >>
> >>> Hi Tim,
> >>>
> >>> I reviewed your contribution and left you some comments on the pr.
> >>> Thanks!
> >>>
> >>> Vladisav
> >>>
> >>> On Wed, Jan 17, 2018 at 10:14 PM, Vladisav Jelisavcic <
> >>> vladis...@gmail.com> wrote:
> >>>
>  Hi Tim,
> 
>  thank you for the contribution!
>  I'll do the review soon and let you know.
> 
> 
> 
>  On Wed, Jan 17, 2018 at 8:56 AM, Yakov Zhdanov 
>  wrote:
> 
> > Thanks Tim! I hope Vlad can review your patch. If this does not
> happen
> > in
> > 2-3 days I will take a look. Can you please let me know on weekend
> if I
> > need to?
> >
> > --Yakov
> >
> > 2018-01-16 23:36 GMT+03:00 Tim Onyschak :
> >
> > > Hey all,
> > >
> > > I created a patch and posted to user group, was told feed back
> would
> > be
> > > left on the jira. I wanted to see if we could get a fix in with
> 2.4,
> > could
> > > somebody please review.
> > >
> > > http://apache-ignite-users.70518.x6.nabble.com/Semaphore-
> > > Stuck-when-no-acquirers-to-assign-permit-td18639.html
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-7090
> > >
> > > https://github.com/apache/ignite/pull/3138
> > >
> > > Thanks,
> > > Tim
> > >
> >
> 
> 
> >>>
> >>
> >
>

Re: Java 9 support

[Yakov Zhdanov]

Did we add what Pavel suggested to README.txt and readme.io documentation?

Yakov Zhdanov,
www.gridgain.com

2018-01-25 14:27 GMT-08:00 Denis Magda :

> Pavel, it’s a good point.
>
> Peter, could you ensure that all Ignite scripts (ignite.sh/bat,
> control.sh/bat, etc.) perform this auto detection of JVM 9 as well? Alex
> K. please do the same for Visor and Web Console scripts.
>
> —
> Denis
>
> > On Jan 25, 2018, at 1:58 AM, Pavel Tupitsyn 
> wrote:
> >
> > I would add that to run Ignite on Java 9 without default scripts one has
> to
> > use the following JVM options:
> >
> >--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED"
> >--add-exports=java.base/sun.nio.ch=ALL-UNNAMED
> >
> > --add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED
> >
> > --add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED
> >
> > Ignite.NET adds these options automatically when Java 9 is detected, no
> > user steps required.
> >
> > Thanks,
> > Pavel
> >
> >
> > On Thu, Jan 25, 2018 at 12:53 PM, vveider  wrote:
> >
> >> Hi, Igniters!
> >>
> >>
> >> Ticket IGNITE-6730 [1] was merged to master (and ignite-2.4) and now we
> >> have
> >> preliminary support of Java 9, which includes:
> >> - compilation with JDK9 with some constraints (scala-2.10 based modules
> >> should be turned off)
> >> - run with JRE9/JDK9 through default scripts (ignite.{sh|bat})
> >>
> >> Also, temporary TC project for Ignite Tests on Java 9 [2] was prepared;
> >> analysis of test problems and corresponding fixes is in progress.
> >>
> >> Please, be advised.
> >>
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-6730
> >> [2] https://ci.ignite.apache.org/project.html?projectId=
> IgniteTests24Java9
> >>
> >>
> >>
> >> --
> >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>
>
>

Re: Partition loss policy to disable cache completely

[Yakov Zhdanov]

No. This is not 100% consistent. Since operations started on prev version
after node has left (but system has not got event yet) would succeed. For
me consistent behavior is to throw exception for "select avg(x) from bla"
if data is currently missing or any data loss occurs in the middle of the
query and return result for cache.get(key); if partition for that key is
still in the grid.

--Yakov

Re: Partition loss policy to disable cache completely

[Valentin Kulichenko]

Yakov,

My suggestion is to introduce new policy (e.g. READ_WRITE_NONE) which works
in the same way as READ_WRITE_SAFE but for all partitions, not only lost
ones. Any operation attempted on a topology version which has at least one
lost partition, should throw an exception. Will this work?

-Val

On Tue, Jan 23, 2018 at 5:20 PM, Yakov Zhdanov  wrote:

> I'm still not sure on what Val has suggested. Dmitry, Val, Do you have any
> concrete API/algorithm in mind?
>
> --Yakov
>

Next Steps: GA Grid: Request to contribute GA library to Apache Ignite

[techbysample]

Denis/Yury, 

Please advise on next steps.

JIRA:  https://issues.apache.org/jira/browse/IGNITE-6899

Per recent comments from Oleg Ignatenko:

"...most recent change looks good to me: the typo in MovieFitnessFunction is
fixed and the rest of code is the same as it was ie everything now looks
right.."

Please advise.

Best,
Turik Campbell




--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Java 9 support

[Denis Magda]

Pavel, it’s a good point.

Peter, could you ensure that all Ignite scripts (ignite.sh/bat, control.sh/bat, 
etc.) perform this auto detection of JVM 9 as well? Alex K. please do the same 
for Visor and Web Console scripts.

—
Denis

> On Jan 25, 2018, at 1:58 AM, Pavel Tupitsyn  wrote:
> 
> I would add that to run Ignite on Java 9 without default scripts one has to
> use the following JVM options:
> 
>--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED"
>--add-exports=java.base/sun.nio.ch=ALL-UNNAMED
> 
> --add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED
> 
> --add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED
> 
> Ignite.NET adds these options automatically when Java 9 is detected, no
> user steps required.
> 
> Thanks,
> Pavel
> 
> 
> On Thu, Jan 25, 2018 at 12:53 PM, vveider  wrote:
> 
>> Hi, Igniters!
>> 
>> 
>> Ticket IGNITE-6730 [1] was merged to master (and ignite-2.4) and now we
>> have
>> preliminary support of Java 9, which includes:
>> - compilation with JDK9 with some constraints (scala-2.10 based modules
>> should be turned off)
>> - run with JRE9/JDK9 through default scripts (ignite.{sh|bat})
>> 
>> Also, temporary TC project for Ignite Tests on Java 9 [2] was prepared;
>> analysis of test problems and corresponding fixes is in progress.
>> 
>> Please, be advised.
>> 
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-6730
>> [2] https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java9
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> 

[GitHub] ignite pull request #3438: IGNITE-7337

[nizhikov]

GitHub user nizhikov opened a pull request:

https://github.com/apache/ignite/pull/3438

IGNITE-7337

Implementation of saving DataFrame to Ignite SQL tables.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nizhikov/ignite IGNITE-7337

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3438.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3438


commit 182e26314f19bd76ead5fa789eea51ae1d097ab1
Author: Nikolay Izhikov 
Date:   2018-01-24T06:37:48Z

IGNITE-7337: Commit in the middle on initial development

commit 04d8697380934558136087a09515972115dd4343
Author: Nikolay Izhikov 
Date:   2018-01-24T06:50:33Z

Merge branch 'master' into IGNITE-7337

commit b47925b6593a1dca888522873793b681d6780d89
Author: Nikolay Izhikov 
Date:   2018-01-24T09:30:00Z

IGNITE-7337: Commit in the middle on initial development

commit 13efd32164082f2abac2e801ece506ff8324ae6f
Author: Nikolay Izhikov 
Date:   2018-01-24T09:33:53Z

Merge branch 'master' into IGNITE-7337

commit 200387d3117506870d0aa9b7e63106a1f6765f21
Author: Nikolay Izhikov 
Date:   2018-01-24T09:48:28Z

Merge branch 'master' into IGNITE-7337

commit fcfb5cbcf02641c88c0d900f56254c2c86406382
Author: Nikolay Izhikov 
Date:   2018-01-24T12:57:29Z

IGNITE-7337: Commit in the middle on initial development

commit 333ee97054d86043348d791da5fc3f9a86b793a8
Author: Nikolay Izhikov 
Date:   2018-01-25T10:08:37Z

IGNITE-7337: Commit in the middle on initial development

commit dc89aa5e565ad99e3410f84685f0a847321203e5
Author: Nikolay Izhikov 
Date:   2018-01-25T18:37:11Z

IGNITE-7337: All tests are passed.




---

[GitHub] ignite pull request #3437: IGNITE-7533: Throttle writing threads according f...

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/3437

IGNITE-7533: Throttle writing threads according fsync progress 

and checkpoint write speed

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7533

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3437.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3437


commit 90befa9d3bc89833eeed42a8814e8a7f51a11448
Author: dpavlov 
Date:   2018-01-25T18:11:07Z

IGNITE-7533: Throttle writing threads according fsync progress and 
checkpoint write speed




---

Re: Ignite Semaphore Bug 7090

[Vladisav Jelisavcic]

Hi Tim,

thank you for delving deeper into the problem,
I left you a comment/question on the JIRA.

Best regards,
Vladisav

On Tue, Jan 23, 2018 at 3:00 PM, Vladisav Jelisavcic 
wrote:

> Hi Tim,
>
> thanks for the update! I left you a comment on Jira.
>
> Best regards,
> Vladisav
>
> On Mon, Jan 22, 2018 at 6:17 PM, Tim Onyschak  wrote:
>
>> Hey Vladisav,
>>
>> I implemented your requests. Take a look, specifically, i created an
>> interface to encapsulate the NodeUpdates and let
>> the DataStructuresProcessor handle the execution by checking for one type
>> as opposed to multiple if checks. In this case it checks for 
>> GridCacheNodeUpdate
>> instance and executes onNodeRemoved. Let me know what you think.
>>
>> Thanks,
>> Tim
>>
>>
>>
>> On Sat, Jan 20, 2018 at 8:10 AM, Vladisav Jelisavcic > > wrote:
>>
>>> Hi Tim,
>>>
>>> I reviewed your contribution and left you some comments on the pr.
>>> Thanks!
>>>
>>> Vladisav
>>>
>>> On Wed, Jan 17, 2018 at 10:14 PM, Vladisav Jelisavcic <
>>> vladis...@gmail.com> wrote:
>>>
 Hi Tim,

 thank you for the contribution!
 I'll do the review soon and let you know.



 On Wed, Jan 17, 2018 at 8:56 AM, Yakov Zhdanov 
 wrote:

> Thanks Tim! I hope Vlad can review your patch. If this does not happen
> in
> 2-3 days I will take a look. Can you please let me know on weekend if I
> need to?
>
> --Yakov
>
> 2018-01-16 23:36 GMT+03:00 Tim Onyschak :
>
> > Hey all,
> >
> > I created a patch and posted to user group, was told feed back would
> be
> > left on the jira. I wanted to see if we could get a fix in with 2.4,
> could
> > somebody please review.
> >
> > http://apache-ignite-users.70518.x6.nabble.com/Semaphore-
> > Stuck-when-no-acquirers-to-assign-permit-td18639.html
> >
> > https://issues.apache.org/jira/browse/IGNITE-7090
> >
> > https://github.com/apache/ignite/pull/3138
> >
> > Thanks,
> > Tim
> >
>


>>>
>>
>

[jira] [Created] (IGNITE-7542) SQL COPY command: implement multipliers (e.g., 'K' for kilo-, 'M' for mega-) in numbers

[Kirill Shirokov (JIRA)]

Kirill Shirokov created IGNITE-7542:
---

 Summary: SQL COPY command: implement multipliers (e.g., 'K' for 
kilo-, 'M' for mega-) in numbers
 Key: IGNITE-7542
 URL: https://issues.apache.org/jira/browse/IGNITE-7542
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Kirill Shirokov


Multipliers can be important for several options, such as batch size.

The syntax could be:

{noformat}
COPY
...
BATCH_SIZE 1.5M
{noformat}

Such number can have a decimal point and an optional multiplier:

K: 1024
M: 1024 * 1024
...and so on for tera- and peta-



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7541) SQL COPY command: implement backend switching option

[Kirill Shirokov (JIRA)]

Kirill Shirokov created IGNITE-7541:
---

 Summary: SQL COPY command: implement backend switching option
 Key: IGNITE-7541
 URL: https://issues.apache.org/jira/browse/IGNITE-7541
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Kirill Shirokov


When we load data using COPY command we can add key/value pairs to the cache 
using different ways:

* Directly calling cache.putAll()
* Loading data via DataStreamer
* etc.

Every backend has its pros and cons. For example, the streamer is fast and 
asynchronous, although it cannot replace value and cannot provide any 
statistics -- such as number of added records. The direct interface is slow and 
synchronous, but provides us with an ability to either replace or skip the 
records with the same key and respond to the user with full statistics.

There shall be an option in the SQL command to switch to particular backend. 
For example:

{noformat}
COPY
FROM 'file'
...
[BACKEND (DIRECT | STREAMER)]

We might have more backends in the future.
{noformat}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7540) Sequential checkpoints cause overwrite of already cleaned & freed offheap page

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-7540:
---

 Summary: Sequential checkpoints cause overwrite of already cleaned 
& freed offheap page
 Key: IGNITE-7540
 URL: https://issues.apache.org/jira/browse/IGNITE-7540
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.4
Reporter: Ilya Kasnacheev
Assignee: Alexey Goncharuk


The sequence of events as follows:

in GridCacheProcessor.onExchangeDone(), 
{color:#660e7a}sharedCtx{color}.database().waitForCheckpoint({color:#008000}"caches
 stop"{color}) is peformed and then cache is destroyed and all its pages are 
freed and cleared asynchronously.

However, it is entirely possible that after waitForCheckpoint(), next 
checkpoint will start immediately. It is typical when a lot of data being 
loaded into Ignite, leading to rapid checkpoint buffer depletion, as well as 
with artificially increased checkpoint frequency, as used in reproducer.

Then, checkpointer will save (overwrite) metadata page:
{code:java}
    at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlockPage(PageMemoryImpl.java:1330)
    at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:428)
    at 
org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryImpl.writeUnlock(PageMemoryImpl.java:422)
    at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.saveStoreMetadata(GridCacheOffheapManager.java:375)
    at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.onCheckpointBegin(GridCacheOffheapManager.java:163)
    at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.markCheckpointBegin(GridCacheDatabaseSharedManager.java:2309)
    at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.doCheckpoint(GridCacheDatabaseSharedManager.java:2088)
    at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager$Checkpointer.body(GridCacheDatabaseSharedManager.java:2013)
    at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
    at java.lang.Thread.run(Thread.java:748){code}
This will happen after cache is already destroyed and even after the page is 
already zeroed by PageMemoryImpl$ClearSegmentRunnable.run().

Then, some new cache is being created, and in 
GridCacheOffheapManager$GridCacheDataStore.getOrAllocatePartitionMetas(), 
pageMem.acquirePage() will return this page, expected zeroed, but actually 
containing metadata for old cache's partition. Then, type == PageIO.T_PART_META 
check will return true and the following exception is issued, leading to cache 
state inconsistency and data loss:
{code:java}
Caused by: java.lang.IllegalStateException: Failed to get page IO instance 
(page content is corrupted)
    at 
org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions.forVersion(IOVersions.java:83)
    at 
org.apache.ignite.internal.processors.cache.persistence.tree.io.IOVersions.forPage(IOVersions.java:95)
    at 
org.apache.ignite.internal.processors.cache.persistence.freelist.PagesList.init(PagesList.java:175)
    at 
org.apache.ignite.internal.processors.cache.persistence.freelist.FreeListImpl.(FreeListImpl.java:370)
    at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore$1.(GridCacheOffheapManager.java:932)
    at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.init0(GridCacheOffheapManager.java:929)
    at 
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:1295)
    at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:344)
    at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3191)
    at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:2571)
    at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2096)
    at 
org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140)
    at 
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.localUpdate(DataStreamProcessor.java:397)
    at 
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:302)
    at 
org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:59)
    at 

[jira] [Created] (IGNITE-7539) SQL COPY command: implement NULL escape sequence handling

[Kirill Shirokov (JIRA)]

Kirill Shirokov created IGNITE-7539:
---

 Summary: SQL COPY command: implement NULL escape sequence handling
 Key: IGNITE-7539
 URL: https://issues.apache.org/jira/browse/IGNITE-7539
 Project: Ignite
  Issue Type: Improvement
Reporter: Kirill Shirokov


There should be a way to specify NULL field value in a file, which we import 
via COPY command.

A popular solution among vendors for CSV format is to have a special escape 
sequence for a NULL value in a field: \N for example.

We need to have a possibility to enable/disable null escape sequence handling 
for all fields or on per-field basis.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7538) Introduce new test project for Ignite 2.0+ with Java 8 / Java 9 compatibility

[Peter Ivanov (JIRA)]

Peter Ivanov created IGNITE-7538:


 Summary: Introduce new test project for Ignite 2.0+ with Java 8 / 
Java 9 compatibility
 Key: IGNITE-7538
 URL: https://issues.apache.org/jira/browse/IGNITE-7538
 Project: Ignite
  Issue Type: Sub-task
Reporter: Peter Ivanov
Assignee: Peter Ivanov


After IGNITE-7203 and IGNITE-6730 there are two separate test project at CI 
which meet the problem of test configuration synchronization. It is necessary 
to devise a solution for overcoming this issue.

Possible approaches:
# Single project for tests with run with different parameters.
Problems:
#* Test History will show history for all builds with any parameters 
combination.
#* Mute test will mute test for all parameters configuration.
# Several project (differentiated by parameter) with build configuration 
synchronisation automation.
Problems:
#* Maintainability — how to seamlessly support build configuration 
synchronisation.

Research for both approaches must be held.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7537) SQL COPY command: implement CSV format options

[Kirill Shirokov (JIRA)]

Kirill Shirokov created IGNITE-7537:
---

 Summary: SQL COPY command: implement CSV format options
 Key: IGNITE-7537
 URL: https://issues.apache.org/jira/browse/IGNITE-7537
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Kirill Shirokov


The following options can be implemented for the CSV format:

* Line separator pattern
* Field separator pattern
* Quoting characters

Escape sequences support are important for this feature. There is a code for 
handling escape sequences in branch ignite-7372 (see IGNITE-7372 for details).




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7536) Document baseline topology feature and its WebConsole screen

[Denis Magda (JIRA)]

Denis Magda created IGNITE-7536:
---

 Summary: Document baseline topology feature and its WebConsole 
screen 
 Key: IGNITE-7536
 URL: https://issues.apache.org/jira/browse/IGNITE-7536
 Project: Ignite
  Issue Type: Task
Reporter: Denis Magda
Assignee: Denis Magda
 Fix For: 2.4


Document Base Line topology:

[https://cwiki.apache.org/confluence/display/IGNITE/IEP-4+Baseline+topology+for+caches]

and how to manage it with Web Console and control.sh script (Ignite activation 
doc has to be updated).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7535) SQL COPY command: implement encoding option

[Kirill Shirokov (JIRA)]

Kirill Shirokov created IGNITE-7535:
---

 Summary: SQL COPY command: implement encoding option
 Key: IGNITE-7535
 URL: https://issues.apache.org/jira/browse/IGNITE-7535
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Kirill Shirokov


The syntax can be something like:

{noformat}
COPY 
FROM 'file' [CHARSET ]
INTO ...
{noformat}

CHARSET is optional. By default the encoding is UTF-8.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7534) ClusterGroupExample duplicate

[Sergey Kozlov (JIRA)]

Sergey Kozlov created IGNITE-7534:
-

 Summary: ClusterGroupExample duplicate
 Key: IGNITE-7534
 URL: https://issues.apache.org/jira/browse/IGNITE-7534
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Sergey Kozlov
 Fix For: 2.4


The examples \{{org.apache.ignite.examples.cluster.ClusterGroupExample}} and 
\{{org.apache.ignite.examples.computegrid.cluster.ClusterGroupExample}} are same



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7533) Throttle writting threads according fsync progress and checkpoint writting speed instead of region fill

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-7533:
--

 Summary: Throttle writting threads according fsync progress and 
checkpoint writting speed instead of region fill
 Key: IGNITE-7533
 URL: https://issues.apache.org/jira/browse/IGNITE-7533
 Project: Ignite
  Issue Type: Improvement
  Components: persistence
Affects Versions: 2.3
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov
 Fix For: 2.5
 Attachments: image (2).png

Throttling implemented under IGNITE-6334 is based on region fill percentage 
(ditry pages ratio) and current checkpoint progress.

But actual progress of writting is based on write operation complete, but not 
on fsync() complete.

Suppose following stage of CP is currently running: most of data is being 
written and fsync is started. Fsync on experiments requires more time than 
write, but throttling is disabled for that stage. There is enough time to 
unthrottled grid to fill remaining 16% of clear pages to get sufficient 75% of 
dirty pages and writes stops.

Fsync progress is to be included in checkpoint progress, but actual fsync 
progress reported by OS is irregular both on Linux and Windows. See picture, 
green line is fsync progress, and yellow is write complete.

Because fsync progress reported is not regular (the differences are 3-4 orders 
of magnitude) it is suggested to implement new speed based throttling policy.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7532) kNN Documentation

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7532:


 Summary: kNN Documentation
 Key: IGNITE-7532
 URL: https://issues.apache.org/jira/browse/IGNITE-7532
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev
 Fix For: 2.4






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3436: IGNITE-7530 .NET: Fix memory usage and performanc...

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/3436

IGNITE-7530 .NET: Fix memory usage and performance for GetAll and query 
cursors in binary mode



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ptupitsyn/ignite ignite-7530

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3436.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3436


commit 219bbf1fc6c2808a16f881a7f2e41adf51fd2916
Author: apopov 
Date:   2018-01-25T09:58:16Z

GetAllBinaryBenchmark issue

commit bb60c552952cc6c14204fc1def9222664cb6bfd3
Author: Pavel Tupitsyn 
Date:   2018-01-25T11:34:11Z

Fix binary object excessive memory copy issue.

commit 0ea15d3e45c915b40aa25c564468095cbce307ee
Author: Pavel Tupitsyn 
Date:   2018-01-25T12:27:47Z

Merge branch 'master' into get-all-binary-issue

commit cc1943e7dc81d6ca1898acbea0d8d0fa0e891a23
Author: Pavel Tupitsyn 
Date:   2018-01-25T12:49:53Z

IGNITE-7530 .NET: Poor performance & excessive memory usage in GetAll and 
query cursors in binary mode

commit 3fd6e47894d4996a3b65b63bd112c65de0082f1f
Author: Pavel Tupitsyn 
Date:   2018-01-25T13:24:16Z

Add CanGetArray guard

commit 725757e0f9f482a2f55bd6dd5248bf4d8b2f71a4
Author: Pavel Tupitsyn 
Date:   2018-01-25T13:31:28Z

Fixing tests

commit db4f89afaa4e905ba08ef8269cca512580e8b9a3
Author: Pavel Tupitsyn 
Date:   2018-01-25T13:43:09Z

цшз

commit e83c1465cd659542c04f5f305b7fc2daad6f2338
Author: Pavel Tupitsyn 
Date:   2018-01-25T13:45:26Z

Code cleanup




---

[GitHub] ignite pull request #3435: IGNITE-7317: SVM Examples

[zaleslaw]

GitHub user zaleslaw opened a pull request:

https://github.com/apache/ignite/pull/3435

IGNITE-7317: SVM Examples

1. Added titanic dataset
2. Fixed pom.xml with licenses
3. Added example
4. Moved datasets to common package
5. Fixed kNN samples

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7317

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3435.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3435


commit b84d1b6ae284edaa510f7eefa84f5594dd1cb83e
Author: zaleslaw 
Date:   2017-11-17T14:44:05Z

Removed incorrect patch

commit 401a464c6bde19bbf1f11d0d32a2aea9facc
Author: zaleslaw 
Date:   2017-11-17T15:11:09Z

Added OLS Distributed support

commit b304deb349a95a8d2f9e62cd53465824a85a13bf
Author: zaleslaw 
Date:   2017-12-08T10:43:08Z

Merge branch 'master' of https://github.com/apache/ignite

commit 10146937099ba58343cf3fbca22d598ab53cd1b0
Author: zaleslaw 
Date:   2017-12-21T09:44:54Z

Merge branch 'master' of https://github.com/apache/ignite

commit 831eda56be1c686f6eba5802ae719dafa96b199a
Author: zaleslaw 
Date:   2018-01-25T11:38:49Z

Merge branch 'master' of https://github.com/apache/ignite

commit a487f529de7504dbc33be1fbdca2431eb2de0070
Author: zaleslaw 
Date:   2018-01-25T13:30:22Z

IGNITE-7317: Added sample, moved datasets, fixed lincenses




---

Re: Ignite diagnostic (SQL system views)

[Alex Plehanov]

Anton, Vladimir, I've made some fixes. There is only one view left and it's
renamed to 'IGNITE.LOCAL_TRANSACTIONS'.

High level design of solution:
When IgniteH2Indexing is starting, it create and start
new GridH2SysViewProcessor, which create and register in H2 (via its own
table engine) all implementations of system views. Each system view
implementation extends base abstract class GridH2SysView. View
implementation describes columns, their types and indexes in constructor
and must override method getRows for data retrieval (this method called by
H2-compatible table and index implementations for ignite system views).
Almost no fixes to existing parsing engine was made, except some places,
where GridH2Table instance was expected, but for system views there is
another class.

New PR: [1].  Please have a look.

[1] https://github.com/apache/ignite/pull/3433

2018-01-24 19:12 GMT+03:00 Anton Vinogradov :

> I've created IEP-13 [1] to cover all cases.
> Feel free to create issues.
>
> [1]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75962769
>
> On Wed, Jan 24, 2018 at 6:10 PM, Vladimir Ozerov 
> wrote:
>
> > Let's start with a single and the most simple view, e.g.
> > LOCAL_TRANSACTIONS. We will review and merge it along with necessary
> > infrastructure. Then will handle the rest view in separate tickets and
> > separate focused discussions.
> >
> > On Wed, Jan 24, 2018 at 5:29 PM, Alex Plehanov 
> > wrote:
> >
> > > 1) It’s not a principal point, I can change schema. The
> > INFORMATION_SCHEMA
> > > was used because it’s already exists and usually used for metadata
> tables
> > > and views. Your proposal is to use schema “IGNITE”, am I understand you
> > > right? BTW, for now, we can’t query another (H2) meta tables from the
> > > INFORMATION_SCHEMA, so, “Ignite system views” is only available views
> to
> > > query from this schema.
> > > 2) Exactly for this reason the IGNITE_INSTANCE view is useful: to
> > determine
> > > which node we are connected to.
> > > 3) As the first phase, in my opinion, local views will be enough.
> > > Performance and caching of distributed views should be discussed at
> next
> > > phases, when distributed views implementation will be planned. In
> current
> > > implementation I tried to use indexing for local views wherever it’s
> > > possible.
> > > 4) I don’t think, that JVM info is more critical information than, for
> > > example, caches or nodes information. When authorization capabilities
> > > planned to implement?
> > >
> > > About local data: yes, we can rename all currently implemented views
> for
> > > the local node data as LOCAL_..., and create (someday) new whole
> cluster
> > > views (which use distributed requests) without prefix or, for example,
> > with
> > > CLUSTER_ prefix. But some views can show all cluster information using
> > only
> > > local node data, without distributed requests (for example
> > > IGNITE_NODE_METRICS, IGNITE_PART_ASSIGNMENT, IGNITE_PART_ALLOCATION,
> > > IGNITE_NODES, etc). Are they local or cluster views in this concept?
> > Which
> > > prefix should be used? And what about caches? Are they local or
> cluster?
> > On
> > > local node we can see cluster wide caches (replicated and distributed)
> > and
> > > caches for current node only. Local caches list may differ from node to
> > > node. Which prefix should be used for this view? And one more, there is
> > no
> > > sense for some views to make them cluster wide (for example
> > > INGNITE_INSTANCE). Should we name it LOCAL_INSTANCE without creating
> > > INSTANCE view?
> > >
> > > So, next steps: split PR, change schema name (IGNITE?), change view
> name
> > > for caches (CACHES, LOCAL_CACHES?)
> > >
> > >
> > > 2018-01-24 13:03 GMT+03:00 Vladimir Ozerov :
> > >
> > > > Hi Alex,
> > > >
> > > > System views could be extremely valuable addition for Ignite.
> Ideally,
> > > user
> > > > should be able to monitor and manage state of the whole cluster with
> a
> > > > single SQL command line. We have plans to implement it for a very
> long
> > > > time. However, this is very sensitive task which should take a lot of
> > > > moving pieces in count, such as usability, consistency, performance,
> > > > security, etc..
> > > >
> > > > Let me point several major concerns I see at the moment:
> > > >
> > > > 1) Usability: INFORMATION_SCHEMA
> > > > This schema is part of SQL ANSI standard. When creating system views,
> > > some
> > > > vendors prefer to store them in completely different predefined
> schema
> > > > (Oracle, MS SQL). Others prefer to keep them in INFORMATION_SCHEMA
> > > > directly. Both approaches could work. However, the latter breaks
> > > separation
> > > > of concerns - we store typical metadata near to possibly sensitive
> > system
> > > > data. Also it makes security management more complex - system data is
> > > very
> > > > sensitive, and now we 

[GitHub] ignite pull request #3434: Ignite-2.4-comp-check

[agoncharuk]

GitHub user agoncharuk opened a pull request:

https://github.com/apache/ignite/pull/3434

Ignite-2.4-comp-check



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.4-comp-check

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3434.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3434


commit 4925ffc10ce8e8287980eaec38b587512568a302
Author: Alexey Goncharuk 
Date:   2018-01-17T12:26:03Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in PageSnapshot

commit bcd68a058683b2f17b7ac60471b6e7aab3e4f6de
Author: Pavel Tupitsyn 
Date:   2018-01-17T12:38:20Z

IGNITE-7301 .NET: Baseline topology

This closes #3352

commit 66b96316a7775ce8a6e2ff4475185d5929e4998b
Author: devozerov 
Date:   2018-01-17T12:54:17Z

Merge branch 'master' into ignite-2.4

commit 268481c1cf7fe57df24be130eb67c3e3a13afe01
Author: Alexey Goncharuk 
Date:   2018-01-17T13:50:34Z

IGNITE-7453 Use GridUnsafe.cleanDirectBuffer in WalStat

commit db0cd105719c8ae713b13b34d9dca0a8cd45d377
Author: Pavel Tupitsyn 
Date:   2018-01-17T14:05:25Z

IGNITE-6776 .NET: Thin client: Add SQL & LINQ example

This closes #3390

commit c214db879101aa5660e2a50b11cd20964c0bc114
Author: Andrey Gura 
Date:   2018-01-17T12:42:41Z

ignite-7450 FileWriteAheadLogManager always uses RandomAccessFileIOFactory 
now

commit 75c27d5e49d7458e46eb46e6f87a445c3f1320ea
Author: Alexey Kuznetsov 
Date:   2018-01-18T02:25:19Z

IGNITE-7274 Web Console: Support multiple statements on Queries screen.
(cherry picked from commit 1926783)

commit 36cc822935387b2150e690c29bc6992dca0563f7
Author: Dmitriy Shabalin 
Date:   2018-01-18T04:49:08Z

IGNITE-7306 Web Console: Fixed export data from tables.
(cherry picked from commit 1bb60ec)

commit d753298b4012894b56f5c9218325211cd84a21d5
Author: Peter Ivanov 
Date:   2018-01-18T06:18:53Z

IGNITE-7107 Apache Ignite RPM packages

* added changelog to package specification

This closes #3396

commit 63445893f1bc75dc9777184499f7eabc1d4e51b1
Author: Denis Mekhanikov 
Date:   2018-01-18T08:36:18Z

IGNITE-3935 Use PeerDeployAware for streamer transformer - Fixes #3378.

Signed-off-by: Alexey Goncharuk 

commit f3f9f2a24b23027cf0c835140322e41a788932ae
Author: Pavel Tupitsyn 
Date:   2018-01-18T09:05:12Z

IGNITE-7413 Fix SqlDmlExample

This closes #3389

commit 1daa7c41bf1460a4d9a2b0c26a7a317f2fca3fb7
Author: Alexey Kuznetsov 
Date:   2018-01-18T10:14:53Z

IGNITE-7461 UI tools: Actualized data storage configuration.
(cherry picked from commit 577e632)

commit cf0080210d24d9dd8b057f959446fac5f8a4ca01
Author: dpavlov 
Date:   2018-01-18T10:53:29Z

IGNITE-7380 Implemented pluggable Direct IO - Fixes #3226.

Signed-off-by: Alexey Goncharuk 

commit dd06d0bd7ef266bfbe156e858b312d1ac86e8982
Author: Pavel Tupitsyn 
Date:   2018-01-18T12:55:49Z

IGNITE-7465 .NET: Fix SqlDdlExample failure with standalone node

commit 57479ec564e1761716da3d5f9feb7a64c396a9f9
Author: Pavel Tupitsyn 
Date:   2018-01-18T13:45:54Z

.NET: Fix CacheLocalTest.TestTxDeadlockDetection

commit bd6be8a4653322905a3b63850c7e033ce3801ce5
Author: Pavel Tupitsyn 
Date:   2018-01-18T18:25:05Z

.NET: Thin client: Fix OP_BINARY_TYPE_GET schema passing format

commit 97564d160586d6d57d300937e6b8877994e35fc7
Author: rkondakov 
Date:   2018-01-19T08:24:51Z

IGNITE-6456: Ability to separately enable or disable JDBC, ODBC and thin 
client endpoints. This closes #3309.

commit d50274ca8875c9680c12e8786ac355a787ba95e0
Author: Yakov Zhdanov 
Date:   2018-01-18T17:57:17Z

Javadoc enhancements - added @see

commit cb2d3cf22388ab19fb2d34ae5bdfc8f1b608db75
Author: Dmitriy Govorukhin 
Date:   2018-01-18T14:14:26Z

IGNITE-7471 Use soft reference for checkpoint entry contents to avoid 
excessive memory usage

commit 3965923369870bb4e8e57e3332c1a1eb1e5f5ed3
Author: rkondakov 
Date:   2018-01-19T09:00:55Z

IGNITE-6772: SQL exception messages became more informative. This closes 
#3342.

commit ba68cb0fa87f776fcd0499d030c333f182611f41
Author: devozerov 
Date:   2018-01-19T09:03:52Z

Merge remote-tracking branch 'origin/ignite-2.4' into ignite-2.4

commit b54c0c8786bd74aa0abb208f537c29f0c4be4b1e
Author: tledkov-gridgain 
Date:   2018-01-19T09:09:34Z

IGNITE-7248: JDBC: fixed schema resolution for streaming mode. 

How to delete all data from SQL table

[Nikolay Izhikov]

Hello, guys.

I wonder what is best way to delete all rows from table?

Other RDBMS offer command like `TRUNCATE TABLE` or similar.

Is it would be faster if user execute `DROP TABLE XXX`, `CREATE TABLE XXX` 
instead of `DELETE FROM XXX`?

[jira] [Created] (IGNITE-7531) SQL: Create data load benchmarks

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7531:
---

 Summary: SQL: Create data load benchmarks
 Key: IGNITE-7531
 URL: https://issues.apache.org/jira/browse/IGNITE-7531
 Project: Ignite
  Issue Type: Task
  Components: sql, yardstick
Reporter: Vladimir Ozerov
Assignee: Pavel Kuznetsov
 Fix For: 2.5


We need to implement a set of data loading benchmarks to better understand how 
fast Ignite is able to consume data. This task consists of two steps:
1) Extend Yardstick capabilities
2) Create set of benchmarks

1) Yardstick
Data load benchmark should be executed in single-shot mode: only one iteration, 
only total execution time is needed, start callback for setup and warmup, stop 
callback for cleanup. 
Currently Yardstick cannot do that, so we need to extend it. Possibly, we can 
control this through new {{boolean BenchmarkDriver.isSingleShot()}} method.

2) Benchmarks 
At first let's focus on thin JDBC driver. The following cases should be 
executed:
2.1) Normal INSERT
2.2) Batched INSERT
2.3) Streaming INSERT (when IGNITE-7253 is ready)
2.4) P. 1-3 with and without dynamically disabled WAL (ALTER TABLE ... 
NOLOGGING)
2.5) P. 1-3 with additional indexes - either created before data load on empty 
table, or after load on table with data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3433: IGNITE-7527 SQL system views: local transactions

[alex-plekhanov]

GitHub user alex-plekhanov opened a pull request:

https://github.com/apache/ignite/pull/3433

IGNITE-7527 SQL system views: local transactions



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alex-plekhanov/ignite ignite-7527

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3433.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3433


commit eb4e5e52c1cb30351d89414dd7ad3bb235b78683
Author: Aleksey Plekhanov 
Date:   2018-01-19T23:45:01Z

IGNITE-7527 SQL system views: local transactions




---

[jira] [Created] (IGNITE-7530) .NET: Poor performance & excessive memory usage in GetAll and query cursors in binary mode

[Pavel Tupitsyn (JIRA)]

Pavel Tupitsyn created IGNITE-7530:
--

 Summary: .NET: Poor performance & excessive memory usage in GetAll 
and query cursors in binary mode
 Key: IGNITE-7530
 URL: https://issues.apache.org/jira/browse/IGNITE-7530
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 2.4


{{GetAll}} and query cursors do not use {{BinaryReader.DetachNext}}. So in 
binary mode for each binary object in a stream we copy entire stream content, 
see {{BinaryReader.ReadAsBinary}}, which calls {{Stream.GetArray()}}, which 
causes copying in {{PlatformMemoryStream}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7529) Web Console: Refactor column filters in grid

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-7529:


 Summary: Web Console: Refactor column filters in grid
 Key: IGNITE-7529
 URL: https://issues.apache.org/jira/browse/IGNITE-7529
 Project: Ignite
  Issue Type: Bug
  Components: wizards
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 2.5


Current implementation is not extendable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Transport compression (not store compression)

[gvvinblade]

Nikita,

I took a look at the changes and see two issues.

The first one is additional dependencies in core module. That means you
should whether make the filter plugable and provide some additional module
for it or put all the necessary classes to ignite-core.

The second is possible bottleneck in NIO threads. Since each worker serves
several connections we may face troubles on big clusters (for example one or
two ignite server nodes and a few dozens client nodes) because
compression/decompression is an expensive operation and happens (in your
implementation) in NIO threads where message processing becames sequential.

The provided benchmarks are quite synthetic (they are local, they involve
only two nodes, they don't use "real" operations like put, get, putAll etc
and don't show a real picture)

We have to be careful with such changes and check all possible scenarios.



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Java 9 support

[Pavel Tupitsyn]

I would add that to run Ignite on Java 9 without default scripts one has to
use the following JVM options:

--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED"
--add-exports=java.base/sun.nio.ch=ALL-UNNAMED

--add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED

--add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED

Ignite.NET adds these options automatically when Java 9 is detected, no
user steps required.

Thanks,
Pavel


On Thu, Jan 25, 2018 at 12:53 PM, vveider  wrote:

> Hi, Igniters!
>
>
> Ticket IGNITE-6730 [1] was merged to master (and ignite-2.4) and now we
> have
> preliminary support of Java 9, which includes:
>  - compilation with JDK9 with some constraints (scala-2.10 based modules
> should be turned off)
>  - run with JRE9/JDK9 through default scripts (ignite.{sh|bat})
>
> Also, temporary TC project for Ignite Tests on Java 9 [2] was prepared;
> analysis of test problems and corresponding fixes is in progress.
>
> Please, be advised.
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-6730
> [2] https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java9
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: Java 9 support

[vveider]

Hi, Igniters!


Ticket IGNITE-6730 [1] was merged to master (and ignite-2.4) and now we have
preliminary support of Java 9, which includes:
 - compilation with JDK9 with some constraints (scala-2.10 based modules
should be turned off)
 - run with JRE9/JDK9 through default scripts (ignite.{sh|bat})

Also, temporary TC project for Ignite Tests on Java 9 [2] was prepared;
analysis of test problems and corresponding fixes is in progress.

Please, be advised.


[1] https://issues.apache.org/jira/browse/IGNITE-6730
[2] https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java9



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[GitHub] ignite pull request #3399: IGNITE-7316: Make Linear SVM for binary classific...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3399


---

Re: Weird FillFactor metric fluctuation.

[Anton Vinogradov]

Denis,

Metrics listed at https://issues.apache.org/jira/browse/IGNITE-7491 are not
relevant to FillFactor.

On Thu, Jan 25, 2018 at 3:17 AM, Denis Magda  wrote:

> There will be a way to get used and free size in bytes in Ignite 2.4:
> https://issues.apache.org/jira/browse/IGNITE-7491
>
> *Anton*, did we use fill factor parameter to calculate the used space for
> the new metrics? If it’s so then the issue reported by Andrey needs to be
> investigated and solved before 2.4 release.
>
> —
> Denis
>
>
> On Jan 24, 2018, at 7:50 AM, Andrey Kuznetsov  wrote:
>
> FillFactor metric is in fact FreeList utilization factor, so the name is
> definitely misleading. Currently we have no metric to estimate free space.
> And we should define what is 'free' in terms of Ignite page memory
> implementation.
>
> 2018-01-24 17:57 GMT+03:00 Andrey Mashenkov :
>
> Hi Ignites,
>
> I've tried to estimate in runtime how much free memory there are on a node.
> Looking at memory metrics API, I need a pageSize, number of allocated pages
> and a fillFactor of these pages.
>
> But it looks like FillFactor metric is broken or useless.
>
> A simple test where new entries are put to cache shows next:
> - number of allocated page is constantly grows which looks ok.
> - FillFactor metric has weird fluctuation. It seems ok if its value will
> slightly decreased when new pages allocated,
> but I observe 2-10+ times difference between checks.
> Also calculated total used memory can also significanlty decreased.
>
>
> Here is a ticket for this issue [1].
> Can someone clarify, is it expected behavior? what is FillFactor metric
> means?
> how this can be fixed or how used memory should be calculated if metrics
> itself is ok?
>
> Thoughts?
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7489
>
> --
> Best regards,
> Andrey V. Mashenkov
>
>
>
>
> --
> Best regards,
>  Andrey Kuznetsov.
>
>
>

[jira] [Created] (IGNITE-7527) SQL system view for current node transactions

[Aleksey Plekhanov (JIRA)]

Aleksey Plekhanov created IGNITE-7527:
-

 Summary: SQL system view for current node transactions
 Key: IGNITE-7527
 URL: https://issues.apache.org/jira/browse/IGNITE-7527
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Aleksey Plekhanov
Assignee: Aleksey Plekhanov


Implement SQL system view to show active transactions on local node.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7525) ODBC: Add a note about connection non-thread safety to docs

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7525:
---

 Summary: ODBC: Add a note about connection non-thread safety to 
docs
 Key: IGNITE-7525
 URL: https://issues.apache.org/jira/browse/IGNITE-7525
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Vladimir Ozerov
Assignee: Igor Sapego
 Fix For: 2.5


Our ODBC connection is not thread safe. Let's mention it in docs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7524) JDBC: Add a note about connection non-thread safety to docs

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7524:
---

 Summary: JDBC: Add a note about connection non-thread safety to 
docs
 Key: IGNITE-7524
 URL: https://issues.apache.org/jira/browse/IGNITE-7524
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Vladimir Ozerov
Assignee: Taras Ledkov
 Fix For: 2.5


Our JDBC connection is not thread safe. Let's mention it in docs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)