[jira] [Commented] (IGNITE-13845) Add checkpoint lock hold metrics
[ https://issues.apache.org/jira/browse/IGNITE-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249519#comment-17249519 ] Ignite TC Bot commented on IGNITE-13845: {panel:title=Branch: [pull/8572/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/8572/head] Base: [master] : New Tests (1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1} {color:#8b}PDS 2{color} [[tests 1|https://ci.ignite.apache.org/viewLog.html?buildId=5790030]] * {color:#013220}IgnitePdsTestSuite2: IgniteDataStorageMetricsSelfTest.testCheckpointMetrics - PASSED{color} {panel} [TeamCity *-- Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=5790058buildTypeId=IgniteTests24Java8_RunAll] > Add checkpoint lock hold metrics > > > Key: IGNITE-13845 > URL: https://issues.apache.org/jira/browse/IGNITE-13845 > Project: Ignite > Issue Type: New Feature >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Fix For: 2.10 > > Time Spent: 10m > Remaining Estimate: 0h > > Add last checkpoint lock hold duration and durations histogram metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13854) Add documentation for the cluster performance profiling tool
Amelchev Nikita created IGNITE-13854: Summary: Add documentation for the cluster performance profiling tool Key: IGNITE-13854 URL: https://issues.apache.org/jira/browse/IGNITE-13854 Project: Ignite Issue Type: Sub-task Reporter: Amelchev Nikita Assignee: Amelchev Nikita Add documentation for the cluster performance profiling tool. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (IGNITE-13451) Provide the ability to set up default performance statistics properties
[ https://issues.apache.org/jira/browse/IGNITE-13451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amelchev Nikita resolved IGNITE-13451. -- Resolution: Done > Provide the ability to set up default performance statistics properties > --- > > Key: IGNITE-13451 > URL: https://issues.apache.org/jira/browse/IGNITE-13451 > Project: Ignite > Issue Type: Sub-task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > > Provide the ability to set up default performance statistics properties: > - DFLT_FILE_MAX_SIZE > - DFLT_BUFFER_SIZE > - DFLT_FLUSH_SIZE > - DFLT_MAX_CACHED_STRINGS_COUNT -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (IGNITE-13252) Provide the ability to cache strings
[ https://issues.apache.org/jira/browse/IGNITE-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amelchev Nikita resolved IGNITE-13252. -- Resolution: Done > Provide the ability to cache strings > > > Key: IGNITE-13252 > URL: https://issues.apache.org/jira/browse/IGNITE-13252 > Project: Ignite > Issue Type: Sub-task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > > Strings that performance statistics writes can be cached with id. It will > reduce affect on performance. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (IGNITE-13452) Create a new performance statistics file instead of deleting the existing
[ https://issues.apache.org/jira/browse/IGNITE-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amelchev Nikita resolved IGNITE-13452. -- Resolution: Done > Create a new performance statistics file instead of deleting the existing > - > > Key: IGNITE-13452 > URL: https://issues.apache.org/jira/browse/IGNITE-13452 > Project: Ignite > Issue Type: Sub-task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > > Create a new performance statistics file instead of deleting the existing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13847) Make GridEncryptionManager#onWalSegmentRemoved async
[ https://issues.apache.org/jira/browse/IGNITE-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249490#comment-17249490 ] Kirill Tkalenko commented on IGNITE-13847: -- Used the [PR's|https://github.com/xtern/ignite/pull/44/files] from the IGNITE-13831 from [~xtern]. > Make GridEncryptionManager#onWalSegmentRemoved async > > > Key: IGNITE-13847 > URL: https://issues.apache.org/jira/browse/IGNITE-13847 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: IEP-18 > Fix For: 2.10 > > > When implementing IGNITE-13831 I was faced with deadlock. > When execute *FileWriteAheadLogManager#rollOver*, begin to clean WAL archive > since we have reached the *DataStorageConfiguration#maxWalArchiveSize*, after > deleting a segment, execute the *GridEncryptionManager#onWalSegmentRemoved* > that wants to write to the metastore, but it will not succeed, since it will > wait for *FileWriteAheadLogManager#rollOver*. > I suggest making the *GridEncryptionManager#onWalSegmentRemoved* asynchronous > in a separate pool, for example, as a *CacheGroupPageScanner#singleExecSvc*. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-13847) Make GridEncryptionManager#onWalSegmentRemoved async
[ https://issues.apache.org/jira/browse/IGNITE-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko reassigned IGNITE-13847: Assignee: Kirill Tkalenko > Make GridEncryptionManager#onWalSegmentRemoved async > > > Key: IGNITE-13847 > URL: https://issues.apache.org/jira/browse/IGNITE-13847 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: IEP-18 > Fix For: 2.10 > > > When implementing IGNITE-13831 I was faced with deadlock. > When execute *FileWriteAheadLogManager#rollOver*, begin to clean WAL archive > since we have reached the *DataStorageConfiguration#maxWalArchiveSize*, after > deleting a segment, execute the *GridEncryptionManager#onWalSegmentRemoved* > that wants to write to the metastore, but it will not succeed, since it will > wait for *FileWriteAheadLogManager#rollOver*. > I suggest making the *GridEncryptionManager#onWalSegmentRemoved* asynchronous > in a separate pool, for example, as a *CacheGroupPageScanner#singleExecSvc*. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13853) Add APIs, examples, language selector to the docs menu
[ https://issues.apache.org/jira/browse/IGNITE-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis A. Magda updated IGNITE-13853: Description: There is user feedback that suggests we need to rework our navigation menus of the website and docs to improve the discoverability of APIs and examples. Let's do the following. *Ignite Website Top-Level Navigation Menu ([https://ignite.apache.org)|https://ignite.apache.org)/]* * Add the "Docs" menu item before the "Features" item. The "Docs" _is a single link_ (not a dropdown menu) that directs to the root of the latest technical documentation: [https://ignite.apache.org/docs/latest/] * Rename "Resources" to "Learn More". Remove "Documentation", "Documentation (Chinese)" and "APIs" from this menu section. * Remove "Events" from the top-level menu. Add the items/links from the menu to the "Community" *Technical Documentation Top-Level Navigation Menu ([https://ignite.apache.org/docs/latest/])* * Remove the Github icon from the menu * Add "APIs" after the "version" selector. The menu lists "Java", "C#/.NET", "C++", "Scala". The "version" selector defines what version of an API is to be opened. For instance, if a user chooses version "2.9.1" (to be released soon) then "Java" of that version is to be opened. If instead, the user wants to see the docs and APIs for "2.9.0", he just needs to selects that version from the version menu. * Add "Examples" after the "APIs". Add the following menu items: ** "Java": [https://github.com/apache/ignite/tree/master/examples] ** "C#/.NET": [https://github.com/apache/ignite/tree/master/modules/platforms/dotnet/examples] ** "C++": [https://github.com/apache/ignite/tree/master/modules/platforms/cpp/examples] ** "Python": [https://github.com/apache/ignite/tree/master/modules/platforms/python/examples] ** "Node.JS": [https://github.com/apache/ignite/tree/master/modules/platforms/nodejs/examples] ** "PHP": [https://github.com/apache/ignite/tree/master/modules/platforms/php/examples] * Add a language selector drop-down list after the "search" text box. See how it's implemented here: [https://www.confluent.io.|https://www.confluent.io./] We'll have just two menu items there: ** "English": points out to the root of the main docs [https://ignite.apache.org/docs/latest/] ** "Chinese": [https://www.ignite-service.cn/doc/java/] was: There is user feedback that suggests we need to rework our navigation menus of the website and docs to improve the discoverability of APIs and examples. Let's do the following. *Ignite Website Top-Level Navigation Menu ([https://ignite.apache.org)|https://ignite.apache.org)/]* * Add the "Docs" menu item before the "Features" item. The "Docs" _is a single link_ (not a dropdown menu) that directs to the root of the latest technical documentation: [https://ignite.apache.org/docs/latest/] * Rename "Resources" to "Learn More". Remove "Documentation", "Documentation (Chinese)" and "APIs" from this menu section. * Remove "Events" from the top-level menu. Add the items/links from the menu to the "Community" *Technical Documentation Top-Level Navigation Menu ([https://ignite.apache.org/docs/latest/])* * Remove the Github icon from the menu * Add "APIs" after the "version" selector. The menu lists "JavaDocs", "C#/.NET", "C++", "Scala". The "version" selector defines what version of an API is to be opened. For instance, if a user chooses version "2.9.1" (to be released soon) then "JavaDocs" of that version is to be opened. If instead, the user wants to see the docs and APIs for "2.9.0", he just needs to selects that version from the version menu. * Add "Examples" after the "APIs". Add the following menu items: ** "Java": [https://github.com/apache/ignite/tree/master/examples] ** "C#/.NET": [https://github.com/apache/ignite/tree/master/modules/platforms/dotnet/examples] ** "C++": [https://github.com/apache/ignite/tree/master/modules/platforms/cpp/examples] ** "Python": [https://github.com/apache/ignite/tree/master/modules/platforms/python/examples] ** "Node.JS": [https://github.com/apache/ignite/tree/master/modules/platforms/nodejs/examples] ** "PHP": [https://github.com/apache/ignite/tree/master/modules/platforms/php/examples] * Add a language selector drop-down list after the "search" text box. See how it's implemented here: [https://www.confluent.io.|https://www.confluent.io./] We'll have just two menu items there: ** "English": points out to the root of the main docs [https://ignite.apache.org/docs/latest/] ** "Chinese": [https://www.ignite-service.cn/doc/java/] > Add APIs, examples, language selector to the docs menu > -- > > Key: IGNITE-13853 > URL: https://issues.apache.org/jira/browse/IGNITE-13853 > Project: Ignite > Issue Type: Improvement >
[jira] [Updated] (IGNITE-13853) Add APIs, examples, language selector to the docs menu
[ https://issues.apache.org/jira/browse/IGNITE-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis A. Magda updated IGNITE-13853: Description: There is user feedback that suggests we need to rework our navigation menus of the website and docs to improve the discoverability of APIs and examples. Let's do the following. *Ignite Website Top-Level Navigation Menu ([https://ignite.apache.org)|https://ignite.apache.org)/]* * Add the "Docs" menu item before the "Features" item. The "Docs" _is a single link_ (not a dropdown menu) that directs to the root of the latest technical documentation: [https://ignite.apache.org/docs/latest/] * Rename "Resources" to "Learn More". Remove "Documentation", "Documentation (Chinese)" and "APIs" from this menu section. * Remove "Events" from the top-level menu. Add the items/links from the menu to the "Community" *Technical Documentation Top-Level Navigation Menu ([https://ignite.apache.org/docs/latest/])* * Remove the Github icon from the menu * Add "APIs" after the "version" selector. The menu lists "JavaDocs", "C#/.NET", "C++", "Scala". The "version" selector defines what version of an API is to be opened. For instance, if a user chooses version "2.9.1" (to be released soon) then "JavaDocs" of that version is to be opened. If instead, the user wants to see the docs and APIs for "2.9.0", he just needs to selects that version from the version menu. * Add "Examples" after the "APIs". Add the following menu items: ** "Java": [https://github.com/apache/ignite/tree/master/examples] ** "C#/.NET": [https://github.com/apache/ignite/tree/master/modules/platforms/dotnet/examples] ** "C++": [https://github.com/apache/ignite/tree/master/modules/platforms/cpp/examples] ** "Python": [https://github.com/apache/ignite/tree/master/modules/platforms/python/examples] ** "Node.JS": [https://github.com/apache/ignite/tree/master/modules/platforms/nodejs/examples] ** "PHP": [https://github.com/apache/ignite/tree/master/modules/platforms/php/examples] * Add a language selector drop-down list after the "search" text box. See how it's implemented here: [https://www.confluent.io.|https://www.confluent.io./] We'll have just two menu items there: ** "English": points out to the root of the main docs [https://ignite.apache.org/docs/latest/] ** "Chinese": [https://www.ignite-service.cn/doc/java/] was: There is user feedback that suggests we need to rework our navigation menus of the website and docs to improve the discoverability of APIs and examples. Let's do the following. *Ignite Website Top-Level Navigation Menu ([https://ignite.apache.org)|https://ignite.apache.org)/]* * Add the "Docs" menu item to the menu placing it before the "Features" item. The "Docs" _is a single link_ (no dropdown menu) that directs to the root of the latest technical documentation: [https://ignite.apache.org/docs/latest/] * Rename "Resources" to "Learn More". Remove "Documentation", "Documentation (Chinese)" and "APIs" from this menu section. * Remove "Events" from the top-level menu. Add the items/links from the menu to the "Community" *Technical Documentation Top-Level Navigation Menu ([https://ignite.apache.org/docs/latest/])* * Remove the Github icon from the menu * Add "APIs" after the "version" selector. The menu lists "JavaDocs", "C#/.NET", "C++", "Scala". The "version" selector defines what version of an API is to be opened. For instance, if a user chooses version "2.9.1" (to be released soon) then "JavaDocs" of that version is to be opened. If instead, the user wants to see the docs and APIs for "2.9.0", he just needs to selects that version from the version menu. * Add "Examples" after the "APIs". Add the following menu items: ** "Java": [https://github.com/apache/ignite/tree/master/examples] ** "C#/.NET": [https://github.com/apache/ignite/tree/master/modules/platforms/dotnet/examples] ** "C++": [https://github.com/apache/ignite/tree/master/modules/platforms/cpp/examples] ** "Python": [https://github.com/apache/ignite/tree/master/modules/platforms/python/examples] ** "Node.JS": [https://github.com/apache/ignite/tree/master/modules/platforms/nodejs/examples] ** "PHP": [https://github.com/apache/ignite/tree/master/modules/platforms/php/examples] * Add a language selector drop-down list after the "search" text box. See how it's implemented here: [https://www.confluent.io.|https://www.confluent.io./] We'll have just two menu items there: ** "English": points out to the root of the main docs [https://ignite.apache.org/docs/latest/] ** "Chinese": https://www.ignite-service.cn/doc/java/ > Add APIs, examples, language selector to the docs menu > -- > > Key: IGNITE-13853 > URL: https://issues.apache.org/jira/browse/IGNITE-13853 > Project: Ignite >
[jira] [Created] (IGNITE-13853) Add APIs, examples, language selector to the docs menu
Denis A. Magda created IGNITE-13853: --- Summary: Add APIs, examples, language selector to the docs menu Key: IGNITE-13853 URL: https://issues.apache.org/jira/browse/IGNITE-13853 Project: Ignite Issue Type: Improvement Components: documentation, website Reporter: Denis A. Magda Fix For: 2.9.1 There is user feedback that suggests we need to rework our navigation menus of the website and docs to improve the discoverability of APIs and examples. Let's do the following. *Ignite Website Top-Level Navigation Menu ([https://ignite.apache.org)|https://ignite.apache.org)/]* * Add the "Docs" menu item to the menu placing it before the "Features" item. The "Docs" _is a single link_ (no dropdown menu) that directs to the root of the latest technical documentation: [https://ignite.apache.org/docs/latest/] * Rename "Resources" to "Learn More". Remove "Documentation", "Documentation (Chinese)" and "APIs" from this menu section. * Remove "Events" from the top-level menu. Add the items/links from the menu to the "Community" *Technical Documentation Top-Level Navigation Menu ([https://ignite.apache.org/docs/latest/])* * Remove the Github icon from the menu * Add "APIs" after the "version" selector. The menu lists "JavaDocs", "C#/.NET", "C++", "Scala". The "version" selector defines what version of an API is to be opened. For instance, if a user chooses version "2.9.1" (to be released soon) then "JavaDocs" of that version is to be opened. If instead, the user wants to see the docs and APIs for "2.9.0", he just needs to selects that version from the version menu. * Add "Examples" after the "APIs". Add the following menu items: ** "Java": [https://github.com/apache/ignite/tree/master/examples] ** "C#/.NET": [https://github.com/apache/ignite/tree/master/modules/platforms/dotnet/examples] ** "C++": [https://github.com/apache/ignite/tree/master/modules/platforms/cpp/examples] ** "Python": [https://github.com/apache/ignite/tree/master/modules/platforms/python/examples] ** "Node.JS": [https://github.com/apache/ignite/tree/master/modules/platforms/nodejs/examples] ** "PHP": [https://github.com/apache/ignite/tree/master/modules/platforms/php/examples] * Add a language selector drop-down list after the "search" text box. See how it's implemented here: [https://www.confluent.io.|https://www.confluent.io./] We'll have just two menu items there: ** "English": points out to the root of the main docs [https://ignite.apache.org/docs/latest/] ** "Chinese": https://www.ignite-service.cn/doc/java/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13853) Add APIs, examples, language selector to the docs menu
[ https://issues.apache.org/jira/browse/IGNITE-13853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249466#comment-17249466 ] Denis A. Magda commented on IGNITE-13853: - [~mstekl], could you please help with this task? > Add APIs, examples, language selector to the docs menu > -- > > Key: IGNITE-13853 > URL: https://issues.apache.org/jira/browse/IGNITE-13853 > Project: Ignite > Issue Type: Improvement > Components: documentation, website >Reporter: Denis A. Magda >Priority: Major > Fix For: 2.9.1 > > > There is user feedback that suggests we need to rework our navigation menus > of the website and docs to improve the discoverability of APIs and examples. > Let's do the following. > > *Ignite Website Top-Level Navigation Menu > ([https://ignite.apache.org)|https://ignite.apache.org)/]* > * Add the "Docs" menu item to the menu placing it before the "Features" > item. The "Docs" _is a single link_ (no dropdown menu) that directs to the > root of the latest technical documentation: > [https://ignite.apache.org/docs/latest/] > * Rename "Resources" to "Learn More". Remove "Documentation", "Documentation > (Chinese)" and "APIs" from this menu section. > * Remove "Events" from the top-level menu. Add the items/links from the menu > to the "Community" > > *Technical Documentation Top-Level Navigation Menu > ([https://ignite.apache.org/docs/latest/])* > * Remove the Github icon from the menu > * Add "APIs" after the "version" selector. The menu lists "JavaDocs", > "C#/.NET", "C++", "Scala". The "version" selector defines what version of an > API is to be opened. For instance, if a user chooses version "2.9.1" (to be > released soon) then "JavaDocs" of that version is to be opened. If instead, > the user wants to see the docs and APIs for "2.9.0", he just needs to selects > that version from the version menu. > * Add "Examples" after the "APIs". Add the following menu items: > ** "Java": [https://github.com/apache/ignite/tree/master/examples] > ** "C#/.NET": > [https://github.com/apache/ignite/tree/master/modules/platforms/dotnet/examples] > ** "C++": > [https://github.com/apache/ignite/tree/master/modules/platforms/cpp/examples] > ** "Python": > [https://github.com/apache/ignite/tree/master/modules/platforms/python/examples] > ** "Node.JS": > [https://github.com/apache/ignite/tree/master/modules/platforms/nodejs/examples] > ** "PHP": > [https://github.com/apache/ignite/tree/master/modules/platforms/php/examples] > * Add a language selector drop-down list after the "search" text box. See > how it's implemented here: > [https://www.confluent.io.|https://www.confluent.io./] We'll have just two > menu items there: > ** "English": points out to the root of the main docs > [https://ignite.apache.org/docs/latest/] > ** "Chinese": https://www.ignite-service.cn/doc/java/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13852) Ascii tables for new unified cli tool
Kirill Gusakov created IGNITE-13852: --- Summary: Ascii tables for new unified cli tool Key: IGNITE-13852 URL: https://issues.apache.org/jira/browse/IGNITE-13852 Project: Ignite Issue Type: Task Reporter: Kirill Gusakov Implement Ascii tables with support of Ansi symbols and custom borders -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12824) Interoperable Ignite.NET Dates
[ https://issues.apache.org/jira/browse/IGNITE-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249138#comment-17249138 ] Nikolay Izhikov edited comment on IGNITE-12824 at 12/14/20, 5:18 PM: - Hello, [~kukushal], [~ptupitsyn] I investigated this issue and find a bug, but not one described :) 1. For now, We can't use {{DateTime}} type as a key or value for a cache. I fixed it in the [PR #8568|https://github.com/apache/ignite/pull/8568]. 2. I just delete explicit exception for {{DateTimeKind.Local}} dates and found no issue. Please, [take a look at my test|https://github.com/apache/ignite/pull/8568/files#diff-978bbc38ff93b1f1a62517f532de7bf4d0a320de32550515a062abb830c58164R959] Can you, please, clarify the circumstances to reproduce the issue from the description. For now, it looks like we just can remove a restriction for non UTC dates and that's all :) was (Author: nizhikov): Hello, [~kukushal], [~ptupitsyn] I investigated this issue and find a bug, but not one described in the description :) 1. For now, We can't use {{DateTime}} type as a key or value for a cache. I fixed it in the [PR #8568|https://github.com/apache/ignite/pull/8568]. 2. I just delete explicit exception for {{DateTimeKind.Local}} dates and found no issue. Please, [take a look at my test|https://github.com/apache/ignite/pull/8568/files#diff-978bbc38ff93b1f1a62517f532de7bf4d0a320de32550515a062abb830c58164R959] Can you, please, clarify the circumstances to reproduce the issue from the description. For now, it looks like we just can remove a restriction for non UTC dates and that's all :) > Interoperable Ignite.NET Dates > -- > > Key: IGNITE-12824 > URL: https://issues.apache.org/jira/browse/IGNITE-12824 > Project: Ignite > Issue Type: Improvement > Components: platforms >Affects Versions: 2.8 >Reporter: Alexey Kukushkin >Assignee: Nikolay Izhikov >Priority: Major > Labels: .NET, iep-54, ignite-3, sbcf > Fix For: 3.0 > > Attachments: IGNITE-12824-from-2.9.0.patch > > Time Spent: 10m > Remaining Estimate: 0h > > *+The Problem+* > Presently .NET API writes dates as composite Ignite objects. Only .NET > clients can read such dates: any other client (JDBC, Java, etc) does not > understand it without custom deserialization. > > It is still possible to configure .NET serialization to write dates as > Ignite dates - see [DateTime Serialization > note|https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility]. > But then Ignite accepts only UTC dates, requiring the application developers > to convert local dates to UTC dates and back. This task is not trivial: > [DateTime.ToUniversalTime|https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1] > uses calendars different from Java (and the .NET calendars are the invalid > ones - especially for pre-1990 dates). > > The motivation for the current default behavior was probably the desire to > keep the Time Zone information: Ignite dates do not store time zones. > > In our experience interoperability is more important than storing time zone > info. > > *+The Solution+* > # Always write .NET dates as portable Ignite dates: get rid of the > {{BinaryReflectiveSerializer.ForceTimestamp}} flag that currently triggers > this behavior. > Keep the {{ForceTimestamp}} flag if saving .NET dates as transparent objects > seems a useful case. > # Automatically convert Local dates to UTC and back *inside* Ignite.NET. > Add a {{BinaryReflectiveSerializer.UtcDate flag}} to disable this > conversion. This prevents loosing the {{DateTime.Kind}} property of UTC > dates. {{UtcDate}} set to true implies the existing behavior: Ignite gets > UTC dates and throws a "Date must be in UTC" exception on an attempt to write > a Local date. The {{UtcDate}} flag is false by default. > # Use [NodaTime|https://nodatime.org/] for UTC<->Local conversions. Noda > time uses Java calendars making the conversion truly portable. > > *+The Benefits+* > # Portable dates is a more frequent use-case than storing time zone info for > every date in Ignite. This becomes default behavior and the developers do not > need to always explicitly configure it. > # Non-trivial code to make the truly portable UTC<->Local conversion is > implemented once inside Ignite instead of having every Ignite.NET application > implementing it. > +*References*+ > * [Dev-List > Discussion|http://apache-ignite-developers.2346864.n4.nabble.com/Interoperable-Ignite-NET-Dates-td49946.html] > * IEP-TBD -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12824) Interoperable Ignite.NET Dates
[ https://issues.apache.org/jira/browse/IGNITE-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249138#comment-17249138 ] Nikolay Izhikov commented on IGNITE-12824: -- Hello, [~kukushal], [~ptupitsyn] I investigated this issue and find a bug, but not one described in the description :) 1. For now, We can't use {{DateTime}} type as a key or value for a cache. I fixed it in the [PR #8568|https://github.com/apache/ignite/pull/8568]. 2. I just delete explicit exception for {{DateTimeKind.Local}} dates and found no issue. Please, [take a look at my test|https://github.com/apache/ignite/pull/8568/files#diff-978bbc38ff93b1f1a62517f532de7bf4d0a320de32550515a062abb830c58164R959] Can you, please, clarify the circumstances to reproduce the issue from the description. For now, it looks like we just can remove a restriction for non UTC dates and that's all :) > Interoperable Ignite.NET Dates > -- > > Key: IGNITE-12824 > URL: https://issues.apache.org/jira/browse/IGNITE-12824 > Project: Ignite > Issue Type: Improvement > Components: platforms >Affects Versions: 2.8 >Reporter: Alexey Kukushkin >Assignee: Nikolay Izhikov >Priority: Major > Labels: .NET, iep-54, ignite-3, sbcf > Fix For: 3.0 > > Attachments: IGNITE-12824-from-2.9.0.patch > > Time Spent: 10m > Remaining Estimate: 0h > > *+The Problem+* > Presently .NET API writes dates as composite Ignite objects. Only .NET > clients can read such dates: any other client (JDBC, Java, etc) does not > understand it without custom deserialization. > > It is still possible to configure .NET serialization to write dates as > Ignite dates - see [DateTime Serialization > note|https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility]. > But then Ignite accepts only UTC dates, requiring the application developers > to convert local dates to UTC dates and back. This task is not trivial: > [DateTime.ToUniversalTime|https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1] > uses calendars different from Java (and the .NET calendars are the invalid > ones - especially for pre-1990 dates). > > The motivation for the current default behavior was probably the desire to > keep the Time Zone information: Ignite dates do not store time zones. > > In our experience interoperability is more important than storing time zone > info. > > *+The Solution+* > # Always write .NET dates as portable Ignite dates: get rid of the > {{BinaryReflectiveSerializer.ForceTimestamp}} flag that currently triggers > this behavior. > Keep the {{ForceTimestamp}} flag if saving .NET dates as transparent objects > seems a useful case. > # Automatically convert Local dates to UTC and back *inside* Ignite.NET. > Add a {{BinaryReflectiveSerializer.UtcDate flag}} to disable this > conversion. This prevents loosing the {{DateTime.Kind}} property of UTC > dates. {{UtcDate}} set to true implies the existing behavior: Ignite gets > UTC dates and throws a "Date must be in UTC" exception on an attempt to write > a Local date. The {{UtcDate}} flag is false by default. > # Use [NodaTime|https://nodatime.org/] for UTC<->Local conversions. Noda > time uses Java calendars making the conversion truly portable. > > *+The Benefits+* > # Portable dates is a more frequent use-case than storing time zone info for > every date in Ignite. This becomes default behavior and the developers do not > need to always explicitly configure it. > # Non-trivial code to make the truly portable UTC<->Local conversion is > implemented once inside Ignite instead of having every Ignite.NET application > implementing it. > +*References*+ > * [Dev-List > Discussion|http://apache-ignite-developers.2346864.n4.nabble.com/Interoperable-Ignite-NET-Dates-td49946.html] > * IEP-TBD -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12892) Clarify WAL archive size configuration
[ https://issues.apache.org/jira/browse/IGNITE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249085#comment-17249085 ] shivakumar commented on IGNITE-12892: - Hi [~YAMolochkov] Could you please consider this for 2.9.1 release ? It seems this code change (also IGNITE-13373 but not sure which one exactly fixed the issue) solves major blocker issue described here [http://apache-ignite-users.70518.x6.nabble.com/Could-not-clear-historyMap-due-to-WAL-reservation-on-cp-td34779.html] [http://apache-ignite-users.70518.x6.nabble.com/connecting-to-visor-shell-permanently-stops-unwanted-WAL-clean-up-td34737.html] Regards Shiva > Clarify WAL archive size configuration > -- > > Key: IGNITE-12892 > URL: https://issues.apache.org/jira/browse/IGNITE-12892 > Project: Ignite > Issue Type: Sub-task >Reporter: Semyon Danilov >Assignee: Semyon Danilov >Priority: Major > Fix For: 2.10 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Actual maximum size of WAL archive that can be reserved for historical > rebalance is calculated as minimum of three properties: > # DataStorageConfiguration#walHistSize (units: number of checkpoints) > # DataStorageConfiguration#maxWalArchiveSize (units: bytes) > # IgniteSystemProperties#IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE > (units: number of checkpoints) > The logic is a little unclear, so I propose following changes: > # Stop using walHistSize at all (it's already deprecated) for WAL truncation > # Use IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE only in case WAL archive > is managed externally (so we limit the quantity of checkpoints stored in > memory, but don't remove WAL files) > # Use -1 for maxWalArchiveSize (instead of Long.MAX_VALUE) to disable WAL > truncation -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13373) WAL segmentns do not released on releaseHistoryForPreloading()
[ https://issues.apache.org/jira/browse/IGNITE-13373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249084#comment-17249084 ] shivakumar commented on IGNITE-13373: - Hi [~YAMolochkov] Could you please consider this for 2.9.1 release ? It seems this code change (also IGNITE-12892) solves major blocker issue described here [http://apache-ignite-users.70518.x6.nabble.com/Could-not-clear-historyMap-due-to-WAL-reservation-on-cp-td34779.html] [http://apache-ignite-users.70518.x6.nabble.com/connecting-to-visor-shell-permanently-stops-unwanted-WAL-clean-up-td34737.html] Regards Shiva > WAL segmentns do not released on releaseHistoryForPreloading() > -- > > Key: IGNITE-13373 > URL: https://issues.apache.org/jira/browse/IGNITE-13373 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Assignee: Alexander Lapin >Priority: Major > Labels: 2.9.1-rc > Fix For: 2.10 > > Time Spent: 20m > Remaining Estimate: 0h > > * Reserve/releaseHistoryForPreloading() was reworked, now we store oldest > WALPointer that was reserved during reserveHistoryForPreloading in > reservedForPreloading field. As a result it's possible to release wall > reservation on releaseHIstoryForPreloading(). > * searchAndReserveCheckpoints() was slightly refactored: now it returns not > only an earliestValidCheckpoints but also an oldest reservedCheckpoint, so > that there’s no need to recalculate it within reserveHistoryForExchange(). > * -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-13190) Core defragmentation functions
[ https://issues.apache.org/jira/browse/IGNITE-13190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249069#comment-17249069 ] Maksim Timonin edited comment on IGNITE-13190 at 12/14/20, 3:56 PM: [~sergeychugunov] [~ibessonov] [~mmuzaf] Hi! I'm was discovering how defragmentation work and found that defragmentation performed only for single index per table [1]. Is it correct behavior? Also there is fixed offset 2 to get the index. It may lead to ArrayOutOfBoundsException in case of rebuilding hash index [2]. But I'm not sure whether this situation is possible or not. [1] [IndexingDefragmentation.java#L124|https://github.com/apache/ignite/blob/dc4b71b230e289e5b0d99b00e0632c9d1d0e6e82/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/defragmentation/IndexingDefragmentation.java#L124] [2] [GridH2Table.java#L1163|https://github.com/apache/ignite/blob/2fbbb676386515ea881e4e61f08864d6bc93225a/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GridH2Table.java#L1163] was (Author: timonin.maksim): [~sergeychugunov] [~ibessonov] [~mmuzaf] Hi! I'm was discovering how defragmentation work and found that defragmentation performed only for single index per table [1]. Is it correct behavior? Also there is fixed offset 2 to get the index. It may lead to ArrayOutOfBoundsException in case of rebuilding hash index [2]. I'm not sure whether this situation is possible or not. [1] [IndexingDefragmentation.java#L124|https://github.com/apache/ignite/blob/dc4b71b230e289e5b0d99b00e0632c9d1d0e6e82/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/defragmentation/IndexingDefragmentation.java#L124] [2] [GridH2Table.java#L1163|https://github.com/apache/ignite/blob/2fbbb676386515ea881e4e61f08864d6bc93225a/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GridH2Table.java#L1163] > Core defragmentation functions > -- > > Key: IGNITE-13190 > URL: https://issues.apache.org/jira/browse/IGNITE-13190 > Project: Ignite > Issue Type: Sub-task >Reporter: Sergey Chugunov >Assignee: Ivan Bessonov >Priority: Major > Labels: IEP-47 > Fix For: 2.10 > > Time Spent: 20h 50m > Remaining Estimate: 0h > > The following set of functions covering defragmentation happy-case needed: > * Initialization of defragmentation manager when node is started in > maintenance mode. > * Information about partition files is gathered by defrag mgr. > * For each partition file corresponding file of defragmented partition is > created and initialized. > * Keys are transferred from old partitions to new partitions. > * Checkpointer is aware of new partition files and flushes defragmented > memory to new partition files. > > No fault-tolerance code nor index defragmentation mappings are needed in this > task. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13190) Core defragmentation functions
[ https://issues.apache.org/jira/browse/IGNITE-13190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249069#comment-17249069 ] Maksim Timonin commented on IGNITE-13190: - [~sergeychugunov] [~ibessonov] [~mmuzaf] Hi! I'm was discovering how defragmentation work and found that defragmentation performed only for single index per table [1]. Is it correct behavior? Also there is fixed offset 2 to get the index. It may lead to ArrayOutOfBoundsException in case of rebuilding hash index [2]. I'm not sure whether this situation is possible or not. [1] [IndexingDefragmentation.java#L124|https://github.com/apache/ignite/blob/dc4b71b230e289e5b0d99b00e0632c9d1d0e6e82/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/defragmentation/IndexingDefragmentation.java#L124] [2] [GridH2Table.java#L1163|https://github.com/apache/ignite/blob/2fbbb676386515ea881e4e61f08864d6bc93225a/modules/indexing/src/main/java/org/apache/ignite/internal/processors/query/h2/opt/GridH2Table.java#L1163] > Core defragmentation functions > -- > > Key: IGNITE-13190 > URL: https://issues.apache.org/jira/browse/IGNITE-13190 > Project: Ignite > Issue Type: Sub-task >Reporter: Sergey Chugunov >Assignee: Ivan Bessonov >Priority: Major > Labels: IEP-47 > Fix For: 2.10 > > Time Spent: 20h 50m > Remaining Estimate: 0h > > The following set of functions covering defragmentation happy-case needed: > * Initialization of defragmentation manager when node is started in > maintenance mode. > * Information about partition files is gathered by defrag mgr. > * For each partition file corresponding file of defragmented partition is > created and initialized. > * Keys are transferred from old partitions to new partitions. > * Checkpointer is aware of new partition files and flushes defragmented > memory to new partition files. > > No fault-tolerance code nor index defragmentation mappings are needed in this > task. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13722) Ignite Spring Data: Ignite resources are not closed properly when closing Spring context.
[ https://issues.apache.org/jira/browse/IGNITE-13722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249060#comment-17249060 ] Mikhail Petrov commented on IGNITE-13722: - [~alex_pl], [~timonin.maksim] Tanks a lot for the review! > Ignite Spring Data: Ignite resources are not closed properly when closing > Spring context. > - > > Key: IGNITE-13722 > URL: https://issues.apache.org/jira/browse/IGNITE-13722 > Project: Ignite > Issue Type: Improvement >Reporter: Mikhail Petrov >Assignee: Mikhail Petrov >Priority: Major > > It's needed to close Ignite thin/thick client that is used by Spring Data for > accessing the Ignite cluster when closing Spring context. > Currently, if Ignite/ClientConfiguration is used to configure Ignite cluster > access for a repository (see RepositoryConfig), instances of > Ignite/IgniteClient remains unclosed after the close of Spring context. > To solve this issue it's proposed to move management of Ignite instances > needed by Spring Data for accessing the cluster to separate configurable > factory bean. It will be auto-registered by Spring Data and will close all > Ignite resources when closing the Spring context. > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-13792) Reconnecting clients trigger failure handler
[ https://issues.apache.org/jira/browse/IGNITE-13792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Uttsel reassigned IGNITE-13792: -- Assignee: Sergey Uttsel > Reconnecting clients trigger failure handler > > > Key: IGNITE-13792 > URL: https://issues.apache.org/jira/browse/IGNITE-13792 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.9 >Reporter: Pavel Vinokurov >Assignee: Sergey Uttsel >Priority: Major > Attachments: UnstableClients.java > > > {code:java} > Dec 01, 2020 9:38:29 PM java.util.logging.LogManager$RootLogger log > SEVERE: JVM will be halted immediately due to the failure: > [failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class > o.a.i.IgniteCheckedException: Affinity for topology version is not > initialized [locNode=b50635ff-0324-431b-bc34-00a6cd36c9e3, > grp=ignite-sys-cache, topVer=AffinityTopologyVersion [topVer=570, > minorTopVer=0], head=AffinityTopologyVersion [topVer=569, minorTopVer=0], > history=[AffinityTopologyVersion [topVer=551, minorTopVer=0], > AffinityTopologyVersion [topVer=552, minorTopVer=0], AffinityTopologyVersion > [topVer=553, minorTopVer=0], AffinityTopologyVersion [topVer=554, > minorTopVer=0], AffinityTopologyVersion [topVer=555, minorTopVer=0], > AffinityTopologyVersion [topVer=556, minorTopVer=0], AffinityTopologyVersion > [topVer=557, minorTopVer=0], AffinityTopologyVersion [topVer=558, > minorTopVer=0], AffinityTopologyVersion [topVer=559, minorTopVer=0], > AffinityTopologyVersion [topVer=560, minorTopVer=0], AffinityTopologyVersion > [topVer=561, minorTopVer=0], AffinityTopologyVersion [topVer=562, > minorTopVer=0], AffinityTopologyVersion [topVer=563, minorTopVer=0], > AffinityTopologyVersion [topVer=564, minorTopVer=0], AffinityTopologyVersion > [topVer=565, minorTopVer=0], AffinityTopologyVersion [topVer=566, > minorTopVer=0], AffinityTopologyVersion [topVer=567, minorTopVer=0], > AffinityTopologyVersion [topVer=568, minorTopVer=0], AffinityTopologyVersion > [topVer=569, minorTopVer=0] > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13848) Premature update SegmentReservationStorage#minReserveIdx during truncate of segments
[ https://issues.apache.org/jira/browse/IGNITE-13848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249039#comment-17249039 ] Ignite TC Bot commented on IGNITE-13848: {panel:title=Branch: [pull/8573/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/8573/head] Base: [master] : New Tests (2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1} {color:#8b}PDS 2{color} [[tests 2|https://ci.ignite.apache.org/viewLog.html?buildId=5789819]] * {color:#013220}IgnitePdsTestSuite2: IgnitePdsReserveWalSegmentsTest.testMinReserveIdx - PASSED{color} * {color:#013220}IgnitePdsTestSuite2: IgnitePdsReserveWalSegmentsWithCompactionTest.testMinReserveIdx - PASSED{color} {panel} [TeamCity *-- Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=5789847buildTypeId=IgniteTests24Java8_RunAll] > Premature update SegmentReservationStorage#minReserveIdx during truncate of > segments > - > > Key: IGNITE-13848 > URL: https://issues.apache.org/jira/browse/IGNITE-13848 > Project: Ignite > Issue Type: Bug > Components: persistence >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Fix For: 2.10 > > Time Spent: 10m > Remaining Estimate: 0h > > It was found premature *SegmentReservationStorage#minReserveIdx* update in > *FileWriteAheadLogManager#truncate*. Which creates the wrong state of the > segments in the archive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13828) Change ducktape dependency to own fork
[ https://issues.apache.org/jira/browse/IGNITE-13828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249036#comment-17249036 ] Anton Vinogradov commented on IGNITE-13828: --- Merged to ignite-ducktape. Thanks for your contribution. > Change ducktape dependency to own fork > -- > > Key: IGNITE-13828 > URL: https://issues.apache.org/jira/browse/IGNITE-13828 > Project: Ignite > Issue Type: Task >Reporter: Sergei Ryzhov >Assignee: Sergei Ryzhov >Priority: Minor > Labels: IEP-56, ducktape > Time Spent: 40m > Remaining Estimate: 0h > > The ducktape repository doesn't contain everything we need, so we need to use > our own fork with extensions > Contains changes: > Fix cluster property of Test > #258 opened on Oct 29 by ivandasch • Review required > https://github.com/confluentinc/ducktape/pull/258 > Fix BackgroundThreadService constructor. > #256 opened on Oct 28 by ivandasch • Review required > https://github.com/confluentinc/ducktape/pull/256 > Add optional clean parameter to Service.start. > #254 opened on Sep 25 by ivandasch • Review required > https://github.com/confluentinc/ducktape/pull/254 > Fix: set DucktapeJSONEncoder as default encoder in reports. > #251 opened on Sep 7 by ivandasch • Review required > https://github.com/confluentinc/ducktape/pull/251 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13851) Document persistence forward compatibility policy
Ilya Kasnacheev created IGNITE-13851: Summary: Document persistence forward compatibility policy Key: IGNITE-13851 URL: https://issues.apache.org/jira/browse/IGNITE-13851 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Ilya Kasnacheev Fix For: 2.10 We have a policy that you can upgrade from 2.x to 2.y, y > x, while keeping your persistence files. Let's document that fact. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13847) Make GridEncryptionManager#onWalSegmentRemoved async
[ https://issues.apache.org/jira/browse/IGNITE-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-13847: - Description: When implementing IGNITE-13831 I was faced with deadlock. When execute *FileWriteAheadLogManager#rollOver*, begin to clean WAL archive since we have reached the *DataStorageConfiguration#maxWalArchiveSize*, after deleting a segment, execute the *GridEncryptionManager#onWalSegmentRemoved* that wants to write to the metastore, but it will not succeed, since it will wait for *FileWriteAheadLogManager#rollOver*. I suggest making the *GridEncryptionManager#onWalSegmentRemoved* asynchronous in a separate pool, for example, as a *CacheGroupPageScanner#singleExecSvc*. was: Make the *GridEncryptionManager#onWalSegmentRemoved* asynchronous to prevent deadlocks in the *FileWriteAheadLogManager*. > Make GridEncryptionManager#onWalSegmentRemoved async > > > Key: IGNITE-13847 > URL: https://issues.apache.org/jira/browse/IGNITE-13847 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Kirill Tkalenko >Priority: Major > Labels: IEP-18 > Fix For: 2.10 > > > When implementing IGNITE-13831 I was faced with deadlock. > When execute *FileWriteAheadLogManager#rollOver*, begin to clean WAL archive > since we have reached the *DataStorageConfiguration#maxWalArchiveSize*, after > deleting a segment, execute the *GridEncryptionManager#onWalSegmentRemoved* > that wants to write to the metastore, but it will not succeed, since it will > wait for *FileWriteAheadLogManager#rollOver*. > I suggest making the *GridEncryptionManager#onWalSegmentRemoved* asynchronous > in a separate pool, for example, as a *CacheGroupPageScanner#singleExecSvc*. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13847) Make GridEncryptionManager#onWalSegmentRemoved async
[ https://issues.apache.org/jira/browse/IGNITE-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-13847: - Description: Make the *GridEncryptionManager#onWalSegmentRemoved* asynchronous to prevent deadlocks in the *FileWriteAheadLogManager*. was:Make the *GridEncryptionManager#onWalSegmentRemoved* asynchronous to prevent deadlocks in the *FileWriteAheadLogManager*. > Make GridEncryptionManager#onWalSegmentRemoved async > > > Key: IGNITE-13847 > URL: https://issues.apache.org/jira/browse/IGNITE-13847 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Kirill Tkalenko >Priority: Major > Labels: IEP-18 > Fix For: 2.10 > > > Make the *GridEncryptionManager#onWalSegmentRemoved* asynchronous to prevent > deadlocks in the *FileWriteAheadLogManager*. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13831) Move WAL archive cleanup from checkpoint to rollover
[ https://issues.apache.org/jira/browse/IGNITE-13831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-13831: - Description: Users expect *DataStorageConfiguration#maxWalArchiveSize* to mean that WAL archive will not exceed this value, but it is not. It seems that to reduce the chance of getting into a situation when we exceed WAL archive, it will be lowed when we clean it when switching to a new segment than at the end of the checkpoint. It is proposed to move the archive cleanup to *FileWriteAheadLogManager#rollOver* when the *DataStorageConfiguration#maxWalArchiveSize* is reached. was: At the moment, WAL archive is cleared at the end of the checkpoint, which does not seem correct and needs to be moved to the *FileWriteAheadLogManager*. It is suggested to clear WAL archive in *FileWriteAheadLogManager#rollOver* when the *DataStorageConfiguration#maxWalArchiveSize* is reached. > Move WAL archive cleanup from checkpoint to rollover > > > Key: IGNITE-13831 > URL: https://issues.apache.org/jira/browse/IGNITE-13831 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Fix For: 2.10 > > Time Spent: 10m > Remaining Estimate: 0h > > Users expect *DataStorageConfiguration#maxWalArchiveSize* to mean that WAL > archive will not exceed this value, but it is not. > It seems that to reduce the chance of getting into a situation when we exceed > WAL archive, it will be lowed when we clean it when switching to a new > segment than at the end of the checkpoint. It is proposed to move the archive > cleanup to *FileWriteAheadLogManager#rollOver* when the > *DataStorageConfiguration#maxWalArchiveSize* is reached. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13850) Improve the TracingSpi interface
Amelchev Nikita created IGNITE-13850: Summary: Improve the TracingSpi interface Key: IGNITE-13850 URL: https://issues.apache.org/jira/browse/IGNITE-13850 Project: Ignite Issue Type: Improvement Reporter: Amelchev Nikita Assignee: Amelchev Nikita Need to investigate and improve the TracingSpi interface: 1. Add support for non-strings tags and log points: primitives, arrays etc 2. Add ability to postpone adding span tags and log points 3. Investigate ability to cache some tags -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-13817) Calcite bug. select count assertionError.
[ https://issues.apache.org/jira/browse/IGNITE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov reassigned IGNITE-13817: - Assignee: Konstantin Orlov > Calcite bug. select count assertionError. > - > > Key: IGNITE-13817 > URL: https://issues.apache.org/jira/browse/IGNITE-13817 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 2.9 >Reporter: Stanilovsky Evgeny >Assignee: Konstantin Orlov >Priority: Major > Labels: calcite > Time Spent: 0.5h > Remaining Estimate: 0h > > request: > {noformat} > SELECT count(*) > FROM > R, > T, > B > WHERE R.BKEY = B.KEY > AND R.ID = T.ID > AND R.VERSION = T.VERSION > AND T.BOOK = '' > AND B.ISL = TRUE > {noformat} > breaks down calcite: > {noformat} > java.lang.AssertionError > at org.apache.calcite.plan.volcano.RelSet.mergeWith(RelSet.java:457) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.merge(VolcanoPlanner.java:1046) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.rename(VolcanoPlanner.java:903) > at org.apache.calcite.plan.volcano.RelSet.mergeWith(RelSet.java:432) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.merge(VolcanoPlanner.java:1046) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.registerSubset(VolcanoPlanner.java:1287) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1166) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:589) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:604) > at > org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:148) > at > org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:268) > at > org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:283) > at > org.apache.calcite.rel.convert.ConverterRule.onMatch(ConverterRule.java:169) > at > org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:229) > at > org.apache.calcite.plan.volcano.TopDownRuleDriver.applyGenerator(TopDownRuleDriver.java:142) > at > org.apache.calcite.plan.volcano.TopDownRuleDriver.access$600(TopDownRuleDriver.java:47) > at > org.apache.calcite.plan.volcano.TopDownRuleDriver$ApplyRule.perform(TopDownRuleDriver.java:519) > at > org.apache.calcite.plan.volcano.TopDownRuleDriver.drive(TopDownRuleDriver.java:101) > at > org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:510) > at > org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:312) > at > org.apache.ignite.internal.processors.query.calcite.prepare.IgnitePlanner.transform(IgnitePlanner.java:258) > at > org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.optimize(ExecutionServiceImpl.java:616) > at > org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.prepareQuery(ExecutionServiceImpl.java:568) > at > org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.prepareSingle(ExecutionServiceImpl.java:542) > at > org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.prepareQuery(ExecutionServiceImpl.java:501) > at > org.apache.ignite.internal.processors.query.calcite.prepare.QueryPlanCacheImpl.queryPlan(QueryPlanCacheImpl.java:84) > at > org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.executeQuery(ExecutionServiceImpl.java:378) > at > org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessor.query(CalciteQueryProcessor.java:240) > at > org.apache.ignite.internal.processors.query.calcite.CalciteQueryProcessorTest.test0(CalciteQueryProcessorTest.java:117) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2373) > at java.lang.Thread.run(Thread.java:748) > {noformat} -- This
[jira] [Commented] (IGNITE-13734) .NET Service loses returned array type information
[ https://issues.apache.org/jira/browse/IGNITE-13734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248895#comment-17248895 ] Yaroslav Molochkov commented on IGNITE-13734: - [~kukushal], I think we need to investigate .net ability to cast results to the needed type, so it might turn out that this ticket is not needed > .NET Service loses returned array type information > -- > > Key: IGNITE-13734 > URL: https://issues.apache.org/jira/browse/IGNITE-13734 > Project: Ignite > Issue Type: Bug > Components: platforms >Affects Versions: 2.9 >Reporter: Alexey Kukushkin >Assignee: Nikolai Kulagin >Priority: Major > > .NET service client receives Object[] instead of strongly typed array from a > .NET service. > There was another already resolved similar issue IGNITE-12823 that addressed > the problem of using arrays as parameters. The problem of using arrays as > results still exists. > h3. Reproducer > A .NET service returning an array of user-defined types is deployed: > {code:c#} > public interface ITestService > { > Parameter[] TestReturnParametersArray(); > } > public sealed class Parameter > { > public int Id { get; set; } > public int[] Values { get; set; } > } > {code} > A .NET client calls the service: > {code:c#} > Parameter[] res = svcProxy.TestReturnParametersArray() > {code} > The service call fails with exception: > {code} > System.InvalidCastException : Unable to cast object of type 'System.Object[]' > to type 'Parameter[]'. > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13748) Schema configuration public API
[ https://issues.apache.org/jira/browse/IGNITE-13748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich updated IGNITE-13748: --- Component/s: sql > Schema configuration public API > --- > > Key: IGNITE-13748 > URL: https://issues.apache.org/jira/browse/IGNITE-13748 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > > Let's implement a public API classes for initial schema declaration. > Considering next: > * It should be possible to create schema without having any Class instance, > like we do now with QueryEntity. > * Automatic schema creation based on annotated classes, like we have in > ccfg.setIndexedTypes() > * Transient fields semantic. > * Using builder pattern for schema configuration. > * Split type-system into internal native-types (introduced GG-13617) and > public API types. The latest ones should be portable (e.g. could be used in > .Net) > * API can be extracted to separate module that will not have any dependency > on other Ignite modules. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13849) Calcite Integration. Add unit tests for all IgniteRel nodes serialization / deserialization
[ https://issues.apache.org/jira/browse/IGNITE-13849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248885#comment-17248885 ] Konstantin Orlov commented on IGNITE-13849: --- One of the possible issue here - missed collation of IgniteExchange > Calcite Integration. Add unit tests for all IgniteRel nodes serialization / > deserialization > --- > > Key: IGNITE-13849 > URL: https://issues.apache.org/jira/browse/IGNITE-13849 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Taras Ledkov >Assignee: Taras Ledkov >Priority: Major > > We have to check serialization for each node (child classes of the IgniteRel). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12892) Clarify WAL archive size configuration
[ https://issues.apache.org/jira/browse/IGNITE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248872#comment-17248872 ] Kirill Tkalenko commented on IGNITE-12892: -- Hi, [~shm]! I think you can ask the release engineer to add this ticket to the [release|https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9.1#ApacheIgnite2.9.1-2.9/2.9.1PerformanceComparison]. It seems to me that most likely the IGNITE-13373 fixed the problem you indicated, I already made a description in the [PR's comment |https://github.com/apache/ignite/pull/8550#issuecomment-744032357]. > Clarify WAL archive size configuration > -- > > Key: IGNITE-12892 > URL: https://issues.apache.org/jira/browse/IGNITE-12892 > Project: Ignite > Issue Type: Sub-task >Reporter: Semyon Danilov >Assignee: Semyon Danilov >Priority: Major > Fix For: 2.10 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Actual maximum size of WAL archive that can be reserved for historical > rebalance is calculated as minimum of three properties: > # DataStorageConfiguration#walHistSize (units: number of checkpoints) > # DataStorageConfiguration#maxWalArchiveSize (units: bytes) > # IgniteSystemProperties#IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE > (units: number of checkpoints) > The logic is a little unclear, so I propose following changes: > # Stop using walHistSize at all (it's already deprecated) for WAL truncation > # Use IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE only in case WAL archive > is managed externally (so we limit the quantity of checkpoints stored in > memory, but don't remove WAL files) > # Use -1 for maxWalArchiveSize (instead of Long.MAX_VALUE) to disable WAL > truncation -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13849) Calcite Integration. Add unit tests for all IgniteRel nodes serialization / deserialization
Taras Ledkov created IGNITE-13849: - Summary: Calcite Integration. Add unit tests for all IgniteRel nodes serialization / deserialization Key: IGNITE-13849 URL: https://issues.apache.org/jira/browse/IGNITE-13849 Project: Ignite Issue Type: Bug Components: sql Reporter: Taras Ledkov Assignee: Taras Ledkov We have to check serialization for each node (child classes of the IgniteRel). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-12892) Clarify WAL archive size configuration
[ https://issues.apache.org/jira/browse/IGNITE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248845#comment-17248845 ] shivakumar edited comment on IGNITE-12892 at 12/14/20, 9:46 AM: Is this code change part of 2.9.1 release ? As per my testing looks like this code change solves the blocking issue reported here [http://apache-ignite-users.70518.x6.nabble.com/Could-not-clear-historyMap-due-to-WAL-reservation-on-cp-td34779.html] [http://apache-ignite-users.70518.x6.nabble.com/connecting-to-visor-shell-permanently-stops-unwanted-WAL-clean-up-td34737.html] If it is not part of 2.9.1 then is it possible to backport to 2.9.0 easily ? Or it has dependency on other code changes? was (Author: shm): Is this code change part of 2.9.1 release ? As pery testing looks like this code change solves the blocking issue reported here [http://apache-ignite-users.70518.x6.nabble.com/Could-not-clear-historyMap-due-to-WAL-reservation-on-cp-td34779.html] [http://apache-ignite-users.70518.x6.nabble.com/connecting-to-visor-shell-permanently-stops-unwanted-WAL-clean-up-td34737.html] > Clarify WAL archive size configuration > -- > > Key: IGNITE-12892 > URL: https://issues.apache.org/jira/browse/IGNITE-12892 > Project: Ignite > Issue Type: Sub-task >Reporter: Semyon Danilov >Assignee: Semyon Danilov >Priority: Major > Fix For: 2.10 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Actual maximum size of WAL archive that can be reserved for historical > rebalance is calculated as minimum of three properties: > # DataStorageConfiguration#walHistSize (units: number of checkpoints) > # DataStorageConfiguration#maxWalArchiveSize (units: bytes) > # IgniteSystemProperties#IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE > (units: number of checkpoints) > The logic is a little unclear, so I propose following changes: > # Stop using walHistSize at all (it's already deprecated) for WAL truncation > # Use IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE only in case WAL archive > is managed externally (so we limit the quantity of checkpoints stored in > memory, but don't remove WAL files) > # Use -1 for maxWalArchiveSize (instead of Long.MAX_VALUE) to disable WAL > truncation -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12892) Clarify WAL archive size configuration
[ https://issues.apache.org/jira/browse/IGNITE-12892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248845#comment-17248845 ] shivakumar commented on IGNITE-12892: - Is this code change part of 2.9.1 release ? As pery testing looks like this code change solves the blocking issue reported here [http://apache-ignite-users.70518.x6.nabble.com/Could-not-clear-historyMap-due-to-WAL-reservation-on-cp-td34779.html] [http://apache-ignite-users.70518.x6.nabble.com/connecting-to-visor-shell-permanently-stops-unwanted-WAL-clean-up-td34737.html] > Clarify WAL archive size configuration > -- > > Key: IGNITE-12892 > URL: https://issues.apache.org/jira/browse/IGNITE-12892 > Project: Ignite > Issue Type: Sub-task >Reporter: Semyon Danilov >Assignee: Semyon Danilov >Priority: Major > Fix For: 2.10 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Actual maximum size of WAL archive that can be reserved for historical > rebalance is calculated as minimum of three properties: > # DataStorageConfiguration#walHistSize (units: number of checkpoints) > # DataStorageConfiguration#maxWalArchiveSize (units: bytes) > # IgniteSystemProperties#IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE > (units: number of checkpoints) > The logic is a little unclear, so I propose following changes: > # Stop using walHistSize at all (it's already deprecated) for WAL truncation > # Use IGNITE_PDS_MAX_CHECKPOINT_MEMORY_HISTORY_SIZE only in case WAL archive > is managed externally (so we limit the quantity of checkpoints stored in > memory, but don't remove WAL files) > # Use -1 for maxWalArchiveSize (instead of Long.MAX_VALUE) to disable WAL > truncation -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13848) Premature update SegmentReservationStorage#minReserveIdx during truncate of segments
Kirill Tkalenko created IGNITE-13848: Summary: Premature update SegmentReservationStorage#minReserveIdx during truncate of segments Key: IGNITE-13848 URL: https://issues.apache.org/jira/browse/IGNITE-13848 Project: Ignite Issue Type: Bug Components: persistence Reporter: Kirill Tkalenko Assignee: Kirill Tkalenko Fix For: 2.10 It was found premature *SegmentReservationStorage#minReserveIdx* update in *FileWriteAheadLogManager#truncate*. Which creates the wrong state of the segments in the archive. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-13618) Provide generated and reflection-based class (de)serializers
[ https://issues.apache.org/jira/browse/IGNITE-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17248442#comment-17248442 ] Andrey Mashenkov edited comment on IGNITE-13618 at 12/14/20, 9:07 AM: -- 1. Unfortunatelly, VarHandles/MethodHandle support is limited in Janino. Seems, it can't correctly compile code and link MethodHandler invoke methods with polymorphic signatures. As we can't write plain code and access fields/call method directly therefore generated code will be very similar to the reflection-based code and hardly possible it could be more efficient. 2. Janino expects class body code only, but JavaPoet can generate only full class code with class signature and package declaration. was (Author: amashenkov): 1. Unfortunatelly, VarHandles/MethodHandle support is limited in Janino. Seems, it can't correctly compile code and link MethodHandler invoke methods with polymorphic signatures. As we can't write plain code and access fields/call method directly therefore generated code will be very similar to the reflection-based code and hardly possible it could be more efficient. 2. Janino expects class body code only, but however JavaPoet can generate only full class code with class signature and package declaration. > Provide generated and reflection-based class (de)serializers > > > Key: IGNITE-13618 > URL: https://issues.apache.org/jira/browse/IGNITE-13618 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Alexey Goncharuk >Assignee: Andrey Mashenkov >Priority: Major > Labels: iep-54, ignite-3 > Attachments: benchmark-results.txt > > > Let's prototype object serializer for type-system described in IEP-54, and > benchmark them to check if generated code approach is a better one. > * For reflection-based serializer we can use Unsafe (see GridUnsafe class) > which works fine for previour versions of Ignite. > * To keep focus on the ticket needs mainly, we can use Janino project for > code generator. > * In addition, reflection-based serialized could have a version using Java9+ > VanHandles and it's could be hidden under 'java-9+' maven-profile. > Hints: > Javapoet can be used as codegenerator, Janino as a compiler, class loader. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13659) TDE - Phase 3. Documentation
[ https://issues.apache.org/jira/browse/IGNITE-13659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Pereslegin updated IGNITE-13659: -- Attachment: (was: html-ver.zip) > TDE - Phase 3. Documentation > > > Key: IGNITE-13659 > URL: https://issues.apache.org/jira/browse/IGNITE-13659 > Project: Ignite > Issue Type: Task > Components: documentation >Reporter: Pavel Pereslegin >Assignee: Pavel Pereslegin >Priority: Major > Labels: IEP-18, tde-phase-3 > Fix For: 2.10 > > Time Spent: 20m > Remaining Estimate: 0h > > Write documentation on the cache encryption key rotation function. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13845) Add checkpoint lock hold metrics
[ https://issues.apache.org/jira/browse/IGNITE-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amelchev Nikita updated IGNITE-13845: - Fix Version/s: 2.10 > Add checkpoint lock hold metrics > > > Key: IGNITE-13845 > URL: https://issues.apache.org/jira/browse/IGNITE-13845 > Project: Ignite > Issue Type: New Feature >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Fix For: 2.10 > > Time Spent: 10m > Remaining Estimate: 0h > > Add last checkpoint lock hold duration and durations histogram metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005)