[jira] [Commented] (IGNITE-6994) Need to document PartitionLossPolicy
[
https://issues.apache.org/jira/browse/IGNITE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355102#comment-16355102
]
Sergey Puchnin commented on IGNITE-6994:
The documentation is updated please proofread it.
https://apacheignite.readme.io/v2.3/docs/cache-modes-24#partition-loss-policies
> Need to document PartitionLossPolicy
>
>
> Key: IGNITE-6994
> URL: https://issues.apache.org/jira/browse/IGNITE-6994
> Project: Ignite
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Sergey Puchnin
>Priority: Critical
> Labels: documentation
> Fix For: 2.4
>
>
> Since 2.0 we have a feature that makes cache(s) unavailable in case of data
> loss; exact behavior is controlled by {{PartitionLossPolicy}}:
> [https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/PartitionLossPolicy.html]
> However, there is no mentioning in the documentation about this. Need to
> provide an explanation of how and when it should be used and provide
> configuration examples.
> The documentation has to address questions and misunderstandings asked in
> these discussions:
> *
> [http://apache-ignite-developers.2346864.n4.nabble.com/Partition-loss-policy-how-to-use-td25341.html]
> *
> [http://apache-ignite-developers.2346864.n4.nabble.com/Partition-loss-policy-to-disable-cache-completely-td26212.html]
> Improve the JavaDoc too whenever is possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IGNITE-6994) Need to document PartitionLossPolicy
[
https://issues.apache.org/jira/browse/IGNITE-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Puchnin reassigned IGNITE-6994:
--
Assignee: Denis Magda (was: Sergey Puchnin)
> Need to document PartitionLossPolicy
>
>
> Key: IGNITE-6994
> URL: https://issues.apache.org/jira/browse/IGNITE-6994
> Project: Ignite
> Issue Type: Task
> Components: documentation
>Affects Versions: 2.3
>Reporter: Valentin Kulichenko
>Assignee: Denis Magda
>Priority: Critical
> Labels: documentation
> Fix For: 2.4
>
>
> Since 2.0 we have a feature that makes cache(s) unavailable in case of data
> loss; exact behavior is controlled by {{PartitionLossPolicy}}:
> [https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/PartitionLossPolicy.html]
> However, there is no mentioning in the documentation about this. Need to
> provide an explanation of how and when it should be used and provide
> configuration examples.
> The documentation has to address questions and misunderstandings asked in
> these discussions:
> *
> [http://apache-ignite-developers.2346864.n4.nabble.com/Partition-loss-policy-how-to-use-td25341.html]
> *
> [http://apache-ignite-developers.2346864.n4.nabble.com/Partition-loss-policy-to-disable-cache-completely-td26212.html]
> Improve the JavaDoc too whenever is possible.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-7496) Review a SQL documentation
Sergey Puchnin created IGNITE-7496: -- Summary: Review a SQL documentation Key: IGNITE-7496 URL: https://issues.apache.org/jira/browse/IGNITE-7496 Project: Ignite Issue Type: Bug Reporter: Sergey Puchnin Assignee: Denis Magda In a discussion on dev list http://apache-ignite-developers.2346864.n4.nabble.com/SparkDataFrame-Query-Optimization-Prototype-tt26249.html was noticed there are some gaps in SQL documentation. It's necessary to review it. I've added next system functions: CASEWHEN Function, CAST, CONVERT, TABLE And for my mind, next functions aren't applicable for Ignite: ARRAY_GET, ARRAY_LENGTH, ARRAY_CONTAINS, CSVREAD, CSVWRITE, DATABASE, DATABASE_PATH, DISK_SPACE_USED, FILE_READ, FILE_WRITE, LINK_SCHEMA, MEMORY_FREE, MEMORY_USED, LOCK_MODE, LOCK_TIMEOUT, READONLY, CURRVAL, AUTOCOMMIT, CANCEL_SESSION, IDENTITY, NEXTVAL, ROWNUM, SCHEMA, SCOPE_IDENTITY, SESSION_ID, SET, TRANSACTION_ID, TRUNCATE_VALUE, USER, H2VERSION -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-3690) Log Message Glossary
[ https://issues.apache.org/jira/browse/IGNITE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16331989#comment-16331989 ] Sergey Puchnin commented on IGNITE-3690: As it was discussed on dev list (http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-not-friendly-for-Monitoring-tt20802.html#none) let's split this activity into two steps and reserver next error ranges for the domains: -- the phase 1 1. 1-10499 UnExpected, UnKnown 2. 10500-12499 Cluster and Topology 10500-10599 Discovery 10600-10699 Segmentation 10700-10799 Node Startup 10800-10899 Communication 10900-10999 Queue 11000-11099 Activate, startup process 11100-11199 Base line topology 11200-11299 Marshaller 11300-11399 Metadata 11400-11499 Topology Validate 3. 12500-14199 Cache and Storage 12500-12599 Partition map exchange 12600-12699 Balancing 12700-12799 Long-running transactions 12800-12899 Checkpoint 12900-12999 Create cache 13000-13099 Destroy cache 13100-13199 Data loading & streaming 4. 14200-15699 SQL 14200-14299 Long-running queries 14300-14399 Parsing 14400-14499 Queries 14500-14599 Scan Queries 14600-14699 SqlLine 5. 15700-17099 Compute 15700-15799 Deployment 15800-15899 spi.checkpoint 15900-15999 spi.collision 11600-16099 Job Schedule -- the phase 2 6. 17100-19199 Service 7. 19200-21199 Security 8. 21200-23199 ML 9. 23200-25199 Integrations 10. 25200-27199 WebConsole 11. 27200-29199 Vendor-Specific 27200-27299 GG FYI: [~dmagda] > Log Message Glossary > > > Key: IGNITE-3690 > URL: https://issues.apache.org/jira/browse/IGNITE-3690 > Project: Ignite > Issue Type: Improvement > Components: documentation >Affects Versions: 1.4 >Reporter: Denis Magda >Assignee: Denis Magda >Priority: Major > Fix For: 2.5 > > > It's time for Ignite to create a page which lists ALL the log messages, their > common causes and potential solutions. > Probably it makes sense to assign an unique ID for the most well-know errors > so that a user is able to look up an error in the glossary quicker. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7222) DiscoverySpi based on Apache ZooKeeper
[ https://issues.apache.org/jira/browse/IGNITE-7222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-7222: --- Labels: IEP-13 (was: ) > DiscoverySpi based on Apache ZooKeeper > -- > > Key: IGNITE-7222 > URL: https://issues.apache.org/jira/browse/IGNITE-7222 > Project: Ignite > Issue Type: Task > Components: general >Reporter: Semen Boikov >Assignee: Semen Boikov > Labels: IEP-13 > Fix For: 2.5 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (IGNITE-7250) Apache Ignite Transactional Subsystem Architecture
[ https://issues.apache.org/jira/browse/IGNITE-7250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16298067#comment-16298067 ] Sergey Puchnin commented on IGNITE-7250: Please use the last version 0.4 PDF files were attached. Google docs: https://docs.google.com/document/d/10reHxrCO7mEP8l7HeRc01mPSEIxCmwwWAxHoIOZo5c8/edit https://docs.google.com/document/d/162dPdfNcz6mLjdPzY6JjM_3zK2Q8nklx3EE5rFzUAns/edit > Apache Ignite Transactional Subsystem Architecture > -- > > Key: IGNITE-7250 > URL: https://issues.apache.org/jira/browse/IGNITE-7250 > Project: Ignite > Issue Type: Task > Components: documentation >Reporter: Denis Magda > Fix For: 2.4 > > Attachments: Apache Ignite Transactions Architecture Eng 0.2.pdf, > Apache Ignite Transactions Architecture Eng 0.4.pdf, Apache Ignite > Transactions Architecture Rus 0.2.pdf, Apache Ignite Transactions > Architecture Rus 0.4.pdf > > > A couple of community members prepared a well-defined document about Ignite > transactions architecture. See the PDF version attached. > Use this document to enrich the existing page on Ignite transactions. Even > more, it makes sense to define a unique category for transactions and create > several pages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-7250) Apache Ignite Transactional Subsystem Architecture
[ https://issues.apache.org/jira/browse/IGNITE-7250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-7250: --- Attachment: Apache Ignite Transactions Architecture Eng 0.4.pdf Apache Ignite Transactions Architecture Rus 0.4.pdf > Apache Ignite Transactional Subsystem Architecture > -- > > Key: IGNITE-7250 > URL: https://issues.apache.org/jira/browse/IGNITE-7250 > Project: Ignite > Issue Type: Task > Components: documentation >Reporter: Denis Magda > Fix For: 2.4 > > Attachments: Apache Ignite Transactions Architecture Eng 0.2.pdf, > Apache Ignite Transactions Architecture Eng 0.4.pdf, Apache Ignite > Transactions Architecture Rus 0.2.pdf, Apache Ignite Transactions > Architecture Rus 0.4.pdf > > > A couple of community members prepared a well-defined document about Ignite > transactions architecture. See the PDF version attached. > Use this document to enrich the existing page on Ignite transactions. Even > more, it makes sense to define a unique category for transactions and create > several pages. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (IGNITE-6959) Split a ttl index tree by partition
[ https://issues.apache.org/jira/browse/IGNITE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin reassigned IGNITE-6959: -- Assignee: Alexey Goncharuk (was: Sergey Puchnin) > Split a ttl index tree by partition > - > > Key: IGNITE-6959 > URL: https://issues.apache.org/jira/browse/IGNITE-6959 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Puchnin >Assignee: Alexey Goncharuk >Priority: Minor > > Now a tree to trace an entries' TTL presented only one per a node. To improve > a performance need to split the tree per partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6959) Split a ttl index tree by partition
[ https://issues.apache.org/jira/browse/IGNITE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6959: --- Description: Now a tree to trace an entries' TTL presented only one per a node. To improve a performance need to split the tree per partition. was: Now a tree to trace an entries' TTL presented only one per a node. To improve a performace need to > Split a ttl index tree by partition > - > > Key: IGNITE-6959 > URL: https://issues.apache.org/jira/browse/IGNITE-6959 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Puchnin >Assignee: Sergey Puchnin >Priority: Minor > > Now a tree to trace an entries' TTL presented only one per a node. To improve > a performance need to split the tree per partition. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6959) Split a ttl index tree by partition
[ https://issues.apache.org/jira/browse/IGNITE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6959: --- Description: Now a tree to trace an entries' TTL presented only one per a node. To improve a performace need to was: h2. Use Case * User starts up cluster of N nodes and fills it with some data. * User splits the cluster into two halves and modifies data in each half independently. * User tries to join two halves back into one - irresolvable conflicts in data for the same key may happen. h2. BaselineTopology Versioning Each BLT contains enough information to check its compatibility with other BLT. If BLT of joining node is not compatible with BLT grid is working on at the moment, node is not allowed to join the grid and must fail with proper exception. > Split a ttl index tree by partition > - > > Key: IGNITE-6959 > URL: https://issues.apache.org/jira/browse/IGNITE-6959 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Puchnin >Assignee: Sergey Puchnin >Priority: Minor > > Now a tree to trace an entries' TTL presented only one per a node. To improve > a performace need to -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6959) Split a ttl index tree by partition
[ https://issues.apache.org/jira/browse/IGNITE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6959: --- Labels: (was: IEP-4) > Split a ttl index tree by partition > - > > Key: IGNITE-6959 > URL: https://issues.apache.org/jira/browse/IGNITE-6959 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Puchnin >Assignee: Sergey Puchnin >Priority: Minor > > h2. Use Case > * User starts up cluster of N nodes and fills it with some data. > * User splits the cluster into two halves and modifies data in each half > independently. > * User tries to join two halves back into one - irresolvable conflicts in > data for the same key may happen. > h2. BaselineTopology Versioning > Each BLT contains enough information to check its compatibility with other > BLT. If BLT of joining node is not compatible with BLT grid is working on at > the moment, node is not allowed to join the grid and must fail with proper > exception. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6959) Split a ttl index tree by partition
Sergey Puchnin created IGNITE-6959: -- Summary: Split a ttl index tree by partition Key: IGNITE-6959 URL: https://issues.apache.org/jira/browse/IGNITE-6959 Project: Ignite Issue Type: Task Components: persistence Reporter: Sergey Puchnin Assignee: Sergey Chugunov h2. Use Case * User starts up cluster of N nodes and fills it with some data. * User splits the cluster into two halves and modifies data in each half independently. * User tries to join two halves back into one - irresolvable conflicts in data for the same key may happen. h2. BaselineTopology Versioning Each BLT contains enough information to check its compatibility with other BLT. If BLT of joining node is not compatible with BLT grid is working on at the moment, node is not allowed to join the grid and must fail with proper exception. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6959) Split a ttl index tree by partition
[ https://issues.apache.org/jira/browse/IGNITE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6959: --- Issue Type: Improvement (was: Task) > Split a ttl index tree by partition > - > > Key: IGNITE-6959 > URL: https://issues.apache.org/jira/browse/IGNITE-6959 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Puchnin >Assignee: Sergey Chugunov > Labels: IEP-4 > > h2. Use Case > * User starts up cluster of N nodes and fills it with some data. > * User splits the cluster into two halves and modifies data in each half > independently. > * User tries to join two halves back into one - irresolvable conflicts in > data for the same key may happen. > h2. BaselineTopology Versioning > Each BLT contains enough information to check its compatibility with other > BLT. If BLT of joining node is not compatible with BLT grid is working on at > the moment, node is not allowed to join the grid and must fail with proper > exception. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6959) Split a ttl index tree by partition
[ https://issues.apache.org/jira/browse/IGNITE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6959: --- Priority: Minor (was: Major) > Split a ttl index tree by partition > - > > Key: IGNITE-6959 > URL: https://issues.apache.org/jira/browse/IGNITE-6959 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Puchnin >Assignee: Sergey Chugunov >Priority: Minor > Labels: IEP-4 > > h2. Use Case > * User starts up cluster of N nodes and fills it with some data. > * User splits the cluster into two halves and modifies data in each half > independently. > * User tries to join two halves back into one - irresolvable conflicts in > data for the same key may happen. > h2. BaselineTopology Versioning > Each BLT contains enough information to check its compatibility with other > BLT. If BLT of joining node is not compatible with BLT grid is working on at > the moment, node is not allowed to join the grid and must fail with proper > exception. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (IGNITE-6959) Split a ttl index tree by partition
[ https://issues.apache.org/jira/browse/IGNITE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin reassigned IGNITE-6959: -- Assignee: Sergey Puchnin (was: Sergey Chugunov) > Split a ttl index tree by partition > - > > Key: IGNITE-6959 > URL: https://issues.apache.org/jira/browse/IGNITE-6959 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Sergey Puchnin >Assignee: Sergey Puchnin >Priority: Minor > Labels: IEP-4 > > h2. Use Case > * User starts up cluster of N nodes and fills it with some data. > * User splits the cluster into two halves and modifies data in each half > independently. > * User tries to join two halves back into one - irresolvable conflicts in > data for the same key may happen. > h2. BaselineTopology Versioning > Each BLT contains enough information to check its compatibility with other > BLT. If BLT of joining node is not compatible with BLT grid is working on at > the moment, node is not allowed to join the grid and must fail with proper > exception. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (IGNITE-6310) BaselineTopology versioning
[ https://issues.apache.org/jira/browse/IGNITE-6310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin reassigned IGNITE-6310: -- Assignee: (was: Sergey Chugunov) > BaselineTopology versioning > --- > > Key: IGNITE-6310 > URL: https://issues.apache.org/jira/browse/IGNITE-6310 > Project: Ignite > Issue Type: Task > Components: persistence >Reporter: Sergey Chugunov > Labels: IEP-4 > > h2. Use Case > * User starts up cluster of N nodes and fills it with some data. > * User splits the cluster into two halves and modifies data in each half > independently. > * User tries to join two halves back into one - irresolvable conflicts in > data for the same key may happen. > h2. BaselineTopology Versioning > Each BLT contains enough information to check its compatibility with other > BLT. If BLT of joining node is not compatible with BLT grid is working on at > the moment, node is not allowed to join the grid and must fail with proper > exception. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6930) Optionally to do not write free list updates to WAL
[
https://issues.apache.org/jira/browse/IGNITE-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Puchnin updated IGNITE-6930:
---
Labels: IEP-8 iep-1 (was: iep-1)
> Optionally to do not write free list updates to WAL
> ---
>
> Key: IGNITE-6930
> URL: https://issues.apache.org/jira/browse/IGNITE-6930
> Project: Ignite
> Issue Type: Task
> Security Level: Public(Viewable by anyone)
> Components: cache
>Reporter: Vladimir Ozerov
>Assignee: Taras Ledkov
> Labels: IEP-8, iep-1
> Fix For: 2.4
>
>
> When cache entry is created, we need to write update the free list. When
> entry is updated, we need to update free list(s) several times. Currently
> free list is persistent structure, so every update to it must be logged to be
> able to recover after crash. This may incur significant overhead, especially
> for small entries.
> E.g. this is how WAL for a single update looks like. "D" - updates with real
> data, "F" - free-list management:
> {code}
> 1. [D] DataRecord [writeEntries=[UnwrapDataEntry[k = key, v = [ BinaryObject
> [idHash=2053299190, hash=1986931360, typeId=-1580729813]], super = [DataEntry
> [cacheId=94416770, op=UPDATE, writeVer=GridCacheVersion [topVer=122147562,
> order=1510667560607, nodeOrder=1], partId=0, partCnt=4, super=WALRecord
> [size=0, chainSize=0, pos=null, type=DATA_RECORD]]
> 2. [F] PagesListRemovePageRecord [rmvdPageId=00010005,
> pageId=00010006, grpId=94416770, super=PageDeltaRecord
> [grpId=94416770, pageId=00010006, super=WALRecord [size=37,
> chainSize=0, pos=null, type=PAGES_LIST_REMOVE_PAGE]]]
> 3. [D] DataPageInsertRecord [super=PageDeltaRecord [grpId=94416770,
> pageId=00010005, super=WALRecord [size=129, chainSize=0, pos=null,
> type=DATA_PAGE_INSERT_RECORD]]]
> 4. [F] PagesListAddPageRecord [dataPageId=00010005,
> super=PageDeltaRecord [grpId=94416770, pageId=00010008,
> super=WALRecord [size=37, chainSize=0, pos=null, type=PAGES_LIST_ADD_PAGE]]]
> 5. [F] DataPageSetFreeListPageRecord [freeListPage=281474976710664,
> super=PageDeltaRecord [grpId=94416770, pageId=00010005,
> super=WALRecord [size=37, chainSize=0, pos=null,
> type=DATA_PAGE_SET_FREE_LIST_PAGE]]]
> 6. [D] ReplaceRecord [io=DataLeafIO[ver=1], idx=0, super=PageDeltaRecord
> [grpId=94416770, pageId=00010004, super=WALRecord [size=47,
> chainSize=0, pos=null, type=BTREE_PAGE_REPLACE]]]
> 7. [F] DataPageRemoveRecord [itemId=0, super=PageDeltaRecord
> [grpId=94416770, pageId=00010005, super=WALRecord [size=30,
> chainSize=0, pos=null, type=DATA_PAGE_REMOVE_RECORD]]]
> 8. [F] PagesListRemovePageRecord [rmvdPageId=00010005,
> pageId=00010008, grpId=94416770, super=PageDeltaRecord
> [grpId=94416770, pageId=00010008, super=WALRecord [size=37,
> chainSize=0, pos=null, type=PAGES_LIST_REMOVE_PAGE]]]
> 9. [F] DataPageSetFreeListPageRecord [freeListPage=0, super=PageDeltaRecord
> [grpId=94416770, pageId=00010005, super=WALRecord [size=37,
> chainSize=0, pos=null, type=DATA_PAGE_SET_FREE_LIST_PAGE]]]
> 10. [F] PagesListAddPageRecord [dataPageId=00010005,
> super=PageDeltaRecord [grpId=94416770, pageId=00010006,
> super=WALRecord [size=37, chainSize=0, pos=null, type=PAGES_LIST_ADD_PAGE]]]
> 11. [F] DataPageSetFreeListPageRecord [freeListPage=281474976710662,
> super=PageDeltaRecord [grpId=94416770, pageId=00010005,
> super=WALRecord [size=37, chainSize=0, pos=null,
> type=DATA_PAGE_SET_FREE_LIST_PAGE]]]
> {code}
> If you sum all space required for operation (size in p.3 is shown incorrectly
> here), you will see that data update required ~300 bytes, so do free list
> update!
> *Proposed solution*
> 1) Optionally do not write free list updates to WAL
> 2) In case of node restart we start with empty free lists, so data inserts
> will have to allocate new pages
> 3) When old data page is read, add it to the free list
> 4) Start a background thread which will iterate over all old data pages and
> re-create the free list, so that eventually all data pages are tracked.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (IGNITE-6587) Ignite watchdog service
[ https://issues.apache.org/jira/browse/IGNITE-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16253348#comment-16253348 ] Sergey Puchnin commented on IGNITE-6587: Old thread list All TCP discovery threads All communication NIO threads (acceptor and workers) Exchange worker Striped pool threads Timeout Worker Checkpointer WAL archiver > Ignite watchdog service > --- > > Key: IGNITE-6587 > URL: https://issues.apache.org/jira/browse/IGNITE-6587 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.2 >Reporter: Alexey Goncharuk >Assignee: Dmitriy Pavlov > Labels: IEP-5 > Fix For: 2.4 > > Attachments: watchdog.sh > > > We need to come up with a 'watchdog service' to monitor for Ignite node local > health and kill the process under some critical conditions. > For example, if one of the mission-critical Ignite threads die, the Ignite > node must be stopped. > At the first glance, the list of critical threads is: > disco-event-worker > tcp-disco-sock-reader > tcp-disco-srvr > tcp-disco-msg-worker > tcp-comm-worker > grid-nio-worker-tcp-comm > exchange-worker > sys-stripe > grid-timeout-worker > db-checkpoint-thread > wal-file-archiver > ttl-cleanup-worker > nio-acceptor > The mechanism should support pluggable components so that self-check can be > extended via plugins. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6587) Ignite watchdog service
[ https://issues.apache.org/jira/browse/IGNITE-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6587: --- Description: We need to come up with a 'watchdog service' to monitor for Ignite node local health and kill the process under some critical conditions. For example, if one of the mission-critical Ignite threads die, the Ignite node must be stopped. At the first glance, the list of critical threads is: disco-event-worker tcp-disco-sock-reader tcp-disco-srvr tcp-disco-msg-worker tcp-comm-worker grid-nio-worker-tcp-comm exchange-worker sys-stripe grid-timeout-worker db-checkpoint-thread wal-file-archiver ttl-cleanup-worker nio-acceptor The mechanism should support pluggable components so that self-check can be extended via plugins. was: We need to come up with a 'watchdog service' to monitor for Ignite node local health and kill the process under some critical conditions. For example, if one of the mission-critical Ignite threads die, the Ignite node must be stopped. At the first glance, the list of critical threads is: All TCP discovery threads All communication NIO threads (acceptor and workers) Exchange worker Striped pool threads Timeout Worker Checkpointer WAL archiver The mechanism should support pluggable components so that self-check can be extended via plugins. > Ignite watchdog service > --- > > Key: IGNITE-6587 > URL: https://issues.apache.org/jira/browse/IGNITE-6587 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.2 >Reporter: Alexey Goncharuk >Assignee: Dmitriy Pavlov > Labels: IEP-5 > Fix For: 2.4 > > Attachments: watchdog.sh > > > We need to come up with a 'watchdog service' to monitor for Ignite node local > health and kill the process under some critical conditions. > For example, if one of the mission-critical Ignite threads die, the Ignite > node must be stopped. > At the first glance, the list of critical threads is: > disco-event-worker > tcp-disco-sock-reader > tcp-disco-srvr > tcp-disco-msg-worker > tcp-comm-worker > grid-nio-worker-tcp-comm > exchange-worker > sys-stripe > grid-timeout-worker > db-checkpoint-thread > wal-file-archiver > ttl-cleanup-worker > nio-acceptor > The mechanism should support pluggable components so that self-check can be > extended via plugins. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6909) Create an API for branching pointing
[ https://issues.apache.org/jira/browse/IGNITE-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6909: --- Description: To prevent offline split brain a branching pointing API should be provided. With BLT two cluster's parts might be activated separately need to prevent an equal join after that. was: To prevent offline split brain a branching pointing API should be provided. With BLT two cluster's parts might be activate separatly > Create an API for branching pointing > > > Key: IGNITE-6909 > URL: https://issues.apache.org/jira/browse/IGNITE-6909 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > To prevent offline split brain a branching pointing API should be provided. > With BLT two cluster's parts might be activated separately need to prevent an > equal join after that. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6909) Create an API for branching pointing
[ https://issues.apache.org/jira/browse/IGNITE-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6909: --- Description: To prevent offline split brain a branching pointing API should be provided. With BLT two cluster's parts might be activate separatly was:Unconditional branching pointing should be created if partial cluster activated > Create an API for branching pointing > > > Key: IGNITE-6909 > URL: https://issues.apache.org/jira/browse/IGNITE-6909 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > To prevent offline split brain a branching pointing API should be provided. > With BLT two cluster's parts might be activate separatly -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6909) Create an API for branching pointing
[ https://issues.apache.org/jira/browse/IGNITE-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6909: --- Summary: Create an API for branching pointing (was: Create an unconditional branching pointing after partial cluster activation) > Create an API for branching pointing > > > Key: IGNITE-6909 > URL: https://issues.apache.org/jira/browse/IGNITE-6909 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > Unconditional branching pointing should be created if partial cluster > activated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6909) Create an unconditional branching pointing after partial cluster activation
[ https://issues.apache.org/jira/browse/IGNITE-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6909: --- Summary: Create an unconditional branching pointing after partial cluster activation (was: Create an unconditional branching pointing after a snapshot restore) > Create an unconditional branching pointing after partial cluster activation > --- > > Key: IGNITE-6909 > URL: https://issues.apache.org/jira/browse/IGNITE-6909 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > Unconditional branching pointing should be created if partial cluster > activated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6909) Create an unconditional branching pointing after a snapshot restore
[ https://issues.apache.org/jira/browse/IGNITE-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6909: --- Description: Unconditional branching pointing should be created if partial cluster activated (was: Unconditional branching pointing should create after a restore cluster from a snapshot ) > Create an unconditional branching pointing after a snapshot restore > --- > > Key: IGNITE-6909 > URL: https://issues.apache.org/jira/browse/IGNITE-6909 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > Unconditional branching pointing should be created if partial cluster > activated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6910) Introduce a force join parameter to clear PDS after branching pointing
[ https://issues.apache.org/jira/browse/IGNITE-6910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6910: --- Description: Need to give an opportunity to clean a PDS on a node that is trying to join after branching pointing. It may be a "forceJoin" or a "forceClean" parameter (was: Unconditional branching pointing should create after a restore cluster from a snapshot ) > Introduce a force join parameter to clear PDS after branching pointing > -- > > Key: IGNITE-6910 > URL: https://issues.apache.org/jira/browse/IGNITE-6910 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > Need to give an opportunity to clean a PDS on a node that is trying to join > after branching pointing. It may be a "forceJoin" or a "forceClean" parameter -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6910) Introduce a force join parameter to clear PDS after branching pointing
[ https://issues.apache.org/jira/browse/IGNITE-6910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6910: --- Description: Need to give an opportunity to clean a PDS on a node that is trying to join after branching pointing. It may be a "force-join" or a "force-clean" parameter (was: Need to give an opportunity to clean a PDS on a node that is trying to join after branching pointing. It may be a "forceJoin" or a "forceClean" parameter) > Introduce a force join parameter to clear PDS after branching pointing > -- > > Key: IGNITE-6910 > URL: https://issues.apache.org/jira/browse/IGNITE-6910 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > Need to give an opportunity to clean a PDS on a node that is trying to join > after branching pointing. It may be a "force-join" or a "force-clean" > parameter -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6910) Introduce a force join parameter to clear PDS after branching pointing
Sergey Puchnin created IGNITE-6910: -- Summary: Introduce a force join parameter to clear PDS after branching pointing Key: IGNITE-6910 URL: https://issues.apache.org/jira/browse/IGNITE-6910 Project: Ignite Issue Type: New Feature Components: persistence Reporter: Sergey Puchnin Fix For: 2.4 Unconditional branching pointing should create after a restore cluster from a snapshot -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6909) Create an unconditional branching pointing after a snapshot restore
[ https://issues.apache.org/jira/browse/IGNITE-6909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6909: --- Description: Unconditional branching pointing should create after a restore cluster from a snapshot (was: During a cluster activation from not full BLT, we should inform a user about which nodes from BLT are missed ) > Create an unconditional branching pointing after a snapshot restore > --- > > Key: IGNITE-6909 > URL: https://issues.apache.org/jira/browse/IGNITE-6909 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > Unconditional branching pointing should create after a restore cluster from a > snapshot -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6909) Create an unconditional branching pointing after a snapshot restore
Sergey Puchnin created IGNITE-6909: -- Summary: Create an unconditional branching pointing after a snapshot restore Key: IGNITE-6909 URL: https://issues.apache.org/jira/browse/IGNITE-6909 Project: Ignite Issue Type: New Feature Components: persistence Reporter: Sergey Puchnin Fix For: 2.4 During a cluster activation from not full BLT, we should inform a user about which nodes from BLT are missed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6906) Inform user about missed nodes from BLT during a cluster activation
[ https://issues.apache.org/jira/browse/IGNITE-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6906: --- Description: During a cluster activation from not full BLT, we should inform a user about which nodes from BLT are missed (was: During a cluster activation, we should inform a user about which nodes from BLT are missed ) > Inform user about missed nodes from BLT during a cluster activation > --- > > Key: IGNITE-6906 > URL: https://issues.apache.org/jira/browse/IGNITE-6906 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > During a cluster activation from not full BLT, we should inform a user about > which nodes from BLT are missed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6908) Check PartitionLossPolicy during a cluster activation
[ https://issues.apache.org/jira/browse/IGNITE-6908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6908: --- Description: During a cluster activation from not full BLT, we should check if a partition lost policy is respected. (was: During a cluster activation, we should inform a user about which nodes from BLT are missed ) > Check PartitionLossPolicy during a cluster activation > -- > > Key: IGNITE-6908 > URL: https://issues.apache.org/jira/browse/IGNITE-6908 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > During a cluster activation from not full BLT, we should check if a partition > lost policy is respected. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6908) Check PartitionLossPolicy during a cluster activation
Sergey Puchnin created IGNITE-6908: -- Summary: Check PartitionLossPolicy during a cluster activation Key: IGNITE-6908 URL: https://issues.apache.org/jira/browse/IGNITE-6908 Project: Ignite Issue Type: New Feature Components: persistence Reporter: Sergey Puchnin Fix For: 2.4 During a cluster activation, we should inform a user about which nodes from BLT are missed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6906) Inform user about missed nodes from BLT during a cluster activation
[ https://issues.apache.org/jira/browse/IGNITE-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6906: --- Description: During a cluster activation we should inform a user about which nodes from BLT are missed (was: A BLT allows joining nodes by consistent ID. It's necessary to provide an information about consistent ID in log file) > Inform user about missed nodes from BLT during a cluster activation > --- > > Key: IGNITE-6906 > URL: https://issues.apache.org/jira/browse/IGNITE-6906 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > During a cluster activation we should inform a user about which nodes from > BLT are missed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6906) Inform user about missed nodes from BLT during a cluster activation
[ https://issues.apache.org/jira/browse/IGNITE-6906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6906: --- Description: During a cluster activation, we should inform a user about which nodes from BLT are missed (was: During a cluster activation we should inform a user about which nodes from BLT are missed ) > Inform user about missed nodes from BLT during a cluster activation > --- > > Key: IGNITE-6906 > URL: https://issues.apache.org/jira/browse/IGNITE-6906 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > During a cluster activation, we should inform a user about which nodes from > BLT are missed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6906) Inform user about missed nodes from BLT during a cluster activation
Sergey Puchnin created IGNITE-6906: -- Summary: Inform user about missed nodes from BLT during a cluster activation Key: IGNITE-6906 URL: https://issues.apache.org/jira/browse/IGNITE-6906 Project: Ignite Issue Type: New Feature Components: persistence Reporter: Sergey Puchnin Fix For: 2.4 A BLT allows joining nodes by consistent ID. It's necessary to provide an information about consistent ID in log file -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (IGNITE-6905) Print a consistent ID into a log file
[ https://issues.apache.org/jira/browse/IGNITE-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin reassigned IGNITE-6905: -- Assignee: (was: Sergey Chugunov) > Print a consistent ID into a log file > - > > Key: IGNITE-6905 > URL: https://issues.apache.org/jira/browse/IGNITE-6905 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin > Labels: IEP-4 > Fix For: 2.4 > > > A BLT allows joining nodes by consistent ID. It's necessary to provide an > information about consistent ID in log file -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6905) Print a consistent ID into a log file
[ https://issues.apache.org/jira/browse/IGNITE-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6905: --- Description: A BLT allows joining nodes by consistent ID. It's necessary to provide an information about consistent ID in log file (was: A BLT allows to join nodes by consistens ID) > Print a consistent ID into a log file > - > > Key: IGNITE-6905 > URL: https://issues.apache.org/jira/browse/IGNITE-6905 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin >Assignee: Sergey Chugunov > Labels: IEP-4 > Fix For: 2.4 > > > A BLT allows joining nodes by consistent ID. It's necessary to provide an > information about consistent ID in log file -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6905) Print a consistent ID into a log file
[ https://issues.apache.org/jira/browse/IGNITE-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6905: --- Description: A BLT allows to join nodes by consistens ID (was: h2. Use Cases * User starts up and activates brand-new grid. On activation BaselineTopology is created and persisted. * User starts up some (not all) nodes of old grid (BLT might be already presented on them) and activates new smaller grid. BLT is recreated with fewer number of nodes and replaces previous BLT in persistent storage. * User starts nodes of old grid (each node already has a BLT available locally). When all nodes presented in BLT are started grid is activated automatically.) > Print a consistent ID into a log file > - > > Key: IGNITE-6905 > URL: https://issues.apache.org/jira/browse/IGNITE-6905 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin >Assignee: Sergey Chugunov > Labels: IEP-4 > Fix For: 2.4 > > > A BLT allows to join nodes by consistens ID -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6905) Print a consistent ID into a log file
Sergey Puchnin created IGNITE-6905: -- Summary: Print a consistent ID into a log file Key: IGNITE-6905 URL: https://issues.apache.org/jira/browse/IGNITE-6905 Project: Ignite Issue Type: New Feature Components: persistence Reporter: Sergey Puchnin Assignee: Sergey Chugunov Fix For: 2.4 h2. Use Cases * User starts up and activates brand-new grid. On activation BaselineTopology is created and persisted. * User starts up some (not all) nodes of old grid (BLT might be already presented on them) and activates new smaller grid. BLT is recreated with fewer number of nodes and replaces previous BLT in persistent storage. * User starts nodes of old grid (each node already has a BLT available locally). When all nodes presented in BLT are started grid is activated automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6905) Print a consistent ID into a log file
[ https://issues.apache.org/jira/browse/IGNITE-6905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6905: --- Labels: IEP-4 (was: IEP-4 Phase-1) > Print a consistent ID into a log file > - > > Key: IGNITE-6905 > URL: https://issues.apache.org/jira/browse/IGNITE-6905 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Puchnin >Assignee: Sergey Chugunov > Labels: IEP-4 > Fix For: 2.4 > > > h2. Use Cases > * User starts up and activates brand-new grid. On activation BaselineTopology > is created and persisted. > * User starts up some (not all) nodes of old grid (BLT might be already > presented on them) and activates new smaller grid. > BLT is recreated with fewer number of nodes and replaces previous BLT in > persistent storage. > * User starts nodes of old grid (each node already has a BLT available > locally). When all nodes presented in BLT are started grid is activated > automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6843) SQL: optionally do not use WAL when executing CREATE INDEX
[
https://issues.apache.org/jira/browse/IGNITE-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Puchnin updated IGNITE-6843:
---
Description:
Inspired by Oracle {{NOLOGGING}} option [1].
When an index is being created through {{CREATE INDEX}} command, every single
index update is written to WAL. Let's introduce special mode where updates are
not written to WAL:
1) Index updates during an index_create operation are not written to WAL
2) When index is ready, force checkpoint and wait for it to happen
3) Purge index data if node crashed before checkpoint
Alternatively, we may even not trigger a checkpoint, hoping that that node will
not crash before the nearest checkpoint is finished. If node crashed during
this time window, the index should be marked as "invalid", and not used for
queries. Then the user should either re-create or rebuild it.
[1]
https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5010.htm#i2182589
was:
Inspired by Oracle {{NOLOGGING}} option [1].
When index is being created through {{CREATE INDEX}} command, every single
index update is written to WAL. Let's introduce special mode where updates are
not written to WAL:
1) Index updates during an index_create operation are not written to WAL
2) When index is ready, force checkpoint and wait for it to happen
3) Purge index data if node crashed before checkpoint
Alternatively, we may even not trigger a checkpoint, hoping that that node will
not crash before the nearest checkpoint is finished. If node crashed during
this time window, index should be marked as "invalid", and not used for
queries. Then user should either re-create or rebuild it.
[1]
https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5010.htm#i2182589
> SQL: optionally do not use WAL when executing CREATE INDEX
> --
>
> Key: IGNITE-6843
> URL: https://issues.apache.org/jira/browse/IGNITE-6843
> Project: Ignite
> Issue Type: Task
> Security Level: Public(Viewable by anyone)
> Components: sql
>Affects Versions: 2.3
>Reporter: Vladimir Ozerov
> Labels: iep-1, performance
> Fix For: 2.4
>
>
> Inspired by Oracle {{NOLOGGING}} option [1].
> When an index is being created through {{CREATE INDEX}} command, every single
> index update is written to WAL. Let's introduce special mode where updates
> are not written to WAL:
> 1) Index updates during an index_create operation are not written to WAL
> 2) When index is ready, force checkpoint and wait for it to happen
> 3) Purge index data if node crashed before checkpoint
> Alternatively, we may even not trigger a checkpoint, hoping that that node
> will not crash before the nearest checkpoint is finished. If node crashed
> during this time window, the index should be marked as "invalid", and not
> used for queries. Then the user should either re-create or rebuild it.
> [1]
> https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5010.htm#i2182589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (IGNITE-6843) SQL: optionally do not use WAL when executing CREATE INDEX
[
https://issues.apache.org/jira/browse/IGNITE-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Puchnin updated IGNITE-6843:
---
Description:
Inspired by Oracle {{NOLOGGING}} option [1].
When an index is being created through {{CREATE INDEX}} command, every single
index update is written to WAL. Let's introduce special mode where updates are
not written to WAL:
1) Index updates during an index_create operation are not written to WAL
2) When the index is ready, force a checkpoint and wait for it to happen
3) Purge index data if node crashed before checkpoint
Alternatively, we may even not trigger a checkpoint, hoping that that node will
not crash before the nearest checkpoint is finished. If node crashed during
this time window, the index should be marked as "invalid", and not used for
queries. Then the user should either re-create or rebuild it.
[1]
https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5010.htm#i2182589
was:
Inspired by Oracle {{NOLOGGING}} option [1].
When an index is being created through {{CREATE INDEX}} command, every single
index update is written to WAL. Let's introduce special mode where updates are
not written to WAL:
1) Index updates during an index_create operation are not written to WAL
2) When index is ready, force checkpoint and wait for it to happen
3) Purge index data if node crashed before checkpoint
Alternatively, we may even not trigger a checkpoint, hoping that that node will
not crash before the nearest checkpoint is finished. If node crashed during
this time window, the index should be marked as "invalid", and not used for
queries. Then the user should either re-create or rebuild it.
[1]
https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5010.htm#i2182589
> SQL: optionally do not use WAL when executing CREATE INDEX
> --
>
> Key: IGNITE-6843
> URL: https://issues.apache.org/jira/browse/IGNITE-6843
> Project: Ignite
> Issue Type: Task
> Security Level: Public(Viewable by anyone)
> Components: sql
>Affects Versions: 2.3
>Reporter: Vladimir Ozerov
> Labels: iep-1, performance
> Fix For: 2.4
>
>
> Inspired by Oracle {{NOLOGGING}} option [1].
> When an index is being created through {{CREATE INDEX}} command, every single
> index update is written to WAL. Let's introduce special mode where updates
> are not written to WAL:
> 1) Index updates during an index_create operation are not written to WAL
> 2) When the index is ready, force a checkpoint and wait for it to happen
> 3) Purge index data if node crashed before checkpoint
> Alternatively, we may even not trigger a checkpoint, hoping that that node
> will not crash before the nearest checkpoint is finished. If node crashed
> during this time window, the index should be marked as "invalid", and not
> used for queries. Then the user should either re-create or rebuild it.
> [1]
> https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5010.htm#i2182589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (IGNITE-6843) SQL: optionally do not use WAL when executing CREATE INDEX
[
https://issues.apache.org/jira/browse/IGNITE-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Puchnin updated IGNITE-6843:
---
Description:
Inspired by Oracle {{NOLOGGING}} option [1].
When index is being created through {{CREATE INDEX}} command, every single
index update is written to WAL. Let's introduce special mode where updates are
not written to WAL:
1) Index updates during an index_create operation are not written to WAL
2) When index is ready, force checkpoint and wait for it to happen
3) Purge index data if node crashed before checkpoint
Alternatively, we may even not trigger a checkpoint, hoping that that node will
not crash before the nearest checkpoint is finished. If node crashed during
this time window, index should be marked as "invalid", and not used for
queries. Then user should either re-create or rebuild it.
[1]
https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5010.htm#i2182589
was:
Inspired by Oracle {{NOLOGGING}} option [1].
When index is being created through {{CREATE INDEX}} command, every single
index update is written to WAL. Let's introduce special mode where updates are
not written to WAL:
1) Index updates are not written to WAL
2) When index is ready, force checkpoint and wait for it to happen
3) Purge index data if node crashed before checkpoint
Alternatively, we may even not trigger a checkpoint, hoping that that node will
not crash before the nearest checkpoint is finished. If node crashed during
this time window, index should be marked as "invalid", and not used for
queries. Then user should either re-create or rebuild it.
[1]
https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5010.htm#i2182589
> SQL: optionally do not use WAL when executing CREATE INDEX
> --
>
> Key: IGNITE-6843
> URL: https://issues.apache.org/jira/browse/IGNITE-6843
> Project: Ignite
> Issue Type: Task
> Security Level: Public(Viewable by anyone)
> Components: sql
>Affects Versions: 2.3
>Reporter: Vladimir Ozerov
> Labels: iep-1, performance
> Fix For: 2.4
>
>
> Inspired by Oracle {{NOLOGGING}} option [1].
> When index is being created through {{CREATE INDEX}} command, every single
> index update is written to WAL. Let's introduce special mode where updates
> are not written to WAL:
> 1) Index updates during an index_create operation are not written to WAL
> 2) When index is ready, force checkpoint and wait for it to happen
> 3) Purge index data if node crashed before checkpoint
> Alternatively, we may even not trigger a checkpoint, hoping that that node
> will not crash before the nearest checkpoint is finished. If node crashed
> during this time window, index should be marked as "invalid", and not used
> for queries. Then user should either re-create or rebuild it.
> [1]
> https://docs.oracle.com/cd/B19306_01/server.102/b14200/statements_5010.htm#i2182589
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (IGNITE-6779) Recreate clients caches after a node join BLT
Sergey Puchnin created IGNITE-6779: -- Summary: Recreate clients caches after a node join BLT Key: IGNITE-6779 URL: https://issues.apache.org/jira/browse/IGNITE-6779 Project: Ignite Issue Type: Bug Security Level: Public (Viewable by anyone) Components: persistence Reporter: Sergey Puchnin Priority: Critical If a node already has caches and is going to join to a BLT than current client caches should be destroyed and re-create server caches. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6694) Ability to turn BaselineTopology handling off
[ https://issues.apache.org/jira/browse/IGNITE-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6694: --- Labels: IEP-4 Phase-1 (was: IEP-4) > Ability to turn BaselineTopology handling off > - > > Key: IGNITE-6694 > URL: https://issues.apache.org/jira/browse/IGNITE-6694 > Project: Ignite > Issue Type: New Feature > Security Level: Public(Viewable by anyone) > Components: persistence >Reporter: Sergey Chugunov > Labels: IEP-4, Phase-1 > Fix For: 2.4 > > > To support cluster upgrade/downgrade scenarios when only part of nodes > supports BaselineTopology special setting to turn BLT handling off is needed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5853) Provide a way to determine which user attributes are used in affinity calculation
[ https://issues.apache.org/jira/browse/IGNITE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5853: --- Labels: IEP-4 Phase-1 (was: IEP-4) > Provide a way to determine which user attributes are used in affinity > calculation > - > > Key: IGNITE-5853 > URL: https://issues.apache.org/jira/browse/IGNITE-5853 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk > Labels: IEP-4, Phase-1 > Fix For: 2.4 > > > Since an affinity function may use user attributes to calculate affinity > distribution, we need to save these attributes to the metastore. However, > storing all the attributes is not very effective, so we need to have a way to > determine which attributes should be stored. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6308) BaselineTopology is created on first cluster activation
[ https://issues.apache.org/jira/browse/IGNITE-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6308: --- Labels: IEP-4 Phase-1 (was: IEP-4) > BaselineTopology is created on first cluster activation > --- > > Key: IGNITE-6308 > URL: https://issues.apache.org/jira/browse/IGNITE-6308 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Chugunov >Assignee: Sergey Chugunov > Labels: IEP-4, Phase-1 > Fix For: 2.3 > > > h2. Use Cases > * User starts up and activates brand-new grid. On activation BaselineTopology > is created and persisted. > * User starts up some (not all) nodes of old grid (BLT might be already > presented on them) and activates new smaller grid. > BLT is recreated with fewer number of nodes and replaces previous BLT in > persistent storage. > * User starts nodes of old grid (each node already has a BLT available > locally). When all nodes presented in BLT are started grid is activated > automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5852) Cache affinity should be calculated wrt baseline topology
[ https://issues.apache.org/jira/browse/IGNITE-5852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5852: --- Labels: IEP-4 Phase-1 (was: IEP-4) > Cache affinity should be calculated wrt baseline topology > - > > Key: IGNITE-5852 > URL: https://issues.apache.org/jira/browse/IGNITE-5852 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk >Assignee: Ilya Lantukh > Labels: IEP-4, Phase-1 > Fix For: 2.4 > > > After we have an API for baseline topology management, we need to provide a > way to operate when real topology differs from the baseline. To facilitate > this, we need to calculate the affinity on the set of nodes which is an > intersection of the baseline topology and current topology -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5850) Introduce an API for baseline topology for persistence-enabled clusters
[ https://issues.apache.org/jira/browse/IGNITE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5850: --- Labels: IEP-4 Phase-1 (was: IEP-4) > Introduce an API for baseline topology for persistence-enabled clusters > --- > > Key: IGNITE-5850 > URL: https://issues.apache.org/jira/browse/IGNITE-5850 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk > Labels: IEP-4, Phase-1 > Fix For: 2.4 > > > Since the persistence was introduced, we require that cluster be started in > an inactive mode and activation happens only manually. > We need to add a concept of baseline topology which is fixed on the first > cluster activation and may be changed later by a user. We need to develop a > consistent API facade for this purpose. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5851) Automatically activate cluster if baseline topology is reached
[ https://issues.apache.org/jira/browse/IGNITE-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5851: --- Labels: IEP-4 Phase-1 (was: IEP-4) > Automatically activate cluster if baseline topology is reached > -- > > Key: IGNITE-5851 > URL: https://issues.apache.org/jira/browse/IGNITE-5851 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk >Assignee: Sergey Chugunov > Labels: IEP-4, Phase-1 > Fix For: 2.4 > > > When we have an API for baseline topology management, we can automatically > activate the cluster once all the nodes from the baseline topology are > started. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6652) Gracefully deny joins of nodes with older version when baseline topology is set up
[ https://issues.apache.org/jira/browse/IGNITE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6652: --- Labels: (was: IEP-4) > Gracefully deny joins of nodes with older version when baseline topology is > set up > -- > > Key: IGNITE-6652 > URL: https://issues.apache.org/jira/browse/IGNITE-6652 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Sergey Puchnin > > As another point of validation need to avoid join old version nodes -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (IGNITE-5854) Validate baseline topology change history during node joins
[ https://issues.apache.org/jira/browse/IGNITE-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin resolved IGNITE-5854. Resolution: Duplicate > Validate baseline topology change history during node joins > --- > > Key: IGNITE-5854 > URL: https://issues.apache.org/jira/browse/IGNITE-5854 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk > Fix For: 2.4 > > > It is possible to have a cluster-split restarts when persistence enabled: > Start 1, 2, 3, 4, update data. Stop > Start 1, 2. Update data. Stop > Start 3, 4. Update data. Stop. > Start 1, 2, 3, 4. Stop > We need to make sure that the second start is not allowed. This can be done > by introducing baseline topology history and topology hash generation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (IGNITE-5854) Validate baseline topology change history during node joins
[ https://issues.apache.org/jira/browse/IGNITE-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin closed IGNITE-5854. -- > Validate baseline topology change history during node joins > --- > > Key: IGNITE-5854 > URL: https://issues.apache.org/jira/browse/IGNITE-5854 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk > Fix For: 2.4 > > > It is possible to have a cluster-split restarts when persistence enabled: > Start 1, 2, 3, 4, update data. Stop > Start 1, 2. Update data. Stop > Start 3, 4. Update data. Stop. > Start 1, 2, 3, 4. Stop > We need to make sure that the second start is not allowed. This can be done > by introducing baseline topology history and topology hash generation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5854) Validate baseline topology change history during node joins
[ https://issues.apache.org/jira/browse/IGNITE-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5854: --- Labels: (was: IEP-4) > Validate baseline topology change history during node joins > --- > > Key: IGNITE-5854 > URL: https://issues.apache.org/jira/browse/IGNITE-5854 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk > Fix For: 2.4 > > > It is possible to have a cluster-split restarts when persistence enabled: > Start 1, 2, 3, 4, update data. Stop > Start 1, 2. Update data. Stop > Start 3, 4. Update data. Stop. > Start 1, 2, 3, 4. Stop > We need to make sure that the second start is not allowed. This can be done > by introducing baseline topology history and topology hash generation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (IGNITE-6311) Initial BaselineTopology is defined with static configuration
[ https://issues.apache.org/jira/browse/IGNITE-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin resolved IGNITE-6311. Resolution: Duplicate > Initial BaselineTopology is defined with static configuration > - > > Key: IGNITE-6311 > URL: https://issues.apache.org/jira/browse/IGNITE-6311 > Project: Ignite > Issue Type: Sub-task > Components: persistence >Reporter: Sergey Chugunov > > h2. Use Case > User is allowed to define target topology with static configuration. > h2. Acceptance Criteria > * New public interface *ClusterActivator* is introduced with single method > boolean activate(Collection nodes); > * Out-of-the-box implementation allows user to specify list of IP addresses > and ports where grid nodes are expected to be started. > * New method *setClusterActivator* is added to configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (IGNITE-6281) Grid Automatic Activation
[ https://issues.apache.org/jira/browse/IGNITE-6281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin closed IGNITE-6281. -- > Grid Automatic Activation > - > > Key: IGNITE-6281 > URL: https://issues.apache.org/jira/browse/IGNITE-6281 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Sergey Chugunov > > h2. Motivation and use cases > Existing activation functionality allows user to perform start/stop > operations on persistent-enabled grids without excessive and unnecessary > rebalancing operations. > Improvement for automatic actiovation simplifies activation process freeing > user from manual grid activation after each restart (or building custom > environment management tools). > Main use case looks like this: user provides information about target grid > topology (number of nodes, IP addresses/ports, etc.); this information is > stored on the first start. Later based on this grid is activated > automatically after restart when that topology is reached. > More information is available in [the > discussion|http://apache-ignite-developers.2346864.n4.nabble.com/Cluster-auto-activation-design-proposal-tt20295.html] > in dev list. > h2. Notes > This JIRA is used as a root for more specific tickets covering particular > aspects of the functionality. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (IGNITE-6321) Nodes joining to active cluster remain in inactive state
[ https://issues.apache.org/jira/browse/IGNITE-6321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin resolved IGNITE-6321. Resolution: Duplicate > Nodes joining to active cluster remain in inactive state > > > Key: IGNITE-6321 > URL: https://issues.apache.org/jira/browse/IGNITE-6321 > Project: Ignite > Issue Type: Sub-task > Components: persistence >Reporter: Sergey Chugunov > > h2. Use Case > After starting up and activating grid user may start more brand-new nodes > that are enabled to join the grid. > However, these nodes must remain in the inactive state. Otherwise, partitions > may be rebalanced to them and on the next grid restart, an auto-activation > feature will lead to some data being unavailable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (IGNITE-6281) Grid Automatic Activation
[ https://issues.apache.org/jira/browse/IGNITE-6281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin resolved IGNITE-6281. Resolution: Duplicate > Grid Automatic Activation > - > > Key: IGNITE-6281 > URL: https://issues.apache.org/jira/browse/IGNITE-6281 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Sergey Chugunov > > h2. Motivation and use cases > Existing activation functionality allows user to perform start/stop > operations on persistent-enabled grids without excessive and unnecessary > rebalancing operations. > Improvement for automatic actiovation simplifies activation process freeing > user from manual grid activation after each restart (or building custom > environment management tools). > Main use case looks like this: user provides information about target grid > topology (number of nodes, IP addresses/ports, etc.); this information is > stored on the first start. Later based on this grid is activated > automatically after restart when that topology is reached. > More information is available in [the > discussion|http://apache-ignite-developers.2346864.n4.nabble.com/Cluster-auto-activation-design-proposal-tt20295.html] > in dev list. > h2. Notes > This JIRA is used as a root for more specific tickets covering particular > aspects of the functionality. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (IGNITE-6311) Initial BaselineTopology is defined with static configuration
[ https://issues.apache.org/jira/browse/IGNITE-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin closed IGNITE-6311. -- > Initial BaselineTopology is defined with static configuration > - > > Key: IGNITE-6311 > URL: https://issues.apache.org/jira/browse/IGNITE-6311 > Project: Ignite > Issue Type: Sub-task > Components: persistence >Reporter: Sergey Chugunov > > h2. Use Case > User is allowed to define target topology with static configuration. > h2. Acceptance Criteria > * New public interface *ClusterActivator* is introduced with single method > boolean activate(Collection nodes); > * Out-of-the-box implementation allows user to specify list of IP addresses > and ports where grid nodes are expected to be started. > * New method *setClusterActivator* is added to configuration. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (IGNITE-6321) Nodes joining to active cluster remain in inactive state
[ https://issues.apache.org/jira/browse/IGNITE-6321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin closed IGNITE-6321. -- > Nodes joining to active cluster remain in inactive state > > > Key: IGNITE-6321 > URL: https://issues.apache.org/jira/browse/IGNITE-6321 > Project: Ignite > Issue Type: Sub-task > Components: persistence >Reporter: Sergey Chugunov > > h2. Use Case > After starting up and activating grid user may start more brand-new nodes > that are enabled to join the grid. > However, these nodes must remain in the inactive state. Otherwise, partitions > may be rebalanced to them and on the next grid restart, an auto-activation > feature will lead to some data being unavailable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6308) BaselineTopology is created on first cluster activation
[ https://issues.apache.org/jira/browse/IGNITE-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6308: --- Issue Type: New Feature (was: Sub-task) Parent: (was: IGNITE-6281) > BaselineTopology is created on first cluster activation > --- > > Key: IGNITE-6308 > URL: https://issues.apache.org/jira/browse/IGNITE-6308 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Chugunov >Assignee: Sergey Chugunov > Labels: IEP-4 > Fix For: 2.3 > > > h2. Use Cases > * User starts up and activates brand-new grid. On activation BaselineTopology > is created and persisted. > * User starts up some (not all) nodes of old grid (BLT might be already > presented on them) and activates new smaller grid. > BLT is recreated with fewer number of nodes and replaces previous BLT in > persistent storage. > * User starts nodes of old grid (each node already has a BLT available > locally). When all nodes presented in BLT are started grid is activated > automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6308) BaselineTopology is created on first cluster activation
[ https://issues.apache.org/jira/browse/IGNITE-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6308: --- Labels: IEP-4 (was: ) > BaselineTopology is created on first cluster activation > --- > > Key: IGNITE-6308 > URL: https://issues.apache.org/jira/browse/IGNITE-6308 > Project: Ignite > Issue Type: New Feature > Components: persistence >Reporter: Sergey Chugunov >Assignee: Sergey Chugunov > Labels: IEP-4 > Fix For: 2.3 > > > h2. Use Cases > * User starts up and activates brand-new grid. On activation BaselineTopology > is created and persisted. > * User starts up some (not all) nodes of old grid (BLT might be already > presented on them) and activates new smaller grid. > BLT is recreated with fewer number of nodes and replaces previous BLT in > persistent storage. > * User starts nodes of old grid (each node already has a BLT available > locally). When all nodes presented in BLT are started grid is activated > automatically. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6281) Grid Automatic Activation
[ https://issues.apache.org/jira/browse/IGNITE-6281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6281: --- Labels: (was: IEP-4) > Grid Automatic Activation > - > > Key: IGNITE-6281 > URL: https://issues.apache.org/jira/browse/IGNITE-6281 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Sergey Chugunov > > h2. Motivation and use cases > Existing activation functionality allows user to perform start/stop > operations on persistent-enabled grids without excessive and unnecessary > rebalancing operations. > Improvement for automatic actiovation simplifies activation process freeing > user from manual grid activation after each restart (or building custom > environment management tools). > Main use case looks like this: user provides information about target grid > topology (number of nodes, IP addresses/ports, etc.); this information is > stored on the first start. Later based on this grid is activated > automatically after restart when that topology is reached. > More information is available in [the > discussion|http://apache-ignite-developers.2346864.n4.nabble.com/Cluster-auto-activation-design-proposal-tt20295.html] > in dev list. > h2. Notes > This JIRA is used as a root for more specific tickets covering particular > aspects of the functionality. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6321) Nodes joining to active cluster remain in inactive state
[ https://issues.apache.org/jira/browse/IGNITE-6321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6321: --- Description: h2. Use Case After starting up and activating grid user may start more brand-new nodes that are enabled to join the grid. However these nodes must remain in the inactive state. Otherwise, partitions may be rebalanced to them and on the next grid restart, an auto-activation feature will lead to some data being unavailable. was: h2. Use Case After starting up and activating grid user may start more brand-new nodes that are enabled to join the grid. However these nodes must remain in inactive state. Otherwise partitions may be rebalanced to them and on next grid restart auto activation feature will lead to some data being unavailable. > Nodes joining to active cluster remain in inactive state > > > Key: IGNITE-6321 > URL: https://issues.apache.org/jira/browse/IGNITE-6321 > Project: Ignite > Issue Type: Sub-task > Components: persistence >Reporter: Sergey Chugunov > > h2. Use Case > After starting up and activating grid user may start more brand-new nodes > that are enabled to join the grid. > However these nodes must remain in the inactive state. Otherwise, partitions > may be rebalanced to them and on the next grid restart, an auto-activation > feature will lead to some data being unavailable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6321) Nodes joining to active cluster remain in inactive state
[ https://issues.apache.org/jira/browse/IGNITE-6321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6321: --- Description: h2. Use Case After starting up and activating grid user may start more brand-new nodes that are enabled to join the grid. However, these nodes must remain in the inactive state. Otherwise, partitions may be rebalanced to them and on the next grid restart, an auto-activation feature will lead to some data being unavailable. was: h2. Use Case After starting up and activating grid user may start more brand-new nodes that are enabled to join the grid. However these nodes must remain in the inactive state. Otherwise, partitions may be rebalanced to them and on the next grid restart, an auto-activation feature will lead to some data being unavailable. > Nodes joining to active cluster remain in inactive state > > > Key: IGNITE-6321 > URL: https://issues.apache.org/jira/browse/IGNITE-6321 > Project: Ignite > Issue Type: Sub-task > Components: persistence >Reporter: Sergey Chugunov > > h2. Use Case > After starting up and activating grid user may start more brand-new nodes > that are enabled to join the grid. > However, these nodes must remain in the inactive state. Otherwise, partitions > may be rebalanced to them and on the next grid restart, an auto-activation > feature will lead to some data being unavailable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (IGNITE-6652) Gracefully deny joins of nodes with older version when baseline topology is set up
[ https://issues.apache.org/jira/browse/IGNITE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin resolved IGNITE-6652. Resolution: Duplicate > Gracefully deny joins of nodes with older version when baseline topology is > set up > -- > > Key: IGNITE-6652 > URL: https://issues.apache.org/jira/browse/IGNITE-6652 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Sergey Puchnin > Labels: IEP-4 > > As another point of validation need to avoid join old version nodes -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (IGNITE-6652) Gracefully deny joins of nodes with older version when baseline topology is set up
[ https://issues.apache.org/jira/browse/IGNITE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin closed IGNITE-6652. -- > Gracefully deny joins of nodes with older version when baseline topology is > set up > -- > > Key: IGNITE-6652 > URL: https://issues.apache.org/jira/browse/IGNITE-6652 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Sergey Puchnin > Labels: IEP-4 > > As another point of validation need to avoid join old version nodes -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (IGNITE-6652) Gracefully deny joins of nodes with older version when baseline topology is set up
[ https://issues.apache.org/jira/browse/IGNITE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16218831#comment-16218831 ] Sergey Puchnin commented on IGNITE-6652: Double for https://issues.apache.org/jira/browse/IGNITE-6310 > Gracefully deny joins of nodes with older version when baseline topology is > set up > -- > > Key: IGNITE-6652 > URL: https://issues.apache.org/jira/browse/IGNITE-6652 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Sergey Puchnin > Labels: IEP-4 > > As another point of validation need to avoid join old version nodes -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6679) Clean up some deprecated cache metrics
Sergey Puchnin created IGNITE-6679: -- Summary: Clean up some deprecated cache metrics Key: IGNITE-6679 URL: https://issues.apache.org/jira/browse/IGNITE-6679 Project: Ignite Issue Type: Improvement Security Level: Public (Viewable by anyone) Components: cache Reporter: Sergey Puchnin Priority: Trivial An old optimistic serializable mode implementation was removed in 2.0 but some cache metrics still present in CacheMetrics interface. Need to clean up and remove these metrics: *TxCommitQueueSize* *TxPrepareQueueSize* *TxStartVersionCountsSize* *TxDhtCommitQueueSize* *TxDhtPrepareQueueSize* *TxDhtStartVersionCountsSize* An algorithm for page eviction was changed and metric *DhtEvictQueueCurrentSize* should be removed also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6587) Ignite watchdog service
[ https://issues.apache.org/jira/browse/IGNITE-6587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6587: --- Labels: IEP-5 (was: ) > Ignite watchdog service > --- > > Key: IGNITE-6587 > URL: https://issues.apache.org/jira/browse/IGNITE-6587 > Project: Ignite > Issue Type: Improvement > Components: general >Affects Versions: 2.2 >Reporter: Alexey Goncharuk >Assignee: Dmitriy Pavlov > Labels: IEP-5 > Fix For: 2.4 > > > We need to come up with a 'watchdog service' to monitor for Ignite node local > health and kill the process under some critical conditions. > For example, if one of the mission-critical Ignite threads die, the Ignite > node must be stopped. > At the first glance, the list of critical threads is: > All TCP discovery threads > All communication NIO threads (acceptor and workers) > Exchange worker > Striped pool threads > Timeout Worker > Checkpointer > WAL archiver > The mechanism should support pluggable components so that self-check can be > extended via plugins. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Closed] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin closed IGNITE-6638. -- > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin > Fix For: 2.4 > > > For further improvement BLT it'd be great to add auto activation > functionality. It allow to reduce stuff intervention. > Next subtasks should be solved: > * To change the metasrore format. Choose a format to store a node list (whole > node list, only id+hash, etc) > * Add a BLT management to WebConsole > * Add a BLT management to CLI > * Save affinity function attributes to metastore > * Check equality configuration between metasore and XML. Reject to start a > cluster if different -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin resolved IGNITE-6638. Resolution: Duplicate looks like a duplicate for https://issues.apache.org/jira/browse/IGNITE-5851 > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin > Fix For: 2.4 > > > For further improvement BLT it'd be great to add auto activation > functionality. It allow to reduce stuff intervention. > Next subtasks should be solved: > * To change the metasrore format. Choose a format to store a node list (whole > node list, only id+hash, etc) > * Add a BLT management to WebConsole > * Add a BLT management to CLI > * Save affinity function attributes to metastore > * Check equality configuration between metasore and XML. Reject to start a > cluster if different -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin reassigned IGNITE-6638: -- Assignee: (was: Dmitriy Pavlov) > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin > Fix For: 2.4 > > > For further improvement BLT it'd be great to add auto activation > functionality. It allow to reduce stuff intervention. > Next subtasks should be solved: > * To change the metasrore format. Choose a format to store a node list (whole > node list, only id+hash, etc) > * Add a BLT management to WebConsole > * Add a BLT management to CLI > * Save affinity function attributes to metastore > * Check equality configuration between metasore and XML. Reject to start a > cluster if different -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6638: --- Labels: (was: IEP-4) > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin >Assignee: Dmitriy Pavlov > Fix For: 2.4 > > > For further improvement BLT it'd be great to add auto activation > functionality. It allow to reduce stuff intervention. > Next subtasks should be solved: > * To change the metasrore format. Choose a format to store a node list (whole > node list, only id+hash, etc) > * Add a BLT management to WebConsole > * Add a BLT management to CLI > * Save affinity function attributes to metastore > * Check equality configuration between metasore and XML. Reject to start a > cluster if different -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6653) Check equality configuration between metasore and XML
[ https://issues.apache.org/jira/browse/IGNITE-6653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6653: --- Description: To introduce another point of configuration validation between cluster (XML, java code) and metasore. Provide detailed information and decline to start cluster if different was:As another point of validation need to avoid join old version nodes > Check equality configuration between metasore and XML > - > > Key: IGNITE-6653 > URL: https://issues.apache.org/jira/browse/IGNITE-6653 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Sergey Puchnin > Labels: IEP-4 > > To introduce another point of configuration validation between cluster (XML, > java code) and metasore. > Provide detailed information and decline to start cluster if different -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6653) Check equality configuration between metasore and XML
Sergey Puchnin created IGNITE-6653: -- Summary: Check equality configuration between metasore and XML Key: IGNITE-6653 URL: https://issues.apache.org/jira/browse/IGNITE-6653 Project: Ignite Issue Type: Improvement Security Level: Public (Viewable by anyone) Affects Versions: 2.3 Reporter: Sergey Puchnin As another point of validation need to avoid join old version nodes -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6652) Gracefully deny joins of nodes with older version when baseline topology is set up
[ https://issues.apache.org/jira/browse/IGNITE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6652: --- Description: As another point of validation need to avoid join old version nodes (was: As an another point of validation ) > Gracefully deny joins of nodes with older version when baseline topology is > set up > -- > > Key: IGNITE-6652 > URL: https://issues.apache.org/jira/browse/IGNITE-6652 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Sergey Puchnin > Labels: IEP-4 > > As another point of validation need to avoid join old version nodes -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6652) Gracefully deny joins of nodes with older version when baseline topology is set up
[ https://issues.apache.org/jira/browse/IGNITE-6652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6652: --- Description: As an another point of validation (was: Baseline should save attributes for AF to metasore) > Gracefully deny joins of nodes with older version when baseline topology is > set up > -- > > Key: IGNITE-6652 > URL: https://issues.apache.org/jira/browse/IGNITE-6652 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Sergey Puchnin > Labels: IEP-4 > > As an another point of validation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6652) Gracefully deny joins of nodes with older version when baseline topology is set up
Sergey Puchnin created IGNITE-6652: -- Summary: Gracefully deny joins of nodes with older version when baseline topology is set up Key: IGNITE-6652 URL: https://issues.apache.org/jira/browse/IGNITE-6652 Project: Ignite Issue Type: Improvement Security Level: Public (Viewable by anyone) Affects Versions: 2.3 Reporter: Sergey Puchnin Baseline should save attributes for AF to metasore -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6651) Baseline includes attributes required for affinity function
[ https://issues.apache.org/jira/browse/IGNITE-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6651: --- Description: Baseline should save attributes for AF to metasore (was: We need to design and implement an effective baseline topology format for metastore so that metastore does not grow too fast.) > Baseline includes attributes required for affinity function > --- > > Key: IGNITE-6651 > URL: https://issues.apache.org/jira/browse/IGNITE-6651 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Sergey Puchnin > Labels: IEP-4 > > Baseline should save attributes for AF to metasore -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6651) Baseline includes only attributes required for affinity function
Sergey Puchnin created IGNITE-6651: -- Summary: Baseline includes only attributes required for affinity function Key: IGNITE-6651 URL: https://issues.apache.org/jira/browse/IGNITE-6651 Project: Ignite Issue Type: Improvement Security Level: Public (Viewable by anyone) Affects Versions: 2.3 Reporter: Sergey Puchnin We need to design and implement an effective baseline topology format for metastore so that metastore does not grow too fast. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6651) Baseline includes attributes required for affinity function
[ https://issues.apache.org/jira/browse/IGNITE-6651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6651: --- Summary: Baseline includes attributes required for affinity function (was: Baseline includes only attributes required for affinity function) > Baseline includes attributes required for affinity function > --- > > Key: IGNITE-6651 > URL: https://issues.apache.org/jira/browse/IGNITE-6651 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Sergey Puchnin > Labels: IEP-4 > > We need to design and implement an effective baseline topology format for > metastore so that metastore does not grow too fast. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5854) Validate baseline topology change history during node joins
[ https://issues.apache.org/jira/browse/IGNITE-5854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5854: --- Labels: IEP-4 (was: ) > Validate baseline topology change history during node joins > --- > > Key: IGNITE-5854 > URL: https://issues.apache.org/jira/browse/IGNITE-5854 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk > Labels: IEP-4 > Fix For: 2.4 > > > It is possible to have a cluster-split restarts when persistence enabled: > Start 1, 2, 3, 4, update data. Stop > Start 1, 2. Update data. Stop > Start 3, 4. Update data. Stop. > Start 1, 2, 3, 4. Stop > We need to make sure that the second start is not allowed. This can be done > by introducing baseline topology history and topology hash generation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5851) Automatically activate cluster if baseline topology is reached
[ https://issues.apache.org/jira/browse/IGNITE-5851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5851: --- Labels: IEP-4 (was: ) > Automatically activate cluster if baseline topology is reached > -- > > Key: IGNITE-5851 > URL: https://issues.apache.org/jira/browse/IGNITE-5851 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk > Labels: IEP-4 > Fix For: 2.4 > > > When we have an API for baseline topology management, we can automatically > activate the cluster once all the nodes from the baseline topology are > started. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5853) Provide a way to determine which user attributes are used in affinity calculation
[ https://issues.apache.org/jira/browse/IGNITE-5853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5853: --- Labels: IEP-4 (was: ) > Provide a way to determine which user attributes are used in affinity > calculation > - > > Key: IGNITE-5853 > URL: https://issues.apache.org/jira/browse/IGNITE-5853 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk > Labels: IEP-4 > Fix For: 2.4 > > > Since an affinity function may use user attributes to calculate affinity > distribution, we need to save these attributes to the metastore. However, > storing all the attributes is not very effective, so we need to have a way to > determine which attributes should be stored. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5852) Cache affinity should be calculated wrt baseline topology
[ https://issues.apache.org/jira/browse/IGNITE-5852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5852: --- Labels: IEP-4 (was: ) > Cache affinity should be calculated wrt baseline topology > - > > Key: IGNITE-5852 > URL: https://issues.apache.org/jira/browse/IGNITE-5852 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk >Assignee: Alexey Goncharuk > Labels: IEP-4 > Fix For: 2.4 > > > After we have an API for baseline topology management, we need to provide a > way to operate when real topology differs from the baseline. To facilitate > this, we need to calculate the affinity on the set of nodes which is an > intersection of the baseline topology and current topology -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-5850) Introduce an API for baseline topology for persistence-enabled clusters
[ https://issues.apache.org/jira/browse/IGNITE-5850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-5850: --- Labels: IEP-4 (was: ) > Introduce an API for baseline topology for persistence-enabled clusters > --- > > Key: IGNITE-5850 > URL: https://issues.apache.org/jira/browse/IGNITE-5850 > Project: Ignite > Issue Type: New Feature > Components: persistence >Affects Versions: 2.1 >Reporter: Alexey Goncharuk > Labels: IEP-4 > Fix For: 2.4 > > > Since the persistence was introduced, we require that cluster be started in > an inactive mode and activation happens only manually. > We need to add a concept of baseline topology which is fixed on the first > cluster activation and may be changed later by a user. We need to develop a > consistent API facade for this purpose. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6650) Introduce effective storage format for baseline topology
[ https://issues.apache.org/jira/browse/IGNITE-6650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6650: --- Labels: IEP-4 (was: ) > Introduce effective storage format for baseline topology > > > Key: IGNITE-6650 > URL: https://issues.apache.org/jira/browse/IGNITE-6650 > Project: Ignite > Issue Type: Improvement > Security Level: Public(Viewable by anyone) >Affects Versions: 2.3 >Reporter: Alexey Goncharuk > Labels: IEP-4 > > We need to design and implement an effective baseline topology format for > metastore so that metastore does not grow too fast. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6638: --- Labels: IEP-4 (was: ) > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin >Assignee: Dmitriy Pavlov > Labels: IEP-4 > Fix For: 2.4 > > > For further improvement BLT it'd be great to add auto activation > functionality. It allow to reduce stuff intervention. > Next subtasks should be solved: > * To change the metasrore format. Choose a format to store a node list (whole > node list, only id+hash, etc) > * Add a BLT management to WebConsole > * Add a BLT management to CLI > * Save affinity function attributes to metastore > * Check equality configuration between metasore and XML. Reject to start a > cluster if different -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6638: --- Description: For further improvement BLT it'd be great to add auto activation functionality. It allow to reduce stuff intervention. Next subtasks should be solved: * To change the metasrore format. Choose a format to store a node list (whole node list, only id+hash, etc) * Add a BLT management to WebConsole * Add a BLT management to CLI * Save affinity function attributes to metastore * Check equality configuration between metasore and XML. Reject to start a cluster if different was: For further improvement BLT it'd be great to add auto activation functionality. It allow to reduce stuff intervention. Next subtasks should be solved: * To change the metasrore format. Choose a format to store a node list (whole node list, only id+hash, etc) * Add a BLT management to WebConsole * Add a BLT management to CLI * Save affinity function attributes to metastore * Check equality configuration between metasore and XML. Reject to start a cluster if different * To support a rolling upgrade procedure for BLT cluster > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin >Assignee: Dmitriy Pavlov > Fix For: 2.4 > > > For further improvement BLT it'd be great to add auto activation > functionality. It allow to reduce stuff intervention. > Next subtasks should be solved: > * To change the metasrore format. Choose a format to store a node list (whole > node list, only id+hash, etc) > * Add a BLT management to WebConsole > * Add a BLT management to CLI > * Save affinity function attributes to metastore > * Check equality configuration between metasore and XML. Reject to start a > cluster if different -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6638: --- Description: For further improvement BLT it'd be great to add auto activation functionality. It allow to reduce stuff intervention. Next subtasks should be solved: * To change the metasrore format. Choose a format to store a node list (whole node list, only id+hash, etc) * Add a BLT management to WebConsole * Add a BLT management to CLI * Save affinity function attributes to metastore * Check equality configuration between metasore and XML. Reject to start a cluster if different * To support a rolling upgrade procedure for BLT cluster was:To support an > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin >Assignee: Dmitriy Pavlov > Fix For: 2.4 > > > For further improvement BLT it'd be great to add auto activation > functionality. It allow to reduce stuff intervention. > Next subtasks should be solved: > * To change the metasrore format. Choose a format to store a node list (whole > node list, only id+hash, etc) > * Add a BLT management to WebConsole > * Add a BLT management to CLI > * Save affinity function attributes to metastore > * Check equality configuration between metasore and XML. Reject to start a > cluster if different > * To support a rolling upgrade procedure for BLT cluster -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6638: --- Description: To support an (was: Currently, the WAL iterator can be obtained from the WAL manager when an Ignite node is up and running. However, it may be extremely useful to read WAL in an 'offline' mode, when a node is not started up. This may be required for crash analysis or to export committed data to some external systems. In the future we can make this even a public interface, however as a starting point, I would like to keep it in private package because moving to the public package will require Iterator and records to be public too. So, as a starting point, we need: * An object that will allow us to get WALIterator instances (probably, should be closeable) * A method on this object which will create an iterator by a file name or file names Using this object should not require an active Ignite instance running.) > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin >Assignee: Dmitriy Pavlov > Fix For: 2.4 > > > To support an -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6638: --- Fix Version/s: (was: 2.1) 2.4 > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin >Assignee: Dmitriy Pavlov > Fix For: 2.4 > > > Currently, the WAL iterator can be obtained from the WAL manager when an > Ignite node is up and running. > However, it may be extremely useful to read WAL in an 'offline' mode, when a > node is not started up. This may be required for crash analysis or to export > committed data to some external systems. > In the future we can make this even a public interface, however as a starting > point, I would like to keep it in private package because moving to the > public package will require Iterator and records to be public too. > So, as a starting point, we need: > * An object that will allow us to get WALIterator instances (probably, > should be closeable) > * A method on this object which will create an iterator by a file name or > file names > Using this object should not require an active Ignite instance running. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6638) Add an ability for auto activate BLT
Sergey Puchnin created IGNITE-6638: -- Summary: Add an ability for auto activate BLT Key: IGNITE-6638 URL: https://issues.apache.org/jira/browse/IGNITE-6638 Project: Ignite Issue Type: Improvement Components: persistence Affects Versions: 2.1 Reporter: Sergey Puchnin Assignee: Dmitriy Pavlov Fix For: 2.1 Currently, the WAL iterator can be obtained from the WAL manager when an Ignite node is up and running. However, it may be extremely useful to read WAL in an 'offline' mode, when a node is not started up. This may be required for crash analysis or to export committed data to some external systems. In the future we can make this even a public interface, however as a starting point, I would like to keep it in private package because moving to the public package will require Iterator and records to be public too. So, as a starting point, we need: * An object that will allow us to get WALIterator instances (probably, should be closeable) * A method on this object which will create an iterator by a file name or file names Using this object should not require an active Ignite instance running. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (IGNITE-6638) Add an ability for auto activate BLT
[ https://issues.apache.org/jira/browse/IGNITE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Puchnin updated IGNITE-6638: --- Affects Version/s: (was: 2.1) 2.2 > Add an ability for auto activate BLT > > > Key: IGNITE-6638 > URL: https://issues.apache.org/jira/browse/IGNITE-6638 > Project: Ignite > Issue Type: Improvement > Components: persistence >Affects Versions: 2.2 >Reporter: Sergey Puchnin >Assignee: Dmitriy Pavlov > Fix For: 2.4 > > > Currently, the WAL iterator can be obtained from the WAL manager when an > Ignite node is up and running. > However, it may be extremely useful to read WAL in an 'offline' mode, when a > node is not started up. This may be required for crash analysis or to export > committed data to some external systems. > In the future we can make this even a public interface, however as a starting > point, I would like to keep it in private package because moving to the > public package will require Iterator and records to be public too. > So, as a starting point, we need: > * An object that will allow us to get WALIterator instances (probably, > should be closeable) > * A method on this object which will create an iterator by a file name or > file names > Using this object should not require an active Ignite instance running. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
