[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14193777#comment-14193777 ] Rakesh R commented on BOOKKEEPER-634: - Thanks [~fpj] for the reminder. Unfortunately I couldn't find enough time to focus on this due to couple of reasons:(, I think I will be able to see it after Nov 7th. Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.1 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, 001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, 002-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-589) Release notes update for non-BC changes
[
https://issues.apache.org/jira/browse/BOOKKEEPER-589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159579#comment-14159579
]
Rakesh R commented on BOOKKEEPER-589:
-
bq.I would follow what Ivan Kelly suggested to keep release flow simple enough.
+1 for this release. In future its good to have {{Hadoop Flags}} as well as
{{Release Note}} custom fields.
Release notes update for non-BC changes
---
Key: BOOKKEEPER-589
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-589
Project: Bookkeeper
Issue Type: Sub-task
Components: Documentation
Reporter: Ivan Kelly
Assignee: Ivan Kelly
Priority: Blocker
Fix For: 4.3.0
4.3.0 client will not be able to connect to 4.2.0 servers, due to something
stupid in the protocol version handling. 4.2.0 and lower will blindly reject
any messages which have a protocolVersion higher than that it's current
version, even if the data format has not changed. The release notes should
reflect this. Also, think about how the protocolVersion is used some more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-789) Update README to reflect bookkeeper modules
[
https://issues.apache.org/jira/browse/BOOKKEEPER-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159581#comment-14159581
]
Rakesh R commented on BOOKKEEPER-789:
-
+1 lgtm
Update README to reflect bookkeeper modules
---
Key: BOOKKEEPER-789
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-789
Project: Bookkeeper
Issue Type: Task
Components: bookkeeper-stats
Reporter: Sijie Guo
Assignee: Sijie Guo
Fix For: 4.3.0
Attachments: BOOKKEEPER-789.diff
{quote}
BookKeeper project contains: section missing the following projects info.
- bookkeeper-stats (Stats API for bookkeeper)
- hedwig-client-jms (Hedwig client jms)
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-589) Release notes update for non-BC changes
[
https://issues.apache.org/jira/browse/BOOKKEEPER-589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159108#comment-14159108
]
Rakesh R commented on BOOKKEEPER-589:
-
I failed to see {{Incompatible change}} field in JIRA, we could update the same
to show the compatibility status. I just checked other hadoop projects there is
a custom field {{Hadoop Flags}} which has this checkbox (for example : ZK, HDFS
projects etc). Also, they have {{Release Note}} field. Truly, I didn't know
whether these will be automatically picked up and reflects in the release
notes, since others are using I feel these would be useful in projects. It
seems admin/infra has to provide these custom fields in the project. Is there
any request flow to the infra team to add extra fields into the project jira?
Release notes update for non-BC changes
---
Key: BOOKKEEPER-589
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-589
Project: Bookkeeper
Issue Type: Sub-task
Components: Documentation
Reporter: Ivan Kelly
Assignee: Ivan Kelly
Priority: Blocker
Fix For: 4.3.0
4.3.0 client will not be able to connect to 4.2.0 servers, due to something
stupid in the protocol version handling. 4.2.0 and lower will blindly reject
any messages which have a protocolVersion higher than that it's current
version, even if the data format has not changed. The release notes should
reflect this. Also, think about how the protocolVersion is used some more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14154466#comment-14154466 ] Rakesh R commented on BOOKKEEPER-634: - Attached a patch based on yesterday's proposal. I'll try to add more testcases. bq. I was expecting a rate limiting. Exactly rate limiting would be more efficient in this situation, permits will be distributed in a fixed rate. Let me try it once. I'm putting one corner case to understand more, say zk is slow and configured throttling(batchsize) 100. Now it will send 100 requests per sec without considering the responses from zk. In worst case, there may be many requests waiting for zk responses like, 100 + 100 + 100 etc. +New Algorithm considering both the cases+ 1) rateLimiter = RateLimiter.create(batchsize); ledgersToBeUpdated = ArrayList(batchsize); 2) iterate and get one ledger. Add it to ledgersToBeUpdated 3) if ledgersToBeUpdated.size() batchsize. If yes then goto 3.1, else goto 3.2 3.1) rateLimiter.acquire(). Go to 2 3.2) loop through ledgersToBeUpdated and do update. Waiting to get all the updation status. 3.3) check the status and update the counter. If errors exit with -1, else clear ledgersToBeUpdated and goto 3.4 3.4) check the limit reaches, if yes exit with 0 else goto 2. ratelimiter will guarantee fixed rate of execution and on the other side batches will guarantee the completion status. Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, 001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, 002-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14154484#comment-14154484 ] Rakesh R commented on BOOKKEEPER-634: - Adding a note - If the new algo doesn't fits well, like you mentioned will make it simple by using only batchsize and will improve this in future. Whats your opinion? Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, 001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, 002-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[
https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156087#comment-14156087
]
Rakesh R commented on BOOKKEEPER-634:
-
[~hustlmsp], [~iv...@yahoo-inc.com] thanks both of you, many points has comeup
in recent discussions. Now its my turn to prepare a design doc capturing all
these points.
bq.you just ignored my comment again, as I said that if user doesn't provide
[-limit], you should iterate over all the ledgers, not just set a value like
Integer.MAX_VALUE.
once again thank you Sijie for the comments. It was a good point and I had
accepted it, also included in my yesterday's patch
{{002-BOOKKEEPER-634-rename-bookieid-in-ledger.patch}}. But I think I need to
improve and make it more clear rather than trying to reuse the
#getOptionIntValue method by using constant Integer.MIN_VALUE represents this
condition. Also, now I could see there is a bug in the condition, anyway I'll
correct it in my next patch.
{code}
+final int limit = getOptionIntValue(cmdLine, limit,
Integer.MIN_VALUE);
+if (limit = 0 limit != Integer.MIN_VALUE) {
+return -1;
+}
{code}
{code}
+if (limit != Integer.MIN_VALUE updatedLedgerCount = limit) {
+break;
+}
{code}
Regards,
Rakesh
Provide admin tool to rename bookie identifier in ledger metadata
-
Key: BOOKKEEPER-634
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-client, bookkeeper-server
Affects Versions: 4.2.1
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.1
Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch,
001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch,
002-BOOKKEEPER-634-rename-bookieid-in-ledger.patch,
BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch,
BOOKKEEPER-634.patch
This JIRA to discuss about admin tool for changing the bookie's IP to
hostname.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[
https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152886#comment-14152886
]
Rakesh R commented on BOOKKEEPER-634:
-
Thank you for the patience. I'd like to do an attempt tonight and hope for a
better one.
I'm planning to incorporate the following changes:
# modify the command format like below, this is to make the command usage
similar to the 'updatecookie'
{code}
updateledger -bookieId hostname|ip [-batchsize N] [-limit N]
{code}
# print the number of ledgers renamed to know the progress.
# use iterator and do update by iterating the ledgers batch by batch(bandwidth
I'm calling as batchsize). Ledgers present in each batch will be executed
concurrently. After completing all the ledgers in the current batch, it will
iterate and collect items for the next batch. This cycle will continue. Default
value is 5. Also, to avoid overwhelm the system, user can define the total
number of ledgers to be considered. Say, if limit=100, it will exit when the
successfully renamed ledger count becomes 100. By default this will be
Integer.MAX. This is just a draft idea, when preparing a patch will get more
clarity.
Provide admin tool to rename bookie identifier in ledger metadata
-
Key: BOOKKEEPER-634
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-client, bookkeeper-server
Affects Versions: 4.2.1
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch,
001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch,
BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch,
BOOKKEEPER-634.patch
This JIRA to discuss about admin tool for changing the bookie's IP to
hostname.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-634: Attachment: 002-BOOKKEEPER-634-rename-bookieid-in-ledger.patch Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, 001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, 002-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151440#comment-14151440 ] Rakesh R commented on BOOKKEEPER-634: - bq.btw, I think we stuck on these two tickets for too long time. we need to move on and make a decision if we want to include these two tickets or not. +1 moving out. its a blocker at all. Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, 001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151499#comment-14151499 ] Rakesh R commented on BOOKKEEPER-634: - Ah! typo in my previous comment. +1 moving out. its not a blocker... Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, 001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151932#comment-14151932 ] Rakesh R commented on BOOKKEEPER-773: - Thank you [~iv...@yahoo-inc.com], [~hustlmsp] [~fpj] for finalizing the solution and reviews. Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch, 008-BOOKKEEPER-773-rename-bookieid.patch, 009-BOOKKEEPER-773-rename-bookieid.patch, 010-BOOKKEEPER-773-rename-bookieid.patch, 011-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14151166#comment-14151166 ] Rakesh R commented on BOOKKEEPER-634: - bq.why couldn't me just print how many ledgers already are renamed? Its good idea. bq.I didn't say the we need to rename in sequence. what I pointed out in my comments again and again, is that you should think of a real production system, you should limit the concurrency of ledger metadata access, to not overwhelm the system. My bad, since I've used BK only with HDFS I always thinking in perspective of HDFS, there the no: of ledgers are comparatively less. Going forward let me try to understand more usecases. How about making the number of concurrent operations configurable by adding a 'bandwidth' parameter(defaulting to 5 operations) - this will be maximum number of ledgers which can be renamed simultaneously. If 6th operation comes, then it will wait. Does this sounds good to you? If agrees I'll try preparing a patch. Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, 001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[
https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150777#comment-14150777
]
Rakesh R commented on BOOKKEEPER-773:
-
Attached new patch. Here 2.3 steps has been modified like:
{code}
2.3.1. check the new cookie matches all the directory cookies. If yes goto
2.3.2 else goto 2.4
2.3.2. Read newcookie from zk. If yes all is fine(since we able to read
oldcookie in 2.2, delete oldcookie) else goto 2.5
{code}
Provide admin tool to rename bookie identifier in Cookies
-
Key: BOOKKEEPER-773
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-server
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch,
002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch,
003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch,
004-BOOKKEEPER-773-rename-bookieid.patch,
005-BOOKKEEPER-773-rename-bookieid.patch,
006-BOOKKEEPER-773-rename-bookieid.patch,
007-BOOKKEEPER-773-rename-bookieid.patch,
008-BOOKKEEPER-773-rename-bookieid.patch,
009-BOOKKEEPER-773-rename-bookieid.patch,
010-BOOKKEEPER-773-rename-bookieid.patch,
011-BOOKKEEPER-773-rename-bookieid.patch
The idea of this JIRA to implement a mechanism to efficiently rename the
bookie identifier present in the Cookies. Cookie information will be present
in:
- ledger journal directories in each Bookie server
- cookies znode in ZooKeeper
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BOOKKEEPER-787) Modify Cookie by removing 'znodeVersion' state/field
Rakesh R created BOOKKEEPER-787: --- Summary: Modify Cookie by removing 'znodeVersion' state/field Key: BOOKKEEPER-787 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-787 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 The idea is to make VersionedCookie instead of maintaining 'znodeVersion' by Cookie. Please see the [discussion thread|https://issues.apache.org/jira/browse/BOOKKEEPER-773?focusedCommentId=14143263page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14143263] for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-787) Modify Cookie by removing 'znodeVersion' state/field
[
https://issues.apache.org/jira/browse/BOOKKEEPER-787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144599#comment-14144599
]
Rakesh R commented on BOOKKEEPER-787:
-
Thanks Ivan for the new idea. Let me try to implement this one.
Between I've one doubt. If we pass 'Version' for writing, internally I need to
cast it to 'ZkVersion' for setting the {{stat#getVersion()}}. Instead I'm
thinking to pass 'ZkVersion' like below.
Also, there is one more case. When writing freshly with {{Version.NEW}}, how
should it returns the new version value to the user. I should have either
returning the new version to the user or have a callback to the user.
Since he knows he is writing/read/delete to ZK I feel this would be fine ?
{code}
DEFAULT_VERSION = Integer.MAX;
ZkVersion version = new ZkVersion(DEFAULT_VERSION);
cookie.writeToZooKeeper(zk, conf, ZkVersion);
{code}
Modify Cookie by removing 'znodeVersion' state/field
Key: BOOKKEEPER-787
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-787
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-server
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
The idea is to make VersionedCookie instead of maintaining 'znodeVersion'
by Cookie.
Please see the [discussion
thread|https://issues.apache.org/jira/browse/BOOKKEEPER-773?focusedCommentId=14143263page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14143263]
for more details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-787) Modify Cookie by removing 'znodeVersion' state/field
[ https://issues.apache.org/jira/browse/BOOKKEEPER-787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144630#comment-14144630 ] Rakesh R commented on BOOKKEEPER-787: - Thanks [~iv...@yahoo-inc.com] for the quick reply. bq.Updating bookieId (using the version returned by readFromZookeeper) Considering that there will not be any usecase of cookie#writeToZooKeeperagain over a fresh/initial cookie. If it comes, it has to do, 1. Initial formatting(using Version.NEW). Creates cookie znode first time. 2. read versioned cookie 3. then write. Modify Cookie by removing 'znodeVersion' state/field Key: BOOKKEEPER-787 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-787 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 The idea is to make VersionedCookie instead of maintaining 'znodeVersion' by Cookie. Please see the [discussion thread|https://issues.apache.org/jira/browse/BOOKKEEPER-773?focusedCommentId=14143263page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14143263] for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-787) Modify Cookie by removing 'znodeVersion' state/field
[ https://issues.apache.org/jira/browse/BOOKKEEPER-787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-787: Attachment: BOOKKEEPER-787.patch Modify Cookie by removing 'znodeVersion' state/field Key: BOOKKEEPER-787 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-787 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-787.patch The idea is to make VersionedCookie instead of maintaining 'znodeVersion' by Cookie. Please see the [discussion thread|https://issues.apache.org/jira/browse/BOOKKEEPER-773?focusedCommentId=14143263page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14143263] for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144887#comment-14144887 ] Rakesh R commented on BOOKKEEPER-773: - Created BOOKKEEPER-787 for making Cookie immutable. Thanks [~iv...@yahoo-inc.com] and [~hustlmsp]. Sometime back there was an offline discussion with [~iv...@yahoo-inc.com] about cookieupdate algo. Since we are making the Cookie immutable, tried to avoid setting bookiehost to the oldcookie in zookeeper with few verification steps. Verification is to make sure that zookeeper and the filesystem is in sync. +New algo:+ 1. prerequisite : update conf useHostNameAsBookieID=true 2. Exec cmd : cookieupdate -bookieId hostname 2.1. check that conf has useHostNameAsBookieID == true. If no return. 2.2. read old cookie, generating a ip based cookie path. If fails all is fine(cookie is hostname based) just return. 2.3. Verification 2.3.1. check the old cookie matches all the directory cookies. If yes goto 2.4 else goto 2.3.2 2.3.2. check oldcookie == newcookie. If yes all is fine(since we able to read oldcookie in 2.2, delete oldcookie) else goto 2.5 2.4. update the old cookie, write to all directories (this will prevent startup if there's a crash half way) 2.5. write the updated cookie to the new zookeeper path 2.6. delete old cookie [~iv...@yahoo-inc.com] Please feel free to correct if I miss anything. Appreciate any comments. Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch, 008-BOOKKEEPER-773-rename-bookieid.patch, 009-BOOKKEEPER-773-rename-bookieid.patch, 010-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-787) Modify Cookie by removing 'znodeVersion' state/field
[ https://issues.apache.org/jira/browse/BOOKKEEPER-787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144888#comment-14144888 ] Rakesh R commented on BOOKKEEPER-787: - Thanks [~iv...@yahoo-inc.com] for the discussions. Please have a look at the patch. Modify Cookie by removing 'znodeVersion' state/field Key: BOOKKEEPER-787 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-787 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-787.patch The idea is to make VersionedCookie instead of maintaining 'znodeVersion' by Cookie. Please see the [discussion thread|https://issues.apache.org/jira/browse/BOOKKEEPER-773?focusedCommentId=14143263page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14143263] for more details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14145927#comment-14145927 ] Rakesh R commented on BOOKKEEPER-773: - *Diff* a) Added 2.3. Verification steps before performing any update operations. b) Removed setting the newbookiehost to the oldcookie, here considering the verification steps will make sure that zookeeper and the filesystem is in sync. Partial failure cases like, case-1) after 2.4 step assume cmd has crashed/failed(say zk is down) Ans: Bookie startup will fail as it sees newbookie in the directories. This will be a hint to the users case-2) after 2.5 step assume cmd has crashed/failed(say zk is down) Ans: Bookie will be able to start with only new configuration. If anyone tries to use old conf, startup will fail as it sees newbookie in the directories. Secondly, users can do cleanup by reexecuting the cmd by making zk up. Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch, 008-BOOKKEEPER-773-rename-bookieid.patch, 009-BOOKKEEPER-773-rename-bookieid.patch, 010-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142946#comment-14142946 ] Rakesh R commented on BOOKKEEPER-773: - bq.I don't know what is the difference between cookie.copy and newBuilder(cookie). They differs in their znodeVersion. #copy - have all the fields of this cookie . This cookie builder is used for setting the newBookieHost to the oldCookie. #newBuilder - have all the fields of this cookie except zNodeVersion, which will be DEFAULT_COOKIE_ZNODE_VERSION. This will be used to create a new cookie in ZK. Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch, 008-BOOKKEEPER-773-rename-bookieid.patch, 009-BOOKKEEPER-773-rename-bookieid.patch, 010-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 010-BOOKKEEPER-773-rename-bookieid.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch, 008-BOOKKEEPER-773-rename-bookieid.patch, 009-BOOKKEEPER-773-rename-bookieid.patch, 010-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142377#comment-14142377 ] Rakesh R commented on BOOKKEEPER-773: - Attached new patch with the following changes: - introduced #copy() method in cookie to avoid modifying this.znodeVersion externally - done changes to incorporate two more scenarios 1) UpdateCookieWithExistingBookieId 2) WhenBothIPaddressAndHostNameCookiesExists Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch, 008-BOOKKEEPER-773-rename-bookieid.patch, 009-BOOKKEEPER-773-rename-bookieid.patch, 010-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142923#comment-14142923 ] Rakesh R commented on BOOKKEEPER-773: - bq.2. I don't understand why you need a copy method. isn't the builder good enough? I don't like adding such method in an immutable object. Previously I tried an approach of #newBuilder(oldcookie) always return the cookie with existing zNodeVersion. And for the (2) case, I set explicitly the default zNodeVersion to the builder. Will this be OK to set the znodeVersion to the builder? (1) Build an oldcookie object with 'newBookiHost' value. Here it should retain the existing zNodeVersion for setting data to cookie znode. (2) Build new cookie object with 'newBookieHost' value. Here it should be with default zNodeVersion for creating a new cookie znode. Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch, 008-BOOKKEEPER-773-rename-bookieid.patch, 009-BOOKKEEPER-773-rename-bookieid.patch, 010-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 009-BOOKKEEPER-773-rename-bookieid.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch, 008-BOOKKEEPER-773-rename-bookieid.patch, 009-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[
https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14142200#comment-14142200
]
Rakesh R commented on BOOKKEEPER-773:
-
Thanks Ivan, Sijie. Attached new patch with the command format
{code}
updatecookie -bookieId hostname|ipaddress
{code}
bq.On a side note, the patch has a lot of ^M(\r) characters at the end of the
line
I modified eclipse configuration. Also done dos2unix for the new patch.
Provide admin tool to rename bookie identifier in Cookies
-
Key: BOOKKEEPER-773
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-server
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch,
002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch,
003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch,
004-BOOKKEEPER-773-rename-bookieid.patch,
005-BOOKKEEPER-773-rename-bookieid.patch,
006-BOOKKEEPER-773-rename-bookieid.patch,
007-BOOKKEEPER-773-rename-bookieid.patch,
008-BOOKKEEPER-773-rename-bookieid.patch,
009-BOOKKEEPER-773-rename-bookieid.patch
The idea of this JIRA to implement a mechanism to efficiently rename the
bookie identifier present in the Cookies. Cookie information will be present
in:
- ledger journal directories in each Bookie server
- cookies znode in ZooKeeper
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 007-BOOKKEEPER-773-rename-bookieid.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 008-BOOKKEEPER-773-rename-bookieid.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch, 007-BOOKKEEPER-773-rename-bookieid.patch, 008-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-784) BookKeeperCloseTest#testLedgerCheck is failing intermittently
[
https://issues.apache.org/jira/browse/BOOKKEEPER-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136852#comment-14136852
]
Rakesh R commented on BOOKKEEPER-784:
-
bq.This wouldn't change the fact that what we are asserting is
non-deterministic. As I said, if the call gets past the check for whether the
client is closed before the client is actually closed, and this is quite
possible, then it will be a different error code returned.
Its really tricky and would be tough to determine whether the call has reached
PerChannelBookieClient's outstanding entry lists. If it reaches this list,
during closure bkclient will error out
PerChannelBookieClient#errorOutOutstandingEntries().
BookKeeperCloseTest#testLedgerCheck is failing intermittently
-
Key: BOOKKEEPER-784
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-784
Project: Bookkeeper
Issue Type: Bug
Reporter: Rakesh R
Assignee: Rakesh R
Labels: test
Fix For: 4.3.0
*Stacktrace*
{code}
java.lang.AssertionError: Should have client closed exception expected:0
but was:-19
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at
org.apache.bookkeeper.client.BookKeeperCloseTest.testLedgerCheck(BookKeeperCloseTest.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
{code}
Link:
https://builds.apache.org/job/bookkeeper-trunk-precommit-build/722/testReport/junit/org.apache.bookkeeper.client/BookKeeperCloseTest/testLedgerCheck/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136861#comment-14136861 ] Rakesh R commented on BOOKKEEPER-782: - bq.what you suggested before should be ok. why you don't use it? Hi [~iv...@yahoo-inc.com], you had previously commented out to use [generateCookie method|https://issues.apache.org/jira/browse/BOOKKEEPER-782?focusedCommentId=14135287page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14135287]. Do you agree by directly using the builder outside. I also feel external entities can access this way and re-build it with new values. Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[
https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137166#comment-14137166
]
Rakesh R commented on BOOKKEEPER-782:
-
[~iv...@yahoo-inc.com] My bad, I didn't get your point. {{generateCookie}} is
static way of building the cookie. But now the case am thinking is to get a
builder from the cookie instance. How can I club both static and instance
together.
Use builder pattern for Cookie
--
Key: BOOKKEEPER-782
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-server
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch,
BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch
It would be good to use builder pattern for Cookie, rather than modifying the
fields in place.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (BOOKKEEPER-784) BookKeeperCloseTest#testLedgerCheck is failing intermittently
[
https://issues.apache.org/jira/browse/BOOKKEEPER-784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rakesh R reassigned BOOKKEEPER-784:
---
Assignee: (was: Rakesh R)
BookKeeperCloseTest#testLedgerCheck is failing intermittently
-
Key: BOOKKEEPER-784
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-784
Project: Bookkeeper
Issue Type: Bug
Reporter: Rakesh R
Labels: test
Fix For: 4.3.0
Attachments:
0001-BOOKKEEPER-784-BookKeeperCloseTest-testLedgerCheck-i.patch
*Stacktrace*
{code}
java.lang.AssertionError: Should have client closed exception expected:0
but was:-19
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at
org.apache.bookkeeper.client.BookKeeperCloseTest.testLedgerCheck(BookKeeperCloseTest.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
{code}
Link:
https://builds.apache.org/job/bookkeeper-trunk-precommit-build/722/testReport/junit/org.apache.bookkeeper.client/BookKeeperCloseTest/testLedgerCheck/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-784) BookKeeperCloseTest#testLedgerCheck is failing intermittently
[
https://issues.apache.org/jira/browse/BOOKKEEPER-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137193#comment-14137193
]
Rakesh R commented on BOOKKEEPER-784:
-
{{testBookKeeperAdmin}} also has similar pre close parts. Please modify this
too.
BookKeeperCloseTest#testLedgerCheck is failing intermittently
-
Key: BOOKKEEPER-784
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-784
Project: Bookkeeper
Issue Type: Bug
Reporter: Rakesh R
Labels: test
Fix For: 4.3.0
Attachments:
0001-BOOKKEEPER-784-BookKeeperCloseTest-testLedgerCheck-i.patch
*Stacktrace*
{code}
java.lang.AssertionError: Should have client closed exception expected:0
but was:-19
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at
org.apache.bookkeeper.client.BookKeeperCloseTest.testLedgerCheck(BookKeeperCloseTest.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
{code}
Link:
https://builds.apache.org/job/bookkeeper-trunk-precommit-build/722/testReport/junit/org.apache.bookkeeper.client/BookKeeperCloseTest/testLedgerCheck/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-782: Attachment: BOOKKEEPER-782.patch Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137213#comment-14137213 ] Rakesh R commented on BOOKKEEPER-782: - Awsome!. Thanks for the clear explanation. Uploaded a new patch including this. Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-782: Attachment: BOOKKEEPER-782.patch Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-783) Avoid running out of fds in MutlipleThreadReadTest
[
https://issues.apache.org/jira/browse/BOOKKEEPER-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137251#comment-14137251
]
Rakesh R commented on BOOKKEEPER-783:
-
There is one minor
[comment|https://issues.apache.org/jira/browse/BOOKKEEPER-783?focusedCommentId=14131236page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14131236]
which I added previously.
Avoid running out of fds in MutlipleThreadReadTest
--
Key: BOOKKEEPER-783
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-783
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-server
Reporter: Sijie Guo
Assignee: Sijie Guo
Labels: test
Fix For: 4.3.0
Attachments: BOOKKEEPER-783.patch
{code}
org.jboss.netty.channel.ChannelException: Failed to create a selector.
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:343)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.init(AbstractNioSelector.java:100)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.init(AbstractNioWorker.java:52)
at
org.jboss.netty.channel.socket.nio.NioWorker.init(NioWorker.java:45)
at
org.jboss.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:45)
at
org.jboss.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:28)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.newWorker(AbstractNioWorkerPool.java:143)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:81)
at
org.jboss.netty.channel.socket.nio.NioWorkerPool.init(NioWorkerPool.java:39)
at
org.jboss.netty.channel.socket.nio.NioWorkerPool.init(NioWorkerPool.java:33)
at
org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.init(NioClientSocketChannelFactory.java:151)
at
org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.init(NioClientSocketChannelFactory.java:116)
at org.apache.bookkeeper.client.BookKeeper.init(BookKeeper.java:204)
at
org.apache.bookkeeper.client.BookKeeperTestClient.init(BookKeeperTestClient.java:50)
at
org.apache.bookkeeper.test.MultipleThreadReadTest.createClients(MultipleThreadReadTest.java:73)
at
org.apache.bookkeeper.test.MultipleThreadReadTest.multiLedgerMultiThreadRead(MultipleThreadReadTest.java:282)
at
org.apache.bookkeeper.test.MultipleThreadReadTest.test1Ledger50ThreadsRead(MultipleThreadReadTest.java:326)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
Caused by: java.io.IOException: Too many open files
at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method)
at sun.nio.ch.EPollArrayWrapper.init(EPollArrayWrapper.java:69)
at sun.nio.ch.EPollSelectorImpl.init(EPollSelectorImpl.java:52)
at
sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18)
at java.nio.channels.Selector.open(Selector.java:209)
at
org.jboss.netty.channel.socket.nio.SelectorUtil.open(SelectorUtil.java:63)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:341)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[
https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135333#comment-14135333
]
Rakesh R commented on BOOKKEEPER-782:
-
FYI, BOOKKEEPER-773 also has similar requirement of updating the
bookieHostId(1). Inorder to do this again it needs to pass another argument
'bookieHostId'' or another overloaded method by taking BookieAddress. If agrees
will prepare one more patch.
(1)Below piece of code is taken from [BOOKKEEPER-773 JIRA patch number -
006-BOOKKEEPER-773-rename-bookieid.patch|https://issues.apache.org/jira/secure/attachment/12668023/006-BOOKKEEPER-773-rename-bookieid.patch].
This will be modified by using the new builder fashion.
{code}
+Cookie cookie = Cookie.readFromZooKeeper(zk, conf);
+cookie = cookie.setBookieHost(newBookieHost);
+cookie.writeToZooKeeper(zk, conf);
{code}
Use builder pattern for Cookie
--
Key: BOOKKEEPER-782
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-server
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch,
BOOKKEEPER-782.patch, BOOKKEEPER-782.patch
It would be good to use builder pattern for Cookie, rather than modifying the
fields in place.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-784) BookKeeperCloseTest#testLedgerCheck is failing intermittently
[
https://issues.apache.org/jira/browse/BOOKKEEPER-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14135369#comment-14135369
]
Rakesh R commented on BOOKKEEPER-784:
-
bq.the slowing of the bookie should avoid this, but thats relying on timers
Yeah, its relaying on timers. One common failure pattern I could see is,
immediately after restarting the bookie server its performing the next
operation, now its hitting Disconnected event and is returning different return
code. Do you think the following changes will help:
# Presently it is delaying the server by 5secs, will increase to a bigger
value(20 or 30secs). Also, we need to increase the read/add timeouts
respectively.
# After restarting will perform another operation to ensure that the existing
bkclient successfully reconnected with the restarted server(during this
operation will reduce the server delay and after this op will set back the
bigger value). After this pre-requisite step it can perform the actual testing.
Is there any better way to see the client reconnection completed?
BookKeeperCloseTest#testLedgerCheck is failing intermittently
-
Key: BOOKKEEPER-784
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-784
Project: Bookkeeper
Issue Type: Bug
Reporter: Rakesh R
Assignee: Rakesh R
Labels: test
Fix For: 4.3.0
*Stacktrace*
{code}
java.lang.AssertionError: Should have client closed exception expected:0
but was:-19
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at
org.apache.bookkeeper.client.BookKeeperCloseTest.testLedgerCheck(BookKeeperCloseTest.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
{code}
Link:
https://builds.apache.org/job/bookkeeper-trunk-precommit-build/722/testReport/junit/org.apache.bookkeeper.client/BookKeeperCloseTest/testLedgerCheck/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-782: Attachment: BOOKKEEPER-782.patch Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136000#comment-14136000 ] Rakesh R commented on BOOKKEEPER-782: - Attached another patch - used #generateCookie(String instanceId). - modified the constructors bq.for bookie id, you should just get back the builder somehow, set it on the builder, and build(). it's simpler since this is only exercised from the conversion tool, not from a bookie itself. I'm thinking to have an overloaded method #generateCookie(BookieSocketAddress addr). Does this sounds good to you? Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134127#comment-14134127 ] Rakesh R commented on BOOKKEEPER-782: - Thanks [~iv...@yahoo-inc.com]. Attached new patch addressing the comments. Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-782: Attachment: BOOKKEEPER-782.patch Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[
https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134961#comment-14134961
]
Rakesh R commented on BOOKKEEPER-782:
-
bq.as I commented before, we shouldn't have setInstanceId in Cookie, as Cookie
is an immutable object.
Actually #setInstanceId is not changing the state of the Cookie, instead it
creates a new cookie object with the given instanceId and return back the new
one. I think method name {{setInstanceId}} gives the impression of changing the
state of the object.
Is that OK, if I directly inline the logic inside Bookie.java like below?
{code}
String instanceId = getInstanceId(zk);
Cookie masterCookie = Cookie.generateCookie(conf);
if (null != instanceId) {
Cookie.Builder builder = Cookie.newBuilder(masterCookie);
builder.setInstanceId(instanceId);
masterCookie = builder.build();
}
{code}
Use builder pattern for Cookie
--
Key: BOOKKEEPER-782
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-server
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch,
BOOKKEEPER-782.patch, BOOKKEEPER-782.patch
It would be good to use builder pattern for Cookie, rather than modifying the
fields in place.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-784) BookKeeperCloseTest#testLedgerCheck is failing intermittently
[
https://issues.apache.org/jira/browse/BOOKKEEPER-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14133125#comment-14133125
]
Rakesh R commented on BOOKKEEPER-784:
-
After analyzing the below exception, there is one common case to be addressed
is - {{restartBookieSlow()}} can cause the bkclient reconnection. This can
generate other exceptions like {{BookieHandleNotAvailableException}} et. So I
think before going to verify the bkclient#close() behavior, it needs to ensure
that the existing bkclient(s) has reconnected with the restarted slow bookie
server.
[Build-729|https://builds.apache.org/job/bookkeeper-trunk-precommit-build/729/testReport/junit/org.apache.bookkeeper.client/BookKeeperCloseTest/testReadLastConfirmed/]
{code}
2014-09-13 12:45:28,563 - INFO - [New I/O worker
#1225:PerChannelBookieClient@631] - Disconnected from bookie channel [id:
0x50881721, /67.195.81.148:50307 : /67.195.81.148:15021]
2014-09-13 12:45:28,563 - INFO - [New I/O worker
#1227:PerChannelBookieClient@631] - Disconnected from bookie channel [id:
0xc2daf545, /67.195.81.148:53456 : /67.195.81.148:15019]
2014-09-13 12:45:28,564 - INFO - [New I/O worker
#1226:PerChannelBookieClient@631] - Disconnected from bookie channel [id:
0x37d0b22c, /67.195.81.148:55687 : /67.195.81.148:15020]
2014-09-13 12:45:28,564 - ERROR -
[BookKeeperClientWorker-3-0:ReadLastConfirmedOp@127] - While readLastConfirmed
ledger: 3 did not hear success responses from all quorums
{code}
BookKeeperCloseTest#testLedgerCheck is failing intermittently
-
Key: BOOKKEEPER-784
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-784
Project: Bookkeeper
Issue Type: Bug
Reporter: Rakesh R
Labels: test
*Stacktrace*
{code}
java.lang.AssertionError: Should have client closed exception expected:0
but was:-19
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at
org.apache.bookkeeper.client.BookKeeperCloseTest.testLedgerCheck(BookKeeperCloseTest.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
{code}
Link:
https://builds.apache.org/job/bookkeeper-trunk-precommit-build/722/testReport/junit/org.apache.bookkeeper.client/BookKeeperCloseTest/testLedgerCheck/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (BOOKKEEPER-784) BookKeeperCloseTest#testLedgerCheck is failing intermittently
[
https://issues.apache.org/jira/browse/BOOKKEEPER-784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rakesh R reassigned BOOKKEEPER-784:
---
Assignee: Rakesh R
BookKeeperCloseTest#testLedgerCheck is failing intermittently
-
Key: BOOKKEEPER-784
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-784
Project: Bookkeeper
Issue Type: Bug
Reporter: Rakesh R
Assignee: Rakesh R
Labels: test
*Stacktrace*
{code}
java.lang.AssertionError: Should have client closed exception expected:0
but was:-19
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at
org.apache.bookkeeper.client.BookKeeperCloseTest.testLedgerCheck(BookKeeperCloseTest.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
{code}
Link:
https://builds.apache.org/job/bookkeeper-trunk-precommit-build/722/testReport/junit/org.apache.bookkeeper.client/BookKeeperCloseTest/testLedgerCheck/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-782: Attachment: BOOKKEEPER-782.patch Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132566#comment-14132566 ] Rakesh R commented on BOOKKEEPER-782: - Attached new patch, here renamed 'CookieBuilder' to 'Builder' Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-782: Attachment: BOOKKEEPER-782.patch Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131160#comment-14131160 ] Rakesh R commented on BOOKKEEPER-782: - Thanks [~hustlmsp]. Please review. Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch, BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-783) Avoid running out of fds in MutlipleThreadReadTest
[
https://issues.apache.org/jira/browse/BOOKKEEPER-783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131236#comment-14131236
]
Rakesh R commented on BOOKKEEPER-783:
-
Could you rename the test method as now it has only 20 threads. Apart from this
+1
{code}
test1Ledger50ThreadsRead
{code}
Avoid running out of fds in MutlipleThreadReadTest
--
Key: BOOKKEEPER-783
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-783
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-server
Reporter: Sijie Guo
Assignee: Sijie Guo
Labels: test
Fix For: 4.3.0
Attachments: BOOKKEEPER-783.patch
{code}
org.jboss.netty.channel.ChannelException: Failed to create a selector.
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:343)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.init(AbstractNioSelector.java:100)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorker.init(AbstractNioWorker.java:52)
at
org.jboss.netty.channel.socket.nio.NioWorker.init(NioWorker.java:45)
at
org.jboss.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:45)
at
org.jboss.netty.channel.socket.nio.NioWorkerPool.createWorker(NioWorkerPool.java:28)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.newWorker(AbstractNioWorkerPool.java:143)
at
org.jboss.netty.channel.socket.nio.AbstractNioWorkerPool.init(AbstractNioWorkerPool.java:81)
at
org.jboss.netty.channel.socket.nio.NioWorkerPool.init(NioWorkerPool.java:39)
at
org.jboss.netty.channel.socket.nio.NioWorkerPool.init(NioWorkerPool.java:33)
at
org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.init(NioClientSocketChannelFactory.java:151)
at
org.jboss.netty.channel.socket.nio.NioClientSocketChannelFactory.init(NioClientSocketChannelFactory.java:116)
at org.apache.bookkeeper.client.BookKeeper.init(BookKeeper.java:204)
at
org.apache.bookkeeper.client.BookKeeperTestClient.init(BookKeeperTestClient.java:50)
at
org.apache.bookkeeper.test.MultipleThreadReadTest.createClients(MultipleThreadReadTest.java:73)
at
org.apache.bookkeeper.test.MultipleThreadReadTest.multiLedgerMultiThreadRead(MultipleThreadReadTest.java:282)
at
org.apache.bookkeeper.test.MultipleThreadReadTest.test1Ledger50ThreadsRead(MultipleThreadReadTest.java:326)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
Caused by: java.io.IOException: Too many open files
at sun.nio.ch.EPollArrayWrapper.epollCreate(Native Method)
at sun.nio.ch.EPollArrayWrapper.init(EPollArrayWrapper.java:69)
at sun.nio.ch.EPollSelectorImpl.init(EPollSelectorImpl.java:52)
at
sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:18)
at java.nio.channels.Selector.open(Selector.java:209)
at
org.jboss.netty.channel.socket.nio.SelectorUtil.open(SelectorUtil.java:63)
at
org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector(AbstractNioSelector.java:341)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (BOOKKEEPER-784) BookKeeperCloseTest#testLedgerCheck is failing intermittently
Rakesh R created BOOKKEEPER-784:
---
Summary: BookKeeperCloseTest#testLedgerCheck is failing
intermittently
Key: BOOKKEEPER-784
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-784
Project: Bookkeeper
Issue Type: Bug
Reporter: Rakesh R
*Stacktrace*
{code}
java.lang.AssertionError: Should have client closed exception expected:0 but
was:-19
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at
org.apache.bookkeeper.client.BookKeeperCloseTest.testLedgerCheck(BookKeeperCloseTest.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
{code}
Link:
https://builds.apache.org/job/bookkeeper-trunk-precommit-build/722/testReport/junit/org.apache.bookkeeper.client/BookKeeperCloseTest/testLedgerCheck/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-784) BookKeeperCloseTest#testLedgerCheck is failing intermittently
[
https://issues.apache.org/jira/browse/BOOKKEEPER-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131353#comment-14131353
]
Rakesh R commented on BOOKKEEPER-784:
-
I ran locally several times without any issues.
Any chances of spurious wakeup?. Since {{final AtomicInteger preRc = new
AtomicInteger(BKException.Code.OK);}} has initialised with OK, couldn't
differentiate the actual operation completion or any other env issues.
BookKeeperCloseTest#testLedgerCheck is failing intermittently
-
Key: BOOKKEEPER-784
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-784
Project: Bookkeeper
Issue Type: Bug
Reporter: Rakesh R
Labels: test
*Stacktrace*
{code}
java.lang.AssertionError: Should have client closed exception expected:0
but was:-19
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at
org.apache.bookkeeper.client.BookKeeperCloseTest.testLedgerCheck(BookKeeperCloseTest.java:501)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
{code}
Link:
https://builds.apache.org/job/bookkeeper-trunk-precommit-build/722/testReport/junit/org.apache.bookkeeper.client/BookKeeperCloseTest/testLedgerCheck/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129694#comment-14129694 ] Rakesh R commented on BOOKKEEPER-773: - bq. This tool should focus on real production use case, not just print fancy information. The idea of printing progress details is to help users if there are many ledgers. Otw it would be hard to determine the completion time. bq.This tool should focus on real production use case, not just print fancy information. The current implementation will definitely have huge side effects on real production traffic, if you don't control the number of requests issued to zookeeper. so -1 on renameBookieIdInLedger I agree with you. I'd like to retain the progress information. How about adding a 'bandwidth' parameter(by default 10 operations) - this will be maximum number of ledgers which can be renamed concurrently. If 11th operation comes it will wait. bq. it is much clear and easier for reviewing and get things checked in faster. Yeah I will split the patches and do cleanup. Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129711#comment-14129711 ] Rakesh R commented on BOOKKEEPER-773: - bq.you pulled all ledgers into a list, which doesn't work if your cluster have a lot of ledgers. Actually I pulled all the ledgers to find the total number of ledgers. Is there any single API to know the total number of ledgers, otw I've to first iterate over all the ledgers and get the total count. Then secondly will get the iterator again and then iterate over all the ledgers asynchronously(as I mentioned earlier will have bandwidth in place to control the concurrency). If I understood correctly you are suggesting to go with one-by-one ledger rename (sequential execution). FYI - Initially I done the same way, but [~iv...@yahoo-inc.com] has given a [comment | https://issues.apache.org/jira/browse/BOOKKEEPER-634?focusedCommentId=14010999page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14010999] to make it concurrent and I also felt it would be useful one. bq. one is using your current solution, while the other one is using ledger iterator. so people could choose what to use. although I don't suggest to have two implementation on this, it would make code maintenance become hard. I also prefer to implement one good solution. Both are not required. Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129730#comment-14129730 ] Rakesh R commented on BOOKKEEPER-773: - bq. otw I've to first iterate over all the ledgers and get the total count. Adding one more point - due to concurrent ledger creation/deletion by other clients, there are chances of mismatching the total count when iterating over the ledger renaming. Since this is not for validation purpose, it can just skip if the pending count goes to negative values. Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (BOOKKEEPER-782) Use builder pattern for Cookie
Rakesh R created BOOKKEEPER-782: --- Summary: Use builder pattern for Cookie Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Improvement Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-782: Issue Type: Sub-task (was: Improvement) Parent: BOOKKEEPER-639 Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-782: Attachment: BOOKKEEPER-782.patch Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-782) Use builder pattern for Cookie
[ https://issues.apache.org/jira/browse/BOOKKEEPER-782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129852#comment-14129852 ] Rakesh R commented on BOOKKEEPER-782: - +Note:+ Cookie#setInstanceId() will return a new cookie object with the new instanceid value. Presently cookie is not exposing internals to the outside through getters(), I just wanted to continue the same behavior. Use builder pattern for Cookie -- Key: BOOKKEEPER-782 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-782 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: BOOKKEEPER-782.patch It would be good to use builder pattern for Cookie, rather than modifying the fields in place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 006-BOOKKEEPER-773-rename-bookieid.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129898#comment-14129898 ] Rakesh R commented on BOOKKEEPER-773: - Thanks [~hustlmsp] for your time and quick replies. bq.in general, all class variables in Cookie should be final. please use a Builder pattern to generate a new Cookie object when modifying it. DONT modify the field in-place, which would usually introduce bugs. I've raised separate JIRA BOOKKEEPER-782 to modify Cookie.java. Kindly review it. Thanks! bq.it is much clear and easier for reviewing and get things checked in faster. Attached a patch which only talks about rename cookie. This patch has to be applied on top of BOOKKEEPER-782 Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch, 006-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-775) Improve MultipleThreadReadTest to reduce flakiness
[ https://issues.apache.org/jira/browse/BOOKKEEPER-775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129942#comment-14129942 ] Rakesh R commented on BOOKKEEPER-775: - [Build-718 MultipleThreadReadTest#test1Ledger50ThreadsRead|https://builds.apache.org/job/bookkeeper-trunk-precommit-build/718/testReport/org.apache.bookkeeper.test/MultipleThreadReadTest/test1Ledger50ThreadsRead] has failures. Is that related to the flakiness discussion here ? Improve MultipleThreadReadTest to reduce flakiness -- Key: BOOKKEEPER-775 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-775 Project: Bookkeeper Issue Type: Test Components: bookkeeper-server Reporter: Sijie Guo Assignee: Sijie Guo Labels: test Fix For: 4.3.0 Attachments: BOOKKEEPER-775.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 005-BOOKKEEPER-773-rename-bookieid.rar Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.rar The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: (was: 005-BOOKKEEPER-773-rename-bookieid.rar) Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 005-BOOKKEEPER-773-rename-bookieid.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128244#comment-14128244 ] Rakesh R commented on BOOKKEEPER-773: - bq. If it is a blocker, then we should hold the release until this is in. No, its not a blocker bq.If this isn't a blocker and isn't ready for 4.3.0, why not push it to 4.4.0? As we know, the feature is already checked in to 4.3.0. New bk installations can make use of this. This tool is needed to update the cookie/ledgers and is required for the existing installations. IMHO if we can reach to an agreement will try push this in 4.3.0. Since it is not a new feature we can consider this to 4.3.1? I've uploaded new patch addressing Sijie's comments. Please have a look. Thanks! Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch, 005-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[
https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14128246#comment-14128246
]
Rakesh R commented on BOOKKEEPER-634:
-
bq.The way that you get whole ledgers into a set and then issues all reads at
the same time is definitely terrible. You should use the LedgerIterator and do
it iterator by iterator, and you have to bind the outstanding metadata reads
and updates to not overwhelm metadata store (zookeeper).
The reason for getting all the ledgers to a set and processing is to print the
execution progress
{code}
BookKeeperAdmin.java
if (TimeUnit.SECONDS.convert(System.nanoTime() - lastReport,
TimeUnit.NANOSECONDS) 5) {
System.out.println(Total ledgers + ledgers.size() + ,
renamed + completedLedgers.size()
+ pending + (ledgers.size() -
completedLedgers.size()));
lastReport = System.nanoTime();
}
{code}
Latest patch is attached to BOOKKEEPER-773.
Provide admin tool to rename bookie identifier in ledger metadata
-
Key: BOOKKEEPER-634
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-client, bookkeeper-server
Affects Versions: 4.2.1
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch,
001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch,
BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch,
BOOKKEEPER-634.patch
This JIRA to discuss about admin tool for changing the bookie's IP to
hostname.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 004-BOOKKEEPER-773-rename-bookieid.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 004-BOOKKEEPER-773-rename-bookieid.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124404#comment-14124404 ] Rakesh R commented on BOOKKEEPER-634: - Thanks for the reminder [~hustlmsp]. As per the discussions in BOOKKEEPER-773, instead of making this logic as a separate command will make this command as an argument in bkrename -ledger. I'll spend sometime tonight and will try best. Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, 001-BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124410#comment-14124410 ] Rakesh R commented on BOOKKEEPER-773: - If I understood your point correctly, should have a parsing logic within the command instead of using 'org.apache.commons.cli.Options. But how to implement #getOptions() and #getUsage() for this command as this have two separate set of commands for -ledger and -cookie, isn't it? Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-778) Flake in TestTryReadLastConfirmed
[ https://issues.apache.org/jira/browse/BOOKKEEPER-778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-778: Labels: tests (was: ) Flake in TestTryReadLastConfirmed - Key: BOOKKEEPER-778 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-778 Project: Bookkeeper Issue Type: Bug Reporter: Ivan Kelly Assignee: Ivan Kelly Labels: tests Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-778-Flake-in-TestTryReadLastConfirmed.patch, TEST-org.apache.bookkeeper.client.TestTryReadLastConfirmed.xml.fail1, TEST-org.apache.bookkeeper.client.TestTryReadLastConfirmed.xml.fail2, TEST-org.apache.bookkeeper.client.TestTryReadLastConfirmed.xml.fail3 Logs attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (BOOKKEEPER-778) Flake in TestTryReadLastConfirmed
[ https://issues.apache.org/jira/browse/BOOKKEEPER-778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-778: Component/s: bookkeeper-client Flake in TestTryReadLastConfirmed - Key: BOOKKEEPER-778 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-778 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-client Reporter: Ivan Kelly Assignee: Ivan Kelly Labels: tests Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-778-Flake-in-TestTryReadLastConfirmed.patch, TEST-org.apache.bookkeeper.client.TestTryReadLastConfirmed.xml.fail1, TEST-org.apache.bookkeeper.client.TestTryReadLastConfirmed.xml.fail2, TEST-org.apache.bookkeeper.client.TestTryReadLastConfirmed.xml.fail3 Logs attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-736) Stats for AutoRecovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14124413#comment-14124413 ] Rakesh R commented on BOOKKEEPER-736: - Thank you [~hustlmsp]. Nice work;-) Stats for AutoRecovery -- Key: BOOKKEEPER-736 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-736 Project: Bookkeeper Issue Type: New Feature Components: bookkeeper-auto-recovery Reporter: Rakesh R Assignee: Sijie Guo Fix For: 4.3.0 Attachments: BOOKKEEPER-736.patch, BOOKKEEPER-736.v2.patch, BOOKKEEPER-736.v3.patch, BOOKKEEPER-736.v4.patch, Prototype-JMX-beans-for-standalone-autorecovery-process.png Idea of this JIRA is to provide jmx interfaces to get the statistics of the auto recovery activities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119941#comment-14119941 ] Rakesh R commented on BOOKKEEPER-773: - I failed to see a way to define '-ledger' and '-cookie' options through org.apache.commons.cli.Options. These are two different options with their own mandatory arguments. Also, atleast one is mandatory for 'bkrename' command. Is there any way to achieve this ? Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-776) Flaky test BookieRecoveryTest
[ https://issues.apache.org/jira/browse/BOOKKEEPER-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119957#comment-14119957 ] Rakesh R commented on BOOKKEEPER-776: - Adding one thought: its good to check any compatibility issues between 4.x.x and 3.x.x. [netty backward compatiblity with older library|http://stackoverflow.com/questions/21762694/netty-backward-compatiblity-with-older-library] Flaky test BookieRecoveryTest - Key: BOOKKEEPER-776 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-776 Project: Bookkeeper Issue Type: Test Components: bookkeeper-client Reporter: Sijie Guo Assignee: Ivan Kelly Labels: test Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-776-Flaky-test-BookieRecoveryTest.patch, TEST-org.apache.bookkeeper.client.BookieRecoveryTest.xml, TEST-org.apache.bookkeeper.client.BookieRecoveryTest.xml.fail1, TEST-org.apache.bookkeeper.client.BookieRecoveryTest.xml.fail2 testMetadataConflictWithRecovery[0](org.apache.bookkeeper.client.BookieRecoveryTest): Not fully replicated -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (BOOKKEEPER-736) Stats for AutoRecovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110379#comment-14110379 ] Rakesh R commented on BOOKKEEPER-736: - bq.The way to keep a default constructor with a NullStatsLogger. I don't see a strong reason to remove them. so I will keep them around. ReplicationWorker, AuditorElector - both are internal entities to the AutoRecovery module and is used only inside AutoRecoveryMain. I was thinking in that way. Stats for AutoRecovery -- Key: BOOKKEEPER-736 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-736 Project: Bookkeeper Issue Type: New Feature Components: bookkeeper-auto-recovery Reporter: Rakesh R Assignee: Sijie Guo Fix For: 4.3.0 Attachments: BOOKKEEPER-736.patch, BOOKKEEPER-736.v2.patch, BOOKKEEPER-736.v3.patch, BOOKKEEPER-736.v4.patch, Prototype-JMX-beans-for-standalone-autorecovery-process.png Idea of this JIRA is to provide jmx interfaces to get the statistics of the auto recovery activities. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110389#comment-14110389 ] Rakesh R commented on BOOKKEEPER-773: - Thanks Sijie for the feedback. I'll rebase it as soon as possible, I'm little busy with other internal schedules:( Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-736) Stats for AutoRecovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110451#comment-14110451 ] Rakesh R commented on BOOKKEEPER-736: - bq. please consider tests. Tests can be modified using NullStatsLogger. Since it is not exposed to anyone, won't be a problem. Stats for AutoRecovery -- Key: BOOKKEEPER-736 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-736 Project: Bookkeeper Issue Type: New Feature Components: bookkeeper-auto-recovery Reporter: Rakesh R Assignee: Sijie Guo Fix For: 4.3.0 Attachments: BOOKKEEPER-736.patch, BOOKKEEPER-736.v2.patch, BOOKKEEPER-736.v3.patch, BOOKKEEPER-736.v4.patch, Prototype-JMX-beans-for-standalone-autorecovery-process.png Idea of this JIRA is to provide jmx interfaces to get the statistics of the auto recovery activities. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-736) Stats for AutoRecovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110504#comment-14110504 ] Rakesh R commented on BOOKKEEPER-736: - There is a nice way to to differentiate things. I've seen @VisibleForTesting annotations 'com.google.common.annotations.VisibleForTesting' in hadoop projects. Do you mind add this for the existing constructors which we are discussing. Apart from this +1 for the patch. Stats for AutoRecovery -- Key: BOOKKEEPER-736 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-736 Project: Bookkeeper Issue Type: New Feature Components: bookkeeper-auto-recovery Reporter: Rakesh R Assignee: Sijie Guo Fix For: 4.3.0 Attachments: BOOKKEEPER-736.patch, BOOKKEEPER-736.v2.patch, BOOKKEEPER-736.v3.patch, BOOKKEEPER-736.v4.patch, Prototype-JMX-beans-for-standalone-autorecovery-process.png Idea of this JIRA is to provide jmx interfaces to get the statistics of the auto recovery activities. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-736) Stats for AutoRecovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110935#comment-14110935 ] Rakesh R commented on BOOKKEEPER-736: - bq. I don't like the idea to mark every this kind of constructor to be visible for testing, which doesn't seems to be exposed only for testing. ok. its just a suggestion, please feel free to ignore if nobody complains this. Stats for AutoRecovery -- Key: BOOKKEEPER-736 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-736 Project: Bookkeeper Issue Type: New Feature Components: bookkeeper-auto-recovery Reporter: Rakesh R Assignee: Sijie Guo Fix For: 4.3.0 Attachments: BOOKKEEPER-736.patch, BOOKKEEPER-736.v2.patch, BOOKKEEPER-736.v3.patch, BOOKKEEPER-736.v4.patch, Prototype-JMX-beans-for-standalone-autorecovery-process.png Idea of this JIRA is to provide jmx interfaces to get the statistics of the auto recovery activities. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-736) Stats for AutoRecovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14108833#comment-14108833 ] Rakesh R commented on BOOKKEEPER-736: - Make sense +1. Stats for AutoRecovery -- Key: BOOKKEEPER-736 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-736 Project: Bookkeeper Issue Type: New Feature Components: bookkeeper-auto-recovery Reporter: Rakesh R Assignee: Sijie Guo Fix For: 4.3.0 Attachments: BOOKKEEPER-736.patch, BOOKKEEPER-736.v2.patch, Prototype-JMX-beans-for-standalone-autorecovery-process.png Idea of this JIRA is to provide jmx interfaces to get the statistics of the auto recovery activities. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-736) Stats for AutoRecovery
[
https://issues.apache.org/jira/browse/BOOKKEEPER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14105737#comment-14105737
]
Rakesh R commented on BOOKKEEPER-736:
-
bq. rereplicate op stats already has a counter for the re-replication
operations.
ok, its counter for re-replication attempts. Actually
ReplicationWorker#rereplicate() is a blocking call and will not return until a
ledger is available for rereplication. It will not give the latency of the
replication attempt if we keep the logic over the #rereplicate() call. Can we
think of moving the latency computation inside #rereplicate() call like below:
{code}
private void rereplicate() {
long ledgerIdToReplicate = underreplicationManager
.getLedgerToRereplicate();
stopwatch.reset().start();
//
//replication logic
//
long latencyMillis = stopwatch.stop().elapsedMillis();
underreplicationManager.markLedgerReplicated(ledgerIdToReplicate);
//...
}
{code}
{quote}bookie process starts up, it marks 3 ledgers as underreplicated, the
counter is 3.
bookie failed and restarted, the counter was reset to 0. those 3 ledgers are
marked as replicated after re-replicated. so the counter becomes -3. what does
-3 mean? it means nothing.
{quote}
Thank you for your patience. Yeah, UrLedgers could not be able to manage as
counter. This has to be queried to the ZooKeeper for accuracy. I have one last
doubt, is there any way to represent a flag UrLedgersExists true/false in
StatsLogger? If yes, one idea to implement this is - ReplicationWorker can
maintain this flag. He has the knowledge of under replicated ledgers in bk
cluster. Using this information he can set the flag to true or false during his
re-replication attempts.
If you feel this is not meaningful, I agree to leave this part now and move
ahead with other areas rather than spending time on this:)
Stats for AutoRecovery
--
Key: BOOKKEEPER-736
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-736
Project: Bookkeeper
Issue Type: New Feature
Components: bookkeeper-auto-recovery
Reporter: Rakesh R
Assignee: Sijie Guo
Fix For: 4.3.0
Attachments: BOOKKEEPER-736.patch, BOOKKEEPER-736.v2.patch,
Prototype-JMX-beans-for-standalone-autorecovery-process.png
Idea of this JIRA is to provide jmx interfaces to get the statistics of the
auto recovery activities.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (BOOKKEEPER-736) Stats for AutoRecovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14098208#comment-14098208 ] Rakesh R commented on BOOKKEEPER-736: - bq. I don't know if NumUrLedgers, NumRereplicatedLedgers is useful, since it doesn't indicated the number of under replicated ledgers in the cluster at a given point At a given point of time the following autorecovery stats gives : # _NumUrLedgers_ - the total number of under replicated ledgers in the BK cluster # _NumRereplicatedLedgers_ - the total number of ledgers which a bookie(say BK1) has successfully re-replicated to itself. IMHO these statistics would be helpful to the administrators to watch the stability of the BK cluster. bq. I would suggest leave them out and re-consider adding them back if there is a use case that need them. It would be good to expose these info. Also, welcome others opinion. Thanks, Rakesh Stats for AutoRecovery -- Key: BOOKKEEPER-736 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-736 Project: Bookkeeper Issue Type: New Feature Components: bookkeeper-auto-recovery Reporter: Rakesh R Assignee: Sijie Guo Fix For: 4.3.0 Attachments: BOOKKEEPER-736.patch, BOOKKEEPER-736.v2.patch, Prototype-JMX-beans-for-standalone-autorecovery-process.png Idea of this JIRA is to provide jmx interfaces to get the statistics of the auto recovery activities. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-774) Flaky test org.apache.bookkeeper.test.ReadOnlyBookieTest.testBookieShouldTurnWritableFromReadOnly
[
https://issues.apache.org/jira/browse/BOOKKEEPER-774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14088873#comment-14088873
]
Rakesh R commented on BOOKKEEPER-774:
-
Good catch. +1 lgtm
Flaky test
org.apache.bookkeeper.test.ReadOnlyBookieTest.testBookieShouldTurnWritableFromReadOnly
-
Key: BOOKKEEPER-774
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-774
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-server
Reporter: Sijie Guo
Assignee: Sijie Guo
Labels: test
Fix For: 4.3.0
Attachments: BOOKKEEPER-774.diff
it failed at https://builds.apache.org/job/bookkeeper-trunk/716 .
It seems that an addEntry hit a ClosedChannelException on file io after the
bookie turned to writable from readonly.
{code}
2014-07-26 07:03:14,584 - ERROR -
[BookieWriteThread-15006-0:WriteEntryProcessorV3@103] - Error writing entry:0
to ledger:4
java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:88)
at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:184)
at
org.apache.bookkeeper.bookie.BufferedChannel.flushInternal(BufferedChannel.java:126)
at
org.apache.bookkeeper.bookie.BufferedChannel.flush(BufferedChannel.java:111)
at
org.apache.bookkeeper.bookie.EntryLogger.createNewLog(EntryLogger.java:371)
at
org.apache.bookkeeper.bookie.EntryLogger.addEntry(EntryLogger.java:623)
at
org.apache.bookkeeper.bookie.InterleavedLedgerStorage.processEntry(InterleavedLedgerStorage.java:296)
at
org.apache.bookkeeper.bookie.InterleavedLedgerStorage.processEntry(InterleavedLedgerStorage.java:283)
at
org.apache.bookkeeper.bookie.InterleavedLedgerStorage.addEntry(InterleavedLedgerStorage.java:196)
at
org.apache.bookkeeper.bookie.LedgerDescriptorImpl.addEntry(LedgerDescriptorImpl.java:80)
at
org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1046)
at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1085)
at
org.apache.bookkeeper.proto.WriteEntryProcessorV3.getAddResponse(WriteEntryProcessorV3.java:99)
at
org.apache.bookkeeper.proto.WriteEntryProcessorV3.run(WriteEntryProcessorV3.java:132)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[
https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087336#comment-14087336
]
Rakesh R commented on BOOKKEEPER-773:
-
Thanks [~fpj] for the comments.
bq.I'm curious to know how often one needs to execute such a command. This is a
rare command to use, yes? I also don't like such long names for commands, what
about bkrename or renamebookie?
Yes, its rare command. Actually we need to rename both (1)ledgers and
(2)cookies and this will be done separately through two different bk commands.
I've included the target entity(ledger/cookie) also in the command to become
more self explanatory. Thats the reason name becomes long.
How about the following, is that fine for you ?
{code}
bkrenameinledger currentBookieId toBookieId
bkrenameincookie -useHostNameAsBookieID true|false
{code}
bq.seems to swap 3 and 4 in this sequence:
Yeah, I'll do it.
Provide admin tool to rename bookie identifier in Cookies
-
Key: BOOKKEEPER-773
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-server
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch,
002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch,
003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch
The idea of this JIRA to implement a mechanism to efficiently rename the
bookie identifier present in the Cookies. Cookie information will be present
in:
- ledger journal directories in each Bookie server
- cookies znode in ZooKeeper
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (BOOKKEEPER-736) Stats for AutoRecovery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14087591#comment-14087591 ] Rakesh R commented on BOOKKEEPER-736: - [~hustlmsp] thanks for the patch. Could you please look at the snapshot image I had attached earlier in this JIRA. Here I tried to expose few details(attributes) where user might be interested. What do you think of including these too. Stats for AutoRecovery -- Key: BOOKKEEPER-736 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-736 Project: Bookkeeper Issue Type: New Feature Components: bookkeeper-auto-recovery Reporter: Rakesh R Assignee: Sijie Guo Fix For: 4.3.0 Attachments: BOOKKEEPER-736.patch, BOOKKEEPER-736.v2.patch, Prototype-JMX-beans-for-standalone-autorecovery-process.png Idea of this JIRA is to provide jmx interfaces to get the statistics of the auto recovery activities. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14083634#comment-14083634 ] Rakesh R commented on BOOKKEEPER-773: - [~hustlmsp] I've reworked and attached new patch addressing the comments. Please look at this. Thanks! Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 003-BOOKKEEPER-773-rename-bookieid-in-cookies.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch, 002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[
https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080832#comment-14080832
]
Rakesh R commented on BOOKKEEPER-773:
-
Thanks [~hustlmsp] for the reviews. After seeing the comments, I think the
command format in the proposed patch will not work well. It will be difficult
to deal with the case of identifying and deleting the old cookie from zk.
_Rename command format_ would be :
{code}
renamebookieincookies -useHostNameAsBookieID true|false
{code}
Sequence would be:
# read cookie from zookeeper. For reading it will invert useHostNameAsBookieID
flag value to the configuration, then read the previous cookie zPath. If cookie
doesn't exists in zk, log cause and exit with -1.
# generate new cookie. Here it will set the given 'useHostNameAsBookieID' flag
value to the configuration and get the Bookie.getBookieAddress(conf). Write
this new cookie to zk.
# update new cookie into local directories.
# create new cookie with new zk path with new bookie id
# delete the old one. Here it will invert useHostNameAsBookieID flag value and
get the previous cookie zPath
bq.change the bookie id and update it back to zookeeper to use same zk path.
I think this is not required and can be removed ?
\\
I've attached new patch to understand the flow better. Whats your opinion ?
Provide admin tool to rename bookie identifier in Cookies
-
Key: BOOKKEEPER-773
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773
Project: Bookkeeper
Issue Type: Sub-task
Components: bookkeeper-server
Reporter: Rakesh R
Assignee: Rakesh R
Fix For: 4.3.0
Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch,
002-BOOKKEEPER-773-rename-bookieid-in-cookies.patch
The idea of this JIRA to implement a mechanism to efficiently rename the
bookie identifier present in the Cookies. Cookie information will be present
in:
- ledger journal directories in each Bookie server
- cookies znode in ZooKeeper
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-634: Summary: Provide admin tool to rename bookie identifier in ledger metadata (was: Provide admin tool to change bookie's identifier from IP to hostname) Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (BOOKKEEPER-634) Provide admin tool to rename bookie identifier in ledger metadata
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-634: Attachment: BOOKKEEPER-634-rename-bookieid-in-ledger.patch Provide admin tool to rename bookie identifier in ledger metadata - Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, BOOKKEEPER-634-rename-bookieid-in-ledger.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
Rakesh R created BOOKKEEPER-773: --- Summary: Provide admin tool to rename bookie identifier in Cookies Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated BOOKKEEPER-773: Attachment: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-773) Provide admin tool to rename bookie identifier in Cookies
[ https://issues.apache.org/jira/browse/BOOKKEEPER-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14074392#comment-14074392 ] Rakesh R commented on BOOKKEEPER-773: - I'm attaching a patch which exposes _renamebookieincookies_ cmd. This admin command has to be executed in each bookie server machine. When executing it will update the given 'tobookieId' into the cookies. Let me try adding test cases, meantime I'd like to know any comments on the approach. Thanks! Provide admin tool to rename bookie identifier in Cookies - Key: BOOKKEEPER-773 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-773 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-server Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 001-BOOKKEEPER-773-rename-bookieid-in-cookies.patch The idea of this JIRA to implement a mechanism to efficiently rename the bookie identifier present in the Cookies. Cookie information will be present in: - ledger journal directories in each Bookie server - cookies znode in ZooKeeper -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to change bookie's identifier from IP to hostname
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068929#comment-14068929 ] Rakesh R commented on BOOKKEEPER-634: - bq. 1) admin stops the bookie. 2) admin runs a tool to update cookie. 3) admin start the bookie. Thanks [~hustlmsp], this is pretty simple. I'll prepare a patch following this approach. Provide admin tool to change bookie's identifier from IP to hostname Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-634) Provide admin tool to change bookie's identifier from IP to hostname
[ https://issues.apache.org/jira/browse/BOOKKEEPER-634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048869#comment-14048869 ] Rakesh R commented on BOOKKEEPER-634: - ping : [~ikelly], [~fpj], [~hustlmsp] Provide admin tool to change bookie's identifier from IP to hostname Key: BOOKKEEPER-634 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-634 Project: Bookkeeper Issue Type: Sub-task Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.2.1 Reporter: Rakesh R Assignee: Rakesh R Fix For: 4.3.0 Attachments: 0001-BOOKKEEPER-634-initial-draft-version.patch, BOOKKEEPER-634.patch, BOOKKEEPER-634.patch This JIRA to discuss about admin tool for changing the bookie's IP to hostname. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-765) bookkeeper script should fall back to java in path if JAVA_HOME is not set
[
https://issues.apache.org/jira/browse/BOOKKEEPER-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14026259#comment-14026259
]
Rakesh R commented on BOOKKEEPER-765:
-
OK cool.
Just one comment. Please include which java going to use in echo message.
{code}
echo JAVA_HOME not set, using java from PATH $JAVA
{code}
bookkeeper script should fall back to java in path if JAVA_HOME is not set
--
Key: BOOKKEEPER-765
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-765
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-server
Reporter: Ivan Kelly
Assignee: Ivan Kelly
Priority: Blocker
Fix For: 4.3.0, 4.2.3
Attachments:
0001-BOOKKEEPER-765-bookkeeper-script-should-fall-b.trunk.patch,
0001-BOOKKEEPER-765-bookkeeper-script-should-fall-back-to.patch
The current behaviour requires JAVA_HOME to be set, which isn't set by
default on all machines. So machines on which bookkeeper ran fine, no longer
work.
Solution is to fallback to java in path is JAVA_HOME not set.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
