[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883601#comment-15883601
 ] 

ASF GitHub Bot commented on BOOKKEEPER-552:
---

Github user reddycharan commented on the issue:

https://github.com/apache/bookkeeper/pull/112
  
It sounds good that LongHLM is not changed. For HierarchicalLM also, it 
sounds good, but I'm not the best person to comment since we are not using it. 
Thanks @knusbaum.


> 64 Bits Ledger ID Generation
> 
>
> Key: BOOKKEEPER-552
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client, bookkeeper-server, hedwig-server
>Reporter: Jiannan Wang
>Assignee: Kyle Nusbaum
> Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, 
> BOOKKEEPER-552.patch
>
>
> This task aims to find and implement 64 bits global unique ledger id 
> generation mechanisms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15883552#comment-15883552
 ] 

ASF GitHub Bot commented on BOOKKEEPER-552:
---

Github user knusbaum commented on the issue:

https://github.com/apache/bookkeeper/pull/112
  
@jvrao @reddycharan 

So my plan is this: Keep LongHierarchicalLedgerManager as is currently in 
master, move HierarchicalLedgerManager to LegacyHierarchicalLedgerManager and 
create the new backwards-compatible one as HierarchicalLedgerManager. This way, 
if you're already using LongHierarchicalLedgerManager, it will still work. If 
you're using the current HierarchicalLedgerManager, that will work, and if you 
(for some unknown reason) want to use the old legacy one, it still exists as 
well.

Thoughts?


> 64 Bits Ledger ID Generation
> 
>
> Key: BOOKKEEPER-552
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client, bookkeeper-server, hedwig-server
>Reporter: Jiannan Wang
>Assignee: Kyle Nusbaum
> Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, 
> BOOKKEEPER-552.patch
>
>
> This task aims to find and implement 64 bits global unique ledger id 
> generation mechanisms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882990#comment-15882990
 ] 

ASF GitHub Bot commented on BOOKKEEPER-552:
---

Github user jvrao commented on the issue:

https://github.com/apache/bookkeeper/pull/112
  
Couple of questions:
1. Charan with this change how does LongHierarchicalLedgerManager users 
(Salesforce) has backward compatibility?
2.  @knusbaum Why not keep LongHierarchicalLedgerManager around? even when 
HierarchicalLedgerManager supporting both ids.?


> 64 Bits Ledger ID Generation
> 
>
> Key: BOOKKEEPER-552
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client, bookkeeper-server, hedwig-server
>Reporter: Jiannan Wang
>Assignee: Kyle Nusbaum
> Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, 
> BOOKKEEPER-552.patch
>
>
> This task aims to find and implement 64 bits global unique ledger id 
> generation mechanisms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-588) SSL support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15882528#comment-15882528
 ] 

ASF GitHub Bot commented on BOOKKEEPER-588:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/97
  
@ kishorekasi  thank you for your review.
I have addressed most of the issues and rebased to current master.

I have not run any benchmark on SSL, so I have no number to share about 
performance degradation


> SSL support
> ---
>
> Key: BOOKKEEPER-588
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-588
> Project: Bookkeeper
>  Issue Type: Sub-task
>Reporter: Ivan Kelly
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
> Attachments: 0001-MutualTLS-for-Bookkeeper.patch, 
> 0004-BOOKKEEPER-588-SSL-support-for-bookkeeper.patch
>
>
> SSL support using startTLS



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880835#comment-15880835
 ] 

ASF GitHub Bot commented on BOOKKEEPER-552:
---

Github user knusbaum commented on the issue:

https://github.com/apache/bookkeeper/pull/112
  
@reddycharan Do you see any problems with this?


> 64 Bits Ledger ID Generation
> 
>
> Key: BOOKKEEPER-552
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client, bookkeeper-server, hedwig-server
>Reporter: Jiannan Wang
>Assignee: Kyle Nusbaum
> Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, 
> BOOKKEEPER-552.patch
>
>
> This task aims to find and implement 64 bits global unique ledger id 
> generation mechanisms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1003) Fix TestDiskChecker so it can be used on /dev/shm

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880618#comment-15880618
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1003:


Github user jvrao commented on the issue:

https://github.com/apache/bookkeeper/pull/107
  
+1 LGTM


> Fix TestDiskChecker so it can be used on /dev/shm
> -
>
> Key: BOOKKEEPER-1003
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1003
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Robert Joseph Evans
>
> Some file systems don't report disk usage by byte (but by block) /dev/shm is 
> one of these.
> We should update the tests to be more lenient.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1001) Make LocalBookiesRegistry.isLocalBookie() public

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880614#comment-15880614
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1001:


Github user jvrao commented on the issue:

https://github.com/apache/bookkeeper/pull/106
  
+1 LGTM


> Make LocalBookiesRegistry.isLocalBookie() public
> 
>
> Key: BOOKKEEPER-1001
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1001
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.4.0
>Reporter: Francesco Caliumi
>Priority: Minor
>  Labels: easyfix, newbie
>   Original Estimate: 0.25h
>  Remaining Estimate: 0.25h
>
> This function is useful for applications using an embedded local bookie, for 
> example in order to create a custom EnsamblePlacementPolicy which prefers the 
> local bookie if present.
> This is an easy fix, I will file a PR.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880609#comment-15880609
 ] 

ASF GitHub Bot commented on BOOKKEEPER-998:
---

Github user jvrao commented on the issue:

https://github.com/apache/bookkeeper/pull/104
  
+1 LGTM. 


> Increased the max entry size to 5MB
> ---
>
> Key: BOOKKEEPER-998
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> In practice we have found the Bookkeeper can handle 5MB entries without 
> issue.  We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the 
> {{BookieNettyServer.maxMessageSize}} and the 
> {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and 
> ideally 5MB.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-816) use native fallocate & sync_file_range to improve journal allocation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880601#comment-15880601
 ] 

ASF GitHub Bot commented on BOOKKEEPER-816:
---

Github user jvrao commented on the issue:

https://github.com/apache/bookkeeper/pull/80
  
@sijie  any perf numbers/stats to show how much improvement this achieved? 


> use native fallocate & sync_file_range to improve journal allocation
> 
>
> Key: BOOKKEEPER-816
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-816
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> it'd better to leverage filesystem fallocate & sync_file_range for journal 
> performance.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878551#comment-15878551
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
@eolivelli  Thanks anyway


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878530#comment-15878530
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
I'm sorry, my +1 is non-binding. I'm not a committer

pinging @sijie 


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878523#comment-15878523
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
@eolivelli Thank you so much for the review :) Could I trouble you to merge 
it into master?


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878364#comment-15878364
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
@eolivelli Done.


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877682#comment-15877682
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
@govind-menon I'm sorry but in your actual code on the PR inside the 
shutdown() method there is still the 'wait(100)'. 
OK for the '100' but we should use Thread.sleep(100) as IMHO 'Object#wait' 
should be used  inside a 'synchronized' block.

What do you think ?


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15876527#comment-15876527
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
@eolivelli All feedback completed.


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872771#comment-15872771
 ] 

ASF GitHub Bot commented on BOOKKEEPER-552:
---

Github user knusbaum commented on the issue:

https://github.com/apache/bookkeeper/pull/112
  
LongHierarchicalLedgerManager is being moved to HierarchicalLedgerManager 
and HierarchicalLedgerManager is being moved to LegacyHierarchicalLedgerManager.

LongHierarchicalLedgerManager (now HierarchicalLedgerManager) has been made 
backwards-compatible so that existing Bookkeeper instances running 
HierarchicalLedgerManager can be upgraded to the new code, which will allow 
64-bit IDs.

This is achieved by having the HierarchicalLedgerManager proxy requests to 
the LegacyHierarchicalLedgerManager for ids which are less than 
Integer.MAX_VALUE
For ids greater than that, the new code is used.

We also add a new id generator. The existing ZkLedgerIdGenerator is limited 
to 31 bits (about 2 million) ledgers because ZooKeeper's sequential id 
generator uses a signed int internally. IDs start going negative if you 
overflow it, which messes some stuff up. 

The new ID generator uses a multi-level directory scheme to overcome this.


> 64 Bits Ledger ID Generation
> 
>
> Key: BOOKKEEPER-552
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client, bookkeeper-server, hedwig-server
>Reporter: Jiannan Wang
>Assignee: Kyle Nusbaum
> Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, 
> BOOKKEEPER-552.patch
>
>
> This task aims to find and implement 64 bits global unique ledger id 
> generation mechanisms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872745#comment-15872745
 ] 

ASF GitHub Bot commented on BOOKKEEPER-552:
---

Github user reddycharan commented on the issue:

https://github.com/apache/bookkeeper/pull/112
  
@knusbaum can you please explain briefly, what you are doing in this change 
and why LongHierarchicalLedgermanager is being removed.


> 64 Bits Ledger ID Generation
> 
>
> Key: BOOKKEEPER-552
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client, bookkeeper-server, hedwig-server
>Reporter: Jiannan Wang
>Assignee: Kyle Nusbaum
> Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, 
> BOOKKEEPER-552.patch
>
>
> This task aims to find and implement 64 bits global unique ledger id 
> generation mechanisms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872726#comment-15872726
 ] 

ASF GitHub Bot commented on BOOKKEEPER-552:
---

Github user knusbaum commented on the issue:

https://github.com/apache/bookkeeper/pull/112
  
@reddycharan Yes, this builds on that. It makes it backwards compatible and 
builds the 64-bit id generation stuff which was missing.


> 64 Bits Ledger ID Generation
> 
>
> Key: BOOKKEEPER-552
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client, bookkeeper-server, hedwig-server
>Reporter: Jiannan Wang
>Assignee: Kyle Nusbaum
> Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, 
> BOOKKEEPER-552.patch
>
>
> This task aims to find and implement 64 bits global unique ledger id 
> generation mechanisms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872721#comment-15872721
 ] 

ASF GitHub Bot commented on BOOKKEEPER-552:
---

Github user reddycharan commented on the issue:

https://github.com/apache/bookkeeper/pull/112
  
Hey @knusbaum we already pushed our commit for 63 bit ledgerID. Can you 
please check that - 


https://github.com/apache/bookkeeper/commit/42e8f1294f206cbe51a6af669cf605833b78bf42



> 64 Bits Ledger ID Generation
> 
>
> Key: BOOKKEEPER-552
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client, bookkeeper-server, hedwig-server
>Reporter: Jiannan Wang
>Assignee: Kyle Nusbaum
> Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, 
> BOOKKEEPER-552.patch
>
>
> This task aims to find and implement 64 bits global unique ledger id 
> generation mechanisms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-552) 64 Bits Ledger ID Generation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872692#comment-15872692
 ] 

ASF GitHub Bot commented on BOOKKEEPER-552:
---

GitHub user knusbaum opened a pull request:

https://github.com/apache/bookkeeper/pull/112

BOOKKEEPER-552: 64 Bits Ledger ID Generation

This PR adds 63-bit ID generation to Bookkeeper and modifies the 
hierarchical ledger structure to be backwards compatible.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/knusbaum/bookkeeper master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/112.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #112


commit 9dda55f377fbcf66da79be558ff6204f3b93f0f6
Author: Kyle Nusbaum 
Date:   2017-02-10T21:52:11Z

Final modifications for 63-bit ledger IDs.




> 64 Bits Ledger ID Generation
> 
>
> Key: BOOKKEEPER-552
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-552
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client, bookkeeper-server, hedwig-server
>Reporter: Jiannan Wang
>Assignee: Kyle Nusbaum
> Attachments: BOOKKEEPER-552.patch, BOOKKEEPER-552.patch, 
> BOOKKEEPER-552.patch
>
>
> This task aims to find and implement 64 bits global unique ledger id 
> generation mechanisms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1007) Explicit LAC: make the interval configurable in milliseconds instead of seconds

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872010#comment-15872010
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1007:


Github user revans2 commented on the issue:

https://github.com/apache/bookkeeper/pull/111
  
With the renamed variables I am +1 (non-binding)


> Explicit LAC: make the interval configurable in milliseconds instead of 
> seconds
> ---
>
> Key: BOOKKEEPER-1007
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1007
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-client
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Critical
> Fix For: 4.5.0
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> in BOOKKEEPER-874 we introduced the "Explicit LAC", but the interval is 
> configurable in "seconds", and I need to achieve sub-second latency.
> So my proposal is to make it configurable in "milliseconds", since 4.5.0 has 
> not yet been released I think it is better to change it as soon as possible



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1007) Explicit LAC: make the interval configurable in milliseconds instead of seconds

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871917#comment-15871917
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1007:


GitHub user eolivelli opened a pull request:

https://github.com/apache/bookkeeper/pull/111

BOOKKEEPER-1007 Explicit LAC: make the interval configurable in ms



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/eolivelli/bookkeeper 
BOOKKEEPER-1007-explicitlac-ms

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/111.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #111


commit 6c07d46546d2bfe6e7208a8699b719679f45e415
Author: eolivelli 
Date:   2017-02-17T14:21:35Z

BOOKKEEPER-1007 Explicit LAC: make the interval configurable in 
milliseconds instead of seconds




> Explicit LAC: make the interval configurable in milliseconds instead of 
> seconds
> ---
>
> Key: BOOKKEEPER-1007
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1007
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-client
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Critical
> Fix For: 4.5.0
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> in BOOKKEEPER-874 we introduced the "Explicit LAC", but the interval is 
> configurable in "seconds", and I need to achieve sub-second latency.
> So my proposal is to make it configurable in "milliseconds", since 4.5.0 has 
> not yet been released I think it is better to change it as soon as possible



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870745#comment-15870745
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
@eolivelli The wait is timed (without needing a notify).


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870673#comment-15870673
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
@govind-menon jenkins build is triggered by any push, without the need to 
reopen the PR


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870658#comment-15870658
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon closed the pull request at:

https://github.com/apache/bookkeeper/pull/109


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870657#comment-15870657
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
@eolivelli I'm trying to retrigger the Jenkins build. Working on your other 
feedback.


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870659#comment-15870659
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


GitHub user govind-menon reopened a pull request:

https://github.com/apache/bookkeeper/pull/109

BOOKKEEPER-1004: Allow bookie garbage collection to be triggered manu…

Ran CompactionTest#testForceGarbageCollection

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/govind-menon/bookkeeper BOOKKEEPER-1004

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/109.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #109


commit 68688fac643ab991873130780703172890891a15
Author: Govind Menon 
Date:   2017-02-14T18:06:44Z

BOOKKEEPER-1004: Allow bookie garbage collection to be triggered manually 
from tests




> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-391) Support Kerberos authentication of bookkeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870136#comment-15870136
 ] 

ASF GitHub Bot commented on BOOKKEEPER-391:
---

GitHub user eolivelli opened a pull request:

https://github.com/apache/bookkeeper/pull/110

BOOKKEEPER-391 Support Kerberos authentication of bookkeeper

This patch contains a very basic AuthProvider which uses JAAS and so 
enables the usage or GSSAPI/Kerberos for BookKeeper authentication

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/eolivelli/bookkeeper BOOKKEEPER-391-kerberos

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/110.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #110


commit d4a26c8dbde264a95dfb055c6775b07915ee757c
Author: eolivelli 
Date:   2016-10-21T15:14:31Z

BOOKKEEPER-391 Support Kerberos authentication of bookkeeper




> Support Kerberos authentication of bookkeeper
> -
>
> Key: BOOKKEEPER-391
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-391
> Project: Bookkeeper
>  Issue Type: New Feature
>  Components: bookkeeper-client, bookkeeper-server
>Reporter: Rakesh R
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
>
> This JIRA to discuss authentication mechanism of bookie clients and server. 
> Assume ZK provides fully secured communication channel using Kerberos based 
> authentication and authorization model. We could also manage and renew users 
> authenticated to BK via Kerberos. There is currently no configuration or 
> hooks for the Bookie process to obtain Kerberos credentials.
> Today an unauthenticated bookie client can easily establish connection with 
> the bookkeeper server. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-391) Support Kerberos authentication of bookkeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870138#comment-15870138
 ] 

ASF GitHub Bot commented on BOOKKEEPER-391:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/110
  
@sijie  I have rebased the patch


> Support Kerberos authentication of bookkeeper
> -
>
> Key: BOOKKEEPER-391
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-391
> Project: Bookkeeper
>  Issue Type: New Feature
>  Components: bookkeeper-client, bookkeeper-server
>Reporter: Rakesh R
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
>
> This JIRA to discuss authentication mechanism of bookie clients and server. 
> Assume ZK provides fully secured communication channel using Kerberos based 
> authentication and authorization model. We could also manage and renew users 
> authenticated to BK via Kerberos. There is currently no configuration or 
> hooks for the Bookie process to obtain Kerberos credentials.
> Today an unauthenticated bookie client can easily establish connection with 
> the bookkeeper server. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-391) Support Kerberos authentication of bookkeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870014#comment-15870014
 ] 

ASF GitHub Bot commented on BOOKKEEPER-391:
---

Github user eolivelli closed the pull request at:

https://github.com/apache/bookkeeper/pull/85


> Support Kerberos authentication of bookkeeper
> -
>
> Key: BOOKKEEPER-391
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-391
> Project: Bookkeeper
>  Issue Type: New Feature
>  Components: bookkeeper-client, bookkeeper-server
>Reporter: Rakesh R
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
>
> This JIRA to discuss authentication mechanism of bookie clients and server. 
> Assume ZK provides fully secured communication channel using Kerberos based 
> authentication and authorization model. We could also manage and renew users 
> authenticated to BK via Kerberos. There is currently no configuration or 
> hooks for the Bookie process to obtain Kerberos credentials.
> Today an unauthenticated bookie client can easily establish connection with 
> the bookkeeper server. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15869783#comment-15869783
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
@govind-menon you can just update your personal branch on github in order 
to update this PR, usually by issueing a "git push --force". there is no need 
to close/reopen the PR


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868415#comment-15868415
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon commented on the issue:

https://github.com/apache/bookkeeper/pull/109
  
The current 2 test failures are being fixed in a separate JIRA and the 3rd 
test failure is known to be flakey - BOOKKEEPER-718


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868400#comment-15868400
 ] 

ASF GitHub Bot commented on BOOKKEEPER-998:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/104
  
For me is +1 but I am not a committer. Usually @sijie merges PR or maybe 
@merlimat


> Increased the max entry size to 5MB
> ---
>
> Key: BOOKKEEPER-998
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> In practice we have found the Bookkeeper can handle 5MB entries without 
> issue.  We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the 
> {{BookieNettyServer.maxMessageSize}} and the 
> {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and 
> ideally 5MB.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868304#comment-15868304
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


GitHub user govind-menon reopened a pull request:

https://github.com/apache/bookkeeper/pull/109

BOOKKEEPER-1004: Allow bookie garbage collection to be triggered manu…

Ran CompactionTest#testForceGarbageCollection

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/govind-menon/bookkeeper BOOKKEEPER-1004

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/109.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #109


commit 68688fac643ab991873130780703172890891a15
Author: Govind Menon 
Date:   2017-02-14T18:06:44Z

BOOKKEEPER-1004: Allow bookie garbage collection to be triggered manually 
from tests




> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868303#comment-15868303
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon closed the pull request at:

https://github.com/apache/bookkeeper/pull/109


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868210#comment-15868210
 ] 

ASF GitHub Bot commented on BOOKKEEPER-998:
---

Github user revans2 commented on the issue:

https://github.com/apache/bookkeeper/pull/104
  
@eolivelli and @jvrao is there anything more I need to do to get this 
merged in?


> Increased the max entry size to 5MB
> ---
>
> Key: BOOKKEEPER-998
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> In practice we have found the Bookkeeper can handle 5MB entries without 
> issue.  We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the 
> {{BookieNettyServer.maxMessageSize}} and the 
> {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and 
> ideally 5MB.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868058#comment-15868058
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon closed the pull request at:

https://github.com/apache/bookkeeper/pull/109


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868060#comment-15868060
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


GitHub user govind-menon reopened a pull request:

https://github.com/apache/bookkeeper/pull/109

BOOKKEEPER-1004: Allow bookie garbage collection to be triggered manu…

Ran CompactionTest#testForceGarbageCollection

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/govind-menon/bookkeeper BOOKKEEPER-1004

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/109.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #109


commit 68688fac643ab991873130780703172890891a15
Author: Govind Menon 
Date:   2017-02-14T18:06:44Z

BOOKKEEPER-1004: Allow bookie garbage collection to be triggered manually 
from tests




> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867988#comment-15867988
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


Github user govind-menon closed the pull request at:

https://github.com/apache/bookkeeper/pull/109


> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.
> This work was originally done by Ivan Kelly and I am just pushing it back to 
> open source



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1004) Allow bookie garbage collection to be triggered manually from tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15866286#comment-15866286
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1004:


GitHub user govind-menon opened a pull request:

https://github.com/apache/bookkeeper/pull/109

BOOKKEEPER-1004: Allow bookie garbage collection to be triggered manu…

Ran CompactionTest#testForceGarbageCollection

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/govind-menon/bookkeeper BOOKKEEPER-1004

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/109.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #109


commit 68688fac643ab991873130780703172890891a15
Author: Govind Menon 
Date:   2017-02-14T18:06:44Z

BOOKKEEPER-1004: Allow bookie garbage collection to be triggered manually 
from tests




> Allow bookie garbage collection to be triggered manually from tests
> ---
>
> Key: BOOKKEEPER-1004
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1004
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.5.0, 4.6.0
>Reporter: Govind Menon
>Priority: Minor
>  Labels: test
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The current gc tests rely on waiting on a timeout for gc to run. It's
> never certain whether it has run or not or if it's still running. 
> This patch allows tests to trigger a gc run and gives the client
> a future to know when it has completed. The gc algorithm is unchangedI but 
> now it runs in a scheduled executor rather than as a
> Thread.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-944) Multiple issues and improvements to BK Compaction.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15864545#comment-15864545
 ] 

ASF GitHub Bot commented on BOOKKEEPER-944:
---

GitHub user reddycharan opened a pull request:

https://github.com/apache/bookkeeper/pull/108

BOOKKEEPER-944: LowWaterMark Storage Threshold

- Current implementation toggles READONLY status of the bookie as soon 
as a directory usage falls below the disk storage threshold.
  Added LowWaterMark parameter that limits such switches.
1. Bookie transition from RW to RONLY only when all the ledger 
dirs usage > HWM (storage threshold)
2. Bookie transition from RONLY to  RW only when total system 
disk usage (ledger/index disks) capacity is < LWM
3. When bookie is in RW mode all disks which are < HWM (storage 
threshold) are RW
- refactored code to remove circular dependency between 
LedgerDirsManager and LedgerDirsMonitor
- currently Bookie won't start as read-only if disk usage is above 
threshold for all disks, instead of that
  start Bookie in Readonly mode
- relevant testcases

Author: Andrey Yegorov 
Co-Author: Charan Reddy Guttapalem 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/reddycharan/bookkeeper lwmhwm

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/108.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #108


commit ed8f31d8d501426c7744fb95b9c1b8eae7a2b6d4
Author: Andrey Yegorov 
Date:   2016-08-29T23:07:00Z

BOOKKEEPER-944: LowWaterMark Storage Threshold

- Current implementation toggles READONLY status of the bookie as soon 
as a directory usage falls below the disk storage threshold.
  Added LowWaterMark parameter that limits such switches.
1. Bookie transition from RW to RONLY only when all the ledger 
dirs usage > HWM (storage threshold)
2. Bookie transition from RONLY to  RW only when total system 
disk usage (ledger/index disks) capacity is < LWM
3. When bookie is in RW mode all disks which are < HWM (storage 
threshold) are RW
- refactored code to remove circular dependency between 
LedgerDirsManager and LedgerDirsMonitor
- currently Bookie won't start as read-only if disk usage is above 
threshold for all disks, instead of that
  start Bookie in Readonly mode
- relevant testcases

Author: Andrey Yegorov 
Co-Author: Charan Reddy Guttapalem 




> Multiple issues and improvements to BK Compaction.
> --
>
> Key: BOOKKEEPER-944
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-944
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.4.0
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Venkateswararao Jujjuri (JV)
>
> We have identified multiple issues with BK compaction.
> This issue is to list all of them in one Jira ticket.
> 1.
> MajorCompaction and MinorCompaction are very basic. Either they do it or 
> won’t do it. Proposal is to  add Low Water Mark(LWM) and High Water Mark(HWM) 
> to the disk space. Have different compaction frequency and re-claim %s when 
> the disk space is < low water mark  ,  >  LWM < HWM, > HWM.
> 2.
> MajorCompaction and Minor Compactions are strictly frequency based. They 
> should at least time of the day based, and also run during low system load, 
> and if the system load raises, reduce the compaction depending on the disk 
> availability 
> 3.
> Current code disables compaction when disk space grows beyond configured 
> threshold. There is no exit from this point. Have an option to keep reserved 
> space for compaction, at least 2 entryLog file sizes when 
> isForceGCAllowWhenNoSpace enabled.
> 4.
> Current code toggles READONLY status of the bookie as soon as it falls below 
> the disk storage threshold. Imagine if we keep 95% as the threshold, Bookie 
> becomes RW as soon as it falls below 95 % and few more writes pushes it above 
> 95 and it turns back to RONLY. Use a set of defines (another set of LWM/HWM?) 
> where Bookie turns RO on high end and won't become RW until it hits low end.
> 5.
> Current code never checks if the compaction is enabled or disabled once the 
> major/minor compaction is started. If the bookie goes > disk threshold (95%) 
> and at that compaction is going on, it never checks until it finishes but 
> 

[jira] [Commented] (BOOKKEEPER-1003) Fix TestDiskChecker so it can be used on /dev/shm

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863856#comment-15863856
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1003:


Github user kishorvpatil commented on the issue:

https://github.com/apache/bookkeeper/pull/107
  
The unit test failures are unrelated.


> Fix TestDiskChecker so it can be used on /dev/shm
> -
>
> Key: BOOKKEEPER-1003
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1003
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.3.1
>Reporter: Robert Joseph Evans
>
> Some file systems don't report disk usage by byte (but by block) /dev/shm is 
> one of these.
> We should update the tests to be more lenient.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1001) Make LocalBookiesRegistry.isLocalBookie() public

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861195#comment-15861195
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1001:


Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/106
  
LGTM +1


> Make LocalBookiesRegistry.isLocalBookie() public
> 
>
> Key: BOOKKEEPER-1001
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1001
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.4.0
>Reporter: Francesco Caliumi
>Priority: Minor
>  Labels: easyfix, newbie
>   Original Estimate: 0.25h
>  Remaining Estimate: 0.25h
>
> This function is useful for applications using an embedded local bookie, for 
> example in order to create a custom EnsamblePlacementPolicy which prefers the 
> local bookie if present.
> This is an easy fix, I will file a PR.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-1001) Make LocalBookiesRegistry.isLocalBookie() public

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861180#comment-15861180
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1001:


GitHub user caiok opened a pull request:

https://github.com/apache/bookkeeper/pull/106

BOOKKEEPER-1001 Make LocalBookiesRegistry.isLocalBookie() public



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/caiok/bookkeeper BOOKKEEPER-1001

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/106.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #106


commit 9262500da231624cf7128526dcf1b46eef0003b9
Author: Francesco Caliumi - Diennea 
Date:   2017-02-10T11:12:53Z

BOOKKEEPER-1001 Make LocalBookiesRegistry.isLocalBookie() public




> Make LocalBookiesRegistry.isLocalBookie() public
> 
>
> Key: BOOKKEEPER-1001
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1001
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Affects Versions: 4.4.0
>Reporter: Francesco Caliumi
>Priority: Minor
>  Labels: easyfix, newbie
>   Original Estimate: 0.25h
>  Remaining Estimate: 0.25h
>
> This function is useful for applications using an embedded local bookie, for 
> example in order to create a custom EnsamblePlacementPolicy which prefers the 
> local bookie if present.
> This is an easy fix, I will file a PR.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859060#comment-15859060
 ] 

ASF GitHub Bot commented on BOOKKEEPER-998:
---

Github user jvrao commented on the issue:

https://github.com/apache/bookkeeper/pull/104
  
1MB is good enough for us at least for now. Did not test anything larger.


> Increased the max entry size to 5MB
> ---
>
> Key: BOOKKEEPER-998
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> In practice we have found the Bookkeeper can handle 5MB entries without 
> issue.  We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the 
> {{BookieNettyServer.maxMessageSize}} and the 
> {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and 
> ideally 5MB.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-999) BookKeeper client can leak threads

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858083#comment-15858083
 ] 

ASF GitHub Bot commented on BOOKKEEPER-999:
---

GitHub user revans2 opened a pull request:

https://github.com/apache/bookkeeper/pull/105

BOOKKEEPER-999: Stop leaking threads in BookKeeper client



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/revans2/bookkeeper BOOKKEEPER-999

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/105.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #105


commit b0e9a3679f76771e80cefbe67de7a1ed33fc42ad
Author: Robert (Bobby) Evans 
Date:   2017-02-08T14:45:30Z

BOOKKEEPER-999: Stop leaking threads in BookKeeper client




> BookKeeper client can leak threads
> --
>
> Key: BOOKKEEPER-999
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-999
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: bookkeeper-client
>Affects Versions: 4.4.0
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> The client constructor
> {{BookKeeper(ClientConfiguration conf, ZooKeeper zk)}}
> in 4.4 and above will create a new NioClientSocketChannelFactory but does not 
> set the ownership of that factory to true so threads are leaked.
> This showed up as a failure in BookieRecoveryTest on MacOS where it has a 
> hard coded limit of about 2000 threads in a single process, and this test was 
> going beyond that.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857472#comment-15857472
 ] 

ASF GitHub Bot commented on BOOKKEEPER-998:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/104
  
I think 1MB limitation was coming before we used netty for the network 
transportation. Increasing it to 5MB sounds okay to me. I don't think it breaks 
any backward compatibility issue. I don't have any strong preference here on 
using a system property or configuration setting for it.

 


> Increased the max entry size to 5MB
> ---
>
> Key: BOOKKEEPER-998
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> In practice we have found the Bookkeeper can handle 5MB entries without 
> issue.  We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the 
> {{BookieNettyServer.maxMessageSize}} and the 
> {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and 
> ideally 5MB.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856815#comment-15856815
 ] 

ASF GitHub Bot commented on BOOKKEEPER-998:
---

Github user revans2 commented on the issue:

https://github.com/apache/bookkeeper/pull/104
  
@eolivelli I added in configs for both the server and the client.  Please 
take a look again.


> Increased the max entry size to 5MB
> ---
>
> Key: BOOKKEEPER-998
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> In practice we have found the Bookkeeper can handle 5MB entries without 
> issue.  We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the 
> {{BookieNettyServer.maxMessageSize}} and the 
> {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and 
> ideally 5MB.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15856249#comment-15856249
 ] 

ASF GitHub Bot commented on BOOKKEEPER-998:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/104
  
@revans2 
IMHO it would be better to make it configurable as system property or in 
ServerConfiguration it would be even better, so it would be more "testable".
I'm not aware of some system property naming convention, maybe you can 
suggest one and then the discuss is open.

IMHO the max entry size can be related to the max Netty frame size but 
maybe we can use two different limits. For instance on the wire we need headers
@sijie what to you think ?

I think that at Salesforce they make large use of big entries (as they 
store BLOBs on BK), maybe JV as some number to share

For me, I'm starting a new project which will need to store "huge" (more 
than 10 MB) entries and so I will need to set this limit to a greater value 
that 5MB



> Increased the max entry size to 5MB
> ---
>
> Key: BOOKKEEPER-998
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> In practice we have found the Bookkeeper can handle 5MB entries without 
> issue.  We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the 
> {{BookieNettyServer.maxMessageSize}} and the 
> {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and 
> ideally 5MB.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-998) Increased the max entry size to 5MB

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854904#comment-15854904
 ] 

ASF GitHub Bot commented on BOOKKEEPER-998:
---

GitHub user revans2 opened a pull request:

https://github.com/apache/bookkeeper/pull/104

BOOKKEEPER-998: Increased the max entry size to 5MB

Full disclosure, most of these changes were actually done by @merlimat For 
this I am mostly just putting them back into open source.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/revans2/bookkeeper BOOKKEEPER-998

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/104.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #104


commit 614d9cd5bfeeedf35f360adb757d1668429bb8ea
Author: Robert (Bobby) Evans 
Date:   2017-02-06T22:33:49Z

BOOKKEEPER-998: Increased the max entry size to 5MB




> Increased the max entry size to 5MB
> ---
>
> Key: BOOKKEEPER-998
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-998
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
>
> In practice we have found the Bookkeeper can handle 5MB entries without 
> issue.  We should make the {{PerChannelBookieClient.MAX_FRAME_LENGTH}}, the 
> {{BookieNettyServer.maxMessageSize}} and the 
> {{EntryLogger.MAX_SANE_ENTRY_SIZE}} size all consistent with one another, and 
> ideally 5MB.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-950) Ledger placement policy to accomodate different storage capacity of bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849176#comment-15849176
 ] 

ASF GitHub Bot commented on BOOKKEEPER-950:
---

Github user rithin-shetty commented on the issue:

https://github.com/apache/bookkeeper/pull/93
  
@eolivelli I have added a new test case for testing the timeout of 
GET_BOOKIE_INFO op. See TestGetBookieInfoTimeout.java


> Ledger placement policy to accomodate different storage capacity of bookies
> ---
>
> Key: BOOKKEEPER-950
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-950
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Rithin Shetty
>Assignee: Rithin Shetty
> Fix For: 4.5.0
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> In our environment, in Salesforce, we are likely to have bookie nodes with 
> different storage capacity: some will have 1TB others might have 3TB. Also, 
> our ledgers are likely going to be long lived. The current ledger placement 
> policy selects the bookies randomly leading to uniform distribution. This 
> would cause some of bookies to reach high utilization while the rest would be 
> underutilized. We need a new ledger placement policy that has higher 
> probability of selecting bookies with higher free disk space than the ones 
> with lower disk free space.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] (BOOKKEEPER-950) Ledger placement policy to accomodate different storage capacity of bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15847569#comment-15847569
 ] 

ASF GitHub Bot commented on BOOKKEEPER-950:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/93
  
LGTM. +1

@eolivelli since you were also reviewing this change, can you take a look 
at the updated pull request?


> Ledger placement policy to accomodate different storage capacity of bookies
> ---
>
> Key: BOOKKEEPER-950
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-950
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Rithin Shetty
>Assignee: Rithin Shetty
> Fix For: 4.5.0
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> In our environment, in Salesforce, we are likely to have bookie nodes with 
> different storage capacity: some will have 1TB others might have 3TB. Also, 
> our ledgers are likely going to be long lived. The current ledger placement 
> policy selects the bookies randomly leading to uniform distribution. This 
> would cause some of bookies to reach high utilization while the rest would be 
> underutilized. We need a new ledger placement policy that has higher 
> probability of selecting bookies with higher free disk space than the ones 
> with lower disk free space.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] (BOOKKEEPER-996) Apache Rat Check Failures

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15847562#comment-15847562
 ] 

ASF GitHub Bot commented on BOOKKEEPER-996:
---

Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/103


> Apache Rat Check Failures
> -
>
> Key: BOOKKEEPER-996
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-996
> Project: Bookkeeper
>  Issue Type: Task
>  Components: build
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> {code}
> [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.7:check 
> (default-cli) on project bookkeeper-server: Too many unapproved licenses: 1 
> -> [Help 1]
> [ERROR] 
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR] 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] (BOOKKEEPER-950) Ledger placement policy to accomodate different storage capacity of bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15847559#comment-15847559
 ] 

ASF GitHub Bot commented on BOOKKEEPER-950:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/93
  
@rithin-shetty I am fixing that by #103 


> Ledger placement policy to accomodate different storage capacity of bookies
> ---
>
> Key: BOOKKEEPER-950
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-950
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Rithin Shetty
>Assignee: Rithin Shetty
> Fix For: 4.5.0
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> In our environment, in Salesforce, we are likely to have bookie nodes with 
> different storage capacity: some will have 1TB others might have 3TB. Also, 
> our ledgers are likely going to be long lived. The current ledger placement 
> policy selects the bookies randomly leading to uniform distribution. This 
> would cause some of bookies to reach high utilization while the rest would be 
> underutilized. We need a new ledger placement policy that has higher 
> probability of selecting bookies with higher free disk space than the ones 
> with lower disk free space.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] (BOOKKEEPER-950) Ledger placement policy to accomodate different storage capacity of bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15847509#comment-15847509
 ] 

ASF GitHub Bot commented on BOOKKEEPER-950:
---

Github user rithin-shetty commented on the issue:

https://github.com/apache/bookkeeper/pull/93
  
@sijie I've addressed your comments. I've uploaded the merged changes. But 
the Jenkins build has been failing with the following error for some time now. 
Do you know how to fix this ? It's very annoying and should be fixed.

[ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.7:check 
(default-cli) on project bookkeeper-server: Too many unapproved licenses: 2 -> 
[Help 1]



> Ledger placement policy to accomodate different storage capacity of bookies
> ---
>
> Key: BOOKKEEPER-950
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-950
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Rithin Shetty
>Assignee: Rithin Shetty
> Fix For: 4.5.0
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> In our environment, in Salesforce, we are likely to have bookie nodes with 
> different storage capacity: some will have 1TB others might have 3TB. Also, 
> our ledgers are likely going to be long lived. The current ledger placement 
> policy selects the bookies randomly leading to uniform distribution. This 
> would cause some of bookies to reach high utilization while the rest would be 
> underutilized. We need a new ledger placement policy that has higher 
> probability of selecting bookies with higher free disk space than the ones 
> with lower disk free space.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BOOKKEEPER-949) Allow entryLog creation even when bookie is in RO mode for compaction

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838529#comment-15838529
 ] 

ASF GitHub Bot commented on BOOKKEEPER-949:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/99
  
the pull request check job shows that there is a findbugs error. can you 
take a look and address it?


> Allow entryLog creation even when bookie is in RO mode for compaction
> -
>
> Key: BOOKKEEPER-949
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-949
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-server
>Affects Versions: 4.3.2
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Venkateswararao Jujjuri (JV)
>Priority: Critical
>
> When storage usage goes above threshold, there won't be any writable ledger 
> dirs available and bookie turns RO. There can be multiple ways bookie can 
> start compaction including isForceGCAllowWhenNoSpace enabled or by other 
> means. When this happens the compaction process need to create a new entrylog 
> to make progress on compaction. Creation of entrylog will fail because there 
> are no writable ledger dirs.
> This is to allow entrylog creation if there is enough disk space even though 
> used space is above threshold. Because after compaction space will stay the 
> same or go below the threshold.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-949) Allow entryLog creation even when bookie is in RO mode for compaction

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838421#comment-15838421
 ] 

ASF GitHub Bot commented on BOOKKEEPER-949:
---

Github user reddycharan commented on the issue:

https://github.com/apache/bookkeeper/pull/99
  
ping..


> Allow entryLog creation even when bookie is in RO mode for compaction
> -
>
> Key: BOOKKEEPER-949
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-949
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-server
>Affects Versions: 4.3.2
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Venkateswararao Jujjuri (JV)
>Priority: Critical
>
> When storage usage goes above threshold, there won't be any writable ledger 
> dirs available and bookie turns RO. There can be multiple ways bookie can 
> start compaction including isForceGCAllowWhenNoSpace enabled or by other 
> means. When this happens the compaction process need to create a new entrylog 
> to make progress on compaction. Creation of entrylog will fail because there 
> are no writable ledger dirs.
> This is to allow entrylog creation if there is enough disk space even though 
> used space is above threshold. Because after compaction space will stay the 
> same or go below the threshold.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-943) Reduce log level of AbstractZkLedgerManager for register/unregister ReadOnlyLedgerHandle

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817408#comment-15817408
 ] 

ASF GitHub Bot commented on BOOKKEEPER-943:
---

Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/98


> Reduce log level of AbstractZkLedgerManager for register/unregister 
> ReadOnlyLedgerHandle
> 
>
> Key: BOOKKEEPER-943
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-943
> Project: Bookkeeper
>  Issue Type: Wish
>  Components: bookkeeper-client
>Affects Versions: 4.4.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Critical
> Fix For: 4.5.0
>
>
> Every time a ledger is opened/closed at least two lines at level INFO are 
> written to the application log.
> In the case of "tailing" a ledger I open/close ledgers many times an so 
> application log are filled up of this kind of lines:
> {code}
> Registered ledger metadata listener ReadOnlyLedgerHandle(lid = 29152, id = 
> 1405806205) on ledger 29152.
> Unregistered ledger metadata listener ReadOnlyLedgerHandle(lid = 29152, id = 
> 1405806205) on ledger 29152.
> {code}
> I would like to decrease the log level to "TRACE" instead of "INFO" in 
> AbstractZkLedgerManager.java
> {code}
> LOG.info("Registered ledger metadata listener {} on ledger {}.", listener, 
> ledgerId);
> LOG.info("Unregistered ledger metadata listener {} on ledger {}.", listener, 
> ledgerId);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-943) Reduce log level of AbstractZkLedgerManager for register/unregister ReadOnlyLedgerHandle

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817402#comment-15817402
 ] 

ASF GitHub Bot commented on BOOKKEEPER-943:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/98
  
+1 merging this.


> Reduce log level of AbstractZkLedgerManager for register/unregister 
> ReadOnlyLedgerHandle
> 
>
> Key: BOOKKEEPER-943
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-943
> Project: Bookkeeper
>  Issue Type: Wish
>  Components: bookkeeper-client
>Affects Versions: 4.4.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Critical
> Fix For: 4.5.0
>
>
> Every time a ledger is opened/closed at least two lines at level INFO are 
> written to the application log.
> In the case of "tailing" a ledger I open/close ledgers many times an so 
> application log are filled up of this kind of lines:
> {code}
> Registered ledger metadata listener ReadOnlyLedgerHandle(lid = 29152, id = 
> 1405806205) on ledger 29152.
> Unregistered ledger metadata listener ReadOnlyLedgerHandle(lid = 29152, id = 
> 1405806205) on ledger 29152.
> {code}
> I would like to decrease the log level to "TRACE" instead of "INFO" in 
> AbstractZkLedgerManager.java
> {code}
> LOG.info("Registered ledger metadata listener {} on ledger {}.", listener, 
> ledgerId);
> LOG.info("Unregistered ledger metadata listener {} on ledger {}.", listener, 
> ledgerId);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-991) bk shell - Get a list of all on disk files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817400#comment-15817400
 ] 

ASF GitHub Bot commented on BOOKKEEPER-991:
---

Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/100


> bk shell - Get a list of all on disk files
> --
>
> Key: BOOKKEEPER-991
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-991
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
> Fix For: 4.5.0
>
>
> BK saves the contents in multiple on disk files, journals, entrylog, and 
> index files.
> We need a way to list all these files in chronology order, so these files can 
> be given as input to other shell operations like ReadJournalCmd.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-992) ReadLog Command Enhancement

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817391#comment-15817391
 ] 

ASF GitHub Bot commented on BOOKKEEPER-992:
---

Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/102


> ReadLog Command Enhancement
> ---
>
> Key: BOOKKEEPER-992
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-992
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
> Fix For: 4.5.0
>
>
> - Take arguments lid and eid and prints just that entry
> - Take arguments of position range and print entries
> which are located in that position range



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-992) ReadLog Command Enhancement

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15817388#comment-15817388
 ] 

ASF GitHub Bot commented on BOOKKEEPER-992:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/102
  
looks good to me +1


> ReadLog Command Enhancement
> ---
>
> Key: BOOKKEEPER-992
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-992
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
>
> - Take arguments lid and eid and prints just that entry
> - Take arguments of position range and print entries
> which are located in that position range



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-963) Allow to use multiple journals in bookie

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802230#comment-15802230
 ] 

ASF GitHub Bot commented on BOOKKEEPER-963:
---

Github user reddycharan commented on the issue:

https://github.com/apache/bookkeeper/pull/71
  
new BookieShell command - ExpandStorageCmd (expandstorage) has been added 
by @rithin-shetty , it should also be updated as part of this change

BOOKKEEPER-948: Provide an option to add more ledger/index directories to a 
bookie

https://github.com/apache/bookkeeper/commit/66515e74d3fb03d21cad4eccb446a90ab7e94597


> Allow to use multiple journals in bookie
> 
>
> Key: BOOKKEEPER-963
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-963
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
> Fix For: 4.5.0
>
>
> By configuring multiple journals, we can take advantage of the IO of multiple
> disks to increase the write throughput of a single bookie.
> Each journal will have its own journal and sync threads and writes will be
> assigned to a particular journal by hashing on the ledger id.
> In addition to using multiple physical disks, there can improvements even by
> using multiple journal on a single SSD device, because these disks can handle
> well multiple concurrent writes in different blocks of the disk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-753) Bookie should run garbage collection before startup when all directories became full

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15798457#comment-15798457
 ] 

ASF GitHub Bot commented on BOOKKEEPER-753:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/81
  
I have checkout your code.
@sijie 
IMHO we can avoid using SettableFuture and switch to CompletableFuture


> Bookie should run garbage collection before startup when all directories 
> became full
> 
>
> Key: BOOKKEEPER-753
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-753
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: bookkeeper-server
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> otherwise, a bookie that ran-out-of disk would never be started up again.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-943) Reduce log level of AbstractZkLedgerManager for register/unregister ReadOnlyLedgerHandle

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15788313#comment-15788313
 ] 

ASF GitHub Bot commented on BOOKKEEPER-943:
---

Github user jvrao commented on the issue:

https://github.com/apache/bookkeeper/pull/98
  
+1 simple fix. :)


> Reduce log level of AbstractZkLedgerManager for register/unregister 
> ReadOnlyLedgerHandle
> 
>
> Key: BOOKKEEPER-943
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-943
> Project: Bookkeeper
>  Issue Type: Wish
>  Components: bookkeeper-client
>Affects Versions: 4.4.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Critical
> Fix For: 4.5.0
>
>
> Every time a ledger is opened/closed at least two lines at level INFO are 
> written to the application log.
> In the case of "tailing" a ledger I open/close ledgers many times an so 
> application log are filled up of this kind of lines:
> {code}
> Registered ledger metadata listener ReadOnlyLedgerHandle(lid = 29152, id = 
> 1405806205) on ledger 29152.
> Unregistered ledger metadata listener ReadOnlyLedgerHandle(lid = 29152, id = 
> 1405806205) on ledger 29152.
> {code}
> I would like to decrease the log level to "TRACE" instead of "INFO" in 
> AbstractZkLedgerManager.java
> {code}
> LOG.info("Registered ledger metadata listener {} on ledger {}.", listener, 
> ledgerId);
> LOG.info("Unregistered ledger metadata listener {} on ledger {}.", listener, 
> ledgerId);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-992) ReadLog Command Enhancement

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784133#comment-15784133
 ] 

ASF GitHub Bot commented on BOOKKEEPER-992:
---

GitHub user reddycharan opened a pull request:

https://github.com/apache/bookkeeper/pull/102

BOOKKEEPER-992: ReadLog Command Enhancement

- Take arguments lid and eid and prints just that entry
- Take arguments of position range and print entries
which are located in that position range

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/reddycharan/bookkeeper readlogcommand

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/102.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #102


commit 84bcf07a9621428ecb9045fb3acbda134832af41
Author: Charan Reddy Guttapalem 
Date:   2016-12-29T00:51:18Z

BOOKKEEPER-992: ReadLog Command Enhancement

- Take arguments lid and eid and prints just that entry
- Take arguments of position range and print entries
which are located in that position range




> ReadLog Command Enhancement
> ---
>
> Key: BOOKKEEPER-992
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-992
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
>
> - Take arguments lid and eid and prints just that entry
> - Take arguments of position range and print entries
> which are located in that position range



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-907) for ReadLedgerEntriesCmd, EntryFormatter should be configurable and HexDumpEntryFormatter should be one of them

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783965#comment-15783965
 ] 

ASF GitHub Bot commented on BOOKKEEPER-907:
---

GitHub user reddycharan opened a pull request:

https://github.com/apache/bookkeeper/pull/101

BOOKKEEPER-907: EntryFormatter should be configur.

for ReadLedgerEntriesCmd, EntryFormatter should
be configurable and HexDumpEntryFormatter should
be one of them.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/reddycharan/bookkeeper entryformatter

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/101.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #101


commit 5a4ee88ed416d23dec50fa5a680c599caa47d717
Author: Charan Reddy Guttapalem 
Date:   2016-12-28T23:05:04Z

BOOKKEEPER-907: EntryFormatter should be configur.

for ReadLedgerEntriesCmd, EntryFormatter should
be configurable and HexDumpEntryFormatter should
be one of them.




> for ReadLedgerEntriesCmd, EntryFormatter should be configurable and 
> HexDumpEntryFormatter should be one of them
> ---
>
> Key: BOOKKEEPER-907
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-907
> Project: Bookkeeper
>  Issue Type: Bug
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
>
> In ReadLedgerEntriesCmd for printing the contents of the Entry, HexDumpFormat 
> is used by default here, but whereas in ReadJournalCmd/ReadLogCmd 
> StringEntryFormatter is used and it is configurable. We should add 
> HexDumpEntryFormatter just like StringEntryFormatter and use the configured 
> formatter for printing the entry.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-991) bk shell - Get a list of all on disk files

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783803#comment-15783803
 ] 

ASF GitHub Bot commented on BOOKKEEPER-991:
---

GitHub user reddycharan opened a pull request:

https://github.com/apache/bookkeeper/pull/100

BOOKKEEPER-991: Get list of disk files

BKShell Command for 
Getting list of DiskFiles -
Journals/EntryLogs/IndexFiles

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/reddycharan/bookkeeper listfilesondisc

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/100.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #100


commit 70557e7d37059e304b5fcdb1f8cc5f8530338d0c
Author: Charan Reddy Guttapalem 
Date:   2016-03-08T02:07:22Z

BOOKKEEPER-991: Get list of disk files

Getting list of DiskFiles -
Journals/EntryLogs/IndexFiles




> bk shell - Get a list of all on disk files
> --
>
> Key: BOOKKEEPER-991
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-991
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
>
> BK saves the contents in multiple on disk files, journals, entrylog, and 
> index files.
> We need a way to list all these files in chronology order, so these files can 
> be given as input to other shell operations like ReadJournalCmd.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-874) Explict LAC from Writer to Bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15771149#comment-15771149
 ] 

ASF GitHub Bot commented on BOOKKEEPER-874:
---

Github user reddycharan commented on the issue:

https://github.com/apache/bookkeeper/pull/89
  
@sijie Addressed most of your comments. Please check the recent commit.

@jvrao FYI


> Explict LAC from Writer to Bookies
> --
>
> Key: BOOKKEEPER-874
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-874
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-client, bookkeeper-server
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Venkateswararao Jujjuri (JV)
>
> Current client API piggy-backs LAC with a write. This is keeps reader one 
> behind the writer. In order to keep reader up to date with writer even when 
> there is a pause in write, proposing the following:
> Writer sends explicit LAC on a configured timeout if there is no write within 
> that period.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-588) SSL support

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15767457#comment-15767457
 ] 

ASF GitHub Bot commented on BOOKKEEPER-588:
---

GitHub user eolivelli opened a pull request:

https://github.com/apache/bookkeeper/pull/97

BOOKKEEPER-588 SSL Support



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/eolivelli/bookkeeper 
BOOKKEEPER-588-ssl-support

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/97.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #97






> SSL support
> ---
>
> Key: BOOKKEEPER-588
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-588
> Project: Bookkeeper
>  Issue Type: Sub-task
>Reporter: Ivan Kelly
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
> Attachments: 0001-MutualTLS-for-Bookkeeper.patch, 
> 0004-BOOKKEEPER-588-SSL-support-for-bookkeeper.patch
>
>
> SSL support using startTLS



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-964) Add concurrent maps and sets for primitive types

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766623#comment-15766623
 ] 

ASF GitHub Bot commented on BOOKKEEPER-964:
---

GitHub user eolivelli opened a pull request:

https://github.com/apache/bookkeeper/pull/96

BOOKKEEPER-964 Ignore findbugs warnings



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/eolivelli/bookkeeper BOOKKEEPER-964-findbugs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/96.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #96


commit 3104c6fafcdafc208b7302fd3080f41a72a060a0
Author: eolivelli 
Date:   2016-12-21T09:46:30Z

Ignore findbugs warnings




> Add concurrent maps and sets for primitive types
> 
>
> Key: BOOKKEEPER-964
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-964
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
> Fix For: 4.5.0
>
>
> In BookKeeper there are many instances of maps and sets that use ledger id
> and entry ids as keys or values. JDK concurrent collections have the overhead
> of boxing all the primitive values into objects (eg: long --> Long) that would
> need to be allocated from the heap. In addition to that, JDK map 
> implementations
> are closed hash tables and they will require at least one more allocation to 
> hold
> the linked-list/tree node.
> There are already available libraries that offer primitive collections with
> zero-allocation, but none of these support concurrent maps/sets.
> We have added a handful of specializations, all based on the same 
> implementation
> idea. We have a hash table which is broken down into multiple sections. Each
> sections, on its own, is an open hash table with linear probing, protected by
> a stamped lock.
> All insertions, lookups and iterations on these collections are allocation 
> free.
> {noformat}
> ConcurrentLongHashMap: Map
> ConcurrentLongHashSet: Set
> ConcurrentLongLongHashMap: Map
> ConcurrentLongLongPairHashMap: Map< Pair, Pair >
> ConcurrentOpenHashMap: Map
> ConcurrentOpenHashSet: Set
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-983) BookieShell Command for LedgerDelete

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765695#comment-15765695
 ] 

ASF GitHub Bot commented on BOOKKEEPER-983:
---

Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/90


> BookieShell Command for LedgerDelete
> 
>
> Key: BOOKKEEPER-983
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-983
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
>
> BookieShell Command for LedgerDelete



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-950) Ledger placement policy to accomodate different storage capacity of bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765379#comment-15765379
 ] 

ASF GitHub Bot commented on BOOKKEEPER-950:
---

Github user rithin-shetty commented on the issue:

https://github.com/apache/bookkeeper/pull/93
  
@sijie thanks for pointing that out. I have now rebased and updated the 
branch. The merge conflict should be gone


> Ledger placement policy to accomodate different storage capacity of bookies
> ---
>
> Key: BOOKKEEPER-950
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-950
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Rithin Shetty
>Assignee: Rithin Shetty
> Fix For: 4.5.0
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> In our environment, in Salesforce, we are likely to have bookie nodes with 
> different storage capacity: some will have 1TB others might have 3TB. Also, 
> our ledgers are likely going to be long lived. The current ledger placement 
> policy selects the bookies randomly leading to uniform distribution. This 
> would cause some of bookies to reach high utilization while the rest would be 
> underutilized. We need a new ledger placement policy that has higher 
> probability of selecting bookies with higher free disk space than the ones 
> with lower disk free space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-391) Support Kerberos authentication of bookkeeper

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765365#comment-15765365
 ] 

ASF GitHub Bot commented on BOOKKEEPER-391:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/85
  
@eolivelli I will take a look at this once BOOKKEEPER-959 is in.


> Support Kerberos authentication of bookkeeper
> -
>
> Key: BOOKKEEPER-391
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-391
> Project: Bookkeeper
>  Issue Type: New Feature
>  Components: bookkeeper-client, bookkeeper-server
>Reporter: Rakesh R
>Assignee: Enrico Olivelli
> Fix For: 4.5.0
>
>
> This JIRA to discuss authentication mechanism of bookie clients and server. 
> Assume ZK provides fully secured communication channel using Kerberos based 
> authentication and authorization model. We could also manage and renew users 
> authenticated to BK via Kerberos. There is currently no configuration or 
> hooks for the Bookie process to obtain Kerberos credentials.
> Today an unauthenticated bookie client can easily establish connection with 
> the bookkeeper server. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-950) Ledger placement policy to accomodate different storage capacity of bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765341#comment-15765341
 ] 

ASF GitHub Bot commented on BOOKKEEPER-950:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/93
  
@rithin-shetty  sorry for late response. I will try to get to this one this 
week.

btw this pull request has conflicts on bookkeeper-server/pom.xml. do you 
mind rebasing it?


> Ledger placement policy to accomodate different storage capacity of bookies
> ---
>
> Key: BOOKKEEPER-950
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-950
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Rithin Shetty
>Assignee: Rithin Shetty
> Fix For: 4.5.0
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> In our environment, in Salesforce, we are likely to have bookie nodes with 
> different storage capacity: some will have 1TB others might have 3TB. Also, 
> our ledgers are likely going to be long lived. The current ledger placement 
> policy selects the bookies randomly leading to uniform distribution. This 
> would cause some of bookies to reach high utilization while the rest would be 
> underutilized. We need a new ledger placement policy that has higher 
> probability of selecting bookies with higher free disk space than the ones 
> with lower disk free space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-873) Enhance CreatedLedger API to accept ledgerId as input

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765340#comment-15765340
 ] 

ASF GitHub Bot commented on BOOKKEEPER-873:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/88
  
@reddycharan sorry for late response. I will try to get to this one this 
week.


> Enhance CreatedLedger API to accept ledgerId as input
> -
>
> Key: BOOKKEEPER-873
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-873
> Project: Bookkeeper
>  Issue Type: Sub-task
>  Components: bookkeeper-client
>Affects Versions: 4.3.1
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Venkateswararao Jujjuri (JV)
> Attachments: writeLedgerTest.NegativeLedgerId, 
> writeLedgerTest.PositiveLedgerId
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-986) Handle Memtable flush failure

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765338#comment-15765338
 ] 

ASF GitHub Bot commented on BOOKKEEPER-986:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/92
  
The change looks good to me. +1


> Handle Memtable flush failure
> -
>
> Key: BOOKKEEPER-986
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-986
> Project: Bookkeeper
>  Issue Type: Bug
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Critical
>
> Consider the following scenario
>  - currently bookie is in writemode but it is close to the diskusagethreshold 
> and also entrymemtable is close to skipListSizeLimit
>  - it received a addentry call, and now the size of the entrymemtable is 
> greater than skipListSizeLimit
>  - so onSizeLimitReached of SortedLedgerStorage will be called and now lets 
> assume that memtable flush has failed because of diskusagethreshold 
>  - because of previous step, bookie would turn to readonly
>  - now lets assume after certain rounds of compaction bookie reclaimed 
> sufficient storage and went back to writemode
>  - now bookie would be receiving addentry calls
>  - because of outstanding snapshot from previous memtable flush failure, it 
> will never retry to flush memtable 
>  - now the memory start keep going up and up and the process memory gets 
> bloated



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-971) update bk codahale stats provider version

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756039#comment-15756039
 ] 

ASF GitHub Bot commented on BOOKKEEPER-971:
---

Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/83


> update bk codahale stats provider version
> -
>
> Key: BOOKKEEPER-971
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-971
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Jia Zhai
>Assignee: Jia Zhai
> Fix For: 4.5.0
>
>
> update bk stats provider: from codahale to yammer.
> Currently io.dropwizard.metrics 3.1.0 is used most widely. will change 
> version to this version, and run the test.
> And would like to change CodahaleMetricsProvider.getMetrics() to public, 
> since this would be used outside package.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-987) BookKeeper build is broken due to the shade plugin for commit ecbb053e6e

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756028#comment-15756028
 ] 

ASF GitHub Bot commented on BOOKKEEPER-987:
---

Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/94


> BookKeeper build is broken due to the shade plugin for commit ecbb053e6e
> 
>
> Key: BOOKKEEPER-987
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-987
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: build
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Blocker
> Fix For: 4.5.0
>
>
> Step to reproduce:
> - clone BookKeeper repository from GitHub
> - checkout master (or at least commit 
> ecbb053e6e873859507e247cae727f4bc8b9f7fa)
> - mvn clean install -DskipTests -X
> The fix is to upgrade the shade plugin to the latest version (2.4.3)
> this is the error
> {code}
> [INFO] Excluding log4j:log4j:jar:1.2.15 from the shaded jar.
> [INFO] Minimizing jar 
> org.apache.bookkeeper:bookkeeper-server:jar:4.5.0-SNAPSHOT
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] bookkeeper . SUCCESS [  0.428 
> s]
> [INFO] compability dependencies ... SUCCESS [  0.016 
> s]
> [INFO] bookkeeper-server-compat400  SUCCESS [  2.772 
> s]
> [INFO] bookkeeper-server-compat410  SUCCESS [  1.159 
> s]
> [INFO] bookkeeper-server-compat420  SUCCESS [  1.370 
> s]
> [INFO] Stats API for bookkeeper ... SUCCESS [  0.422 
> s]
> [INFO] bookkeeper-server .. FAILURE [  4.002 
> s]
> [INFO] bookkeeper-benchmark ... SKIPPED
> [INFO] Stats provider for twitter-stats package ... SKIPPED
> [INFO] Stats provider for twitter-ostrich package . SKIPPED
> [INFO] Stats provider for codahale metrics  SKIPPED
> [INFO] bookkeeper-stats-providers . SKIPPED
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 10.328 s
> [INFO] Finished at: 2016-12-16T17:06:08+01:00
> [INFO] Final Memory: 46M/998M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-shade-plugin:2.1:shade (default) on project 
> bookkeeper-server: Error creating shaded jar: 46848 -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute 
> goal org.apache.maven.plugins:maven-shade-plugin:2.1:shade (default) on 
> project bookkeeper-server: Error creating shaded jar: 46848
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>   at 
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>   at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>   at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>   at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>   at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>   at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>   at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>   at 
> 

[jira] [Commented] (BOOKKEEPER-984) BookieClientTest.testWriteGaps tested

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756024#comment-15756024
 ] 

ASF GitHub Bot commented on BOOKKEEPER-984:
---

Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/91


>  BookieClientTest.testWriteGaps tested
> --
>
> Key: BOOKKEEPER-984
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-984
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: bookkeeper-server
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> This test is failed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-987) BookKeeper build is broken due to the shade plugin for commit ecbb053e6e

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756017#comment-15756017
 ] 

ASF GitHub Bot commented on BOOKKEEPER-987:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/94
  
the change looks good to me. +1

I am going to merge the changes to bring the ci back.


> BookKeeper build is broken due to the shade plugin for commit ecbb053e6e
> 
>
> Key: BOOKKEEPER-987
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-987
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: build
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Blocker
> Fix For: 4.5.0
>
>
> Step to reproduce:
> - clone BookKeeper repository from GitHub
> - checkout master (or at least commit 
> ecbb053e6e873859507e247cae727f4bc8b9f7fa)
> - mvn clean install -DskipTests -X
> The fix is to upgrade the shade plugin to the latest version (2.4.3)
> this is the error
> {code}
> [INFO] Excluding log4j:log4j:jar:1.2.15 from the shaded jar.
> [INFO] Minimizing jar 
> org.apache.bookkeeper:bookkeeper-server:jar:4.5.0-SNAPSHOT
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] bookkeeper . SUCCESS [  0.428 
> s]
> [INFO] compability dependencies ... SUCCESS [  0.016 
> s]
> [INFO] bookkeeper-server-compat400  SUCCESS [  2.772 
> s]
> [INFO] bookkeeper-server-compat410  SUCCESS [  1.159 
> s]
> [INFO] bookkeeper-server-compat420  SUCCESS [  1.370 
> s]
> [INFO] Stats API for bookkeeper ... SUCCESS [  0.422 
> s]
> [INFO] bookkeeper-server .. FAILURE [  4.002 
> s]
> [INFO] bookkeeper-benchmark ... SKIPPED
> [INFO] Stats provider for twitter-stats package ... SKIPPED
> [INFO] Stats provider for twitter-ostrich package . SKIPPED
> [INFO] Stats provider for codahale metrics  SKIPPED
> [INFO] bookkeeper-stats-providers . SKIPPED
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 10.328 s
> [INFO] Finished at: 2016-12-16T17:06:08+01:00
> [INFO] Final Memory: 46M/998M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-shade-plugin:2.1:shade (default) on project 
> bookkeeper-server: Error creating shaded jar: 46848 -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute 
> goal org.apache.maven.plugins:maven-shade-plugin:2.1:shade (default) on 
> project bookkeeper-server: Error creating shaded jar: 46848
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>   at 
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>   at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>   at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>   at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>   at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>   at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>   at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>   at 
> 

[jira] [Commented] (BOOKKEEPER-987) BookKeeper build is broken due to the shade plugin for commit ecbb053e6e

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755796#comment-15755796
 ] 

ASF GitHub Bot commented on BOOKKEEPER-987:
---

Github user rithin-shetty commented on the issue:

https://github.com/apache/bookkeeper/pull/94
  
I've seen this error too. The fix LGTM +1.


> BookKeeper build is broken due to the shade plugin for commit ecbb053e6e
> 
>
> Key: BOOKKEEPER-987
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-987
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: build
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Blocker
> Fix For: 4.5.0
>
>
> Step to reproduce:
> - clone BookKeeper repository from GitHub
> - checkout master (or at least commit 
> ecbb053e6e873859507e247cae727f4bc8b9f7fa)
> - mvn clean install -DskipTests -X
> The fix is to upgrade the shade plugin to the latest version (2.4.3)
> this is the error
> {code}
> [INFO] Excluding log4j:log4j:jar:1.2.15 from the shaded jar.
> [INFO] Minimizing jar 
> org.apache.bookkeeper:bookkeeper-server:jar:4.5.0-SNAPSHOT
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] bookkeeper . SUCCESS [  0.428 
> s]
> [INFO] compability dependencies ... SUCCESS [  0.016 
> s]
> [INFO] bookkeeper-server-compat400  SUCCESS [  2.772 
> s]
> [INFO] bookkeeper-server-compat410  SUCCESS [  1.159 
> s]
> [INFO] bookkeeper-server-compat420  SUCCESS [  1.370 
> s]
> [INFO] Stats API for bookkeeper ... SUCCESS [  0.422 
> s]
> [INFO] bookkeeper-server .. FAILURE [  4.002 
> s]
> [INFO] bookkeeper-benchmark ... SKIPPED
> [INFO] Stats provider for twitter-stats package ... SKIPPED
> [INFO] Stats provider for twitter-ostrich package . SKIPPED
> [INFO] Stats provider for codahale metrics  SKIPPED
> [INFO] bookkeeper-stats-providers . SKIPPED
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 10.328 s
> [INFO] Finished at: 2016-12-16T17:06:08+01:00
> [INFO] Final Memory: 46M/998M
> [INFO] 
> 
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-shade-plugin:2.1:shade (default) on project 
> bookkeeper-server: Error creating shaded jar: 46848 -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute 
> goal org.apache.maven.plugins:maven-shade-plugin:2.1:shade (default) on 
> project bookkeeper-server: Error creating shaded jar: 46848
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>   at 
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>   at 
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>   at 
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>   at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>   at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>   at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>   at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>   at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>   at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>   at 
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>  

[jira] [Commented] (BOOKKEEPER-970) Bump zookeeper version to 3.5

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15751528#comment-15751528
 ] 

ASF GitHub Bot commented on BOOKKEEPER-970:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/79
  
I have double checked and now a new compile-time dependency appeared: 
net.java.dev.javacc:javacc:jar:5.0:compile

I have already opened a JIRA for that problem 
https://issues.apache.org/jira/browse/ZOOKEEPER-2460
mean while can we exclude it from BK pom ?

thanks 


> Bump zookeeper version to 3.5
> -
>
> Key: BOOKKEEPER-970
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-970
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-client, bookkeeper-server
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> in DL, we need to leverage the asynchronous version of 'multi' in zookeeper. 
> so this jira is to bump the zookeeper version to 3.5 to support async multi.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-984) BookieClientTest.testWriteGaps tested

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15751518#comment-15751518
 ] 

ASF GitHub Bot commented on BOOKKEEPER-984:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/91
  
LGTM +1
The test is not failing anymore on my laptop


>  BookieClientTest.testWriteGaps tested
> --
>
> Key: BOOKKEEPER-984
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-984
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: bookkeeper-server
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> This test is failed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-950) Ledger placement policy to accomodate different storage capacity of bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749274#comment-15749274
 ] 

ASF GitHub Bot commented on BOOKKEEPER-950:
---

GitHub user rithin-shetty opened a pull request:

https://github.com/apache/bookkeeper/pull/93

BOOKKEEPER-950: Ledger placement policy to accomodate different stora…

…ge capacity of bookies

This change introduces Disk weight based ledger placement. Currently free 
disk space is the only supported
weight for a bookie. This change also introduces a new protocol message 
between bk client and server
called GET_BOOKIE_INFO. This message is used by the client to retrieve the 
free disk space info from
all the bookies. The existing placement policies: DefaultPlacementPolicy 
and RackAwarePlacementPolicy
have been enhanced to make use of the weight while selecting bookies. New 
test cases have been added to
test RackawarePlacement with weights. A new test class has been added to 
test the weight based selection
algorithm in a stand alone fashion.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rithin-shetty/bookkeeper 
weightBasedPlacementDec13

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/93.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #93


commit e78b11d46ae4c3bdd58082d2b4ffa6e9ce06c68d
Author: Rithin 
Date:   2016-12-14T04:56:15Z

BOOKKEEPER-950: Ledger placement policy to accomodate different storage 
capacity of bookies

This change introduces Disk weight based ledger placement. Currently free 
disk space is the only supported
weight for a bookie. This change also introduces a new protocol message 
between bk client and server
called GET_BOOKIE_INFO. This message is used by the client to retrieve the 
free disk space info from
all the bookies. The existing placement policies: DefaultPlacementPolicy 
and RackAwarePlacementPolicy
have been enhanced to make use of the weight while selecting bookies. New 
test cases have been added to
test RackawarePlacement with weights. A new test class has been added to 
test the weight based selection
algorithm in a stand alone fashion.




> Ledger placement policy to accomodate different storage capacity of bookies
> ---
>
> Key: BOOKKEEPER-950
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-950
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Rithin Shetty
>Assignee: Rithin Shetty
> Fix For: 4.5.0
>
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> In our environment, in Salesforce, we are likely to have bookie nodes with 
> different storage capacity: some will have 1TB others might have 3TB. Also, 
> our ledgers are likely going to be long lived. The current ledger placement 
> policy selects the bookies randomly leading to uniform distribution. This 
> would cause some of bookies to reach high utilization while the rest would be 
> underutilized. We need a new ledger placement policy that has higher 
> probability of selecting bookies with higher free disk space than the ones 
> with lower disk free space.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-874) Explict LAC from Writer to Bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15749189#comment-15749189
 ] 

ASF GitHub Bot commented on BOOKKEEPER-874:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/89
  
@jvrao 

isn't the lac buffer also just cached?

I am fine with the lac buffer (although that is not perfect, as I mentioned 
above).

i think the middle ground for this patch here is :
piggyback lac and explicit lac should be just two ways to advance lac. we 
should treat them as same for advancing lac in bookie.

- when an entry is added, it should advance the lac in FileInfo and trigger 
listeners about lac is advanced
- when an explicit lac flush happened, it should do the same thing.

does that work for you? 




> Explict LAC from Writer to Bookies
> --
>
> Key: BOOKKEEPER-874
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-874
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-client, bookkeeper-server
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Venkateswararao Jujjuri (JV)
>
> Current client API piggy-backs LAC with a write. This is keeps reader one 
> behind the writer. In order to keep reader up to date with writer even when 
> there is a pause in write, proposing the following:
> Writer sends explicit LAC on a configured timeout if there is no write within 
> that period.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-874) Explict LAC from Writer to Bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15747191#comment-15747191
 ] 

ASF GitHub Bot commented on BOOKKEEPER-874:
---

Github user jvrao commented on the issue:

https://github.com/apache/bookkeeper/pull/89
  
I totally agree that we don't need to have two lac.

We have piggyback LAC on the disk. Your change is to keep it in the cache.
The change we are introducing is digested LAC buffer. This is going to be
more up to date.

Here is what I would propose:
- Have piggyback on the disk; just like how we have it now.
- ExplicitLAC on the fileInfo as we have introduced.
- I am guessing we need to make this work with/without explicitLAC enabled.
If so we need a function in bookie to look at these two versions and get
the latest one.
- Let your Listeners/waiters work on the most updated/available LAC on the
bookie.

On Tue, Dec 13, 2016 at 6:46 PM, Sijie Guo  wrote:

> *@sijie* commented on this pull request.
> --
>
> In bookkeeper-server/src/main/java/org/apache/bookkeeper/
> bookie/FileInfo.java :
>
> > @@ -112,6 +114,29 @@ public long getSizeSinceLastwrite() {
>  return sizeSinceLastwrite;
>  }
>
> +public ByteBuffer getExplicitLac() {
>
> sure we can discuss this in this week's meeting.
>
> I think there should be only one LAC concept in the bookie server side.
> The LAC is indicating the last add confirmed for a ledger, no matter it is
> a piggyback lac or explicit lac. piggyback and explicit are just different
> ways on advancing the LAC. It should be unified. Otherwise, it is not 
going
> to scale.
>
> You patch here is about advancing lac while our change is about
> notifications when lac is advanced. they are not conflicted and are
> actually resolving different problems. I would like to see a unified
> solution - e.g have only one lac or lac buffer in the FileInfo. it would 
be
> updated each time lac is advanced (no matter it is advanced or 
piggybacked)
> and notify any listeners listening on the lac changes.
>
> Also, I don't like the approach that we did for digesting/checksumming the
> add entries, as it already made the server hard to verify
> digestion/checksum before persisting the entries. We should not use this
> approach for explicit lac. Since we are introducing new type of requests,
> we probably should use this chance to make things right.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> , or mute the thread
> 

> .
>



-- 
Jvrao
---
First they ignore you, then they laugh at you, then they fight you, then
you win. - Mahatma Gandhi



> Explict LAC from Writer to Bookies
> --
>
> Key: BOOKKEEPER-874
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-874
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-client, bookkeeper-server
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Venkateswararao Jujjuri (JV)
>
> Current client API piggy-backs LAC with a write. This is keeps reader one 
> behind the writer. In order to keep reader up to date with writer even when 
> there is a pause in write, proposing the following:
> Writer sends explicit LAC on a configured timeout if there is no write within 
> that period.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-963) Allow to use multiple journals in bookie

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746713#comment-15746713
 ] 

ASF GitHub Bot commented on BOOKKEEPER-963:
---

Github user jvrao commented on the issue:

https://github.com/apache/bookkeeper/pull/71
  
Crap I lost all my review comments; Will start from the beginning again. 


> Allow to use multiple journals in bookie
> 
>
> Key: BOOKKEEPER-963
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-963
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Matteo Merli
>Assignee: Matteo Merli
> Fix For: 4.5.0
>
>
> By configuring multiple journals, we can take advantage of the IO of multiple
> disks to increase the write throughput of a single bookie.
> Each journal will have its own journal and sync threads and writes will be
> assigned to a particular journal by hashing on the ledger id.
> In addition to using multiple physical disks, there can improvements even by
> using multiple journal on a single SSD device, because these disks can handle
> well multiple concurrent writes in different blocks of the disk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-986) Handle Memtable flush failure

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15746264#comment-15746264
 ] 

ASF GitHub Bot commented on BOOKKEEPER-986:
---

GitHub user reddycharan opened a pull request:

https://github.com/apache/bookkeeper/pull/92

BOOKKEEPER-986: Handle memtable flush failure

- If the memtable flush is failed previously then
for the next addEntry call it will try to flush the
existing snapshot

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/reddycharan/bookkeeper 
handlememtableflushfailure

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/92.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #92


commit bc77cedf56e9308bea1da280e16b855646479311
Author: Charan Reddy Guttapalem 
Date:   2016-09-13T18:44:33Z

BOOKKEEPER-986: Handle memtable flush failure

- If the memtable flush is failed previously then
for the next addEntry call it will try to flush the
existing snapshot




> Handle Memtable flush failure
> -
>
> Key: BOOKKEEPER-986
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-986
> Project: Bookkeeper
>  Issue Type: Bug
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Critical
>
> Consider the following scenario
>  - currently bookie is in writemode but it is close to the diskusagethreshold 
> and also entrymemtable is close to skipListSizeLimit
>  - it received a addentry call, and now the size of the entrymemtable is 
> greater than skipListSizeLimit
>  - so onSizeLimitReached of SortedLedgerStorage will be called and now lets 
> assume that memtable flush has failed because of diskusagethreshold 
>  - because of previous step, bookie would turn to readonly
>  - now lets assume after certain rounds of compaction bookie reclaimed 
> sufficient storage and went back to writemode
>  - now bookie would be receiving addentry calls
>  - because of outstanding snapshot from previous memtable flush failure, it 
> will never retry to flush memtable 
>  - now the memory start keep going up and up and the process memory gets 
> bloated



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-959) ClientAuthProvider and BookieAuthProvider Public API used Protobuf Shaded classes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744544#comment-15744544
 ] 

ASF GitHub Bot commented on BOOKKEEPER-959:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/67
  
ping @merlimat 


> ClientAuthProvider and BookieAuthProvider Public API used Protobuf Shaded 
> classes
> -
>
> Key: BOOKKEEPER-959
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-959
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.4.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Blocker
> Fix For: 4.5.0
>
>
> With 4.4.0 we introduced the ability to implement custom authentication 
> plugins.
> The new interfaces ClientAuthProvider and BookieAuthProvider depend on 
> ExtensionRegistry, which is a shaded dependency.
> As a consequence it is not possibile to implement any custom auth provider in 
> code outside the project, because shaded/relocated dependencies cannot be 
> used.
> We need to break the actual interface and introduce a new way to implement 
> such plugins in a portable way



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-984) BookieClientTest.testWriteGaps tested

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744467#comment-15744467
 ] 

ASF GitHub Bot commented on BOOKKEEPER-984:
---

GitHub user sijie opened a pull request:

https://github.com/apache/bookkeeper/pull/91

BOOKKEEPER-984: Fix BookieClientTest.testWriteGaps

in commit 9359d682a1598e30364eca6021d976f911e055b2, the third field (last 
add confirmed) in the message. changed the test to include the lac in the test 
so the test logic can work as expected.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/bookkeeper sijie/BOOKKEEPER-984

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/91.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #91


commit b80194677e1c04b0711d557a8d5bd25e536c1fbe
Author: Sijie Guo 
Date:   2016-12-13T07:43:31Z

Fix BookieClientTest.testWriteGaps




>  BookieClientTest.testWriteGaps tested
> --
>
> Key: BOOKKEEPER-984
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-984
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: bookkeeper-server
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> This test is failed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-983) BookieShell Command for LedgerDelete

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744472#comment-15744472
 ] 

ASF GitHub Bot commented on BOOKKEEPER-983:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/90
  
LGTM +1


> BookieShell Command for LedgerDelete
> 
>
> Key: BOOKKEEPER-983
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-983
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
>
> BookieShell Command for LedgerDelete



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-983) BookieShell Command for LedgerDelete

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743905#comment-15743905
 ] 

ASF GitHub Bot commented on BOOKKEEPER-983:
---

GitHub user reddycharan opened a pull request:

https://github.com/apache/bookkeeper/pull/90

BOOKKEEPER-983: BookieShell LedgerDelete Command

BookieShell Command for LedgerDelete

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/reddycharan/bookkeeper ledgerdeletecommand

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/90.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #90


commit e86fe0c7a2c5314f460901f0123d85f4177d4d78
Author: Charan Reddy Guttapalem 
Date:   2016-05-04T23:34:19Z

BOOKKEEPER-983: BookieShell LedgerDelete Command

BookieShell Command for LedgerDelete




> BookieShell Command for LedgerDelete
> 
>
> Key: BOOKKEEPER-983
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-983
> Project: Bookkeeper
>  Issue Type: New Feature
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
>
> BookieShell Command for LedgerDelete



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-966) change the bookieServer cmdline to make conf-file and option co-exist

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15741110#comment-15741110
 ] 

ASF GitHub Bot commented on BOOKKEEPER-966:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/75
  
@sijie @jiazhai OK.  I will create the new issue for advanced options.
This changes loooks good to me.
+1



> change the bookieServer cmdline to make conf-file and option co-exist
> -
>
> Key: BOOKKEEPER-966
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-966
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Jia Zhai
>Assignee: Jia Zhai
>
> Currently, when using bookieServer cmdline to start a bookie, you will either 
> give it a cofiguration file by "-c booke.conf"; or add some options like 
> "   " in a fix 
> sequential.
> It may not satisfy some of the requirement. So changed it to be co-exist for 
> configuration file and options.
> By this change, it will first use settings in configuration file; and then 
> use options to overwrite some of the settings, if there are some options 
> provided.
> Here is an example after this change:
> BookieServer -c bookie.conf -z localhost:2181 -m /bookkeeper/ledgers -p 3181 
> -j /mnt/journal -l "/mnt/ledger1 /mnt/ledger2 /mnt/ledger3”
> Here, in this command:
> -z is for “Zookeeper client instance”;
> -m is for "Zookeeper ledgers root path for bookies";
> -p is for "bookie service port exported";
> -j is for "bookie journal directory";
> -l is for "bookie ledgers directories".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-874) Explict LAC from Writer to Bookies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736442#comment-15736442
 ] 

ASF GitHub Bot commented on BOOKKEEPER-874:
---

GitHub user reddycharan opened a pull request:

https://github.com/apache/bookkeeper/pull/89

BOOKKEEPER-874: Explict LAC from Writer to Bookies

Introduce a new feature for sending explicit LAC to bookies.
A writable LedgerHandle creates a timer thread to send explicit LACs
at the intervals specified through configuration paramenter,
explicitLacInterval. If this is set to zero, this feature is disabled,
no timer thread is created.

Explicit LAC is sent only if the client did not get a chance to send
LAC through piggyback method for "explicitLacInterval" time.
To implement this, introduced two new protocol messages to the
Bookkeeper protocol -  WRITE_LAC and READ_LAC, in addition to its
current READ_ENTRY and ADD_ENTRY.

Reviewed-by: Charan Reddy Guttapalem 
Signed-off-by: Venkateswararao Jujjuri (JV) 
Signed-off-by: Charan Reddy Guttapalem 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/reddycharan/bookkeeper explicitlacsinglecommit

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bookkeeper/pull/89.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #89


commit d1db9f84aa17eaed97ed81304beb3afe35c4475f
Author: JV 
Date:   2015-10-22T05:23:25Z

BOOKKEEPER-874: Explict LAC from Writer to Bookies

Introduce a new feature for sending explicit LAC to bookies.
A writable LedgerHandle creates a timer thread to send explicit LACs
at the intervals specified through configuration paramenter,
explicitLacInterval. If this is set to zero, this feature is disabled,
no timer thread is created.

Explicit LAC is sent only if the client did not get a chance to send
LAC through piggyback method for "explicitLacInterval" time.
To implement this, introduced two new protocol messages to the
Bookkeeper protocol -  WRITE_LAC and READ_LAC, in addition to its
current READ_ENTRY and ADD_ENTRY.

Reviewed-by: Charan Reddy Guttapalem 
Signed-off-by: Venkateswararao Jujjuri (JV) 
Signed-off-by: Charan Reddy Guttapalem 




> Explict LAC from Writer to Bookies
> --
>
> Key: BOOKKEEPER-874
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-874
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-client, bookkeeper-server
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Venkateswararao Jujjuri (JV)
>
> Current client API piggy-backs LAC with a write. This is keeps reader one 
> behind the writer. In order to keep reader up to date with writer even when 
> there is a pause in write, proposing the following:
> Writer sends explicit LAC on a configured timeout if there is no write within 
> that period.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-852) Release LedgerDescriptor and master-key objects when not used anymore

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15730978#comment-15730978
 ] 

ASF GitHub Bot commented on BOOKKEEPER-852:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/78
  
I guess this change depends on the other changes. I will try to get the 
other changes in today.


> Release LedgerDescriptor and master-key objects when not used anymore
> -
>
> Key: BOOKKEEPER-852
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-852
> Project: Bookkeeper
>  Issue Type: Bug
>Affects Versions: 4.3.0
>Reporter: Matteo Merli
>Assignee: Matteo Merli
> Fix For: 4.5.0
>
> Attachments: 
> 0001-BOOKKEEPER-852-Release-LedgerDescriptor-and-master-k.patch
>
>
> Maps with ledger descriptors and master-keys are not cleaned after a ledger 
> gets deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-959) ClientAuthProvider and BookieAuthProvider Public API used Protobuf Shaded classes

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15730973#comment-15730973
 ] 

ASF GitHub Bot commented on BOOKKEEPER-959:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/67
  
ping @merlimat 


> ClientAuthProvider and BookieAuthProvider Public API used Protobuf Shaded 
> classes
> -
>
> Key: BOOKKEEPER-959
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-959
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: bookkeeper-client, bookkeeper-server
>Affects Versions: 4.4.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Blocker
> Fix For: 4.5.0
>
>
> With 4.4.0 we introduced the ability to implement custom authentication 
> plugins.
> The new interfaces ClientAuthProvider and BookieAuthProvider depend on 
> ExtensionRegistry, which is a shaded dependency.
> As a consequence it is not possibile to implement any custom auth provider in 
> code outside the project, because shaded/relocated dependencies cannot be 
> used.
> We need to break the actual interface and introduce a new way to implement 
> such plugins in a portable way



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-967) Create new testsuite for testing RackAwareEnsemblePlacementPolicy using ScriptBasedMapping.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15730972#comment-15730972
 ] 

ASF GitHub Bot commented on BOOKKEEPER-967:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/86
  
LGTM +1


> Create new testsuite for testing RackAwareEnsemblePlacementPolicy using 
> ScriptBasedMapping.
> ---
>
> Key: BOOKKEEPER-967
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-967
> Project: Bookkeeper
>  Issue Type: Test
>Reporter: Charan Reddy Guttapalem
>Assignee: Charan Reddy Guttapalem
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-966) change the bookieServer cmdline to make conf-file and option co-exist

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15729066#comment-15729066
 ] 

ASF GitHub Bot commented on BOOKKEEPER-966:
---

Github user jiazhai commented on the issue:

https://github.com/apache/bookkeeper/pull/75
  
@eolivelli That idea is great. How about open another new issue to tracking 
this requirement?


> change the bookieServer cmdline to make conf-file and option co-exist
> -
>
> Key: BOOKKEEPER-966
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-966
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Jia Zhai
>Assignee: Jia Zhai
>
> Currently, when using bookieServer cmdline to start a bookie, you will either 
> give it a cofiguration file by "-c booke.conf"; or add some options like 
> "   " in a fix 
> sequential.
> It may not satisfy some of the requirement. So changed it to be co-exist for 
> configuration file and options.
> By this change, it will first use settings in configuration file; and then 
> use options to overwrite some of the settings, if there are some options 
> provided.
> Here is an example after this change:
> BookieServer -c bookie.conf -z localhost:2181 -m /bookkeeper/ledgers -p 3181 
> -j /mnt/journal -l "/mnt/ledger1 /mnt/ledger2 /mnt/ledger3”
> Here, in this command:
> -z is for “Zookeeper client instance”;
> -m is for "Zookeeper ledgers root path for bookies";
> -p is for "bookie service port exported";
> -j is for "bookie journal directory";
> -l is for "bookie ledgers directories".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BOOKKEEPER-966) change the bookieServer cmdline to make conf-file and option co-exist

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15729005#comment-15729005
 ] 

ASF GitHub Bot commented on BOOKKEEPER-966:
---

Github user sijie commented on the issue:

https://github.com/apache/bookkeeper/pull/75
  
@eolivelli +1 for your idea.


> change the bookieServer cmdline to make conf-file and option co-exist
> -
>
> Key: BOOKKEEPER-966
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-966
> Project: Bookkeeper
>  Issue Type: Improvement
>Reporter: Jia Zhai
>Assignee: Jia Zhai
>
> Currently, when using bookieServer cmdline to start a bookie, you will either 
> give it a cofiguration file by "-c booke.conf"; or add some options like 
> "   " in a fix 
> sequential.
> It may not satisfy some of the requirement. So changed it to be co-exist for 
> configuration file and options.
> By this change, it will first use settings in configuration file; and then 
> use options to overwrite some of the settings, if there are some options 
> provided.
> Here is an example after this change:
> BookieServer -c bookie.conf -z localhost:2181 -m /bookkeeper/ledgers -p 3181 
> -j /mnt/journal -l "/mnt/ledger1 /mnt/ledger2 /mnt/ledger3”
> Here, in this command:
> -z is for “Zookeeper client instance”;
> -m is for "Zookeeper ledgers root path for bookies";
> -p is for "bookie service port exported";
> -j is for "bookie journal directory";
> -l is for "bookie ledgers directories".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)