[jira] [Commented] (BOOKKEEPER-1041) Multiple active entrylogs

[Enrico Olivelli (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016981#comment-16016981
 ] 

Enrico Olivelli commented on BOOKKEEPER-1041:
-

[~jujjuri] [~reddychara...@gmail.com]
This sound very interesting. Now I can see clearly way JV wrote on the mailing 
list that maybe clients could send some hint to the bookies that a ledger has 
been gracefully deleted/closed


> Multiple active entrylogs
> -
>
> Key: BOOKKEEPER-1041
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1041
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Charan Reddy Guttapalem
>
> Current bookkeeper is tuned for rotational HDDs. It has one active entrylog, 
> and all the ledger/entries go to the same entrylog until it is rotated out. 
> This is perfect for HDDs as seeks and moving head allover the disk platter is 
> very expensive. But this is very inefficient for SSDs, as each SSD can handle 
> multiple parallel writers, also this method is extremely inefficient for 
> compaction as it causes write amplification and inefficient disk space usage.
> Our proposal is to have multiple active entrylogs and a configuration param 
> on how many parallel entrylogs the system can have. This way one can have 
> ability to configure to have less (may be  one) ledger per entrylog.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (BOOKKEEPER-1071) BookieRecoveryTest is failing due to a Netty4 IllegalReferenceCountException

[Matteo Merli (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matteo Merli reassigned BOOKKEEPER-1071:


Assignee: Matteo Merli

> BookieRecoveryTest is failing due to a Netty4 IllegalReferenceCountException
> 
>
> Key: BOOKKEEPER-1071
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1071
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: bookkeeper-server
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Matteo Merli
>Priority: Blocker
> Fix For: 4.5.0
>
>
> this is the killer commit:
> e44c7388399e5589cf44e38c58bb84c74da544af BOOKKEEPER-1069: If client uses V2 
> proto, set the connection to always decode V2 messages
> this commit was working:
> 0f81461d2d1dc5cf9db4de9a46599d7d64e3dac6 BOOKKEEPER-1048: Use ByteBuf in 
> LedgerStorage interface
> {code}
> 2017-05-18 04:50:39,691 - WARN  - [bookie-io-4:Slf4JLogger@151] - An 
> exceptionCaught() event was fired, and it reached at the tail of the 
> pipeline. It usually means the last handler in the pipeline did not handl
> e the exception.
> io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
> at 
> io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:101)
> at 
> io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:89)
> at 
> io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
> at 
> io.netty.channel.DefaultChannelPipeline.onUnhandledInboundMessage(DefaultChannelPipeline.java:1169)
> at 
> io.netty.channel.DefaultChannelPipeline$TailContext.channelRead(DefaultChannelPipeline.java:1221)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> org.apache.bookkeeper.proto.BookieRequestHandler.channelRead(BookieRequestHandler.java:77)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
> at 
> org.apache.bookkeeper.proto.AuthHandler$ServerSideHandler.channelRead(AuthHandler.java:90)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
> at 
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
> at 
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:1017)
> 

[jira] [Commented] (BOOKKEEPER-1071) BookieRecoveryTest is failing due to a Netty4 IllegalReferenceCountException

[Matteo Merli (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015905#comment-16015905
 ] 

Matteo Merli commented on BOOKKEEPER-1071:
--

Sure, looking at it.

> BookieRecoveryTest is failing due to a Netty4 IllegalReferenceCountException
> 
>
> Key: BOOKKEEPER-1071
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1071
> Project: Bookkeeper
>  Issue Type: Bug
>  Components: bookkeeper-server
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Matteo Merli
>Priority: Blocker
> Fix For: 4.5.0
>
>
> this is the killer commit:
> e44c7388399e5589cf44e38c58bb84c74da544af BOOKKEEPER-1069: If client uses V2 
> proto, set the connection to always decode V2 messages
> this commit was working:
> 0f81461d2d1dc5cf9db4de9a46599d7d64e3dac6 BOOKKEEPER-1048: Use ByteBuf in 
> LedgerStorage interface
> {code}
> 2017-05-18 04:50:39,691 - WARN  - [bookie-io-4:Slf4JLogger@151] - An 
> exceptionCaught() event was fired, and it reached at the tail of the 
> pipeline. It usually means the last handler in the pipeline did not handl
> e the exception.
> io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
> at 
> io.netty.buffer.AbstractReferenceCountedByteBuf.release0(AbstractReferenceCountedByteBuf.java:101)
> at 
> io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:89)
> at 
> io.netty.util.ReferenceCountUtil.release(ReferenceCountUtil.java:84)
> at 
> io.netty.channel.DefaultChannelPipeline.onUnhandledInboundMessage(DefaultChannelPipeline.java:1169)
> at 
> io.netty.channel.DefaultChannelPipeline$TailContext.channelRead(DefaultChannelPipeline.java:1221)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> org.apache.bookkeeper.proto.BookieRequestHandler.channelRead(BookieRequestHandler.java:77)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
> at 
> org.apache.bookkeeper.proto.AuthHandler$ServerSideHandler.channelRead(AuthHandler.java:90)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
> at 
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
> at 
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
> at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
> at 
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
> at 
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady

[jira] [Commented] (BOOKKEEPER-1041) Multiple active entrylogs

[Charan Reddy Guttapalem (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015895#comment-16015895
 ] 

Charan Reddy Guttapalem commented on BOOKKEEPER-1041:
-

Issue: 

In Bookie's EntryLogger, we are having only one current active entryLog and all 
the ledger/entries go to the same entrylog. This is perfect for HDDs as file 
syncs, seeks and moving head allover the disk platter is very expensive. But 
having single active Entry Log is  inefficient for SSDs, as each SSD can handle 
multiple parallel writers. Also, having single active EntryLog (irrespective of 
LedgerStorage type - interleaved/sorted), is inefficient for compaction, since 
entries of multiple ledgers will end up in an entrylog.

Also in SortedLedgerStorage , in the addEntry request we flush EntryMemtable, 
if it reaches the sizelimit. Because of this we are observing unpredictable 
tail latency for addEntry request. When EntryMemTable snapshot of size (64 MB) 
is flushed all at once, this may affect the journal addentry latency. Also, if 
the rate of new add requests surpasses the rate at which the EntryMemTable's 
previous snapshot is flushed, then at a point the current EntryMemTable map 
will reach the limit and since the previous snapshot flush is in progress, 
EntryMemTable will throttle new addRequests, which would affect addEntry 
latency.

Goals:

The main purpose of this feature is to have efficient Garbagecollection story 
by minimizing the amount of compactions required and the ability to reclaim the 
deleted ledger's space quicker. Also with this feature we can lay foreground 
for switching to InterleavedLedgerStorage from SortedLedgerStorage and get 
predictable tail latency. 

Proposal:

So proposal here is to have multiple active entrylogs. Which will help with 
compaction performance and make SortedLedgerStorage redundant.

Design Overview:

- is to have server configuration specifying number of active entry logs per 
ledgerdir.
- for backward compatibility (for existing behaviour) that config can be set to 
0. 
- round-robin method will be used for choosing the active entry log for the 
current ledger in EntryLogger.addEntry method
- if the total number of active entrylogs is more than or equal to number of 
active ledgers, then we get almost exclusivity
- For implementing Round-Robin approach, we need to maintain state information 
of mapping of ledgerId to SlotId
- there will be numberofledgerdirs*numberofactiveentrylogsperledgerdir slots. a 
slot is mapped to ledgerdir, but the activeentrylog of that slot will be 
rotated when it reaches the capacity.
- By knowing the SlotId we can get the corresponding entryLogId associated to 
that slot.
- If there is no entry for current ledger in the map, then we pick the next in 
order slot and add the mapping entry to the map.
- Since Bookie won't  be informed about the writeclose of the ledger, there is 
no easy way to know when to remove the mapping entry from the map. Considering 
it is just  mapentry, we may compromise on evicting 
policy. We can just use some Cache, which has eviction policy, timebased on 
last access
- If a ledgerdir becomes full, then all the slots having entrylogs in that 
ledgerdir, should become inactive. The existing mappings, mappings of active 
ledgers to these slots (active entrylogs), should be updated to available 
active slots.
- when ledgerdir becomes writable again, then the slots which were inactive 
should be made active and become eligible for round-robin distribution
- For this feature I need to make changes to checkpoint logic. Currently with 
BOOKKEEPER-564 change, we are scheduling checkpoint only when current entrylog 
file is rotated. So we dont call 'flushCurrentLog' when we checkpoint. But for 
this feature, since there are going to be multiple active entrylogs, scheduling 
checkpoint when entrylog file is rotated, is not an option. So I need to call 
flushCurrentLogs when checkpoint is made for every 'flushinterval' period

> Multiple active entrylogs
> -
>
> Key: BOOKKEEPER-1041
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1041
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Venkateswararao Jujjuri (JV)
>Assignee: Charan Reddy Guttapalem
>
> Current bookkeeper is tuned for rotational HDDs. It has one active entrylog, 
> and all the ledger/entries go to the same entrylog until it is rotated out. 
> This is perfect for HDDs as seeks and moving head allover the disk platter is 
> very expensive. But this is very inefficient for SSDs, as each SSD can handle 
> multiple parallel writers, also this method is extremely inefficient for 
> compaction as it causes write amplification and inefficient disk space usage.
> Our proposal is to have multiple act

[jira] [Commented] (BOOKKEEPER-1070) bk-merge-pr.py use apache-rat:check goal instead of rat:rat

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015815#comment-16015815
 ] 

ASF GitHub Bot commented on BOOKKEEPER-1070:


Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/158


> bk-merge-pr.py use apache-rat:check goal instead of rat:rat
> ---
>
> Key: BOOKKEEPER-1070
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1070
> Project: Bookkeeper
>  Issue Type: Task
>  Components: build
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Critical
> Fix For: 4.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] bookkeeper pull request #158: BOOKKEEPER-1070 bk-merge-pr.py use apache-rat:...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/bookkeeper/pull/158


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (BOOKKEEPER-1070) bk-merge-pr.py use apache-rat:check goal instead of rat:rat

[Enrico Olivelli (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BOOKKEEPER-1070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enrico Olivelli resolved BOOKKEEPER-1070.
-
Resolution: Fixed

Issue resolved by merging pull request 158
[https://github.com/apache/bookkeeper/pull/158]

{noformat}
commit 7da9ed293db87103efcc28b7e706c0b386131ec1
Author: Enrico Olivelli 
AuthorDate: Thu May 18 16:21:37 2017 +0200
Commit: eolivelli 
CommitDate: Thu May 18 16:21:37 2017 +0200

BOOKKEEPER-1070: bk-merge-pr.py use apache-rat:check goal

Author: eolivelli 

Reviewers: Jia Zhai 

Closes #158 from eolivelli/BOOKKEEPER-1070-rat

{noformat}


> bk-merge-pr.py use apache-rat:check goal instead of rat:rat
> ---
>
> Key: BOOKKEEPER-1070
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-1070
> Project: Bookkeeper
>  Issue Type: Task
>  Components: build
>Affects Versions: 4.5.0
>Reporter: Enrico Olivelli
>Assignee: Enrico Olivelli
>Priority: Critical
> Fix For: 4.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Build failed in Jenkins: bookkeeper-master #1753

[Apache Jenkins Server]

See 

--
[...truncated 30.01 KB...]
Running org.apache.bookkeeper.client.TestBookieHealthCheck
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.133 sec - in 
org.apache.bookkeeper.client.TestBookieHealthCheck
Running org.apache.bookkeeper.client.TestGetBookieInfoTimeout
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.839 sec - in 
org.apache.bookkeeper.client.TestGetBookieInfoTimeout
Running org.apache.bookkeeper.client.RoundRobinDistributionScheduleTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.194 sec - in 
org.apache.bookkeeper.client.RoundRobinDistributionScheduleTest
Running org.apache.bookkeeper.client.UpdateLedgerCmdTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.983 sec - in 
org.apache.bookkeeper.client.UpdateLedgerCmdTest
Running org.apache.bookkeeper.client.TestLedgerFragmentReplication
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.433 sec - in 
org.apache.bookkeeper.client.TestLedgerFragmentReplication
Running org.apache.bookkeeper.client.BookieRecoveryTest
Tests run: 120, Failures: 0, Errors: 6, Skipped: 0, Time elapsed: 255.749 sec 
<<< FAILURE! - in org.apache.bookkeeper.client.BookieRecoveryTest
ensurePasswordUsedForOldLedgers[0](org.apache.bookkeeper.client.BookieRecoveryTest)
  Time elapsed: 34.022 sec  <<< ERROR!
org.apache.bookkeeper.client.BKException$BKLedgerRecoveryException
at 
org.apache.bookkeeper.client.BookieRecoveryTest.ensurePasswordUsedForOldLedgers(BookieRecoveryTest.java:995)

ensurePasswordUsedForOldLedgers[1](org.apache.bookkeeper.client.BookieRecoveryTest)
  Time elapsed: 26.398 sec  <<< ERROR!
org.apache.bookkeeper.client.BKException$BKLedgerRecoveryException
at 
org.apache.bookkeeper.client.BookieRecoveryTest.ensurePasswordUsedForOldLedgers(BookieRecoveryTest.java:995)

ensurePasswordUsedForOldLedgers[2](org.apache.bookkeeper.client.BookieRecoveryTest)
  Time elapsed: 33.44 sec  <<< ERROR!
org.apache.bookkeeper.client.BKException$BKLedgerRecoveryException
at 
org.apache.bookkeeper.client.BookieRecoveryTest.ensurePasswordUsedForOldLedgers(BookieRecoveryTest.java:995)

ensurePasswordUsedForOldLedgers[3](org.apache.bookkeeper.client.BookieRecoveryTest)
  Time elapsed: 26.256 sec  <<< ERROR!
org.apache.bookkeeper.client.BKException$BKLedgerRecoveryException
at 
org.apache.bookkeeper.client.BookieRecoveryTest.ensurePasswordUsedForOldLedgers(BookieRecoveryTest.java:995)

ensurePasswordUsedForOldLedgers[4](org.apache.bookkeeper.client.BookieRecoveryTest)
  Time elapsed: 33.431 sec  <<< ERROR!
org.apache.bookkeeper.client.BKException$BKLedgerRecoveryException
at 
org.apache.bookkeeper.client.BookieRecoveryTest.ensurePasswordUsedForOldLedgers(BookieRecoveryTest.java:995)

ensurePasswordUsedForOldLedgers[5](org.apache.bookkeeper.client.BookieRecoveryTest)
  Time elapsed: 26.367 sec  <<< ERROR!
org.apache.bookkeeper.client.BKException$BKLedgerRecoveryException
at 
org.apache.bookkeeper.client.BookieRecoveryTest.ensurePasswordUsedForOldLedgers(BookieRecoveryTest.java:995)

Running org.apache.bookkeeper.client.LedgerCloseTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.177 sec - in 
org.apache.bookkeeper.client.LedgerCloseTest
Running org.apache.bookkeeper.client.LedgerRecoveryTest
Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.105 sec - 
in org.apache.bookkeeper.client.LedgerRecoveryTest
Running org.apache.bookkeeper.client.TestTryReadLastConfirmed
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.666 sec - in 
org.apache.bookkeeper.client.TestTryReadLastConfirmed
Running org.apache.bookkeeper.client.UpdateLedgerOpTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.915 sec - in 
org.apache.bookkeeper.client.UpdateLedgerOpTest
Running org.apache.bookkeeper.client.TestWeightedRandomSelection
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.167 sec - in 
org.apache.bookkeeper.client.TestWeightedRandomSelection
Running org.apache.bookkeeper.client.TestBookieWatcher
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.139 sec - in 
org.apache.bookkeeper.client.TestBookieWatcher
Running org.apache.bookkeeper.client.BookieWriteLedgerTest
Tests run: 120, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 97.112 sec - 
in org.apache.bookkeeper.client.BookieWriteLedgerTest
Running org.apache.bookkeeper.client.GenericEnsemblePlacementPolicyTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.739 sec - in 
org.apache.bookkeeper.client.GenericEnsemblePlacementPolicyTest
Running org.apache.bookkeeper.client.TestAddEntryQuorumTimeout
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.948 sec - in 
org.apache.bookkeeper.client.TestAddEntryQuorumTimeout
Running org.apache.bookkeeper.client.BookKeepe

[jira] [Commented] (BOOKKEEPER-816) use native fallocate & sync_file_range to improve journal allocation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015588#comment-16015588
 ] 

ASF GitHub Bot commented on BOOKKEEPER-816:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/80
  
@jiazhai try to install package cmake then try again, maybe you will get 
new errors, and you will need to install locally additional libraries



> use native fallocate & sync_file_range to improve journal allocation
> 
>
> Key: BOOKKEEPER-816
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-816
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> it'd better to leverage filesystem fallocate & sync_file_range for journal 
> performance.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] bookkeeper issue #80: BOOKKEEPER-816: use native fallocate to improve journa...

[eolivelli]

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/80
  
@jiazhai try to install package cmake then try again, maybe you will get 
new errors, and you will need to install locally additional libraries



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BOOKKEEPER-816) use native fallocate & sync_file_range to improve journal allocation

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BOOKKEEPER-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015585#comment-16015585
 ] 

ASF GitHub Bot commented on BOOKKEEPER-816:
---

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/80
  
@sijie 
I have pushed here a fix for builds on Fedora and Centos

can you pick up this commit too (if it sounds look good to you) ?
https://github.com/eolivelli/bookkeeper/tree/bookkeeper-816-native

maybe you could also add this property to the surefire configuration in 
order to use native libraries in JUnit tests if present

`-Djava.library.path=${project.build.directory}/native/target/usr/local/lib`


> use native fallocate & sync_file_range to improve journal allocation
> 
>
> Key: BOOKKEEPER-816
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-816
> Project: Bookkeeper
>  Issue Type: Improvement
>  Components: bookkeeper-server
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 4.5.0
>
>
> it'd better to leverage filesystem fallocate & sync_file_range for journal 
> performance.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] bookkeeper issue #80: BOOKKEEPER-816: use native fallocate to improve journa...

[eolivelli]

Github user eolivelli commented on the issue:

https://github.com/apache/bookkeeper/pull/80
  
@sijie 
I have pushed here a fix for builds on Fedora and Centos

can you pick up this commit too (if it sounds look good to you) ?
https://github.com/eolivelli/bookkeeper/tree/bookkeeper-816-native

maybe you could also add this property to the surefire configuration in 
order to use native libraries in JUnit tests if present

`-Djava.library.path=${project.build.directory}/native/target/usr/local/lib`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---