[jira] [Commented] (DL-135) Cleanup System.out and System.err usage

[Liang Xie (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15744081#comment-15744081
 ] 

Liang Xie commented on DL-135:
--

A trivial patch :  https://github.com/apache/incubator-distributedlog/pull/70

> Cleanup System.out and System.err usage
> ---
>
> Key: DL-135
> URL: https://issues.apache.org/jira/browse/DL-135
> Project: DistributedLog
>  Issue Type: Task
>Reporter: Sijie Guo
>Assignee: Liang Xie
>  Labels: help-wanted
>
> Based on the code analysis here : 
> https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/
> There are 272 calls to System.out and System.err usage. Most of them might 
> just come from distributedlog tool. But let's still audit the code to make 
> sure we are using System.out and System.err correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #70: cleanup System.out

[xieliang]

GitHub user xieliang opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/70

cleanup System.out



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xieliang/incubator-distributedlog DL-135

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/70.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #70


commit 94b4dfbc52c7f31c41f4202c4ab7ddb560fc0812
Author: xieliang 
Date:   2016-12-13T04:03:21Z

cleanup System.out




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-135) Cleanup System.out and System.err usage

[Liang Xie (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15744071#comment-15744071
 ] 

Liang Xie commented on DL-135:
--

cd incubator-distributedlog ,  grep -R "System.err" .  shows all of them 
are from *Tool , *Admin , *Benchmark, distributedlog-tutorials*,  so no further 
work need to be done.

and grep -R "System.out" seems shows only one line code needs to be modified 
(all others are from Tool class or tutorials)

> Cleanup System.out and System.err usage
> ---
>
> Key: DL-135
> URL: https://issues.apache.org/jira/browse/DL-135
> Project: DistributedLog
>  Issue Type: Task
>Reporter: Sijie Guo
>  Labels: help-wanted
>
> Based on the code analysis here : 
> https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/
> There are 272 calls to System.out and System.err usage. Most of them might 
> just come from distributedlog tool. But let's still audit the code to make 
> sure we are using System.out and System.err correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (DL-135) Cleanup System.out and System.err usage

[Liang Xie (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DL-135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liang Xie reassigned DL-135:


Assignee: Liang Xie

> Cleanup System.out and System.err usage
> ---
>
> Key: DL-135
> URL: https://issues.apache.org/jira/browse/DL-135
> Project: DistributedLog
>  Issue Type: Task
>Reporter: Sijie Guo
>Assignee: Liang Xie
>  Labels: help-wanted
>
> Based on the code analysis here : 
> https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/
> There are 272 calls to System.out and System.err usage. Most of them might 
> just come from distributedlog tool. But let's still audit the code to make 
> sure we are using System.out and System.err correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DL-97) Remove unused methods in BKLogHandler

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-97?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743940#comment-15743940
 ] 

ASF GitHub Bot commented on DL-97:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/69

DL-97: Remove unused methods in BKLogHandler

merge code change on remove unused methods in BKLogHandler.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-97

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/69.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #69


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998

commit f18fe172fb4559bcabb7173d1cd0b9d27fc23c93
Author: Sijie Guo 
Date:   2016-07-11T17:21:44Z

Make the zookeeper client used by bookke

[GitHub] incubator-distributedlog pull request #69: DL-97: Remove unused methods in B...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/69

DL-97: Remove unused methods in BKLogHandler

merge code change on remove unused methods in BKLogHandler.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-97

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/69.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #69


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998

commit f18fe172fb4559bcabb7173d1cd0b9d27fc23c93
Author: Sijie Guo 
Date:   2016-07-11T17:21:44Z

Make the zookeeper client used by bookkeeper client retry on session expires

* the zookeeper client used by bookkeeper client is purely for metadata 
accesses, so we should retry on session expires.
* remove the unnessary zookeeper session handling in bk log handler. 

[jira] [Commented] (DL-94) Make the zookeeper client used by bookkeeper client retry on session expires

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743935#comment-15743935
 ] 

ASF GitHub Bot commented on DL-94:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/67

DL-94: Make the zookeeper client used by bookkeeper client retry on session 
expires

merge twitter's change on session expires

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-94

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/67.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #67


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998

commit f18fe172fb4559bcabb7173d1cd0b9d27fc23c93
Author: Sijie Guo 
Date:   2016-07-11T17:21:44Z

Make the zookeeper

[jira] [Commented] (DL-95) Assign host ip client id if the client id is undefined

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-95?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743937#comment-15743937
 ] 

ASF GitHub Bot commented on DL-95:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/68

DL-95: Assign host ip client id if the client id is undefined

merge twitter's change on client id

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-95

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/68.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #68


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998

commit f18fe172fb4559bcabb7173d1cd0b9d27fc23c93
Author: Sijie Guo 
Date:   2016-07-11T17:21:44Z

Make the zookeeper client used by bookkeeper cl

[GitHub] incubator-distributedlog pull request #68: DL-95: Assign host ip client id i...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/68

DL-95: Assign host ip client id if the client id is undefined

merge twitter's change on client id

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-95

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/68.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #68


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998

commit f18fe172fb4559bcabb7173d1cd0b9d27fc23c93
Author: Sijie Guo 
Date:   2016-07-11T17:21:44Z

Make the zookeeper client used by bookkeeper client retry on session expires

* the zookeeper client used by bookkeeper client is purely for metadata 
accesses, so we should retry on session expires.
* remove the unnessary zookeeper session handling in bk log handler. as we 

[jira] [Commented] (DL-92) Simplify the state transition on stream

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-92?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743932#comment-15743932
 ] 

ASF GitHub Bot commented on DL-92:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/65

DL-92: Simplify the state transition on stream

merge twitter's change on simplifying the stream state transition

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-92

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/65.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #65


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047




> Simplify the state transition on stream
> ---
>
> Key: DL-92
> URL: https://issues.apache.org/jira/browse/DL-92
> Project: DistributedLog
>  Issue Type: Improvement
>  Components: distributedlog-service
>Affects Versions: 0.4.0
>Reporter: Sijie Guo
>Assignee: Leigh Stewart
>
> Simplify the state transition on stream
> * the stream is created on INITIALIZING
> * when the stream is started, it would start transition from INITIALIZING 
> to INITIALIZED
> * it would serve stream operations when the stream is INITIALIZED
> * it would be turned to ERROR when encountered exceptions.
> * the stream would be closed when service operation timeout or 
> encountered any exceptions. it would first be removed from  acquired mapping
> * the stream would be removed from cached mapping depends on probation 
> time.



--
This mess

[GitHub] incubator-distributedlog pull request #67: DL-94: Make the zookeeper client ...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/67

DL-94: Make the zookeeper client used by bookkeeper client retry on session 
expires

merge twitter's change on session expires

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-94

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/67.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #67


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998

commit f18fe172fb4559bcabb7173d1cd0b9d27fc23c93
Author: Sijie Guo 
Date:   2016-07-11T17:21:44Z

Make the zookeeper client used by bookkeeper client retry on session expires

* the zookeeper client used by bookkeeper client is purely for metadata 
accesses, so we should retry on session expires.
* remove the unnessary zookeeper session handli

[GitHub] incubator-distributedlog pull request #66: DL-93: LogSegmentMetadataStore sh...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/66

DL-93: LogSegmentMetadataStore should only notify when the list of log 
segments is updated

merge twitter's change on log segment metadata store.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-93

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/66.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #66


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-93) LogSegmentMetadataStore should only notify when the list of log segments is updated

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743934#comment-15743934
 ] 

ASF GitHub Bot commented on DL-93:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/66

DL-93: LogSegmentMetadataStore should only notify when the list of log 
segments is updated

merge twitter's change on log segment metadata store.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-93

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/66.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #66


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998




> LogSegmentMetadataStore should only notify when the list of log segments is 
> updated
> --

[GitHub] incubator-distributedlog pull request #65: DL-92: Simplify the state transit...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/65

DL-92: Simplify the state transition on stream

merge twitter's change on simplifying the stream state transition

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-92

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/65.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #65


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-91) Be able to close writer within a timeout duration

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743929#comment-15743929
 ] 

ASF GitHub Bot commented on DL-91:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/64

DL-91: Be able to close writer within a timeout duration

merge twitter's change on closing writer.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-91

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/64.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #64


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340




> Be able to close writer within a timeout duration
> -
>
> Key: DL-91
> URL: https://issues.apache.org/jira/browse/DL-91
> Project: DistributedLog
>  Issue Type: Bug
>  Components: distributedlog-client
>Reporter: Sijie Guo
>Assignee: Leigh Stewart
>
> If a stream is hanging on closing the writer on write proxy, it will cause 
> STEAM_UNAVAILABLE.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DL-91) Be able to close writer within a timeout duration

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743927#comment-15743927
 ] 

ASF GitHub Bot commented on DL-91:
--

Github user sijie closed the pull request at:

https://github.com/apache/incubator-distributedlog/pull/63


> Be able to close writer within a timeout duration
> -
>
> Key: DL-91
> URL: https://issues.apache.org/jira/browse/DL-91
> Project: DistributedLog
>  Issue Type: Bug
>  Components: distributedlog-client
>Reporter: Sijie Guo
>Assignee: Leigh Stewart
>
> If a stream is hanging on closing the writer on write proxy, it will cause 
> STEAM_UNAVAILABLE.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #64: DL-91: Be able to close writer wi...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/64

DL-91: Be able to close writer within a timeout duration

merge twitter's change on closing writer.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-91

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/64.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #64


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-91) Be able to close writer within a timeout duration

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743924#comment-15743924
 ] 

ASF GitHub Bot commented on DL-91:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/63

DL-91: Be able to close writer within a timeout duration

merge twitter's change on closing writer.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-96

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/63.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #63


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998

commit f18fe172fb4559bcabb7173d1cd0b9d27fc23c93
Author: Sijie Guo 
Date:   2016-07-11T17:21:44Z

Make the zookeeper client used by bookkeeper c

[jira] [Commented] (DL-89) add flag to enable thrift mux on DL monitor service

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743920#comment-15743920
 ] 

ASF GitHub Bot commented on DL-89:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/61

DL-89: add flag to enable thrift mux on DL monitor service

merge twitter's change from David Rusek.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-89

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/61.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #61


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555




> add flag to enable thrift mux on DL monitor service
> ---
>
> Key: DL-89
> URL: https://issues.apache.org/jira/browse/DL-89
> Project: DistributedLog
>  Issue Type: Bug
>  Components: distributedlog-service
>Reporter: Sijie Guo
>Assignee: David Rusek
> Fix For: 0.4.0
>
>
> provide the ability to enable thriftmux on monitor service



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #63: DL-91: Be able to close writer wi...

[sijie]

Github user sijie closed the pull request at:

https://github.com/apache/incubator-distributedlog/pull/63


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-distributedlog pull request #63: DL-91: Be able to close writer wi...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/63

DL-91: Be able to close writer within a timeout duration

merge twitter's change on closing writer.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-96

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/63.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #63


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled

commit 0a18f564ba7e4d03f4c6a0859e5478535a59befd
Author: Leigh Stewart 
Date:   2016-12-13T00:46:11Z

Be able to close the writer within a timeout period

RB_ID=841340

commit f19e7564ff4a1ec1b5d6f2683db190d739df99bb
Author: Leigh Stewart 
Date:   2016-12-13T00:49:26Z

Simplify the state transition on stream

* the stream is created on INITIALIZING
* when the stream is started, it would start transition from INITIALIZING 
to INITIALIZED
* it would serve stream operations when the stream is INITIALIZED
* it would be turned to ERROR when encountered exceptions.
* the stream would be closed when service operation timeout or encountered 
any exceptions. it would first be removed from acquired mapping
* the stream would be removed from cached mapping depends on probation time.

RB_ID=848047

commit f008f751ec0a020a670903b01c709007ada87381
Author: Sijie Guo 
Date:   2016-07-08T22:22:55Z

Allow configuring flush interval in micros for distributedlog multi streams 
writer

RB_ID=846974

commit 72a786e78540fb5a3f8e27f9d71d3616d06d1548
Author: Sijie Guo 
Date:   2016-07-11T17:10:55Z

LogSegmentMetadataStore should only notify when the list of log segments is 
updated

Currently it notifies the listeners not only when there is a change but 
also when session expires. it would break the readahead loop and cause readers 
have to wait until it is able to connect to zookeeper again.

With this change, it would only notify when the list of log segments is 
updated. If it disconnects to zookeeper, the listener won't be notified and it 
would keep reading from the log segments it knows.

RB_ID=842998

commit f18fe172fb4559bcabb7173d1cd0b9d27fc23c93
Author: Sijie Guo 
Date:   2016-07-11T17:21:44Z

Make the zookeeper client used by bookkeeper client retry on session expires

* the zookeeper client used by bookkeeper client is purely for metadata 
accesses, so we should retry on session expires.
* remove the unnessary zookeeper session handling in bk log handler. as we 

[jira] [Commented] (DL-90) Don't use stack and codec together for configuring thriftmux

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-90?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743921#comment-15743921
 ] 

ASF GitHub Bot commented on DL-90:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/62

DL-90: Don't use stack and codec together for configuring thriftmux

merge Twitter's change on thriftmux

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-90

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/62.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #62


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled




> Don't use stack and codec together for configuring thriftmux
> 
>
> Key: DL-90
> URL: https://issues.apache.org/jira/browse/DL-90
> Project: DistributedLog
>  Issue Type: Bug
>  Components: distributedlog-client, distributedlog-service
>Affects Versions: 0.4.0
>Reporter: Sijie Guo
>Assignee: David Rusek
>
> - Don't use stack and codec together for configuring thriftmux
> - Remove codec if the thriftmux is disabled



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #62: DL-90: Don't use stack and codec ...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/62

DL-90: Don't use stack and codec together for configuring thriftmux

merge Twitter's change on thriftmux

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-90

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/62.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #62


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555

commit 800b867b705a786094b9a591e0ebc64fa934f632
Author: Dave Rusek 
Date:   2016-12-13T00:41:47Z

Don't use stack and codec together for configuring thriftmux

- Don't use stack and codec together for configuring thriftmux
- Remove codec if the thriftmux is disabled




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-88) remove watches when unregister children watches

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-88?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743918#comment-15743918
 ] 

ASF GitHub Bot commented on DL-88:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/60

DL-88: remove watches when unregister children watches

merge twitter's change from Sijie Guo

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-88

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/60.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #60


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073




> remove watches when unregister children watches
> ---
>
> Key: DL-88
> URL: https://issues.apache.org/jira/browse/DL-88
> Project: DistributedLog
>  Issue Type: New Feature
>  Components: distributedlog-core
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 0.4.0
>
>
> use the new zk removeWatches api to remove watchers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #61: DL-89: add flag to enable thrift ...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/61

DL-89: add flag to enable thrift mux on DL monitor service

merge twitter's change from David Rusek.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-89

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/61.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #61


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073

commit d3a97bc0dde0c25516840725599ac46fa03601ab
Author: Dave Rusek 
Date:   2016-06-06T23:50:25Z

dl: add flag to enable thrift mux on DL Client

RB_ID=839555




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-distributedlog pull request #60: DL-88: remove watches when unregi...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/60

DL-88: remove watches when unregister children watches

merge twitter's change from Sijie Guo

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-88

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/60.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #60


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.

commit b571d3b4adcc140acca881979782474c27459d34
Author: Sijie Guo 
Date:   2016-05-24T04:01:57Z

DL: remove watches when unregister children watches

RB_ID=833858

commit 98dc9ab2df8af7af26ca68dbe93be3d420cda417
Author: Sijie Guo 
Date:   2016-05-27T00:06:31Z

 Remove watcher only when it registered watcher

 (reduce the annoying logging from zookeeper client about "Failed to find 
watcher!"

RB_ID=837073




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-87) Introduce periodic keepalive control record in writer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-87?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743914#comment-15743914
 ] 

ASF GitHub Bot commented on DL-87:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/59

DL-87: Introduce periodic keepalive control record in writer

merge twitter's change from Leigh Stewart

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-87

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/59.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #59


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.




> Introduce periodic keepalive control record in writer
> -
>
> Key: DL-87
> URL: https://issues.apache.org/jira/browse/DL-87
> Project: DistributedLog
>  Issue Type: New Feature
>Reporter: Sijie Guo
>Assignee: Leigh Stewart
> Fix For: 0.4.0
>
>
> the writer will periodically write 'keepalive' control record to make sure 
> the stream is alive. so if the write proxy is disconnected from bookies, the 
> control record will fail to write hence write proxy gets a chance to drop the 
> ownership.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #59: DL-87: Introduce periodic keepali...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/59

DL-87: Introduce periodic keepalive control record in writer

merge twitter's change from Leigh Stewart

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-87

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/59.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #59


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945

commit 517c77c164cb989ae9829cbd80bf2e492eb8e364
Author: Leigh Stewart 
Date:   2016-12-13T00:33:33Z

Introduce periodic keepalive control record in writer

* so the writer will periodically write 'keepalive' control record to 
make sure the stream is alive. so if the write proxy is disconnected from 
bookies, the control record will fail to write hence write proxy gets a chance 
to drop the ownership.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-86) Improve handling of lock conflicts in zk session lock

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743912#comment-15743912
 ] 

ASF GitHub Bot commented on DL-86:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/58

DL-86: Improve handling of lock conflicts in zk session lock

merge twitter's change from Sijie Guo.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-86

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/58.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #58


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945




> Improve handling of lock conflicts in zk session lock
> -
>
> Key: DL-86
> URL: https://issues.apache.org/jira/browse/DL-86
> Project: DistributedLog
>  Issue Type: Improvement
>  Components: distributedlog-core
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 0.4.0
>
>
> - lock reacquire could happen in foreground and background thread. so use a 
> semaphore to make sure there is only on e  outstanding  acquire 
> operation. and  check if already hold lock before reacquire.
> - fix handling zk sibling znode logic. as the znode is sequential znode, 
> the name would be different each time. so onlycomparing the client id and 
> session id of the znodes



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DL-85) Publish both per-stream stats and aggregation stats

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743909#comment-15743909
 ] 

ASF GitHub Bot commented on DL-85:
--

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/57

DL-85: Publish both per-stream stats and aggregation stats

merge twittter's changes from Jordan Bull.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-85

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/57.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #57


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803




> Publish both per-stream stats and aggregation stats
> ---
>
> Key: DL-85
> URL: https://issues.apache.org/jira/browse/DL-85
> Project: DistributedLog
>  Issue Type: Improvement
>Reporter: Sijie Guo
>Assignee: Jordan Bull
> Fix For: 0.4.0
>
>
> when publishing per stream stats, also publish aggregated stats



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #58: DL-86: Improve handling of lock c...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/58

DL-86: Improve handling of lock conflicts in zk session lock

merge twitter's change from Sijie Guo.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-86

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/58.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #58


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803

commit 7b46a9ac6bb5d520366069c244332347ef019e8e
Author: Sijie Guo 
Date:   2016-05-23T23:49:19Z

Improve handling of lock conflicts in zk session lock

- lock reacquire could happen in foreground and background thread. so use a 
semaphore to make sure there is only on e outstanding  acquire operation. and  
check if already hold lock before reacquire.
- fix handling zk sibling znode logic. as the znode is sequential znode, 
the name would be different each time. so only comparing the client id and 
session id of the znodes

RB_ID=833945




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-distributedlog pull request #57: DL-85: Publish both per-stream st...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/57

DL-85: Publish both per-stream stats and aggregation stats

merge twittter's changes from Jordan Bull.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog merge/DL-85

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/57.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #57


commit 904b8986b46fc908cfd1a0ee05b35dce77d12f5c
Author: Jordan Bull 
Date:   2016-05-13T18:27:19Z

when publishing per partition stats, also publish per stream stats

RB_ID=820062

commit 00919605f1bfa3f3e0c8514dc63552df4eeb3cc2
Author: Jordan Bull 
Date:   2016-05-18T17:18:34Z

use zero padded partition strings in stats

RB_ID=833803




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-134) Add DistributedLog User MailList to pom file

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743897#comment-15743897
 ] 

ASF GitHub Bot commented on DL-134:
---

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/56#discussion_r92087267
  
--- Diff: pom.xml ---
@@ -52,6 +52,13 @@
   
   
 
+  DistributedLog User
+  
user-subscr...@distributedlog.incubator.apache.org
+  
user-unsubscr...@distributedlog.incubator.apache.org
+  u...@distributedlog.incubator.apache.org
+  
http://www.mail-archive.com/user%distributedlog.incubator.apache.org
--- End diff --

I don't think we need to encode the @ in the xml. I updated the pom file to 
use 'user@distributedlog' now.


> Add DistributedLog User MailList to pom file
> 
>
> Key: DL-134
> URL: https://issues.apache.org/jira/browse/DL-134
> Project: DistributedLog
>  Issue Type: Task
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 0.4.0
>
>
> Add distributedlog user mail list to pom file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #56: DL-134: Add distributedlog user m...

[sijie]

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/56#discussion_r92087267
  
--- Diff: pom.xml ---
@@ -52,6 +52,13 @@
   
   
 
+  DistributedLog User
+  
user-subscr...@distributedlog.incubator.apache.org
+  
user-unsubscr...@distributedlog.incubator.apache.org
+  u...@distributedlog.incubator.apache.org
+  
http://www.mail-archive.com/user%distributedlog.incubator.apache.org
--- End diff --

I don't think we need to encode the @ in the xml. I updated the pom file to 
use 'user@distributedlog' now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [jira] [Created] (DL-139) Follow up on "Code Analysis of Twitter’s DistributedLog"

[Yiming Zang]

Interesting analysis

On Mon, Dec 12, 2016 at 5:39 PM, Sijie Guo (JIRA)  wrote:

> Sijie Guo created DL-139:
> 
>
>  Summary: Follow up on "Code Analysis of Twitter’s
> DistributedLog"
>  Key: DL-139
>  URL: https://issues.apache.org/jira/browse/DL-139
>  Project: DistributedLog
>   Issue Type: Task
> Reporter: Sijie Guo
>
>
> https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/
>
> Kiuwan did a code analysis on distributedlog code base and provided quite
> a few good suggestions. This is the ticket to follow up the suggestions.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>

"Code Analysis of Twitter’s DistributedLog"

[Sijie Guo]

There is a blog post about "code analysis of distributedlog
". It has
a few good suggestions. I've created a jira DL-139
 for tracking the items to
improve the code quality based on the suggestions. If you are interested in
working on them, feel free to work on them.

- Sijie

[jira] [Created] (DL-139) Follow up on "Code Analysis of Twitter’s DistributedLog"

[Sijie Guo (JIRA)]

Sijie Guo created DL-139:


 Summary: Follow up on "Code Analysis of Twitter’s DistributedLog"
 Key: DL-139
 URL: https://issues.apache.org/jira/browse/DL-139
 Project: DistributedLog
  Issue Type: Task
Reporter: Sijie Guo


https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/

Kiuwan did a code analysis on distributedlog code base and provided quite a few 
good suggestions. This is the ticket to follow up the suggestions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DL-138) Improve the 'synchronized' usage

[Sijie Guo (JIRA)]

Sijie Guo created DL-138:


 Summary: Improve the 'synchronized' usage 
 Key: DL-138
 URL: https://issues.apache.org/jira/browse/DL-138
 Project: DistributedLog
  Issue Type: Task
Reporter: Sijie Guo


{quote}
191 Methods are synchronized using the ‘synchronized’ modifier in the 
declaration. It is not a good programming practise to use this modifier at 
method level. Normally, it is more recommendable to implement synchronized code 
blocks inside methods. The reason is very simple. If other people modify these 
methods later, it will not be easy for them to realise that the entire method 
is synchronized. They will surely expand the method with code that is not 
necessary to synchronize. In the long term, the application will become more 
and more linear in execution, taking way less advantage of the hardware 
resources, and will be slower.
{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DL-138) Improve the 'synchronized' usage

[Sijie Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DL-138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sijie Guo updated DL-138:
-
Description: 
{quote}
191 Methods are synchronized using the ‘synchronized’ modifier in the 
declaration. It is not a good programming practise to use this modifier at 
method level. Normally, it is more recommendable to implement synchronized code 
blocks inside methods. The reason is very simple. If other people modify these 
methods later, it will not be easy for them to realise that the entire method 
is synchronized. They will surely expand the method with code that is not 
necessary to synchronize. In the long term, the application will become more 
and more linear in execution, taking way less advantage of the hardware 
resources, and will be slower.
{quote}

https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/

  was:
{quote}
191 Methods are synchronized using the ‘synchronized’ modifier in the 
declaration. It is not a good programming practise to use this modifier at 
method level. Normally, it is more recommendable to implement synchronized code 
blocks inside methods. The reason is very simple. If other people modify these 
methods later, it will not be easy for them to realise that the entire method 
is synchronized. They will surely expand the method with code that is not 
necessary to synchronize. In the long term, the application will become more 
and more linear in execution, taking way less advantage of the hardware 
resources, and will be slower.
{quote}


> Improve the 'synchronized' usage 
> -
>
> Key: DL-138
> URL: https://issues.apache.org/jira/browse/DL-138
> Project: DistributedLog
>  Issue Type: Task
>Reporter: Sijie Guo
>
> {quote}
> 191 Methods are synchronized using the ‘synchronized’ modifier in the 
> declaration. It is not a good programming practise to use this modifier at 
> method level. Normally, it is more recommendable to implement synchronized 
> code blocks inside methods. The reason is very simple. If other people modify 
> these methods later, it will not be easy for them to realise that the entire 
> method is synchronized. They will surely expand the method with code that is 
> not necessary to synchronize. In the long term, the application will become 
> more and more linear in execution, taking way less advantage of the hardware 
> resources, and will be slower.
> {quote}
> https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DL-137) Make sure all TODO labels are associated with Jiras

[Sijie Guo (JIRA)]

Sijie Guo created DL-137:


 Summary: Make sure all TODO labels are associated with Jiras
 Key: DL-137
 URL: https://issues.apache.org/jira/browse/DL-137
 Project: DistributedLog
  Issue Type: Task
Reporter: Sijie Guo


21 ‘TODO’ labels. It is good practise to periodically revise all ‘TODO’ or 
‘TBD’ labels in the code. These are marks that programmers leave to remind that 
there are things pending implementation that may not be done yet. Some of these 
labels refer to changes in names or to re-factorisings of pending code, but the 
majority indicate that some functionality is not yet supported.

https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DL-136) Avoid empty catch blocks

[Sijie Guo (JIRA)]

Sijie Guo created DL-136:


 Summary: Avoid empty catch blocks
 Key: DL-136
 URL: https://issues.apache.org/jira/browse/DL-136
 Project: DistributedLog
  Issue Type: Task
Reporter: Sijie Guo


Based on the code analysis here : 
https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/, there are 22 
Times: Catching exceptions with empty bodies.

{quote}
There are exceptions of different types: InterruptedException, KeeperException, 
Exception, etc.. Even though we think that these exceptions should never occur, 
if they are captured it means that they are technically possible. In case that 
they do happen, it will be very difficult to know what is happening and to 
diagnose the problem. It is very recommendable, at least to add a log trace, in 
case these exceptions are triggered.
{quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DL-136) Avoid empty catch blocks

[Sijie Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DL-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sijie Guo updated DL-136:
-
Labels: help-wanted  (was: )

> Avoid empty catch blocks
> 
>
> Key: DL-136
> URL: https://issues.apache.org/jira/browse/DL-136
> Project: DistributedLog
>  Issue Type: Task
>Reporter: Sijie Guo
>  Labels: help-wanted
>
> Based on the code analysis here : 
> https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/, there are 22 
> Times: Catching exceptions with empty bodies.
> {quote}
> There are exceptions of different types: InterruptedException, 
> KeeperException, Exception, etc.. Even though we think that these exceptions 
> should never occur, if they are captured it means that they are technically 
> possible. In case that they do happen, it will be very difficult to know what 
> is happening and to diagnose the problem. It is very recommendable, at least 
> to add a log trace, in case these exceptions are triggered.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DL-135) Cleanup System.out and System.err usage

[Sijie Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DL-135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sijie Guo updated DL-135:
-
Labels: help-wanted  (was: )

> Cleanup System.out and System.err usage
> ---
>
> Key: DL-135
> URL: https://issues.apache.org/jira/browse/DL-135
> Project: DistributedLog
>  Issue Type: Task
>Reporter: Sijie Guo
>  Labels: help-wanted
>
> Based on the code analysis here : 
> https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/
> There are 272 calls to System.out and System.err usage. Most of them might 
> just come from distributedlog tool. But let's still audit the code to make 
> sure we are using System.out and System.err correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DL-135) Cleanup System.out and System.err usage

[Sijie Guo (JIRA)]

Sijie Guo created DL-135:


 Summary: Cleanup System.out and System.err usage
 Key: DL-135
 URL: https://issues.apache.org/jira/browse/DL-135
 Project: DistributedLog
  Issue Type: Task
Reporter: Sijie Guo


Based on the code analysis here : 
https://www.kiuwan.com/blog/analyzing_distributedlog_twitter-2/

There are 272 calls to System.out and System.err usage. Most of them might just 
come from distributedlog tool. But let's still audit the code to make sure we 
are using System.out and System.err correctly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DL-92) Simplify the state transition on stream

[Sijie Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DL-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sijie Guo updated DL-92:

Assignee: Leigh Stewart  (was: Sijie Guo)

> Simplify the state transition on stream
> ---
>
> Key: DL-92
> URL: https://issues.apache.org/jira/browse/DL-92
> Project: DistributedLog
>  Issue Type: Improvement
>  Components: distributedlog-service
>Affects Versions: 0.4.0
>Reporter: Sijie Guo
>Assignee: Leigh Stewart
>
> Simplify the state transition on stream
> * the stream is created on INITIALIZING
> * when the stream is started, it would start transition from INITIALIZING 
> to INITIALIZED
> * it would serve stream operations when the stream is INITIALIZED
> * it would be turned to ERROR when encountered exceptions.
> * the stream would be closed when service operation timeout or 
> encountered any exceptions. it would first be removed from  acquired mapping
> * the stream would be removed from cached mapping depends on probation 
> time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DL-91) Be able to close writer within a timeout duration

[Sijie Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DL-91?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sijie Guo updated DL-91:

Assignee: Leigh Stewart

> Be able to close writer within a timeout duration
> -
>
> Key: DL-91
> URL: https://issues.apache.org/jira/browse/DL-91
> Project: DistributedLog
>  Issue Type: Bug
>  Components: distributedlog-client
>Reporter: Sijie Guo
>Assignee: Leigh Stewart
>
> If a stream is hanging on closing the writer on write proxy, it will cause 
> STEAM_UNAVAILABLE.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DL-127) Fix coverall report

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743692#comment-15743692
 ] 

Hudson commented on DL-127:
---

UNSTABLE: Integrated in Jenkins build distributedlog-nightly-build #153 (See 
[https://builds.apache.org/job/distributedlog-nightly-build/153/])
DL-127: Fix coverall report (sijieg: rev 
3fceccc6cbce5c41c52510172ab6925d3beec35b)
* (edit) distributedlog-core/pom.xml
* (edit) distributedlog-protocol/pom.xml
* (edit) pom.xml
* (edit) distributedlog-service/pom.xml


> Fix coverall report
> ---
>
> Key: DL-127
> URL: https://issues.apache.org/jira/browse/DL-127
> Project: DistributedLog
>  Issue Type: Bug
>  Components: builds
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 0.4.0
>
>
> Make coverall reports work



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DL-90) Don't use stack and codec together for configuring thriftmux

[Sijie Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DL-90?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sijie Guo updated DL-90:

Assignee: David Rusek  (was: Sijie Guo)

> Don't use stack and codec together for configuring thriftmux
> 
>
> Key: DL-90
> URL: https://issues.apache.org/jira/browse/DL-90
> Project: DistributedLog
>  Issue Type: Bug
>  Components: distributedlog-client, distributedlog-service
>Affects Versions: 0.4.0
>Reporter: Sijie Guo
>Assignee: David Rusek
>
> - Don't use stack and codec together for configuring thriftmux
> - Remove codec if the thriftmux is disabled



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DL-87) Introduce periodic keepalive control record in writer

[Sijie Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DL-87?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sijie Guo updated DL-87:

Assignee: Leigh Stewart  (was: Sijie Guo)

> Introduce periodic keepalive control record in writer
> -
>
> Key: DL-87
> URL: https://issues.apache.org/jira/browse/DL-87
> Project: DistributedLog
>  Issue Type: New Feature
>Reporter: Sijie Guo
>Assignee: Leigh Stewart
> Fix For: 0.4.0
>
>
> the writer will periodically write 'keepalive' control record to make sure 
> the stream is alive. so if the write proxy is disconnected from bookies, the 
> control record will fail to write hence write proxy gets a chance to drop the 
> ownership.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DL-85) Publish both per-stream stats and aggregation stats

[Sijie Guo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DL-85?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sijie Guo updated DL-85:

Assignee: Jordan Bull  (was: Yiming Zang)

> Publish both per-stream stats and aggregation stats
> ---
>
> Key: DL-85
> URL: https://issues.apache.org/jira/browse/DL-85
> Project: DistributedLog
>  Issue Type: Improvement
>Reporter: Sijie Guo
>Assignee: Jordan Bull
> Fix For: 0.4.0
>
>
> when publishing per stream stats, also publish aggregated stats



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #56: Add distributedlog user mail list...

[franckcuny]

Github user franckcuny commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/56#discussion_r92070883
  
--- Diff: pom.xml ---
@@ -52,6 +52,13 @@
   
   
 
+  DistributedLog User
+  
user-subscr...@distributedlog.incubator.apache.org
+  
user-unsubscr...@distributedlog.incubator.apache.org
+  u...@distributedlog.incubator.apache.org
+  
http://www.mail-archive.com/user%distributedlog.incubator.apache.org
--- End diff --

user@distributedlog

https://www.mail-archive.com/user@distributedlog.incubator.apache.org/

do you need to encode the @ in the xml though ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-distributedlog pull request #56: Add distributedlog user mail list...

[sijie]

GitHub user sijie opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/56

Add distributedlog user mail list to pom file



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sijie/incubator-distributedlog sijie/DL-134

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/56.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #56


commit a8efaa579c1502147b6a2e7b95cbf07ab58cc28f
Author: Sijie Guo 
Date:   2016-12-13T00:05:27Z

Add distributedlog user mail list to pom file




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DL-134) Add DistributedLog User MailList to pom file

[Sijie Guo (JIRA)]

Sijie Guo created DL-134:


 Summary: Add DistributedLog User MailList to pom file
 Key: DL-134
 URL: https://issues.apache.org/jira/browse/DL-134
 Project: DistributedLog
  Issue Type: Task
Reporter: Sijie Guo
Assignee: Sijie Guo
 Fix For: 0.4.0


Add distributedlog user mail list to pom file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DL-127) Fix coverall report

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743533#comment-15743533
 ] 

ASF GitHub Bot commented on DL-127:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-distributedlog/pull/53


> Fix coverall report
> ---
>
> Key: DL-127
> URL: https://issues.apache.org/jira/browse/DL-127
> Project: DistributedLog
>  Issue Type: Bug
>  Components: builds
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 0.4.0
>
>
> Make coverall reports work



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #53: DL-127: Fix coverall report

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-distributedlog/pull/53


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

incubator-distributedlog git commit: DL-127: Fix coverall report

[sijie]

Repository: incubator-distributedlog
Updated Branches:
  refs/heads/master 8e619cd73 -> 3fceccc6c


DL-127: Fix coverall report

Author: Sijie Guo 

Reviewers: Franck Cuny 

Closes #53 from sijie/sijie/remove_reporting


Project: http://git-wip-us.apache.org/repos/asf/incubator-distributedlog/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-distributedlog/commit/3fceccc6
Tree: 
http://git-wip-us.apache.org/repos/asf/incubator-distributedlog/tree/3fceccc6
Diff: 
http://git-wip-us.apache.org/repos/asf/incubator-distributedlog/diff/3fceccc6

Branch: refs/heads/master
Commit: 3fceccc6cbce5c41c52510172ab6925d3beec35b
Parents: 8e619cd
Author: Sijie Guo 
Authored: Mon Dec 12 15:37:54 2016 -0800
Committer: Sijie Guo 
Committed: Mon Dec 12 15:37:54 2016 -0800

--
 distributedlog-core/pom.xml |  4 
 distributedlog-protocol/pom.xml |  6 ++
 distributedlog-service/pom.xml  |  6 ++
 pom.xml | 26 +++---
 4 files changed, 27 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-distributedlog/blob/3fceccc6/distributedlog-core/pom.xml
--
diff --git a/distributedlog-core/pom.xml b/distributedlog-core/pom.xml
index 20d3139..4da1849 100644
--- a/distributedlog-core/pom.xml
+++ b/distributedlog-core/pom.xml
@@ -118,6 +118,10 @@
   commons-cli
   commons-cli
 
+
+  org.slf4j
+  slf4j-log4j12
+
   
 
 

http://git-wip-us.apache.org/repos/asf/incubator-distributedlog/blob/3fceccc6/distributedlog-protocol/pom.xml
--
diff --git a/distributedlog-protocol/pom.xml b/distributedlog-protocol/pom.xml
index 48b749a..4417d14 100644
--- a/distributedlog-protocol/pom.xml
+++ b/distributedlog-protocol/pom.xml
@@ -30,6 +30,12 @@
   org.apache.bookkeeper.stats
   bookkeeper-stats-api
   ${bookkeeper.version}
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
 
 
   org.apache.thrift

http://git-wip-us.apache.org/repos/asf/incubator-distributedlog/blob/3fceccc6/distributedlog-service/pom.xml
--
diff --git a/distributedlog-service/pom.xml b/distributedlog-service/pom.xml
index aa5849e..b7b6ff8 100644
--- a/distributedlog-service/pom.xml
+++ b/distributedlog-service/pom.xml
@@ -63,6 +63,12 @@
   org.apache.zookeeper
   zookeeper
   ${zookeeper.version}
+  
+
+  org.slf4j
+  slf4j-log4j12
+
+  
 
 
   jdiff

http://git-wip-us.apache.org/repos/asf/incubator-distributedlog/blob/3fceccc6/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 6441a26..db07f12 100644
--- a/pom.xml
+++ b/pom.xml
@@ -195,6 +195,17 @@
 coveralls-maven-plugin
 4.1.0
   
+  
+org.codehaus.mojo
+cobertura-maven-plugin
+2.7
+
+  
+html
+xml
+  
+
+  
 
   
   
@@ -212,19 +223,4 @@
   http://maven.twttr.com
 
   
-  
-
-  
-org.codehaus.mojo
-cobertura-maven-plugin
-2.7
-
-  
-html
-xml
-  
-
-  
-
-  
 

Re: I have some problem when I run distributedlog-benchmark.I need help!

[Sijie Guo]

On Mon, Dec 12, 2016 at 12:19 AM, 康斯淇  wrote:

>
>
> Could you please do me a favor? I’m a beginner of DL. These days I run the
> DL-benchmark and I encounter some problems.
>


First to clarify what does benchmark module do.

In distributedlog, we didn't follow the approach of "writing x millions
records, timing it and calculating a throughput number". As we felt the
approach is a bit problematic - since the number is most likely calculated
when the resources are under-utilized and meaningless as the guideline for
real production setup.

Instead, we follow a USE-like 
principle for running any benchmarks - a benchmark should be used for find
the saturation point for the desired metrics and the saturation point is
the limit of the system and can be used as a guideline for production
capacity planning.

Throughput and Latency are the two desired metrics for distributedlog
benchmarks. What we measured in distributedlog benchmark is how much
latency (50 percentile, 99 percentile, and 99.9 percentile) that DL can
achieve under a given throughput.

In order to get such information, you need to configure to use a stats
provider to collect the stats exposed by the benchmark. You can read this
page
http://distributedlog.incubator.apache.org/docs/latest/admin_guide/monitoring.html#stats-provider
for more information. One suggestion is using Codahale

metrics to collect the metric. It can expose the stats into csv file and
then you can plot the stats using any graph tool to understand the
relationship between throughput and latency.

Hope this make sense.

But if the approach of "writing x millions records, timing it and
calculating a throughput number" will be convenient to you, feel free to
modify the benchmark to achieve that.


>
>
> 1. I have a doubt what the benchmark actually tests. For instance, I found
> in the class com.twitter.benchmark.stream.AsyncReaderBenchmark, it
> records the time to open the reader and the time to read records. But after
> I run the class, I found only the first time is recorded in file
> dbench.log. So I have no idea about how it shows performance of
> AsyncLogReader and whether it test the throughput of AsyncLogReader.
>

In this test case, the time on read records measure the latency on reading
a record.


>
>
> 2. Another problem is LedgerReadBenchmark. I found when read in this way,
> it starts a thread which the other two didn’t. I would have thought I
> should do some comparsion between AsyncReaderBenchmark/SyncReaderBenchmark
> and LedgerReadBenchmark. So what LedgerReadBenchmark actually tests?
>

This benchmark is to test the reading entries using bk raw api directly. It
is to see if there is any difference between reading bk entries directly
and reading records using DL library. It is used to find if there is any
overhead introduced by the DL library.


>
>
> 3. I found both of AsyncLogReader and LogReader read records sequentially
> and AsyncLogReader shows better performance. I want to know the different
> usage scenario of them.
>

They are almost same. Currently LogReader (sync reader) is based on
AsyncLogReader.

The difference here you can think about is - AsyncLogReader is more like a
'event' driven (callback like) reader. The future is notified or triggered
when record is available. LogReader is a more 'synchronous' reader, more
like polling records.


>
>
> 4. what is the difference between entry and log record?
>

A log record is the logic user data unit. An entry is a batch of records,
which is an I/O unit.


>
>
> Looking forward to your answer!
>

Let me know if all these answered your questions. If you need more
clarification and have more questions, please let me know.

- Sijie


>
>
> --
>
> 康斯淇
> 中国科学院计算技术研究所  先进计算中心
> 电话：18781961524  E-mail：kangs...@ict.ac.cn
> 地址：北京市海淀区中关村科学院南路6号 100190
>
>
>

[GitHub] incubator-distributedlog pull request #55: Hack week cleanup

[franckcuny]

GitHub user franckcuny opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/55

Hack week cleanup

This is still in progress.

- remove trailing white spaces in a few places
- fix some scripts (not rely on git); style consistency
- validate that the vagrant configuration works

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/franckcuny/incubator-distributedlog 
fcuny/hack-week-cleanup

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/55.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #55


commit 5800829fc9a0751dcc41195550025f77aba7eb39
Author: Franck Cuny 
Date:   2016-12-12T18:43:19Z

Remove Dockerfile and docker directory.

There's already a docker file at the root of the repository. In
addition, this Dockerfile would not work, since you can't create an
image by copying content from a directory outside, since you get the
following error:

  % docker build .
  ...
  Forbidden path outside the build context: .. ()

commit 9bf33fc94b58b09b21a1d318c7001a04397b6225
Author: Franck Cuny 
Date:   2016-12-12T18:45:54Z

Create a valid snapshot.

commit 179eddc269653608750ab9c344205f52d4315553
Author: Franck Cuny 
Date:   2016-12-12T21:55:29Z

Ignore .vagrant directory created by vagrant.

commit a7ac7c051fcae3c86e785298c29c736d575580d4
Author: Franck Cuny 
Date:   2016-12-12T21:56:26Z

Git ignore pid files

commit 480bc348694a19fc0be9e78d1f83a45e08e609e8
Author: Franck Cuny 
Date:   2016-12-12T22:17:18Z

Remove comment from Dockerfile

Remove trailing white spaces too.

commit 21d89f8e93d0110bb92f7adc4f85e7e17a490489
Author: Franck Cuny 
Date:   2016-12-12T22:28:15Z

Do not assume git is available in the shell scripts.

Almost all the shell scripts were using `git` to figure out the root
directory of distributedlog, but we can't assume it is available on the
host. We could also not have a .git directory if the package is shipped
in a tarball or other package format.

Reformat some of the scripts to use consistent indentation level.

commit 27f90f13aed5ce87c171024402d05a7975a8ac26
Author: Franck Cuny 
Date:   2016-12-12T22:30:26Z

Rename bookkeeper hosts from node to bk.

It is more obvious than the generic name.

commit a3fc4ef3c85f053b6433b87b64ce72fc7a295659
Author: Franck Cuny 
Date:   2016-12-12T22:31:11Z

Remove trailing white spaces in the doc




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-127) Fix coverall report

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742985#comment-15742985
 ] 

ASF GitHub Bot commented on DL-127:
---

Github user sijie commented on the issue:

https://github.com/apache/incubator-distributedlog/pull/53
  
@franckcuny @leighst can any of you review this one? the coverall plugin 
was added in a wrong section which cause the coverall report can't be generated 
correctly. I'd like to merge this so the master branch can generate the right 
coverall report.


> Fix coverall report
> ---
>
> Key: DL-127
> URL: https://issues.apache.org/jira/browse/DL-127
> Project: DistributedLog
>  Issue Type: Bug
>  Components: builds
>Reporter: Sijie Guo
>Assignee: Sijie Guo
> Fix For: 0.4.0
>
>
> Make coverall reports work



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog issue #53: DL-127: Fix coverall report

[sijie]

Github user sijie commented on the issue:

https://github.com/apache/incubator-distributedlog/pull/53
  
@franckcuny @leighst can any of you review this one? the coverall plugin 
was added in a wrong section which cause the coverall report can't be generated 
correctly. I'd like to merge this so the master branch can generate the right 
coverall report.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-129) Enable check style for distributedlog-protocol module

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742976#comment-15742976
 ] 

ASF GitHub Bot commented on DL-129:
---

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92026632
  
--- Diff: 
distributedlog-build-tools/src/main/resources/distributedlog/checkstyle.xml ---
@@ -0,0 +1,443 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+  
+
+  
+
+  
+
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+  
+
+  
+
+  
+  
+
+
+
+  
+  
+
+  
+
+  
+  
+
+  
+  
+
+
+  
+  
+
+
+
+
+
+  
+  
+  
+
+
+
+  
+  
+  
+  
+  
+  
+  
+
+
+
+  
+
+
+
+  
--- End diff --

add 'bk-shade' to avoid importing shaded bookkeeper dependencies?


> Enable check style for distributedlog-protocol module
> -
>
> Key: DL-129
> URL: https://issues.apache.org/jira/browse/DL-129
> Project: DistributedLog
>  Issue Type: Sub-task
>  Components: distributedlog-protocol
>Reporter: Xi Liu
>Assignee: Xi Liu
> Fix For: 0.4.0
>
>
> Enable check style plugin for distributedlog-protocol module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DL-129) Enable check style for distributedlog-protocol module

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742975#comment-15742975
 ] 

ASF GitHub Bot commented on DL-129:
---

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92026383
  
--- Diff: 
distributedlog-build-tools/src/main/resources/distributedlog/checkstyle.xml ---
@@ -0,0 +1,443 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+  
+
+  
+
+  
+
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+  
+
+  
+
+  
+  
+
+
+
+  
+  
+
+  
+
+  
+  
+
+  
+  
+
+
+  
+  
+
+
+
+
+
+  
+  
+  
+
+
+
+  
+  
+  
+  
+  
+  
+  
+
+
+
--- End diff --

+1 for avoiding start import


> Enable check style for distributedlog-protocol module
> -
>
> Key: DL-129
> URL: https://issues.apache.org/jira/browse/DL-129
> Project: DistributedLog
>  Issue Type: Sub-task
>  Components: distributedlog-protocol
>Reporter: Xi Liu
>Assignee: Xi Liu
> Fix For: 0.4.0
>
>
> Enable check style plugin for distributedlog-protocol module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #54: DL-129: Enable checkstyle plugin ...

[sijie]

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92025594
  
--- Diff: distributedlog-build-tools/pom.xml ---
@@ -0,0 +1,30 @@
+
--- End diff --

I like the idea to organize the checkstyle files into a separated module so 
that we don't need to have configuration on each module.

I think we should do the similar thing for findbugs settings. Can you move 
the findbug settings to builds-tool module? (It can be a separate jira and pull 
request).

Also it might be clear to move the eclipse format settings and the merge-pr 
script to builds-tool module too.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-129) Enable check style for distributedlog-protocol module

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742979#comment-15742979
 ] 

ASF GitHub Bot commented on DL-129:
---

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92026310
  
--- Diff: 
distributedlog-build-tools/src/main/resources/distributedlog/checkstyle.xml ---
@@ -0,0 +1,443 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+  
+
+  
+
+  
+
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+  
+
+  
+
+  
+  
+
+
+
+  
+  
+
+  
+
+  
+  
+
+  
+  
+
+
+  
+  
+
+
+
+
+
+  
+  
+  
+
+
+
+  
+  
+  
+  
+  
--- End diff --

nice. sorting the imports looks really good to me.


> Enable check style for distributedlog-protocol module
> -
>
> Key: DL-129
> URL: https://issues.apache.org/jira/browse/DL-129
> Project: DistributedLog
>  Issue Type: Sub-task
>  Components: distributedlog-protocol
>Reporter: Xi Liu
>Assignee: Xi Liu
> Fix For: 0.4.0
>
>
> Enable check style plugin for distributedlog-protocol module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DL-129) Enable check style for distributedlog-protocol module

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742977#comment-15742977
 ] 

ASF GitHub Bot commented on DL-129:
---

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92025594
  
--- Diff: distributedlog-build-tools/pom.xml ---
@@ -0,0 +1,30 @@
+
--- End diff --

I like the idea to organize the checkstyle files into a separated module so 
that we don't need to have configuration on each module.

I think we should do the similar thing for findbugs settings. Can you move 
the findbug settings to builds-tool module? (It can be a separate jira and pull 
request).

Also it might be clear to move the eclipse format settings and the merge-pr 
script to builds-tool module too.


> Enable check style for distributedlog-protocol module
> -
>
> Key: DL-129
> URL: https://issues.apache.org/jira/browse/DL-129
> Project: DistributedLog
>  Issue Type: Sub-task
>  Components: distributedlog-protocol
>Reporter: Xi Liu
>Assignee: Xi Liu
> Fix For: 0.4.0
>
>
> Enable check style plugin for distributedlog-protocol module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DL-129) Enable check style for distributedlog-protocol module

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742978#comment-15742978
 ] 

ASF GitHub Bot commented on DL-129:
---

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92025930
  
--- Diff: 
distributedlog-build-tools/src/main/resources/distributedlog/checkstyle.xml ---
@@ -0,0 +1,443 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+  
+
+  
+
+  
+
+
--- End diff --

it might be good to make sure there is a jira associated with TODO item?

E.g. 

"TODO: {@link jira_link}"?


> Enable check style for distributedlog-protocol module
> -
>
> Key: DL-129
> URL: https://issues.apache.org/jira/browse/DL-129
> Project: DistributedLog
>  Issue Type: Sub-task
>  Components: distributedlog-protocol
>Reporter: Xi Liu
>Assignee: Xi Liu
> Fix For: 0.4.0
>
>
> Enable check style plugin for distributedlog-protocol module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #54: DL-129: Enable checkstyle plugin ...

[sijie]

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92025930
  
--- Diff: 
distributedlog-build-tools/src/main/resources/distributedlog/checkstyle.xml ---
@@ -0,0 +1,443 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+  
+
+  
+
+  
+
+
--- End diff --

it might be good to make sure there is a jira associated with TODO item?

E.g. 

"TODO: {@link jira_link}"?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-distributedlog pull request #54: DL-129: Enable checkstyle plugin ...

[sijie]

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92026383
  
--- Diff: 
distributedlog-build-tools/src/main/resources/distributedlog/checkstyle.xml ---
@@ -0,0 +1,443 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+  
+
+  
+
+  
+
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+  
+
+  
+
+  
+  
+
+
+
+  
+  
+
+  
+
+  
+  
+
+  
+  
+
+
+  
+  
+
+
+
+
+
+  
+  
+  
+
+
+
+  
+  
+  
+  
+  
+  
+  
+
+
+
--- End diff --

+1 for avoiding start import


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-distributedlog pull request #54: DL-129: Enable checkstyle plugin ...

[sijie]

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92026310
  
--- Diff: 
distributedlog-build-tools/src/main/resources/distributedlog/checkstyle.xml ---
@@ -0,0 +1,443 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+  
+
+  
+
+  
+
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+  
+
+  
+
+  
+  
+
+
+
+  
+  
+
+  
+
+  
+  
+
+  
+  
+
+
+  
+  
+
+
+
+
+
+  
+  
+  
+
+
+
+  
+  
+  
+  
+  
--- End diff --

nice. sorting the imports looks really good to me.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-distributedlog pull request #54: DL-129: Enable checkstyle plugin ...

[sijie]

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/54#discussion_r92026632
  
--- Diff: 
distributedlog-build-tools/src/main/resources/distributedlog/checkstyle.xml ---
@@ -0,0 +1,443 @@
+
+
+http://www.puppycrawl.com/dtds/configuration_1_3.dtd";>
+
+
+
+
+
+  
+
+  
+
+  
+
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+
+
+
+  
+
+  
+  
+
+  
+
+  
+  
+
+
+
+  
+  
+
+  
+
+  
+  
+
+  
+  
+
+
+  
+  
+
+
+
+
+
+  
+  
+  
+
+
+
+  
+  
+  
+  
+  
+  
+  
+
+
+
+  
+
+
+
+  
--- End diff --

add 'bk-shade' to avoid importing shaded bookkeeper dependencies?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (DL-81) Build the distributedlog release procedure

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DL-81?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15742942#comment-15742942
 ] 

ASF GitHub Bot commented on DL-81:
--

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/51#discussion_r92024690
  
--- Diff: website/community/release_guide.md ---
@@ -0,0 +1,507 @@
+---
+layout: default
+title: "DistributedLog Release Guide"
+permalink: /community/release-guide/
+---
+
+# Apache DistributedLog Release Guide
+
+* TOC
+{:toc}
+
+This page documents the procedure to make an Apache DistributedLog 
release. Creit to the 
[Beam](http://distributedlog.incubator.apache.org/contribute/release-guide/) 
project. We've borrow liberally from their documentation.
--- End diff --

Good catch! Will fix it.


> Build the distributedlog release procedure
> --
>
> Key: DL-81
> URL: https://issues.apache.org/jira/browse/DL-81
> Project: DistributedLog
>  Issue Type: Task
>  Components: documentation
>Reporter: Sijie Guo
>Assignee: Sijie Guo
>Priority: Blocker
> Fix For: 0.4.0
>
>
> this is the ticket to track building the release procedure for distributedlog



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-distributedlog pull request #51: (WIP) DL-81: Build the distribute...

[sijie]

Github user sijie commented on a diff in the pull request:


https://github.com/apache/incubator-distributedlog/pull/51#discussion_r92024690
  
--- Diff: website/community/release_guide.md ---
@@ -0,0 +1,507 @@
+---
+layout: default
+title: "DistributedLog Release Guide"
+permalink: /community/release-guide/
+---
+
+# Apache DistributedLog Release Guide
+
+* TOC
+{:toc}
+
+This page documents the procedure to make an Apache DistributedLog 
release. Creit to the 
[Beam](http://distributedlog.incubator.apache.org/contribute/release-guide/) 
project. We've borrow liberally from their documentation.
--- End diff --

Good catch! Will fix it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-distributedlog pull request #54: Enable checkstyle plugin for dist...

[xiliuant]

GitHub user xiliuant opened a pull request:

https://github.com/apache/incubator-distributedlog/pull/54

Enable checkstyle plugin for distributedlog-protocol module

- add build-tools module for check style configurations
- fix style problems in distributedlog-protocol
  * sorted import
  * fix javadoc
  * constant static name and static name 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xiliuant/incubator-distributedlog 
xi/checkstyle

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-distributedlog/pull/54.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #54


commit 86a996c3efddb1c03c1e011ed1298101e63b0ef9
Author: Xi Liu 
Date:   2016-12-12T17:19:49Z

Enable checkstyle plugin for distributedlog-protocol module




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DL-132) Enable check style for distributedlog-service module

[Xi Liu (JIRA)]

Xi Liu created DL-132:
-

 Summary: Enable check style for distributedlog-service module
 Key: DL-132
 URL: https://issues.apache.org/jira/browse/DL-132
 Project: DistributedLog
  Issue Type: Sub-task
  Components: distributedlog-service
Reporter: Xi Liu
 Fix For: 0.4.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DL-133) Enable check style for distributedlog-benchmark module

[Xi Liu (JIRA)]

Xi Liu created DL-133:
-

 Summary: Enable check style for distributedlog-benchmark module
 Key: DL-133
 URL: https://issues.apache.org/jira/browse/DL-133
 Project: DistributedLog
  Issue Type: Sub-task
  Components: distributedlog-benchmark
Reporter: Xi Liu
 Fix For: 0.4.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DL-131) Enable check style for distributedlog-client module

[Xi Liu (JIRA)]

Xi Liu created DL-131:
-

 Summary: Enable check style for distributedlog-client module
 Key: DL-131
 URL: https://issues.apache.org/jira/browse/DL-131
 Project: DistributedLog
  Issue Type: Sub-task
  Components: distributedlog-client
Reporter: Xi Liu
 Fix For: 0.4.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DL-130) Enable check style for distributedlog-core module

[Xi Liu (JIRA)]

Xi Liu created DL-130:
-

 Summary: Enable check style for distributedlog-core module
 Key: DL-130
 URL: https://issues.apache.org/jira/browse/DL-130
 Project: DistributedLog
  Issue Type: Sub-task
  Components: distributedlog-core
Reporter: Xi Liu
 Fix For: 0.4.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DL-129) Enable check style for distributedlog-protocol module

[Xi Liu (JIRA)]

Xi Liu created DL-129:
-

 Summary: Enable check style for distributedlog-protocol module
 Key: DL-129
 URL: https://issues.apache.org/jira/browse/DL-129
 Project: DistributedLog
  Issue Type: Sub-task
  Components: distributedlog-protocol
Reporter: Xi Liu
Assignee: Xi Liu
 Fix For: 0.4.0


Enable check style plugin for distributedlog-protocol module



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

I have some problem when I run distributedlog-benchmark.I need help!

[康斯淇]

Could you please do me a favor? I’m a beginner of DL. These days I run the 
DL-benchmark and I encounter some problems.


1. I have a doubt what the benchmark actually tests. For instance, I found in 
the class com.twitter.benchmark.stream.AsyncReaderBenchmark, it records the 
time to open the reader and the time to read records. But after I run the 
class, I found only the first time is recorded in file dbench.log. So I have no 
idea about how it shows performance of AsyncLogReader and whether it test the 
throughput of AsyncLogReader.


2. Another problem is LedgerReadBenchmark. I found when read in this way, it 
starts a thread which the other two didn’t. I would have thought I should do 
some comparsion between AsyncReaderBenchmark/SyncReaderBenchmark and 
LedgerReadBenchmark. So what LedgerReadBenchmark actually tests?


3. I found both of AsyncLogReader and LogReader read records sequentially and 
AsyncLogReader shows better performance. I want to know the different usage 
scenario of them.


4. what is the difference between entry and log record?


Looking forward to your answer!


--

康斯淇 
中国科学院计算技术研究所  先进计算中心
电话：18781961524  E-mail：kangs...@ict.ac.cn
地址：北京市海淀区中关村科学院南路6号　 100190