[jira] [Resolved] (KAFKA-3309) Update Protocol Documentation WIP patch

[Ashish K Singh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish K Singh resolved KAFKA-3309.
---
Resolution: Duplicate

> Update Protocol Documentation WIP patch
> ---
>
> Key: KAFKA-3309
> URL: https://issues.apache.org/jira/browse/KAFKA-3309
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Ashish K Singh
>Assignee: Ashish K Singh
>
> https://github.com/apache/kafka/pull/970 has the WIP patch. Update to reflect 
> changes made for KIP-35.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Protocol Documentation WIP

[Grant Henke]

FYI. I have updated the pull request here:
https://github.com/apache/kafka/pull/970



On Wed, Mar 9, 2016 at 10:18 PM, Guozhang Wang  wrote:

> Hey Ewen,
>
> For "its own page" which location are you specifically thinking about?
>
> Guozhang
>
> On Wed, Mar 9, 2016 at 6:48 AM, Grant Henke  wrote:
>
> > Ewen,
> >
> > I am okay with that. I agree something is better than nothing. I can
> touch
> > up the patch today and open some jiras to track future work.
> >
> > Thanks,
> > Grant
> >
> > On Wed, Mar 9, 2016 at 1:50 AM, Ewen Cheslack-Postava  >
> > wrote:
> >
> > > Agreed w/ Jay's comment on the PR that this is a great idea. What if we
> > > included this as is, but instead of putting it all on the docs page we
> > put
> > > it on its own page? This:
> > >
> > > 1. Gets us something useful wrt protocol docs immediately. As you point
> > > out, this is already useful to a bunch of people.
> > > 2. Doesn't block on us figuring out a better way of managing docs
> > (although
> > > this may need adjustment once we choose a different format for docs).
> > > 3. Gets the code checked in so any requests for refinements around
> > protocol
> > > docs can often be resolved with a patch to the code for autogenerating
> > > them.
> > >
> > > -Ewen
> > >
> > > On Wed, Feb 24, 2016 at 11:57 PM, Grant Henke 
> > wrote:
> > >
> > > > We have gotten a lot of requests for updating the protocol
> > documentation:
> > > >
> > > >- KAFKA-3124: Update protocol wiki page to reflect latest
> > > >request/response formats
> > > >- KAFKA-2865: Improve Request API Error Code Documentation
> > > >- KAFKA-3099: Please update A+Guide+To+The+Kafka+Protocol Wiki
> > > >- KAFKA-2495: Protocol guide only partially updated after
> > > >ConsumerMetadata* addition?
> > > >- KAFKA-2494: Document ReplicaId in OffsetRequest in the protocol
> > > guide
> > > >
> > > > Generating some docs to go either in the wiki or on the actual
> > > > documentation site would definitely be valuable and ensure our
> protocol
> > > > docs are up to date. Although the output could be fairly large, so I
> am
> > > not
> > > > sure it should go on the main page of our "one page" documentation
> > site,
> > > as
> > > > the page is already fairly long to scroll.
> > > >
> > > > I made a very rough, super hacky, implementation that outputs some
> > useful
> > > > data in a similar format to the existing wiki. There is a pull
> request
> > > and
> > > > a gist of the output for anyone that it could help in the mean time
> > > (links
> > > > below).  In the patch I listed some "todo" comments for things I
> would
> > > > still like to see in the output. I don't have time to polish up the
> > patch
> > > > to completion now, due to other priorities.
> > > >
> > > > Is anyone interested in finishing the patch? If so please feel free
> to
> > > use
> > > > or ignore the code shared.
> > > >
> > > >- Pull request: https://github.com/apache/kafka/pull/970
> > > >- Gist of output:
> > > >
> > > >
> > >
> >
> https://gist.github.com/granthenke/dae9e60b5e548acead58#file-kafka_protocol-html
> > > >- In browser preview of Gist:
> > > >
> > > >
> > >
> >
> https://rawgit.com/granthenke/dae9e60b5e548acead58/raw/a2da4279d6fd11e8056927695c7c091e58607f38/kafka_protocol.html
> > > >
> > > > Thanks,
> > > > Grant
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > gr...@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Ewen
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
>
>
> --
> -- Guozhang
>



-- 
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke

Re: Protocol Documentation WIP

[Guozhang Wang]

Hey Ewen,

For "its own page" which location are you specifically thinking about?

Guozhang

On Wed, Mar 9, 2016 at 6:48 AM, Grant Henke  wrote:

> Ewen,
>
> I am okay with that. I agree something is better than nothing. I can touch
> up the patch today and open some jiras to track future work.
>
> Thanks,
> Grant
>
> On Wed, Mar 9, 2016 at 1:50 AM, Ewen Cheslack-Postava 
> wrote:
>
> > Agreed w/ Jay's comment on the PR that this is a great idea. What if we
> > included this as is, but instead of putting it all on the docs page we
> put
> > it on its own page? This:
> >
> > 1. Gets us something useful wrt protocol docs immediately. As you point
> > out, this is already useful to a bunch of people.
> > 2. Doesn't block on us figuring out a better way of managing docs
> (although
> > this may need adjustment once we choose a different format for docs).
> > 3. Gets the code checked in so any requests for refinements around
> protocol
> > docs can often be resolved with a patch to the code for autogenerating
> > them.
> >
> > -Ewen
> >
> > On Wed, Feb 24, 2016 at 11:57 PM, Grant Henke 
> wrote:
> >
> > > We have gotten a lot of requests for updating the protocol
> documentation:
> > >
> > >- KAFKA-3124: Update protocol wiki page to reflect latest
> > >request/response formats
> > >- KAFKA-2865: Improve Request API Error Code Documentation
> > >- KAFKA-3099: Please update A+Guide+To+The+Kafka+Protocol Wiki
> > >- KAFKA-2495: Protocol guide only partially updated after
> > >ConsumerMetadata* addition?
> > >- KAFKA-2494: Document ReplicaId in OffsetRequest in the protocol
> > guide
> > >
> > > Generating some docs to go either in the wiki or on the actual
> > > documentation site would definitely be valuable and ensure our protocol
> > > docs are up to date. Although the output could be fairly large, so I am
> > not
> > > sure it should go on the main page of our "one page" documentation
> site,
> > as
> > > the page is already fairly long to scroll.
> > >
> > > I made a very rough, super hacky, implementation that outputs some
> useful
> > > data in a similar format to the existing wiki. There is a pull request
> > and
> > > a gist of the output for anyone that it could help in the mean time
> > (links
> > > below).  In the patch I listed some "todo" comments for things I would
> > > still like to see in the output. I don't have time to polish up the
> patch
> > > to completion now, due to other priorities.
> > >
> > > Is anyone interested in finishing the patch? If so please feel free to
> > use
> > > or ignore the code shared.
> > >
> > >- Pull request: https://github.com/apache/kafka/pull/970
> > >- Gist of output:
> > >
> > >
> >
> https://gist.github.com/granthenke/dae9e60b5e548acead58#file-kafka_protocol-html
> > >- In browser preview of Gist:
> > >
> > >
> >
> https://rawgit.com/granthenke/dae9e60b5e548acead58/raw/a2da4279d6fd11e8056927695c7c091e58607f38/kafka_protocol.html
> > >
> > > Thanks,
> > > Grant
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
> >
> >
> > --
> > Thanks,
> > Ewen
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>



-- 
-- Guozhang

Re: Protocol Documentation WIP

[Grant Henke]

Ewen,

I am okay with that. I agree something is better than nothing. I can touch
up the patch today and open some jiras to track future work.

Thanks,
Grant

On Wed, Mar 9, 2016 at 1:50 AM, Ewen Cheslack-Postava 
wrote:

> Agreed w/ Jay's comment on the PR that this is a great idea. What if we
> included this as is, but instead of putting it all on the docs page we put
> it on its own page? This:
>
> 1. Gets us something useful wrt protocol docs immediately. As you point
> out, this is already useful to a bunch of people.
> 2. Doesn't block on us figuring out a better way of managing docs (although
> this may need adjustment once we choose a different format for docs).
> 3. Gets the code checked in so any requests for refinements around protocol
> docs can often be resolved with a patch to the code for autogenerating
> them.
>
> -Ewen
>
> On Wed, Feb 24, 2016 at 11:57 PM, Grant Henke  wrote:
>
> > We have gotten a lot of requests for updating the protocol documentation:
> >
> >- KAFKA-3124: Update protocol wiki page to reflect latest
> >request/response formats
> >- KAFKA-2865: Improve Request API Error Code Documentation
> >- KAFKA-3099: Please update A+Guide+To+The+Kafka+Protocol Wiki
> >- KAFKA-2495: Protocol guide only partially updated after
> >ConsumerMetadata* addition?
> >- KAFKA-2494: Document ReplicaId in OffsetRequest in the protocol
> guide
> >
> > Generating some docs to go either in the wiki or on the actual
> > documentation site would definitely be valuable and ensure our protocol
> > docs are up to date. Although the output could be fairly large, so I am
> not
> > sure it should go on the main page of our "one page" documentation site,
> as
> > the page is already fairly long to scroll.
> >
> > I made a very rough, super hacky, implementation that outputs some useful
> > data in a similar format to the existing wiki. There is a pull request
> and
> > a gist of the output for anyone that it could help in the mean time
> (links
> > below).  In the patch I listed some "todo" comments for things I would
> > still like to see in the output. I don't have time to polish up the patch
> > to completion now, due to other priorities.
> >
> > Is anyone interested in finishing the patch? If so please feel free to
> use
> > or ignore the code shared.
> >
> >- Pull request: https://github.com/apache/kafka/pull/970
> >- Gist of output:
> >
> >
> https://gist.github.com/granthenke/dae9e60b5e548acead58#file-kafka_protocol-html
> >- In browser preview of Gist:
> >
> >
> https://rawgit.com/granthenke/dae9e60b5e548acead58/raw/a2da4279d6fd11e8056927695c7c091e58607f38/kafka_protocol.html
> >
> > Thanks,
> > Grant
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>
>
>
> --
> Thanks,
> Ewen
>



-- 
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke

Re: Protocol Documentation WIP

[Ewen Cheslack-Postava]

Agreed w/ Jay's comment on the PR that this is a great idea. What if we
included this as is, but instead of putting it all on the docs page we put
it on its own page? This:

1. Gets us something useful wrt protocol docs immediately. As you point
out, this is already useful to a bunch of people.
2. Doesn't block on us figuring out a better way of managing docs (although
this may need adjustment once we choose a different format for docs).
3. Gets the code checked in so any requests for refinements around protocol
docs can often be resolved with a patch to the code for autogenerating them.

-Ewen

On Wed, Feb 24, 2016 at 11:57 PM, Grant Henke  wrote:

> We have gotten a lot of requests for updating the protocol documentation:
>
>- KAFKA-3124: Update protocol wiki page to reflect latest
>request/response formats
>- KAFKA-2865: Improve Request API Error Code Documentation
>- KAFKA-3099: Please update A+Guide+To+The+Kafka+Protocol Wiki
>- KAFKA-2495: Protocol guide only partially updated after
>ConsumerMetadata* addition?
>- KAFKA-2494: Document ReplicaId in OffsetRequest in the protocol guide
>
> Generating some docs to go either in the wiki or on the actual
> documentation site would definitely be valuable and ensure our protocol
> docs are up to date. Although the output could be fairly large, so I am not
> sure it should go on the main page of our "one page" documentation site, as
> the page is already fairly long to scroll.
>
> I made a very rough, super hacky, implementation that outputs some useful
> data in a similar format to the existing wiki. There is a pull request and
> a gist of the output for anyone that it could help in the mean time (links
> below).  In the patch I listed some "todo" comments for things I would
> still like to see in the output. I don't have time to polish up the patch
> to completion now, due to other priorities.
>
> Is anyone interested in finishing the patch? If so please feel free to use
> or ignore the code shared.
>
>- Pull request: https://github.com/apache/kafka/pull/970
>- Gist of output:
>
> https://gist.github.com/granthenke/dae9e60b5e548acead58#file-kafka_protocol-html
>- In browser preview of Gist:
>
> https://rawgit.com/granthenke/dae9e60b5e548acead58/raw/a2da4279d6fd11e8056927695c7c091e58607f38/kafka_protocol.html
>
> Thanks,
> Grant
> --
> Grant Henke
> Software Engineer | Cloudera
> gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>



-- 
Thanks,
Ewen

[jira] [Commented] (KAFKA-3309) Update Protocol Documentation WIP patch

[Ashish K Singh (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15185233#comment-15185233
 ] 

Ashish K Singh commented on KAFKA-3309:
---

[~granthenke] it might need an update based on whether or not we add the 
deprecation list in Protocol. Based on recent development, it looks like we 
need a mechanism for deprecation, however that mechanism could be as simple as 
manually updating docs to reflect deprecation. Based on how discussion for 
KIP-35 goes, we can modify the intent or close the JIRA. Sounds reasonable?

> Update Protocol Documentation WIP patch
> ---
>
> Key: KAFKA-3309
> URL: https://issues.apache.org/jira/browse/KAFKA-3309
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Ashish K Singh
>Assignee: Ashish K Singh
>
> https://github.com/apache/kafka/pull/970 has the WIP patch. Update to reflect 
> changes made for KIP-35.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-3309) Update Protocol Documentation WIP patch

[Grant Henke (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15185114#comment-15185114
 ] 

Grant Henke commented on KAFKA-3309:


[~singhashish] This patch is not in yet. So I may update it as protocol changes 
are committed, but I don't think we need to track it. There is no work that 
needs to be done to update it other than rebasing since the output is 
auto-generated. 

> Update Protocol Documentation WIP patch
> ---
>
> Key: KAFKA-3309
> URL: https://issues.apache.org/jira/browse/KAFKA-3309
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Ashish K Singh
>Assignee: Ashish K Singh
>
> https://github.com/apache/kafka/pull/970 has the WIP patch. Update to reflect 
> changes made for KIP-35.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KAFKA-3309) Update Protocol Documentation WIP patch

[Ashish K Singh (JIRA)]

Ashish K Singh created KAFKA-3309:
-

 Summary: Update Protocol Documentation WIP patch
 Key: KAFKA-3309
 URL: https://issues.apache.org/jira/browse/KAFKA-3309
 Project: Kafka
  Issue Type: Sub-task
Reporter: Ashish K Singh
Assignee: Ashish K Singh


https://github.com/apache/kafka/pull/970 has the WIP patch. Update to reflect 
changes made for KIP-35.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Protocol Documentation WIP

[Grant Henke]

We have gotten a lot of requests for updating the protocol documentation:

   - KAFKA-3124: Update protocol wiki page to reflect latest
   request/response formats
   - KAFKA-2865: Improve Request API Error Code Documentation
   - KAFKA-3099: Please update A+Guide+To+The+Kafka+Protocol Wiki
   - KAFKA-2495: Protocol guide only partially updated after
   ConsumerMetadata* addition?
   - KAFKA-2494: Document ReplicaId in OffsetRequest in the protocol guide

Generating some docs to go either in the wiki or on the actual
documentation site would definitely be valuable and ensure our protocol
docs are up to date. Although the output could be fairly large, so I am not
sure it should go on the main page of our "one page" documentation site, as
the page is already fairly long to scroll.

I made a very rough, super hacky, implementation that outputs some useful
data in a similar format to the existing wiki. There is a pull request and
a gist of the output for anyone that it could help in the mean time (links
below).  In the patch I listed some "todo" comments for things I would
still like to see in the output. I don't have time to polish up the patch
to completion now, due to other priorities.

Is anyone interested in finishing the patch? If so please feel free to use
or ignore the code shared.

   - Pull request: https://github.com/apache/kafka/pull/970
   - Gist of output:
   
https://gist.github.com/granthenke/dae9e60b5e548acead58#file-kafka_protocol-html
   - In browser preview of Gist:
   
https://rawgit.com/granthenke/dae9e60b5e548acead58/raw/a2da4279d6fd11e8056927695c7c091e58607f38/kafka_protocol.html

Thanks,
Grant
-- 
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke