Sure, I can try cutting out PTransform. We could also look into reducing noise by: - removing undoc-members from the config [1] (this would make it so only objects with a docstring are added to the generated docs) - adding :meta private:` to docstrings for objects we don't want publicly visible
[1] https://github.com/apache/beam/blob/243128a8fc52798e1b58b0cf1a271d95ee7aa241/sdks/python/scripts/generate_pydoc.sh#L48 On Tue, Apr 6, 2021 at 1:17 PM Robert Bradshaw <rober...@google.com> wrote: > Way too many things are inherited from PTransform, can we at least cut > that out? > > On Tue, Apr 6, 2021 at 1:09 PM Brian Hulette <bhule...@google.com> wrote: > >> Just wanted to bump this - does anyone have concerns with the way the API >> docs look when inherited members are included? >> >> On Wed, Mar 31, 2021 at 5:23 PM Brian Hulette <bhule...@google.com> >> wrote: >> >>> I staged my current working copy built from head here [1], see >>> CombinePerKey here [2]. Note it also has a few other changes, most notably >>> I excluded several internal-only modules that are currently in our API docs >>> (I will PR this soon regardless). >>> >>> > are these inherited members grouped in such a way that it makes it >>> easy to ignore them once they get to "low" in the stack? >>> There doesn't seem to be any grouping, but it does look like inherited >>> members are added at the end. >>> >>> > If it can't be per-module, is there a "nice" set of ancestors to avoid >>> (as it seems this option takes such an argument). >>> Ah good point, I missed this. I suppose we could avoid basic constructs >>> like PTransform, DoFn, etc. I'm not sure how realistic that is though. It >>> would be nice if this argument worked the other way >>> >>> [1] https://theneuralbit.github.io/beam-site/pydoc/inherited-members >>> [2] >>> https://theneuralbit.github.io/beam-site/pydoc/inherited-members/apache_beam.transforms.core.html#apache_beam.transforms.core.CombinePerKey >>> >>> On Wed, Mar 31, 2021 at 4:45 PM Robert Bradshaw <rober...@google.com> >>> wrote: >>> >>>> +1 to an example. In particular, are these inherited members grouped in >>>> such a way that it makes it easy to ignore them once they get to "low" in >>>> the stack? If it can't be per-module, is there a "nice" set of ancestors to >>>> avoid (as it seems this option takes such an argument). >>>> >>>> On Wed, Mar 31, 2021 at 4:23 PM Pablo Estrada <pabl...@google.com> >>>> wrote: >>>> >>>>> Do you have an example of what it would look like when released? >>>>> >>>>> On Wed, Mar 31, 2021 at 4:16 PM Brian Hulette <bhule...@google.com> >>>>> wrote: >>>>> >>>>>> I'm working on generating useful API docs for the DataFrame API >>>>>> (BEAM-12074). In doing so, one thing I've found would be very helpful is >>>>>> if >>>>>> we could include docstrings for inherited members in the API docs. That >>>>>> way >>>>>> docstrings for operations defined in DeferredDataFrameOrSeries [1], will >>>>>> be >>>>>> propagated to DeferredDataFrame [2] and DeferredSeries, and the former >>>>>> can >>>>>> be hidden entirely. This would be more consistent with the pandas >>>>>> documentation [3]. >>>>>> >>>>>> It looks like we can do this by specifying :inherited-members: [4], >>>>>> but this will apply to _all_ of our API docs, there doesn't seem to be a >>>>>> way to restrict it to a particular module. This seems generally useful to >>>>>> me, but it would be a significant change, so I wanted to see if there are >>>>>> any objections from dev@ before doing this. >>>>>> >>>>>> An example of the kind of change this would produce: any PTransform >>>>>> sub-classes, e.g. CombinePerKey [5], would now include docstrings for >>>>>> every >>>>>> PTransform member, e.g. with_input_types [6], and display_data [7]. >>>>>> >>>>>> Would there be any objections to that? >>>>>> >>>>>> Thanks, >>>>>> Brian >>>>>> >>>>>> [1] >>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.dataframe.frames.html#apache_beam.dataframe.frames.DeferredDataFrameOrSeries >>>>>> [2] >>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.dataframe.frames.html#apache_beam.dataframe.frames.DeferredDataFrame >>>>>> [3] https://pandas.pydata.org/docs/reference/frame.html >>>>>> [4] >>>>>> https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html >>>>>> [5] >>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.transforms.core.html?highlight=combineperkey#apache_beam.transforms.core.CombinePerKey >>>>>> [6] >>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.transforms.ptransform.html#apache_beam.transforms.ptransform.PTransform.with_input_types >>>>>> [7] >>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.transforms.display.html#apache_beam.transforms.display.HasDisplayData.display_data >>>>>> >>>>>
