Looking at [1] I see all the methods on the DF but not the pydoc. Will the Pydoc be included later somehow? Best -P.
[1] https://theneuralbit.github.io/beam-site/pydoc/inherited-members/apache_beam.dataframe.frames.html On Thu, Apr 8, 2021 at 1:05 PM Brian Hulette <[email protected]> wrote: > I found a slightly hacky way to enable :inherited-members: just for the > DataFrame API. I can add the option to the .rst output generated by > sphinx-apidoc, before we run sphinx-build [1]. > > I'm fine just doing that instead of turning it on globally. > > [1] > https://github.com/TheNeuralBit/beam/blob/e26760937f7a34fd72578b65f716098c74e4380b/sdks/python/scripts/generate_pydoc.sh#L86 > > On Tue, Apr 6, 2021 at 1:50 PM Brian Hulette <[email protected]> wrote: > >> Sure, I can try cutting out PTransform. >> >> We could also look into reducing noise by: >> - removing undoc-members from the config [1] (this would make it so only >> objects with a docstring are added to the generated docs) >> - adding :meta private:` to docstrings for objects we don't want publicly >> visible >> >> [1] >> https://github.com/apache/beam/blob/243128a8fc52798e1b58b0cf1a271d95ee7aa241/sdks/python/scripts/generate_pydoc.sh#L48 >> >> On Tue, Apr 6, 2021 at 1:17 PM Robert Bradshaw <[email protected]> >> wrote: >> >>> Way too many things are inherited from PTransform, can we at least cut >>> that out? >>> >>> On Tue, Apr 6, 2021 at 1:09 PM Brian Hulette <[email protected]> >>> wrote: >>> >>>> Just wanted to bump this - does anyone have concerns with the way the >>>> API docs look when inherited members are included? >>>> >>>> On Wed, Mar 31, 2021 at 5:23 PM Brian Hulette <[email protected]> >>>> wrote: >>>> >>>>> I staged my current working copy built from head here [1], see >>>>> CombinePerKey here [2]. Note it also has a few other changes, most notably >>>>> I excluded several internal-only modules that are currently in our API >>>>> docs >>>>> (I will PR this soon regardless). >>>>> >>>>> > are these inherited members grouped in such a way that it makes it >>>>> easy to ignore them once they get to "low" in the stack? >>>>> There doesn't seem to be any grouping, but it does look like inherited >>>>> members are added at the end. >>>>> >>>>> > If it can't be per-module, is there a "nice" set of ancestors to >>>>> avoid (as it seems this option takes such an argument). >>>>> Ah good point, I missed this. I suppose we could avoid basic >>>>> constructs like PTransform, DoFn, etc. I'm not sure how realistic that is >>>>> though. It would be nice if this argument worked the other way >>>>> >>>>> [1] https://theneuralbit.github.io/beam-site/pydoc/inherited-members >>>>> [2] >>>>> https://theneuralbit.github.io/beam-site/pydoc/inherited-members/apache_beam.transforms.core.html#apache_beam.transforms.core.CombinePerKey >>>>> >>>>> On Wed, Mar 31, 2021 at 4:45 PM Robert Bradshaw <[email protected]> >>>>> wrote: >>>>> >>>>>> +1 to an example. In particular, are these inherited members grouped >>>>>> in such a way that it makes it easy to ignore them once they get to "low" >>>>>> in the stack? If it can't be per-module, is there a "nice" set of >>>>>> ancestors >>>>>> to avoid (as it seems this option takes such an argument). >>>>>> >>>>>> On Wed, Mar 31, 2021 at 4:23 PM Pablo Estrada <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Do you have an example of what it would look like when released? >>>>>>> >>>>>>> On Wed, Mar 31, 2021 at 4:16 PM Brian Hulette <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> I'm working on generating useful API docs for the DataFrame API >>>>>>>> (BEAM-12074). In doing so, one thing I've found would be very helpful >>>>>>>> is if >>>>>>>> we could include docstrings for inherited members in the API docs. >>>>>>>> That way >>>>>>>> docstrings for operations defined in DeferredDataFrameOrSeries [1], >>>>>>>> will be >>>>>>>> propagated to DeferredDataFrame [2] and DeferredSeries, and the former >>>>>>>> can >>>>>>>> be hidden entirely. This would be more consistent with the pandas >>>>>>>> documentation [3]. >>>>>>>> >>>>>>>> It looks like we can do this by specifying :inherited-members: [4], >>>>>>>> but this will apply to _all_ of our API docs, there doesn't seem to be >>>>>>>> a >>>>>>>> way to restrict it to a particular module. This seems generally useful >>>>>>>> to >>>>>>>> me, but it would be a significant change, so I wanted to see if there >>>>>>>> are >>>>>>>> any objections from dev@ before doing this. >>>>>>>> >>>>>>>> An example of the kind of change this would produce: any PTransform >>>>>>>> sub-classes, e.g. CombinePerKey [5], would now include docstrings for >>>>>>>> every >>>>>>>> PTransform member, e.g. with_input_types [6], and display_data [7]. >>>>>>>> >>>>>>>> Would there be any objections to that? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Brian >>>>>>>> >>>>>>>> [1] >>>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.dataframe.frames.html#apache_beam.dataframe.frames.DeferredDataFrameOrSeries >>>>>>>> [2] >>>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.dataframe.frames.html#apache_beam.dataframe.frames.DeferredDataFrame >>>>>>>> [3] https://pandas.pydata.org/docs/reference/frame.html >>>>>>>> [4] >>>>>>>> https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html >>>>>>>> [5] >>>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.transforms.core.html?highlight=combineperkey#apache_beam.transforms.core.CombinePerKey >>>>>>>> [6] >>>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.transforms.ptransform.html#apache_beam.transforms.ptransform.PTransform.with_input_types >>>>>>>> [7] >>>>>>>> https://beam.apache.org/releases/pydoc/2.27.0/apache_beam.transforms.display.html#apache_beam.transforms.display.HasDisplayData.display_data >>>>>>>> >>>>>>>
