So what would need to be done to get this process kick started? I see a few components here:
1. Develop the table in sys (sys.functions) that stores the information about the function. For this I propose this for discussions name - The name of the function description - The Description of the function docgroup- This should be the groupings under "SQL Functions" in the documentation. See * below for proposal tags - A way to tag functions for easy searching like select from sys.functions where tags like '%string%' arguments - The Ordered list of arguments and their types return - What is returned (and it's type) Examples: Usage examples that can display nicely in both select * from sys.functions as well as provide enough information for use in HTML Docs. * docgroup. Currently, in the documentation, the nesting is as below (I didn't expand all the stuff on SQL Window and Nested Data). I want to outline my proposal for to hand handle this... So I propose moving "SQL Window Functions", "Nested Data Functions" and "Query Directory Functions" all under SQL Functions and adding another group here "Common Functions". This will now be the "Functions root". The Docgroup field in sys.function will have levels and grouping separated by ":". Thus, firstlevel:second:level or firstlevel:secondlevel:thirdlevel This will allow us to keep the grouping in the sys.functions table, and make it so that on every release, the documentation could be updated with a query. Note the level "SQL Functions" would not be represented in the docgroup. Here are some examples of functions LOWER() Current Doc Placement: SQL Functions -> String Manipulation Proposed docgroup: 'Common Functions:String Manipulation' How it would appear on the Doc page: SQL Functions -> Common Functions -> String Manipulation COUNT() Current Doc Placement: SQL Functions -> SQL Window Functions -> Aggregate Window Functions Proposed docgroup: 'SQL Window Functions:Aggregate Window Functions' How it would appear on the Doc pages: SQL Functions -> SQL Window Functions -> Aggregate Window Functions Basically it would be used to add the raw HTML to the final page in the docgroup (Aggregate Window Functions) Current Doc Layout: SQL Reference SQL Reference Introduction -> Data Types Lexical Structure Operators -> SQL Functions ->-> About SQL Function Examples ->-> Math and Trig ->->Data Type Conversion ->->Date/Time Functions and Arithmetic ->->String Manipulation ->->Aggregate and Aggregate Statistical ->-> Functions for handling Nulls -> SQL Window Functions ->-> ... ->-> ... -> Nested Data Functions ->->... ->->... -> Query Directory Functions 2. Once we get to a certain point in development on sys.functions, we need a call to arms. We need to come up with an initial "fill" of sys.functions. For that we'll need to take current data fill it in, as well as getting a list of all the ninja functions that have been added to drill and not documented... not sure how get those with out an intense code/jira review. 3. Come up with new function proposal guidelines. If you do a pull request with a function, what will need to be included for your pull request to be approved? We should not allow functions to be added to Drill without a basic doc addition. 4. Update procedures? This is complicated, but done well, it could really put the knowledge and analyst needs right in the system itself! John On Tue, Feb 28, 2017 at 12:20 PM, John Omernik <j...@omernik.com> wrote: > You could also generate documentation updates via query at each release. > This would be a great feature, move the information close to the analysts > hands, I love how that would work. (I think I remember some talk about > extending sys.options to be self documenting as well.... ) > > > > On Tue, Feb 28, 2017 at 12:11 PM, Jinfeng Ni <j...@apache.org> wrote: > >> Regarding the list of functions (build-in or UDF), someone once >> suggested that we make the functions self-documented by adding a >> sys.functions table. >> >> select * from sys.functions where name like '%SPLIT%'; >> >> return function_name, parameter_list, description etc. >> >> This way, use could simply query sys.functions using Drill. >> >> >> >> >> >> >> >> On Tue, Feb 28, 2017 at 9:02 AM, Jinfeng Ni <j...@apache.org> wrote: >> >> 4. I think as part of developer review and pull requests that add >> >> functions/functionality should require a pull request to also provide a >> >> documentation update. This helps to ensure that the docs keep up to >> date, >> >> as well as keeping users appraised of what is happening... i.e. it's a >> good >> >> "feeling" to see a great tool like Drill "improving" with new >> >> functionality. >> >> >> >> Please, folks, we need to do some one time clean up (go back through >> pull >> >> requests to ensure all functions are documented up to now) and then >> then >> >> get processes in place to ensure ongoing updates. >> >> >> > >> > That's a good suggestion. We should try our best to keep the code and >> > doc in sync. >> > >> > +1 >> > >
