All of War and Peace is only 3MB. Let people document however they want. Don't over-optimize for problems that have never occurred.
On Fri, Mar 3, 2017 at 3:19 PM, Kunal Khatua <kkha...@mapr.com> wrote: > It might be, incase someone begins to dump a massive design doc into the > comment field for a view's JSON. > > > I'm also not sure about how this information can be consumed. If it is > through CLI, either we rely on the SQLLine shell to trim the output, or not > worry at all. I'm assuming we'd also probably want something like a > > DESCRIBE VIEW ... > > to be enhanced to something like > > DESCRIBE VIEW WITH COMMENTARY ... > > > A 1KB field is quite generous IMHO. That's more than 7 tweets to describe > something ! [?] > > > Kunal Khatua > > ________________________________ > From: Ted Dunning <ted.dunn...@gmail.com> > Sent: Friday, March 3, 2017 12:56:44 PM > To: user > Subject: Re: Discussion: Comments in Drill Views > > It it really necessary to put a technical limit in to prevent people from > OVER-documenting views? > > > What is the last time you saw code that had too many comments in it? > > > > On Thu, Mar 2, 2017 at 8:42 AM, John Omernik <j...@omernik.com> wrote: > > > So I think on your worry that's an easily definable "abuse" condition... > > i.e. if we set a limit of say 1024 characters, that provides ample space > > for descriptions, but at 1kb per view, that's an allowable condition, > i.e. > > it would be hard to abuse it ... or am I missing something? > > > > On Wed, Mar 1, 2017 at 8:08 PM, Kunal Khatua <kkha...@mapr.com> wrote: > > > > > +1 > > > > > > > > > I this this can be very useful. The only worry is of someone abusing > it, > > > so we probably should have a limit on the size of this? Not sure else > it > > > could be exposed and consumed. > > > > > > > > > Kunal Khatua > > > > > > Engineering > > > > > > [MapR]<http://www.mapr.com/> > > > > > > www.mapr.com<http://www.mapr.com/> > > > > > > ________________________________ > > > From: John Omernik <j...@omernik.com> > > > Sent: Wednesday, March 1, 2017 9:55:27 AM > > > To: user > > > Subject: Re: Discussion: Comments in Drill Views > > > > > > Sorry, I let this idea drop (I didn't follow up and found when > searching > > > for something else...) Any other thoughts on this idea? > > > > > > Should I open a JIRA if people think it would be handy? > > > > > > On Thu, Jun 23, 2016 at 4:02 PM, Ted Dunning <ted.dunn...@gmail.com> > > > wrote: > > > > > > > This is very interesting. I love docstrings in Lisp and Python and > > > Javadoc > > > > in Java. > > > > > > > > Basically this is like that, but for SQL. Very helpful. > > > > > > > > On Thu, Jun 23, 2016 at 11:48 AM, John Omernik <j...@omernik.com> > > wrote: > > > > > > > > > I am looking for discussion here. A colleague was asking me how to > > add > > > > > comments to the metadata of a view. (He's new to Drill, thus the > > idea > > > of > > > > > not having metadata for a table is one he's warming up to). > > > > > > > > > > That got me thinking... why couldn't we use Drill Views to store > > > > > table/field comments? This could be a great way to help add > > contextual > > > > > information for users. Here's some current observations when I > issue > > a > > > > > describe view_myview > > > > > > > > > > > > > > > 1. I get three columns ,COLUMN_NAME, DATA_TYPE, and IS_NULLABLE > > > > > 2. Even thought the underlying parquet table has types, the view > does > > > not > > > > > pass the types for the underlying parquet files through. (The type > > is > > > > ANY) > > > > > 3. The data for the view is all just a json file that could be > easily > > > > > extended. > > > > > > > > > > > > > > > So, a few things would be a nice to have > > > > > > > > > > 1. Table comments. when I issue a describe table, if the view has > a > > > > > "Description" field, then having that print out as a description > for > > > the > > > > > whole view would be nice. This is harder, I think because it's not > > > just > > > > > extending the view information. > > > > > > > > > > 2. Column comments: A text field that could be added to the view, > > and > > > > just > > > > > print out another column with description. This would be very > > helpful. > > > > > While Drill being schemaless is awesome, the ability to add > > information > > > > to > > > > > known data, is huge. > > > > > > > > > > 3. Ability to to use the types from the Parquet files (without > > manually > > > > > specifying each type). If we could provide an option to View > > creation > > > to > > > > > attempt to infer type, that would be handy. I realize that folks > are > > > > using > > > > > the LIMIT 0 to get metadata, but describe could be done well too. > > > > > > > > > > 4. Ability, using ANSI Sql to update the view column descriptions > and > > > the > > > > > description for the view itself. > > > > > > > > > > 5. I believe Avro has the ability to add this information to the > > files, > > > > so > > > > > if the data exists outside of views (such as in AVRO files) should > we > > > > > present it to the user in describe table events as well? > > > > > > > > > > Curious if folks think this would be valuable, how much work an > > > addition > > > > > like this would be to Drill, and other thoughts in general. > > > > > > > > > > > > > > > John > > > > > > > > > > > > > > >
