[ https://issues.apache.org/jira/browse/DRILL-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027489#comment-15027489 ]
Jacques Nadeau commented on DRILL-4130: --------------------------------------- I also agree: get table options out of the session level. System makes sense for system defaults. > Ability to set settings at Table or View level rather than SESSION or SYSTEM > ---------------------------------------------------------------------------- > > Key: DRILL-4130 > URL: https://issues.apache.org/jira/browse/DRILL-4130 > Project: Apache Drill > Issue Type: Improvement > Components: Metadata > Affects Versions: 1.3.0 > Environment: All > Reporter: John Omernik > Labels: administration, settings > Fix For: Future > > > There are a number of settings within drill for handling data that due to low > level of granularity there may be unintended data reading consequences. A few > examples include: > store.json.read_numbers_as_double > and > store.json.all_text_mode > (There are likely more, these are some I've worked with) > The documentation on https://drill.apache.org/docs/json-data-model/ outlines > how when dealing with certain types of data, that these settings can be > helpful for reading, and indeed some queries fail with a suggestion to change > these settings. > A few points here. 1. The documentation suggests alter system commands. This > is not ideal as it changes the default way drill handles data for all users > AND not all users will (should) have the privs to enter this command. The > documentation at a minimum should show alter session (or provide a clearer > understanding of the difference) > But even with alter session, that affects reads for all JSON files for that > session, when in reality, the reasoning behind the setting is to be able to > read a specific table that has poorly formed JSON. Thus, issuing a command > that alters how Drill reads all JSON in order to read one table of JSON could > have unintended consequences, especially for a user who just wants to be able > to read things and issues commands without thinking things through. > Now as an administrator, there are two use cases here. One is I have a table > of poorly formed JSON that requires one of these settings, and I can't change > the source, therefore, can I create a view that makes it so all reads of this > table are done with the more permissive setting? Setting these in a view > would be very helpful from an administrator perspective for known bad data > sources. Keep users from having to think about it, and let them do their > exploration. > The other use case, is the ability for a user to set a session level read > that only applies for the table being read. alter session set > "%tablename%.store.json.read_numbers_as_double = true" (and have the errors > that display use that as the default suggestion) that way, the user can issue > the command, but not have downstream consequences in their session while > reading other tables. > Either case is valuable to an administrator, and could help prevent data read > issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)