> On April 8, 2015, 6:20 a.m., Rajat Khandelwal wrote: > > Can we be sure that other default values of hiveconf are useless for the > > driver?
I hope the question matters only for HiveDriver. The answer is Yes. In this case of HiveDriver, whatever conf that is getting passed is only overlay on top of HiveConf(that is already created by HiveServer) for the query. And overlay shouldnt pass simple default values as actually HiveServer would have overridden some of the values in its hive-site.xml. The issue we noticed with respect to NPE is because core-default, but all variables overridden in hive-site.xml will also be ignored, when defaults are passed on top of it. Earlier to LENS-480 (without the fix for priority), the conf that as passed was only query conf. That was missing conf set at driver level. With passing driver level conf and with this fix, default values are not getting passed - the correct overlay config will be passed now. - Amareshwari ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32958/#review79311 ----------------------------------------------------------- On April 8, 2015, 5:42 a.m., Amareshwari Sriramadasu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32958/ > ----------------------------------------------------------- > > (Updated April 8, 2015, 5:42 a.m.) > > > Review request for lens, Jaideep dhok and Rajat Khandelwal. > > > Bugs: LENS-480 > https://issues.apache.org/jira/browse/LENS-480 > > > Repository: lens > > > Description > ------- > > NPE happens because the fs.default.name name config is set to default > file:/// . This is happening because the driver configuration is getting > created with HiveConf() which loads all hive-default.xml and > core-default.xml. We need to create configuration without loading defaults. > As HiveConf() does not allow to create one with the changes here create a > Configuration without loading defaults and passes to driver.configure() > > > Diffs > ----- > > lens-server/src/main/java/org/apache/lens/server/LensApplication.java > dd3a4b3 > lens-server/src/main/java/org/apache/lens/server/LensServer.java fe348a1 > lens-server/src/main/java/org/apache/lens/server/LensServerConf.java > 38cd14f > lens-server/src/main/java/org/apache/lens/server/LensServices.java 1484e47 > > lens-server/src/main/java/org/apache/lens/server/LensServletContextListener.java > a801e3c > > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java > 3e0e0db > lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java > 1f3fe68 > lens-server/src/test/java/org/apache/lens/server/TestLensServer.java > d7ac06c > lens-server/src/test/java/org/apache/lens/server/TestServiceProvider.java > 5dfe780 > > lens-server/src/test/java/org/apache/lens/server/query/TestEventService.java > 3684af1 > > lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java > bce0ca9 > > lens-server/src/test/java/org/apache/lens/server/session/TestSessionClassLoaders.java > 6f6208a > > lens-server/src/test/java/org/apache/lens/server/session/TestSessionExpiry.java > 6a30e90 > > lens-server/src/test/java/org/apache/lens/server/stats/TestLogStatisticsStore.java > ce74f9d > > lens-server/src/test/java/org/apache/lens/server/stats/TestStatisticsLogPartitionHandler.java > 2084ced > > lens-server/src/test/java/org/apache/lens/server/user/TestUserConfigLoader.java > 5187f2e > lens-server/src/test/resources/hivedriver-site.xml e4208b4 > lens-server/src/test/resources/jdbcdriver-site.xml 752372a > > Diff: https://reviews.apache.org/r/32958/diff/ > > > Testing > ------- > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Lens Checkstyle Rules ............................. SUCCESS [2.061s] > [INFO] Lens .............................................. SUCCESS [2.009s] > [INFO] Lens API .......................................... SUCCESS [5.618s] > [INFO] Lens API for server and extensions ................ SUCCESS [6.956s] > [INFO] Lens Cube ......................................... SUCCESS [2:19.753s] > [INFO] Lens DB storage ................................... SUCCESS [10.287s] > [INFO] Lens Query Library ................................ SUCCESS [5.015s] > [INFO] Lens Hive Driver .................................. SUCCESS [2:31.739s] > [INFO] Lens Driver for JDBC .............................. SUCCESS [19.822s] > [INFO] Lens Server ....................................... SUCCESS [4:41.906s] > [INFO] Lens client ....................................... SUCCESS [21.169s] > [INFO] Lens CLI .......................................... SUCCESS [2:39.705s] > [INFO] Lens Examples ..................................... SUCCESS [0.870s] > [INFO] Lens Distribution ................................. SUCCESS [9.193s] > [INFO] Lens ML Lib ....................................... SUCCESS [1:00.907s] > [INFO] Lens ML Ext Distribution .......................... SUCCESS [2.513s] > [INFO] Lens Regression ................................... SUCCESS [0.507s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 14:40.934s > [INFO] Finished at: Wed Apr 08 05:25:56 UTC 2015 > [INFO] Final Memory: 96M/921M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Amareshwari Sriramadasu > >
