Khurram Faraaz created DRILL-2700: ------------------------------------- Summary: Missing/deleted drillbit.log files on cluster are not reported Key: DRILL-2700 URL: https://issues.apache.org/jira/browse/DRILL-2700 Project: Apache Drill Issue Type: Bug Components: Tools, Build & Test Affects Versions: 0.9.0 Environment: 4 node cluster on CentOS Reporter: Khurram Faraaz Assignee: Steven Phillips
Missing drillbit.log files on cluster, under /opt/mapr/drill/drill-0.9.0/logs/ are not reported as missing. Instead, user is able to issue and execute SQL queries from sqlline prompt. Failing queries are also executed and errors/Exceptions are reported on sqlline prompt, but there is no corresponding drillbit.log file where details must be written to in the case of passing/failing queries. I have manually deleted the drillbit.log (users can accidentally delete drillbit.log), and continue to execute SQL from sqlline prompt, and in the case of failing SQL there will be no drillbit.log file since it was accidentally deleted, and hence stacktrace and other diagnostic information is lost. {code} [root@centos-01 bin]# clush -g khurram service mapr-warden stop 10.10.100.201: stopping WARDEN 10.10.100.202: stopping WARDEN 10.10.100.203: stopping WARDEN 10.10.100.204: stopping WARDEN 10.10.100.202: looking to stop mapr-core processes not started by warden 10.10.100.203: looking to stop mapr-core processes not started by warden 10.10.100.204: looking to stop mapr-core processes not started by warden 10.10.100.201: looking to stop mapr-core processes not started by warden [root@centos-01 bin]# clush -g khurram service mapr-warden start 10.10.100.204: Starting WARDEN, logging to /opt/mapr/logs/warden.log. 10.10.100.203: Starting WARDEN, logging to /opt/mapr/logs/warden.log. 10.10.100.202: Starting WARDEN, logging to /opt/mapr/logs/warden.log. 10.10.100.201: Starting WARDEN, logging to /opt/mapr/logs/warden.log. 10.10.100.204: . 10.10.100.203: . 10.10.100.204: For diagnostics look at /opt/mapr/logs/ for createsystemvolumes.log, warden.log and configured services log files 10.10.100.202: . 10.10.100.201: . 10.10.100.203: For diagnostics look at /opt/mapr/logs/ for createsystemvolumes.log, warden.log and configured services log files 10.10.100.202: For diagnostics look at /opt/mapr/logs/ for createsystemvolumes.log, warden.log and configured services log files 10.10.100.201: For diagnostics look at /opt/mapr/logs/ for createsystemvolumes.log, warden.log and configured services log files [root@centos-01 bin]# clush -g khurram ls -lrt /opt/mapr/drill/drill-0.9.0/logs/drillbit.log 10.10.100.201: -rw-r--r-- 1 mapr mapr 17522 Apr 6 20:49 /opt/mapr/drill/drill-0.9.0/logs/drillbit.log 10.10.100.204: -rw-r--r-- 1 mapr mapr 17522 Apr 6 20:49 /opt/mapr/drill/drill-0.9.0/logs/drillbit.log 10.10.100.202: -rw-r--r-- 1 mapr mapr 17522 Apr 6 20:49 /opt/mapr/drill/drill-0.9.0/logs/drillbit.log 10.10.100.203: -rw-r--r-- 1 mapr mapr 17522 Apr 6 20:49 /opt/mapr/drill/drill-0.9.0/logs/drillbit.log Manually remove drillbit.log files from cluster nodes [root@centos-01 bin]# clush -g khurram rm -rf /opt/mapr/drill/drill-0.9.0/logs/drillbit.log [root@centos-01 bin]# clush -g khurram ls -lrt /opt/mapr/drill/drill-0.9.0/logs/drillbit.log 10.10.100.201: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log: No such file or directory clush: 10.10.100.201: exited with exit code 2 10.10.100.204: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log: No such file or directory clush: 10.10.100.204: exited with exit code 2 10.10.100.203: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log: No such file or directory clush: 10.10.100.203: exited with exit code 2 10.10.100.202: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log: No such file or directory clush: 10.10.100.202: exited with exit code 2 [root@centos-01 bin]# ./sqlline -u "jdbc:drill:" Drill log directory: /opt/mapr/drill/drill-0.9.0/logs sqlline version 1.1.6 0: jdbc:drill:> use sys; +------------+------------+ | ok | summary | +------------+------------+ | true | Default schema changed to 'sys' | +------------+------------+ 1 row selected (1.199 seconds) 0: jdbc:drill:> select * from sys.version; +------------+----------------+-------------+-------------+------------+ | commit_id | commit_message | commit_time | build_email | build_time | +------------+----------------+-------------+-------------+------------+ | 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580: Exit early from HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53 EDT | Unknown | 26.03.2015 @ 16:59:31 EDT | +------------+----------------+-------------+-------------+------------+ 1 row selected (0.608 seconds) 0: jdbc:drill:> select * from sys.drillbits; +------------+------------+--------------+------------+------------+ | hostname | user_port | control_port | data_port | current | +------------+------------+--------------+------------+------------+ | centos-02.qa.lab | 31010 | 31011 | 31012 | true | | centos-01.qa.lab | 31010 | 31011 | 31012 | false | | centos-04.qa.lab | 31010 | 31011 | 31012 | false | | centos-03.qa.lab | 31010 | 31011 | 31012 | false | +------------+------------+--------------+------------+------------+ 4 rows selected (0.207 seconds) 0: jdbc:drill:> !q Closing: org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection Note that there are no drillbit.log files available. [root@centos-01 bin]# clush -g khurram ls -lrt /opt/mapr/drill/drill-0.9.0/logs/drillbit.log 10.10.100.201: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log: No such file or directory clush: 10.10.100.201: exited with exit code 2 10.10.100.203: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log: No such file or directory clush: 10.10.100.203: exited with exit code 2 10.10.100.204: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log: No such file or directory 10.10.100.202: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log: No such file or directory clush: 10.10.100.204: exited with exit code 2 clush: 10.10.100.202: exited with exit code 2 {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)