Re: move drill log directory to HDFS

[Paul Rogers]

I don’t believe it is possible to store Drill logs in HDFS using an HDFS URL. 
The log files are set up via environment variables in the drill-config.sh 
script and are expected to be normal file system files.

However, if you have the ability to NFS mount your distributed file system, 
then you can easily accomplish your goal. (MapR MFS is such an example, others 
may support this as well.) Let’s say that you mount your DFS at /mnt/dfs. Then, 
you can add the following to your drill-env.sh script:

export DRILL_LOG_DIR=/mnt/dfs/drill/`hostname`

The bit about hostname is needed to put the logs for each Drillbit into a 
separate directory. Feel free to use any other technique you like to label the 
directory.

As an alternative, we are making good progress in finalizing a YARN hosting 
solution for Drill. Once this is available (probably Drill 1.9), then YARN will 
take care of collecting your Drill log files.

- Paul

> On Aug 26, 2016, at 6:46 AM, Andries Engelbrecht  
> wrote:
> 
> https://drill.apache.org/docs/persistent-configuration-storage/ 
> 
> 
> 
> --Andries
> 
>> On Aug 26, 2016, at 4:25 AM, Anup Tiwari  wrote:
>> 
>> Also please note that I have tried below in all node's drill-env.sh but its
>> not working.
>> 
>> export DRILL_LOG_DIR="hdfs://namenode:9000/tmp/drilllogs/"
>> 
>> 
>> Regards,
>> *Anup Tiwari*
>> 
>> 
>> On Fri, Aug 26, 2016 at 4:06 PM, Anup Tiwari 
>> wrote:
>> 
>>> Hi All,
>>> 
>>> We are trying to move drill logs directory from local file system to HDFS
>>> so that we can refer only one location rather than each node's log
>>> directory.
>>> 
>>> Can anyone help me on this?
>>> 
>>> Regards,
>>> *Anup Tiwari*
>>> 
>>> 
> 

Re: move drill log directory to HDFS

[Andries Engelbrecht]

https://drill.apache.org/docs/persistent-configuration-storage/ 



--Andries

> On Aug 26, 2016, at 4:25 AM, Anup Tiwari  wrote:
> 
> Also please note that I have tried below in all node's drill-env.sh but its
> not working.
> 
> export DRILL_LOG_DIR="hdfs://namenode:9000/tmp/drilllogs/"
> 
> 
> Regards,
> *Anup Tiwari*
> 
> 
> On Fri, Aug 26, 2016 at 4:06 PM, Anup Tiwari 
> wrote:
> 
>> Hi All,
>> 
>> We are trying to move drill logs directory from local file system to HDFS
>> so that we can refer only one location rather than each node's log
>> directory.
>> 
>> Can anyone help me on this?
>> 
>> Regards,
>> *Anup Tiwari*
>> 
>> 

Re: move drill log directory to HDFS

[Anup Tiwari]

Also please note that I have tried below in all node's drill-env.sh but its
not working.

export DRILL_LOG_DIR="hdfs://namenode:9000/tmp/drilllogs/"


Regards,
*Anup Tiwari*


On Fri, Aug 26, 2016 at 4:06 PM, Anup Tiwari 
wrote:

> Hi All,
>
> We are trying to move drill logs directory from local file system to HDFS
> so that we can refer only one location rather than each node's log
> directory.
>
> Can anyone help me on this?
>
> Regards,
> *Anup Tiwari*
>
>

move drill log directory to HDFS

[Anup Tiwari]

Hi All,

We are trying to move drill logs directory from local file system to HDFS
so that we can refer only one location rather than each node's log
directory.

Can anyone help me on this?

Regards,
*Anup Tiwari*