Re: Inode for STS

[ayan guha]

Hi

Thanks for this. However, I am interested in regular deletion of temp while
server is up. Additionally, the link says it is not of use for multi-user
environment. Any other idea? is there any variation of cleaner.ttl?

On Mon, Jul 18, 2016 at 8:00 PM, Chanh Le  wrote:

> Hi Ayan,
> I seem like you mention this
> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.start.cleanup.scratchdir
> Default it was set false by default.
>
>
>
>
>
> On Jul 13, 2016, at 5:01 PM, ayan guha  wrote:
>
> Thanks Christophe. Any comment from Spark dev community member would
> really helpful on the Jira.
>
> What I saw today is shutting down the thrift server process lead to a
> clean up. Also, we started removing any empty folders from /tmp. Is there
> any other or better method?
>
> On Wed, Jul 13, 2016 at 5:25 PM, Christophe Préaud <
> christophe.pre...@kelkoo.com> wrote:
>
>> Hi Ayan,
>>
>> I have opened a JIRA about this issues, but there are no answer so far:
>> SPARK-15401 
>>
>> Regards,
>> Christophe.
>>
>>
>> On 13/07/16 05:54, ayan guha wrote:
>>
>> Hi
>>
>> We are running Spark Thrift Server as a long running application.
>> However,  it looks like it is filling up /tmp/hive folder with lots of
>> small files and directories with no file in them, blowing out inode limit
>> and preventing any connection with "No Space Left in Device" issue.
>>
>> What is the best way to clean up those small files periodically?
>>
>> --
>> Best Regards,
>> Ayan Guha
>>
>>
>>
>> --
>> Kelkoo SAS
>> Société par Actions Simplifiée
>> Au capital de € 4.168.964,30
>> Siège social : 158 Ter Rue du Temple 75003 Paris
>> 425 093 069 RCS Paris
>>
>> Ce message et les pièces jointes sont confidentiels et établis à
>> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
>> destinataire de ce message, merci de le détruire et d'en avertir
>> l'expéditeur.
>>
>
>
>
> --
> Best Regards,
> Ayan Guha
>
>
>


-- 
Best Regards,
Ayan Guha

Re: Inode for STS

[Chanh Le]

Hi Ayan,
I seem like you mention this 
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.start.cleanup.scratchdir
 

Default it was set false by default.





> On Jul 13, 2016, at 5:01 PM, ayan guha  wrote:
> 
> Thanks Christophe. Any comment from Spark dev community member would really 
> helpful on the Jira.
> 
> What I saw today is shutting down the thrift server process lead to a clean 
> up. Also, we started removing any empty folders from /tmp. Is there any other 
> or better method? 
> 
> On Wed, Jul 13, 2016 at 5:25 PM, Christophe Préaud 
> > wrote:
> Hi Ayan,
> 
> I have opened a JIRA about this issues, but there are no answer so far: 
> SPARK-15401 
> 
> Regards,
> Christophe.
> 
> 
> On 13/07/16 05:54, ayan guha wrote:
>> Hi
>> 
>> We are running Spark Thrift Server as a long running application. However,  
>> it looks like it is filling up /tmp/hive folder with lots of small files and 
>> directories with no file in them, blowing out inode limit and preventing any 
>> connection with "No Space Left in Device" issue. 
>> 
>> What is the best way to clean up those small files periodically? 
>> 
>> -- 
>> Best Regards,
>> Ayan Guha
> 
> 
> Kelkoo SAS
> Société par Actions Simplifiée
> Au capital de € 4.168.964,30
> Siège social : 158 Ter Rue du Temple 75003 Paris
> 425 093 069 RCS Paris
> 
> Ce message et les pièces jointes sont confidentiels et établis à l'attention 
> exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
> message, merci de le détruire et d'en avertir l'expéditeur.
> 
> 
> 
> -- 
> Best Regards,
> Ayan Guha

Re: Inode for STS

[ayan guha]

Thanks Christophe. Any comment from Spark dev community member would really
helpful on the Jira.

What I saw today is shutting down the thrift server process lead to a clean
up. Also, we started removing any empty folders from /tmp. Is there any
other or better method?

On Wed, Jul 13, 2016 at 5:25 PM, Christophe Préaud <
christophe.pre...@kelkoo.com> wrote:

> Hi Ayan,
>
> I have opened a JIRA about this issues, but there are no answer so far:
> SPARK-15401 
>
> Regards,
> Christophe.
>
>
> On 13/07/16 05:54, ayan guha wrote:
>
> Hi
>
> We are running Spark Thrift Server as a long running application. However,
>  it looks like it is filling up /tmp/hive folder with lots of small files
> and directories with no file in them, blowing out inode limit and
> preventing any connection with "No Space Left in Device" issue.
>
> What is the best way to clean up those small files periodically?
>
> --
> Best Regards,
> Ayan Guha
>
>
>
> --
> Kelkoo SAS
> Société par Actions Simplifiée
> Au capital de € 4.168.964,30
> Siège social : 158 Ter Rue du Temple 75003 Paris
> 425 093 069 RCS Paris
>
> Ce message et les pièces jointes sont confidentiels et établis à
> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
> destinataire de ce message, merci de le détruire et d'en avertir
> l'expéditeur.
>



-- 
Best Regards,
Ayan Guha

Re: Inode for STS

[Christophe Préaud]

Hi Ayan,

I have opened a JIRA about this issues, but there are no answer so far: 
SPARK-15401

Regards,
Christophe.

On 13/07/16 05:54, ayan guha wrote:
Hi

We are running Spark Thrift Server as a long running application. However,  it 
looks like it is filling up /tmp/hive folder with lots of small files and 
directories with no file in them, blowing out inode limit and preventing any 
connection with "No Space Left in Device" issue.

What is the best way to clean up those small files periodically?

--
Best Regards,
Ayan Guha



Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention 
exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce 
message, merci de le détruire et d'en avertir l'expéditeur.