Re: SSD support in HDFS

[Alexey Babutin]

You can use  BTIER  in linux.(I don't know its
stable or not).With FreeBSD  you can use ZFS +L2ARC on SSD.


On Thu, Jun 13, 2013 at 6:35 AM, Michael Segel wrote:

> I could have sworn there was a thread on this already. (Maybe the HBase
> list?)
>
> Andrew P. kinda nailed it when he talked about the fact that you had to
> write the replication(s).
>
> If you wanted improved performance, why not look at the hybrid drives that
> have a small SSD buffer and a spinning disk?
>
> I don't know but it may be what you're looking for.
>
> HTH
>
> -Mike
>
> On Jun 12, 2013, at 5:18 AM, Lucas Stanley  wrote:
>
> Thanks Chris and Phil.
>
>
> On Tue, Jun 11, 2013 at 1:31 PM, Chris Nauroth 
> wrote:
>
>> Hi Lucas,
>>
>> HDFS does not have this capability right now, but there has been some
>> preliminary discussion around adding features to support it.  You might
>> want to follow jira issues HDFS-2832 and HDFS-4672 if you'd like to receive
>> notifications about the discussion.
>>
>> https://issues.apache.org/jira/browse/HDFS-2832
>> https://issues.apache.org/jira/browse/HDFS-4672
>>
>> Chris Nauroth
>> Hortonworks
>> http://hortonworks.com/
>>
>>
>>
>> On Mon, Jun 10, 2013 at 6:57 PM, Lucas Stanley wrote:
>>
>>> Hi,
>>>
>>> Is it possible to tell Apache HDFS to store some files on SSD and the
>>> rest of the files on spinning disks?
>>>
>>> So if each on my nodes has 1 SSD and 5 spinning disks, can I configure a
>>> directory in HDFS to put all files in that dir on the SSD?
>>>
>>> I think Intel's Hadoop distribution is working on some SSD support right?
>>>
>>
>>
>
>

Re: SSD support in HDFS

[Michael Segel]

I could have sworn there was a thread on this already. (Maybe the HBase list?) 

Andrew P. kinda nailed it when he talked about the fact that you had to write 
the replication(s). 

If you wanted improved performance, why not look at the hybrid drives that have 
a small SSD buffer and a spinning disk? 

I don't know but it may be what you're looking for. 

HTH

-Mike

On Jun 12, 2013, at 5:18 AM, Lucas Stanley  wrote:

> Thanks Chris and Phil.
> 
> 
> On Tue, Jun 11, 2013 at 1:31 PM, Chris Nauroth  
> wrote:
> Hi Lucas,
> 
> HDFS does not have this capability right now, but there has been some 
> preliminary discussion around adding features to support it.  You might want 
> to follow jira issues HDFS-2832 and HDFS-4672 if you'd like to receive 
> notifications about the discussion.
> 
> https://issues.apache.org/jira/browse/HDFS-2832
> https://issues.apache.org/jira/browse/HDFS-4672
> 
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
> 
> 
> 
> On Mon, Jun 10, 2013 at 6:57 PM, Lucas Stanley  wrote:
> Hi, 
> 
> Is it possible to tell Apache HDFS to store some files on SSD and the rest of 
> the files on spinning disks?
> 
> So if each on my nodes has 1 SSD and 5 spinning disks, can I configure a 
> directory in HDFS to put all files in that dir on the SSD?
> 
> I think Intel's Hadoop distribution is working on some SSD support right?
> 
> 

Re: SSD support in HDFS

[Lucas Stanley]

Thanks Chris and Phil.


On Tue, Jun 11, 2013 at 1:31 PM, Chris Nauroth wrote:

> Hi Lucas,
>
> HDFS does not have this capability right now, but there has been some
> preliminary discussion around adding features to support it.  You might
> want to follow jira issues HDFS-2832 and HDFS-4672 if you'd like to receive
> notifications about the discussion.
>
> https://issues.apache.org/jira/browse/HDFS-2832
> https://issues.apache.org/jira/browse/HDFS-4672
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Mon, Jun 10, 2013 at 6:57 PM, Lucas Stanley wrote:
>
>> Hi,
>>
>> Is it possible to tell Apache HDFS to store some files on SSD and the
>> rest of the files on spinning disks?
>>
>> So if each on my nodes has 1 SSD and 5 spinning disks, can I configure a
>> directory in HDFS to put all files in that dir on the SSD?
>>
>> I think Intel's Hadoop distribution is working on some SSD support right?
>>
>
>

Re: SSD support in HDFS

[Chris Nauroth]

Hi Lucas,

HDFS does not have this capability right now, but there has been some
preliminary discussion around adding features to support it.  You might
want to follow jira issues HDFS-2832 and HDFS-4672 if you'd like to receive
notifications about the discussion.

https://issues.apache.org/jira/browse/HDFS-2832
https://issues.apache.org/jira/browse/HDFS-4672

Chris Nauroth
Hortonworks
http://hortonworks.com/



On Mon, Jun 10, 2013 at 6:57 PM, Lucas Stanley  wrote:

> Hi,
>
> Is it possible to tell Apache HDFS to store some files on SSD and the rest
> of the files on spinning disks?
>
> So if each on my nodes has 1 SSD and 5 spinning disks, can I configure a
> directory in HDFS to put all files in that dir on the SSD?
>
> I think Intel's Hadoop distribution is working on some SSD support right?
>

SSD support in HDFS

[Lucas Stanley]

Hi,

Is it possible to tell Apache HDFS to store some files on SSD and the rest
of the files on spinning disks?

So if each on my nodes has 1 SSD and 5 spinning disks, can I configure a
directory in HDFS to put all files in that dir on the SSD?

I think Intel's Hadoop distribution is working on some SSD support right?