Re: FileSystem and FileContext Janitor, at your service !

2014-03-06 Thread Jay Vyas
, Jay Vyas jayunit...@gmail.com wrote: Hi HCFS Community :) This is Jay... Some of you know me I hack on a broad range of file system and hadoop ecosystem interoperability stuff. I just wanted to introduce myself and let you folks know im going to be working to help clean up

In-Memory Reference FS implementations

2014-03-06 Thread Jay Vyas
to, and modify their particular file system to declare semantics using the matching FS as a template. -- Jay Vyas http://jayunit100.blogspot.com

Re: In-Memory Reference FS implementations

2014-03-06 Thread Jay Vyas
do you consider that native S3 FS a real reference implementation for blob stores? or just something that , by mere chance, we are able to use as a ref. impl.

Re: In-Memory Reference FS implementations

2014-03-06 Thread Jay Vyas
be nice to see this or something like it integrated with Hadoop. I fear that a lot of applications are not ready for eventual consistency, and may never be, leading to the feeling that Hadoop on S3 is buggy. Colin On Thu, Mar 6, 2014 at 10:42 AM, Jay Vyas jayunit...@gmail.com wrote: do

Re: In-Memory Reference FS implementations

2014-03-06 Thread Jay Vyas
, 2014 at 4:01 PM, Steve Loughran ste...@hortonworks.comwrote: Lets get the HADOOP-9361 stuff in (it lives alongside FileSystemContractBaseTest) and you can work off that. On 6 March 2014 18:57, Jay Vyas jayunit...@gmail.com wrote: Thanks Colin: that's a good example of why we want To unify

FileSystem and FileContext Janitor, at your service !

2014-03-05 Thread Jay Vyas
, and it will benefit us all immensly in the long run as well. -- Jay Vyas http://jayunit100.blogspot.com

[jira] [Created] (HDFS-6027) hadoop-fuse-dfs is not strictly consistent for cp actions file contents.

2014-02-27 Thread jay vyas (JIRA)
jay vyas created HDFS-6027: -- Summary: hadoop-fuse-dfs is not strictly consistent for cp actions file contents. Key: HDFS-6027 URL: https://issues.apache.org/jira/browse/HDFS-6027 Project: Hadoop HDFS

[jira] [Created] (HDFS-5909) NameNode : Can some features be implemented in a pure YARN app, to synergize broader uniformity for HCFS distributed FS metadata operations?

2014-02-07 Thread jay vyas (JIRA)
jay vyas created HDFS-5909: -- Summary: NameNode : Can some features be implemented in a pure YARN app, to synergize broader uniformity for HCFS distributed FS metadata operations? Key: HDFS-5909 URL: https

Re: HDFS read/write data throttling

2013-11-18 Thread Jay Vyas
Where is the jira for this? Sent from my iPhone On Nov 18, 2013, at 1:25 PM, Andrew Wang andrew.w...@cloudera.com wrote: Thanks for asking, here's a link: http://www.umbrant.com/papers/socc12-cake.pdf I don't think there's a recording of my talk unfortunately. I'll also copy my

Re: mapred replication

2013-08-16 Thread Jay Vyas
, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Jay Vyas http://jayunit100.blogspot.com

Re: Backwards compatibility of FileSystem interface.

2013-05-31 Thread Jay Vyas
://issues.apache.org/jira/browse/HADOOP-6886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel On Fri, May 31, 2013 at 6:09 AM, Steve Loughran ste...@hortonworks.comwrote: Jay, this is is more a common-dev question, as it goes beyond hdfs On 31 May 2013 02:07, Jay Vyas jayunit...@gmail.com

Backwards compatibility of FileSystem interface.

2013-05-30 Thread Jay Vyas
be entirely phased out. This is in reference to some issues with HBASE and the createNonRecursive implementation which varies across different hadoop versions. -- Jay Vyas http://jayunit100.blogspot.com

transfer - CreateSocketForPipeline : hardcoded length of 2?

2013-04-23 Thread Jay Vyas
node, rather than 1 node one? Maybe I'm misunderstanding something about the way the pipeline works so thanks for helping and apologies if this question is a little silly. -- Jay Vyas http://jayunit100.blogspot.com

Convention question for using DFSConfigKey constants : are zeros magic?

2013-04-19 Thread Jay Vyas
hadoop dependent configuration code. We'd like to follow hadoopy idioms if possible.. -- Jay Vyas http://jayunit100.blogspot.com

Re: Convention question for using DFSConfigKey constants : are zeros magic?

2013-04-19 Thread Jay Vyas
your answer as No...? Thats fine.. but just clarifying :) ? On Fri, Apr 19, 2013 at 2:23 PM, Aaron T. Myers a...@cloudera.com wrote: Hi Jay, On Sat, Apr 20, 2013 at 1:10 AM, Jay Vyas jayunit...@gmail.com wrote: I recently looked into the HDFS source tree to determine idioms with respect

Re: testHDFSConf.xml

2013-04-11 Thread Jay Vyas
. May be it something you would benefit from. Cos On Wed, Apr 10, 2013 at 10:43AM, Colin McCabe wrote: On Wed, Apr 10, 2013 at 10:16 AM, Jay Vyas jayunit...@gmail.com wrote: Hello HDFS brethren ! I've noticed that the testHDFSConf.xml has alot of references to supergroup

testHDFSConf.xml

2013-04-10 Thread Jay Vyas
for FileSystem implementations Depending on feedback im certainly willing to submit and put in a first pass at a more modular version of this file. Its in many ways a very generalizable component of the hdfs trunk. Thanks! -- Jay Vyas http://jayunit100.blogspot.com