Re: Experience using binary packages on various Hadoop distros

2015-04-06 Thread Dean Chen
This would be great for those of us running on HDP. At eBay we recently ran in to few problems using the generic Hadoop lib. Two off of the top of my head: * Needed to included our custom Hadoop client due to custom keberos integration * Minor difference in HDFS protocol causing the following

Re: Experience using binary packages on various Hadoop distros

2015-03-24 Thread Matei Zaharia
Just a note, one challenge with the BYOH version might be that users who download that can't run in local mode without also having Hadoop. But if we describe it correctly then hopefully it's okay. Matei On Mar 24, 2015, at 3:05 PM, Patrick Wendell pwend...@gmail.com wrote: Hey All, For

Experience using binary packages on various Hadoop distros

2015-03-24 Thread Patrick Wendell
Hey All, For a while we've published binary packages with different Hadoop client's pre-bundled. We currently have three interfaces to a Hadoop cluster (a) the HDFS client (b) the YARN client (c) the Hive client. Because (a) and (b) are supposed to be backwards compatible interfaces. My working