Is it possible to implement transpose with PigLatin/any other MR language?

Hi, Is it possible to implement transpose operation of rows into columns and vice versa... i.e. col1 col2 col3 col4 col5 col6 col7 col8 col9 col10 col11 col12 can this be converted to col1 col4 col7 col10 col2 col5 col8 col11 col3 col6 col9 col12 Is this even possible with map reduce? If yes

Re: Is it possible to implement transpose with PigLatin/any other MR language?

Hi, Its possible in Map/Reduce. Look into the code here https://github.com/zinnia-phatak-dev/Nectar/tree/master/Nectar-regression/src/main/java/com/zinnia/nectar/regression/hadoop/primitive/mapreduce 2012/6/21 Subir S > Hi, > > Is it possible to implement transpose operation of rows into colu

Re: Fail to build Trunk...

> > Sorry for spamming! > > Philip, please see this issue < > https://issues.apache.org/jira/browse/HDFS-3549> > Ah, yes. That patch works. Thanks Prajakta; not spam at all!

Re: Error starting MRAppMaster

Okay, the below problem was fixed after commenting out the property to use CapacityScheduler from my yarn-site.xml. I found a reference to this in this JIRA . YARN now uses the default FIFO scheduler. Is the Dispatcher used in the CapacitySched

Re: Single disk failure (with HDFS-457 applied) causes data node to die

Peter, I believe that this will help you: https://ccp.cloudera.com/display/CDHDOC/CDH3+Deployment+on+a+Cluster DataNode Configuration By default, the failure of a single dfs.data.dir will cause the HDFS DataNode process to shut down, which results in the NameNode scheduling additional replicas f

Re: Is it possible to implement transpose with PigLatin/any other MR language?

While it may be fine for many cases, If I'm reading the Nectar code correctly, that transpose doesn't guarantee anything about the order of rows within each column. In other words, transposing: a - b -c d - e - f g - h - i may give you different permutations of "a - d - g" as the first row, depe

Re: Is it possible to implement transpose with PigLatin/any other MR language?

That may be true, I have not read through the code very closely, if you have multiple reduces, so you can run it with a single reduce or you can write a custom partitioner to do it. You only need to know the length of the column, and then you can divide them up appropriately, kind of like how

Yuan Jin is out of the office.

I will be out of the office starting 06/22/2012 and will not return until 06/25/2012. I am out of office, and will reply you when I am back.

Re: Error starting MRAppMaster

This has got nothing to do with the scheduler. I believe this has got to do with some compilation issue. How did you build hadoop? Also, I found that the repo at github (which is a mirror of git repo at apache) doesn't always pick all the commits immediately. You are better off checking out f

Re: Error starting MRAppMaster

Hi Vinod I am actually using the 'mvn package -Pdist -DskipTests -Dtar' command as described in the BUILDING.txt file after cloning Hadoop. Then I am using the eclipse goal with mvn to set it up for eclipse as described in < http://wiki.apache.org/hadoop/EclipseEnvironment> page. To setup YARN, I