Re: How to print values in console while running MapReduce application

2017-10-04 Thread Tanvir Rahman
Hello Demian, Thanks for the answer. 1. I am using Java for writing the MapReduce Application. Can you tell me how to do it in JAVA? 2. In the mapper or reducer function, which command did you use to write the output? Is it going to write it in Log folder? I have multiple nodes and

Re: How to print values in console while running MapReduce application

2017-10-04 Thread Demian Kurejwowski
i did the same tutorial, i think they only way  is doing it  outside hadoop.in the command line:cat folder/* | python mapper.py | sort | python reducer El Miércoles, 4 de octubre, 2017 16:20:31, Tanvir Rahman escribió: Hello,I have a small cluster and I am running MapReduce WordCount a

Re: How to print values in console while running MapReduce application

2017-10-04 Thread Sultan Alamro
Hi, The easiest way is to open a new window and display the log file as follow tail -f /path/to/log/file.log Best, Sultan > On Oct 4, 2017, at 5:20 PM, Tanvir Rahman wrote: > > Hello, > I have a small cluster and I am running MapReduce WordCount application in > it. > I want to print some va

How to print values in console while running MapReduce application

2017-10-04 Thread Tanvir Rahman
Hello, I have a small cluster and I am running MapReduce WordCount application in it. I want to print some variable values in the console(where you can see the map and reduce job progress and other job information) for debugging purpose while running the MapReduce application. What is the easiest

[ANNOUNCE] Apache Hadoop 3.0.0-beta1 has been released

2017-10-04 Thread Andrew Wang
Hi all, I'm pleased to announce the release of Apache Hadoop 3.0.0-beta1. This is our first beta release in the 3.0.0 release line, and is planned to be the last release before 3.0.0 GA. Beta releases are API stable but come with no guarantee of quality, and are not intended for production use. 3

different behaviour linux/Unix vs windows when load spark context in scala method called from R function using rscala package

2017-10-04 Thread Simone Pallotta
In my R code, I am using rscala package to bridge to a scala method. in scala method I have initialized a spark context to be used later. R code: s <- scala(classpath = "", heap.maximum = "4g") assign("WrappeR",s$.it.wrapper.r.Wrapper) WrappeR$init() where init is a scala function and Wrapp

different behaviour linux/Unix vs windows when load spark context in scala method called from R function using rscala package

2017-10-04 Thread Simone Pallotta
In my R code, I am using rscala package to bridge to a scala method. in scala method I have initialized a spark context to be used later. R code: s <- scala(classpath = "", heap.maximum = "4g") assign("WrappeR",s$.it.wrapper.r.Wrapper) WrappeR$init() where init is a scala function and Wrapp