Hi Sarath,
I will try to reproduce the problem.
Thanks,
Yin
On Wed, Jul 23, 2014 at 11:32 PM, Sarath Chandra
sarathchandra.jos...@algofusiontech.com wrote:
Hi Michael,
Sorry for the delayed response.
I'm using Spark 1.0.1 (pre-built version for hadoop 1). I'm running spark
programs on
Hi Sarath,
Have you tried the current branch 1.0? If not, can you give it a try and
see if the problem can be resolved?
Thanks,
Yin
On Thu, Jul 24, 2014 at 11:17 AM, Yin Huai yh...@databricks.com wrote:
Hi Sarath,
I will try to reproduce the problem.
Thanks,
Yin
On Wed, Jul 23,
Added below 2 lines just before the sql query line -
*...*
*file1_schema.count;*
*file2_schema.count;*
*...*
and it started working. But I couldn't get the reason.
Can someone please explain me? What was happening earlier and what is
happening with addition of these 2 lines?
~Sarath
On Thu,
What version are you running? Could you provide a jstack of the driver and
executor when it is hanging?
On Thu, Jul 17, 2014 at 10:55 AM, Sarath Chandra
sarathchandra.jos...@algofusiontech.com wrote:
Added below 2 lines just before the sql query line -
*...*
*file1_schema.count;*
Hi All,
I'm trying to do a simple record matching between 2 files and wrote
following code -
*import org.apache.spark.sql.SQLContext;*
*import org.apache.spark.rdd.RDD*
*object SqlTest {*
* case class Test(fld1:String, fld2:String, fld3:String, fld4:String,
fld4:String, fld5:Double,
Check your executor logs for the output or if your data is not big collect it
in the driver and print it.
On Jul 16, 2014, at 9:21 AM, Sarath Chandra
sarathchandra.jos...@algofusiontech.com wrote:
Hi All,
I'm trying to do a simple record matching between 2 files and wrote following
Hi Soumya,
Data is very small, 500+ lines in each file.
Removed last 2 lines and placed this at the end
matched.collect().foreach(println);. Still no luck. It's been more than
5min, the execution is still running.
Checked logs, nothing in stdout. In stderr I don't see anything going
wrong, all
When you submit your job, it should appear on the Spark UI. Same with the
REPL. Make sure you job is submitted to the cluster properly.
On Wed, Jul 16, 2014 at 10:08 AM, Sarath Chandra
sarathchandra.jos...@algofusiontech.com wrote:
Hi Soumya,
Data is very small, 500+ lines in each file.
Yes it is appearing on the Spark UI, and remains there with state as
RUNNING till I press Ctrl+C in the terminal to kill the execution.
Barring the statements to create the spark context, if I copy paste the
lines of my code in spark shell, runs perfectly giving the desired output.
~Sarath
On
Can you try submitting a very simple job to the cluster.
On Jul 16, 2014, at 10:25 AM, Sarath Chandra
sarathchandra.jos...@algofusiontech.com wrote:
Yes it is appearing on the Spark UI, and remains there with state as
RUNNING till I press Ctrl+C in the terminal to kill the execution.
Yes Soumya, I did it.
First I tried with the example available in the documentation (example
using people table and finding teenagers). After successfully running it, I
moved on to this one which is starting point to a bigger requirement for
which I'm evaluating Spark SQL.
On Wed, Jul 16, 2014
What if you just run something like:
*sc.textFile(hdfs://localhost:54310/user/hduser/file1.csv).count()*
On Wed, Jul 16, 2014 at 10:37 AM, Sarath Chandra
sarathchandra.jos...@algofusiontech.com wrote:
Yes Soumya, I did it.
First I tried with the example available in the documentation
Hi Michael,
Tried it. It's correctly printing the line counts of both the files. Here's
what I tried -
*Code:*
*package test*
*object Test4 {*
* case class Test(fld1: String, *
* fld2: String, *
* fld3: String, *
* fld4: String, *
* fld5: String, *
* fld6: Double, *
* fld7:
13 matches
Mail list logo
