Hello
We have 2 tables (tab1, tab2) exposed using hive. The data is in different
hdfs folders. We are trying to join these 2 tables on certain single column
using sparkR join. But inspite of join columns having same values, it
returns zero rows.
But when I run the same join sql in hive, from
Resolved this issue after reinstalling r, rstudio. Had issues with earlier
installation.
On Jan 22, 2016 6:48 PM, "Sandeep Khurana" <sand...@infoworks.io> wrote:
> This problem is fixed by restarting R from R studio. Now see
>
> 16/01/22 08:08:38 INFO HiveMetaStore:
Hello
I installed spark in a folder. I start bin/sparkR on console. Then I
execute below command and all work fine. I can see the data as well.
hivecontext <<- sparkRHive.init(sc) ;
df <- loadDF(hivecontext, "/someHdfsPath", "orc")
showDF(df)
But when I give same to rstudio, it throws the
with Hive support
in rstudio while running same code and hive-site.xml is present in the .
It works in sparkR shell.
Any ideas?
On Fri, Jan 22, 2016 at 4:35 PM, Sandeep Khurana <sand...@infoworks.io>
wrote:
> Hello
>
> I installed spark in a folder. I start bin/sparkR on console. T
uot;Yanbo Liang" <yblia...@gmail.com>
> wrote:
>
> You should ensure your sqlContext is HiveContext.
>
> sc <- sparkR.init()
>
> sqlContext <- sparkRHive.init(sc)
>
>
> 2016-01-06 20:35 GMT+08:00 Sandeep Khurana <sand...@infoworks.io>:
>
> Fel
"loadDF", sqlContext,
source, options)
3
read.df(sqlContext, path, source, schema, ...)
2
loadDF(hivecontext, filepath, "orc")
On Tue, Jan 12, 2016 at 2:41 PM, Sandeep Khurana <sand...@infoworks.io>
wrote:
> Running this gave
>
> 16/01/12 04:06:54 INFO BlockMana
tmail.com>
> wrote:
>
> Yes, as Yanbo suggested, it looks like there is something wrong with the
> sqlContext.
>
> Could you forward us your code please?
>
>
>
>
>
> On Wed, Jan 6, 2016 at 5:52 AM -0800, "Yanbo Liang" <yblia...@gmail.com>
&g
.
On Tue, Jan 12, 2016 at 3:05 PM, Sandeep Khurana <sand...@infoworks.io>
wrote:
> Complete stacktrace is. Can it be something wih java versions?
>
>
> stop("invalid jobj ", value$id)
> 8
> writeJobj(con, object)
> 7
> writeObject(con, a)
> 6
> writeA
1/", "orc")
>
>
> Is there a reason you want to call stop? If you do, you would need to call
> the line hivecontext <- sparkRHive.init(sc) again.
>
>
> _
> From: Sandeep Khurana <sand...@infoworks.io>
> Sent: Tuesday, Jan
there is any error
> message there.
>
> _
> From: Prem Sure <premsure...@gmail.com>
> Sent: Tuesday, January 5, 2016 8:12 AM
> Subject: Re: sparkR ORC support.
> To: Sandeep Khurana <sand...@infoworks.io>
> Cc: spark users <user@spark.
Also, do I need to setup hive in spark as per the link
http://stackoverflow.com/questions/26360725/accesing-hive-tables-in-spark ?
We might need to copy hdfs-site.xml file to spark conf directory ?
On Tue, Jan 5, 2016 at 8:28 PM, Sandeep Khurana <sand...@infoworks.io>
wrote:
> Deepak
>
> results <- sql(hivecontext, "FROM test SELECT id","")
>
> Thanks
> Deepak
>
>
> On Tue, Jan 5, 2016 at 5:49 PM, Sandeep Khurana <sand...@infoworks.io>
> wrote:
>
>> Thanks Deepak.
>>
>> I tried this as well.
Hello
I need to read an ORC files in hdfs in R using spark. I am not able to find
a package to do that.
Can anyone help with documentation or example for this purpose?
--
Architect
Infoworks.io
http://Infoworks.io
e if ORC can be read directly in R.
> But there can be a workaround .First create hive table on top of ORC files
> and then access hive table in R.
>
> Thanks
> Deepak
>
> On Tue, Jan 5, 2016 at 4:57 PM, Sandeep Khurana <sand...@infoworks.io>
> wrote:
>
>>
14 matches
Mail list logo
