RE: sparkR issues ?

2016-03-18 Thread Sun, Rui 
Sorry. I am wrong. The issue is not related to as.data.frame(). It seems to be 
related to DataFrame naming conflict between s4vectors and SparkR.
Refer to https://issues.apache.org/jira/browse/SPARK-12148


From: Sun, Rui [mailto:rui@intel.com]
Sent: Wednesday, March 16, 2016 9:33 AM
To: Alex Kozlov <ale...@gmail.com>; roni <roni.epi...@gmail.com>
Cc: user@spark.apache.org
Subject: RE: sparkR issues ?

I have submitted https://issues.apache.org/jira/browse/SPARK-13905 and a PR for 
it.

From: Alex Kozlov [mailto:ale...@gmail.com]
Sent: Wednesday, March 16, 2016 12:52 AM
To: roni <roni.epi...@gmail.com<mailto:roni.epi...@gmail.com>>
Cc: Sun, Rui <rui@intel.com<mailto:rui@intel.com>>; 
user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: sparkR issues ?

Hi Roni, you can probably rename the as.data.frame in 
$SPARK_HOME/R/pkg/R/DataFrame.R and re-install SparkR by running install-dev.sh

On Tue, Mar 15, 2016 at 8:46 AM, roni 
<roni.epi...@gmail.com<mailto:roni.epi...@gmail.com>> wrote:
Hi ,
 Is there a work around for this?
 Do i need to file a bug for this?
Thanks
-R

On Tue, Mar 15, 2016 at 12:28 AM, Sun, Rui 
<rui@intel.com<mailto:rui@intel.com>> wrote:
It seems as.data.frame() defined in SparkR convers the versions in R base 
package.
We can try to see if we can change the implementation of as.data.frame() in 
SparkR to avoid such covering.

From: Alex Kozlov [mailto:ale...@gmail.com<mailto:ale...@gmail.com>]
Sent: Tuesday, March 15, 2016 2:59 PM
To: roni <roni.epi...@gmail.com<mailto:roni.epi...@gmail.com>>
Cc: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: sparkR issues ?

This seems to be a very unfortunate name collision.  SparkR defines it's own 
DataFrame class which shadows what seems to be your own definition.

Is DataFrame something you define?  Can you rename it?

On Mon, Mar 14, 2016 at 10:44 PM, roni 
<roni.epi...@gmail.com<mailto:roni.epi...@gmail.com>> wrote:
Hi,
 I am working with bioinformatics and trying to convert some scripts to sparkR 
to fit into other spark jobs.

I tries a simple example from a bioinf lib and as soon as I start sparkR 
environment it does not work.

code as follows -
countData <- matrix(1:100,ncol=4)
condition <- factor(c("A","A","B","B"))
dds <- DESeqDataSetFromMatrix(countData, DataFrame(condition), ~ condition)

Works if i dont initialize the sparkR environment.
 if I do library(SparkR) and sqlContext <- sparkRSQL.init(sc)  it gives 
following error

> dds <- DESeqDataSetFromMatrix(countData, as.data.frame(condition), ~ 
> condition)
Error in DataFrame(colData, row.names = rownames(colData)) :
  cannot coerce class "data.frame" to a DataFrame

I am really stumped. I am not using any spark function , so i would expect it 
to work as a simple R code.
why it does not work?

Appreciate  the help
-R




--
Alex Kozlov
(408) 507-4987<tel:%28408%29%20507-4987>
(650) 887-2135<tel:%28650%29%20887-2135> efax
ale...@gmail.com<mailto:ale...@gmail.com>




--
Alex Kozlov
(408) 507-4987
(650) 887-2135 efax
ale...@gmail.com<mailto:ale...@gmail.com>

RE: sparkR issues ?

2016-03-15 Thread Sun, Rui 
I have submitted https://issues.apache.org/jira/browse/SPARK-13905 and a PR for 
it.

From: Alex Kozlov [mailto:ale...@gmail.com]
Sent: Wednesday, March 16, 2016 12:52 AM
To: roni <roni.epi...@gmail.com>
Cc: Sun, Rui <rui@intel.com>; user@spark.apache.org
Subject: Re: sparkR issues ?

Hi Roni, you can probably rename the as.data.frame in 
$SPARK_HOME/R/pkg/R/DataFrame.R and re-install SparkR by running install-dev.sh

On Tue, Mar 15, 2016 at 8:46 AM, roni 
<roni.epi...@gmail.com<mailto:roni.epi...@gmail.com>> wrote:
Hi ,
 Is there a work around for this?
 Do i need to file a bug for this?
Thanks
-R

On Tue, Mar 15, 2016 at 12:28 AM, Sun, Rui 
<rui@intel.com<mailto:rui@intel.com>> wrote:
It seems as.data.frame() defined in SparkR convers the versions in R base 
package.
We can try to see if we can change the implementation of as.data.frame() in 
SparkR to avoid such covering.

From: Alex Kozlov [mailto:ale...@gmail.com<mailto:ale...@gmail.com>]
Sent: Tuesday, March 15, 2016 2:59 PM
To: roni <roni.epi...@gmail.com<mailto:roni.epi...@gmail.com>>
Cc: user@spark.apache.org<mailto:user@spark.apache.org>
Subject: Re: sparkR issues ?

This seems to be a very unfortunate name collision.  SparkR defines it's own 
DataFrame class which shadows what seems to be your own definition.

Is DataFrame something you define?  Can you rename it?

On Mon, Mar 14, 2016 at 10:44 PM, roni 
<roni.epi...@gmail.com<mailto:roni.epi...@gmail.com>> wrote:
Hi,
 I am working with bioinformatics and trying to convert some scripts to sparkR 
to fit into other spark jobs.

I tries a simple example from a bioinf lib and as soon as I start sparkR 
environment it does not work.

code as follows -
countData <- matrix(1:100,ncol=4)
condition <- factor(c("A","A","B","B"))
dds <- DESeqDataSetFromMatrix(countData, DataFrame(condition), ~ condition)

Works if i dont initialize the sparkR environment.
 if I do library(SparkR) and sqlContext <- sparkRSQL.init(sc)  it gives 
following error

> dds <- DESeqDataSetFromMatrix(countData, as.data.frame(condition), ~ 
> condition)
Error in DataFrame(colData, row.names = rownames(colData)) :
  cannot coerce class "data.frame" to a DataFrame

I am really stumped. I am not using any spark function , so i would expect it 
to work as a simple R code.
why it does not work?

Appreciate  the help
-R




--
Alex Kozlov
(408) 507-4987<tel:%28408%29%20507-4987>
(650) 887-2135<tel:%28650%29%20887-2135> efax
ale...@gmail.com<mailto:ale...@gmail.com>




--
Alex Kozlov
(408) 507-4987
(650) 887-2135 efax
ale...@gmail.com<mailto:ale...@gmail.com>

Re: sparkR issues ?

2016-03-15 Thread Alex Kozlov 
Hi Roni, you can probably rename the as.data.frame in
$SPARK_HOME/R/pkg/R/DataFrame.R and re-install SparkR by running
install-dev.sh

On Tue, Mar 15, 2016 at 8:46 AM, roni <roni.epi...@gmail.com> wrote:

> Hi ,
>  Is there a work around for this?
>  Do i need to file a bug for this?
> Thanks
> -R
>
> On Tue, Mar 15, 2016 at 12:28 AM, Sun, Rui <rui@intel.com> wrote:
>
>> It seems as.data.frame() defined in SparkR convers the versions in R base
>> package.
>>
>> We can try to see if we can change the implementation of as.data.frame()
>> in SparkR to avoid such covering.
>>
>>
>>
>> *From:* Alex Kozlov [mailto:ale...@gmail.com]
>> *Sent:* Tuesday, March 15, 2016 2:59 PM
>> *To:* roni <roni.epi...@gmail.com>
>> *Cc:* user@spark.apache.org
>> *Subject:* Re: sparkR issues ?
>>
>>
>>
>> This seems to be a very unfortunate name collision.  SparkR defines it's
>> own DataFrame class which shadows what seems to be your own definition.
>>
>>
>>
>> Is DataFrame something you define?  Can you rename it?
>>
>>
>>
>> On Mon, Mar 14, 2016 at 10:44 PM, roni <roni.epi...@gmail.com> wrote:
>>
>> Hi,
>>
>>  I am working with bioinformatics and trying to convert some scripts to
>> sparkR to fit into other spark jobs.
>>
>>
>>
>> I tries a simple example from a bioinf lib and as soon as I start sparkR
>> environment it does not work.
>>
>>
>>
>> code as follows -
>>
>> countData <- matrix(1:100,ncol=4)
>>
>> condition <- factor(c("A","A","B","B"))
>>
>> dds <- DESeqDataSetFromMatrix(countData, DataFrame(condition), ~
>> condition)
>>
>>
>>
>> Works if i dont initialize the sparkR environment.
>>
>>  if I do library(SparkR) and sqlContext <- sparkRSQL.init(sc)  it gives
>> following error
>>
>>
>>
>> > dds <- DESeqDataSetFromMatrix(countData, as.data.frame(condition), ~
>> condition)
>>
>> Error in DataFrame(colData, row.names = rownames(colData)) :
>>
>>   cannot coerce class "data.frame" to a DataFrame
>>
>>
>>
>> I am really stumped. I am not using any spark function , so i would
>> expect it to work as a simple R code.
>>
>> why it does not work?
>>
>>
>>
>> Appreciate  the help
>>
>> -R
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> Alex Kozlov
>> (408) 507-4987
>> (650) 887-2135 efax
>> ale...@gmail.com
>>
>
>


-- 
Alex Kozlov
(408) 507-4987
(650) 887-2135 efax
ale...@gmail.com

Re: sparkR issues ?

2016-03-15 Thread roni 
Hi ,
 Is there a work around for this?
 Do i need to file a bug for this?
Thanks
-R

On Tue, Mar 15, 2016 at 12:28 AM, Sun, Rui <rui@intel.com> wrote:

> It seems as.data.frame() defined in SparkR convers the versions in R base
> package.
>
> We can try to see if we can change the implementation of as.data.frame()
> in SparkR to avoid such covering.
>
>
>
> *From:* Alex Kozlov [mailto:ale...@gmail.com]
> *Sent:* Tuesday, March 15, 2016 2:59 PM
> *To:* roni <roni.epi...@gmail.com>
> *Cc:* user@spark.apache.org
> *Subject:* Re: sparkR issues ?
>
>
>
> This seems to be a very unfortunate name collision.  SparkR defines it's
> own DataFrame class which shadows what seems to be your own definition.
>
>
>
> Is DataFrame something you define?  Can you rename it?
>
>
>
> On Mon, Mar 14, 2016 at 10:44 PM, roni <roni.epi...@gmail.com> wrote:
>
> Hi,
>
>  I am working with bioinformatics and trying to convert some scripts to
> sparkR to fit into other spark jobs.
>
>
>
> I tries a simple example from a bioinf lib and as soon as I start sparkR
> environment it does not work.
>
>
>
> code as follows -
>
> countData <- matrix(1:100,ncol=4)
>
> condition <- factor(c("A","A","B","B"))
>
> dds <- DESeqDataSetFromMatrix(countData, DataFrame(condition), ~ condition)
>
>
>
> Works if i dont initialize the sparkR environment.
>
>  if I do library(SparkR) and sqlContext <- sparkRSQL.init(sc)  it gives
> following error
>
>
>
> > dds <- DESeqDataSetFromMatrix(countData, as.data.frame(condition), ~
> condition)
>
> Error in DataFrame(colData, row.names = rownames(colData)) :
>
>   cannot coerce class "data.frame" to a DataFrame
>
>
>
> I am really stumped. I am not using any spark function , so i would expect
> it to work as a simple R code.
>
> why it does not work?
>
>
>
> Appreciate  the help
>
> -R
>
>
>
>
>
>
>
> --
>
> Alex Kozlov
> (408) 507-4987
> (650) 887-2135 efax
> ale...@gmail.com
>

Re: sparkR issues ?

2016-03-15 Thread roni 
Alex,
 No I have not defined he "dataframe" its the spark default Dataframe. That
line is just casting Factor as datarame to send to the function.
Thanks
-R

On Mon, Mar 14, 2016 at 11:58 PM, Alex Kozlov  wrote:

> This seems to be a very unfortunate name collision.  SparkR defines it's
> own DataFrame class which shadows what seems to be your own definition.
>
> Is DataFrame something you define?  Can you rename it?
>
> On Mon, Mar 14, 2016 at 10:44 PM, roni  wrote:
>
>> Hi,
>>  I am working with bioinformatics and trying to convert some scripts to
>> sparkR to fit into other spark jobs.
>>
>> I tries a simple example from a bioinf lib and as soon as I start sparkR
>> environment it does not work.
>>
>> code as follows -
>> countData <- matrix(1:100,ncol=4)
>> condition <- factor(c("A","A","B","B"))
>> dds <- DESeqDataSetFromMatrix(countData, DataFrame(condition), ~
>> condition)
>>
>> Works if i dont initialize the sparkR environment.
>>  if I do library(SparkR) and sqlContext <- sparkRSQL.init(sc)  it gives
>> following error
>>
>> > dds <- DESeqDataSetFromMatrix(countData, as.data.frame(condition), ~
>> condition)
>> Error in DataFrame(colData, row.names = rownames(colData)) :
>>   cannot coerce class "data.frame" to a DataFrame
>>
>> I am really stumped. I am not using any spark function , so i would
>> expect it to work as a simple R code.
>> why it does not work?
>>
>> Appreciate  the help
>> -R
>>
>>
>
>
> --
> Alex Kozlov
> (408) 507-4987
> (650) 887-2135 efax
> ale...@gmail.com
>

RE: sparkR issues ?

2016-03-15 Thread Sun, Rui 
It seems as.data.frame() defined in SparkR convers the versions in R base 
package.
We can try to see if we can change the implementation of as.data.frame() in 
SparkR to avoid such covering.

From: Alex Kozlov [mailto:ale...@gmail.com]
Sent: Tuesday, March 15, 2016 2:59 PM
To: roni <roni.epi...@gmail.com>
Cc: user@spark.apache.org
Subject: Re: sparkR issues ?

This seems to be a very unfortunate name collision.  SparkR defines it's own 
DataFrame class which shadows what seems to be your own definition.

Is DataFrame something you define?  Can you rename it?

On Mon, Mar 14, 2016 at 10:44 PM, roni 
<roni.epi...@gmail.com<mailto:roni.epi...@gmail.com>> wrote:
Hi,
 I am working with bioinformatics and trying to convert some scripts to sparkR 
to fit into other spark jobs.

I tries a simple example from a bioinf lib and as soon as I start sparkR 
environment it does not work.

code as follows -
countData <- matrix(1:100,ncol=4)
condition <- factor(c("A","A","B","B"))
dds <- DESeqDataSetFromMatrix(countData, DataFrame(condition), ~ condition)

Works if i dont initialize the sparkR environment.
 if I do library(SparkR) and sqlContext <- sparkRSQL.init(sc)  it gives 
following error

> dds <- DESeqDataSetFromMatrix(countData, as.data.frame(condition), ~ 
> condition)
Error in DataFrame(colData, row.names = rownames(colData)) :
  cannot coerce class "data.frame" to a DataFrame

I am really stumped. I am not using any spark function , so i would expect it 
to work as a simple R code.
why it does not work?

Appreciate  the help
-R




--
Alex Kozlov
(408) 507-4987
(650) 887-2135 efax
ale...@gmail.com<mailto:ale...@gmail.com>

Re: sparkR issues ?

2016-03-15 Thread Alex Kozlov 
This seems to be a very unfortunate name collision.  SparkR defines it's
own DataFrame class which shadows what seems to be your own definition.

Is DataFrame something you define?  Can you rename it?

On Mon, Mar 14, 2016 at 10:44 PM, roni  wrote:

> Hi,
>  I am working with bioinformatics and trying to convert some scripts to
> sparkR to fit into other spark jobs.
>
> I tries a simple example from a bioinf lib and as soon as I start sparkR
> environment it does not work.
>
> code as follows -
> countData <- matrix(1:100,ncol=4)
> condition <- factor(c("A","A","B","B"))
> dds <- DESeqDataSetFromMatrix(countData, DataFrame(condition), ~ condition)
>
> Works if i dont initialize the sparkR environment.
>  if I do library(SparkR) and sqlContext <- sparkRSQL.init(sc)  it gives
> following error
>
> > dds <- DESeqDataSetFromMatrix(countData, as.data.frame(condition), ~
> condition)
> Error in DataFrame(colData, row.names = rownames(colData)) :
>   cannot coerce class "data.frame" to a DataFrame
>
> I am really stumped. I am not using any spark function , so i would expect
> it to work as a simple R code.
> why it does not work?
>
> Appreciate  the help
> -R
>
>


-- 
Alex Kozlov
(408) 507-4987
(650) 887-2135 efax
ale...@gmail.com