[
https://issues.apache.org/jira/browse/SPARK-25527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649990#comment-16649990
]
Ran Haim commented on SPARK-25527:
--
Any update?
> Job stuck waiting for last stage to
[
https://issues.apache.org/jira/browse/SPARK-25527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628392#comment-16628392
]
Ran Haim commented on SPARK-25527:
--
Hi, actually as I said there is one task (in the la
[
https://issues.apache.org/jira/browse/SPARK-25527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim updated SPARK-25527:
-
Attachment: threaddumpjob.txt
> Job stuck waiting for last stage to start
>
Ran Haim created SPARK-25527:
Summary: Job stuck waiting for last stage to start
Key: SPARK-25527
URL: https://issues.apache.org/jira/browse/SPARK-25527
Project: Spark
Issue Type: Bug
C
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16248837#comment-16248837
]
Ran Haim edited comment on SPARK-22481 at 11/12/17 12:31 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16248837#comment-16248837
]
Ran Haim commented on SPARK-22481:
--
Hi,
I create this simple test that will show how slo
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247913#comment-16247913
]
Ran Haim commented on SPARK-22481:
--
No, it is not.
> CatalogImpl.refreshTable is slow
>
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247858#comment-16247858
]
Ran Haim commented on SPARK-22481:
--
Ok, I'll create a simple app to reproduce this, late
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247852#comment-16247852
]
Ran Haim commented on SPARK-22481:
--
I will double check on Sunday, but sparksession.tabl
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247838#comment-16247838
]
Ran Haim commented on SPARK-22481:
--
I can check it again on Sunday.
I don't know why it
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247828#comment-16247828
]
Ran Haim edited comment on SPARK-22481 at 11/10/17 5:47 PM:
I
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247828#comment-16247828
]
Ran Haim commented on SPARK-22481:
--
It is as I wrote above.
It takes 1 minute in 2.1.1 a
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16246176#comment-16246176
]
Ran Haim commented on SPARK-22481:
--
It takes about 2 seconds to create the dataset...i n
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim updated SPARK-22481:
-
Description:
CatalogImpl.refreshTable was updated in 2.1.1 and since than it has become
really slow.
The
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim updated SPARK-22481:
-
Description:
CatalogImpl.refreshTable was updated in 2.1.1 and since than it has become
really slow.
The
Ran Haim created SPARK-22481:
Summary: CatalogImpl.refreshTable is slow
Key: SPARK-22481
URL: https://issues.apache.org/jira/browse/SPARK-22481
Project: Spark
Issue Type: Bug
Components
[
https://issues.apache.org/jira/browse/SPARK-22481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim updated SPARK-22481:
-
Description:
CatalogImpl.refreshTable was updated in 2.1.1 and since than it has become
really slow.
The
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931619#comment-15931619
]
Ran Haim commented on SPARK-17436:
--
Hi,
I think we need to reopen this.
It seems that on
[
https://issues.apache.org/jira/browse/SPARK-19360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15837503#comment-15837503
]
Ran Haim commented on SPARK-19360:
--
Hi,
I added my own storage handler actually - and no
Ran Haim created SPARK-19360:
Summary: Spark 2.X does not support stored by cluase
Key: SPARK-19360
URL: https://issues.apache.org/jira/browse/SPARK-19360
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833367#comment-15833367
]
Ran Haim commented on SPARK-17436:
--
Hi,
I did not actually use 2.1 yet - so I cannot be
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15680994#comment-15680994
]
Ran Haim edited comment on SPARK-17436 at 11/20/16 12:06 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim updated SPARK-17436:
-
Description:
update
***
It seems that in spark 2.1 code, the sorting issue is resolved.
The s
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim updated SPARK-17436:
-
Description:
*** update
It seems that in spark 2.0 code, the sorting issue is resolved.
The sorter do
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim updated SPARK-17436:
-
Description:
update
***
It seems that in spark 2.0 code, the sorting issue is resolved.
The s
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim updated SPARK-17436:
-
Priority: Minor (was: Major)
> dataframe.write sometimes does not keep sorting
> ---
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim updated SPARK-17436:
-
Description:
When using partition by, datawriter can sometimes mess up an ordered dataframe.
The proble
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15680994#comment-15680994
]
Ran Haim edited comment on SPARK-17436 at 11/20/16 11:30 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15680994#comment-15680994
]
Ran Haim edited comment on SPARK-17436 at 11/20/16 11:29 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15680994#comment-15680994
]
Ran Haim commented on SPARK-17436:
--
Sure.
Basically I propose to stop using UnsafeKVExte
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15679204#comment-15679204
]
Ran Haim edited comment on SPARK-17436 at 11/19/16 12:44 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15679204#comment-15679204
]
Ran Haim commented on SPARK-17436:
--
Hi,
When you want to write your data to orc files or
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15674032#comment-15674032
]
Ran Haim commented on SPARK-17436:
--
I have basiaclly cloned the repository from https://
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15674032#comment-15674032
]
Ran Haim edited comment on SPARK-17436 at 11/17/16 3:48 PM:
I
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673230#comment-15673230
]
Ran Haim commented on SPARK-17436:
--
I am running thee build on linux, so this is not it.
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664233#comment-15664233
]
Ran Haim edited comment on SPARK-17436 at 11/14/16 3:45 PM:
S
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664233#comment-15664233
]
Ran Haim commented on SPARK-17436:
--
So...
Can you help?
Or at least point me to someone
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663358#comment-15663358
]
Ran Haim commented on SPARK-17436:
--
Hey, I'll do a pull and try again
How to I gain
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663345#comment-15663345
]
Ran Haim commented on SPARK-17436:
--
Look, I think this is a serious bug - it makes parq
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663343#comment-15663343
]
Ran Haim commented on SPARK-17436:
--
but the whole thing is that the partitioning here ac
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15661640#comment-15661640
]
Ran Haim edited comment on SPARK-17436 at 11/13/16 3:37 PM:
H
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15661640#comment-15661640
]
Ran Haim commented on SPARK-17436:
--
Hi,
I only got a chance to work on it now.
I saw tha
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611483#comment-15611483
]
Ran Haim edited comment on SPARK-17436 at 10/27/16 10:35 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611483#comment-15611483
]
Ran Haim commented on SPARK-17436:
--
usually you partition the data, and then you order i
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611457#comment-15611457
]
Ran Haim edited comment on SPARK-17436 at 10/27/16 10:32 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611457#comment-15611457
]
Ran Haim edited comment on SPARK-17436 at 10/27/16 10:22 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611457#comment-15611457
]
Ran Haim commented on SPARK-17436:
--
Of course it does, every technology that supports pa
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611331#comment-15611331
]
Ran Haim commented on SPARK-17436:
--
anyone?
> dataframe.write sometimes does not keep s
[
https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15471291#comment-15471291
]
Ran Haim commented on SPARK-17436:
--
I tries it under 1.6.1, but I did not see anything t
Ran Haim created SPARK-17436:
Summary: dataframe.write sometimes does not keep sorting
Key: SPARK-17436
URL: https://issues.apache.org/jira/browse/SPARK-17436
Project: Spark
Issue Type: Bug
A
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354851#comment-15354851
]
Ran Haim edited comment on SPARK-15731 at 6/29/16 9:14 AM:
---
I h
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15354851#comment-15354851
]
Ran Haim commented on SPARK-15731:
--
I have tested it under 1.6.1, and writing the data u
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351026#comment-15351026
]
Ran Haim commented on SPARK-15731:
--
Oh, OK - I will try to do this on 1.6 locally and th
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350996#comment-15350996
]
Ran Haim commented on SPARK-15731:
--
Yes, I will try it locally.
But this might be the pr
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15351003#comment-15351003
]
Ran Haim commented on SPARK-15731:
--
wait a minute, you are not addressing any issues on
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15350983#comment-15350983
]
Ran Haim commented on SPARK-15731:
--
You did not answer any of my questions, it will help
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim reopened SPARK-15731:
--
Hi,
MapR upgraded Spark to 1.6.1 and I still get this behavior described.
Did you test it on a file standal
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315887#comment-15315887
]
Ran Haim commented on SPARK-15731:
--
Fine, you can close it - if it works.
I will use a w
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315884#comment-15315884
]
Ran Haim commented on SPARK-15731:
--
I am using Mapr distribution and they provide spark
[
https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315782#comment-15315782
]
Ran Haim commented on SPARK-15731:
--
Hi Kevin.
it is pretty much the same, the only diff
Ran Haim created SPARK-15731:
Summary: orc writer directory permissions
Key: SPARK-15731
URL: https://issues.apache.org/jira/browse/SPARK-15731
Project: Spark
Issue Type: Bug
Reporter
[
https://issues.apache.org/jira/browse/SPARK-10872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307775#comment-15307775
]
Ran Haim commented on SPARK-10872:
--
is there a work around for this?
I need this for uni
[
https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15284698#comment-15284698
]
Ran Haim edited comment on SPARK-15348 at 5/16/16 3:09 PM:
---
Thi
[
https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15284698#comment-15284698
]
Ran Haim commented on SPARK-15348:
--
If I have a transnational table in hive, I cannot us
[
https://issues.apache.org/jira/browse/SPARK-15349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Haim closed SPARK-15349.
Resolution: Duplicate
> Hive ACID
> -
>
> Key: SPARK-15349
> URL: h
Ran Haim created SPARK-15349:
Summary: Hive ACID
Key: SPARK-15349
URL: https://issues.apache.org/jira/browse/SPARK-15349
Project: Spark
Issue Type: New Feature
Reporter: Ran Haim
Sp
Ran Haim created SPARK-15348:
Summary: Hive ACID
Key: SPARK-15348
URL: https://issues.apache.org/jira/browse/SPARK-15348
Project: Spark
Issue Type: New Feature
Reporter: Ran Haim
Sp
67 matches
Mail list logo
