[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Description:
Step to reproduce:
1)Use SPak Streaming application.
2) inside foreachRDD of an
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev resolved SPARK-31427.
Resolution: Invalid
> Spark Structure streaming read data twice per every micro-batch.
> -
[
https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev resolved SPARK-34204.
Fix Version/s: 3.1.1
Resolution: Fixed
> When use input_file_name() func all column
[
https://issues.apache.org/jira/browse/SPARK-35094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17751630#comment-17751630
]
Nick Hryhoriev commented on SPARK-35094:
I miss understand, do the issue resolve
[
https://issues.apache.org/jira/browse/SPARK-42034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686914#comment-17686914
]
Nick Hryhoriev edited comment on SPARK-42034 at 2/10/23 7:36 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-42034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686914#comment-17686914
]
Nick Hryhoriev commented on SPARK-42034:
I can do PR, but I need someone to targ
Nick Hryhoriev created SPARK-42034:
--
Summary: QueryExecutionListener and Observation API, df.observe do
not work with `foreach` action.
Key: SPARK-42034
URL: https://issues.apache.org/jira/browse/SPARK-42034
[
https://issues.apache.org/jira/browse/SPARK-36263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17676265#comment-17676265
]
Nick Hryhoriev commented on SPARK-36263:
Hi, I find this feature does not work f
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344312#comment-17344312
]
Nick Hryhoriev commented on SPARK-11844:
[~sungwy92]
spark.hadoop.fs.s3a.fast.u
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342447#comment-17342447
]
Nick Hryhoriev commented on SPARK-11844:
I turn off `fast.data.upload` in spark
[
https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17340038#comment-17340038
]
Nick Hryhoriev commented on SPARK-34204:
I write every simple code to avoid it,
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334577#comment-17334577
]
Nick Hryhoriev commented on SPARK-11844:
[~Xu_Guang_Lv] My issue is also always
[
https://issues.apache.org/jira/browse/SPARK-35094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-35094:
---
Description:
I use spark 3.1.1 and 3.0.2.
Function `from_json` return wrong result with Pe
[
https://issues.apache.org/jira/browse/SPARK-35094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-35094:
---
Description:
I use spark 3.1.1 and 3.0.2.
Function `from_json` return wrong result with Per
Nick Hryhoriev created SPARK-35094:
--
Summary: Spark from_json(JsonToStruct) function return wrong
value in permissive mode in case best effort
Key: SPARK-35094
URL: https://issues.apache.org/jira/browse/SPARK-35
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317066#comment-17317066
]
Nick Hryhoriev edited comment on SPARK-11844 at 4/12/21, 3:05 PM:
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317874#comment-17317874
]
Nick Hryhoriev commented on SPARK-31427:
thx [~kabhwan], yes my only concern rea
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317066#comment-17317066
]
Nick Hryhoriev edited comment on SPARK-11844 at 4/8/21, 2:22 PM:
-
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317066#comment-17317066
]
Nick Hryhoriev edited comment on SPARK-11844 at 4/8/21, 11:50 AM:
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317066#comment-17317066
]
Nick Hryhoriev edited comment on SPARK-11844 at 4/8/21, 10:51 AM:
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317066#comment-17317066
]
Nick Hryhoriev edited comment on SPARK-11844 at 4/8/21, 10:49 AM:
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317066#comment-17317066
]
Nick Hryhoriev edited comment on SPARK-11844 at 4/8/21, 10:35 AM:
[
https://issues.apache.org/jira/browse/SPARK-11844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317066#comment-17317066
]
Nick Hryhoriev commented on SPARK-11844:
I have experienced the same issue with
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17301281#comment-17301281
]
Nick Hryhoriev edited comment on SPARK-31427 at 3/14/21, 9:26 PM:
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17301281#comment-17301281
]
Nick Hryhoriev commented on SPARK-31427:
hi [~xccui] , after some deep investiga
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-31427:
---
Affects Version/s: 2.4.7
3.0.1
3.0.2
> Spark S
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300777#comment-17300777
]
Nick Hryhoriev edited comment on SPARK-31427 at 3/13/21, 10:38 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300777#comment-17300777
]
Nick Hryhoriev edited comment on SPARK-31427 at 3/13/21, 10:21 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300777#comment-17300777
]
Nick Hryhoriev edited comment on SPARK-31427 at 3/13/21, 10:20 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300777#comment-17300777
]
Nick Hryhoriev edited comment on SPARK-31427 at 3/13/21, 10:16 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300777#comment-17300777
]
Nick Hryhoriev edited comment on SPARK-31427 at 3/13/21, 9:41 AM:
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300777#comment-17300777
]
Nick Hryhoriev commented on SPARK-31427:
And even more strange, when I use only
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300775#comment-17300775
]
Nick Hryhoriev commented on SPARK-31427:
In spark 3 on GUI, I even can see that
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300773#comment-17300773
]
Nick Hryhoriev edited comment on SPARK-31427 at 3/13/21, 9:24 AM:
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300773#comment-17300773
]
Nick Hryhoriev edited comment on SPARK-31427 at 3/13/21, 9:22 AM:
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17300773#comment-17300773
]
Nick Hryhoriev commented on SPARK-31427:
I check spark 3, it's the same.
Maybe i
[
https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-34204:
---
Description:
input_file_name() function damage applying projection to the physical plan of
[
https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-34204:
---
Description:
input_file_name() function damage applying projection to the physical plan of
[
https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270326#comment-17270326
]
Nick Hryhoriev commented on SPARK-34204:
Look like `spark_partition_id` have sam
[
https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-34204:
---
Summary: When use input_file_name() func all column from file appeared in
physical plan of q
[
https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-34204:
---
Description:
input_file_name() function damage applying projection to the physical plan of
[
https://issues.apache.org/jira/browse/SPARK-34204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-34204:
---
Description:
input_file_name() function damage applying projection to the physical plan of
Nick Hryhoriev created SPARK-34204:
--
Summary: input_file_name() func damage applying projection to
physical plan of query.
Key: SPARK-34204
URL: https://issues.apache.org/jira/browse/SPARK-34204
Proj
[
https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188342#comment-17188342
]
Nick Hryhoriev edited comment on SPARK-12837 at 9/1/20, 10:12 AM:
[
https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188342#comment-17188342
]
Nick Hryhoriev commented on SPARK-12837:
I have the same issue with the dataset.
[
https://issues.apache.org/jira/browse/SPARK-30364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084033#comment-17084033
]
Nick Hryhoriev commented on SPARK-30364:
I have the same issue with the spark ap
[
https://issues.apache.org/jira/browse/SPARK-31427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082163#comment-17082163
]
Nick Hryhoriev commented on SPARK-31427:
[~kabhwan] I will try to do it, but do
Nick Hryhoriev created SPARK-31427:
--
Summary: Spark Structure streaming read data twice per every
micro-batch.
Key: SPARK-31427
URL: https://issues.apache.org/jira/browse/SPARK-31427
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-19407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968226#comment-16968226
]
Nick Hryhoriev commented on SPARK-19407:
[~uncleGen]
> defaultFS is used FileSy
[
https://issues.apache.org/jira/browse/SPARK-19407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968221#comment-16968221
]
Nick Hryhoriev edited comment on SPARK-19407 at 11/6/19 9:40 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-19407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968221#comment-16968221
]
Nick Hryhoriev edited comment on SPARK-19407 at 11/6/19 9:39 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-19407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968221#comment-16968221
]
Nick Hryhoriev edited comment on SPARK-19407 at 11/6/19 9:37 AM:
-
[
https://issues.apache.org/jira/browse/SPARK-19407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968221#comment-16968221
]
Nick Hryhoriev commented on SPARK-19407:
I have the same issue with spark 2.4.4.
[
https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004733#comment-16004733
]
Nick Hryhoriev edited comment on SPARK-5594 at 5/12/17 6:32 AM:
[
https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004733#comment-16004733
]
Nick Hryhoriev commented on SPARK-5594:
---
Hi,
i have the same issue.
But in spark 2.1
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Description:
Step to reproduce:
1)Use SPak Streaming application.
2) inside foreachRDD of any
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Description:
Step to reproduce:
1)Use SPak Streaming application.
2) inside foreachRDD of any
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Description:
Step to reproduce:
1)Use SPak Streaming application.
2) inside foreachRDD of any
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Description:
Step to reproduce:
1)Use SPak Streaming application.
2) inside foreachRDD of any
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Description:
Step to reproduce:
1)Use SPak Streaming application.
2) inside foreachRDD of any
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940113#comment-15940113
]
Nick Hryhoriev commented on SPARK-20080:
It's clear issue. Even with a link to co
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Summary: Spak streaming application do not throw serialisation exception in
foreachRDD (was:
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev reopened SPARK-20080:
Problem with loosing exception still exist.
> Spak streaming up do not throw serialisation exc
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15940069#comment-15940069
]
Nick Hryhoriev commented on SPARK-20080:
[~sowen] You don't understand problem.
T
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Description:
When i try use or init org.slf4j.Logger inside foreachPartition, that extracted
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Description:
up vote
0
down vote
favorite
When i try use or init org.slf4j.Logger inside fore
[
https://issues.apache.org/jira/browse/SPARK-20080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Hryhoriev updated SPARK-20080:
---
Description:
up vote
0
down vote
favorite
When i try use or init org.slf4j.Logger inside fore
Nick Hryhoriev created SPARK-20080:
--
Summary: Spak streaming up do not throw serialisation exception in
foreachRDD
Key: SPARK-20080
URL: https://issues.apache.org/jira/browse/SPARK-20080
Project: Spa
68 matches
Mail list logo