[
https://issues.apache.org/jira/browse/SPARK-20703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085559#comment-16085559
]
Steve Loughran commented on SPARK-20703:
----------------------------------------
Regarding a patch for this, what do people suggest as a good policy for failures
FNFE: log@debug, return 0
IOE: log@info, full stack @debug, return 0
or treat FNFE the same as any other IOE (which is most likely to be a transient
network failure, given by this point we know that the user has connectivity,
auth and write access to the bucket)?
# I would like the option of printing that full stack, so that if the time
comes to debug it, saying "edit this log setting" is enough to make problems
visible in production.
# Yet I don't want to log the full stack as a default as that can lead to
panic, and it also tends to mean when there are real problems in the code
people may just paste that stack in, rather than one for the real problem.
> Add an operator for writing data out
> ------------------------------------
>
> Key: SPARK-20703
> URL: https://issues.apache.org/jira/browse/SPARK-20703
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: Reynold Xin
> Assignee: Liang-Chi Hsieh
> Fix For: 2.3.0
>
>
> We should add an operator for writing data out. Right now in the explain plan
> / UI there is no way to tell whether a query is writing data out, and also
> there is no way to associate metrics with data writes. It'd be tremendously
> valuable to do this for adding metrics and for visibility.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
