[
https://issues.apache.org/jira/browse/SPARK-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-2127.
------------------------------
Resolution: Duplicate
The PR that closes SPARK-3377 also closed the PR for this JIRA, and it looks
like this is a subset of SPARK-3377.
> Use application specific folders to dump metrics via CsvSink
> ------------------------------------------------------------
>
> Key: SPARK-2127
> URL: https://issues.apache.org/jira/browse/SPARK-2127
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 1.0.0
> Reporter: Rahul Singhal
> Priority: Minor
>
> Currently when using the CsvSink, all application's csv metrics are dumped in
> the root folder (configured via "*.sink.csv.director" in metrics.properties).
> Also, some files that have common names (e.g. "jvm.PS-MarkSweep.count.csv")
> are reused. And if one is running the same application multiple times, the
> metrics get appended to previously existing files.
> This makes it harder to parse these files and extract the information that
> one might be looking for. I suggest that a unique folder is created every
> time an application is run and use it to dump the metrics from that
> particular run only. This unique folder could be created similar the one that
> is currently craeted for logging application events (e.g.
> "spark-pi-1402484928439").
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
