I mostly use Amazon S3 for reading input data and writing output data for my
spark jobs. I want to know the numbers of bytes read & written by my job
from S3.
In hadoop, there are FileSystemCounters for this, is there something similar
in spark ? If there is, can you please guide me on how to use
This bug still exists in Spark-1.4.0. Is there a workaround for it?
https://issues.apache.org/jira/browse/SPARK-7944
Thanks,
Alex
Whatever you do, DO NOT use the built-in JIRA 'releases' feature to migrate
issues from 1.4.0 to another version: the JIRA feature will have the
side-effect of automatically changing the target versions for issues that
have been closed, which is going to be really confusing. I've made this
mistake
Question: what would happen if I cleared Target Version for everything
still marked Target Version = 1.4.0? There are 76 right now, and
clearly that's not correct.
56 were opened by committers, including issues like "Do X for 1.4".
I'd like to understand whether these are resolved but just weren't
