[
https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379817#comment-14379817
]
Steve Loughran commented on SPARK-6479:
---------------------------------------
As part of my ongoing work to take formality to the OSS datacentre stack beyond
just ZK
([1|http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/filesystem/index.html],[2|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/src/main/tla/yarnregistry.tla]
, I'm willing to help define this API in TLA+, as well as derive tests from it
which can be used as a basis of asserting compliance.
FWIW defining the failure modes —specifically what exceptions are expected on
problems— tends to be hard;
concurrency & consistency something you can define but which isn't easy to test
outside multinode functional load tests.
> Create off-heap block storage API (internal)
> --------------------------------------------
>
> Key: SPARK-6479
> URL: https://issues.apache.org/jira/browse/SPARK-6479
> Project: Spark
> Issue Type: Improvement
> Components: Block Manager, Spark Core
> Reporter: Reynold Xin
> Attachments: SparkOffheapsupportbyHDFS.pdf
>
>
> Would be great to create APIs for off-heap block stores, rather than doing a
> bunch of if statements everywhere.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
