Github user RongGu commented on the pull request:
https://github.com/apache/incubator-spark/pull/468#issuecomment-35198329
Hey @pwendell, I have processed the lower level comments put forward by you
and updated the code in this PR. Next, I will modify the namespace of tachyon
as suggested by @mateiz.
My respondence to your high level feedback is as below:
Tests -
Currently, I have put a small UTs about TacyonStore in the
BlockManagerSuite.scala. I also set a switch parameter for disabling or
enabling the test on Tahcyon. I would add more UTs into it. BTW, I have try
some Spark applications on my my computer, this PR works well on Tachyon:)
Documentation -
Thank you and @ alig for having already given a lot of advice about the
Documentation. I will follow your advice and @haoyuan will also help me with
that.
Code Style -
Sorry about that. I have fixed a lot of them. When the PR comes to its code
freeze stage, I will check the whole code's style again.
Update the maven build to include Tachyon as well.-
I have done this now. Thanks for reminding.
The amount of recursive dependencies for tachyon is troubling.-
Yes, that's right. Actually, this is also a work in Tachyon @haoyuan is
talking with me these days offline, we are planning to separate Tachyon into
different component projects as Tachyon-Client, Tachyon-Server, Tachyon-Common
(like core, mllib, etc. in Spark). When this work is done, Spark will just need
to include Tachyon-Client into the build file, then it will avoid a lot of jar
dependencies. I am planning to do this when this PR's work is finished.
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
[email protected] or file a JIRA ticket with INFRA.
