Hi Folks,
ARM is becoming famous lately in its processing capability and has got the
potential to run Bigdata workloads.
Many users have been moving to ARM machines due to its low cost.
In the past there were attempts to compile Hadoop on ARM (Rasberry PI) for
experimental purposes. Today ARM arc
Hi,
Thanks Vinaya for bring this up and thanks Sheng for the idea. A separate
branch with it's own ARM CI seems a really good idea.
By doing this we won't break any of the undergoing development in trunk and
a CI can be a very good way to show what are the
current problems and what have been fixed
+1, for the branch idea. Just FYI, Your biggest problem is proving that
Hadoop and the downstream projects work correctly after you upgrade core
components like Protobuf.
So while branching and working on a branch is easy, merging back after you
upgrade some of these core components is insanely har
For HBase, we purged all the protobuf related things from the public API,
and then upgraded to a shaded and relocated version of protobuf. We have
created a repo for this:
https://github.com/apache/hbase-thirdparty
But since the hadoop dependencies still pull in the protobuf 2.5 jars, our
coproce
Thanx Vinay for the initiative, Makes sense to add support for different
architectures.
+1, for the branch idea.
Good Luck!!!
-Ayush
> On 03-Sep-2019, at 6:19 AM, 张铎(Duo Zhang) wrote:
>
> For HBase, we purged all the protobuf related things from the public API,
> and then upgraded to a shaded
BTW, I also noticed that the Hadoop-trunk-Commit job has been failling for
over 2 month related to the Protobuf problem .
According to the latest successful build log:
https://builds.apache.org/job/Hadoop-trunk-Commit/lastSuccessfulBuild/consoleFull
the
os version was ubuntu 14.04 and for the jobs
Suggest to use org.xolstice.maven.plugins:protobuf-maven-plugin to generate
the protobuf code. It will download the protoc binary from the maven
central so we do not need to install protoc on the build machine any more.
Zhenyu Zheng 于2019年9月4日周三 下午5:27写道：
> BTW, I also noticed that the Hadoop-tr
Thanks Vinay for starting the thread.
I agree to Anu's view point related to protobuf. And with the suggestion
pointed out by Duo Zhang, if we can make use
of org.xolstice.maven.plugins:protobuf-maven-plugin, our upgrade to 3.0.0
of protobuf will also be more easier.
However i think its better to
Hi all,
Thanks for the response.
As I see, protobuf upgrade is long pending and most awaited one.
@Sunil
Protobuf upgrade looks to be a non-trivial task.
Thanks @Duo Zhang for the suggestion of
'org.xolstice.maven.plugins:protobuf-maven-plugin'. This solves the problem
of dependency on build envi
Yes, I think that is what Sunil and I are trying to suggest; the complex
dependencies like Protobuf, if you do it in the trunk you have a better
change of getting it done. Otherwise, at merge point random downstream
applications which you have never heard of will object, and Hadoop
compatibility ru
Thanks @Anu
I understand the concern. I took it in different manner.
Anyway, since protobuf upgrade looks huge, and need everyone's eyes on
changes as early as possible, its better to do it trunk itself.
I was able to come to successfull attempt of upgrading protobuf as per
suggestion of stack i
Hi All,
Thanks Vinaya bring the whole thing up and everyone else for discussing,
providing thoughts and doing actuall coding.
I noticed that protobuf upgrading has been the hottest discussion point
about the whole thread and Vinayakumar, Duo Zhang and Akira Ajisaka have
done alot about protocbuf
Hi All,
Some updates for ARM CI, our team has succesfully donated ARM resources and
setup an ARM CI for Apache Spark:
https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-arm/
it will set to periodic job and then PR trigger when we think it is stable
enough.
I really hope we can do
Thanks Zhenyu Zheng for the offer to donate machines for ARM CI.
We will definitely make use of it. Let me check how we can set up Jenkins
job for ARM.
-Thanks
Vinay
On Tue, 22 Oct 2019, 1:04 pm Zhenyu Zheng,
wrote:
> Hi All,
>
> Some updates for ARM CI, our team has succesfully donated ARM r
14 matches
Mail list logo
