On 30/07/11 01:09, Rottinghuis, Joep wrote:
Thanks for the replies.
To elaborate on why I want to build on a server w/o Internet access:
Build should not reach out to Internet and grab jars from unverified sources
w/o md5 hash check etc.
automated hash checking is flawed for various reasons
On 30/07/11 01:17, Alejandro Abdelnur wrote:
Joep,
Ivy& Maven pull JARs from the maven repos you specify.
Maven verifies checksums and I assume Ivy does.
by default? If so, that's an improvement.
You could turn your verified ~/.m2 into a Maven proxy and switch fetching
JARs not found in th
FWIW, the ASF Infra guys aren't that keen on more machines they won't have
physical control of. There may be some wiggle room on that, but they'll
definitely need sudo and they would much prefer non-single-purpose machines.
Just a heads up from my talks with Infra lately.
A.
On Mon, Aug 1, 2011 a
On Sun, Jul 31, 2011 at 9:57 PM, Tom White wrote:
> HADOOP-6671 has now received two +1s (one from Eric Yang and one from
> me), so I would like to commit it on Tuesday at 16:00 GMT
> (http://s.apache.org/6nx). I'll also update the Jenkins jobs for
> running test-patch and performing the nightly b
HADOOP-6671 is now in trunk. After you do an update you will need to
use Maven 3 to build Common. There are instructions at
https://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-common/BUILDING.txt
HDFS and MapReduce still use Ant, and the instructions for
cross-project builds are in
http:/
A couple of questions for Nigel and Andrew:
1. What do you mean by "non-single-purpose machines" ? (I guess, machines
they can use for non-Hadoop projects as well ?)
2. Do these machines have to be physical boxes or virtual machines be okay
too ? (sudo on virtual machines would be easier to sell
You should check with ASF Infra for the actual answers - I'm just relaying
my impressions from talking with them.
1) They don't want "targeted donations" - i.e., a donated machine that is
solely for builds of a specific set of projects, rather than one that is
available for builds of any project.
On Aug 1, 2011, at 11:22 PM, Nigel Daley wrote:
> Ideally the hardware is:
> * hosted and OS managed by the donor,
> * publicly addressable on the internet,
> * running Ubuntu or CentOS, and
> * sudo access can be given to Apache's Jenkins admins so they can create
> accounts for committers as
It seems to me that HW diversity is good for test machines, but not for
build machines. Build machines you want to be rock solid and repeatable.
Test machines you want to span the range of real-world variability.
IMHO,
--Matt
On Tue, Aug 2, 2011 at 10:56 AM, Allen Wittenauer wrote:
>
> On Aug
As a build guy by trade, I tend to like the idea of doing builds (with unit
tests) on a relative diversity of platforms. But the underlying hardware
should be consistent across the platforms, so that any variations in results
can be narrowed down to the platform differences rather than hardware
dif
Milind.
>From about 30 boxes Y! gave to ASF at least (I believe there were more, but at
least that much) 10 of them were used _exclusively_ as Hadoop build/test
machines (e.g. Hudson slaves) and a number of others (about 5 at least) were
used for Hadoop deployment testing _exclusively_. So, I thin
That's good to know, Kos.
Is there any place I can find the actual usage (in terms of utilization /
memory-cpu-disk usage) of these build machines ?
Based on the Amazon EC2 pricing (http://aws.amazon.com/ec2/pricing/), I
would like to determine which type of instances would be adequate, and how
m
As I mentioned before, I believe that ASF Infra explicitly does *not* want
donated EC2 instances, but again, check with them to be sure and as to what
the donation process is regardless.
A.
On Tue, Aug 2, 2011 at 12:01 PM, wrote:
> That's good to know, Kos.
>
> Is there any place I can find the
Having talked to various folks in the community about their willingness to
stabilize and use 0.22 and make it production-quality, here is what I
propose:
1. Cut a release 0.22.0 without mapreduce-2178 patch, with
hadoop.security.authentication set to simple (I.e. No authentication).
Make sure that
On Tue, Aug 2, 2011 at 12:19 PM, wrote:
> Having talked to various folks in the community about their willingness to
> stabilize and use 0.22 and make it production-quality, here is what I
> propose:
>
> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
> hadoop.security.authentication s
0.22 seems to be loaded enough to add value even with
a couple of the bugs like mentioned. This is a significant step towards
field-testing of 0.23 and so on and will give to applications on top of Hadoop
a chance to with latest Apache release of what Hadoop might look like in the
near future.
+1
On Aug 2, 2011, at 12:19 PM,
wrote:
> 1. Cut a release 0.22.0 without mapreduce-2178 patch, with
> hadoop.security.authentication set to simple (I.e. No authentication).
> Make sure that MR-2178 is highlighted as known-issue in the top-level dir
> (create a KNOWN-ISSUES.txt ?), and carry out a
On Tue, Aug 2, 2011 at 12:25 PM, Eli Collins wrote:
> On Tue, Aug 2, 2011 at 12:19 PM, wrote:
>> Having talked to various folks in the community about their willingness to
>> stabilize and use 0.22 and make it production-quality, here is what I
>> propose:
>>
>> 1. Cut a release 0.22.0 without m
+1 to removing the TaskController code for the 0.22 release.
In the MR-279 branch, the security issues identified in MAPREDUCE-2178 have
been taken care of (to my knowledge).
On Aug 2, 2011, at 12:58 PM, Eli Collins wrote:
> On Tue, Aug 2, 2011 at 12:25 PM, Eli Collins wrote:
>> On Tue, Aug 2
Eli,
I hope we don't have to remove the TaskTracker *grin*.
Devaraj/Allen,
You mean LinuxTaskController only, right ?
The default is DefaultTaskController, which is not relevant to these
discussions.
+1. This is the easiest way to get 0.22.0 out.
- Milind
---
Milind Bhandarkar
Greenplum Labs,
Yes... I meant the LTC.
On Aug 2, 2011, at 1:23 PM, wrote:
> Eli,
>
> I hope we don't have to remove the TaskTracker *grin*.
>
> Devaraj/Allen,
>
> You mean LinuxTaskController only, right ?
>
> The default is DefaultTaskController, which is not relevant to these
> discussions.
>
> +1. Th
+1 (non-binding). Sounds good to me.
--Matt
On Tue, Aug 2, 2011 at 1:27 PM, Devaraj Das wrote:
> Yes... I meant the LTC.
>
> On Aug 2, 2011, at 1:23 PM, wrote:
>
> > Eli,
> >
> > I hope we don't have to remove the TaskTracker *grin*.
> >
> > Devaraj/Allen,
> >
> > You mean LinuxTaskControlle
With the HADOOP-6671 commit the way of building hadoop common has changed
significantly.
While the wiki explains these changes, and there is a BUILDING.txt
directory, still I guess things will hit many of of you.
Because of this I've put together some brief notes.
Thanks.
Alejandro
---
Thanks for all your work and updates on this, Alejandro! It's much
better/easier to work with :)
I did have a few issues/questions:
First, I still had to manually add 'target/generated-src/test/java' to my
build path sources in eclipse. I don't know if this is due to something I
did wrong, but I
Jeffrey,
Thanks.
Regarding adding the 'target/generated-src/test/java' dir to the build path.
You are correct, you have to add it manually to your IDE (I use IntelliJ and
it is the same story). But unless you need to debug through the generated
code you don't need to do so (doing a 'mvn test -Dsk
On 8/2/11 5:21 PM, "Alejandro Abdelnur" wrote:
> Regarding adding the 'target/generated-src/test/java' dir to the build path.
> You are correct, you have to add it manually to your IDE (I use IntelliJ and
> it is the same story). But unless you need to debug through the generated
> code you don't
On Tue, Aug 2, 2011 at 1:23 PM, wrote:
> Eli,
>
> I hope we don't have to remove the TaskTracker *grin*.
>
> Devaraj/Allen,
>
> You mean LinuxTaskController only, right ?
>
Yup, meant the LTC.
Thanks,
Eli
On Tue, Aug 2, 2011 at 3:47 PM, Jeffrey Naisbitt wrote:
> On 8/2/11 5:21 PM, "Alejandro Abdelnur" wrote:
>> Regarding adding the 'target/generated-src/test/java' dir to the build path.
>> You are correct, you have to add it manually to your IDE (I use IntelliJ and
>> it is the same story). But un
(the following discusses religious practices ... please don't break into
flames)
In the past, the simplest approach I have seen for dealing with this is to
simply put the generated code under the normal source dir and check it in.
This is particularly handy with Thrift since it is common for user
On Tue, Aug 2, 2011 at 3:21 PM, Alejandro Abdelnur wrote:
> Jeffrey,
>
> Thanks.
>
> Regarding adding the 'target/generated-src/test/java' dir to the build path.
> You are correct, you have to add it manually to your IDE (I use IntelliJ and
> it is the same story). But unless you need to debug thr
Hi Folks,
I've talked to the folks at Yahoo about the build machines and am happy to
report that an end to the blackout is in sight.
1) They need to reimage the machines, which is in progress and machines should
be restored within a week (pessimistically).
2) They plan to reach out to the apac
Luke,
Let me try that, thanks for the tip.
Alejandro
On Tue, Aug 2, 2011 at 7:09 PM, Luke Lu wrote:
> On Tue, Aug 2, 2011 at 3:21 PM, Alejandro Abdelnur
> wrote:
> > Jeffrey,
> >
> > Thanks.
> >
> > Regarding adding the 'target/generated-src/test/java' dir to the build
> path.
> > You are cor
Thanks Eric.
thats great news.
--I
On Aug 3, 2011, at 1:05 PM, Eric Baldeschwieler wrote:
> Hi Folks,
>
> I've talked to the folks at Yahoo about the build machines and am happy to
> report that an end to the blackout is in sight.
>
> 1) They need to reimage the machines, which is in progress
Yes, donating hardware to ASF is tricky for all the reasons you've outlined,
not the least of which you can't donate to a specific project. That's why I
emphasized **access** to hardware in my request.
Background of current ASF build machines:
1) Y! officially donated 4 machines to ASF, hostin
+1.
n.
On Aug 2, 2011, at 1:35 PM, Matt Foley wrote:
> +1 (non-binding). Sounds good to me.
> --Matt
>
>
> On Tue, Aug 2, 2011 at 1:27 PM, Devaraj Das wrote:
>
>> Yes... I meant the LTC.
>>
>> On Aug 2, 2011, at 1:23 PM, wrote:
>>
>>> Eli,
>>>
>>> I hope we don't have to remove the Tas
Great! Thanks for getting the info.
Cheers,
n.
On Aug 2, 2011, at 8:05 PM, Eric Baldeschwieler wrote:
> Hi Folks,
>
> I've talked to the folks at Yahoo about the build machines and am happy to
> report that an end to the blackout is in sight.
>
> 1) They need to reimage the machines, which i
36 matches
Mail list logo
