Folks,
In the release numbering discussions that we had less than 3 months ago, there
was a suggestion about 0.22 release being an active release, and that it should
be reserved a major release number, since it has major changes (especially in
HDFS, since a viable HA based on a working standby
You can take a look at distributed shell.
- milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)
On 4/5/12 4
My (non-binding) vote is: 1, 2, 3, 4, 5
- milind
>On Mar 19, 2012, at 6:06 PM, Arun C Murthy wrote:
>
>> We've discussed several options:
>>
>> (1) Rename branch-0.22 to branch-2, rename branch-0.23 to branch-3.
>> (2) Rename branch-0.23 to branch-3, keep branch-0.22 as-is i.e. leave a
>>hole.
>
Arun,
As Konstantin has noted in the email below:
> If the community decides to rename .22 to 2 I will be glad to work on it.
My inclination (as I have communicated to several people at apachecon) is
to upgrade our clusters from 1.0 to 0.23 (whatever it is called when it
becomes stable). The re
I agree with Konstantin. In previous discussion, I had suggested
simultaneous renumbering, but for some reason it was not considered.
(For history buffs: I upgraded from Windows 1.0 to Windows 3.1 straight.
Windows 2.0 did not have many features that made it compelling to upgrade.
It did not seem
+1
Downloaded, installed on single node, (simple authentication), ran hadoop
mapred examples successfully.
- Milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, pas
-1 (non-binding)
While I support renaming 0.20.205.1 as 1.0.0 and branch-0.20-security as
branch-1.x.x, I think that renumbering other branches in play (0.22, 0.23,
0.24) should be done in the same transaction. Otherwise, the goal of this
whole exercise (I.e. Reducing user-confusion) will be defea
Arun,
You beat me to start this discussion :-)
I was at Apachecon recently, and based on the questions and comments from
several attendees for the hadoop sessions, as well as the hadoop meetup
afterwards, it was clear that users are perplexed about our versioning
strategies.
In addition, Doug an
Oh, that is very interesting ! Would I have to write my own
container-launcher for that ? Or does existing container launcher support
it ?
- Milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the
Steve,
>I would suggest moving up projects on the side first, layers on top,
>like your MPI-on-YARN stuff, where there is little or no installed base
>to worry about
That is exactly the project for which I though some of the new I/O
interfaces would reduce the amount of code, (plus auto-freed res
>I believe you mean when will it be compatible with Java 7, not require it
>(based on all you mention above, other than the word 'required'). Is that
>the case?
I meant, "when will I be able to use some of the JDK 7 APIs in Hadoop code
?", which means JDK7 is required for Hadoop.
- milind
---
Hi Folks,
While I have seen the wiki on which java versions to use currently to run
Hadoop, I have not seen any discussion about the roadmap of java version
compatibility with future hadoop versions.
Recently, Oracle retired the "Operating System Distributor License for
Java" (DLJ) [http://robila
Steve,
>
>you can improve Hadoop to make it more agile; my defunct Hadoop
>lifecycle branch did a lot of that, but you have to have everyone else
>using Hadoop to be willing to let the changes go in -and those changes
>mustn't impose a cost or risk to the physical cluster model.
Until Hadoop 0.
Steve,
>Summary: I'm not sure that HDFS is the right FS in this world, as it
>contains a lot of assumptions about system stability and HDD persistence
>that aren't valid any more. With the ability to plug in new placers you
>could do tricks like ensure 1 replica lives in a persistent blockstore
>(
Jagane,
I understand your use case, I think, and so here are my thoughts, inline:
>1. Hbase support, i.e. working scale tested Append and Hflush in HDFS
Absolutely. Hbase (and other components of the stack that do not follow
the MapReduce paradigm) are increasingly important. It is important to
Jagane,
I think you have forgotten one major deciding factor:
Which version is *your* vendor committed to support ?
If you are at the same place where you were the last time we met, you have
no other choice but to go with 0.20.206. It's in the contract ! :-)
- Milind
---
Milind Bhandarkar
Gree
Excellent. 0.9.2 it is, then ! Thanks @squarecog.
- milind
On 9/30/11 11:26 AM, "Dmitriy Ryaboy" wrote:
>9.1 just went up for vote, and holding up a release to add 0.22
>compatibility seems ill-advised.
>I would be willing to help get 9.2 out in short order to provide
>compatibility with 22, a
3. There's 0.22 which is going to combine the API of 0.21 with the fixes
>of 0.20.20x *and* will be the last release of the MR1.0 engine. For that
>last reason, I think there's value in pushing it out, though it's going
>to take time, and there's a risk of it adding another branch to be
>maintaine
Dmitry,
So, your suggestion would be to try and get 0.9.1 compatible with Hadoop
0.22 ? Would there be an interest from pig committers to see that through,
as a 0.9.2 perhaps ?
- Milind
---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the autho
Roman,
What is your definition of pig 0.8.2 ? Is it 0.8.1 + Hadoop 0.22
compatibility ?
Joep has already made 0.8.1 work with Hadoop 0.22.
- milind
On 9/29/11 3:52 PM, "Roman Shaposhnik" wrote:
>2011/9/29 :
>> [CC'ng Pig-dev].
>>
>> In terms of interest, what is the community more interested
[CC'ng Pig-dev].
In terms of interest, what is the community more interested in seeing work
with Hadoop 0.22 ? Pig 0.8.x or Pig.0.9.
I took a look at pig 0.9 added features:
New parser (still has corner cases)
UDFs in languages other than Java
Macros
Hcatalog support
As far as existing installa
This is great progress folks !!!
- Milind
On 9/27/11 9:55 PM, "J. Rottinghuis" wrote:
>Thanks Roman,
>
>Will have to give BigTop a try...
>With the patch for https://issues.apache.org/jira/browse/PIG-2277 applied
>Pig (0.8 for now, and 0.9 in the works) will compile and run against 0.22
>as
>w
Never mind. It was an oblique reference to client not writing to a file
for a long time, so hdfs recovering the lease. (RM=client, release=file,
recovery=transferring RM role. :-)
- milind
On 9/9/11 2:58 PM, "Konstantin Boudnik" wrote:
>On Fri, Sep 09, 2011 at 05:55PM, milind.bhandar...@emc.com
But did he do the lease recovery (for more info: HDFS-265) ? I haven seen
the initiation of lease recovery by Konst, but haven't seen acks.
- milind
On 9/9/11 2:13 PM, "Konstantin Boudnik" wrote:
>KOnstantin has stepped forward a couple of days ago ;)
>
>On Fri, Sep 09, 2011 at 05:08PM, milind
Has anyone seen the RM for 0.22 lately ?
- milind
On 9/9/11 1:46 PM, "Roy T. Fielding" wrote:
>On Sep 9, 2011, at 11:08 AM, Eli Collins wrote:
>> The release manager - not the developers - are responsible for and
>> have the final say as to what patches get merge to their branch.
>
>That is sim
I am very interested in hearing what the community thinks about this
issue. I believe what happens here has long-term consequences.
AKAIK, the bylaws do not mention anything about "abandoning release". If
past is any indication, even if releases 0.19.*, and 0.21.* were not
adopted by large hadoop
>
>
>
>For example, https://issues.apache.org/jira/browse/HDFS-2288 can cause
>HBase to fail to recover its WAL during a crash scenario. There are
>some others that I'll be likely working through in the coming months.
Thanks Todd.
Will go through it to test against 0.22.
- milind
---
Milind Bha
FWIW, Stack has already done the work needed to make sure that Hbase works
with Hadoop 0.22 branch, and I suppose if
https://issues.apache.org/jira/browse/MAPREDUCE-2767 is committed, it
removes the last blocker from 0.22.0, so that it can be released.
I am cc'ng hbase-dev, since this is relevant
>
>
>
>It still has to be asked whether the changes meet the criterion of
>manageable risk. That definitely should be evaluated
>after we see the proposed patches.
Matt,
As a release manager, it is up to you to decide what the criterion for
inclusion is. And that was what I was asking.
- milind
Thanks for the prompt response Eric.
I have been traveling, and could reply to you immediately.
How long do you estimate 0.23 (or 0.22) would take to stabilize ? Based on
the past experience of release 0.19, even if it was not deployed at 1000+
node-scale, eventually, it did stabilize, and severa
> At the same time, we certainly do not wish 0.20-security to be viewed as a
> "trunk"; it is important
that all patches go in trunk first, and only patches of manageable risk and
high value to production users, should go into
the sustaining releases.
Matt,
With all due respect, I have heard fr
Nigel,
I have created https://issues.apache.org/jira/browse/MAPREDUCE-2767, and
uploaded the patch.
Please review. Thanks.
- milind
---
Milind Bhandarkar
Greenplum Labs, EMC
((Disclaimer: Opinions expressed in this email are those of the author,
and do
not necessarily represent the views of any
I had this exact same argument with Arun Murthy at the hadoop dev meeting
about checking in protobuf generated code in the MR-279 branch. That same
evening, he had to download entire xcode and install it on his new Mac to
build the MR-279 branch :-)
Hadoop recordio also follows the same approach.
Eli,
I hope we don't have to remove the TaskTracker *grin*.
Devaraj/Allen,
You mean LinuxTaskController only, right ?
The default is DefaultTaskController, which is not relevant to these
discussions.
+1. This is the easiest way to get 0.22.0 out.
- Milind
---
Milind Bhandarkar
Greenplum Labs,
Having talked to various folks in the community about their willingness to
stabilize and use 0.22 and make it production-quality, here is what I
propose:
1. Cut a release 0.22.0 without mapreduce-2178 patch, with
hadoop.security.authentication set to simple (I.e. No authentication).
Make sure that
That's good to know, Kos.
Is there any place I can find the actual usage (in terms of utilization /
memory-cpu-disk usage) of these build machines ?
Based on the Amazon EC2 pricing (http://aws.amazon.com/ec2/pricing/), I
would like to determine which type of instances would be adequate, and how
m
A couple of questions for Nigel and Andrew:
1. What do you mean by "non-single-purpose machines" ? (I guess, machines
they can use for non-Hadoop projects as well ?)
2. Do these machines have to be physical boxes or virtual machines be okay
too ? (sudo on virtual machines would be easier to sell
37 matches
Mail list logo
