Re: Heads up - 2.0.5-beta
On Apr 26, 2013, at 12:07 PM, Eli Collins wrote: > Arun, Suresh, > > Mind reviewing the following page Karthik put together on > compatibility? http://wiki.apache.org/hadoop/Compatibility Sure. Will do. I just opened https://issues.apache.org/jira/browse/HADOOP-9517 to ensure we capture it for posterity. Karthik - Would you like to take a crack at it? The wiki would be a good starting point. thanks, Arun
[jira] [Created] (HADOOP-9517) Define Hadoop Compatibility
Arun C Murthy created HADOOP-9517: - Summary: Define Hadoop Compatibility Key: HADOOP-9517 URL: https://issues.apache.org/jira/browse/HADOOP-9517 Project: Hadoop Common Issue Type: Bug Components: documentation Reporter: Arun C Murthy As we get ready to call hadoop-2 stable we need to better define 'Hadoop Compatibility'. http://wiki.apache.org/hadoop/Compatibility is a start, let's document requirements clearly and completely. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Heads up - 2.0.5-beta
Konstantin, On Apr 26, 2013, at 4:34 PM, Konstantin Shvachko wrote: > Do you think we can call the version you proposed to release > 2.1.0 or 2.1.0-beta? > > The proposed new features imho do not exactly conform with the idea > of dot-dot release, but definitely qualify for a major number change. > I am just trying to avoid rather ugly 2.0.4.1 versions, which of course > also possible. I'm agnostic to the schemes. During the long discussion we had just 2 months ago, I proposed that 2.1.x be the beta series initially. The feedback and consensus was that it wasn't the right numbering scheme: http://s.apache.org/1j4 thanks, Arun
[jira] [Resolved] (HADOOP-1773) Hadoop build (ant) hangs while setting up init target, build process hangs
[ https://issues.apache.org/jira/browse/HADOOP-1773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-1773. Resolution: Cannot Reproduce Fix Version/s: 3.0.0 resolving as a can't reproduce now that trunk is on Maven > Hadoop build (ant) hangs while setting up init target, build process hangs > -- > > Key: HADOOP-1773 > URL: https://issues.apache.org/jira/browse/HADOOP-1773 > Project: Hadoop Common > Issue Type: Bug > Components: build >Affects Versions: 0.15.0 > Environment: Fedora7 on x64, java version "1.6.0_02", latest hadoop > (svn version as of 8/21/07), ant version 1.7.0 (but same problem with 1.6.5) >Reporter: Yuri Pradkin > Fix For: 3.0.0 > > > Ant hangs during a build process, eventually (in hours) dies of heap > exhaustion. > The problematic lines in build.xml seem to be in the init target (fileset > operations): > - > - > - > - > - > - > - > - > - > - > - > - > Commenting them out or deleting allows build to proceed to sucessful > completion. Not being an expert, > in either xml or ant, I'm not sure what exactly I'm missing with those lines > gone, but at least I'm able to compile. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Heads up - 2.0.5-beta
On Thu, Apr 25, 2013 at 6:36 PM, Suresh Srinivas wrote: > Thanks for starting this discussion. I volunteer to do a final review of > protocol changes, so we can avoid incompatible changes to API and wire > protocol post 2.0.5 in Common and HDFS. > > We have been working really hard on the following features. I would like to > get into 2.x and see it reach HDFS users: > # Snapshots > # NFS gateway for HDFS > # HDFS-347 unix domain socket based short circuits > # Windows support Thanks Suresh. It would be great to see Windows support and Snapshots pushed out with 2.0.5 and get picked up by users. -Arpit
Re: Heads up - 2.0.5-beta
Arun, Suresh, Very exciting to hear about this final push to stable Hadoop 2. But I have a problem. Either with the plan or with the version number. I'll be arguing for the number change below rather than the plan. 1. Based on features listed by Suresh it looks that you plan a heavy feature-full release. 2. You are saying you want to complete this within a month (or so). 3. You would like to give it beta quality mark. Not saying it is impossible. But in line with the common saying "You can have fast, good or big: pick two" (a little rephrasing here) I would like to propose to leave some gap between 2.0.4 and the next version so that just in case there was a version to put bug fixes on top of the last release. Do you think we can call the version you proposed to release 2.1.0 or 2.1.0-beta? The proposed new features imho do not exactly conform with the idea of dot-dot release, but definitely qualify for a major number change. I am just trying to avoid rather ugly 2.0.4.1 versions, which of course also possible. Thanks, --Konstantin On Thu, Apr 25, 2013 at 6:36 PM, Suresh Srinivas wrote: > Thanks for starting this discussion. I volunteer to do a final review of > protocol changes, so we can avoid incompatible changes to API and wire > protocol post 2.0.5 in Common and HDFS. > > We have been working really hard on the following features. I would like to > get into 2.x and see it reach HDFS users: > # Snapshots > # NFS gateway for HDFS > # HDFS-347 unix domain socket based short circuits > # Windows support > > Other HDFS folks please let me know if I missed anything. > > To ensure a timely release of 2.0.5-beta, we should not hold back for > individual features. However, I would like to make necessary API and/or > protocol changes right-away. This will allow us to adding features in > subsequent releases e.g. hadoop-2.2 or hadoop-2.3 etc without breaking > compatibility. For e.g. even if we don't complete NFS support, making > FileID related changes in 2.0.5-beta will ensure future compatbility. > > Regards, > Suresh > > > > On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy > wrote: > > > Gang, > > > > With hadoop-2.0.4-alpha released, I'd like 2.0.4 to be the final of our > > hadoop-2.x alphas. We have made lots of progress on hadoop-2.x and I > > believe we are nearly there, exciting times! > > > > As we have discussed previously, I hope to do a final push to stabilize > > hadoop-2.x, release a hadoop-2.0.5-beta in the next month or so; and then > > declare hadoop-2.1 as stable this summer after a short period of > intensive > > testing. > > > > With that in mind, I really want to make a serious push to lock down > APIs > > and wire-protocols for hadoop-2.0.5-beta. Thus, we can confidently > support > > hadoop-2.x in a compatible manner in the future. So, it's fine to add new > > features, but please ensure that all APIs are frozen for > hadoop-2.0.5-beta > > > > Vinod is helping out on the YARN/MR side and has tagged a number of > final > > changes (including some the final API incompatibilities) we'd like to > push > > in before we call hadoop-2.x as ready to be supported (Target Version set > > to 2.0.5-beta): > > http://s.apache.org/target-hadoop-2.0.5-beta > > Thanks Vinod! (Note some of the sub-tasks of umbrella jiras may not be > > tagged, but their necessity is implied). > > > > Similarly on HDFS side, can someone please help out by tagging features, > > bug-fixes, protocol/API changes etc.? This way we can ensure HDFS APIs & > > protocols are locked down too - I'd really appreciate it! > > > > thanks, > > Arun > > > > > > -- > > Arun C. Murthy > > Hortonworks Inc. > > http://hortonworks.com/ > > > > > > > > > -- > http://hortonworks.com/download/ >
Re: Heads up - 2.0.5-beta
Arun, Could you please define the release plan and put it into vote. In accordance with the ByLaws. After this discussion of course. http://hadoop.apache.org/bylaws.html Release Plan Defines the timetable and actions for a release. The plan also nominates a Release Manager. Lazy majority of active committers Do I understand correctly you volunteering for RM? Just to clarify. Suresh had already put a list of features for HDFS and common. So you probably need to indicate features for MapReduce and Yarn. Thanks, --Konstantin On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy wrote: > Gang, > > With hadoop-2.0.4-alpha released, I'd like 2.0.4 to be the final of our > hadoop-2.x alphas. We have made lots of progress on hadoop-2.x and I > believe we are nearly there, exciting times! > > As we have discussed previously, I hope to do a final push to stabilize > hadoop-2.x, release a hadoop-2.0.5-beta in the next month or so; and then > declare hadoop-2.1 as stable this summer after a short period of intensive > testing. > > With that in mind, I really want to make a serious push to lock down APIs > and wire-protocols for hadoop-2.0.5-beta. Thus, we can confidently support > hadoop-2.x in a compatible manner in the future. So, it's fine to add new > features, but please ensure that all APIs are frozen for hadoop-2.0.5-beta > > Vinod is helping out on the YARN/MR side and has tagged a number of final > changes (including some the final API incompatibilities) we'd like to push > in before we call hadoop-2.x as ready to be supported (Target Version set > to 2.0.5-beta): > http://s.apache.org/target-hadoop-2.0.5-beta > Thanks Vinod! (Note some of the sub-tasks of umbrella jiras may not be > tagged, but their necessity is implied). > > Similarly on HDFS side, can someone please help out by tagging features, > bug-fixes, protocol/API changes etc.? This way we can ensure HDFS APIs & > protocols are locked down too - I'd really appreciate it! > > thanks, > Arun > > > -- > Arun C. Murthy > Hortonworks Inc. > http://hortonworks.com/ > > >
Re: Heads up - 2.0.5-beta
On Fri, Apr 26, 2013 at 2:42 PM, Suresh Srinivas wrote: > Eli, I will post a more detailed reply soon. But one small correction: > > > I'm also not sure there's currently consensus on what an incompatible >> change is. For example, I think HADOOP-9151 is incompatible because it >> broke client/server wire compatibility with previous releases and any >> change that breaks wire compatibility is incompatible. Suresh felt it >> was not an incompatible change because it did not affect API >> compatibility (ie PB is not considered part of the API) and the change >> occurred while v2 is in alpha. >> > > This is not correct. I did not say it was not an incompatible change. > It was indeed an incompatible wire protocol change. My argument was, > the phase of development we were in, we could not mark wire protocol > as stable and not make any incompatible change. But once 2.0.5-beta > is out, as had discussed earlier, we should not make further incompatible > changes to wire protocol. Sorry for the confusion, I misinterpreted your comments on the jira (specifically, "This is an incompatible change: I disagree." and "see my argument that about why this is not incompatible.") to indicate that you thought it was not incompatible. > > -- > http://hortonworks.com/download/
Re: Heads up - 2.0.5-beta
Eli, I will post a more detailed reply soon. But one small correction: I'm also not sure there's currently consensus on what an incompatible > change is. For example, I think HADOOP-9151 is incompatible because it > broke client/server wire compatibility with previous releases and any > change that breaks wire compatibility is incompatible. Suresh felt it > was not an incompatible change because it did not affect API > compatibility (ie PB is not considered part of the API) and the change > occurred while v2 is in alpha. > This is not correct. I did not say it was not an incompatible change. It was indeed an incompatible wire protocol change. My argument was, the phase of development we were in, we could not mark wire protocol as stable and not make any incompatible change. But once 2.0.5-beta is out, as had discussed earlier, we should not make further incompatible changes to wire protocol. -- http://hortonworks.com/download/
Re: Heads up - 2.0.5-beta
If protocol compatibility of v2 and v3 is a goal, HADOOP-8990 should be a blocker for v2. __Luke On Fri, Apr 26, 2013 at 12:07 PM, Eli Collins wrote: > On Fri, Apr 26, 2013 at 11:15 AM, Arun C Murthy > wrote: > > > > On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik wrote: > > > >> On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy > wrote: > >> > >>> With that in mind, I really want to make a serious push to lock down > APIs and wire-protocols for hadoop-2.0.5-beta. > >>> Thus, we can confidently support hadoop-2.x in a compatible manner in > the future. So, it's fine to add new features, > >>> but please ensure that all APIs are frozen for hadoop-2.0.5-beta > >> > >> Arun, since it sounds like you have a pretty definite idea > >> in mind for what you want 'beta' label to actually mean, > >> could you, please, share the exact criteria? > > > > Sorry, I'm not sure if this is exactly what you are looking for but, as > I mentioned above, the primary aim would be make the final set of required > API/write-protocol changes so that we can call it a 'beta' i.e. once > 2.0.5-beta ships users & downstream projects can be confident about forward > compatibility in hadoop-2.x line. Obviously, we might discover a blocker > bug post 2.0.5 which *might* necessitate an unfortunate change - but that > should be an outstanding exception. > > Arun, Suresh, > > Mind reviewing the following page Karthik put together on > compatibility? http://wiki.apache.org/hadoop/Compatibility > > I think we should do something similar to what Sanjay proposed in > HADOOP-5071 for Hadoop v2. If we get on the same page on > compatibility terms/APIs then we can quickly draft the policy, at > least for the things we've already got consensus on. I think our new > developers, users, downstream projects, and partners would really > appreciate us making this clear. If people like the content we can > move it to the Hadoop website and maintain it in svn like the bylaws. > > The reason I think we need to do so is because there's been confusion > about what types of compatibility we promise and some open questions > which I'm not sure everyone is clear on. Examples: > - Are we going to preserve Hadoop v3 clients against v2 servers now > that we have protobuf support? (I think so..) > - Can we break rolling upgrade of daemons in updates post GA? (I don't > think so..) > - Do we disallow HDFS metadata changes that require an HDFS upgrade in > an update? (I think so..) > - Can we remove methods from v2 and v2 updates that were deprecated in > v0.20-22? (Unclear) > - Will we preserve binary compatibility for MR2 going forward? (I think > so..) > - Does the ability to support multiple versions of MR simultaneously > via MR2 change the MR API compatibility story? (I don't think so..) > - Are the RM protocols sufficiently stable to disallow incompatible > changes potentially required by non-MR projects? (Unclear, most large > Yarn deployments I'm aware of are running 0.23, not v2 alphas) > > I'm also not sure there's currently consensus on what an incompatible > change is. For example, I think HADOOP-9151 is incompatible because it > broke client/server wire compatibility with previous releases and any > change that breaks wire compatibility is incompatible. Suresh felt it > was not an incompatible change because it did not affect API > compatibility (ie PB is not considered part of the API) and the change > occurred while v2 is in alpha. Not sure we need to go through the > whole exercise of what's allowed in an alpha and beta (water under the > bridge, hopefully), but I do think we should clearly define an > incompatible change. It's fine that v2 has been a bit wild wild west > in the alpha development stage but I think we need to get a little > more rigorous. > > Thanks, > Eli >
Re: Heads up - 2.0.5-beta
On Fri, Apr 26, 2013 at 11:15 AM, Arun C Murthy wrote: > > On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik wrote: > >> On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy wrote: >> >>> With that in mind, I really want to make a serious push to lock down APIs >>> and wire-protocols for hadoop-2.0.5-beta. >>> Thus, we can confidently support hadoop-2.x in a compatible manner in the >>> future. So, it's fine to add new features, >>> but please ensure that all APIs are frozen for hadoop-2.0.5-beta >> >> Arun, since it sounds like you have a pretty definite idea >> in mind for what you want 'beta' label to actually mean, >> could you, please, share the exact criteria? > > Sorry, I'm not sure if this is exactly what you are looking for but, as I > mentioned above, the primary aim would be make the final set of required > API/write-protocol changes so that we can call it a 'beta' i.e. once > 2.0.5-beta ships users & downstream projects can be confident about forward > compatibility in hadoop-2.x line. Obviously, we might discover a blocker bug > post 2.0.5 which *might* necessitate an unfortunate change - but that should > be an outstanding exception. Arun, Suresh, Mind reviewing the following page Karthik put together on compatibility? http://wiki.apache.org/hadoop/Compatibility I think we should do something similar to what Sanjay proposed in HADOOP-5071 for Hadoop v2. If we get on the same page on compatibility terms/APIs then we can quickly draft the policy, at least for the things we've already got consensus on. I think our new developers, users, downstream projects, and partners would really appreciate us making this clear. If people like the content we can move it to the Hadoop website and maintain it in svn like the bylaws. The reason I think we need to do so is because there's been confusion about what types of compatibility we promise and some open questions which I'm not sure everyone is clear on. Examples: - Are we going to preserve Hadoop v3 clients against v2 servers now that we have protobuf support? (I think so..) - Can we break rolling upgrade of daemons in updates post GA? (I don't think so..) - Do we disallow HDFS metadata changes that require an HDFS upgrade in an update? (I think so..) - Can we remove methods from v2 and v2 updates that were deprecated in v0.20-22? (Unclear) - Will we preserve binary compatibility for MR2 going forward? (I think so..) - Does the ability to support multiple versions of MR simultaneously via MR2 change the MR API compatibility story? (I don't think so..) - Are the RM protocols sufficiently stable to disallow incompatible changes potentially required by non-MR projects? (Unclear, most large Yarn deployments I'm aware of are running 0.23, not v2 alphas) I'm also not sure there's currently consensus on what an incompatible change is. For example, I think HADOOP-9151 is incompatible because it broke client/server wire compatibility with previous releases and any change that breaks wire compatibility is incompatible. Suresh felt it was not an incompatible change because it did not affect API compatibility (ie PB is not considered part of the API) and the change occurred while v2 is in alpha. Not sure we need to go through the whole exercise of what's allowed in an alpha and beta (water under the bridge, hopefully), but I do think we should clearly define an incompatible change. It's fine that v2 has been a bit wild wild west in the alpha development stage but I think we need to get a little more rigorous. Thanks, Eli
Re: Use hadoop.relaxed.worker.version.check to allow versions in the same major version?
Thanks for the pointers, Eli. Glad HDFS already takes care of this - I ll follow up on the MR JIRA. On Fri, Apr 26, 2013 at 11:09 AM, Eli Collins wrote: > Hey Karthik, > > We already support this for HDFS, see HDFS-2983 (Relax the build > version check to permit rolling upgrades within a release). > We should do so for Yarn as well, MAPREDUCE-4150 tracks this. I don't > think Ahmed is working on it so would be great for you or someone else > to take it. > > Thanks, > Eli > > On Fri, Apr 26, 2013 at 10:21 AM, Karthik Kambatla > wrote: > > Hi devs, > > > > Given that we have API compatibility within a major release, I was > > wondering if it would make sense to augment the worker version check to > > allow workers from a different minor/point release in the same major > > release? The motivation for this is rolling upgrade within a major > release. > > > > We could use the same property hadoop.relaxed.worker.version.check for > this > > or add another property. Thoughts? > > > > Thanks > > Karthik >
Re: Heads up - 2.0.5-beta
On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik wrote: > On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy wrote: > >> With that in mind, I really want to make a serious push to lock down APIs >> and wire-protocols for hadoop-2.0.5-beta. >> Thus, we can confidently support hadoop-2.x in a compatible manner in the >> future. So, it's fine to add new features, >> but please ensure that all APIs are frozen for hadoop-2.0.5-beta > > Arun, since it sounds like you have a pretty definite idea > in mind for what you want 'beta' label to actually mean, > could you, please, share the exact criteria? Sorry, I'm not sure if this is exactly what you are looking for but, as I mentioned above, the primary aim would be make the final set of required API/write-protocol changes so that we can call it a 'beta' i.e. once 2.0.5-beta ships users & downstream projects can be confident about forward compatibility in hadoop-2.x line. Obviously, we might discover a blocker bug post 2.0.5 which *might* necessitate an unfortunate change - but that should be an outstanding exception. Hope that helps. thanks, Arun
Re: Use hadoop.relaxed.worker.version.check to allow versions in the same major version?
Hey Karthik, We already support this for HDFS, see HDFS-2983 (Relax the build version check to permit rolling upgrades within a release). We should do so for Yarn as well, MAPREDUCE-4150 tracks this. I don't think Ahmed is working on it so would be great for you or someone else to take it. Thanks, Eli On Fri, Apr 26, 2013 at 10:21 AM, Karthik Kambatla wrote: > Hi devs, > > Given that we have API compatibility within a major release, I was > wondering if it would make sense to augment the worker version check to > allow workers from a different minor/point release in the same major > release? The motivation for this is rolling upgrade within a major release. > > We could use the same property hadoop.relaxed.worker.version.check for this > or add another property. Thoughts? > > Thanks > Karthik
Re: Heads up - 2.0.5-beta
On Apr 25, 2013, at 6:36 PM, Suresh Srinivas wrote: >> On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy wrote: >> >> Similarly on HDFS side, can someone please help out by tagging features, >> bug-fixes, protocol/API changes etc.? This way we can ensure HDFS APIs & >> protocols are locked down too - I'd really appreciate it! > > To ensure a timely release of 2.0.5-beta, we should not hold back for > individual features. However, I would like to make necessary API and/or > protocol changes right-away. This will allow us to adding features in > subsequent releases e.g. hadoop-2.2 or hadoop-2.3 etc without breaking > compatibility. +1, sounds like a good plan. Thanks! Arun
[jira] [Created] (HADOOP-9516) Enable spnego filters only if kerberos is enabled
Daryn Sharp created HADOOP-9516: --- Summary: Enable spnego filters only if kerberos is enabled Key: HADOOP-9516 URL: https://issues.apache.org/jira/browse/HADOOP-9516 Project: Hadoop Common Issue Type: Sub-task Components: security Affects Versions: 2.0.0-alpha, 3.0.0 Reporter: Daryn Sharp Assignee: Daryn Sharp Spnego filters are currently enabled if security is enabled - which is predicated on security=kerberos. With the advent of the PLAIN authentication method, the filters should only be enabled if kerberos is enabled. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Use hadoop.relaxed.worker.version.check to allow versions in the same major version?
Hi devs, Given that we have API compatibility within a major release, I was wondering if it would make sense to augment the worker version check to allow workers from a different minor/point release in the same major release? The motivation for this is rolling upgrade within a major release. We could use the same property hadoop.relaxed.worker.version.check for this or add another property. Thoughts? Thanks Karthik
[jira] [Created] (HADOOP-9515) Add general interface for NFS and Mount
Brandon Li created HADOOP-9515: -- Summary: Add general interface for NFS and Mount Key: HADOOP-9515 URL: https://issues.apache.org/jira/browse/HADOOP-9515 Project: Hadoop Common Issue Type: New Feature Reporter: Brandon Li Assignee: Brandon Li These is the general interface implementation for NFS and Mount protocol, e.g., some protocol related data structures and etc. It doesn't include the file system specific implementations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HADOOP-9507) LocalFileSystem rename() is broken in some cases when destination exists
[
https://issues.apache.org/jira/browse/HADOOP-9507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mostafa Elhemali reopened HADOOP-9507:
--
Assignee: Daryn Sharp (was: Mostafa Elhemali)
Daryn: please read the repro and result more carefully. mv /foo /bar doesn't
produce /foo/bar, it produces /bar with wrong contents in there (/X/X instead
of /X); at least on Windows in branch-1-win (again, haven't tried in other
branches).
> LocalFileSystem rename() is broken in some cases when destination exists
>
>
> Key: HADOOP-9507
> URL: https://issues.apache.org/jira/browse/HADOOP-9507
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Reporter: Mostafa Elhemali
>Assignee: Daryn Sharp
>Priority: Minor
> Attachments: HADOOP-9507.branch-1-win.patch
>
>
> The rename() method in RawLocalFileSystem uses FileUtil.copy() without
> realizing that FileUtil.copy() has a special behavior that if you're copying
> /foo to /bar and /bar exists and is a directory, it'll copy /foo inside /bar
> instead of overwriting it, which is not what rename() wants. So you end up
> with weird behaviors like in this repro:
> {code}
> c:
> cd \
> md Foo
> md Bar
> md Foo\X
> md Bar\X
> hadoop fs -mv file:///c:/Foo file:///c:/Bar
> {code}
> At the end of this, you would expect to find only Bar\X, but you instead find
> Bar\X\X.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9507) LocalFileSystem rename() is broken in some cases when destination exists
[
https://issues.apache.org/jira/browse/HADOOP-9507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp resolved HADOOP-9507.
-
Resolution: Invalid
> LocalFileSystem rename() is broken in some cases when destination exists
>
>
> Key: HADOOP-9507
> URL: https://issues.apache.org/jira/browse/HADOOP-9507
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
>Reporter: Mostafa Elhemali
>Assignee: Mostafa Elhemali
>Priority: Minor
> Attachments: HADOOP-9507.branch-1-win.patch
>
>
> The rename() method in RawLocalFileSystem uses FileUtil.copy() without
> realizing that FileUtil.copy() has a special behavior that if you're copying
> /foo to /bar and /bar exists and is a directory, it'll copy /foo inside /bar
> instead of overwriting it, which is not what rename() wants. So you end up
> with weird behaviors like in this repro:
> {code}
> c:
> cd \
> md Foo
> md Bar
> md Foo\X
> md Bar\X
> hadoop fs -mv file:///c:/Foo file:///c:/Bar
> {code}
> At the end of this, you would expect to find only Bar\X, but you instead find
> Bar\X\X.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9514) Hadoop CLI's have inconsistent usages
Thomas Graves created HADOOP-9514: - Summary: Hadoop CLI's have inconsistent usages Key: HADOOP-9514 URL: https://issues.apache.org/jira/browse/HADOOP-9514 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 2.0.4-alpha, 3.0.0 Reporter: Thomas Graves Many of the hadoop command line interfaces (yarn/mapred/hdfs/hadoop) and subcommands have inconsistent usages, in many cases have options that don't apply (-archives/-files/-jt), and due to the usage of GenericOptionsParser print the usage as "bin/hadoop command [genericOptions] [commandOptions]" even though you were running yarn or hdfs commands. This makes for a bad user experience and its confusing as to what options are really available and how to use them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9513) Wrong file permissions in hadoop-1.1.2-1.x86_64.rpm package
Javi Roman created HADOOP-9513: -- Summary: Wrong file permissions in hadoop-1.1.2-1.x86_64.rpm package Key: HADOOP-9513 URL: https://issues.apache.org/jira/browse/HADOOP-9513 Project: Hadoop Common Issue Type: Improvement Affects Versions: 1.1.2 Environment: Red Hat 6.x / CentOS 6.x Reporter: Javi Roman Priority: Trivial Missing execution bit in shell scripts: /usr/sbin/start-*.sh /usr/sbin/stop-*.sh /usr/sbin/slaves.sh /usr/sbin/update-hadoop-env.sh $ ls -l /usr/sbin/start-all.sh -rw-r--r-- 1 root root 1166 Jan 31 03:08 /usr/sbin/start-all.sh -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
