Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
-1, +1, -1 Python has fairly inconsistent support across all major OS vendors. It is hard to get it right unless the scripts are all designed to make use of Python 2.4. However, Python 2.4 doesn't have necessary OS features to make Python useful in runtime or build environment unless you write a lot of custom modules. Which defeats the purpose to use python as intermediate layer to do OS dependent work. Jruby may be a better choice. regards, Eric On Sat, Dec 1, 2012 at 12:28 PM, Joep Rottinghuis wrote: > 0, 0, -1 (non-binding) > > Joep > > On Nov 24, 2012, at 12:13 PM, Matt Foley wrote: > > > For discussion, please see previous thread "[PROPOSAL] introduce Python > as > > build-time and run-time dependency for Hadoop and throughout Hadoop > stack". > > > > This vote consists of three separate items: > > > > 1. Contributors shall be allowed to use Python as a platform-independent > > scripting language for build-time tasks, and add Python as a build-time > > dependency. > > Please vote +1, 0, -1. > > > > 2. Contributors shall be encouraged to use Maven tasks in combination > with > > either plug-ins or Groovy scripts to do cross-platform build-time tasks, > > even under ant in Hadoop-1. > > Please vote +1, 0, -1. > > > > 3. Contributors shall be allowed to use Python as a platform-independent > > scripting language for run-time tasks, and add Python as a run-time > > dependency. > > Please vote +1, 0, -1. > > > > Note that voting -1 on #1 and +1 on #2 essentially REQUIRES contributors > to > > use Maven plug-ins or Groovy as the only means of cross-platform > build-time > > tasks, or to simply continue using platform-dependent scripts as is being > > done today. > > > > Vote closes at 12:30pm PST on Saturday 1 December. > > - > > Personally, my vote is +1, +1, +1. > > I think #2 is preferable to #1, but still has many unknowns in it, and > > until those are worked out I don't want to delay moving to cross-platform > > scripts for build-time tasks. > > > > Best regards, > > --Matt >
[jira] [Resolved] (HADOOP-8301) Common (hadoop-tools) side of MAPREDUCE-4172
[ https://issues.apache.org/jira/browse/HADOOP-8301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harsh J resolved HADOOP-8301. - Resolution: Won't Fix Patches were too broad and have gone stale. Will address these forms of issue over separate, smaller and more divided JIRAs in future. Closing out parent JIRA MAPREDUCE-4172, and hence closing out this. > Common (hadoop-tools) side of MAPREDUCE-4172 > > > Key: HADOOP-8301 > URL: https://issues.apache.org/jira/browse/HADOOP-8301 > Project: Hadoop Common > Issue Type: Task > Components: build >Affects Versions: 3.0.0 >Reporter: Harsh J >Assignee: Harsh J > > Patches on MAPREDUCE-4172 (for MR-relevant projects) that requires to run off > of Hadoop Common project for Hadoop QA. > One sub-task per hadoop-tools submodule will be added here for reviews. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9110) winutils ls off-by-one error indexing MONTHS array can cause access violation
[ https://issues.apache.org/jira/browse/HADOOP-9110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas resolved HADOOP-9110. - Resolution: Fixed Fix Version/s: trunk-win 1-win Hadoop Flags: Reviewed Patch committed to both branch-1-win and branch-trunk-win. Thank you Chris. > winutils ls off-by-one error indexing MONTHS array can cause access violation > - > > Key: HADOOP-9110 > URL: https://issues.apache.org/jira/browse/HADOOP-9110 > Project: Hadoop Common > Issue Type: Bug > Components: util >Affects Versions: 1-win, trunk-win >Reporter: Chris Nauroth >Assignee: Chris Nauroth > Fix For: 1-win, trunk-win > > Attachments: HADOOP-9110-branch-1-win.1.patch, > HADOOP-9110-branch-trunk-win.1.patch > > > In ls.c, the LsPrintLine function uses the wMonth field of a SYSTEMTIME > struct to index into MONTHS, an array of 12 elements containing string > representations of the months. The wMonth field is 1-based (1=January), but > indexing into an array is zero-based. This gives the wrong month, and since > we just crossed into month 12, it also has started causing an access > violation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
0, 0, -1 (non-binding) Joep On Nov 24, 2012, at 12:13 PM, Matt Foley wrote: > For discussion, please see previous thread "[PROPOSAL] introduce Python as > build-time and run-time dependency for Hadoop and throughout Hadoop stack". > > This vote consists of three separate items: > > 1. Contributors shall be allowed to use Python as a platform-independent > scripting language for build-time tasks, and add Python as a build-time > dependency. > Please vote +1, 0, -1. > > 2. Contributors shall be encouraged to use Maven tasks in combination with > either plug-ins or Groovy scripts to do cross-platform build-time tasks, > even under ant in Hadoop-1. > Please vote +1, 0, -1. > > 3. Contributors shall be allowed to use Python as a platform-independent > scripting language for run-time tasks, and add Python as a run-time > dependency. > Please vote +1, 0, -1. > > Note that voting -1 on #1 and +1 on #2 essentially REQUIRES contributors to > use Maven plug-ins or Groovy as the only means of cross-platform build-time > tasks, or to simply continue using platform-dependent scripts as is being > done today. > > Vote closes at 12:30pm PST on Saturday 1 December. > - > Personally, my vote is +1, +1, +1. > I think #2 is preferable to #1, but still has many unknowns in it, and > until those are worked out I don't want to delay moving to cross-platform > scripts for build-time tasks. > > Best regards, > --Matt
Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
On Sat, Dec 1, 2012 at 2:44 AM, Steve Loughran wrote: > WinNT Bat/CMD files are the worst possible scripting language invented. At > the very least, .py should be the language of choice there The scripts should not have so much logic that .bat files are a problem. Doug
Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
On 30 November 2012 13:40, Radim Kolar wrote: > > inline ant scripts >>> >>> =0. Ant's versioning is stricter; you can pull down the exact Jar >>> versions, >>> and some of us in the Ant team worked very hard to get it going >>> everywhere. >>> You don't gain anything by going to .py >>> >> there are sh scripts inside maven ant plugin stuff > Which is because there are some things you can't do in Java -run rpmbuild to pick up file permissions and hanging symlinks that only become valid on deployment. The reason Ant is used to start them is Maven views trying to run native scripts as a forbidden action - probably popping up some patronising text "you are trying to run a shell script, please look at maven.apache.org/wiki/whymavenwontletyoudothings/ to understand this; they also view building RPMs as not something to encourage either. (but we digress into an ant vs maven argument. I do actually appreciate the consistent target naming across projects and the ability for the IDE to set up structure, it's just the entire underlying architecture and implementation that I dislike)
Re: [VOTE] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
On 1 December 2012 01:08, Eli Collins wrote: > -1, 0, -1 > > IIUC the only platform we plan to add support for that we can't easily > support today (w/o an emulation layer like cygwin) is Windows, and it > seems like making the bash scripts simpler and having parallel bat > files is IMO a better approach. > > WinNT Bat/CMD files are the worst possible scripting language invented. At the very least, .py should be the language of choice there
