On Thu, 3 Jun 2021 at 21:42, /dev /local/ca <devloca...@gmail.com> wrote:
> Greetings: > > Are any Hadoop developers working/running on Windows 10 Pro x64 (AMD)? > > If so, how did you get Hadoop to work/run? > > --- > I am told that to run on Windows 10 Pro x64, that I need to set: > 1. HADOOP_HOME ( I set to C:\hadoop) > 2. create %HADOOP_HOME%/bin > 3. and put two files in there (hadoop.dll, winutils.exe) > > and to put %HADOOP_HOME%/bin in my path. > > I searched and found hadoop.dll and winutils.exe here: > https://github.com/cdarlint/winutils > > but when I tried using 3.2.1 , I get (when running) > > java.io.IOException: Cannot run program "C:\hadoop\bin\winutils.exe": > CreateProcess error=216, This version of %1 is not compatible with the > version of Windows you're running. Check your computer's system information > and then contact the software publisher > Are you sure you have 64 bit windows? https://thegeekpage.com/fix-this-version-of-the-file-is-not-compatible-with-the-version-of-windows-youre-running-problem-in-windows-10/ > > --- > I tried following the instructions for compiling hadoop.dll and > winutils.exe, but it is simply not humanly possible. If so, try it, follow > the instructions and you will see. > > --- > I want to ask if anyone has, or knows where i can get ; hadoop.dll and > winutils.exe or can please help me compile. > I have Windows 10 Pro with cygwin installed. I do not want to install > anything else beyond this as it super craps out my machine when I install > any developer tool or IDE from Microsoft. > > -- > I needed to get Hadoop on Windows up and running a month ago, I am way > behind and the project is in jeopardy. > So Jesus please, if anyone is doing this (compiling), please let me know. > > -- > Also why can't there be a /binaries/win/3.2.1/* directory somewhere, these > can be downloaded. Many thousands of people, beyond myself, are having > problems with this each and every day. > > Every place I go, Windows 10 Pro is on the desk for development use. I do > not see that developers are giving Linux boxes for development. yes we > deploy to Linux, Cloud, but IDE's run on Windows - few use Macs here and > there. I'm not making this up, look at the numbers. > I put a fair amount of effort in a few years back to at least ensure that when PATH wasn't set up right the error message would be different ( https://issues.apache.org/jira/browse/HADOOP-10775). And as you are aware, initially I and then later someone else went to the effort of building windows binaries. But: I don't have that VM or the MSDN license to recreate it. > > If you are surrounded by people that run macs and Linux as their primary > development machine, you are in a bubble, We are the people who get to field support calls related to hadoop clusters of hundreds of nodes playing up in obscure ways. Having laptops/desktops which match what gets used there helps. Is that a bubble? Probably. But having started coding for Windows in the time of Windows 2.04 I feel your pain. And I have no regrets switching to a linux desktop when I went full time into server-side coding because it eliminated so much of the subtle differences between your development box and production systems. Java hides a lot of the differences, as can Python, but as soon as you go near: filenames, processes, networking, scheduling, making sense of error messages, even installing the exact same set of binaries -having your desktop be "a unix", and ideally the same linux distro as you are deploying too- makes your life a lot easier. > it's predominantly Windows out there in the dev world. > Please accept my condolences. Except in the special cases "I am doing windows client apps", "I have code for MSEdge/IE" and "I enjoy an IT department managing an OS which they reboot on a random schedule", moving off it is better (OK, I'm omitting "I like a laptop whose power management works" and "I like audio to work, including things like zoom", both of which preclude linux) Have you tried Windows Subsystem for Linux? Given its a linux container deployed by the WinNT microkerrnel, there's no fundamental reason why it wouldn't work https://docs.microsoft.com/en-us/windows/wsl/install-win10 We're actually in the week of qualifying the Hadoop 3.3.1 release. If you can see if it works -and if not, help identify why not- that would be good
