[jira] [Resolved] (HADOOP-10358) libhadoop doesn't compile on Mac OS X
[
https://issues.apache.org/jira/browse/HADOOP-10358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Maykov resolved HADOOP-10358.
--
Resolution: Duplicate
Resolving dupe
> libhadoop doesn't compile on Mac OS X
> -
>
> Key: HADOOP-10358
> URL: https://issues.apache.org/jira/browse/HADOOP-10358
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Environment: Mac OS X 10.8.5
> Oracle JDK 1.7.0_51
>Reporter: Ilya Maykov
>Priority: Minor
> Attachments: HADOOP-10358-fix-hadoop-common-native-on-os-x.patch
>
>
> The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
> on mac) fails to compile on Mac OS X. The problem is in
> hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
> at lines 76-78:
> [exec]
> /Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
> error: invalid operands to binary expression ('void' and 'int')
> [exec] if(setnetgrent(cgroup) == 1) {
> [exec] ~~~ ^ ~
> There are two problems in the code:
> 1) The #ifndef guard only checks for __FreeBSD__ but should check for either
> one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its
> syscalls from FreeBSD rather than Linux, and thus the setnetgrent() syscall
> returns void.
> 2) setnetgrentCalledFlag = 1 is set outside the #ifndef guard, but the
> syscall is only invoked inside the guard. This means that on FreeBSD,
> endnetgrent() can be called in the cleanup code without a corresponding
> setnetgrent() invocation.
> I have a patch that fixes both issues (will attach in a bit). With this
> patch, I'm able to compile libhadoop.dylib on Mac OS X, which in turn lets me
> install native snappy, lzo, etc compressor libraries on my client. That lets
> me run commands like 'hadoop fs -text somefile.lzo' from the macbook rather
> than having to ssh to a linux box, etc.
> Note that this patch only fixes the native build of hadoop-common-project.
> Some other components of hadoop still fail to build their native components,
> but libhadoop.dylib is enough for the client.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HADOOP-10359) Native bzip2 compression support is broken on non-Linux systems
[
https://issues.apache.org/jira/browse/HADOOP-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13948299#comment-13948299
]
Ilya Maykov commented on HADOOP-10359:
--
[~hadoopqa] As far as I know, the build bot doesn't currently build native
libraries on Mac OS X (if it builds anything on Mac OS X), so there are no
automated tests that I could add to test this change.
Manual testing performed:
- built libhadoop.dylib on Mac OS X (requires this patch and the patch for
HADOOP-9648)
- run "hadoop checknative" to verify that bzip2 compression is using the native
code:
{{$ hadoop checknative
14/03/26 11:46:41 INFO bzip2.Bzip2Factory: Successfully loaded & initialized
native-bzip2 library system-native
14/03/26 11:46:41 INFO zlib.ZlibFactory: Successfully loaded & initialized
native-zlib library
14/03/26 11:46:41 INFO Configuration.deprecation: hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
Native library checking:
hadoop: true
/Users/ilyam/repos/vendor/hadoop_distros/hadoop-2.2.0-cdh5.0.0-beta-2/lib/native/libhadoop.1.0.0.dylib
zlib: true /usr/lib/libz.1.dylib
snappy: true /usr/local/lib/libsnappy.1.dylib
lz4:true revision:99
bzip2: true /usr/lib/libbz2.1.0.dylib}}
> Native bzip2 compression support is broken on non-Linux systems
> ---
>
> Key: HADOOP-10359
> URL: https://issues.apache.org/jira/browse/HADOOP-10359
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
>Affects Versions: 2.2.0
> Environment: Mac OS X 10.8.5
> Oracle JDK 1.7.0_51
> Hadoop 2.2.0-CDH5.0.0-beta-2
>Reporter: Ilya Maykov
>Priority: Minor
> Attachments: HADOOP-10359-native-bzip2-for-os-x.patch
>
>
> While testing the patch for HADOOP-9648, I noticed that the bzip2 native
> compressor/decompressor support wasn't working properly. I dug around a bit
> and got native bzip2 support to work on my macbook. Will attach a patch in a
> bit. (This probably needs to be tested on FreeBSD / Windows / Linux, but I
> don't have the time to set up the necessary VMs to do it. I assume the build
> bot will test Linux).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HADOOP-10359) Native bzip2 compression support is broken on non-Linux systems
[ https://issues.apache.org/jira/browse/HADOOP-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Maykov updated HADOOP-10359: - Status: Patch Available (was: Open) Attaching the patch with "submit patch" option, hopefully that changes status to "Patch Available" ... > Native bzip2 compression support is broken on non-Linux systems > --- > > Key: HADOOP-10359 > URL: https://issues.apache.org/jira/browse/HADOOP-10359 > Project: Hadoop Common > Issue Type: Improvement > Components: native >Affects Versions: 2.2.0 > Environment: Mac OS X 10.8.5 > Oracle JDK 1.7.0_51 > Hadoop 2.2.0-CDH5.0.0-beta-2 >Reporter: Ilya Maykov >Priority: Minor > Attachments: HADOOP-10359-native-bzip2-for-os-x.patch > > > While testing the patch for HADOOP-9648, I noticed that the bzip2 native > compressor/decompressor support wasn't working properly. I dug around a bit > and got native bzip2 support to work on my macbook. Will attach a patch in a > bit. (This probably needs to be tested on FreeBSD / Windows / Linux, but I > don't have the time to set up the necessary VMs to do it. I assume the build > bot will test Linux). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HADOOP-9648) Fix build native library on mac osx
[ https://issues.apache.org/jira/browse/HADOOP-9648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909479#comment-13909479 ] Ilya Maykov commented on HADOOP-9648: - If committers are not fully comfortable with making the proposed changes to hadoop-yarn-server, then maybe this could be broken up into 2 issues? 1) The hadoop-common-project changes. Fixing that would allow libhadoop.dylib to build on Mac OS X and fix the client (I duplicated this part of the issue unknowingly in HADOOP-10358). 2) The hadoop-yarn-project changes. This might be seen as the more risky part of this patch, and maybe less useful since I suspect most people interested in getting native code working on Macs run hadoop on Linux clusters but interact with them from Mac clients (I could be wrong of course). > Fix build native library on mac osx > --- > > Key: HADOOP-9648 > URL: https://issues.apache.org/jira/browse/HADOOP-9648 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.0.4, 1.2.0, 1.1.2, 2.0.5-alpha >Reporter: Kirill A. Korinskiy >Assignee: Binglin Chang > Attachments: HADOOP-9648-native-osx.1.0.4.patch, > HADOOP-9648-native-osx.1.1.2.patch, HADOOP-9648-native-osx.1.2.0.patch, > HADOOP-9648-native-osx.2.0.5-alpha-rc1.patch, HADOOP-9648.v2.patch > > > Some patches for fixing build a hadoop native library on os x 10.7/10.8. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HADOOP-9648) Fix build native library on mac osx
[ https://issues.apache.org/jira/browse/HADOOP-9648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909473#comment-13909473 ] Ilya Maykov commented on HADOOP-9648: - This patch works for me, but I've only tested the client. Not sure if anyone actually plans to run YARN containers on a cluster of Macs (or even if that's possible to do, even with this patch) so the hadoop-yarn-project changes are harder to test in a real environment ... > Fix build native library on mac osx > --- > > Key: HADOOP-9648 > URL: https://issues.apache.org/jira/browse/HADOOP-9648 > Project: Hadoop Common > Issue Type: Bug >Affects Versions: 1.0.4, 1.2.0, 1.1.2, 2.0.5-alpha >Reporter: Kirill A. Korinskiy >Assignee: Binglin Chang > Attachments: HADOOP-9648-native-osx.1.0.4.patch, > HADOOP-9648-native-osx.1.1.2.patch, HADOOP-9648-native-osx.1.2.0.patch, > HADOOP-9648-native-osx.2.0.5-alpha-rc1.patch, HADOOP-9648.v2.patch > > > Some patches for fixing build a hadoop native library on os x 10.7/10.8. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HADOOP-10359) Native bzip2 compression support is broken on non-Linux systems
[ https://issues.apache.org/jira/browse/HADOOP-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909468#comment-13909468 ] Ilya Maykov commented on HADOOP-10359: -- This is obviously a minor issue since Bzip2 has a pure-java implementation, but it's nice to be able to run 'hadoop checknative -a' on my mac and not get an error. > Native bzip2 compression support is broken on non-Linux systems > --- > > Key: HADOOP-10359 > URL: https://issues.apache.org/jira/browse/HADOOP-10359 > Project: Hadoop Common > Issue Type: Improvement > Components: native >Affects Versions: 2.2.0 > Environment: Mac OS X 10.8.5 > Oracle JDK 1.7.0_51 > Hadoop 2.2.0-CDH5.0.0-beta-2 >Reporter: Ilya Maykov >Priority: Minor > Attachments: HADOOP-10359-native-bzip2-for-os-x.patch > > > While testing the patch for HADOOP-9648, I noticed that the bzip2 native > compressor/decompressor support wasn't working properly. I dug around a bit > and got native bzip2 support to work on my macbook. Will attach a patch in a > bit. (This probably needs to be tested on FreeBSD / Windows / Linux, but I > don't have the time to set up the necessary VMs to do it. I assume the build > bot will test Linux). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HADOOP-10359) Native bzip2 compression support is broken on non-Linux systems
[ https://issues.apache.org/jira/browse/HADOOP-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909467#comment-13909467 ] Ilya Maykov commented on HADOOP-10359: -- @ [~cnauroth] Ok, I stand corrected about the Windows build. Still, the default HADOOP_BZIP2_LIBRARY #define will not work for FreeBSD or Mac OS X w/o this change ... > Native bzip2 compression support is broken on non-Linux systems > --- > > Key: HADOOP-10359 > URL: https://issues.apache.org/jira/browse/HADOOP-10359 > Project: Hadoop Common > Issue Type: Improvement > Components: native >Affects Versions: 2.2.0 > Environment: Mac OS X 10.8.5 > Oracle JDK 1.7.0_51 > Hadoop 2.2.0-CDH5.0.0-beta-2 >Reporter: Ilya Maykov >Priority: Minor > Attachments: HADOOP-10359-native-bzip2-for-os-x.patch > > > While testing the patch for HADOOP-9648, I noticed that the bzip2 native > compressor/decompressor support wasn't working properly. I dug around a bit > and got native bzip2 support to work on my macbook. Will attach a patch in a > bit. (This probably needs to be tested on FreeBSD / Windows / Linux, but I > don't have the time to set up the necessary VMs to do it. I assume the build > bot will test Linux). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HADOOP-10359) Native bzip2 compression support is broken on non-Linux systems
[ https://issues.apache.org/jira/browse/HADOOP-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909182#comment-13909182 ] Ilya Maykov commented on HADOOP-10359: -- Could a committer please code review? > Native bzip2 compression support is broken on non-Linux systems > --- > > Key: HADOOP-10359 > URL: https://issues.apache.org/jira/browse/HADOOP-10359 > Project: Hadoop Common > Issue Type: Improvement > Components: native >Affects Versions: 2.2.0 > Environment: Mac OS X 10.8.5 > Oracle JDK 1.7.0_51 > Hadoop 2.2.0-CDH5.0.0-beta-2 >Reporter: Ilya Maykov >Priority: Minor > Attachments: HADOOP-10359-native-bzip2-for-os-x.patch > > > While testing the patch for HADOOP-9648, I noticed that the bzip2 native > compressor/decompressor support wasn't working properly. I dug around a bit > and got native bzip2 support to work on my macbook. Will attach a patch in a > bit. (This probably needs to be tested on FreeBSD / Windows / Linux, but I > don't have the time to set up the necessary VMs to do it. I assume the build > bot will test Linux). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HADOOP-10359) Native bzip2 compression support is broken on non-Linux systems
[ https://issues.apache.org/jira/browse/HADOOP-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909167#comment-13909167 ] Ilya Maykov commented on HADOOP-10359: -- Note: I believe that native support is broken on non-linux systems because the header file unconditionally does #define HADOOP_BZIP2_LIBRARY "libbz2.so.1" which I think is only correct for Linux according to some comments found in CMakeLists.txt (Windows should have a ".dll", FreeBSD a ."so" without the version, and Mac OS X a ".version.dylib"). However I've only verified that it's broken on Mac OS X. > Native bzip2 compression support is broken on non-Linux systems > --- > > Key: HADOOP-10359 > URL: https://issues.apache.org/jira/browse/HADOOP-10359 > Project: Hadoop Common > Issue Type: Improvement > Components: native >Affects Versions: 2.2.0 > Environment: Mac OS X 10.8.5 > Oracle JDK 1.7.0_51 > Hadoop 2.2.0-CDH5.0.0-beta-2 >Reporter: Ilya Maykov >Priority: Minor > Attachments: HADOOP-10359-native-bzip2-for-os-x.patch > > > While testing the patch for HADOOP-9648, I noticed that the bzip2 native > compressor/decompressor support wasn't working properly. I dug around a bit > and got native bzip2 support to work on my macbook. Will attach a patch in a > bit. (This probably needs to be tested on FreeBSD / Windows / Linux, but I > don't have the time to set up the necessary VMs to do it. I assume the build > bot will test Linux). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (HADOOP-10359) Native bzip2 compression support is broken on non-Linux systems
Ilya Maykov created HADOOP-10359: Summary: Native bzip2 compression support is broken on non-Linux systems Key: HADOOP-10359 URL: https://issues.apache.org/jira/browse/HADOOP-10359 Project: Hadoop Common Issue Type: Improvement Components: native Affects Versions: 2.2.0 Environment: Mac OS X 10.8.5 Oracle JDK 1.7.0_51 Hadoop 2.2.0-CDH5.0.0-beta-2 Reporter: Ilya Maykov Priority: Minor Attachments: HADOOP-10359-native-bzip2-for-os-x.patch While testing the patch for HADOOP-9648, I noticed that the bzip2 native compressor/decompressor support wasn't working properly. I dug around a bit and got native bzip2 support to work on my macbook. Will attach a patch in a bit. (This probably needs to be tested on FreeBSD / Windows / Linux, but I don't have the time to set up the necessary VMs to do it. I assume the build bot will test Linux). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HADOOP-10359) Native bzip2 compression support is broken on non-Linux systems
[ https://issues.apache.org/jira/browse/HADOOP-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Maykov updated HADOOP-10359: - Attachment: HADOOP-10359-native-bzip2-for-os-x.patch > Native bzip2 compression support is broken on non-Linux systems > --- > > Key: HADOOP-10359 > URL: https://issues.apache.org/jira/browse/HADOOP-10359 > Project: Hadoop Common > Issue Type: Improvement > Components: native >Affects Versions: 2.2.0 > Environment: Mac OS X 10.8.5 > Oracle JDK 1.7.0_51 > Hadoop 2.2.0-CDH5.0.0-beta-2 >Reporter: Ilya Maykov >Priority: Minor > Attachments: HADOOP-10359-native-bzip2-for-os-x.patch > > > While testing the patch for HADOOP-9648, I noticed that the bzip2 native > compressor/decompressor support wasn't working properly. I dug around a bit > and got native bzip2 support to work on my macbook. Will attach a patch in a > bit. (This probably needs to be tested on FreeBSD / Windows / Linux, but I > don't have the time to set up the necessary VMs to do it. I assume the build > bot will test Linux). -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HADOOP-10358) libhadoop doesn't compile on Mac OS X
[
https://issues.apache.org/jira/browse/HADOOP-10358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13909153#comment-13909153
]
Ilya Maykov commented on HADOOP-10358:
--
Yes, this is a duplicate. My patch is smaller and only fixes
hadoop-common-project without changing hadoop-yarn-project. I'll make a comment
in the other issue.
> libhadoop doesn't compile on Mac OS X
> -
>
> Key: HADOOP-10358
> URL: https://issues.apache.org/jira/browse/HADOOP-10358
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Environment: Mac OS X 10.8.5
> Oracle JDK 1.7.0_51
>Reporter: Ilya Maykov
>Priority: Minor
> Attachments: HADOOP-10358-fix-hadoop-common-native-on-os-x.patch
>
>
> The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
> on mac) fails to compile on Mac OS X. The problem is in
> hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
> at lines 76-78:
> [exec]
> /Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
> error: invalid operands to binary expression ('void' and 'int')
> [exec] if(setnetgrent(cgroup) == 1) {
> [exec] ~~~ ^ ~
> There are two problems in the code:
> 1) The #ifndef guard only checks for __FreeBSD__ but should check for either
> one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its
> syscalls from FreeBSD rather than Linux, and thus the setnetgrent() syscall
> returns void.
> 2) setnetgrentCalledFlag = 1 is set outside the #ifndef guard, but the
> syscall is only invoked inside the guard. This means that on FreeBSD,
> endnetgrent() can be called in the cleanup code without a corresponding
> setnetgrent() invocation.
> I have a patch that fixes both issues (will attach in a bit). With this
> patch, I'm able to compile libhadoop.dylib on Mac OS X, which in turn lets me
> install native snappy, lzo, etc compressor libraries on my client. That lets
> me run commands like 'hadoop fs -text somefile.lzo' from the macbook rather
> than having to ssh to a linux box, etc.
> Note that this patch only fixes the native build of hadoop-common-project.
> Some other components of hadoop still fail to build their native components,
> but libhadoop.dylib is enough for the client.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HADOOP-10358) libhadoop doesn't compile on Mac OS X
[
https://issues.apache.org/jira/browse/HADOOP-10358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Maykov updated HADOOP-10358:
-
Description:
The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
on mac) fails to compile on Mac OS X. The problem is in
hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
at lines 76-78:
[exec]
/Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
error: invalid operands to binary expression ('void' and 'int')
[exec] if(setnetgrent(cgroup) == 1) {
[exec] ~~~ ^ ~
There are two problems in the code:
1) The #ifndef guard only checks for __FreeBSD__ but should check for either
one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its syscalls
from FreeBSD rather than Linux, and thus the setnetgrent() syscall returns void.
2) setnetgrentCalledFlag = 1 is set outside the #ifndef guard, but the syscall
is only invoked inside the guard. This means that on FreeBSD, endnetgrent() can
be called in the cleanup code without a corresponding setnetgrent() invocation.
I have a patch that fixes both issues (will attach in a bit). With this patch,
I'm able to compile libhadoop.dylib on Mac OS X, which in turn lets me install
native snappy, lzo, etc compressor libraries on my client. That lets me run
commands like 'hadoop fs -text somefile.lzo' from the macbook rather than
having to ssh to a linux box, etc.
Note that this patch only fixes the native build of hadoop-common-project. Some
other components of hadoop still fail to build their native components, but
libhadoop.dylib is enough for the client.
was:
The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
on mac) fails to compile on Mac OS X. The problem is in
hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
at lines 76-78:
[exec]
/Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
error: invalid operands to binary expression ('void' and 'int')
[exec] if(setnetgrent(cgroup) == 1) {
[exec] ~~~ ^ ~
There are two problems in the code:
1) The #ifndef guard only checks for __FreeBSD__ but should check for either
one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its syscalls
from FreeBSD rather than Linux, and thus the setnetgrent() syscall returns void.
2) setnetgrentCalledFlag = 1 is called outside the #ifndef guard, but the
syscall is only called inside the guard. This means that on FreeBSD,
endnetgrent() can be called in the cleanup code without a corresponding
setnetgrent() invocation.
I have a patch that fixes both issues (will attach in a bit). With this patch,
I'm able to compile libhadoop.dylib on Mac OS X, which in turn lets me install
native snappy, lzo, etc compressor libraries on my client. That lets me run
commands like 'hadoop fs -text somefile.lzo' from the macbook rather than
having to ssh to a linux box, etc.
Note that this patch only fixes the native build of hadoop-common-project. Some
other components of hadoop still fail to build their native components, but
libhadoop.dylib is enough for the client.
> libhadoop doesn't compile on Mac OS X
> -
>
> Key: HADOOP-10358
> URL: https://issues.apache.org/jira/browse/HADOOP-10358
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Environment: Mac OS X 10.8.5
> Oracle JDK 1.7.0_51
>Reporter: Ilya Maykov
>Priority: Minor
> Attachments: HADOOP-10358-fix-hadoop-common-native-on-os-x.patch
>
>
> The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
> on mac) fails to compile on Mac OS X. The problem is in
> hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
> at lines 76-78:
> [exec]
> /Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
> error: invalid operands to binary expression ('void' and 'int')
> [exec] if(setnetgrent(cgroup) == 1) {
> [exec] ~~~ ^ ~
> There are two problems in the code:
> 1) The #ifndef guard only checks for __FreeBSD__ but should check for either
> one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its
> syscalls from FreeBSD rather than Linux, and thus the setnetgrent() syscall
> returns void.
> 2) setnetgrentCalledFlag = 1 is set outside the #
[jira] [Updated] (HADOOP-10358) libhadoop doesn't compile on Mac OS X
[
https://issues.apache.org/jira/browse/HADOOP-10358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Maykov updated HADOOP-10358:
-
Attachment: HADOOP-10358-fix-hadoop-common-native-on-os-x.patch
Patch attached.
> libhadoop doesn't compile on Mac OS X
> -
>
> Key: HADOOP-10358
> URL: https://issues.apache.org/jira/browse/HADOOP-10358
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Environment: Mac OS X 10.8.5
> Oracle JDK 1.7.0_51
>Reporter: Ilya Maykov
>Priority: Minor
> Attachments: HADOOP-10358-fix-hadoop-common-native-on-os-x.patch
>
>
> The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
> on mac) fails to compile on Mac OS X. The problem is in
> hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
> at lines 76-78:
> [exec]
> /Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
> error: invalid operands to binary expression ('void' and 'int')
> [exec] if(setnetgrent(cgroup) == 1) {
> [exec] ~~~ ^ ~
> There are two problems in the code:
> 1) The #ifndef guard only checks for __FreeBSD__ but should check for either
> one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its
> syscalls from FreeBSD rather than Linux, and thus the setnetgrent() syscall
> returns void.
> 2) setnetgrentCalledFlag = 1 is called outside the #ifndef guard, but the
> syscall is only called inside the guard. This means that on FreeBSD,
> endnetgrent() can be called in the cleanup code without a corresponding
> setnetgrent() invocation.
> I have a patch that fixes both issues (will attach in a bit). With this
> patch, I'm able to compile libhadoop.dylib on Mac OS X, which in turn lets me
> install native snappy, lzo, etc compressor libraries on my client. That lets
> me run commands like 'hadoop fs -text somefile.lzo' from the macbook rather
> than having to ssh to a linux box, etc.
> Note that this patch only fixes the native build of hadoop-common-project.
> Some other components of hadoop still fail to build their native components,
> but libhadoop.dylib is enough for the client.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (HADOOP-10358) libhadoop doesn't compile on Mac OS X
Ilya Maykov created HADOOP-10358:
Summary: libhadoop doesn't compile on Mac OS X
Key: HADOOP-10358
URL: https://issues.apache.org/jira/browse/HADOOP-10358
Project: Hadoop Common
Issue Type: Improvement
Components: native
Environment: Mac OS X 10.8.5
Oracle JDK 1.7.0_51
Reporter: Ilya Maykov
Priority: Minor
The native component of hadoop-common (libhadoop.so on linux, libhadoop.dylib
on mac) fails to compile on Mac OS X. The problem is in
hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c
at lines 76-78:
[exec]
/Users/ilyam/src/github/apache/hadoop-common/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.c:77:26:
error: invalid operands to binary expression ('void' and 'int')
[exec] if(setnetgrent(cgroup) == 1) {
[exec] ~~~ ^ ~
There are two problems in the code:
1) The #ifndef guard only checks for __FreeBSD__ but should check for either
one of __FreeBSD__ or __APPLE__. This is because Mac OS X inherits its syscalls
from FreeBSD rather than Linux, and thus the setnetgrent() syscall returns void.
2) setnetgrentCalledFlag = 1 is called outside the #ifndef guard, but the
syscall is only called inside the guard. This means that on FreeBSD,
endnetgrent() can be called in the cleanup code without a corresponding
setnetgrent() invocation.
I have a patch that fixes both issues (will attach in a bit). With this patch,
I'm able to compile libhadoop.dylib on Mac OS X, which in turn lets me install
native snappy, lzo, etc compressor libraries on my client. That lets me run
commands like 'hadoop fs -text somefile.lzo' from the macbook rather than
having to ssh to a linux box, etc.
Note that this patch only fixes the native build of hadoop-common-project. Some
other components of hadoop still fail to build their native components, but
libhadoop.dylib is enough for the client.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
