[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge updated HDFS-3296:
-
Attachment: HDFS-3296.004.patch
I'd like to share a patch I have been using. Not totally polished or thoroughly
tested yet.
Patch 004
* Work around domain socket path limitation by using "/tmp" instead of
$java.io.tmpdir
* On Mac, {{shutdown0}} can not unblock {{accept0}} which still holds on to a
refcount, thus {{DomainSocket#close}} stuck in the loop. Fix the issue by using
the "self-pipe" method. See {{accept1}} implementation for details.
* Add syslog calls to DomainSocket.c for easier debugging
* Set DYLD_LIBRARY_PATH to hadoop-hdfs-native-client pom. All libhdfs tests
pass including the zerocopy test.
TODO:
* {{TestDomainSocket#testAsyncCloseDuringWrite}} failed
* Pass all ShortCircuitRead tests
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
>Assignee: Chris Nauroth
> Attachments: HDFS-3296.001.patch, HDFS-3296.002.patch,
> HDFS-3296.003.patch, HDFS-3296.004.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated HDFS-3296:
-
Target Version/s: (was: 2.8.0)
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
>Assignee: Chris Nauroth
> Attachments: HDFS-3296.001.patch, HDFS-3296.002.patch,
> HDFS-3296.003.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-3296:
Attachment: HDFS-3296.003.patch
[~aw], I have filed HADOOP-13177 with a one-line patch for the Surefire
configuration change to set {{DYLD_LIBRARY_PATH}}. We can commit that one to
move ahead with Jenkins runs on OS X.
I'm also attaching a rebased v003 patch here for just the change in
hadoop-hdfs-native-client. We won't want to commit this one, because this will
just make {{test_libhdfs_zerocopy_hdfs_static}} hang indefinitely. We need to
get to the bottom of the domain socket issues on OS X before we can commit this
one.
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
>Assignee: Chris Nauroth
> Attachments: HDFS-3296.001.patch, HDFS-3296.002.patch,
> HDFS-3296.003.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-3296:
Status: Open (was: Patch Available)
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
>Assignee: Chris Nauroth
> Attachments: HDFS-3296.001.patch, HDFS-3296.002.patch,
> HDFS-3296.003.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-3296:
Attachment: HDFS-3296.002.patch
Patch v002 is a rebased patch for the required build changes. I still consider
this effectively blocked on HADOOP-11957, where we need to address domain
socket test failures on Mac.
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
>Assignee: Chris Nauroth
> Attachments: HDFS-3296.001.patch, HDFS-3296.002.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HDFS-3296:
---
Labels: (was: BB2015-05-TBR)
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
>Assignee: Chris Nauroth
> Attachments: HDFS-3296.001.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated HDFS-3296:
---
Labels: BB2015-05-TBR (was: )
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
>Assignee: Chris Nauroth
> Labels: BB2015-05-TBR
> Attachments: HDFS-3296.001.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-3296:
Component/s: libhdfs
Target Version/s: 2.7.0
Assignee: Chris Nauroth
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
>Assignee: Chris Nauroth
> Attachments: HDFS-3296.001.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-3296:
Attachment: HDFS-3296.001.patch
This is almost working in current trunk, as long as we set
{{DYLD_LIBRARY_PATH}}. (Patch attached.) However, this then runs into a
problem with test_libhdfs_zerocopy hanging. There appears to be some
difference in domain socket handling on Mac (or BSDs in general) that our code
isn't handling well. I'm still investigating.
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
> Attachments: HDFS-3296.001.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-3296) Running libhdfs tests in mac fails
[
https://issues.apache.org/jira/browse/HDFS-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-3296:
Status: Patch Available (was: Open)
> Running libhdfs tests in mac fails
> --
>
> Key: HDFS-3296
> URL: https://issues.apache.org/jira/browse/HDFS-3296
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: libhdfs
>Reporter: Amareshwari Sriramadasu
> Attachments: HDFS-3296.001.patch
>
>
> Running "ant -Dcompile.c++=true -Dlibhdfs=true test-c++-libhdfs" on Mac fails
> with following error:
> {noformat}
> [exec] dyld: lazy symbol binding failed: Symbol not found:
> _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec] dyld: Symbol not found: _JNI_GetCreatedJavaVMs
> [exec] Referenced from:
> /Users/amareshwari.sr/workspace/hadoop/build/c++/Mac_OS_X-x86_64-64/lib/libhdfs.0.dylib
> [exec] Expected in: flat namespace
> [exec]
> [exec]
> /Users/amareshwari.sr/workspace/hadoop/src/c++/libhdfs/tests/test-libhdfs.sh:
> line 122: 39485 Trace/BPT trap: 5 CLASSPATH=$HADOOP_CONF_DIR:$CLASSPATH
> LD_PRELOAD="$LIB_JVM_DIR/libjvm.so:$LIBHDFS_INSTALL_DIR/libhdfs.so:"
> $LIBHDFS_BUILD_DIR/$HDFS_TEST
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
