[jira] [Created] (HDFS-1802) Web interface of namenode does not work probably
Web interface of namenode does not work probably Key: HDFS-1802 URL: https://issues.apache.org/jira/browse/HDFS-1802 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 0.21.0 Environment: 0.21.1 on Oracle Solaris 10u7 X64 Reporter: XiaoboGu I have done the required steps point by FAQ of hadoop :http://wiki.apache.org/hadoop/FAQ#Why_do_files_and_directories_show_up_as_DrWho_and.2BAC8-or_user_names_are_missing.2BAC8-weird.3F but the web interface of namenode still have problems: 1. The "Browse the filesystem" link in page "http://solsvr:50070/dfshealth.jsp"; can't be started. 2. The "Go back to DFS home " link in page http://solsvr:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F, points to "http://localhost:50070/dfshealth.jsp";, which should be "http://namenode-server:50070/dfshealth.jsp"; > -bash-3.00$ more /etc/hosts > # > # Internet host table > # > ::1 localhost > 127.0.0.1 localhost > 192.168.72.4solsvr loghost > -bash-3.00$ I have only one entry in both slaves and masters file under $hadoop_home/conf. solsvr -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1802) Web interface of namenode does not work probably
[ https://issues.apache.org/jira/browse/HDFS-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiaoboGu updated HDFS-1802: --- Environment: 0.21.0 on Oracle Solaris 10u7 X64 (was: 0.21.1 on Oracle Solaris 10u7 X64) > Web interface of namenode does not work probably > > > Key: HDFS-1802 > URL: https://issues.apache.org/jira/browse/HDFS-1802 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node >Affects Versions: 0.21.0 > Environment: 0.21.0 on Oracle Solaris 10u7 X64 >Reporter: XiaoboGu > > I have done the required steps point by FAQ of hadoop > :http://wiki.apache.org/hadoop/FAQ#Why_do_files_and_directories_show_up_as_DrWho_and.2BAC8-or_user_names_are_missing.2BAC8-weird.3F > but the web interface of namenode still have problems: > 1. The "Browse the filesystem" link in page > "http://solsvr:50070/dfshealth.jsp"; can't be started. > 2. The "Go back to DFS home " link in page > http://solsvr:50075/browseDirectory.jsp?namenodeInfoPort=50070&dir=%2F, > points to "http://localhost:50070/dfshealth.jsp";, which should be > "http://namenode-server:50070/dfshealth.jsp"; > > -bash-3.00$ more /etc/hosts > > # > > # Internet host table > > # > > ::1 localhost > > 127.0.0.1 localhost > > 192.168.72.4solsvr loghost > > -bash-3.00$ > I have only one entry in both slaves and masters file under > $hadoop_home/conf. > solsvr -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1580) Add interface for generic Write Ahead Logging mechanisms
[ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated HDFS-1580: - Attachment: HDFS-1580.diff > Add interface for generic Write Ahead Logging mechanisms > > > Key: HDFS-1580 > URL: https://issues.apache.org/jira/browse/HDFS-1580 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ivan Kelly > Attachments: HDFS-1580+1521.diff, HDFS-1580.diff, HDFS-1580.diff, > HDFS-1580.diff, generic_wal_iface.pdf, generic_wal_iface.pdf, > generic_wal_iface.pdf, generic_wal_iface.txt > > -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1725) Cleanup FSImage construction
[ https://issues.apache.org/jira/browse/HDFS-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015224#comment-13015224 ] Ivan Kelly commented on HDFS-1725: -- Just to clarify the issue I'm trying to address with this JIRA while it's fresh in my mind (just hit the same problem again on HDFS-1580). In HDFS-1580 I'm trying to add the ability to have a editlog output stream which is not backed by a StorageDirectory. Currently images and edits are both stored in a StorageDirectory. Currently when FSImage and FSEditLog are created, the image and edit URIs may or may not be set. If they are not set on construction, NNStorage.setStorageDirectories is called to set them. This isn't a problem because FSEditLog only uses StorageDirectory backed output streams and as such can get the list of locations to write to from storage, which is exactly what it does. On FSEditLog.open() is iterates through all EDITS type directories in the storage object, and creates a stream for it. Obviously this will not work for anything but StorageDirectories and it's precisely this problem which I was trying to address here. If NNStorage.setStorageDirectories() is used, you have to go to NNStorage for the location list when calling FSEditLog.open as the list could change at any time. The solution I proposed was to mandate that you always set the StorageDirectories on construction. This means that FSEditLog can get the list on construction from FSImage and never has to query NNStorage. The only other solution I can think of is to add a setStorageDirectories call to FSImage which calls the same call on NNStorage and sets the list for FSEditLog. This would work, but does seem as elegant as just setting them on construction. > Cleanup FSImage construction > > > Key: HDFS-1725 > URL: https://issues.apache.org/jira/browse/HDFS-1725 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ivan Kelly >Assignee: Ivan Kelly > Fix For: 0.23.0 > > Attachments: HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff, > HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff > > > FSImage construction is messy. Sometimes the storagedirectories in use are > set straight away, sometimes they are not. This makes it hard for anything > under FSImage (i.e. FSEditLog) to make assumptions about what it can use. > Therefore, this patch makes FSImage set the storage directories in use during > construction, and never allows them to change. If you want to change > storagedirectories you create a new image. > Also, all the construction code should be the same with the only difference > being the parameters passed. When not passed, these should get sensible > defaults. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1725) Cleanup FSImage construction
[ https://issues.apache.org/jira/browse/HDFS-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015180#comment-13015180 ] Hadoop QA commented on HDFS-1725: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12475315/HDFS-1725.diff against trunk revision 1087900. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestFileConcurrentReader -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/312//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/312//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/312//console This message is automatically generated. > Cleanup FSImage construction > > > Key: HDFS-1725 > URL: https://issues.apache.org/jira/browse/HDFS-1725 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ivan Kelly >Assignee: Ivan Kelly > Fix For: 0.23.0 > > Attachments: HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff, > HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff > > > FSImage construction is messy. Sometimes the storagedirectories in use are > set straight away, sometimes they are not. This makes it hard for anything > under FSImage (i.e. FSEditLog) to make assumptions about what it can use. > Therefore, this patch makes FSImage set the storage directories in use during > construction, and never allows them to change. If you want to change > storagedirectories you create a new image. > Also, all the construction code should be the same with the only difference > being the parameters passed. When not passed, these should get sensible > defaults. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1725) Cleanup FSImage construction
[ https://issues.apache.org/jira/browse/HDFS-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated HDFS-1725: - Attachment: HDFS-1725.diff I've removed everything that was related to ensuring that image and edits directories are set from the start. In all it doesn't do very much now, just pushes the FSImage() code around a bit and removes FSImage(URI). I initially created this JIRA because it was possible to change the directories underneath FSImage without going through FSImage. This made it hard to make assumptions about the state of FSImage. This patch in its current form does not fix that, so I wouldn't recommend it for submission. > Cleanup FSImage construction > > > Key: HDFS-1725 > URL: https://issues.apache.org/jira/browse/HDFS-1725 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Ivan Kelly >Assignee: Ivan Kelly > Fix For: 0.23.0 > > Attachments: HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff, > HDFS-1725.diff, HDFS-1725.diff, HDFS-1725.diff > > > FSImage construction is messy. Sometimes the storagedirectories in use are > set straight away, sometimes they are not. This makes it hard for anything > under FSImage (i.e. FSEditLog) to make assumptions about what it can use. > Therefore, this patch makes FSImage set the storage directories in use during > construction, and never allows them to change. If you want to change > storagedirectories you create a new image. > Also, all the construction code should be the same with the only difference > being the parameters passed. When not passed, these should get sensible > defaults. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1580) Add interface for generic Write Ahead Logging mechanisms
[ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015162#comment-13015162 ] Ivan Kelly commented on HDFS-1580: -- Applies on top of 1073 branch. > Add interface for generic Write Ahead Logging mechanisms > > > Key: HDFS-1580 > URL: https://issues.apache.org/jira/browse/HDFS-1580 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ivan Kelly > Attachments: HDFS-1580+1521.diff, HDFS-1580.diff, HDFS-1580.diff, > generic_wal_iface.pdf, generic_wal_iface.pdf, generic_wal_iface.pdf, > generic_wal_iface.txt > > -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1580) Add interface for generic Write Ahead Logging mechanisms
[ https://issues.apache.org/jira/browse/HDFS-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated HDFS-1580: - Attachment: HDFS-1580.diff First drop of implementation of most recent design. Changes are intended to be minimal. Compiles, but doesn't pass tests yet. > Add interface for generic Write Ahead Logging mechanisms > > > Key: HDFS-1580 > URL: https://issues.apache.org/jira/browse/HDFS-1580 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ivan Kelly > Attachments: HDFS-1580+1521.diff, HDFS-1580.diff, HDFS-1580.diff, > generic_wal_iface.pdf, generic_wal_iface.pdf, generic_wal_iface.pdf, > generic_wal_iface.txt > > -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
