[jira] [Updated] (HDFS-4950) newly created files in fuse_dfs appear to be length 0 for a while
[
https://issues.apache.org/jira/browse/HDFS-4950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-4950:
---
Component/s: fuse-dfs
Target Version/s: 2.2.0
Affects Version/s: 2.2.0
newly created files in fuse_dfs appear to be length 0 for a while
-
Key: HDFS-4950
URL: https://issues.apache.org/jira/browse/HDFS-4950
Project: Hadoop HDFS
Issue Type: Bug
Components: fuse-dfs
Affects Versions: 2.2.0
Reporter: Colin Patrick McCabe
For some reason, newly created files in fuse_dfs appear to be length 0 for a
while.
{code}
cmccabe@keter:~ echo hi hi
cmccabe@keter:~ mv hi /mnt/tmp/hi
cmccabe@keter:~ ls -l /mnt/tmp
total 0
-rw-r--r-- 1 cmccabe users 0 Jul 2 13:24 hi
cmccabe@keter:~ cat /mnt/tmp/hi
cmccabe@keter:~ cat /mnt/tmp/hi
hi
{code}
Disabling FUSE attribute caching fixes this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4950) newly created files in fuse_dfs appear to be length 0 for a while
[
https://issues.apache.org/jira/browse/HDFS-4950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-4950:
---
Attachment: 2013-07-02.incorrect-attr-trace.txt
2013-07-02.correct-noattr-trace.txt
here are two traces that demonstrate the problem. The issue seems to be that
after the fuse RELEASE operation, we never call GETATTR again when attribute
caching is enabled.
One easy fix for this would be to disable attribute caching entirely. This
would certainly fix the bug, but it might result in lower performance. As you
can see from the correct noattr trace, many more GETATTR operations are done
in this, all of which will hit the NameNode.
Can we live with the fuse attribute cache? This raises the question of how
we're supposed to invalidate the fuse_dfs attribute cache. I wasn't able to
find any documentation about this. I can see that fuse is checking the
attributes of the root directory after the release.
{code}
unique: 18, success, outsize: 16
unique: 19, opcode: RELEASE (18), nodeid: 2, insize: 64, pid: 0
release[140595351837776] flags: 0x8001
unique: 19, success, outsize: 16
unique: 20, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 6597
getattr /
unique: 20, success, outsize: 120
{code}
Is it possible that FUSE expects something to change there if a new file has
been added?
newly created files in fuse_dfs appear to be length 0 for a while
-
Key: HDFS-4950
URL: https://issues.apache.org/jira/browse/HDFS-4950
Project: Hadoop HDFS
Issue Type: Bug
Components: fuse-dfs
Affects Versions: 2.2.0
Reporter: Colin Patrick McCabe
Attachments: 2013-07-02.correct-noattr-trace.txt,
2013-07-02.incorrect-attr-trace.txt
For some reason, newly created files in fuse_dfs appear to be length 0 for a
while.
{code}
cmccabe@keter:~ echo hi hi
cmccabe@keter:~ mv hi /mnt/tmp/hi
cmccabe@keter:~ ls -l /mnt/tmp
total 0
-rw-r--r-- 1 cmccabe users 0 Jul 2 13:24 hi
cmccabe@keter:~ cat /mnt/tmp/hi
cmccabe@keter:~ cat /mnt/tmp/hi
hi
{code}
Disabling FUSE attribute caching fixes this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4950) newly created files in fuse_dfs appear to be length 0 for a while due to attribute caching
[
https://issues.apache.org/jira/browse/HDFS-4950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Patrick McCabe updated HDFS-4950:
---
Summary: newly created files in fuse_dfs appear to be length 0 for a while
due to attribute caching (was: newly created files in fuse_dfs appear to be
length 0 for a while)
newly created files in fuse_dfs appear to be length 0 for a while due to
attribute caching
--
Key: HDFS-4950
URL: https://issues.apache.org/jira/browse/HDFS-4950
Project: Hadoop HDFS
Issue Type: Bug
Components: fuse-dfs
Affects Versions: 2.2.0
Reporter: Colin Patrick McCabe
Attachments: 2013-07-02.correct-noattr-trace.txt,
2013-07-02.incorrect-attr-trace.txt
For some reason, newly created files in fuse_dfs appear to be length 0 for a
while.
{code}
cmccabe@keter:~ echo hi hi
cmccabe@keter:~ mv hi /mnt/tmp/hi
cmccabe@keter:~ ls -l /mnt/tmp
total 0
-rw-r--r-- 1 cmccabe users 0 Jul 2 13:24 hi
cmccabe@keter:~ cat /mnt/tmp/hi
cmccabe@keter:~ cat /mnt/tmp/hi
hi
{code}
Disabling FUSE attribute caching fixes this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
