[jira] [Commented] (FILEUPLOAD-288) Linux dentry cache memory leak due to inappropriate temporary file detection
[
https://issues.apache.org/jira/browse/FILEUPLOAD-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16264756#comment-16264756
]
ASF GitHub Bot commented on FILEUPLOAD-288:
---
Github user jochenw commented on the issue:
https://github.com/apache/commons-fileupload/pull/12
Applied to master, and b1_3 branches, thank you!
> Linux dentry cache memory leak due to inappropriate temporary file detection
>
>
> Key: FILEUPLOAD-288
> URL: https://issues.apache.org/jira/browse/FILEUPLOAD-288
> Project: Commons FileUpload
> Issue Type: Improvement
> Environment: OS version: CentOS release 6.7 (Final)
> Kenel: 2.6.32-573.26.1.el6.x86_64
> JVM: Oracle 1.8.0_102-b14
> Tomcat: 8.0.33.0
> commons-fileupload : 1.3.3
>Reporter: fangwentong
>
> We use commons-fileupload:1.3.3 to handle multipart request, param
> {{file-size-threshold}} was set to 200K. Our API handle multipart request in
> high QPS, the size of request body is about 20KB, which means those parts was
> saved in memory, not written to disk temporary file.
> But, in our production environment, size of dentry cache grows up in high
> speed. here are some analysis:
> in {{strace}} result, plenty of {{stat}} system call invoked for tmp files
> which not exists.
> {noformat}
> [pid 111310] <... stat resumed> {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
> [pid 57844] <... stat resumed> 0x7f08e80b84b0) = -1 ENOENT (No such file or
> directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624923.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624922.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624921.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624920.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624919.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624918.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 69798] stat("/tmp",
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624917.tmp",
>
> [pid 69798] <... stat resumed> {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
> [pid 57844] <... stat resumed> 0x7f08e80b84b0) = -1 ENOENT (No such file or
> directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624916.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624915.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624914.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624913.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624912.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624911.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624910.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624909.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624908.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624907.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624906.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624905.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624904.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624903.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624902.tmp",
> 0x7f08e
[jira] [Commented] (FILEUPLOAD-288) Linux dentry cache memory leak due to inappropriate temporary file detection
[
https://issues.apache.org/jira/browse/FILEUPLOAD-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16264743#comment-16264743
]
ASF GitHub Bot commented on FILEUPLOAD-288:
---
Github user asfgit closed the pull request at:
https://github.com/apache/commons-fileupload/pull/12
> Linux dentry cache memory leak due to inappropriate temporary file detection
>
>
> Key: FILEUPLOAD-288
> URL: https://issues.apache.org/jira/browse/FILEUPLOAD-288
> Project: Commons FileUpload
> Issue Type: Improvement
> Environment: OS version: CentOS release 6.7 (Final)
> Kenel: 2.6.32-573.26.1.el6.x86_64
> JVM: Oracle 1.8.0_102-b14
> Tomcat: 8.0.33.0
> commons-fileupload : 1.3.3
>Reporter: fangwentong
>
> We use commons-fileupload:1.3.3 to handle multipart request, param
> {{file-size-threshold}} was set to 200K. Our API handle multipart request in
> high QPS, the size of request body is about 20KB, which means those parts was
> saved in memory, not written to disk temporary file.
> But, in our production environment, size of dentry cache grows up in high
> speed. here are some analysis:
> in {{strace}} result, plenty of {{stat}} system call invoked for tmp files
> which not exists.
> {noformat}
> [pid 111310] <... stat resumed> {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
> [pid 57844] <... stat resumed> 0x7f08e80b84b0) = -1 ENOENT (No such file or
> directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624923.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624922.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624921.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624920.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624919.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624918.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 69798] stat("/tmp",
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624917.tmp",
>
> [pid 69798] <... stat resumed> {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
> [pid 57844] <... stat resumed> 0x7f08e80b84b0) = -1 ENOENT (No such file or
> directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624916.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624915.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624914.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624913.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624912.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624911.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624910.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624909.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624908.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624907.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624906.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624905.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624904.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624903.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624902.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pi
[jira] [Commented] (FILEUPLOAD-288) Linux dentry cache memory leak due to inappropriate temporary file detection
[
https://issues.apache.org/jira/browse/FILEUPLOAD-288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16263839#comment-16263839
]
ASF GitHub Bot commented on FILEUPLOAD-288:
---
GitHub user fangwentong opened a pull request:
https://github.com/apache/commons-fileupload/pull/12
FILEUPLOAD-288: detect file only if the file written to disk.
see https://issues.apache.org/jira/browse/FILEUPLOAD-288
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/fangwentong/commons-fileupload master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/commons-fileupload/pull/12.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12
commit b28019b27ab06847815de02ed69fffdfa6bf18d1
Author: fangwentong
Date: 2017-11-23T05:26:05Z
FILEUPLOAD-288: detect file only if the file written to disk.
> Linux dentry cache memory leak due to inappropriate temporary file detection
>
>
> Key: FILEUPLOAD-288
> URL: https://issues.apache.org/jira/browse/FILEUPLOAD-288
> Project: Commons FileUpload
> Issue Type: Improvement
> Environment: OS version: CentOS release 6.7 (Final)
> Kenel: 2.6.32-573.26.1.el6.x86_64
> JVM: Oracle 1.8.0_102-b14
> Tomcat: 8.0.33.0
> commons-fileupload : 1.3.3
>Reporter: fangwentong
>
> We use commons-fileupload:1.3.3 to handle multipart request, param
> {{file-size-threshold}} was set to 200K. Our API handle multipart request in
> high QPS, the size of request body is about 20KB, which means those parts was
> saved in memory, not written to disk temporary file.
> But, in our production environment, size of dentry cache grows up in high
> speed. here are some analysis:
> in {{strace}} result, plenty of {{stat}} system call invoked for tmp files
> which not exists.
> {noformat}
> [pid 111310] <... stat resumed> {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
> [pid 57844] <... stat resumed> 0x7f08e80b84b0) = -1 ENOENT (No such file or
> directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624923.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624922.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624921.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624920.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624919.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624918.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 69798] stat("/tmp",
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624917.tmp",
>
> [pid 69798] <... stat resumed> {st_mode=S_IFDIR|0755, st_size=6, ...}) = 0
> [pid 57844] <... stat resumed> 0x7f08e80b84b0) = -1 ENOENT (No such file or
> directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624916.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624915.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624914.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624913.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624912.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624911.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624910.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624909.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624908.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 57844]
> stat("/tmp/upload_75a3e205_c3f1_43e7_8165_34f6a7a02227_518624907.tmp",
> 0x7f08e80b84b0) = -1 ENOENT (No such file or directory)
> [pid 578
