[jira] [Commented] (FLINK-5166) TextInputFormatTest.testNestedFileRead
[ https://issues.apache.org/jira/browse/FLINK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850972#comment-15850972 ] ASF GitHub Bot commented on FLINK-5166: --- Github user shijinkui closed the pull request at: https://github.com/apache/flink/pull/2876 > TextInputFormatTest.testNestedFileRead > -- > > Key: FLINK-5166 > URL: https://issues.apache.org/jira/browse/FLINK-5166 > Project: Flink > Issue Type: Bug > Components: Batch Connectors and Input/Output Formats, Tests >Reporter: shijinkui > > `mvn clean package -P \!scala-2.11,scala-2.11 -U` > Failed tests: > TextInputFormatTest.testNestedFileRead:140 Test erroneous > Tests run: 846, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (FLINK-5166) TextInputFormatTest.testNestedFileRead
[
https://issues.apache.org/jira/browse/FLINK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15708815#comment-15708815
]
ASF GitHub Bot commented on FLINK-5166:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/2876#discussion_r90249472
--- Diff:
flink-java/src/test/java/org/apache/flink/api/java/io/TextInputFormatTest.java
---
@@ -90,23 +92,23 @@ public void testSimpleRead() {
@Test
public void testNestedFileRead() {
- String[] dirs = new String[] {"tmp/first/", "tmp/second/"};
- List expectedFiles = new ArrayList<>();
-
try {
- for (String dir: dirs) {
+ String tmpDirPath =
CommonTestUtils.createTempDirectory().getPath();
+ String[] dirs = new String[]{tmpDirPath + "/first/",
tmpDirPath + "/second/"};
+ List expectedFiles = new ArrayList<>();
+ for (String dir : dirs) {
// create input file
File tmpDir = new File(dir);
- if (!tmpDir.exists()) {
- tmpDir.mkdirs();
+ if (!tmpDir.exists() && tmpDir.mkdirs()) {
+ tmpDir.deleteOnExit();
}
--- End diff --
this section could be a bit cleaner imo:
```
File parentDir = CommonTestUtils.createTempDirectory();
String[] dirs = new String[]{"first", "second"};
...
File tmpDir= new File(parentDir , dir);
```
> TextInputFormatTest.testNestedFileRead
> --
>
> Key: FLINK-5166
> URL: https://issues.apache.org/jira/browse/FLINK-5166
> Project: Flink
> Issue Type: Bug
> Components: Batch Connectors and Input/Output Formats, Tests
>Reporter: shijinkui
>
> `mvn clean package -P \!scala-2.11,scala-2.11 -U`
> Failed tests:
> TextInputFormatTest.testNestedFileRead:140 Test erroneous
> Tests run: 846, Failures: 1, Errors: 0, Skipped: 0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-5166) TextInputFormatTest.testNestedFileRead
[
https://issues.apache.org/jira/browse/FLINK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15708780#comment-15708780
]
ASF GitHub Bot commented on FLINK-5166:
---
Github user shijinkui commented on a diff in the pull request:
https://github.com/apache/flink/pull/2876#discussion_r90246314
--- Diff:
flink-java/src/test/java/org/apache/flink/api/java/io/TextInputFormatTest.java
---
@@ -90,7 +91,9 @@ public void testSimpleRead() {
@Test
public void testNestedFileRead() {
- String[] dirs = new String[] {"tmp/first/", "tmp/second/"};
+ String tmpDirName = "tmp-" +
UUID.randomUUID().toString().substring(0, 8);
--- End diff --
CommonTestUtils in core package have no function createTempDirectory, have
add one.
> TextInputFormatTest.testNestedFileRead
> --
>
> Key: FLINK-5166
> URL: https://issues.apache.org/jira/browse/FLINK-5166
> Project: Flink
> Issue Type: Bug
> Components: Batch Connectors and Input/Output Formats, Tests
>Reporter: shijinkui
>
> `mvn clean package -P \!scala-2.11,scala-2.11 -U`
> Failed tests:
> TextInputFormatTest.testNestedFileRead:140 Test erroneous
> Tests run: 846, Failures: 1, Errors: 0, Skipped: 0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-5166) TextInputFormatTest.testNestedFileRead
[
https://issues.apache.org/jira/browse/FLINK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15708231#comment-15708231
]
ASF GitHub Bot commented on FLINK-5166:
---
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/2876#discussion_r90209936
--- Diff:
flink-java/src/test/java/org/apache/flink/api/java/io/TextInputFormatTest.java
---
@@ -90,7 +91,9 @@ public void testSimpleRead() {
@Test
public void testNestedFileRead() {
- String[] dirs = new String[] {"tmp/first/", "tmp/second/"};
+ String tmpDirName = "tmp-" +
UUID.randomUUID().toString().substring(0, 8);
--- End diff --
We could reuse `CommonTestUtils#createTempDirectory()' in flink-runtime by
copying the method into the CommonTestUtils class in `flink-test-utils-junit`.
> TextInputFormatTest.testNestedFileRead
> --
>
> Key: FLINK-5166
> URL: https://issues.apache.org/jira/browse/FLINK-5166
> Project: Flink
> Issue Type: Bug
> Components: Batch Connectors and Input/Output Formats, Tests
>Reporter: shijinkui
>
> `mvn clean package -P \!scala-2.11,scala-2.11 -U`
> Failed tests:
> TextInputFormatTest.testNestedFileRead:140 Test erroneous
> Tests run: 846, Failures: 1, Errors: 0, Skipped: 0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-5166) TextInputFormatTest.testNestedFileRead
[ https://issues.apache.org/jira/browse/FLINK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15699759#comment-15699759 ] ASF GitHub Bot commented on FLINK-5166: --- Github user shijinkui commented on the issue: https://github.com/apache/flink/pull/2876 > The test should be changed to use more specific folder names including a random component. If the directory we are trying to create already exists we should not delete everything there (since we can't guarantee that we created those files), but instead try a different directory. @zentol That's file. Also delete the tmp dir when exist. > TextInputFormatTest.testNestedFileRead > -- > > Key: FLINK-5166 > URL: https://issues.apache.org/jira/browse/FLINK-5166 > Project: Flink > Issue Type: Bug > Components: Batch Connectors and Input/Output Formats, Tests >Reporter: shijinkui > > `mvn clean package -P \!scala-2.11,scala-2.11 -U` > Failed tests: > TextInputFormatTest.testNestedFileRead:140 Test erroneous > Tests run: 846, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-5166) TextInputFormatTest.testNestedFileRead
[ https://issues.apache.org/jira/browse/FLINK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15698373#comment-15698373 ] ASF GitHub Bot commented on FLINK-5166: --- Github user zentol commented on the issue: https://github.com/apache/flink/pull/2876 The test should be changed to use more specific folder names including a random component. If the directory we are trying to create already exists we should most definitely not just delete everything there, but instead try a different directory. > TextInputFormatTest.testNestedFileRead > -- > > Key: FLINK-5166 > URL: https://issues.apache.org/jira/browse/FLINK-5166 > Project: Flink > Issue Type: Bug > Components: Batch Connectors and Input/Output Formats, Tests >Reporter: shijinkui > > `mvn clean package -P \!scala-2.11,scala-2.11 -U` > Failed tests: > TextInputFormatTest.testNestedFileRead:140 Test erroneous > Tests run: 846, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-5166) TextInputFormatTest.testNestedFileRead
[
https://issues.apache.org/jira/browse/FLINK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15697655#comment-15697655
]
ASF GitHub Bot commented on FLINK-5166:
---
GitHub user shijinkui opened a pull request:
https://github.com/apache/flink/pull/2876
[FLINK-5166] TextInputFormatTest.testNestedFileRead
- [x] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira
title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the
JIRA id)
- [x] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [x] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis
build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shijinkui/flink FLINK-5166-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2876.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2876
commit 085b0fb94e20877c2eec3d2facd593045e26fb1b
Author: shijinkui
Date: 2016-11-26T09:42:17Z
[FLINK-5166] TextInputFormatTest.testNestedFileRead
> TextInputFormatTest.testNestedFileRead
> --
>
> Key: FLINK-5166
> URL: https://issues.apache.org/jira/browse/FLINK-5166
> Project: Flink
> Issue Type: Bug
> Components: Batch Connectors and Input/Output Formats, Tests
>Reporter: shijinkui
>
> `mvn clean package -P \!scala-2.11,scala-2.11 -U`
> Failed tests:
> TextInputFormatTest.testNestedFileRead:140 Test erroneous
> Tests run: 846, Failures: 1, Errors: 0, Skipped: 0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-5166) TextInputFormatTest.testNestedFileRead
[ https://issues.apache.org/jira/browse/FLINK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15697644#comment-15697644 ] shijinkui commented on FLINK-5166: -- Debug, found that: there four tmp file in the `expectedFiles` list, as there are two file in `paths`, so when execute `assertTrue(expectedFiles.get(i).equals(paths.get(i)));`, get null by the index. tmp file such as: flink/flink-java/tmp/first/TextInputFormatTest3439492861909451525.tmp It's strangely when i delete the tmp files, then run the test, it's ok. I think we can clear the directory before test. I add two line code in the PR. > TextInputFormatTest.testNestedFileRead > -- > > Key: FLINK-5166 > URL: https://issues.apache.org/jira/browse/FLINK-5166 > Project: Flink > Issue Type: Bug > Components: Batch Connectors and Input/Output Formats, Tests >Reporter: shijinkui > > `mvn clean package -P \!scala-2.11,scala-2.11 -U` > Failed tests: > TextInputFormatTest.testNestedFileRead:140 Test erroneous > Tests run: 846, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-5166) TextInputFormatTest.testNestedFileRead
[ https://issues.apache.org/jira/browse/FLINK-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15697607#comment-15697607 ] Chesnay Schepler commented on FLINK-5166: - can you include the exception that you got? > TextInputFormatTest.testNestedFileRead > -- > > Key: FLINK-5166 > URL: https://issues.apache.org/jira/browse/FLINK-5166 > Project: Flink > Issue Type: Bug > Components: Batch Connectors and Input/Output Formats, Tests >Reporter: shijinkui > > `mvn clean package -P \!scala-2.11,scala-2.11 -U` > Failed tests: > TextInputFormatTest.testNestedFileRead:140 Test erroneous > Tests run: 846, Failures: 1, Errors: 0, Skipped: 0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
