Github user parthchandra commented on a diff in the pull request:
https://github.com/apache/drill/pull/500#discussion_r67028137
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/TestSelectWithOption.java ---
@@ -89,30 +88,51 @@ public void testTabFieldDelimiter() throws Exception {
listOf("2", "b"));
}
- @Test @Ignore // It does not look like lineDelimiter is working
- public void testTextLineDelimiter() throws Exception {
+ @Test
+ public void testSingleTextLineDelimiter() throws Exception {
+ String tableName = genCSVTable("testSingleTextLineDelimiter",
+ "a|b|c");
+
+ testWithResult(format("select columns from table(%s(type => 'TeXT',
lineDelimiter => '|'))", tableName),
+ listOf("a"),
+ listOf("b"),
+ listOf("c"));
+ }
+
+ @Test
+ // '\n' is treated as standard delimiter
+ // if user has indicated custom line delimiter but input file contains
'\n', split will occur on both
+ public void testCustomTextLineDelimiterAndNewLine() throws Exception {
String tableName = genCSVTable("testTextLineDelimiter",
- "\"b\"|\"0\"",
- "\"b\"|\"1\"",
- "\"b\"|\"2\"");
+ "b|1",
+ "b|2");
testWithResult(format("select columns from table(%s(type => 'TeXT',
lineDelimiter => '|'))", tableName),
- listOf("\"b\""),
- listOf("\"0\"", "\"b\""),
- listOf("\"1\"", "\"b\""),
- listOf("\"2\"")
- );
+ listOf("b"),
+ listOf("1"),
+ listOf("b"),
+ listOf("2"));
}
@Test
public void testTextLineDelimiterWithCarriageReturn() throws Exception {
String tableName =
genCSVTable("testTextLineDelimiterWithCarriageReturn",
- "1,a\r",
- "2,b\r");
+ "1, a\r",
+ "2, b\r");
String lineDelimiter = new String(new char[]{92, 114, 92, 110}); //
represents \r\n
testWithResult(format("select columns from table(%s(type=>'TeXT',
lineDelimiter => '%s'))", tableName, lineDelimiter),
- listOf("1,a"),
- listOf("2,b"));
+ listOf("1, a"),
+ listOf("2, b"));
+ }
+
+ @Test
+ public void testMultiByteLineDelimiter() throws Exception {
+ String tableName = genCSVTable("testMultiByteLineDelimiter",
+ "1abc2abc3abc");
--- End diff --
I was expecting a test where the data has part of a multibyte delimiter in
it.
Data - ab1abc2abc3abc
delimiter - abc
Should output - ab1, 2, 3
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
