KyleGrains commented on a change in pull request #1044:
URL: https://github.com/apache/orc/pull/1044#discussion_r809642307
##########
File path: tools/test/TestCSVFileImport.cc
##########
@@ -90,3 +92,36 @@ TEST (TestCSVFileImport, testTimezoneOption) {
EXPECT_EQ("", error);
}
}
+
+TEST (TestCSVFileImport, testLongString) {
+ // create an ORC file from importing the CSV file
+ const std::string pgm1 = findProgram("tools/src/csv-import");
+ const std::string csvFile = "/tmp/test_csv_import_test_long_string.csv";
+ const std::string orcFile = "/tmp/test_csv_import_test_long_string.orc";
+ const std::string schema = "'struct<_a:string,b_:binary,_c:varchar(10)>'";
+ std::string output;
+ std::string error;
+
+ std::ofstream csvFileStream(csvFile, std::ios::binary | std::ios::out |
std::ios::trunc);
+ if(csvFileStream.is_open())
+ {
+ std::string longStr;
+ longStr.resize(4 * 1024 * 1024 + 1, 'x');
+ csvFileStream << "str1," << longStr << ",var1\n";
+ csvFileStream << "str2," << longStr << ",var2\n";
+ csvFileStream.close();
+ }
+
+ EXPECT_EQ(0, runProgram({pgm1, schema, csvFile, orcFile}, output, error));
+ EXPECT_EQ("", error);
+
+ // verify the ORC file content
+ const std::string pgm2 = findProgram("tools/src/orc-contents");
+ std::string option = "--columns=0,2";
+ const std::string expected =
+ "{\"_a\": \"str1\", \"_c\": \"var1\"}\n"
+ "{\"_a\": \"str2\", \"_c\": \"var2\"}\n";
+ EXPECT_EQ(0, runProgram({pgm2, option, orcFile}, output, error));
+ EXPECT_EQ(expected, output);
+ EXPECT_EQ("", error);
Review comment:
I am not sure if this is a code problem or some other issues, the output
like:
$make package test-out
...
Test project /home/kai/github/orc/build
Start 1: orc-test
1/2 Test #1: orc-test ......................... Passed 7.05 sec
Start 2: tool-test
2/2 Test #2: tool-test ........................***Failed 17.12 sec
ORC version: 1.8.0-SNAPSHOT
example dir = /home/kai/github/orc/examples
build dir = /home/kai/github/orc/build
[==========] Running 85 tests from 10 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from TestCSVFileImport
[ RUN ] TestCSVFileImport.test10rows
[ OK ] TestCSVFileImport.test10rows (15 ms)
[ RUN ] TestCSVFileImport.testTimezoneOption
[ OK ] TestCSVFileImport.testTimezoneOption (29 ms)
[ RUN ] TestCSVFileImport.testLongString
/home/kai/github/orc/tools/test/TestCSVFileImport.cc:124: Failure
Expected: 0
To be equal to: runProgram({pgm2, option, orcFile}, output, error)
Which is: 1
/home/kai/github/orc/tools/test/TestCSVFileImport.cc:125: Failure
Expected: expected
Which is: "{\"_a\": \"str1\", \"_c\": \"var1\"}\n{\"_a\": \"str2\",
\"_c\": \"var2\"}\n"
To be equal to: output
Which is: ""
With diff:
@@ -1,2 +1,1 @@
-{\"_a\": \"str1\", \"_c\": \"var1\"}
-{\"_a\": \"str2\", \"_c\": \"var2\"}\n
+""
/home/kai/github/orc/tools/test/TestCSVFileImport.cc:126: Failure
Expected: ""
To be equal to: error
Which is: "Caught exception in
/tmp/test_csv_import_test_long_string.orc: File size too small\n"
[ FAILED ] TestCSVFileImport.testLongString (58 ms)
$git diff main --stat
tools/test/TestCSVFileImport.cc | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)
$md5sum tools/test/TestCSVFileImport.cc
73b578e945183da5c3fdd8d16afda5c6 tools/test/TestCSVFileImport.cc
$cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
