[
https://issues.apache.org/jira/browse/PHOENIX-53?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920165#comment-13920165
]
James Violette commented on PHOENIX-53:
---------------------------------------
The patch did not change the table name condition check in PhoenixRuntime. I am
not sure why it did not work. I did not find any tests for
PhoenixRuntime.main() that would exercise these command line tool parameter
configurations.
String tableName = null;
if (TABLE_OPTION.equals(args[i])) {
if (++i == args.length || tableName != null) {
usageError();
}
tableName = args[i];
}
if (fileName.endsWith(SQL_FILE_EXT)) {
PhoenixRuntime.executeStatements(conn, new
FileReader(args[i]), Collections.emptyList());
} else if (fileName.endsWith(CSV_FILE_EXT)) {
if (tableName == null) {
tableName =
fileName.substring(fileName.lastIndexOf(File.separatorChar) + 1,
fileName.length()-CSV_FILE_EXT.length());
}
CSVCommonsLoader csvLoader =
new CSVCommonsLoader(conn, tableName, columns,
isStrict, customMetaCharacters);
csvLoader.upsert(fileName);
} else {
usageError();
}
There is an existing problem in the PhoenixRuntime.
$ cp examples/web_stat.csv examples/WEB_STAT.csv
$ ./bin/psql.py blvdevhdp04 examples/web_stat.sql examples/WEB_STAT.csv
Found
/home/jviolette.adm/git/apache-phoenix/bin/../phoenix-assembly/target/phoenix-3.0.0-SNAPSHOT-client.jar
Running
java -cp
".:/home/jviolette.adm/git/apache-phoenix/bin:/home/jviolette.adm/git/apache-phoenix/bin/../phoenix-assembly/target/phoenix-3.0.0-SNAPSHOT-client.jar"
-Dlog4j.configuration=file:/home/jviolette.adm/git/apache-phoenix/bin/log4j.properties
org.apache.phoenix.util.PhoenixRuntime blvdevhdp04 examples/web_stat.sql
examples/WEB_STAT.csv
no rows upserted
Time: 0.301 sec(s)
csv columns from database.
phoenix columnInfo length=7, HOST CHAR, DOMAIN VARCHAR, FEATURE VARCHAR, DATE
DATE, CORE BIGINT, DB BIGINT, ACTIVE_VISITOR INTEGER
CSV Upsert complete. 39 rows upserted
Time: 0.056 sec(s)
$ ./bin/psql.py blvdevhdp04 examples/web_stat.sql examples/web_stat.csv
no rows upserted
Time: 0.328 sec(s)
csv columns from database.
phoenix columnInfo length=0,
java.lang.ArrayIndexOutOfBoundsException: 6
at
org.apache.phoenix.util.CSVCommonsLoader.upsert(CSVCommonsLoader.java:279)
at
org.apache.phoenix.util.CSVCommonsLoader.upsert(CSVCommonsLoader.java:189)
at org.apache.phoenix.util.PhoenixRuntime.main(PhoenixRuntime.java:203)
> Replace CSV loader with Apache Commons CSV loader
> -------------------------------------------------
>
> Key: PHOENIX-53
> URL: https://issues.apache.org/jira/browse/PHOENIX-53
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 2.2.3, 3.0.0
> Reporter: James Violette
> Labels: patch
> Fix For: 3.0.0
>
> Attachments: colInfoToString.patch,
> commons-csv-1.0-SNAPSHOT-sources.jar, commons-csv-1.0-SNAPSHOT.jar,
> csvloaderoverrides.diff, csvloadersource.diff,
> incubator-phoenix-commons-csv-rev2-3.0.0.patch
>
>
> in org.apache.phoenix.util.CSVLoader, the upsert fails if it encounters an
> empty line. This occurs if all lines end with the new line character and the
> reader returns an empty line at the end. Other issues, such as encapsulated
> meta characters also occur.
> The fix is to replace the opencsv library with the current apache commons-csv
> library.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
