[
https://issues.apache.org/jira/browse/PHOENIX-2239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabriel Reid updated PHOENIX-2239:
----------------------------------
Attachment: PHOENIX-2239.patch
[~jamestaylor] just to be sure/clear, you're talking about the non-MR loader
(i.e. via psql) right? The MR-based loader (CsvBulkLoadTool) doesn't currently
have a --strict option.
It looks like the --strict option in psql is currently only used to enforce
that the list of columns coming from the CSV file matches up with columns in
the table. Here's a patch that causes the ingest tool to fail if there is a
failure to parse or upsert a given row as well.
Adding something like this into the MR-based loader would be more tricky, as
stopping the full job from a single MR task isn't easily done I think (MR will
just retry the task a few times).
> Honor strict option when catching data error during CSV load
> ------------------------------------------------------------
>
> Key: PHOENIX-2239
> URL: https://issues.apache.org/jira/browse/PHOENIX-2239
> Project: Phoenix
> Issue Type: Bug
> Reporter: James Taylor
> Attachments: PHOENIX-2239.patch
>
>
> The Phoenix CSV Bulk Loader has a strict option (i.e. -s or --strict) which
> is intended to abort the load if a data error is found. Looks like we're not
> passing this option through the errorOnRecord method, but I think that'd be a
> good (and likely pretty easy) improvement:
> {code}
> @Override
> public void errorOnRecord(CSVRecord csvRecord, String errorMessage) {
> LOG.error("Error upserting record {}: {}", csvRecord,
> errorMessage);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
