[
https://issues.apache.org/jira/browse/FLINK-1208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220745#comment-14220745
]
ASF GitHub Bot commented on FLINK-1208:
---------------------------------------
Github user FelixNeutatz commented on a diff in the pull request:
https://github.com/apache/incubator-flink/pull/201#discussion_r20706975
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/io/DelimitedInputFormat.java
---
@@ -486,8 +536,42 @@ public boolean reachedEnd() {
@Override
public OT nextRecord(OT record) throws IOException {
if (readLine()) {
- return readRecord(record, this.currBuffer,
this.currOffset, this.currLen);
+ if (commentPrefix != null) {
+ //check record for comments
+ Boolean isComment = true;
+ for (int i = 0; i < commentPrefix.length; i++) {
+ if (commentPrefix[i] !=
this.currBuffer[this.currOffset + i]) {
+ isComment = false;
+ break;
+ }
+ }
+ if (isComment) {
+ this.commentCount++;
+ return nextRecord(record);
+ }
+ }
+
+ OT returnRecord = readRecord(record, this.currBuffer,
this.currOffset, this.currLen);
+
+ if (returnRecord == null) {
--- End diff --
Invalid lines are not skipped by default: CsvReader.class
```java
protected boolean ignoreInvalidLines = false;
```
> Skip comment lines in CSV input format. Allow user to specify comment
> character.
> --------------------------------------------------------------------------------
>
> Key: FLINK-1208
> URL: https://issues.apache.org/jira/browse/FLINK-1208
> Project: Flink
> Issue Type: Improvement
> Components: Java API, Scala API
> Affects Versions: 0.8-incubating
> Reporter: Aljoscha Krettek
> Assignee: Felix Neutatz
> Priority: Minor
> Labels: starter
>
> The current skipFirstLine is limited. Skipping arbitrary lines that start
> with a certain character would be much more flexible while still easy to
> implement.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
