[jira] [Commented] (FLINK-1437) Bug in PojoSerializer's copy() method
[
https://issues.apache.org/jira/browse/FLINK-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317981#comment-14317981
]
Timo Walther commented on FLINK-1437:
-
Thanks. I change the Fixed Version field.
Bug in PojoSerializer's copy() method
-
Key: FLINK-1437
URL: https://issues.apache.org/jira/browse/FLINK-1437
Project: Flink
Issue Type: Bug
Components: Java API
Reporter: Timo Walther
Assignee: Timo Walther
Fix For: 0.9, 0.8.1
The PojoSerializer's {{copy()}} method does not work properly with {{null}}
values. An exception could look like:
{code}
Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
terminated due to an exception: null
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:792)
Caused by: java.io.EOFException
at
org.apache.flink.runtime.io.disk.RandomAccessInputView.nextSegment(RandomAccessInputView.java:83)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.advance(AbstractPagedInputView.java:159)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readByte(AbstractPagedInputView.java:270)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readUnsignedByte(AbstractPagedInputView.java:277)
at org.apache.flink.types.StringValue.copyString(StringValue.java:839)
at
org.apache.flink.api.common.typeutils.base.StringSerializer.copy(StringSerializer.java:83)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.copy(PojoSerializer.java:261)
at
org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:449)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1303)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:788)
{code}
I'm working on a fix for that...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1437) Bug in PojoSerializer's copy() method
[
https://issues.apache.org/jira/browse/FLINK-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14317801#comment-14317801
]
Stephan Ewen commented on FLINK-1437:
-
I think the fix is in the master, I will try to pick it onto the
{{release-0.8}} branch...
Bug in PojoSerializer's copy() method
-
Key: FLINK-1437
URL: https://issues.apache.org/jira/browse/FLINK-1437
Project: Flink
Issue Type: Bug
Components: Java API
Reporter: Timo Walther
Assignee: Timo Walther
Fix For: 0.9
The PojoSerializer's {{copy()}} method does not work properly with {{null}}
values. An exception could look like:
{code}
Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
terminated due to an exception: null
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:792)
Caused by: java.io.EOFException
at
org.apache.flink.runtime.io.disk.RandomAccessInputView.nextSegment(RandomAccessInputView.java:83)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.advance(AbstractPagedInputView.java:159)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readByte(AbstractPagedInputView.java:270)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readUnsignedByte(AbstractPagedInputView.java:277)
at org.apache.flink.types.StringValue.copyString(StringValue.java:839)
at
org.apache.flink.api.common.typeutils.base.StringSerializer.copy(StringSerializer.java:83)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.copy(PojoSerializer.java:261)
at
org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:449)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1303)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:788)
{code}
I'm working on a fix for that...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1437) Bug in PojoSerializer's copy() method
[
https://issues.apache.org/jira/browse/FLINK-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303175#comment-14303175
]
ASF GitHub Bot commented on FLINK-1437:
---
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/342
Bug in PojoSerializer's copy() method
-
Key: FLINK-1437
URL: https://issues.apache.org/jira/browse/FLINK-1437
Project: Flink
Issue Type: Bug
Components: Java API
Reporter: Timo Walther
Assignee: Timo Walther
The PojoSerializer's {{copy()}} method does not work properly with {{null}}
values. An exception could look like:
{code}
Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
terminated due to an exception: null
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:792)
Caused by: java.io.EOFException
at
org.apache.flink.runtime.io.disk.RandomAccessInputView.nextSegment(RandomAccessInputView.java:83)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.advance(AbstractPagedInputView.java:159)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readByte(AbstractPagedInputView.java:270)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readUnsignedByte(AbstractPagedInputView.java:277)
at org.apache.flink.types.StringValue.copyString(StringValue.java:839)
at
org.apache.flink.api.common.typeutils.base.StringSerializer.copy(StringSerializer.java:83)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.copy(PojoSerializer.java:261)
at
org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:449)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1303)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:788)
{code}
I'm working on a fix for that...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1437) Bug in PojoSerializer's copy() method
[
https://issues.apache.org/jira/browse/FLINK-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14303028#comment-14303028
]
ASF GitHub Bot commented on FLINK-1437:
---
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/342#issuecomment-72623753
Ah, just saw that the tests are there. My bad...
Bug in PojoSerializer's copy() method
-
Key: FLINK-1437
URL: https://issues.apache.org/jira/browse/FLINK-1437
Project: Flink
Issue Type: Bug
Components: Java API
Reporter: Timo Walther
Assignee: Timo Walther
The PojoSerializer's {{copy()}} method does not work properly with {{null}}
values. An exception could look like:
{code}
Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
terminated due to an exception: null
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:792)
Caused by: java.io.EOFException
at
org.apache.flink.runtime.io.disk.RandomAccessInputView.nextSegment(RandomAccessInputView.java:83)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.advance(AbstractPagedInputView.java:159)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readByte(AbstractPagedInputView.java:270)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readUnsignedByte(AbstractPagedInputView.java:277)
at org.apache.flink.types.StringValue.copyString(StringValue.java:839)
at
org.apache.flink.api.common.typeutils.base.StringSerializer.copy(StringSerializer.java:83)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.copy(PojoSerializer.java:261)
at
org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:449)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1303)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:788)
{code}
I'm working on a fix for that...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1437) Bug in PojoSerializer's copy() method
[
https://issues.apache.org/jira/browse/FLINK-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291934#comment-14291934
]
ASF GitHub Bot commented on FLINK-1437:
---
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/342#issuecomment-71475896
This will probably conflict with https://github.com/apache/flink/pull/316.
Bug in PojoSerializer's copy() method
-
Key: FLINK-1437
URL: https://issues.apache.org/jira/browse/FLINK-1437
Project: Flink
Issue Type: Bug
Components: Java API
Reporter: Timo Walther
Assignee: Timo Walther
The PojoSerializer's {{copy()}} method does not work properly with {{null}}
values. An exception could look like:
{code}
Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
terminated due to an exception: null
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:792)
Caused by: java.io.EOFException
at
org.apache.flink.runtime.io.disk.RandomAccessInputView.nextSegment(RandomAccessInputView.java:83)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.advance(AbstractPagedInputView.java:159)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readByte(AbstractPagedInputView.java:270)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readUnsignedByte(AbstractPagedInputView.java:277)
at org.apache.flink.types.StringValue.copyString(StringValue.java:839)
at
org.apache.flink.api.common.typeutils.base.StringSerializer.copy(StringSerializer.java:83)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.copy(PojoSerializer.java:261)
at
org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:449)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1303)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:788)
{code}
I'm working on a fix for that...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1437) Bug in PojoSerializer's copy() method
[
https://issues.apache.org/jira/browse/FLINK-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14291931#comment-14291931
]
ASF GitHub Bot commented on FLINK-1437:
---
GitHub user twalthr opened a pull request:
https://github.com/apache/flink/pull/342
[FLINK-1437][Java API] Fixes copy() methods in PojoSerializer for null
values.
See description in FLINK-1437.
PR includes tests cases.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/twalthr/flink PojoCopyFix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/342.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #342
commit f6917765fb599de74ab89580f22feb2096ca946c
Author: twalthr i...@twalthr.com
Date: 2015-01-26T15:09:24Z
[FLINK-1437][Java API] Fixes copy() methods in PojoSerializer for null
values
Bug in PojoSerializer's copy() method
-
Key: FLINK-1437
URL: https://issues.apache.org/jira/browse/FLINK-1437
Project: Flink
Issue Type: Bug
Components: Java API
Reporter: Timo Walther
Assignee: Timo Walther
The PojoSerializer's {{copy()}} method does not work properly with {{null}}
values. An exception could look like:
{code}
Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
terminated due to an exception: null
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:792)
Caused by: java.io.EOFException
at
org.apache.flink.runtime.io.disk.RandomAccessInputView.nextSegment(RandomAccessInputView.java:83)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.advance(AbstractPagedInputView.java:159)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readByte(AbstractPagedInputView.java:270)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readUnsignedByte(AbstractPagedInputView.java:277)
at org.apache.flink.types.StringValue.copyString(StringValue.java:839)
at
org.apache.flink.api.common.typeutils.base.StringSerializer.copy(StringSerializer.java:83)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.copy(PojoSerializer.java:261)
at
org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:449)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1303)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:788)
{code}
I'm working on a fix for that...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (FLINK-1437) Bug in PojoSerializer's copy() method
[
https://issues.apache.org/jira/browse/FLINK-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14287801#comment-14287801
]
Stephan Ewen commented on FLINK-1437:
-
Good catch. The POJO serializer can probably handle this, without need to
update all individual serializers?
Bug in PojoSerializer's copy() method
-
Key: FLINK-1437
URL: https://issues.apache.org/jira/browse/FLINK-1437
Project: Flink
Issue Type: Bug
Components: Java API
Reporter: Timo Walther
Assignee: Timo Walther
The PojoSerializer's {{copy()}} method does not work properly with {{null}}
values. An exception could look like:
{code}
Caused by: java.io.IOException: Thread 'SortMerger spilling thread'
terminated due to an exception: null
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:792)
Caused by: java.io.EOFException
at
org.apache.flink.runtime.io.disk.RandomAccessInputView.nextSegment(RandomAccessInputView.java:83)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.advance(AbstractPagedInputView.java:159)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readByte(AbstractPagedInputView.java:270)
at
org.apache.flink.runtime.memorymanager.AbstractPagedInputView.readUnsignedByte(AbstractPagedInputView.java:277)
at org.apache.flink.types.StringValue.copyString(StringValue.java:839)
at
org.apache.flink.api.common.typeutils.base.StringSerializer.copy(StringSerializer.java:83)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializer.copy(PojoSerializer.java:261)
at
org.apache.flink.runtime.operators.sort.NormalizedKeySorter.writeToOutput(NormalizedKeySorter.java:449)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$SpillingThread.go(UnilateralSortMerger.java:1303)
at
org.apache.flink.runtime.operators.sort.UnilateralSortMerger$ThreadBase.run(UnilateralSortMerger.java:788)
{code}
I'm working on a fix for that...
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
