[jira] [Resolved] (SPARK-26016) Encoding not working when using a map / mapPartitions call
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26016. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23962 [https://github.com/apache/spark/pull/23962] > Encoding not working when using a map / mapPartitions call > -- > > Key: SPARK-26016 > URL: https://issues.apache.org/jira/browse/SPARK-26016 > Project: Spark > Issue Type: Bug > Components: Java API >Affects Versions: 2.4.0 >Reporter: Chris Caspanello >Assignee: Sean Owen >Priority: Major > Fix For: 3.0.0 > > Attachments: spark-sandbox.zip > > > Attached you will find a project with unit tests showing the issue at hand. > If I read in a ISO-8859-1 encoded file and simply write out what was read; > the contents in the part file matches what was read. Which is great. > However, the second I use a map / mapPartitions function it looks like the > encoding is not correct. In addition a simple collectAsList and writing that > list of strings to a file does not work either. I don't think I'm doing > anything wrong. Can someone please investigate? I think this is a bug. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-26016) Encoding not working when using a map / mapPartitions call
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26016. -- Resolution: Invalid > Encoding not working when using a map / mapPartitions call > -- > > Key: SPARK-26016 > URL: https://issues.apache.org/jira/browse/SPARK-26016 > Project: Spark > Issue Type: Bug > Components: Java API >Affects Versions: 2.4.0 >Reporter: Chris Caspanello >Priority: Major > Attachments: spark-sandbox.zip > > > Attached you will find a project with unit tests showing the issue at hand. > If I read in a ISO-8859-1 encoded file and simply write out what was read; > the contents in the part file matches what was read. Which is great. > However, the second I use a map / mapPartitions function it looks like the > encoding is not correct. In addition a simple collectAsList and writing that > list of strings to a file does not work either. I don't think I'm doing > anything wrong. Can someone please investigate? I think this is a bug. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org