[ https://issues.apache.org/jira/browse/CARBONDATA-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ajantha Bhat resolved CARBONDATA-3565. -------------------------------------- Fix Version/s: 2.0.0 Resolution: Fixed > Binary to string issue when loading dataframe data in NewRddIterator > -------------------------------------------------------------------- > > Key: CARBONDATA-3565 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3565 > Project: CarbonData > Issue Type: Bug > Components: spark-integration > Affects Versions: 1.6.0 > Reporter: ChenKai > Priority: Major > Fix For: 2.0.0 > > Time Spent: 6h > Remaining Estimate: 0h > > * issue > Spark DataFrame(SQL) load complex binary data to a hive table, the data will > be broken when reading out. I see in RddIterator, the data will be converted > to a string, and then be converted back. > * test case > Binary data can be *DataOutputStream#writeDouble* and so on. > * discussion > I think *CarbonScalaUtil#getString* operation can be removed now. I dig deep > into the code in 2016, the code was used in kettle *CsvInput* (commit: > 0018756d). But the code has been removed now, I think this converting > operation is a little redundant. (UPDATE: The follow-up code GenericParser > will use this string-convert logic, should consider here.) -- This message was sent by Atlassian Jira (v8.3.4#803005)