[GitHub] [spark] HyukjinKwon commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768830169 cc @dongjoon-hyun and @cloud-fan, would you guys mind taking a look for this PR too? this is also related to parquet decimals

[GitHub] [spark] HyukjinKwon commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768630015 @razajafri, do you mind clarifying PR description? For exmaple, I thought you meant writing out to files or somewhere by: > Spark should read it as a long but write

[GitHub] [spark] HyukjinKwon commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

2021-01-27 Thread GitBox
HyukjinKwon commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768243423 > ... but write it as an int by downcasting it ... @razajafri would you mind pointing out where it happens?