[ https://issues.apache.org/jira/browse/SPARK-30855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Roy updated SPARK-30855: ------------------------------- Description: Here's a public repo containing a scala test case that reproduces the issue * g...@github.com:benoitroy/spark-30855.git Simply execute the *Spark30855Tests* class. was: Im in the early phase of writting a custom spark datasource for messagepack. I have a basic implementation running in most cases but have encountered this issue when I tried to implement a UnaryExpression, in much the same way 'from _json' and 'from_avro' do, that converts a binary value (holding messagepack) into spark catalyst types. When using 'from_msgpack', which returns one InternalRow, all is well. However, if I use 'from_msgpack2' which uses 'explode' to return multiple InternalRow, any select follow up selects within the df results in a org.apache.spark.sql.catalyst.errors.package$TreeNodeException: Binding attribute, tree: _gen_alias_40#40 Couldn't find _gen_alias_40#40 in [result#4|#4] Example code that produces the issue: {code:java} df.select(from_msgpack2(col("value"), mp_schema).as("result")) .select("result.*") .show() {code} I've attached the code base as an archive to help you reproduce the issue. The test class org.apache.spark.sql.messagepack.MessagePackSparkIssue contains test cases that will help you reproduce the exception. It is entirely possible, and overall likely, that this is caused by a bad implementation, or wrong expectation, on my part. Any guidance help, feedback on the cause of this exception would be greatly appreciated. I've scoured the traditional sources for help to resolve this but haven't found anything related this this case. Cheers and Thanks. > Issue with Binding attribute: Couldn't find _gen_alias_ when using 'explode' > function. > -------------------------------------------------------------------------------------- > > Key: SPARK-30855 > URL: https://issues.apache.org/jira/browse/SPARK-30855 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.0.0 > Reporter: Benoit Roy > Priority: Major > > Here's a public repo containing a scala test case that reproduces the issue > > * g...@github.com:benoitroy/spark-30855.git > > Simply execute the *Spark30855Tests* class. > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org