Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21657 Actually the issue happened because I removed the mapping: https://github.com/apache/spark/commit/64fad0b519cf35b8c0a0dec18dd3df9488a5ed25#diff-d19881aceddcaa5c60620fdcda99b4c4L79 I would propose to revert it back, and remove all those "expensive" (comparing to look up `tokenIndexArr`) `map`s and the projection: https://github.com/apache/spark/pull/21657/files#diff-a549ac2e19ee7486911e2e6403444d9dR156
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org