[GitHub] spark pull request #20796: [SPARK-23649][SQL] Skipping chars disallowed in U...

[cloud-fan]

Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20796#discussion_r175543480
  
    --- Diff: 
common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java 
---
    @@ -791,4 +795,21 @@ public void trimRightWithTrimString() {
         assertEquals(fromString("头"), 
fromString("头a???/").trimRight(fromString("数?/*&^%a")));
         assertEquals(fromString("头"), fromString("头数b数数 
[").trimRight(fromString(" []数b")));
       }
    +
    +  @Test
    +  public void skipWrongFirstByte() {
    +    int[] wrongFirstBytes = {
    --- End diff --
    
    what will happen if we print UTF8String with invalid bytes?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org