[jira] [Assigned] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives (.har)
[ https://issues.apache.org/jira/browse/SPARK-39910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39910: --- Assignee: Christophe Préaud > DataFrameReader API cannot read files from hadoop archives (.har) > - > > Key: SPARK-39910 > URL: https://issues.apache.org/jira/browse/SPARK-39910 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.0.3, 3.1.3, 3.3.0, 3.2.2 >Reporter: Christophe Préaud >Assignee: Christophe Préaud >Priority: Minor > Labels: DataFrameReader, pull-request-available > > Reading a file from an hadoop archive using the DataFrameReader API returns > an empty Dataset: > {code:java} > scala> val df = > spark.read.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719") > df: org.apache.spark.sql.Dataset[String] = [value: string] > scala> df.count > res7: Long = 0 {code} > > On the other hand, reading the same file, from the same hadoop archive, but > using the RDD API yields the correct result: > {code:java} > scala> val df = > sc.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719").toDF("value") > df: org.apache.spark.sql.DataFrame = [value: string] > scala> df.count > res8: Long = 5589 {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives (.har)
[ https://issues.apache.org/jira/browse/SPARK-39910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-39910: -- Assignee: Apache Spark > DataFrameReader API cannot read files from hadoop archives (.har) > - > > Key: SPARK-39910 > URL: https://issues.apache.org/jira/browse/SPARK-39910 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.0.3, 3.1.3, 3.3.0, 3.2.2 >Reporter: Christophe Préaud >Assignee: Apache Spark >Priority: Minor > Labels: DataFrameReader, pull-request-available > > Reading a file from an hadoop archive using the DataFrameReader API returns > an empty Dataset: > {code:java} > scala> val df = > spark.read.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719") > df: org.apache.spark.sql.Dataset[String] = [value: string] > scala> df.count > res7: Long = 0 {code} > > On the other hand, reading the same file, from the same hadoop archive, but > using the RDD API yields the correct result: > {code:java} > scala> val df = > sc.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719").toDF("value") > df: org.apache.spark.sql.DataFrame = [value: string] > scala> df.count > res8: Long = 5589 {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives (.har)
[ https://issues.apache.org/jira/browse/SPARK-39910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-39910: -- Assignee: (was: Apache Spark) > DataFrameReader API cannot read files from hadoop archives (.har) > - > > Key: SPARK-39910 > URL: https://issues.apache.org/jira/browse/SPARK-39910 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.0.3, 3.1.3, 3.3.0, 3.2.2 >Reporter: Christophe Préaud >Priority: Minor > Labels: DataFrameReader, pull-request-available > > Reading a file from an hadoop archive using the DataFrameReader API returns > an empty Dataset: > {code:java} > scala> val df = > spark.read.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719") > df: org.apache.spark.sql.Dataset[String] = [value: string] > scala> df.count > res7: Long = 0 {code} > > On the other hand, reading the same file, from the same hadoop archive, but > using the RDD API yields the correct result: > {code:java} > scala> val df = > sc.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719").toDF("value") > df: org.apache.spark.sql.DataFrame = [value: string] > scala> df.count > res8: Long = 5589 {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives (.har)
[ https://issues.apache.org/jira/browse/SPARK-39910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-39910: -- Assignee: (was: Apache Spark) > DataFrameReader API cannot read files from hadoop archives (.har) > - > > Key: SPARK-39910 > URL: https://issues.apache.org/jira/browse/SPARK-39910 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.0.3, 3.1.3, 3.3.0, 3.2.2 >Reporter: Christophe Préaud >Priority: Minor > Labels: DataFrameReader, pull-request-available > > Reading a file from an hadoop archive using the DataFrameReader API returns > an empty Dataset: > {code:java} > scala> val df = > spark.read.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719") > df: org.apache.spark.sql.Dataset[String] = [value: string] > scala> df.count > res7: Long = 0 {code} > > On the other hand, reading the same file, from the same hadoop archive, but > using the RDD API yields the correct result: > {code:java} > scala> val df = > sc.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719").toDF("value") > df: org.apache.spark.sql.DataFrame = [value: string] > scala> df.count > res8: Long = 5589 {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-39910) DataFrameReader API cannot read files from hadoop archives (.har)
[ https://issues.apache.org/jira/browse/SPARK-39910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-39910: -- Assignee: Apache Spark > DataFrameReader API cannot read files from hadoop archives (.har) > - > > Key: SPARK-39910 > URL: https://issues.apache.org/jira/browse/SPARK-39910 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.0.3, 3.1.3, 3.3.0, 3.2.2 >Reporter: Christophe Préaud >Assignee: Apache Spark >Priority: Minor > Labels: DataFrameReader, pull-request-available > > Reading a file from an hadoop archive using the DataFrameReader API returns > an empty Dataset: > {code:java} > scala> val df = > spark.read.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719") > df: org.apache.spark.sql.Dataset[String] = [value: string] > scala> df.count > res7: Long = 0 {code} > > On the other hand, reading the same file, from the same hadoop archive, but > using the RDD API yields the correct result: > {code:java} > scala> val df = > sc.textFile("har:///user/preaudc/logs/lead/jp/2022/202207.har/20220719").toDF("value") > df: org.apache.spark.sql.DataFrame = [value: string] > scala> df.count > res8: Long = 5589 {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org