[ https://issues.apache.org/jira/browse/SPARK-5685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14311931#comment-14311931 ]
Nicholas Chammas commented on SPARK-5685: ----------------------------------------- [~joshrosen] - What do you think of adding a warning like this? > Show warning when users open text files compressed with non-splittable > algorithms like gzip > ------------------------------------------------------------------------------------------- > > Key: SPARK-5685 > URL: https://issues.apache.org/jira/browse/SPARK-5685 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Reporter: Nicholas Chammas > Priority: Minor > > This is a usability or user-friendliness issue. > It's extremely common for people to load a text file compressed with gzip, > process it, and then wonder why only 1 core in their cluster is doing any > work. > Some examples: > * http://stackoverflow.com/q/28127119/877069 > * http://stackoverflow.com/q/27531816/877069 > I'm not sure how this problem can be generalized, but at the very least it > would be helpful if Spark displayed some kind of warning in the common case > when someone opens a gzipped file with {{sc.textFile}}. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org