How to get the help or explanation for the functions in Spark shell?
Hi All, I am new to Spark. In the Spark shell, how can I get the help or explanation for those functions that I can use for a variable or RDD? For example, after I input a RDD's name with a dot (.) at the end, if I press the Tab key, a list of functions that I can use for this RDD will be displayed, but I dont know how to use these functions. Your help is greatly appreciated. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-get-the-help-or-explanation-for-the-functions-in-Spark-shell-tp7191.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: How to get the help or explanation for the functions in Spark shell?
You can consult the docs at : https://spark.apache.org/docs/latest/api/scala/index.html#package In particular, the rdd docs contain the explanation of each method : https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.rdd.RDD Kr, Gerard On Jun 8, 2014 1:00 PM, Carter gyz...@hotmail.com wrote: Hi All, I am new to Spark. In the Spark shell, how can I get the help or explanation for those functions that I can use for a variable or RDD? For example, after I input a RDD's name with a dot (.) at the end, if I press the Tab key, a list of functions that I can use for this RDD will be displayed, but I dont know how to use these functions. Your help is greatly appreciated. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-get-the-help-or-explanation-for-the-functions-in-Spark-shell-tp7191.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: How to get the help or explanation for the functions in Spark shell?
Thank you very much Gerard. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-get-the-help-or-explanation-for-the-functions-in-Spark-shell-tp7191p7193.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: How to get the help or explanation for the functions in Spark shell?
In PySpark you can also do help(my_rdd) and get a nice help page of methods available. 2014년 6월 8일 일요일, Cartergyz...@hotmail.com님이 작성한 메시지: Thank you very much Gerard. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-get-the-help-or-explanation-for-the-functions-in-Spark-shell-tp7191p7193.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
