Great to know, thanks Xiangrui.
*Sebastián Ramírez* Diseñador de Algoritmos <http://www.senseta.com> ________________ Tel: (+571) 795 7950 ext: 1012 Cel: (+57) 300 370 77 10 Calle 73 No 7 - 06 Piso 4 Linkedin: co.linkedin.com/in/tiangolo/ Twitter: @tiangolo <https://twitter.com/tiangolo> Email: sebastian.rami...@senseta.com www.senseta.com On Tue, Feb 24, 2015 at 1:23 AM, Xiangrui Meng <men...@gmail.com> wrote: > FYI, in 1.3 we support save/load tree models in Scala and Java. We will > add save/load support to Python soon. -Xiangrui > > On Mon, Feb 23, 2015 at 2:57 PM, Sebastián Ramírez < > sebastian.rami...@senseta.com> wrote: > >> In your log it says: >> >> pickle.PicklingError: Can't pickle <type 'thread.lock'>: it's not found >> as thread.lock >> >> As far as I know, you can't pickle Spark models. If you go to the >> documentation for Pickle you can see that you can pickle only simple Python >> structures and code (written in Python), at least as I understand: >> https://docs.python.org/2/library/pickle.html#what-can-be-pickled-and-unpickled >> >> To save a model you can call: model.toDebugString() >> <http://spark.apache.org/docs/latest/api/python/pyspark.mllib.html#pyspark.mllib.tree.DecisionTreeModel.toDebugString> >> >> That gives you a string in pseudo-code that you can save to a file. Then, >> you can parse that pseudo code to write a proper script that runs the >> Decision Tree. Actually, that's what I did for a Random Forest (an ensamble >> of Decision Trees). >> >> Hope that helps, >> >> >> *Sebastián Ramírez* >> Diseñador de Algoritmos >> >> <http://www.senseta.com> >> ________________ >> Tel: (+571) 795 7950 ext: 1012 >> Cel: (+57) 300 370 77 10 >> Calle 73 No 7 - 06 Piso 4 >> Linkedin: co.linkedin.com/in/tiangolo/ >> Twitter: @tiangolo <https://twitter.com/tiangolo> >> Email: sebastian.rami...@senseta.com >> www.senseta.com >> >> On Mon, Feb 23, 2015 at 4:55 AM, Jaggu <jagana...@gmail.com> wrote: >> >>> Hi Team, >>> I was trying to save a DecisionTree model from Pyspark using joblib. >>> It is giving me the following error http://pastebin.com/82CFhPNn . Any >>> clue >>> how to resolve the same or save a model. >>> >>> Best regards >>> >>> Jagan >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/Pyspark-save-Decison-Tree-Module-with-joblib-pickle-tp21765.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> >>> >> >> *----------------------------------------------------* >> *This e-mail transmission, including any attachments, is intended only >> for the named recipient(s) and may contain information that is privileged, >> confidential and/or exempt from disclosure under applicable law. If you >> have received this transmission in error, or are not the named >> recipient(s), please notify Senseta immediately by return e-mail and >> permanently delete this transmission, including any attachments.* >> > > -- *----------------------------------------------------* *This e-mail transmission, including any attachments, is intended only for the named recipient(s) and may contain information that is privileged, confidential and/or exempt from disclosure under applicable law. If you have received this transmission in error, or are not the named recipient(s), please notify Senseta immediately by return e-mail and permanently delete this transmission, including any attachments.*
