We tried but didn’t get much benefits from Python Dataset, as Python is dynamic typed and there is not much we can do to optimize running python functions.
> On 31 May 2017, at 3:36 AM, Cyanny LIANG <lgrcya...@gmail.com> wrote: > > Hi, > Since DataSet API has become a common way to process structured data in spark > 2.0, and Scala , Java API support dataset now, and When will python dataset > API release? or are there some plans? > Consider that, in our production environment, many users love to use python > API, which has many machine learning tools, so python DataSet API will be > very helpful for us. Really looking forward to it. > I searched some jira issues about this: > https://issues.apache.org/jira/browse/SPARK-12776 > <https://issues.apache.org/jira/browse/SPARK-12776> > https://issues.apache.org/jira/browse/SPARK-9999 > <https://issues.apache.org/jira/browse/SPARK-9999> > > And int this blog: > https://databricks.com/blog/2016/01/04/introducing-apache-spark-datasets.html > <https://databricks.com/blog/2016/01/04/introducing-apache-spark-datasets.html> > it said, python API will be supported in spark 2.0 > > -- > Best & Regards > Cyanny LIANG > email: lgrcya...@gmail.com <mailto:lgrcya...@gmail.com>
