Unsubscribe

2023-07-31 Thread Ali Bajwa
Unsubscribe

Re: PySpark: slicing issue with dataframes

2015-05-03 Thread Ali Bajwa
Friendly reminder on this one. Just wanted to get a confirmation that this is not by design before I logged a JIRA Thanks! Ali On Tue, Apr 28, 2015 at 9:53 AM, Ali Bajwa ali.ba...@gmail.com wrote: Hi experts, Trying to use the slicing functionality in strings as part of a Spark program

PySpark: slicing issue with dataframes

2015-04-28 Thread Ali Bajwa
', 'Day']}) a = hc.createDataFrame(A) print A b = a.select(a.Firstname[:2]) print b.toPandas() c = a.select(a.Lastname[2:]) print c.toPandas() Output: Firstname Lastname 0 JamesJones 1 AliBajwa 2Daniel Day SUBSTR(Firstname, 0, 2) 0 Ja 1

Re: Question regarding join with multiple columns with pyspark

2015-04-28 Thread Ali Bajwa
, joiningKey=u'1994~1', mn=1, name=u'AA', yr=1994) Row(joiningKey=u'1993~1', mn=1, price=100, yr=1993, joiningKey=u'1993~1', mn=1, name=u'A', yr=1993) Row(joiningKey=u'2000~1', mn=1, price=1, yr=2000, joiningKey=u'2000~1', mn=1, name=u'AAA', yr=2000) On Sat, Apr 25, 2015 at 10:18 AM, Ali Bajwa

Re: Question regarding join with multiple columns with pyspark

2015-04-24 Thread Ali Bajwa
Any ideas on this? Any sample code to join 2 data frames on two columns? Thanks Ali On Apr 23, 2015, at 1:05 PM, Ali Bajwa ali.ba...@gmail.com wrote: Hi experts, Sorry if this is a n00b question or has already been answered... Am trying to use the data frames API in python to join 2

Question regarding join with multiple columns with pyspark

2015-04-23 Thread Ali Bajwa
Hi experts, Sorry if this is a n00b question or has already been answered... Am trying to use the data frames API in python to join 2 dataframes with more than 1 column. The example I've seen in the documentation only shows a single column - so I tried this: Example code import pandas