RE: [SparkR] creating dataframe from json file

2015-07-15 Thread Sun, Rui
is not so complete. You may use scala documentation as reference, and try if some method is supported in SparkR. From: jianshu Weng [jian...@gmail.com] Sent: Wednesday, July 15, 2015 9:37 PM To: Sun, Rui Cc: user@spark.apache.org Subject: Re: [SparkR] creating dataframe

RE: [SparkR] creating dataframe from json file

2015-07-15 Thread Sun, Rui
suppose df - jsonFile(sqlContext, json file) You can extract hashtags.text as a Column object using the following command: t - getField(df$hashtags, text) and then you can perform operations on the column. You can extract hashtags.text as a DataFrame using the following command: t -

Re: [SparkR] creating dataframe from json file

2015-07-15 Thread jianshu Weng
Thanks. t - getField(df$hashtags, text) does return a Column. But when I tried to call t - getField(df$hashtags, text), it would give an error: Error: All select() inputs must resolve to integer column positions. The following do not: * getField(df$hashtags, text) In fact, the text field in df