Re: fetching and joining data from two different clusters

It is a proprietary solution to an open source problem Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw * http://talebzadehmich.wordpress.com

Re: fetching and joining data from two different clusters

Sorry cannot help you there - I do not know the cost for isilon. I also cannot predict what the majority will do ... > On 18. Jun 2017, at 21:49, Mich Talebzadeh wrote: > > thanks Jorn. > > I have been told that Hadoop 3 (alpha testing now) will support Docking and

Re: fetching and joining data from two different clusters

thanks Jorn. I have been told that Hadoop 3 (alpha testing now) will support Docking and virtualised Hadoop clusters Also if we decided to use something like Isolin and blue data to create zoning (meaning two different Hadoop clusters migrated to Isolin storage each residing on its

Re: how to call udf with parameters

What version of spark you are using? I cannot reproduce your error: scala> spark.version res9: String = 2.1.1 scala> val dataset = Seq((0, "hello"), (1, "world")).toDF("id", "text") dataset: org.apache.spark.sql.DataFrame = [id: int, text: string] scala> import org.apache.spark.sql.functions.udf

Re: [Spark Sql/ UDFs] Spark and Hive UDFs parity

I assume you use Scala to implement your UDFs. In this case, Scala language itself provides some options already for you. If you want to control more logic when UDFs init, you can define a Scala object, def your UDF as part of it, then the object in Scala will behavior like Singleton pattern

Unsubscribe

 Thanks & Best Regards, Engr. Palash Gupta Consultant, OSS/CEM/Big Data Skype: palash2494 https://www.linkedin.com/in/enggpalashgupta

the scheme in stream reader

Hi all, L set the scheme for DataStreamReader but when I print the scheme.It just printed: root |--value:string (nullable=true) My code is val line = ss.readStream.format("socket") .option("ip",xxx) .option("port",xxx) .scheme(StructField("name",StringType??::(StructField("age",