RE: [Spark SQL]: Slow insertInto overwrite if target table has many partitions

Hi, How do I get the filename from textFileStream Using streaming. Thanks a mill Standard Bank email disclaimer and confidentiality note Please go to www.standardbank.co.za/site/homepage/emaildisclaimer.html to read our email disclaimer and confidentiality note. Kindly email

RE: Dose pyspark supports python3.6？

Dear Spark users I have been asked to provide a presentation / business case as to why to use spark and java as ingestion tool for HDFS and HIVE And why to move away from an etl tool. Could you be so kind as to provide with some pros and cons to this. I have the following : Pros: In house

RE: Is Spark suited for this use case?

Hi, We basically have the same scenario but worldwide as we have bigger Datasets we use OGG --> local --> Sqoop Into Hadoop. By all means you can have spark reading the oracle tables and then do some changes to data in need which will not be done on scoop qry. Ie fraudulent detection on

Re: TTransportException when using Spark 1.6.0 on top of Tachyon 0.8.2

Hey, Jia Zou I'm curious about this exception, the error log you showed that the exception is related to unlockBlock, could you upload your full master.log and worker.log under tachyon/logs directory? Best, Cheng 在 2016年1月29日星期五 UTC+8上午11:11:19，Calvin Jia写道： > > Hi, > > Thanks for the