You can't use df as the name of the return from the try and the name of the match variable in success. You also probably want to match the name of the variable in the match with the return from the match.
So val df = Try(spark.read. format("jdbc"). option("url", jdbcUrl). option("dbtable", HiveSchema+"."+HiveTable). option("user", HybridServerUserName). option("password", HybridServerPassword). load()) match { * case Success(validDf) => validDf* case Failure(e) => throw new Exception("Error Encountered reading Hive table") } On Thu, Oct 1, 2020 at 5:53 PM Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > > Many thanks SEan. > > > Maybe I misunderstood your point? > > > var DF = Try(spark.read. > > format("jdbc"). > > option("url", jdbcUrl). > > option("dbtable", HiveSchema+"."+HiveTable). > > option("user", HybridServerUserName). > > option("password", HybridServerPassword). > > load()) match { > > * case Success(DF) => HiveDF* > > case Failure(e) => throw new Exception("Error > Encountered reading Hive table") > > } > > Still getting the error > > > <console>:74: error: recursive method DF needs type > > case Success(DF) => HiveDF > > Do I need to define DF as DataFrame beforehand because at that moment > Spark does not know what DF type is > > Thanks again > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Thu, 1 Oct 2020 at 23:08, Sean Owen <sro...@gmail.com> wrote: > >> You are reusing HiveDF for two vars and it ends up ambiguous. Just rename >> one. >> >> On Thu, Oct 1, 2020, 5:02 PM Mich Talebzadeh <mich.talebza...@gmail.com> >> wrote: >> >>> Hi, >>> >>> >>> Spark version 2.3.3 on Google Dataproc >>> >>> >>> I am trying to use databricks to other databases >>> >>> >>> https://spark.apache.org/docs/latest/sql-data-sources-jdbc.html >>> >>> >>> to read from Hive table on Prem using Spark in Cloud >>> >>> >>> This works OK without a Try enclosure. >>> >>> >>> import spark.implicits._ >>> >>> import scala.util.{Try, Success, Failure} >>> >>> val HiveDF = Try(spark.read. >>> >>> format("jdbc"). >>> >>> option("url", jdbcUrl). >>> >>> option("dbtable", HiveSchema+"."+HiveTable). >>> >>> option("user", HybridServerUserName). >>> >>> option("password", HybridServerPassword). >>> >>> load()) match { >>> >>> case Success(HiveDF) => HiveDF >>> >>> case Failure(e) => throw new Exception("Error >>> Encountered reading Hive table") >>> >>> } >>> >>> However, with Try I am getting the following error >>> >>> >>> <console>:66: error: recursive value HiveDF needs type >>> >>> case Success(HiveDF) => HiveDF >>> >>> Wondering what is causing this. I have used it before (say reading from >>> an XML file) and it worked the, >>> >>> Thanks >>> >>> >>> >>> >>> >>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>> any loss, damage or destruction of data or any other property which may >>> arise from relying on this email's technical content is explicitly >>> disclaimed. The author will in no case be liable for any monetary damages >>> arising from such loss, damage or destruction. >>> >>> >>> >>