Spark and Oozie

2019-07-18 Thread Dennis Suhari
Dear experts, I am using Spark for processing data from HDFS (hadoop). These Spark application are data pipelines, data wrangling and machine learning applications. Thus Spark submits its job using YARN. This also works well. For scheduling I am now trying to use Apache Oozie, but I am facin

Re: In Apache Spark JIRA, spark/dev/github_jira_sync.py not running properly

2019-07-18 Thread Hyukjin Kwon
Hi all, Seems this issue is re-happening again. Seems the PR link is properly created in the corresponding JIRA but it doesn't change the JIRA's status from OPEN to IN-PROGRESS. See, for instance, https://issues.apache.org/jira/browse/SPARK-28443 https://issues.apache.org/jira/browse/SPARK-28440

Using Custom Version of Hive with Spark

2019-07-18 Thread Valeriy Trofimov
Hi All, I've created test tables in HiveCLI (druid1, druid2) and test tables in Beeline (beeline1, beeline2). I want to be able to access Hive tables in Beeline and Beeline tables in Hive. Is it possible to do? I've set up hive-site.xml for both Hive and Spark to use the same warehouse thinking