[ https://issues.apache.org/jira/browse/LIVY-368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16222678#comment-16222678 ]
Devin Boyer commented on LIVY-368: ---------------------------------- This issue still seems to occur in some situations in Livy 0.4. Adding my case as a comment, but if it seems more appropriate as a separate bug, I can open a new bug as well. I have a simple job I submitted as a batch session that adds a partition to a Hive table: {{from pyspark.sql import SparkSession spark = SparkSession.builder.enableHiveSupport().getOrCreate() spark.sql("ALTER TABLE mytable ADD IF NOT EXISTS PARTITION (partition_info) location 's3://mydatalocation/on/s3'") spark.stop() }} Without the spark.stop() line, the YARN application continues on forever (state = RUNNING, looking at the ResourceManager output) and the Livy state for the batch session stays as 'started'. When I add the spark.stop() line, the application does get marked as FINISHED by YARN, but Livy still reports the batch session as 'started'. > Livy batch state is not being updated properly. > ----------------------------------------------- > > Key: LIVY-368 > URL: https://issues.apache.org/jira/browse/LIVY-368 > Project: Livy > Issue Type: Bug > Components: Batch > Affects Versions: 0.3 > Reporter: Sneha Hegde > > I tried using an Http client to connect to livy server and work in batch > mode, it works fine. But I don't see the status being updated properly. It > always shows the status as "starting" even when it is running in batch mode. -- This message was sent by Atlassian JIRA (v6.4.14#64029)