[ https://issues.apache.org/jira/browse/LIVY-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Saisai Shao closed LIVY-428. ---------------------------- Resolution: Duplicate > Does livy support concurrency for centain spark session? > -------------------------------------------------------- > > Key: LIVY-428 > URL: https://issues.apache.org/jira/browse/LIVY-428 > Project: Livy > Issue Type: Bug > Components: API > Affects Versions: 0.4 > Environment: livy 0.4 > spark2.2.0-cloudera1 > Reporter: suheng.cloud > Attachments: spark-fair.png > > > To use spark sql concurrency for internal fair schedule > in [post] /sessions/{id}/statements use param "conf": > {"spark.scheduler.mode":"FAIR","spark.scheduler.allocation.file":"/home/.../spark-schedule.xml"} > spark-schedule.xml is as follows: > <allocations> > <pool name="FAIR"> > <schedulingMode>FAIR</schedulingMode> > </pool> > </allocations> > Log shows tow jobs surely be added into fair pool: > INFO scheduler.FairSchedulableBuilder: Added task set TaskSet_10.0 tasks to > pool FAIR > ... > INFO scheduler.FairSchedulableBuilder: Added task set TaskSet_11.0 tasks to > pool FAIR > But from spark web console the second job starts only when the first finished. > I also tried with zeppelin, which concurrency works well with spark but not > work in livy. > Did I miss some configuration? Thanks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)