> On Oct. 16, 2018, 1:47 p.m., Sahil Takiar wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java > > Line 352 (original) > > <https://reviews.apache.org/r/69022/diff/2/?file=2097991#file2097991line361> > > > > why remove this? > > denys kuzmenko wrote: > it's not required. close() method is covered with the lock, and > activeJobs is a concurrent collection > > Sahil Takiar wrote: > what happens if a job is submitted after `hasTimedOut` returns true? > > denys kuzmenko wrote: > I see. However existing lock won't help as it doesn't prevent other > threads from adding new queries. > > public void onQuerySubmission(String queryId) { > activeJobs.add(queryId); > } > > we might need to cover this with separate lock (onQueryCompletion, > onQuerySubmission, triggerTimeout) > What do you think? > > denys kuzmenko wrote: > what happens if a job is submitted after hasTimedOut returns true? > current Session will be closed and a new one will be opened. > > denys kuzmenko wrote: > there might be an issue when 2nd session checkes state and get isOpen and > before it's reaching the submit phase, 1st one calls the close. > I think we need synchronization for active sessions manipulation.
fixed. reverted back to the original locking. Above tricky case will be handled by preventing new queries to execute open() before close() is complete. - denys ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69022/#review209617 ----------------------------------------------------------- On Oct. 15, 2018, 7:21 p.m., denys kuzmenko wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69022/ > ----------------------------------------------------------- > > (Updated Oct. 15, 2018, 7:21 p.m.) > > > Review request for hive, Sahil Takiar and Adam Szita. > > > Bugs: HIVE-20737 > https://issues.apache.org/jira/browse/HIVE-20737 > > > Repository: hive-git > > > Description > ------- > > 1. Local SparkContext is shared between user sessions and should be closed > only when there is no active. > 2. Possible race condition in SparkSession.open() in case when user queries > run in parallel within the same session. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java > 72ff53e3bd > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java > bb50129518 > > ql/src/test/org/apache/hadoop/hive/ql/exec/spark/TestLocalHiveSparkClient.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/69022/diff/2/ > > > Testing > ------- > > Added TestLocalHiveSparkClient test > > > File Attachments > ---------------- > > HIVE-20737.7.patch > > https://reviews.apache.org/media/uploaded/files/2018/10/15/9cf8a2b3-9ec1-4316-81d0-3cd124b1a9fd__HIVE-20737.7.patch > > > Thanks, > > denys kuzmenko > >
