Code stops working after a few executions

2019-04-23 Thread Peter Wicks (pwicks)
I am working in Livy v0.4 with Python.  I'm using sessions.  If I run the same 
Python code over and over again it will work around four times, then on the 
next time I get the error:

Unexpected character ('#' (code 35)): expected a valid value (number, String, 
array, object, 'true', 'false' or 'null')
at [Source: #; line: 1, column: 2]

There aren't any #'s in my code, and the code is identical between runs 
anyways... I made it identical following user reports of runs failing with this 
error message to try and reproduce the error.

Once this error starts appearing the session is useless, running code on it 
again does not succeed, though the status still shows as idle.

Any ideas on what might be going on here?

Thanks!
  Peter


Accessing Detailed Livy Session Information (session name?)

2019-04-12 Thread Peter Wicks (pwicks)
Greetings,

I have a custom service that connects to Livy, v0.4 soon to be v0.5 once we go 
to HDP3. If sessions already exist it logs the session ID's and starts using 
them, if sessions don't exist it creates new ones. The problem is the account 
used to launch the Livy sessions is not unique to this service, nor is the kind 
of session. So sometimes it grabs other people's sessions and absconds off with 
them. Also, there are multiple instances of the service, running under the same 
account, and they are not supposed to use each other's sessions... that's not 
working out so well.

The service names the sessions, but I can't find any way to retrieve detailed 
session data so that I can update the service to check if the Livy Session 
belongs to the service or not.

I found some older comments 2016/2017 about retrieving Livy sessions by name. I 
don't really need that, I just want to be able to read the name through the 
regular sessions REST call.

Any REST calls I missed, or undocumented calls... that can help?

Thanks,
  Peter

Ref: 
https://github.com/meisam/livy/wiki/Design-doc-for-Livy-41:-Accessing-sessions-by-name,
 https://issues.cloudera.org/browse/LIVY-41