Re: [galaxy-dev] latest galaxy-central version

[Robert Baertsch]

Carl,
Thanks for building a great tool. 

Can you fix the library performance problem?

Steps to reproduce:
1. http://su2c-dev.ucsc.edu/library
2. open library TCGA Pancan. 

It takes more than 20 seconds!
-Robert

On Nov 5, 2013, at 1:29 PM, Carl Eberhard  wrote:

> Thanks for the thorough report, Robert.
> 
> I've added some sanity checking to the panel in default:11240:c271cdb443c6
> 
> My only guess as to why the panel javascript is trying to load at all in the 
> library section is a link to the controller/datasets methods (unhide, delete, 
> etc.) that still assume histories are in their own frames. Still 
> investigating.
> 
> 
> On Tue, Nov 5, 2013 at 12:06 PM, Robert Baertsch  
> wrote:
> Carl,
> Just to make sure I just deleted my tree and did a fresh checkout and tried a 
> third time. It happened all three times. I'm using a postgres database and 
> this time I started with a fresh config files and just changed the setting 
> below.
> 
> I didn't see the javascript error in Chrome, perhaps it is some strangeness 
> in Firebug. I've seen strange errors before in Firebug.
> 
> Attached are two screen grab of the lastest try before and after browser 
> refresh.
> 
> changeset:   11227:151b7d3b2f1b
> branch:  stable
> tag: tip
> parent:  11219:5c789ab4144a
> user:Carl Eberhard 
> date:Tue Nov 05 11:33:06 2013 -0500
> summary: History panel: fix naming bug in purge async error handling
> 
> -Robert
> 
> 29c29
> < port = 8585
> ---
> > #port = 8080
> 34c34
> < host = 0.0.0.0
> ---
> > #host = 127.0.0.1
> 96d95
> < database_connection = postgresql://localhost/medbookgalaxycentral3
> 542c541
> < allow_library_path_paste = True
> ---
> > #allow_library_path_paste = False
> 599c598
> < admin_users = baert...@soe.ucsc.edu
> ---
> > #admin_users = None
> 
> 
> 
> 
> 
> On Nov 5, 2013, at 8:21 AM, Carl Eberhard  wrote:
> 
>> Hello, Robert
>> 
>> I'm having difficulty reproducing this with a fresh install of 
>> galaxy_central:default:11226:c67b9518c1e0.
>> 
>> Is this an intermittent error or does it happen reliably each time with the 
>> steps above? Is it still the same javascript error you mentioned above?
>> 
>> I'll investigate further.
>> 
>> 
>> On Mon, Nov 4, 2013 at 7:29 PM, Robert Baertsch  
>> wrote:
>> I updated to the stable release and reproduced the issue.
>> 
>> Step to reproduce
>> 1. go to admin
>> 2. Manage data libraries
>> 3. add dataset
>> 4. select Upload files from filesystem paths
>> 5. paste full path to any bam file.
>> 6. leave defaults: auto-detect and copy files into galaxy
>> 7. select role to restrict access
>> 8. click upload to start
>> 
>> Screen shows strange formatting and "Job is running" for Bam file.
>> 
>> python /data/galaxy-central/tools/data_source/upload.py /data/galaxy-central 
>> /data/galaxy-central/database/tmp/tmpJoasJl 
>> /data/galaxy-central/database/tmp/tmpzZO8t1 
>> 8877:/data/galaxy-central/database/job_working_directory/004/4548/dataset_8877_files:/data/galaxy-central/database/files/008/dataset_8877.dat
>> 
>> If I do a firefox refresh and go back to the library, the formatting is 
>> normal.
>> 
>> I'm assuming the fix is to just render the page before waiting for the job 
>> to complete.
>> -Robert
>> 
>> On Nov 4, 2013, at 12:45 PM, Martin Čech  wrote:
>> 
>>> Hello,
>>> 
>>> I have also seen some of these errors while developing libraries. The 
>>> library code is not in central however it might be related to recent 
>>> changes to the history panel. Carl Eberhard might now more, adding him to 
>>> the conversation.
>>> 
>>> --Marten
>>> 
>>> 
>>> On Mon, Nov 4, 2013 at 2:45 PM, Robert Baertsch  
>>> wrote:
>>> It keeps doing posts, and I'm not seeing any new errors. 
>>> 
>>> POST http://su2c-dev.ucsc.edu:8383/library_common/library_item_updates
>>> 200 OK   121ms
>>> 
>>> When I did a browser refresh, I got the following javascript error: (I 
>>> am logged in)
>>> 
>>> Galaxy.currUser is undefined on Line 631 in history-panel.js
>>> 
>>> 
>>> When I opened the data library where the bam file was copying, everything 
>>> is rendered ok.  It seems the browser refresh fixed things.
>>> 
>>> -Robert
>>> 
>>> 
>>> On Nov 4, 2013, at 11:14 AM, James Taylor  wrote:
>>> 
 Robert, I'm not sure what is going on here, other than that the javascript 
 that converts buttons into dropdown menus has not fired. 
 
 Are there any javascript errors? 
 
 Marten is working on rewriting libraries, and we will be eliminating the 
 progressive loading popupmenus for something much more efficient, but this 
 also might indicate a bug so let us know if there is anything odd in the 
 console. 
 
 
 --
 James Taylor, Associate Professor, Biology/CS, Emory University
 
 
 On Mon, Nov 4, 2013 at 1:58 PM, Robert Baertsch  
 wrote:
 HI James,
 I just pulled in the latest code to see how you changed from iframe to 
 divs. Very exciting update.
>>>

Re: [galaxy-dev] latest galaxy-central version

[Martin Čech]

Hi Robert,

We are currently working on improving data libraries including the speed
enhancements. In the meantime one of our users also reported that he is
using for the dataset folders multiple libraries - each having less
folder-depth which for him improves performance significantly. Maybe it
will be of use to you.

--Martin


On Tue, Nov 5, 2013 at 5:19 PM, Robert Baertsch
wrote:

> Carl,
> Thanks for building a great tool.
>
> Can you fix the library performance problem?
>
> Steps to reproduce:
> 1. http://su2c-dev.ucsc.edu/library
> 2. open library TCGA Pancan.
>
> It takes more than 20 seconds!
> -Robert
>
> On Nov 5, 2013, at 1:29 PM, Carl Eberhard  wrote:
>
> Thanks for the thorough report, Robert.
>
> I've added some sanity checking to the panel in default:11240:c271cdb443c6
>
> My only guess as to why the panel javascript is trying to load at all in
> the library section is a link to the controller/datasets methods (unhide,
> delete, etc.) that still assume histories are in their own frames. Still
> investigating.
>
>
> On Tue, Nov 5, 2013 at 12:06 PM, Robert Baertsch wrote:
>
>> Carl,
>> Just to make sure I just deleted my tree and did a fresh checkout and
>> tried a third time. It happened all three times. I'm using a postgres
>> database and this time I started with a fresh config files and just changed
>> the setting below.
>>
>> I didn't see the javascript error in Chrome, perhaps it is some
>> strangeness in Firebug. I've seen strange errors before in Firebug.
>>
>> Attached are two screen grab of the lastest try before and after browser
>> refresh.
>>
>> changeset:   11227:151b7d3b2f1b
>> branch:  stable
>> tag: tip
>> parent:  11219:5c789ab4144a
>> user:Carl Eberhard 
>> date:Tue Nov 05 11:33:06 2013 -0500
>> summary: History panel: fix naming bug in purge async error handling
>>
>> -Robert
>>
>> 29c29
>> < port = 8585
>> ---
>> > #port = 8080
>> 34c34
>> < host = 0.0.0.0
>> ---
>> > #host = 127.0.0.1
>> 96d95
>> < database_connection = postgresql://localhost/medbookgalaxycentral3
>> 542c541
>> < allow_library_path_paste = True
>> ---
>> > #allow_library_path_paste = False
>> 599c598
>> < admin_users = baert...@soe.ucsc.edu
>> ---
>> > #admin_users = None
>>
>>
>> 
>> 
>>
>> On Nov 5, 2013, at 8:21 AM, Carl Eberhard 
>> wrote:
>>
>> Hello, Robert
>>
>> I'm having difficulty reproducing this with a fresh install of
>> galaxy_central:default:11226:c67b9518c1e0.
>>
>> Is this an intermittent error or does it happen reliably each time with
>> the steps above? Is it still the same javascript error you mentioned above?
>>
>> I'll investigate further.
>>
>>
>> On Mon, Nov 4, 2013 at 7:29 PM, Robert Baertsch wrote:
>>
>>> I updated to the stable release and reproduced the issue.
>>>
>>> Step to reproduce
>>> 1. go to admin
>>> 2. Manage data 
>>> libraries
>>> 3. add dataset
>>> 4. select Upload files from filesystem paths
>>> 5. paste full path to any bam file.
>>> 6. leave defaults: auto-detect and copy files into galaxy
>>> 7. select role to restrict access
>>> 8. click upload to start
>>>
>>> Screen shows strange formatting and "Job is running" for Bam file.
>>>
>>> python /data/galaxy-central/tools/data_source/upload.py
>>> /data/galaxy-central /data/galaxy-central/database/tmp/tmpJoasJl
>>> /data/galaxy-central/database/tmp/tmpzZO8t1
>>> 8877:/data/galaxy-central/database/job_working_directory/004/4548/dataset_8877_files:/data/galaxy-central/database/files/008/dataset_8877.dat
>>>
>>> If I do a firefox refresh and go back to the library, the formatting is
>>> normal.
>>>
>>> I'm assuming the fix is to just render the page before waiting for the
>>> job to complete.
>>> -Robert
>>>
>>> On Nov 4, 2013, at 12:45 PM, Martin Čech  wrote:
>>>
>>> Hello,
>>>
>>> I have also seen some of these errors while developing libraries. The
>>> library code is not in central however it might be related to recent
>>> changes to the history panel. Carl Eberhard might now more, adding him to
>>> the conversation.
>>>
>>> --Marten
>>>
>>>
>>> On Mon, Nov 4, 2013 at 2:45 PM, Robert Baertsch 
>>> wrote:
>>>
 It keeps doing posts, and I'm not seeing any new errors.

 POST http://su2c-dev.ucsc.edu:8383/library_common/library_item_updates
 200 OK 121ms
 When I did a browser refresh, I got the following javascript error:
 (I am logged in)

  Galaxy.currUser is undefined on Line 631 in history-panel.js


 When I opened the data library where the bam file was copying,
 everything is rendered ok.  It seems the browser refresh fixed things.

 -Robert


 On Nov 4, 2013, at 11:14 AM, James Taylor 
 wrote:

 Robert, I'm not sure what is going on here, other than that the
 javascript that converts buttons into dropdown menus has not fired.

 Are there any javascript errors?

 Marten is working on rewriting libraries, and we will 

Re: [galaxy-dev] latest galaxy-central version

[Carl Eberhard]

Hello, Robert

I'm having difficulty reproducing this with a fresh install of
galaxy_central:default:11226:c67b9518c1e0.

Is this an intermittent error or does it happen reliably each time with the
steps above? Is it still the same javascript error you mentioned above?

I'll investigate further.


On Mon, Nov 4, 2013 at 7:29 PM, Robert Baertsch wrote:

> I updated to the stable release and reproduced the issue.
>
> Step to reproduce
> 1. go to admin
> 2. Manage data 
> libraries
> 3. add dataset
> 4. select Upload files from filesystem paths
> 5. paste full path to any bam file.
> 6. leave defaults: auto-detect and copy files into galaxy
> 7. select role to restrict access
> 8. click upload to start
>
> Screen shows strange formatting and "Job is running" for Bam file.
>
> python /data/galaxy-central/tools/data_source/upload.py
> /data/galaxy-central /data/galaxy-central/database/tmp/tmpJoasJl
> /data/galaxy-central/database/tmp/tmpzZO8t1
> 8877:/data/galaxy-central/database/job_working_directory/004/4548/dataset_8877_files:/data/galaxy-central/database/files/008/dataset_8877.dat
>
> If I do a firefox refresh and go back to the library, the formatting is
> normal.
>
> I'm assuming the fix is to just render the page before waiting for the job
> to complete.
> -Robert
>
> On Nov 4, 2013, at 12:45 PM, Martin Čech  wrote:
>
> Hello,
>
> I have also seen some of these errors while developing libraries. The
> library code is not in central however it might be related to recent
> changes to the history panel. Carl Eberhard might now more, adding him to
> the conversation.
>
> --Marten
>
>
> On Mon, Nov 4, 2013 at 2:45 PM, Robert Baertsch wrote:
>
>> It keeps doing posts, and I'm not seeing any new errors.
>>
>> POST http://su2c-dev.ucsc.edu:8383/library_common/library_item_updates
>> 200 OK 121ms
>> When I did a browser refresh, I got the following javascript error:
>> (I am logged in)
>>
>>  Galaxy.currUser is undefined on Line 631 in history-panel.js
>>
>>
>> When I opened the data library where the bam file was copying, everything
>> is rendered ok.  It seems the browser refresh fixed things.
>>
>> -Robert
>>
>>
>> On Nov 4, 2013, at 11:14 AM, James Taylor  wrote:
>>
>> Robert, I'm not sure what is going on here, other than that the
>> javascript that converts buttons into dropdown menus has not fired.
>>
>> Are there any javascript errors?
>>
>> Marten is working on rewriting libraries, and we will be eliminating the
>> progressive loading popupmenus for something much more efficient, but this
>> also might indicate a bug so let us know if there is anything odd in the
>> console.
>>
>>
>> --
>> James Taylor, Associate Professor, Biology/CS, Emory University
>>
>>
>> On Mon, Nov 4, 2013 at 1:58 PM, Robert Baertsch wrote:
>>
>>> HI James,
>>> I just pulled in the latest code to see how you changed from iframe to
>>> divs. Very exciting update.
>>>
>>>
>>> I tried importing a bam file into the library using the admin tool that
>>> reads the file directly from a path on the server.
>>>
>>> During the copy operation, the screen gets into a weird state.
>>> ps -ef shows these two processes running:
>>>  python /data/galaxy-central/tools/data_source/upload.py
>>> /data/galaxy-central /data/galaxy-central/database/tmp/tmpywxnFE
>>> /data/galaxy-central/database/tmp/tmpXkEvSY
>>> 8876:/data/galaxy-central/database/job_working_directory/004/4547/dataset_8876_files:/data/galaxy-central/database/files/008/dataset_8876.dat
>>>  samtools sort /data/galaxy-central/database/files/008/dataset_8876.dat
>>> /tmp/tmpNbcggh/sorted
>>>
>>> Do you finish rendering the screen after the copy operation is complete?
>>>
>>> -Robert
>>>
>>> 
>>>
>>
>>
>>
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   http://lists.bx.psu.edu/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] latest galaxy-central version

[Robert Baertsch]

I updated to the stable release and reproduced the issue.

Step to reproduce
1. go to admin
2. Manage data libraries
3. add dataset
4. select Upload files from filesystem paths
5. paste full path to any bam file.
6. leave defaults: auto-detect and copy files into galaxy
7. select role to restrict access
8. click upload to start

Screen shows strange formatting and "Job is running" for Bam file.

python /data/galaxy-central/tools/data_source/upload.py /data/galaxy-central 
/data/galaxy-central/database/tmp/tmpJoasJl 
/data/galaxy-central/database/tmp/tmpzZO8t1 
8877:/data/galaxy-central/database/job_working_directory/004/4548/dataset_8877_files:/data/galaxy-central/database/files/008/dataset_8877.dat

If I do a firefox refresh and go back to the library, the formatting is normal.

I'm assuming the fix is to just render the page before waiting for the job to 
complete.
-Robert

On Nov 4, 2013, at 12:45 PM, Martin Čech  wrote:

> Hello,
> 
> I have also seen some of these errors while developing libraries. The library 
> code is not in central however it might be related to recent changes to the 
> history panel. Carl Eberhard might now more, adding him to the conversation.
> 
> --Marten
> 
> 
> On Mon, Nov 4, 2013 at 2:45 PM, Robert Baertsch  wrote:
> It keeps doing posts, and I'm not seeing any new errors. 
> 
> POST http://su2c-dev.ucsc.edu:8383/library_common/library_item_updates
> 200 OK 121ms
> 
> When I did a browser refresh, I got the following javascript error: (I am 
> logged in)
> 
> Galaxy.currUser is undefined on Line 631 in history-panel.js
> 
> 
> When I opened the data library where the bam file was copying, everything is 
> rendered ok.  It seems the browser refresh fixed things.
> 
> -Robert
> 
> 
> On Nov 4, 2013, at 11:14 AM, James Taylor  wrote:
> 
>> Robert, I'm not sure what is going on here, other than that the javascript 
>> that converts buttons into dropdown menus has not fired. 
>> 
>> Are there any javascript errors? 
>> 
>> Marten is working on rewriting libraries, and we will be eliminating the 
>> progressive loading popupmenus for something much more efficient, but this 
>> also might indicate a bug so let us know if there is anything odd in the 
>> console. 
>> 
>> 
>> --
>> James Taylor, Associate Professor, Biology/CS, Emory University
>> 
>> 
>> On Mon, Nov 4, 2013 at 1:58 PM, Robert Baertsch  
>> wrote:
>> HI James,
>> I just pulled in the latest code to see how you changed from iframe to divs. 
>> Very exciting update.
>> 
>> 
>> I tried importing a bam file into the library using the admin tool that 
>> reads the file directly from a path on the server.
>> 
>> During the copy operation, the screen gets into a weird state. 
>> ps -ef shows these two processes running:
>>  python /data/galaxy-central/tools/data_source/upload.py 
>> /data/galaxy-central /data/galaxy-central/database/tmp/tmpywxnFE 
>> /data/galaxy-central/database/tmp/tmpXkEvSY 
>> 8876:/data/galaxy-central/database/job_working_directory/004/4547/dataset_8876_files:/data/galaxy-central/database/files/008/dataset_8876.dat
>>  samtools sort /data/galaxy-central/database/files/008/dataset_8876.dat 
>> /tmp/tmpNbcggh/sorted
>> 
>> Do you finish rendering the screen after the copy operation is complete?
>> 
>> -Robert
>> 
>> 
>> 
> 
> 
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
> 
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
> 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] latest galaxy-central version

[Martin Čech]

Hello,

I have also seen some of these errors while developing libraries. The
library code is not in central however it might be related to recent
changes to the history panel. Carl Eberhard might now more, adding him to
the conversation.

--Marten


On Mon, Nov 4, 2013 at 2:45 PM, Robert Baertsch wrote:

> It keeps doing posts, and I'm not seeing any new errors.
>
> POST http://su2c-dev.ucsc.edu:8383/library_common/library_item_updates
> 200 OK
> 121ms
> When I did a browser refresh, I got the following javascript error: (I
> am logged in)
>
> Galaxy.currUser is undefined on Line 631 in history-panel.js
>
>
> When I opened the data library where the bam file was copying, everything
> is rendered ok.  It seems the browser refresh fixed things.
>
> -Robert
>
>
> On Nov 4, 2013, at 11:14 AM, James Taylor  wrote:
>
> Robert, I'm not sure what is going on here, other than that the javascript
> that converts buttons into dropdown menus has not fired.
>
> Are there any javascript errors?
>
> Marten is working on rewriting libraries, and we will be eliminating the
> progressive loading popupmenus for something much more efficient, but this
> also might indicate a bug so let us know if there is anything odd in the
> console.
>
>
> --
> James Taylor, Associate Professor, Biology/CS, Emory University
>
>
> On Mon, Nov 4, 2013 at 1:58 PM, Robert Baertsch wrote:
>
>> HI James,
>> I just pulled in the latest code to see how you changed from iframe to
>> divs. Very exciting update.
>>
>>
>> I tried importing a bam file into the library using the admin tool that
>> reads the file directly from a path on the server.
>>
>> During the copy operation, the screen gets into a weird state.
>> ps -ef shows these two processes running:
>>  python /data/galaxy-central/tools/data_source/upload.py
>> /data/galaxy-central /data/galaxy-central/database/tmp/tmpywxnFE
>> /data/galaxy-central/database/tmp/tmpXkEvSY
>> 8876:/data/galaxy-central/database/job_working_directory/004/4547/dataset_8876_files:/data/galaxy-central/database/files/008/dataset_8876.dat
>>  samtools sort /data/galaxy-central/database/files/008/dataset_8876.dat
>> /tmp/tmpNbcggh/sorted
>>
>> Do you finish rendering the screen after the copy operation is complete?
>>
>> -Robert
>>
>> 
>>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] latest galaxy-central version

[Robert Baertsch]

It keeps doing posts, and I'm not seeing any new errors. 

POST http://su2c-dev.ucsc.edu:8383/library_common/library_item_updates
200 OK
121ms

When I did a browser refresh, I got the following javascript error: (I am 
logged in)

Galaxy.currUser is undefined on Line 631 in history-panel.js


When I opened the data library where the bam file was copying, everything is 
rendered ok.  It seems the browser refresh fixed things.

-Robert


On Nov 4, 2013, at 11:14 AM, James Taylor  wrote:

> Robert, I'm not sure what is going on here, other than that the javascript 
> that converts buttons into dropdown menus has not fired. 
> 
> Are there any javascript errors? 
> 
> Marten is working on rewriting libraries, and we will be eliminating the 
> progressive loading popupmenus for something much more efficient, but this 
> also might indicate a bug so let us know if there is anything odd in the 
> console. 
> 
> 
> --
> James Taylor, Associate Professor, Biology/CS, Emory University
> 
> 
> On Mon, Nov 4, 2013 at 1:58 PM, Robert Baertsch  wrote:
> HI James,
> I just pulled in the latest code to see how you changed from iframe to divs. 
> Very exciting update.
> 
> 
> I tried importing a bam file into the library using the admin tool that reads 
> the file directly from a path on the server.
> 
> During the copy operation, the screen gets into a weird state. 
> ps -ef shows these two processes running:
>  python /data/galaxy-central/tools/data_source/upload.py /data/galaxy-central 
> /data/galaxy-central/database/tmp/tmpywxnFE 
> /data/galaxy-central/database/tmp/tmpXkEvSY 
> 8876:/data/galaxy-central/database/job_working_directory/004/4547/dataset_8876_files:/data/galaxy-central/database/files/008/dataset_8876.dat
>  samtools sort /data/galaxy-central/database/files/008/dataset_8876.dat 
> /tmp/tmpNbcggh/sorted
> 
> Do you finish rendering the screen after the copy operation is complete?
> 
> -Robert
> 
> 
> 

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/