[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Victor Grousset/tuxayo  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|Pushed to oldstable |RESOLVED
 CC||vic...@tuxayo.net

--- Comment #54 from Victor Grousset/tuxayo  ---
Not backported to oldoldstable (19.05.x). Feel free to ask if it's needed.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Aleisha Amohia  changed:

   What|Removed |Added

 Version(s)|20.11.00, 20.05.06  |20.11.00, 20.05.06,
released in||19.11.12
 Status|Pushed to stable|Pushed to oldstable

--- Comment #53 from Aleisha Amohia  ---
backported to 19.11.x for 19.11.12

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Aleisha Amohia  changed:

   What|Removed |Added

 Status|Pushed to master|Pushed to stable
 CC||alei...@catalyst.net.nz

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #52 from David Cook  ---
Using CGI::Emulate::PSGI::Streaming (version 1.0.1 since 1.0.0 is buggy) and a
custom Plack::Component I created based on Plack::App::WrapCGI, I was able to
stream 65MB of bib records from /cgi-bin/koha/tools/export.pl in 13 minutes
using koha-testing-docker (and an external database on a different server). 

I could explore a solution using that tech, although I think that I'm still
more inclined to moving forward with Bug 26791, as replacing export.pl provides
a good example for replacing CGI scripts with Mojolicious controllers.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Lucas Gass  changed:

   What|Removed |Added

 CC||lu...@bywatersolutions.com
 Version(s)|20.11.00|20.11.00, 20.05.06
released in||

--- Comment #51 from Lucas Gass  ---
backported to 20.05.x for 20.05.06

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #50 from David Cook  ---
Hurray!

I won't have work time to spend on Bug 26791, but sometime soon I could share
the work I did to get it working in Mojolicious. It'll need a fair bit more
work to be production ready, but it could be the path forward for fixing all
these proxy exceptions.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #49 from Jonathan Druart  
---
Pushed to master for 20.11, thanks to everybody involved!

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Jonathan Druart  changed:

   What|Removed |Added

 Version(s)||20.11.00
released in||
 Status|Passed QA   |Pushed to master

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Katrin Fischer  changed:

   What|Removed |Added

 Attachment #112224|0   |1
is obsolete||

--- Comment #48 from Katrin Fischer  ---
Created attachment 112448
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=112448=edit
Bug 8437: Exclude export.pl from plack

When attempting to download large files from Koha plack can timeout

Excluding the script from plack is a simple fix until we have a more permanent
fix for this
issue.

To test:
1 - Try to export your entire DB from Tools->Export
2 - If big enough, it fails
3 - Apply patch, copy changes to /etc/koha/apache-shared-intranet-plack.conf
4 - Restart all the things
5 - Repeat export, it succeeds

Signed-off-by: David Cook 

Signed-off-by: Katrin Fischer 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Katrin Fischer  changed:

   What|Removed |Added

   Patch complexity|--- |Trivial patch
 Status|Signed Off  |Passed QA

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Katrin Fischer  changed:

   What|Removed |Added

 QA Contact||katrin.fisc...@bsz-bw.de

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Nick Clemens  changed:

   What|Removed |Added

   Assignee|gmcha...@gmail.com  |n...@bywatersolutions.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #47 from David Cook  ---
Side note: It's harder to make this work in Mojolicious than I thought it
would, but only because Mojolicious is non-blocking by design. I think I should
be able to create something that should work quite well in the end...

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

David Cook  changed:

   What|Removed |Added

   See Also||https://bugs.koha-community
   ||.org/bugzilla3/show_bug.cgi
   ||?id=26791

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

David Cook  changed:

   What|Removed |Added

 Attachment #109632|0   |1
is obsolete||

--- Comment #46 from David Cook  ---
Created attachment 112224
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=112224=edit
Bug 8437: Exclude export.pl from plack

When attempting to download large files from Koha plack can timeout

Excluding the script from plack is a simple fix until we have a more permanent
fix for this
issue.

To test:
1 - Try to export your entire DB from Tools->Export
2 - If big enough, it fails
3 - Apply patch, copy changes to /etc/koha/apache-shared-intranet-plack.conf
4 - Restart all the things
5 - Repeat export, it succeeds

Signed-off-by: David Cook 

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

David Cook  changed:

   What|Removed |Added

 Status|Needs Signoff   |Signed Off

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #45 from David Cook  ---
(In reply to David Cook from comment #44)
> (In reply to Nick Clemens from comment #30)
> > Created attachment 109632 [details] [review] [review]
> > Bug 8437: Exclude export.pl from plack
> > 
> > When attempting to download large files from Koha plack can timeout
> > 
> > Excluding the script from plack is a simple fix until we have a more
> > permanent fix for this
> > issue.
> > 
> > To test:
> > 1 - Try to export your entire DB from Tools->Export
> > 2 - If big enough, it fails
> > 3 - Apply patch, copy changes to /etc/koha/apache-shared-intranet-plack.conf
> > 4 - Restart all the things
> > 5 - Repeat export, it succeeds
> 
> My test plan:
> 0. Using koha-testing-docker
> 1. cp debian/templates/apache-shared-intranet-plack.conf
> /etc/koha/apache-shared-intranet-plack.conf
> 2. enable "backup_db_via_tools" in /etc/koha/sites/kohadev/koha-conf.xml
> 3. echo 'flush_all' | nc -q 1 memcached 11211
> 4. service apache2 restart
> 5. koha-plack --restart kohadev
> 6. Go to http://localhost:8081/cgi-bin/koha/tools/export.pl
> 7. Note "Unfortunately, no backups are available."
> 
> So that's not going to work... but I have another idea.

My test plan:
0. Using koha-testing-docker
1. cp debian/templates/apache-shared-intranet-plack.conf
/etc/koha/apache-shared-intranet-plack.conf
2. Point koha-testing-docker at a database with a large number of biblios (e.g.
40,000 biblios)
3. echo 'flush_all' | nc -q 1 memcached 11211
4. restart_all
5. Go to http://localhost:8081/cgi-bin/koha/tools/export.pl
6. Click "Don't export items"
7. Click "Export bibliographic records"
8. Note after about 5 seconds that a download for koha.mrc starts. 

Perfect. Cheers, Nick.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #44 from David Cook  ---
(In reply to Nick Clemens from comment #30)
> Created attachment 109632 [details] [review]
> Bug 8437: Exclude export.pl from plack
> 
> When attempting to download large files from Koha plack can timeout
> 
> Excluding the script from plack is a simple fix until we have a more
> permanent fix for this
> issue.
> 
> To test:
> 1 - Try to export your entire DB from Tools->Export
> 2 - If big enough, it fails
> 3 - Apply patch, copy changes to /etc/koha/apache-shared-intranet-plack.conf
> 4 - Restart all the things
> 5 - Repeat export, it succeeds

My test plan:
0. Using koha-testing-docker
1. cp debian/templates/apache-shared-intranet-plack.conf
/etc/koha/apache-shared-intranet-plack.conf
2. enable "backup_db_via_tools" in /etc/koha/sites/kohadev/koha-conf.xml
3. echo 'flush_all' | nc -q 1 memcached 11211
4. service apache2 restart
5. koha-plack --restart kohadev
6. Go to http://localhost:8081/cgi-bin/koha/tools/export.pl
7. Note "Unfortunately, no backups are available."

So that's not going to work... but I have another idea.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

David Cook  changed:

   What|Removed |Added

  Attachment #11177|0   |1
is obsolete||

--- Comment #43 from David Cook  ---
Comment on attachment 11177
  --> https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=11177
Bug 8437: variable scoping for tools/export.pl

I think that this one is obsolete at this point.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #42 from David Cook  ---
In any case, I think we do need to solve the immediate problem ASAP, so I'm
going to go ahead with Nick's patch.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #41 from David Cook  ---
I do have another idea... based on what Dobrica said at
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437#c7

When using Plack, we could re-direct away from export.pl to a URL where we have
mounted a PSGI application (e.g. we could use Mojo::Serve::PSGI like we do with
the REST API).

This would also give us a path forward to transition away from CGI to Plack.

The tools/export.pl file is actually pretty short, and the majority of the
logic is in Perl modules (the "model" in MVC terms). The .pl files are
basically just overloaded controllers (in MVC terms). 

I don't have time today, but maybe I would be willing to work on a solution for
this. We could keep the existing template file (or "view" in MVC terms), and
just provide an alternative controller. 

We could deprecate tools/export.pl and focus in favour on a PSGI controller.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #40 from David Cook  ---
I can't figure out why the problem with CGI::Emulate::PSGI::Streaming is
happening, but I've reported the bug. It's unusable until it's fixed though.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #39 from David Cook  ---
That truncated response is 32489 bytes long... 

I'm guessing the issue is with CGI::Parse::PSGI::Streaming...

According to the tools/export.pl, I'm writing a body of 33441 characters...

Looking at CGI::Parse::PSGI::Streaming, I'm only seeing 32768 characters (which
includes the HTTP headers and the HTTP body), and 32768 bytes is the same as
32KB...

CGI::Parse::PSGI::Streaming::Handle shows the full response in the buffer and
says it is 33750 characters long... 

Ah, I think it's a bug in CGI::Parse::PSGI::Streaming::Handle. For some reason,
the author takes the data printed to STDOUT, creates an in-memory file handle
stored in a scalar variable, and prints the contents of that "STDOUT" to the
in-memory file handle, and that's when it is truncated from 33750 characters to
32768 characters...

This is the line in CGI::Parse::PSGI::Streaming::Handle which seemingly breaks
things:
print {$self->{fh}} substr($buf, $offset, $len);

The weird thing is that I'm trying to reproduce the problem in a more stripped
back program and I can't do it. It just works...

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #38 from David Cook  ---
For some reason, my HTML response is being cut off here:

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #37 from David Cook  ---
(In reply to David Cook from comment #36)
> Hmm the difference seems to be in the HTML...

Ahh my custom handler isn't sending a complete HTML response for some reason. I
think a Perl error. I think I must not be setting up the CGI env quite right.
But so close...

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #36 from David Cook  ---
Hmm the difference seems to be in the HTML...

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #35 from David Cook  ---
One alternative could be to use
https://metacpan.org/pod/Plack::App::CGIBin::Streaming. 

Another would be a custom handler using CGI::Emulate::PSGI::Streaming:
cpan2deb CGI::Emulate::PSGI::Streaming
apt install ./libcgi-emulate-psgi-streaming-perl_1.0.0-1_all.deb

I'm exploring the second option now (via a custom Koha::Plack::App::StreamCGI
module)...

It seems like the custom handler is returning an empty koha.mrc file after
about 10 seconds, and I've only processed about 6000 out of 4 records...

Of course, as soon as I say that, suddenly it seems to start working without me
doing anything different. 

Although I notice that usually my custom handler doesn't load some Javascript
or CSS quite right... and I hit the "Export bibliographic records" button from
a different window...

But still... progress... I've streamed 10,000 bib records (10MB) out of Starman
via Apache.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #34 from David Cook  ---
(In reply to David Cook from comment #33)
> As Dobrica mentioned, it does appear that Starman is buffering the whole
> response before sending it. I am guessing this is due to our usage of
> Plack::App::CGIBin... but I'll investigate further.

If I understand correctly, this will never work for us, so long as we're using
Plack::App::CGIBin to serve export.pl. 

At https://metacpan.org/release/Plack/source/lib/Plack/App/CGIBin.pm#L47,
Plack::App::CGIBin uses Plack::App::WrapCGI. 

Plack::App::WrapCGI uses CGI::Emulate::PSGI at
https://metacpan.org/source/Plack::App::WrapCGI#L87.

CGI::Emulate::PSGI writes the response to a temporary file before returning the
response (as per
https://metacpan.org/release/CGI-Emulate-PSGI/source/lib/CGI/Emulate/PSGI.pm#L19).

If you look at CGI::Emulate::PSGI::Streaming
(https://metacpan.org/release/CGI-Emulate-PSGI-Streaming/source/lib/CGI/Emulate/PSGI/Streaming.pm),
you can see how that handler returns a closure rather than a tuple (as noted at
http://www.catalystframework.org/calendar/2013/10).

In order to do a streaming response, you need to leverage that $responder
coderef in the closure.

Of course, this should be doable. I have an idea germinating.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #33 from David Cook  ---
Ok, so I'm running a large bib export using export.pl, and I'm monitoring the
Apache with "LogLevel Debug".

I see a few things that might be interesting, but the key interesting thing is
this:

[Thu Oct 22 00:26:23.676618 2020] [reqtimeout:info] [pid 60205] [client
172.21.0.1:55460] AH01382: Request header read timeout

As Dobrica mentioned, it does appear that Starman is buffering the whole
response before sending it. I am guessing this is due to our usage of
Plack::App::CGIBin... but I'll investigate further.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #32 from David Cook  ---
I've bumped into this one again, so I'm going to have a go at solving it for
real.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #31 from David Cook  ---
Based on my analysis on Bug 26128, I'm wondering what kind of timeouts we're
talking about here.

In the bug I note above, the timeout used by CGI was effectively 2 times as
long as the timeout used when proxying to Plack/Starman.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

David Cook  changed:

   What|Removed |Added

   See Also||https://bugs.koha-community
   ||.org/bugzilla3/show_bug.cgi
   ||?id=22417

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #30 from Nick Clemens  ---
Created attachment 109632
  -->
https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=109632=edit
Bug 8437: Exclude export.pl from plack

When attempting to download large files from Koha plack can timeout

Excluding the script from plack is a simple fix until we have a more permanent
fix for this
issue.

To test:
1 - Try to export your entire DB from Tools->Export
2 - If big enough, it fails
3 - Apply patch, copy changes to /etc/koha/apache-shared-intranet-plack.conf
4 - Restart all the things
5 - Repeat export, it succeeds

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Nick Clemens  changed:

   What|Removed |Added

 Status|In Discussion   |Needs Signoff

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Martin Renvoize  changed:

   What|Removed |Added

 CC||martin.renvoize@ptfs-europe
   ||.com
   Severity|critical|normal

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #29 from David Cook  ---
(In reply to Kyle M Hall from comment #24)
> Avoiding plack will 'fix' the script, but most likely cause everyone to stop
> looking for an actual solution. That's my worry. That being said, it's
> probably better to get it working in the near-term.

I'm looking at export.pl and thinking how export.pl's job should be to
authenticate the user, process user input about what to download, and then
direct the user somewhere else better suited to serving files...

This is kind of interesting although perhaps not relevant in our hybrid
CGI/PSGI way of doing things, and it's old so doesn't take into account that
X-Sendfile seems deprecated:
http://www.catalystframework.org/calendar/2014/5

But it does make the point that trying to serve a large file from Perl is going
to be slow. 

It might be worth looking at frameworks using Plack to see how they do it. For
example send_file and _send_file in Dancer:
https://metacpan.org/release/Dancer/source/lib/Dancer.pm. 

Or we could look at
https://metacpan.org/release/Plack/source/lib/Plack/App/File.pm, although that
looks like it uses PSGI magic to send the file too, which would be tricky in
our hybrid environment.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #28 from David Cook  ---
(In reply to Liz Rea from comment #26)
> What if we change dump strategy for large database, and use mydumper and
> myloader? 
> 
> https://dotlayer.com/extremely-fast-mysql-backup-restore-using-
> mydumpermyloader/

I like the sound of that, but I'm not sure that it's relevant to this issue,
since we're just sending an already existing backup?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #27 from Liz Rea  ---
OPTIONALLY, of course.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Liz Rea  changed:

   What|Removed |Added

 CC||wizzy...@gmail.com

--- Comment #26 from Liz Rea  ---
What if we change dump strategy for large database, and use mydumper and
myloader? 

https://dotlayer.com/extremely-fast-mysql-backup-restore-using-mydumpermyloader/

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Jonathan Druart  changed:

   What|Removed |Added

   See Also||https://bugs.koha-community
   ||.org/bugzilla3/show_bug.cgi
   ||?id=22481

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Fridolin SOMERS  changed:

   What|Removed |Added

 CC||fridolin.som...@biblibre.co
   ||m
   See Also||https://bugs.koha-community
   ||.org/bugzilla3/show_bug.cgi
   ||?id=17240

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #25 from David Cook  ---
(In reply to Kyle M Hall from comment #24)
> Avoiding plack will 'fix' the script, but most likely cause everyone to stop
> looking for an actual solution. That's my worry. That being said, it's
> probably better to get it working in the near-term.

This is my worry as well but I didn't want to say it and seem like a naysayer.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #24 from Kyle M Hall  ---
(In reply to Barton Chittenden from comment #23)
> (In reply to Magnus Enger from comment #22)
> 
> > Would it make sense to add export.pl until someone comes up with a proper
> > solution? Or are there so few people downloading huge files that it is not
> > worth it?
> 
> I think it's probably worth it -- it's a one-line change, and it's easy to
> back out when we finally fix it. Nick, Kyle, Larry, care to weigh in?

That's up to the RM :)

Avoiding plack will 'fix' the script, but most likely cause everyone to stop
looking for an actual solution. That's my worry. That being said, it's probably
better to get it working in the near-term.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

David Cook  changed:

   What|Removed |Added

 CC||dc...@prosentient.com.au

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #23 from Barton Chittenden  ---
(In reply to Magnus Enger from comment #22)

> Would it make sense to add export.pl until someone comes up with a proper
> solution? Or are there so few people downloading huge files that it is not
> worth it?

I think it's probably worth it -- it's a one-line change, and it's easy to back
out when we finally fix it. Nick, Kyle, Larry, care to weigh in?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Barton Chittenden  changed:

   What|Removed |Added

 CC||k...@bywatersolutions.com,
   ||la...@bywatersolutions.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #22 from Magnus Enger  ---
(In reply to Barton Chittenden from comment #21)
> ... the "!" means "Don't run this under plack".

Thanks! There are already a number of pages listed here:
http://git.koha-community.org/gitweb/?p=koha.git;a=blob;f=debian/templates/apache-shared-intranet-plack.conf

Would it make sense to add export.pl until someone comes up with a proper
solution? Or are there so few people downloading huge files that it is not
worth it?

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #21 from Barton Chittenden  ---
(In reply to Magnus Enger from comment #19)
> (In reply to Nick Clemens from comment #18)
> > Hi Nancy, We have also worked around this by simply excluding that page from
> > plack, it avoids the issue and allows exports to complete.
> 
> Anyone got an example of how to do that? Is it something we should do in the
> strandard config, until someone comes up with a better solution? 
> 
> I think the second patch from Dobrica should be a separate bug.

Magnus, take a look at /etc/koha/apache-shared-intranet-plack.conf, you'll find
lines that look like this:

ProxyPass "/cgi-bin/koha/offline_circ/process_koc.pl" "!"

... the "!" means "Don't run this under plack".

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Magnus Enger  changed:

   What|Removed |Added

 Status|NEW |In Discussion

--- Comment #20 from Magnus Enger  ---
I have taken the liberty of moving the second patch from Dobrica to a dedicated
bug: Bug 22143. I also signed it off. Obsoleting the patch here to avoid
confusion. 

Also setting this bug to "In discussion", since there does not seem to be a
clear plan for solving the main problem.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Magnus Enger  changed:

   What|Removed |Added

  Attachment #11355|0   |1
is obsolete||

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Magnus Enger  changed:

   What|Removed |Added

   See Also||https://bugs.koha-community
   ||.org/bugzilla3/show_bug.cgi
   ||?id=22143

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Magnus Enger  changed:

   What|Removed |Added

 CC||mag...@libriotech.no

--- Comment #19 from Magnus Enger  ---
(In reply to Nick Clemens from comment #18)
> Hi Nancy, We have also worked around this by simply excluding that page from
> plack, it avoids the issue and allows exports to complete.

Anyone got an example of how to do that? Is it something we should do in the
strandard config, until someone comes up with a better solution? 

I think the second patch from Dobrica should be a separate bug.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Josef Moravec  changed:

   What|Removed |Added

 CC||josef.mora...@gmail.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #18 from Nick Clemens  ---
Hi Nancy, We have also worked around this by simply excluding that page from
plack, it avoids the issue and allows exports to complete.

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #17 from Jonathan Druart  
---
(In reply to NancyK. from comment #16)
> This bug is going to cause us a really big problem.  We are doing weekly
> exports and have plans for more.  Please fix this soon.  
> Thanks
> 
> NancyK

This will not be fixed soon as nobody as a solution.
You should consider using the CLI script misc/export_records.pl for your weekly
export (with a cronjob even).

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

NancyK.  changed:

   What|Removed |Added

 CC||nkee...@washoecounty.us

--- Comment #16 from NancyK.  ---
This bug is going to cause us a really big problem.  We are doing weekly
exports and have plans for more.  Please fix this soon.  
Thanks

NancyK

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

--- Comment #15 from Barton Chittenden  ---
(In reply to Jonathan Druart from comment #14)
> (In reply to Dobrica Pavlinusic from comment #7)
> > Correct way to fix this is to issue redirect under plack in download_backup
> > to some plack handled URL so we don't read whole file in memory.
> 
> The solution seems to be here, but I don't understand what Dobrica meant.

Dobrica,

Can you clarify and/or submit a further patch? This issue is really causing
issues for large libraries.

Thanks,

--Barton

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-bugs] [Bug 8437] Large database backups and large exports from export.pl fail under plack

[bugzilla-daemon]

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=8437

Barton Chittenden  changed:

   What|Removed |Added

Summary|Plack and database backup   |Large database backups and
   ||large exports from
   ||export.pl fail under plack
 CC||bar...@bywatersolutions.com

-- 
You are receiving this mail because:
You are watching all bug changes.
___
Koha-bugs mailing list
Koha-bugs@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-bugs
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/