Re: Serving very large pdf files with django

[Gary Roach]

Thanks you all for you contributions

I have decided to break down the pdf files into individual jpg pages and 
serve them that way. I plan on putting each document in its own folder 
and then have the individual pages listed within the folder. I haven't 
worked out the exact retrieval scheme yet but it will probably be 
something like:


Documents
Document 1
;:
;jpg 1
::
:   jpg n
;
Document n

This should work and allow fast recovery.

Thanks again for your help

Gary R


On 09/04/2016 05:37 AM, Avraham Serour wrote:
if you need to server the pdf then it will be downloaded to the user 
computer, even if you are visualizing in browser with js.


What you can do is show screenshots of the pdf pages, then you would 
be serving individual jpgs for each page and not a single 100mb file.


You can create previews for each page using something like imagemagick 
as mentioned before and create a layout to view that using django 
template.




On Sun, Sep 4, 2016 at 9:52 AM, <06use...@gmail.com 
> wrote:


Hello,

Maybe you could use ImageMagick to show one page of the pdf as an
image ? This is what I have implemented on my project : first page
of pdf files can be pre-viewed before user uploads the whole file
(if needed). In your case, pre-viewing could be parametrized to
show page ‘n’ of the pdf file, probably. In my project,
pre-viewing image is generated when pdf file is uploaded and saved
by admin in myproject/media/files directory. Once pdf files are in
media directory, users can preview those pdf files and tnen upload
them, if needed (only first page can be pre-viewed by user. This
is an implementation choice).


http://www.imagemagick.org/script/index.php





Implementation principles/credit can be found here :

http://www.yaconiello.com/blog/auto-generating-pdf-covers/





I am attaching the signals file_post_save file_pre_save, for
information.


Cheers.

Django 1.8
Python 3.4


Note : to install ImageMagick:

sudo apt-get install libmagickwand-dev imagemagick libmagickcore-dev

The convert command allows to convert any page of pdf file into
png image (for instance), eg :

convert -thumbnail 1280 test.pdf[0] test_1280.png
Several parameters exist to tune image quality (in my case, need
to use density, trim and quality).

*models.p**y*

[…]

###

# SIGNALS #

###

from django.db.models.signals import post_save, pre_delete

from django.dispatch import receiver

from myproject.settings import MEDIA_ROOT

import subprocess


# What to do after a File is saved - receiver definition

def file_post_save(sender, instance=False, **kwargs):

# This post save function creates a thumbnail for the File

file = File.objects.get(pk=instance.pk )

command = "convert -density 300 -trim -thumbnail %s %s%s[0]
-quality 100 %s%s" % (file.thumbnail_size, MEDIA_ROOT, file.file,
MEDIA_ROOT, file.thumbnail)

proc = subprocess.Popen(command,

shell=True,

stdin=subprocess.PIPE,

stdout=subprocess.PIPE,

stderr=subprocess.PIPE,

)

stdout_value = proc.communicate()[0]

post_save.connect(file_post_save, sender=File,
dispatch_uid="file_post_save_uid")

# What to do before a File is deleted - receiver definition

def file_pre_delete(sender, instance=False, **kwargs):

# This pre delete function deletes file and thumbnail from media
directory

file = File.objects.get(pk=instance.pk )

command = "rm %s%s %s%s" % (MEDIA_ROOT, file.file, MEDIA_ROOT,
file.thumbnail)

proc = subprocess.Popen(command,

shell=True,

stdin=subprocess.PIPE,

stdout=subprocess.PIPE,

stderr=subprocess.PIPE,

)

stdout_value = proc.communicate()[0]

pre_delete.connect(file_pre_delete, sender=File,
dispatch_uid="file_pre_delete_uid")

-- 
You received this message because you are subscribed to the Google

Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to django-users+unsubscr...@googlegroups.com
.
To post to this group, send email to django-users@googlegroups.com
.
Visit this group at https://groups.google.com/group/django-users
.
To view this discussion on the web visit

https://groups.google.com/d/msgid/django-users/e409aafa-3133-47b1-b465-7d74501d0cac%40googlegroups.com

Re: Serving very large pdf files with django

[Avraham Serour]

if you need to server the pdf then it will be downloaded to the user
computer, even if you are visualizing in browser with js.

What you can do is show screenshots of the pdf pages, then you would be
serving individual jpgs for each page and not a single 100mb file.

You can create previews for each page using something like imagemagick as
mentioned before and create a layout to view that using django template.



On Sun, Sep 4, 2016 at 9:52 AM, <06use...@gmail.com> wrote:

> Hello,
>
> Maybe you could use ImageMagick to show one page of the pdf as an image ?
> This is what I have implemented on my project : first page of pdf files can
> be pre-viewed before user uploads the whole file (if needed). In your case,
> pre-viewing could be parametrized to show page ‘n’ of the pdf file,
> probably. In my project, pre-viewing image is generated when pdf file is
> uploaded and saved by admin in myproject/media/files directory. Once pdf
> files are in media directory, users can preview those pdf files and tnen
> upload them, if needed (only first page can be pre-viewed by user. This is
> an implementation choice).
>
>
> http://www.imagemagick.org/script/index.php
>
>
> 
>
> Implementation principles/credit can be found here :
>
> http://www.yaconiello.com/blog/auto-generating-pdf-covers/
>
>
> 
>
> I am attaching the signals file_post_save file_pre_save, for information.
>
>
> Cheers.
>
> Django 1.8
> Python 3.4
>
>
> Note : to install ImageMagick:
>
> sudo apt-get install libmagickwand-dev imagemagick libmagickcore-dev
>
> The convert command allows to convert any page of pdf file into png image
> (for instance), eg :
>
> convert -thumbnail 1280 test.pdf[0] test_1280.png
> Several parameters exist to tune image quality (in my case, need to use
> density, trim and quality).
>
>
>
> *models.p**y*
>
> […]
>
> ###
>
> # SIGNALS #
>
> ###
>
> from django.db.models.signals import post_save, pre_delete
>
> from django.dispatch import receiver
>
> from myproject.settings import MEDIA_ROOT
>
> import subprocess
>
>
> # What to do after a File is saved - receiver definition
>
> def file_post_save(sender, instance=False, **kwargs):
>
> # This post save function creates a thumbnail for the File
>
> file = File.objects.get(pk=instance.pk)
>
> command = "convert -density 300 -trim -thumbnail %s %s%s[0]
> -quality 100 %s%s" % (file.thumbnail_size, MEDIA_ROOT, file.file,
> MEDIA_ROOT, file.thumbnail)
>
> proc = subprocess.Popen(command,
>
> shell=True,
>
> stdin=subprocess.PIPE,
>
> stdout=subprocess.PIPE,
>
> stderr=subprocess.PIPE,
>
> )
>
> stdout_value = proc.communicate()[0]
>
> post_save.connect(file_post_save, sender=File,
> dispatch_uid="file_post_save_uid")
>
>
>
> # What to do before a File is deleted - receiver definition
>
> def file_pre_delete(sender, instance=False, **kwargs):
>
> # This pre delete function deletes file and thumbnail from
> media directory
>
> file = File.objects.get(pk=instance.pk)
>
> command = "rm %s%s %s%s" % (MEDIA_ROOT, file.file,
> MEDIA_ROOT, file.thumbnail)
>
> proc = subprocess.Popen(command,
>
> shell=True,
>
> stdin=subprocess.PIPE,
>
> stdout=subprocess.PIPE,
>
> stderr=subprocess.PIPE,
>
> )
>
> stdout_value = proc.communicate()[0]
>
> pre_delete.connect(file_pre_delete, sender=File,
> dispatch_uid="file_pre_delete_uid")
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-users/e409aafa-3133-47b1-b465-7d74501d0cac%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAFWa6tLODtEQ_5Tde-Axe3khKKzDCa58%3D-u3%2Bnk7vx3F6uQ

Re: Serving very large pdf files with django

[06userit]

 

Hello,

Maybe you could use ImageMagick to show one page of the pdf as an image ? 
This is what I have implemented on my project : first page of pdf files can 
be pre-viewed before user uploads the whole file (if needed). In your case, 
pre-viewing could be parametrized to show page ‘n’ of the pdf file, 
probably. In my project, pre-viewing image is generated when pdf file is 
uploaded and saved by admin in myproject/media/files directory. Once pdf 
files are in media directory, users can preview those pdf files and tnen 
upload them, if needed (only first page can be pre-viewed by user. This is 
an implementation choice).


http://www.imagemagick.org/script/index.php




Implementation principles/credit can be found here :

http://www.yaconiello.com/blog/auto-generating-pdf-covers/




I am attaching the signals file_post_save file_pre_save, for information.


Cheers.

Django 1.8
Python 3.4


Note : to install ImageMagick:

sudo apt-get install libmagickwand-dev imagemagick libmagickcore-dev

The convert command allows to convert any page of pdf file into png image 
(for instance), eg :

convert -thumbnail 1280 test.pdf[0] test_1280.png
Several parameters exist to tune image quality (in my case, need to use 
density, trim and quality).

 

*models.p**y*

[…]

###

# SIGNALS #

###

from django.db.models.signals import post_save, pre_delete

from django.dispatch import receiver

from myproject.settings import MEDIA_ROOT

import subprocess


# What to do after a File is saved - receiver definition

def file_post_save(sender, instance=False, **kwargs):

# This post save function creates a thumbnail for the File

file = File.objects.get(pk=instance.pk)

command = "convert -density 300 -trim -thumbnail %s %s%s[0] 
-quality 100 %s%s" % (file.thumbnail_size, MEDIA_ROOT, file.file, 
MEDIA_ROOT, file.thumbnail)

proc = subprocess.Popen(command,

shell=True,

stdin=subprocess.PIPE,

stdout=subprocess.PIPE,

stderr=subprocess.PIPE,

)

stdout_value = proc.communicate()[0]

post_save.connect(file_post_save, sender=File, 
dispatch_uid="file_post_save_uid")

 

# What to do before a File is deleted - receiver definition

def file_pre_delete(sender, instance=False, **kwargs):

# This pre delete function deletes file and thumbnail from 
media directory

file = File.objects.get(pk=instance.pk)

command = "rm %s%s %s%s" % (MEDIA_ROOT, file.file, MEDIA_ROOT, 
file.thumbnail)

proc = subprocess.Popen(command,

shell=True,

stdin=subprocess.PIPE,

stdout=subprocess.PIPE,

stderr=subprocess.PIPE,

)

stdout_value = proc.communicate()[0]

pre_delete.connect(file_pre_delete, sender=File, 
dispatch_uid="file_pre_delete_uid")

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/e409aafa-3133-47b1-b465-7d74501d0cac%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Serving very large pdf files with django

[ADEWALE ADISA]

hi,
My view is that instead of combining all the images in one pdf, its better
to leave it as image and serve them individually by publishing link to each
image with text describing each page. Then the user can view any page they
are interested in. 'cause that single file is too large.
On Sep 3, 2016 6:12 PM, "Gary Roach"  wrote:

> Hi all,
>
> I am working on a project where I need to serve up large (100 -150 MB)
> static pdf files for viewing. The pdf files are jpg photos of pages from
> old log books. Downloading into the user's system is out of the question
> for obvious reasons. In addition the user may only need to see one or two
> pages out of a document. I plan on putting the static files in their own
> directory outside of the django programming and the postgresql database. I
> have looked at several addons to django but can't seem to get a "feel" for
> the whole problem. Has anyone had a similar problem and do they have a good
> solution .
>
> All help will be sincerely appreciated.
>
> OS Debian Linux
> Desktop KDE4
> Python 3.5
> Django 1.9.x
>
> Gary R.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/django-users/b5e154b5-3c01-6f39-f4a1-6e285093aa45%40verizon.net.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAMGzuy8XNUWXugEL0z3nOr_G2MYGz8x59NsUGM%2BZaHfW0AbYHA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Serving very large pdf files with django

[ludovic coues]

You have a few solution. You already excluded the user downloading the
full document. You can display the document directly in your page,
with solution like pdfjs, or you can extract the individual image from
the document on your server and display directly the image.

If you show the pdf, I fear your only solution is that the user
download the whole document in their browser. Same problem as with
downloading the full document. A solution would be to split the
document into individual page.

2016-09-03 19:11 GMT+02:00 Gary Roach :
> Hi all,
>
> I am working on a project where I need to serve up large (100 -150 MB)
> static pdf files for viewing. The pdf files are jpg photos of pages from old
> log books. Downloading into the user's system is out of the question for
> obvious reasons. In addition the user may only need to see one or two pages
> out of a document. I plan on putting the static files in their own directory
> outside of the django programming and the postgresql database. I have looked
> at several addons to django but can't seem to get a "feel" for the whole
> problem. Has anyone had a similar problem and do they have a good solution .
>
> All help will be sincerely appreciated.
>
> OS Debian Linux
> Desktop KDE4
> Python 3.5
> Django 1.9.x
>
> Gary R.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/b5e154b5-3c01-6f39-f4a1-6e285093aa45%40verizon.net.
> For more options, visit https://groups.google.com/d/optout.



-- 

Cordialement, Coues Ludovic
+336 148 743 42

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAEuG%2BTbmgJVmMy4%3DxTZVedawYmfxp7HF0MeDXWxq878G0pQoqw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Serving very large pdf files with django

[Gary Roach]

Hi all,

I am working on a project where I need to serve up large (100 -150 MB) 
static pdf files for viewing. The pdf files are jpg photos of pages from 
old log books. Downloading into the user's system is out of the question 
for obvious reasons. In addition the user may only need to see one or 
two pages out of a document. I plan on putting the static files in their 
own directory outside of the django programming and the postgresql 
database. I have looked at several addons to django but can't seem to 
get a "feel" for the whole problem. Has anyone had a similar problem and 
do they have a good solution .


All help will be sincerely appreciated.

OS Debian Linux
Desktop KDE4
Python 3.5
Django 1.9.x

Gary R.

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/b5e154b5-3c01-6f39-f4a1-6e285093aa45%40verizon.net.
For more options, visit https://groups.google.com/d/optout.