Re: Add custom autoreload file tracking options setting

[Florian Apolloner]

On Thursday, January 5, 2017 at 11:14:08 PM UTC+1, Josh Smeaton wrote:
>
> > I am -0 to -1 for the debugger -- I've seen to many sites out there 
> running with DEBUG=True, enabling RCE ootb seems to be pretty horrible.
>
> But it's so incredibly useful. And we already show the django debug page 
> for errors with DEBUG=True that exposes enough secrets to allow a 
> sufficient attacker to gain access.
>

What exactly? Last time I checked SECRET_KEY and other dangerous stuff was 
blanked out as good as possible. Do not get me wrong, it is certainly not 
"safe" to show the debug page, but leaking information versus RCE is a 
different story… Even if the debug page leaks enough information to login 
as admin, you do not neccessarily compromise the OS, whereas the werkzeug 
debugger gives you at least user access to the OS.

And truth to be told, I can count the instance where the werkzeug debugger 
would have been useful on one hand -- the traces are usually more than 
enough.
 

> If we could, by default, block the debugger in a similar way that django 
> debug toolbar does, would that be appropriate? That is, checks for DEBUG 
> and HOST etc?
>

I am not deploying debug toolbar anywhere, so I cannot tell. That said, 
people have DEBUG=True etc in prod, if we open those installations for RCE, 
that would still suck (defense in depth and everything). Also 
https://labs.detectify.com/2015/10/02/how-patreon-got-hacked-publicly-exposed-werkzeug-debugger/
 
-- even though it is old, it is a nice sign that it happens…

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/91eb120a-e777-4bbe-ad63-345d6983cfba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add custom autoreload file tracking options setting

[Josh Smeaton]

> I am -0 to -1 for the debugger -- I've seen to many sites out there 
running with DEBUG=True, enabling RCE ootb seems to be pretty horrible.

But it's so incredibly useful. And we already show the django debug page 
for errors with DEBUG=True that exposes enough secrets to allow a 
sufficient attacker to gain access. If we could, by default, block the 
debugger in a similar way that django debug toolbar does, would that be 
appropriate? That is, checks for DEBUG and HOST etc?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/65ab0096-d7e9-4817-8fa2-0d2296f97671%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add custom autoreload file tracking options setting

[Stratos Moros]

I’m -0 on the change. I could move to +0 if I understood why the use 
case described here requires watching additional files.


A different use case we've run into is non-python configuration files. 
Our settings.py reads a few variables off a toml file and it would be 
nice if we could configure runserver to reload automatically when that 
toml file changes.


That said, I agree that auto reloading should probably live outside 
Django.


--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/D8E34EDB-0BF8-4D78-A567-FB0EE372E631%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add custom autoreload file tracking options setting

[Michael Manfre]

On Thu, Jan 5, 2017 at 8:06 AM Florian Apolloner 
wrote:

>
>
> On Thursday, January 5, 2017 at 1:38:44 PM UTC+1, Sam Willis wrote:
>
> Could one options be to replace the current devserver with the one from
> Werkzeug? It already uses watchdog (similar to watchman) for monitoring
> file system events and is well maintained. With Django now allowing
> dependancies, this seems like something that doesn't necessarily need to be
> developed internally.
>
>
> Watchdog seems to require a .c extension file on MacOSX at least, so as
> long as there are no wheels, that does not seem to be an option.
>
> The Werkzeug devserver also has some niceties like an interactive debugger
> and ssl hosting with an automatically issued self signed certificate. It
> could be implemented behind the current management command
>
>
> Self-Signed SSL sounds somewhat nice, I am -0 to -1 for the debugger --
> I've seen to many sites out there running with DEBUG=True, enabling RCE
> ootb seems to be pretty horrible.
>

I think anyone running devserver in prod deserves a breakpoint induced
outage.

Regards,
Michael Manfre

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAGdCwBsshYVoNS04Rii2Xm1qi6J4mQPaBrNja%3DGwLVhQQ4RqNw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add custom autoreload file tracking options setting

[Florian Apolloner]

On Thursday, January 5, 2017 at 1:38:44 PM UTC+1, Sam Willis wrote:
>
> Could one options be to replace the current devserver with the one from 
> Werkzeug? It already uses watchdog (similar to watchman) for monitoring 
> file system events and is well maintained. With Django now allowing 
> dependancies, this seems like something that doesn't necessarily need to be 
> developed internally.
>

Watchdog seems to require a .c extension file on MacOSX at least, so as 
long as there are no wheels, that does not seem to be an option.

The Werkzeug devserver also has some niceties like an interactive debugger 
> and ssl hosting with an automatically issued self signed certificate. It 
> could be implemented behind the current management command 
>

Self-Signed SSL sounds somewhat nice, I am -0 to -1 for the debugger -- 
I've seen to many sites out there running with DEBUG=True, enabling RCE 
ootb seems to be pretty horrible.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/93d787ad-dd98-4ae5-8251-8eddd4b97d3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add custom autoreload file tracking options setting

[Sam Willis]

Could one options be to replace the current devserver with the one from 
Werkzeug? It already uses watchdog (similar to watchman) for monitoring 
file system events and is well maintained. With Django now allowing 
dependancies, this seems like something that doesn't necessarily need to be 
developed internally.

The Werkzeug devserver also has some niceties like an interactive debugger 
and ssl hosting with an automatically issued self signed certificate. It 
could be implemented behind the current management command relatively 
easily.

There is already an implementation as part of django-extentions that I 
believe is well used and battle tested 
- http://django-extensions.readthedocs.io/en/latest/runserver_plus.html 


On Thursday, January 5, 2017 at 9:11:53 AM UTC, Aymeric Augustin wrote:
>
> On 4 Jan 2017, at 23:31, Bobby Mozumder > 
> wrote: 
>
> > I guess I could just use Watchman to restart the Django development 
> server as needed? 
>
>
> If you find a way to tell watchman to run `django-admin runserver 
> --noreload` and restart it whenever a file in the current directory 
> changes, that should do the job. 
>
> Unfortunately the APIs exposed by watchman don’t make this trivial. 
> They’re intended to run a build process that will terminate, while the 
> development server will keep running. 
>
> -- 
> Aymeric. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6d346a13-b096-42f8-a15b-a2814a81eef5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Add custom autoreload file tracking options setting

[Aymeric Augustin]

On 4 Jan 2017, at 23:31, Bobby Mozumder  wrote:

> I guess I could just use Watchman to restart the Django development server as 
> needed?


If you find a way to tell watchman to run `django-admin runserver --noreload` 
and restart it whenever a file in the current directory changes, that should do 
the job.

Unfortunately the APIs exposed by watchman don’t make this trivial. They’re 
intended to run a build process that will terminate, while the development 
server will keep running.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/A7512959-281B-4679-9E04-93EEB78C33A1%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Add custom autoreload file tracking options setting

[Tim Graham]

No, there's a cached template 
loader: 
https://docs.djangoproject.com/en/dev/ref/templates/api/#django.template.loaders.cached.Loader

On Wednesday, January 4, 2017 at 7:00:31 PM UTC-5, Bobby Mozumder wrote:
>
>
> On Jan 4, 2017, at 5:40 PM, Adam Johnson > 
> wrote:
>
> How do people serve development Javascript & CSS files?  These days 
>> Javascript & CSS involves a large build process.  Are we forced to manually 
>> restart the development server every time Javascript changes?
>
>
> Django just serves them from the filesystem, and doesn't cache them 
> itself. You reload the page in your browser, and Django serves the files 
> from disk
>
> Does the Django development server restart if you edit Templates?  Because 
>> it should also do that.
>
>
> Similarly, by default templates aren't cached, so on each view the dev 
> server loads the templates from disk, parses them, and renders them.
>  
>
>
> Does it do the same under production environments - read and interpret 
> each template file on every request?  
>
> Is there interest in making a much faster version of this flow?
>
>
> On 4 January 2017 at 22:31, Bobby Mozumder  > wrote:
>
>>
>> > On Jan 4, 2017, at 4:47 PM, Aymeric Augustin <
>> aymeric@polytechnique.org > wrote:
>> >
>> > Hello Bobby,
>> >
>> >> On 4 Jan 2017, at 22:25, Bobby Mozumder > > wrote:
>> >>
>> >> It’s actually called once on app startup during DB connection via a 
>> Signal.
>> >
>> > Unless I missed something, since the development server creates a new 
>> connection to the database for each request — Python’s threaded socket 
>> server creates a new thread for each connection and database connections 
>> are thread local in Django — prepared statements will be refreshed for each 
>> request. So I don’t think you need autoreloading here.
>>
>> Hmm I thought the test server used permanent connections if CONN_MAX_AGE 
>> was set to None?  Could have sworn it was that way for a while?
>>
>> In any case, I just tried the Javascript & CSS Makefile with a simple one 
>> line change to one file, and it took 4 seconds.  That’s going to be way too 
>> long to do on each request.
>>
>> How do people serve development Javascript & CSS files?  These days 
>> Javascript & CSS involves a large build process.  Are we forced to manually 
>> restart the development server every time Javascript changes?
>>
>> I think it should be just like changing Python files and it autoreloads 
>> Javascript & CSS as it changes.
>>
>> Also, I haven’t used the Django Template system or Jinja in over a year.  
>> Does the Django development server restart if you edit Templates?  Because 
>> it should also do that.
>>
>> This is what I’m getting at with this autoreload.. it should include the 
>> full system, not just the Python files.
>>
>> >
>> >> (This whole process helps speed up the view response generation 
>> times.  I can generate a full page in about 4ms, and that includes GZip.)
>> >
>> > Off topic, but I’m jealous…
>> >
>>
>> Also, my average page generates in 1-2ms, and fully cached html serves in 
>> 200 microseconds, and that include Gzip, since I cache Gzip (which makes my 
>> cache 10x bigger).   There’s lots that I can contribute here, including 
>> non-blocking analytics stored in teh database.  If I can make it “generic” 
>> enough I’ll try to publish it, but the code is really manual.  Maybe 
>> instead of a formal toolset, I could post a methodology manual?  It’s 
>> really tedious though, involving materialized database views & all sorts of 
>> other techniques.
>>
>> BTW My site is at http://www.futureclaw.com
>>
>> > 
>> >
>> > I still believe we should stop maintaining an autoreloader as soon as 
>> possible. Django’s autoreloader is annoyingly slow, highly inefficient, 
>> moderately well designed, and a gigantic pain to maintain. I’m more scared 
>> of django.utils.autoreload than of django.db.models.related before it was 
>> cleaned up.
>> >
>> > I wish one day someone will take the time to write a good autoreloading 
>> dev server based on watchman. This would solve the problem discussed here 
>> because watchman watches all files in the current directory. The correct 
>> way to do this is to throw away the current design and start from scratch.
>> >
>> > Watchman is smart enough to wait until you’ve finished a git operation 
>> to trigger a reload. Once such polished tech has become available, trying 
>> to compete with a thread that checks the mtime of all known files every 
>> second isn’t funny anymore. In fact it’s just sad.
>> >
>> > 
>> >
>> > While I don’t care much about django.utils.autoreloader because I want 
>> to kill it, I’m reluctant to add public APIs such as the proposed setting, 
>> which may not make sense with an hypothetical better autoreloader.
>> >
>> > I’m -0 on the change. I could move to +0 if I understood why the use 
>> case described here requires watching additional files.
>> >
>>
>> I guess I could just use Watchman to restart the Django devel

Re: Add custom autoreload file tracking options setting

[Bobby Mozumder]

> On Jan 4, 2017, at 5:40 PM, Adam Johnson  wrote:
> 
> How do people serve development Javascript & CSS files?  These days 
> Javascript & CSS involves a large build process.  Are we forced to manually 
> restart the development server every time Javascript changes?
> 
> Django just serves them from the filesystem, and doesn't cache them itself. 
> You reload the page in your browser, and Django serves the files from disk
> 
> Does the Django development server restart if you edit Templates?  Because it 
> should also do that.
> 
> Similarly, by default templates aren't cached, so on each view the dev server 
> loads the templates from disk, parses them, and renders them.
>  

Does it do the same under production environments - read and interpret each 
template file on every request?  

Is there interest in making a much faster version of this flow?

> 
> On 4 January 2017 at 22:31, Bobby Mozumder  > wrote:
> 
> > On Jan 4, 2017, at 4:47 PM, Aymeric Augustin 
> >  > > wrote:
> >
> > Hello Bobby,
> >
> >> On 4 Jan 2017, at 22:25, Bobby Mozumder  >> > wrote:
> >>
> >> It’s actually called once on app startup during DB connection via a Signal.
> >
> > Unless I missed something, since the development server creates a new 
> > connection to the database for each request — Python’s threaded socket 
> > server creates a new thread for each connection and database connections 
> > are thread local in Django — prepared statements will be refreshed for each 
> > request. So I don’t think you need autoreloading here.
> 
> Hmm I thought the test server used permanent connections if CONN_MAX_AGE was 
> set to None?  Could have sworn it was that way for a while?
> 
> In any case, I just tried the Javascript & CSS Makefile with a simple one 
> line change to one file, and it took 4 seconds.  That’s going to be way too 
> long to do on each request.
> 
> How do people serve development Javascript & CSS files?  These days 
> Javascript & CSS involves a large build process.  Are we forced to manually 
> restart the development server every time Javascript changes?
> 
> I think it should be just like changing Python files and it autoreloads 
> Javascript & CSS as it changes.
> 
> Also, I haven’t used the Django Template system or Jinja in over a year.  
> Does the Django development server restart if you edit Templates?  Because it 
> should also do that.
> 
> This is what I’m getting at with this autoreload.. it should include the full 
> system, not just the Python files.
> 
> >
> >> (This whole process helps speed up the view response generation times.  I 
> >> can generate a full page in about 4ms, and that includes GZip.)
> >
> > Off topic, but I’m jealous…
> >
> 
> Also, my average page generates in 1-2ms, and fully cached html serves in 200 
> microseconds, and that include Gzip, since I cache Gzip (which makes my cache 
> 10x bigger).   There’s lots that I can contribute here, including 
> non-blocking analytics stored in teh database.  If I can make it “generic” 
> enough I’ll try to publish it, but the code is really manual.  Maybe instead 
> of a formal toolset, I could post a methodology manual?  It’s really tedious 
> though, involving materialized database views & all sorts of other techniques.
> 
> BTW My site is at http://www.futureclaw.com 
> 
> > 
> >
> > I still believe we should stop maintaining an autoreloader as soon as 
> > possible. Django’s autoreloader is annoyingly slow, highly inefficient, 
> > moderately well designed, and a gigantic pain to maintain. I’m more scared 
> > of django.utils.autoreload than of django.db.models.related before it was 
> > cleaned up.
> >
> > I wish one day someone will take the time to write a good autoreloading dev 
> > server based on watchman. This would solve the problem discussed here 
> > because watchman watches all files in the current directory. The correct 
> > way to do this is to throw away the current design and start from scratch.
> >
> > Watchman is smart enough to wait until you’ve finished a git operation to 
> > trigger a reload. Once such polished tech has become available, trying to 
> > compete with a thread that checks the mtime of all known files every second 
> > isn’t funny anymore. In fact it’s just sad.
> >
> > 
> >
> > While I don’t care much about django.utils.autoreloader because I want to 
> > kill it, I’m reluctant to add public APIs such as the proposed setting, 
> > which may not make sense with an hypothetical better autoreloader.
> >
> > I’m -0 on the change. I could move to +0 if I understood why the use case 
> > described here requires watching additional files.
> >
> 
> I guess I could just use Watchman to restart the Django development server as 
> needed?
> 
> I’m OK with that.. my only goal is to make sure I don’t have to restart 
> Django manually every time I edit CSS/Javascript/SQL/HTML.
> 
> 

Re: Add custom autoreload file tracking options setting

[Adam Johnson]

>
> How do people serve development Javascript & CSS files?  These days
> Javascript & CSS involves a large build process.  Are we forced to manually
> restart the development server every time Javascript changes?


Django just serves them from the filesystem, and doesn't cache them itself.
You reload the page in your browser, and Django serves the files from disk

Does the Django development server restart if you edit Templates?  Because
> it should also do that.


Similarly, by default templates aren't cached, so on each view the dev
server loads the templates from disk, parses them, and renders them.


On 4 January 2017 at 22:31, Bobby Mozumder  wrote:

>
> > On Jan 4, 2017, at 4:47 PM, Aymeric Augustin  polytechnique.org> wrote:
> >
> > Hello Bobby,
> >
> >> On 4 Jan 2017, at 22:25, Bobby Mozumder  wrote:
> >>
> >> It’s actually called once on app startup during DB connection via a
> Signal.
> >
> > Unless I missed something, since the development server creates a new
> connection to the database for each request — Python’s threaded socket
> server creates a new thread for each connection and database connections
> are thread local in Django — prepared statements will be refreshed for each
> request. So I don’t think you need autoreloading here.
>
> Hmm I thought the test server used permanent connections if CONN_MAX_AGE
> was set to None?  Could have sworn it was that way for a while?
>
> In any case, I just tried the Javascript & CSS Makefile with a simple one
> line change to one file, and it took 4 seconds.  That’s going to be way too
> long to do on each request.
>
> How do people serve development Javascript & CSS files?  These days
> Javascript & CSS involves a large build process.  Are we forced to manually
> restart the development server every time Javascript changes?
>
> I think it should be just like changing Python files and it autoreloads
> Javascript & CSS as it changes.
>
> Also, I haven’t used the Django Template system or Jinja in over a year.
> Does the Django development server restart if you edit Templates?  Because
> it should also do that.
>
> This is what I’m getting at with this autoreload.. it should include the
> full system, not just the Python files.
>
> >
> >> (This whole process helps speed up the view response generation times.
> I can generate a full page in about 4ms, and that includes GZip.)
> >
> > Off topic, but I’m jealous…
> >
>
> Also, my average page generates in 1-2ms, and fully cached html serves in
> 200 microseconds, and that include Gzip, since I cache Gzip (which makes my
> cache 10x bigger).   There’s lots that I can contribute here, including
> non-blocking analytics stored in teh database.  If I can make it “generic”
> enough I’ll try to publish it, but the code is really manual.  Maybe
> instead of a formal toolset, I could post a methodology manual?  It’s
> really tedious though, involving materialized database views & all sorts of
> other techniques.
>
> BTW My site is at http://www.futureclaw.com
>
> > 
> >
> > I still believe we should stop maintaining an autoreloader as soon as
> possible. Django’s autoreloader is annoyingly slow, highly inefficient,
> moderately well designed, and a gigantic pain to maintain. I’m more scared
> of django.utils.autoreload than of django.db.models.related before it was
> cleaned up.
> >
> > I wish one day someone will take the time to write a good autoreloading
> dev server based on watchman. This would solve the problem discussed here
> because watchman watches all files in the current directory. The correct
> way to do this is to throw away the current design and start from scratch.
> >
> > Watchman is smart enough to wait until you’ve finished a git operation
> to trigger a reload. Once such polished tech has become available, trying
> to compete with a thread that checks the mtime of all known files every
> second isn’t funny anymore. In fact it’s just sad.
> >
> > 
> >
> > While I don’t care much about django.utils.autoreloader because I want
> to kill it, I’m reluctant to add public APIs such as the proposed setting,
> which may not make sense with an hypothetical better autoreloader.
> >
> > I’m -0 on the change. I could move to +0 if I understood why the use
> case described here requires watching additional files.
> >
>
> I guess I could just use Watchman to restart the Django development server
> as needed?
>
> I’m OK with that.. my only goal is to make sure I don’t have to restart
> Django manually every time I edit CSS/Javascript/SQL/HTML.
>
> Is that the recommended workflow for this?  I never looked into outside
> tools to manage this, but it seems it would be a lot easier to add a list
> of files or directories to our settings and have Django manage autoreload,
> than to bring in an outside tool.  (I can add directories to watch to the
> pull request if needed)
>
> > Best regards,
> >
> > --
> > Aymeric.
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Dj

Re: Add custom autoreload file tracking options setting

[Bobby Mozumder]

> On Jan 4, 2017, at 4:47 PM, Aymeric Augustin 
>  wrote:
> 
> Hello Bobby,
> 
>> On 4 Jan 2017, at 22:25, Bobby Mozumder  wrote:
>> 
>> It’s actually called once on app startup during DB connection via a Signal.
> 
> Unless I missed something, since the development server creates a new 
> connection to the database for each request — Python’s threaded socket server 
> creates a new thread for each connection and database connections are thread 
> local in Django — prepared statements will be refreshed for each request. So 
> I don’t think you need autoreloading here.

Hmm I thought the test server used permanent connections if CONN_MAX_AGE was 
set to None?  Could have sworn it was that way for a while?

In any case, I just tried the Javascript & CSS Makefile with a simple one line 
change to one file, and it took 4 seconds.  That’s going to be way too long to 
do on each request.

How do people serve development Javascript & CSS files?  These days Javascript 
& CSS involves a large build process.  Are we forced to manually restart the 
development server every time Javascript changes?  

I think it should be just like changing Python files and it autoreloads 
Javascript & CSS as it changes.

Also, I haven’t used the Django Template system or Jinja in over a year.  Does 
the Django development server restart if you edit Templates?  Because it should 
also do that.

This is what I’m getting at with this autoreload.. it should include the full 
system, not just the Python files.

> 
>> (This whole process helps speed up the view response generation times.  I 
>> can generate a full page in about 4ms, and that includes GZip.)
> 
> Off topic, but I’m jealous…
> 

Also, my average page generates in 1-2ms, and fully cached html serves in 200 
microseconds, and that include Gzip, since I cache Gzip (which makes my cache 
10x bigger).   There’s lots that I can contribute here, including non-blocking 
analytics stored in teh database.  If I can make it “generic” enough I’ll try 
to publish it, but the code is really manual.  Maybe instead of a formal 
toolset, I could post a methodology manual?  It’s really tedious though, 
involving materialized database views & all sorts of other techniques.  

BTW My site is at http://www.futureclaw.com 

> 
> 
> I still believe we should stop maintaining an autoreloader as soon as 
> possible. Django’s autoreloader is annoyingly slow, highly inefficient, 
> moderately well designed, and a gigantic pain to maintain. I’m more scared of 
> django.utils.autoreload than of django.db.models.related before it was 
> cleaned up.
> 
> I wish one day someone will take the time to write a good autoreloading dev 
> server based on watchman. This would solve the problem discussed here because 
> watchman watches all files in the current directory. The correct way to do 
> this is to throw away the current design and start from scratch.
> 
> Watchman is smart enough to wait until you’ve finished a git operation to 
> trigger a reload. Once such polished tech has become available, trying to 
> compete with a thread that checks the mtime of all known files every second 
> isn’t funny anymore. In fact it’s just sad.
> 
> 
> 
> While I don’t care much about django.utils.autoreloader because I want to 
> kill it, I’m reluctant to add public APIs such as the proposed setting, which 
> may not make sense with an hypothetical better autoreloader.
> 
> I’m -0 on the change. I could move to +0 if I understood why the use case 
> described here requires watching additional files.
> 

I guess I could just use Watchman to restart the Django development server as 
needed?

I’m OK with that.. my only goal is to make sure I don’t have to restart Django 
manually every time I edit CSS/Javascript/SQL/HTML.

Is that the recommended workflow for this?  I never looked into outside tools 
to manage this, but it seems it would be a lot easier to add a list of files or 
directories to our settings and have Django manage autoreload, than to bring in 
an outside tool.  (I can add directories to watch to the pull request if needed)

> Best regards,
> 
> -- 
> Aymeric.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/BAAB82B3-974C-4240-B9C6-E81F04D9604F%40polytechnique.org.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, se

Re: Add custom autoreload file tracking options setting

[Aymeric Augustin]

Hello Bobby,

> On 4 Jan 2017, at 22:25, Bobby Mozumder  wrote:
> 
> It’s actually called once on app startup during DB connection via a Signal.

Unless I missed something, since the development server creates a new 
connection to the database for each request — Python’s threaded socket server 
creates a new thread for each connection and database connections are thread 
local in Django — prepared statements will be refreshed for each request. So I 
don’t think you need autoreloading here.

> (This whole process helps speed up the view response generation times.  I can 
> generate a full page in about 4ms, and that includes GZip.)

Off topic, but I’m jealous…



I still believe we should stop maintaining an autoreloader as soon as possible. 
Django’s autoreloader is annoyingly slow, highly inefficient, moderately well 
designed, and a gigantic pain to maintain. I’m more scared of 
django.utils.autoreload than of django.db.models.related before it was cleaned 
up.

I wish one day someone will take the time to write a good autoreloading dev 
server based on watchman. This would solve the problem discussed here because 
watchman watches all files in the current directory. The correct way to do this 
is to throw away the current design and start from scratch.

Watchman is smart enough to wait until you’ve finished a git operation to 
trigger a reload. Once such polished tech has become available, trying to 
compete with a thread that checks the mtime of all known files every second 
isn’t funny anymore. In fact it’s just sad.



While I don’t care much about django.utils.autoreloader because I want to kill 
it, I’m reluctant to add public APIs such as the proposed setting, which may 
not make sense with an hypothetical better autoreloader.

I’m -0 on the change. I could move to +0 if I understood why the use case 
described here requires watching additional files.

Best regards,

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/BAAB82B3-974C-4240-B9C6-E81F04D9604F%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Add custom autoreload file tracking options setting

[Bobby Mozumder]

That’ll make development server access times really slow.  There’s a pretty 
long Makefile also that builds Javascript.  We really shouldn’t have to rebuild 
Javascript on every page view, especially since I call interactive API requests 
with these views.

-bobby

> On Jan 4, 2017, at 4:18 PM, Adam Johnson  wrote:
> 
> For that use case I'd suggest just re-executing prepare_db_queries on every 
> page view when DEBUG=True. This is similar to how Django's template loaders 
> work without the cached loader wrapping them.
> 
> On 4 January 2017 at 21:12, Bobby Mozumder  > wrote:
> OK here is some example code snippet where I load prepared SQL statements:
> 
> 
> class FastDetailView(DetailView,FastView):
> 
> c = connection.cursor()
> 
> SQL_VIEW_DIRS = {
> 'fashion': (
> 'include/sql/materializedviews/headlines',
> 'include/sql/materializedviews/latestCollections',
> 'include/sql/materializedviews/allSeasons',
> 'include/sql/materializedviews/fullSeason',
> 'include/sql/materializedviews/gallery',
> 'include/sql/materializedviews/indexView',
> 'include/sql/materializedviews/cover',
> 'include/sql/materializedviews/latestSeasonView',
> 'include/sql/materializedviews/seasonView',
> 'include/sql/materializedviews/collectionView',
> 'include/sql/materializedviews/latestCollectionsJSON',
> 'include/sql/materializedviews/collectionCardJSON',
> 'include/sql/materializedviews/indexJSON',
> 'include/sql/materializedviews/categoryJSON',
> 'include/sql/materializedviews/articleJSON',
> 'include/sql/triggers/globals',
> 'include/sql/triggers/brand',
> 'include/sql/triggers/collection',
> 'include/sql/triggers/collectionlookassignment',
> 'include/sql/triggers/cover',
> 'include/sql/triggers/look',
> 'include/sql/triggers/photo',
> 'include/sql/triggers/season',
> 'include/sql/triggers/fashion_headlinesviewmat',
> 'include/sql/triggers/fashion_latestcollectionsviewmat',
> 'include/sql/triggers/fashion_allseasonsviewmat',
> 'include/sql/triggers/fashion_fullseasonviewmat',
> 'include/sql/triggers/fashion_galleryviewmat',
> 'include/sql/triggers/fashion_coverviewmat',
> 'include/sql/triggers/fashion_indexviewmat',
> 'include/sql/triggers/fashion_latestseasonviewmat',
> 'include/sql/triggers/fashion_seasonviewmat',
> 'include/sql/triggers/fashion_collectionviewmat',
> 'include/sql/triggers/fashion_collectioncardjsonviewmat',
> ),
> 'analytics': (
> 'include/sql/analytics',
> ),
> }
> 
> MATERIALIZED_VIEWS = True
> 
> @classmethod
> def prepare_db_queries(self):
> logger.info ('Reading fashion prepared SQL 
> statements')
> cursor = connection.cursor()
> for sql_view_dir in SQL_VIEW_DIRS['fashion']:
> file_name = sql_view_dir + '/prepare.sql'
> try:
> with open(file_name, 'r') as file:
> sql_prepare=file.read().strip()
> if sql_prepare:
> cursor.execute(sql_prepare)
> except (OSError, IOError) as e:
> pass
> except e:
> logger.info ('Error reading SQL file: 
> %s' % file_name)
> raise e
> if MATERIALIZED_VIEWS:
> file_name = sql_view_dir + '/prepare_materialized.sql'
> try:
> with open(file_name, 'r') as file:
> sql_prepare=file.read().strip()
> if sql_prepare:
> cursor.execute(sql_prepare)
> except (OSError, IOError) as e:
> pass
> except e:
> logger.info ('Error reading SQL 
> file: %s' % file_name)
> raise e
> 
> 
> It’s a custom view class that basically reads SQL from a separate list of 
> files on initialization, and executes those SQL files.  
> 
> If I edit these SQL files, it won't restart the development server.
> 
> -bobby
> 
>> On Jan 4, 2017, at 4:03 PM, Tim Graham > > wrote:
>> 
>> Could you give us a code snippet (sample view, perhaps) demonstrating how 
>> this caching happens?
>> 
>> On Wednesday, January 4, 2017 at 3:57:31 PM UTC-5, Bobby Mozumder wrote:
>> Hi, 
>> 
>> Right now, Django only tracks Python module files for autoreload during 
>> development. As a project starts to include more custom include files, such 
>> as Javascript, SQL, Makefiles, etc.., the autoreload function doesn't apply 

Re: Add custom autoreload file tracking options setting

[Bobby Mozumder]

It’s actually called once on app startup during DB connection via a Signal.

Here is my app.py:

from django.apps import AppConfig
from .signals import *
from django.utils import autoreload

class FashionAppConfig(AppConfig):
name = 'fashion'
verbose_name = "Fashion"

And here is my signals.py:

from django.dispatch import receiver
from django.db.backends.signals import connection_created
from .page import factory as pageFactory
from .api import APIFactory


@receiver(connection_created, dispatch_uid="dbConnectionInitiated")
def prepareSQL(sender, **kwargs):
pageFactory.prepare_db_queries()
APIFactory.prepare_db_queries()
pass


My FastDetailView class is a subclass of factory , and this is where 
prepare_db_queries happens.

I also have separate code that loads in Javascript into this FastDetailView 
class.

Basically I don’t read html from the template at all during the 
Request/Response cycle.  I instead pre-load all my DB Queries and Javascript.

(This whole process helps speed up the view response generation times.  I can 
generate a full page in about 4ms, and that includes GZip.)

-bobby

> On Jan 4, 2017, at 4:17 PM, Tim Graham  wrote:
> 
> When is prepare_db_queries() called? During a request/response cycle? I 
> doesn't look like any caching is happening so I still doesn't see why the 
> server needs to restart to pickup changes to the SQL files.
> 
> On Wednesday, January 4, 2017 at 4:12:27 PM UTC-5, Bobby Mozumder wrote:
> OK here is some example code snippet where I load prepared SQL statements:
> 
> 
> class FastDetailView(DetailView,FastView):
> 
> c = connection.cursor()
> 
> SQL_VIEW_DIRS = {
> 'fashion': (
> 'include/sql/materializedviews/headlines',
> 'include/sql/materializedviews/latestCollections',
> 'include/sql/materializedviews/allSeasons',
> 'include/sql/materializedviews/fullSeason',
> 'include/sql/materializedviews/gallery',
> 'include/sql/materializedviews/indexView',
> 'include/sql/materializedviews/cover',
> 'include/sql/materializedviews/latestSeasonView',
> 'include/sql/materializedviews/seasonView',
> 'include/sql/materializedviews/collectionView',
> 'include/sql/materializedviews/latestCollectionsJSON',
> 'include/sql/materializedviews/collectionCardJSON',
> 'include/sql/materializedviews/indexJSON',
> 'include/sql/materializedviews/categoryJSON',
> 'include/sql/materializedviews/articleJSON',
> 'include/sql/triggers/globals',
> 'include/sql/triggers/brand',
> 'include/sql/triggers/collection',
> 'include/sql/triggers/collectionlookassignment',
> 'include/sql/triggers/cover',
> 'include/sql/triggers/look',
> 'include/sql/triggers/photo',
> 'include/sql/triggers/season',
> 'include/sql/triggers/fashion_headlinesviewmat',
> 'include/sql/triggers/fashion_latestcollectionsviewmat',
> 'include/sql/triggers/fashion_allseasonsviewmat',
> 'include/sql/triggers/fashion_fullseasonviewmat',
> 'include/sql/triggers/fashion_galleryviewmat',
> 'include/sql/triggers/fashion_coverviewmat',
> 'include/sql/triggers/fashion_indexviewmat',
> 'include/sql/triggers/fashion_latestseasonviewmat',
> 'include/sql/triggers/fashion_seasonviewmat',
> 'include/sql/triggers/fashion_collectionviewmat',
> 'include/sql/triggers/fashion_collectioncardjsonviewmat',
> ),
> 'analytics': (
> 'include/sql/analytics',
> ),
> }
> 
> MATERIALIZED_VIEWS = True
> 
> @classmethod
> def prepare_db_queries(self):
> logger.info ('Reading fashion prepared SQL 
> statements')
> cursor = connection.cursor()
> for sql_view_dir in SQL_VIEW_DIRS['fashion']:
> file_name = sql_view_dir + '/prepare.sql'
> try:
> with open(file_name, 'r') as file:
> sql_prepare=file.read().strip()
> if sql_prepare:
> cursor.execute(sql_prepare)
> except (OSError, IOError) as e:
> pass
> except e:
> logger.info ('Error reading SQL file: 
> %s' % file_name)
> raise e
> if MATERIALIZED_VIEWS:
> file_name = sql_view_dir + '/prepare_materialized.sql'
> try:
> with open(file_name, 'r') as file:
> sql_prepare=file.read().strip()
> if sql_prepare:
> cursor.execute(sql_prepare)
> except (OSError, IOError) as e:
> pass
> except e:
>

Re: Add custom autoreload file tracking options setting

[Adam Johnson]

For that use case I'd suggest just re-executing prepare_db_queries on every
page view when DEBUG=True. This is similar to how Django's template loaders
work without the cached loader wrapping them.

On 4 January 2017 at 21:12, Bobby Mozumder  wrote:

> OK here is some example code snippet where I load prepared SQL statements:
>
> class FastDetailView(DetailView,FastView):
>
> c = connection.cursor()
>
> SQL_VIEW_DIRS = {
> 'fashion': (
> 'include/sql/materializedviews/headlines',
> 'include/sql/materializedviews/latestCollections',
> 'include/sql/materializedviews/allSeasons',
> 'include/sql/materializedviews/fullSeason',
> 'include/sql/materializedviews/gallery',
> 'include/sql/materializedviews/indexView',
> 'include/sql/materializedviews/cover',
> 'include/sql/materializedviews/latestSeasonView',
> 'include/sql/materializedviews/seasonView',
> 'include/sql/materializedviews/collectionView',
> 'include/sql/materializedviews/latestCollectionsJSON',
> 'include/sql/materializedviews/collectionCardJSON',
> 'include/sql/materializedviews/indexJSON',
> 'include/sql/materializedviews/categoryJSON',
> 'include/sql/materializedviews/articleJSON',
> 'include/sql/triggers/globals',
> 'include/sql/triggers/brand',
> 'include/sql/triggers/collection',
> 'include/sql/triggers/collectionlookassignment',
> 'include/sql/triggers/cover',
> 'include/sql/triggers/look',
> 'include/sql/triggers/photo',
> 'include/sql/triggers/season',
> 'include/sql/triggers/fashion_headlinesviewmat',
> 'include/sql/triggers/fashion_latestcollectionsviewmat',
> 'include/sql/triggers/fashion_allseasonsviewmat',
> 'include/sql/triggers/fashion_fullseasonviewmat',
> 'include/sql/triggers/fashion_galleryviewmat',
> 'include/sql/triggers/fashion_coverviewmat',
> 'include/sql/triggers/fashion_indexviewmat',
> 'include/sql/triggers/fashion_latestseasonviewmat',
> 'include/sql/triggers/fashion_seasonviewmat',
> 'include/sql/triggers/fashion_collectionviewmat',
> 'include/sql/triggers/fashion_collectioncardjsonviewmat',
> ),
> 'analytics': (
> 'include/sql/analytics',
> ),
> }
>
> MATERIALIZED_VIEWS = True
>
> @classmethod
> def prepare_db_queries(self):
> logger.info('Reading fashion prepared SQL statements')
> cursor = connection.cursor()
> for sql_view_dir in SQL_VIEW_DIRS['fashion']:
> file_name = sql_view_dir + '/prepare.sql'
> try:
> with open(file_name, 'r') as file:
> sql_prepare=file.read().strip()
> if sql_prepare:
> cursor.execute(sql_prepare)
> except (OSError, IOError) as e:
> pass
> except e:
> logger.info('Error reading SQL file: %s' % file_name)
> raise e
> if MATERIALIZED_VIEWS:
> file_name = sql_view_dir + '/prepare_materialized.sql'
> try:
> with open(file_name, 'r') as file:
> sql_prepare=file.read().strip()
> if sql_prepare:
> cursor.execute(sql_prepare)
> except (OSError, IOError) as e:
> pass
> except e:
> logger.info('Error reading SQL file: %s' % file_name)
> raise e
>
>
> It’s a custom view class that basically reads SQL from a separate list of
> files on initialization, and executes those SQL files.
>
> If I edit these SQL files, it won't restart the development server.
>
> -bobby
>
> On Jan 4, 2017, at 4:03 PM, Tim Graham  wrote:
>
> Could you give us a code snippet (sample view, perhaps) demonstrating how
> this caching happens?
>
> On Wednesday, January 4, 2017 at 3:57:31 PM UTC-5, Bobby Mozumder wrote:
>>
>> Hi,
>>
>> Right now, Django only tracks Python module files for autoreload during
>> development. As a project starts to include more custom include files, such
>> as Javascript, SQL, Makefiles, etc.., the autoreload function doesn't apply
>> to these.
>>
>> For my use case, I have custom view functions that call in separate SQL &
>> Javascript files.  (I don’t use the template system.)
>>
>> If I edit these Javascript & SQL files, the Django server doesn’t
>> autoreload.
>>
>> So, I made a pull-request where we can add a manual list of files to
>> track for autoreload: https://github.com/django/django/pull/7791
>>
>> In your project's settings.py file, assign a variable TRACK_FILES
>> containing a list of full file paths to 

Re: Add custom autoreload file tracking options setting

[Tim Graham]

When is prepare_db_queries() called? During a request/response cycle? I 
doesn't look like any caching is happening so I still doesn't see why the 
server needs to restart to pickup changes to the SQL files.

On Wednesday, January 4, 2017 at 4:12:27 PM UTC-5, Bobby Mozumder wrote:
>
> OK here is some example code snippet where I load prepared SQL statements:
>
> class FastDetailView(DetailView,FastView):
>
> c = connection.cursor()
>
> SQL_VIEW_DIRS = {
> 'fashion': (
> 'include/sql/materializedviews/headlines',
> 'include/sql/materializedviews/latestCollections',
> 'include/sql/materializedviews/allSeasons',
> 'include/sql/materializedviews/fullSeason',
> 'include/sql/materializedviews/gallery',
> 'include/sql/materializedviews/indexView',
> 'include/sql/materializedviews/cover',
> 'include/sql/materializedviews/latestSeasonView',
> 'include/sql/materializedviews/seasonView',
> 'include/sql/materializedviews/collectionView',
> 'include/sql/materializedviews/latestCollectionsJSON',
> 'include/sql/materializedviews/collectionCardJSON',
> 'include/sql/materializedviews/indexJSON',
> 'include/sql/materializedviews/categoryJSON',
> 'include/sql/materializedviews/articleJSON',
> 'include/sql/triggers/globals',
> 'include/sql/triggers/brand',
> 'include/sql/triggers/collection',
> 'include/sql/triggers/collectionlookassignment',
> 'include/sql/triggers/cover',
> 'include/sql/triggers/look',
> 'include/sql/triggers/photo',
> 'include/sql/triggers/season',
> 'include/sql/triggers/fashion_headlinesviewmat',
> 'include/sql/triggers/fashion_latestcollectionsviewmat',
> 'include/sql/triggers/fashion_allseasonsviewmat',
> 'include/sql/triggers/fashion_fullseasonviewmat',
> 'include/sql/triggers/fashion_galleryviewmat',
> 'include/sql/triggers/fashion_coverviewmat',
> 'include/sql/triggers/fashion_indexviewmat',
> 'include/sql/triggers/fashion_latestseasonviewmat',
> 'include/sql/triggers/fashion_seasonviewmat',
> 'include/sql/triggers/fashion_collectionviewmat',
> 'include/sql/triggers/fashion_collectioncardjsonviewmat',
> ),
> 'analytics': (
> 'include/sql/analytics',
> ),
> }
>
> MATERIALIZED_VIEWS = True
>
> @classmethod
> def prepare_db_queries(self):
> logger.info('Reading fashion prepared SQL statements')
> cursor = connection.cursor()
> for sql_view_dir in SQL_VIEW_DIRS['fashion']:
> file_name = sql_view_dir + '/prepare.sql'
> try:
> with open(file_name, 'r') as file:
> sql_prepare=file.read().strip()
> if sql_prepare:
> cursor.execute(sql_prepare)
> except (OSError, IOError) as e:
> pass
> except e:
> logger.info('Error reading SQL file: %s' % file_name)
> raise e
> if MATERIALIZED_VIEWS:
> file_name = sql_view_dir + '/prepare_materialized.sql'
> try:
> with open(file_name, 'r') as file:
> sql_prepare=file.read().strip()
> if sql_prepare:
> cursor.execute(sql_prepare)
> except (OSError, IOError) as e:
> pass
> except e:
> logger.info('Error reading SQL file: %s' % file_name)
> raise e
>
>
> It’s a custom view class that basically reads SQL from a separate list of 
> files on initialization, and executes those SQL files.  
>
> If I edit these SQL files, it won't restart the development server.
>
> -bobby
>
> On Jan 4, 2017, at 4:03 PM, Tim Graham > 
> wrote:
>
> Could you give us a code snippet (sample view, perhaps) demonstrating how 
> this caching happens?
>
> On Wednesday, January 4, 2017 at 3:57:31 PM UTC-5, Bobby Mozumder wrote:
>>
>> Hi, 
>>
>> Right now, Django only tracks Python module files for autoreload during 
>> development. As a project starts to include more custom include files, such 
>> as Javascript, SQL, Makefiles, etc.., the autoreload function doesn't apply 
>> to these. 
>>
>> For my use case, I have custom view functions that call in separate SQL & 
>> Javascript files.  (I don’t use the template system.) 
>>
>> If I edit these Javascript & SQL files, the Django server doesn’t 
>> autoreload. 
>>
>> So, I made a pull-request where we can add a manual list of files to 
>> track for autoreload: https://github.com/django/django/pull/7791 
>>
>> In your project's settings.py file, assign a variable TRA

Re: Add custom autoreload file tracking options setting

[Bobby Mozumder]

OK here is some example code snippet where I load prepared SQL statements:


class FastDetailView(DetailView,FastView):

c = connection.cursor()

SQL_VIEW_DIRS = {
'fashion': (
'include/sql/materializedviews/headlines',
'include/sql/materializedviews/latestCollections',
'include/sql/materializedviews/allSeasons',
'include/sql/materializedviews/fullSeason',
'include/sql/materializedviews/gallery',
'include/sql/materializedviews/indexView',
'include/sql/materializedviews/cover',
'include/sql/materializedviews/latestSeasonView',
'include/sql/materializedviews/seasonView',
'include/sql/materializedviews/collectionView',
'include/sql/materializedviews/latestCollectionsJSON',
'include/sql/materializedviews/collectionCardJSON',
'include/sql/materializedviews/indexJSON',
'include/sql/materializedviews/categoryJSON',
'include/sql/materializedviews/articleJSON',
'include/sql/triggers/globals',
'include/sql/triggers/brand',
'include/sql/triggers/collection',
'include/sql/triggers/collectionlookassignment',
'include/sql/triggers/cover',
'include/sql/triggers/look',
'include/sql/triggers/photo',
'include/sql/triggers/season',
'include/sql/triggers/fashion_headlinesviewmat',
'include/sql/triggers/fashion_latestcollectionsviewmat',
'include/sql/triggers/fashion_allseasonsviewmat',
'include/sql/triggers/fashion_fullseasonviewmat',
'include/sql/triggers/fashion_galleryviewmat',
'include/sql/triggers/fashion_coverviewmat',
'include/sql/triggers/fashion_indexviewmat',
'include/sql/triggers/fashion_latestseasonviewmat',
'include/sql/triggers/fashion_seasonviewmat',
'include/sql/triggers/fashion_collectionviewmat',
'include/sql/triggers/fashion_collectioncardjsonviewmat',
),
'analytics': (
'include/sql/analytics',
),
}

MATERIALIZED_VIEWS = True

@classmethod
def prepare_db_queries(self):
logger.info('Reading fashion prepared SQL statements')
cursor = connection.cursor()
for sql_view_dir in SQL_VIEW_DIRS['fashion']:
file_name = sql_view_dir + '/prepare.sql'
try:
with open(file_name, 'r') as file:
sql_prepare=file.read().strip()
if sql_prepare:
cursor.execute(sql_prepare)
except (OSError, IOError) as e:
pass
except e:
logger.info('Error reading SQL file: %s' % file_name)
raise e
if MATERIALIZED_VIEWS:
file_name = sql_view_dir + '/prepare_materialized.sql'
try:
with open(file_name, 'r') as file:
sql_prepare=file.read().strip()
if sql_prepare:
cursor.execute(sql_prepare)
except (OSError, IOError) as e:
pass
except e:
logger.info('Error reading SQL file: %s' % file_name)
raise e


It’s a custom view class that basically reads SQL from a separate list of files 
on initialization, and executes those SQL files.  

If I edit these SQL files, it won't restart the development server.

-bobby

> On Jan 4, 2017, at 4:03 PM, Tim Graham  wrote:
> 
> Could you give us a code snippet (sample view, perhaps) demonstrating how 
> this caching happens?
> 
> On Wednesday, January 4, 2017 at 3:57:31 PM UTC-5, Bobby Mozumder wrote:
> Hi, 
> 
> Right now, Django only tracks Python module files for autoreload during 
> development. As a project starts to include more custom include files, such 
> as Javascript, SQL, Makefiles, etc.., the autoreload function doesn't apply 
> to these. 
> 
> For my use case, I have custom view functions that call in separate SQL & 
> Javascript files.  (I don’t use the template system.) 
> 
> If I edit these Javascript & SQL files, the Django server doesn’t autoreload. 
> 
> So, I made a pull-request where we can add a manual list of files to track 
> for autoreload: https://github.com/django/django/pull/7791 
>  
> 
> In your project's settings.py file, assign a variable TRACK_FILES containing 
> a list of full file paths to track. This will track files to autoreload the 
> development run server as these files are updated. 
> 
> Is this OK?  This pull request is a basic option and I’m sure it can get more 
> complicated than that (directory tracking, Makefiles, Javscript builds, 
> etc..) 
> 
> -bobby
> 
> -- 
> You received this message because you are subscribed to the Google Groups

Re: Add custom autoreload file tracking options setting

[Tim Graham]

Could you give us a code snippet (sample view, perhaps) demonstrating how 
this caching happens?

On Wednesday, January 4, 2017 at 3:57:31 PM UTC-5, Bobby Mozumder wrote:
>
> Hi, 
>
> Right now, Django only tracks Python module files for autoreload during 
> development. As a project starts to include more custom include files, such 
> as Javascript, SQL, Makefiles, etc.., the autoreload function doesn't apply 
> to these. 
>
> For my use case, I have custom view functions that call in separate SQL & 
> Javascript files.  (I don’t use the template system.) 
>
> If I edit these Javascript & SQL files, the Django server doesn’t 
> autoreload. 
>
> So, I made a pull-request where we can add a manual list of files to track 
> for autoreload: https://github.com/django/django/pull/7791 
>
> In your project's settings.py file, assign a variable TRACK_FILES 
> containing a list of full file paths to track. This will track files to 
> autoreload the development run server as these files are updated. 
>
> Is this OK?  This pull request is a basic option and I’m sure it can get 
> more complicated than that (directory tracking, Makefiles, Javscript 
> builds, etc..) 
>
> -bobby

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c857c334-6388-4e10-8367-ffbee08acc10%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Add custom autoreload file tracking options setting

[Bobby Mozumder]

Hi,

Right now, Django only tracks Python module files for autoreload during 
development. As a project starts to include more custom include files, such as 
Javascript, SQL, Makefiles, etc.., the autoreload function doesn't apply to 
these.

For my use case, I have custom view functions that call in separate SQL & 
Javascript files.  (I don’t use the template system.)

If I edit these Javascript & SQL files, the Django server doesn’t autoreload. 

So, I made a pull-request where we can add a manual list of files to track for 
autoreload: https://github.com/django/django/pull/7791

In your project's settings.py file, assign a variable TRACK_FILES containing a 
list of full file paths to track. This will track files to autoreload the 
development run server as these files are updated.

Is this OK?  This pull request is a basic option and I’m sure it can get more 
complicated than that (directory tracking, Makefiles, Javscript builds, etc..)

-bobby

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/77BE6024-04E1-4DC1-89BD-24091F151FFC%40gmail.com.
For more options, visit https://groups.google.com/d/optout.