Re: [Twisted-Python] twisted.web with dynamic + static content

[Glyph]

> On Nov 27, 2018, at 2:37 PM, Jeff Grimmett  wrote:
> 
> Thanks, Tom.  
> 
> Is there a limit to the smallness of the proposed changes?  For example, if I 
> had a possible two-line change to 
> https://twistedmatrix.com/documents/current/api/twisted.internet.wxreactor.html
>  
> 
>  (which looks like it would be done in the docstring), is it okay to just 
> have a PR for that or does it need to be bundled with more changes?
> 
> Thanks,
> 
> Jeff 

Smaller changes are always better!  Sometimes changes need to be larger and 
don't make sense when split up, so it's OK to propose large things, but a 
one-liner is the easiest possible thing to review, and will generally get 
integrated faster.

-g___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web with dynamic + static content

[Jeff Grimmett]

Thanks, Tom.

Is there a limit to the smallness of the proposed changes?  For example, if
I had a possible two-line change to
https://twistedmatrix.com/documents/current/api/twisted.internet.wxreactor.html
(which looks like it would be done in the docstring), is it okay to just
have a PR for that or does it need to be bundled with more changes?

Thanks,

Jeff


On Wed, Nov 21, 2018 at 1:31 AM Tom Most  wrote:

> On Mon, Nov 19, 2018, at 3:06 PM, Jeff Grimmett wrote:
>
> Glyph,
>
> Belated (I was in the thick of a few unrelated crises) thanks - it helps
> to understand a little bit of the background on the way it works.  I have
> made adjustments and everything works the way I was hoping for now.
>
> I wish I understood it a little better, if for no other reason than to
> contribute some documentation tweaks.  Just how would that work, anyway?
> The contribution model for Twisted is a little opaque from where I am
> sitting. I'd love to help in areas where I am actually capable of doing :)
>
> Regards,
>
> Jeff
>
>
> Hi Jeff,
>
> The Twisted development process is documented here on the wiki
> .
> It boils down to:
>
> 1. Create a ticket  (on Trac)
> 2. Create a pull request (on GitHub)
> 3. Wait for a review (and address any feedback the reviewer gives)
>
> There is more on the wiki. The same process applies to code and
> documentation.
>
> Documentation changes are really helpful, especially because it is
> difficult for the experienced to see the rough edges that catch new users.
> Your confusion is a bug, and particularly with respect to the Resource
> model, a not uncommon one. Let's fix it!
>
> ---Tom
>
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web with dynamic + static content

[Tom Most]

On Mon, Nov 19, 2018, at 3:06 PM, Jeff Grimmett wrote:
> Glyph,
> 
> Belated (I was in the thick of a few unrelated crises) thanks - it
> helps to understand a little bit of the background on the way it
> works.  I have made adjustments and everything works the way I was
> hoping for now.> 
> I wish I understood it a little better, if for no other reason than to
> contribute some documentation tweaks.  Just how would that work,
> anyway? The contribution model for Twisted is a little opaque from
> where I am sitting. I'd love to help in areas where I am actually
> capable of doing :)> 
> Regards,
> 
> Jeff 

Hi Jeff,

The Twisted development process is documented here on the wiki[1]. It
boils down to:
1. Create a ticket[2] (on Trac)
2. Create a pull request (on GitHub)
3. Wait for a review (and address any feedback the reviewer gives)

There is more on the wiki. The same process applies to code and
documentation.
Documentation changes are really helpful, especially because it is
difficult for the experienced to see the rough edges that catch new
users. Your confusion is a bug, and particularly with respect to the
Resource model, a not uncommon one. Let's fix it!
---Tom




Links:

  1. 
https://twistedmatrix.com/trac/wiki/TwistedDevelopment#GettingYourPatchAccepted
  2. https://twistedmatrix.com/trac/newticket
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web with dynamic + static content

[Jeff Grimmett]

Glyph,

Belated (I was in the thick of a few unrelated crises) thanks - it helps to
understand a little bit of the background on the way it works.  I have made
adjustments and everything works the way I was hoping for now.

I wish I understood it a little better, if for no other reason than to
contribute some documentation tweaks.  Just how would that work, anyway?
The contribution model for Twisted is a little opaque from where I am
sitting. I'd love to help in areas where I am actually capable of doing :)

Regards,

Jeff


On Thu, Nov 1, 2018 at 3:44 AM Glyph  wrote:

> Hi Jeff,
>
> Thanks for using Twisted.
>
> Here's a version with some small changes that works, and is self-contained.
>
> import sys
>
> from twisted.internet import reactor, endpoints
> from twisted.web import server
> from twisted.web.resource import Resource
> from twisted.web.static import Data
>
> sys.path.append('lib')
>
> content = """
> 
> 
> 
> 
> 
> 
> 
> This is a test
> 
> 
> """
>
> class tServer(Resource):
> def render_GET(self, request):
> return bytes(content, "utf-8")
>
> if __name__ == "__main__":
> root = Resource()
> static_collection = Resource()
> static_collection.putChild(b"test.css", Data(b".twistedTest {color:
> red;}", "text/css"))
> root.putChild(b"static", static_collection)
> root.putChild(b"", tServer())
>
> site = server.Site(root)
> endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
> endpoint.listen(site)
>
> reactor.run()
> print("Shutting down!")
>
>
> The problem with your first version was 'isLeaf', as Donal suggested.
> However, the problem was not simply that the flag was set, but rather what
> the flag means, and why it works that way.
>
> The root resource in any web server is a *collection*.  Which is to say,
> under normal circumstances, the root resource never has render_* invoked on
> it; you can't render it, because it's impossible, in the HTTP protocol, to
> spell a URL that doesn't start with "/".
>
> isLeaf changes this, and says "this resource is responsible for rendering
> all of its children; traversal stops here".  That means that it starts
> invoking render_GET to render "/", but also to render *every other path
> on the server*, including (unfortunately for you) /static/test.css.
>
> The modified example above instead uses a Resource() as the collection,
> and inserts a '' child for the index, and a separate 'static' child for the
> static index.  You can use a static.File for a directory here instead of a
> static resource, and anywhere you see putChild, you could also use a
> dynamic resource which overrides getChild to return the object rather than
> inserting it in advance.
>
> Of course, you might wonder what the point of 'isLeaf' is if it short
> circuits this stuff and makes it impossible to tell the difference between
> resources.
>
> Given that you have a directory, you want to use a static.File child
> resource and almost certainly *don't* want to set isLeaf; however, you
> might be wondering how one would even use isLeaf if it just cuts off the
> ability to tell the difference between resources.  The documentation on
> this is not great - it doesn't even appear as an attribute in the API
> reference, just an oblique reference in the docstring for
> https://twistedmatrix.com/documents/current/api/twisted.web.resource.Resource.html#getChild.
> But, the 'prepath' and 'postpath' attributes, lists of bytes, will tell you
> about where in the request traversal cycle you are, and allow you to
> distinguish which content to render directly within the body of render_*,
> rather than having to route to the right object using Twisted APIs.  So
> here's a working version with isLeaf=True:
>
> import sys
>
> from twisted.internet import reactor, endpoints
> from twisted.web import server
> from twisted.web.resource import Resource
>
> content = """
> 
> 
> 
> 
> 
> 
> 
> This is a test
> 
> 
> """
>
> css = """
> .twistedTest {
> color: red;
> }
> """
>
> class tServer(Resource):
> isLeaf = True
> def render_GET(self, request):
> if request.postpath == [b'']:
> request.setHeader("content-type", "text/html")
> return bytes(content, "utf-8")
> elif request.postpath == [b'static', b'test.css']:
> request.setHeader("content-type", "text/css")
> return bytes(css, 'utf-8')
> else:
> request.setResponseCode(404)
> return b'not found'
>
> if __name__ == "__main__":
> site = server.Site(tServer())
> endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
> endpoint.listen(site)
>
> reactor.run()
> print("Shutting down!")
>
>
> I hope this clears up the request traversal model a little bit.
>
> -glyph
>
>
> On Oct 31, 2018, at 2:15 PM, Jeff Grimmett  wrote:
>
> Tried that, I get a big
>
> No Such Resource
>
> No such child resource.
> back.  Watching it in FF's development panel, I see a 404

Re: [Twisted-Python] twisted.web with dynamic + static content

[Glyph]

Hi Jeff,

Thanks for using Twisted.

Here's a version with some small changes that works, and is self-contained.

import sys

from twisted.internet import reactor, endpoints
from twisted.web import server
from twisted.web.resource import Resource
from twisted.web.static import Data

sys.path.append('lib')

content = """







This is a test


"""

class tServer(Resource):
def render_GET(self, request):
return bytes(content, "utf-8")

if __name__ == "__main__":
root = Resource()
static_collection = Resource()
static_collection.putChild(b"test.css", Data(b".twistedTest {color: red;}", 
"text/css"))
root.putChild(b"static", static_collection)
root.putChild(b"", tServer())

site = server.Site(root)
endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
endpoint.listen(site)

reactor.run()
print("Shutting down!")

The problem with your first version was 'isLeaf', as Donal suggested.  However, 
the problem was not simply that the flag was set, but rather what the flag 
means, and why it works that way.

The root resource in any web server is a collection.  Which is to say, under 
normal circumstances, the root resource never has render_* invoked on it; you 
can't render it, because it's impossible, in the HTTP protocol, to spell a URL 
that doesn't start with "/".

isLeaf changes this, and says "this resource is responsible for rendering all 
of its children; traversal stops here".  That means that it starts invoking 
render_GET to render "/", but also to render every other path on the server, 
including (unfortunately for you) /static/test.css.

The modified example above instead uses a Resource() as the collection, and 
inserts a '' child for the index, and a separate 'static' child for the static 
index.  You can use a static.File for a directory here instead of a static 
resource, and anywhere you see putChild, you could also use a dynamic resource 
which overrides getChild to return the object rather than inserting it in 
advance.

Of course, you might wonder what the point of 'isLeaf' is if it short circuits 
this stuff and makes it impossible to tell the difference between resources.

Given that you have a directory, you want to use a static.File child resource 
and almost certainly don't want to set isLeaf; however, you might be wondering 
how one would even use isLeaf if it just cuts off the ability to tell the 
difference between resources.  The documentation on this is not great - it 
doesn't even appear as an attribute in the API reference, just an oblique 
reference in the docstring for 
https://twistedmatrix.com/documents/current/api/twisted.web.resource.Resource.html#getChild
 
.
  But, the 'prepath' and 'postpath' attributes, lists of bytes, will tell you 
about where in the request traversal cycle you are, and allow you to 
distinguish which content to render directly within the body of render_*, 
rather than having to route to the right object using Twisted APIs.  So here's 
a working version with isLeaf=True:

import sys

from twisted.internet import reactor, endpoints
from twisted.web import server
from twisted.web.resource import Resource

content = """







This is a test


"""

css = """
.twistedTest {
color: red;
}
"""

class tServer(Resource):
isLeaf = True
def render_GET(self, request):
if request.postpath == [b'']:
request.setHeader("content-type", "text/html")
return bytes(content, "utf-8")
elif request.postpath == [b'static', b'test.css']:
request.setHeader("content-type", "text/css")
return bytes(css, 'utf-8')
else:
request.setResponseCode(404)
return b'not found'

if __name__ == "__main__":
site = server.Site(tServer())
endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
endpoint.listen(site)

reactor.run()
print("Shutting down!")

I hope this clears up the request traversal model a little bit.

-glyph


> On Oct 31, 2018, at 2:15 PM, Jeff Grimmett  wrote:
> 
> Tried that, I get a big 
> 
> No Such Resource
> 
> No such child resource.
> 
> back.  Watching it in FF's development panel, I see a 404 come back for /.  
> /static doesn't get served at all, of course.
> 
> This, however, DID work.
> 
> class tServer(Resource):
> isLeaf = False
> 
> def getChild(self, path, request):
> print('You know what you doing.')
> 
> if path == b'':
> print("Rendering /")
> return self
> 
> return Resource.getChild(self, path, request)
> 
> def render_GET(self, request):
> return bytes(content, "utf-8")
> 
> (ignore my printf debugging plz)
> 
> So, Thanks! :)
> 
> Regards,
> 
> Jeff 
> 
> 
> On Tue, Oct 30, 2018 at 6:42 PM Donal McMullan  > wrote:
> Try replacing:
> isLeaf = True
> with
> isLeaf = Fa

Re: [Twisted-Python] twisted.web with dynamic + static content

[Jeff Grimmett]

Tried that, I get a big

No Such Resource

No such child resource.
back.  Watching it in FF's development panel, I see a 404 come back for /.
/static doesn't get served at all, of course.

This, however, DID work.

class tServer(Resource):
isLeaf = False

def getChild(self, path, request):
print('You know what you doing.')

if path == b'':
print("Rendering /")
return self

return Resource.getChild(self, path, request)

def render_GET(self, request):
return bytes(content, "utf-8")

(ignore my printf debugging plz)

So, Thanks! :)

Regards,

Jeff


On Tue, Oct 30, 2018 at 6:42 PM Donal McMullan 
wrote:

> Try replacing:
> isLeaf = True
> with
> isLeaf = False
>
>
> On Tue, 30 Oct 2018 at 21:32, Jeff Grimmett  wrote:
>
>> I'm sure I'm overlooking something obvious here but I just can't get my
>> head around it.
>>
>> Here's the setup: twisted.web server that generates dynamic content.
>> Child that serves up static content, e.g. css and favoicon.  However, the
>> static content isn't making it. Instead, any hit to localhost/static
>> actually yields up a copy of / again.
>>
>> Here's the server code
>>
>> import sys
>>
>> from twisted.internet import reactor, endpoints
>> from twisted.web import server
>> from twisted.web.resource import Resource
>> from twisted.web.static import File
>>
>> sys.path.append('lib')
>>
>> content = """
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> This is a test
>> 
>> 
>> """
>>
>>
>> class tServer(Resource):
>> isLeaf = True
>>
>> def render_GET(self, request):
>> return bytes(content, "utf-8")
>>
>>
>> if __name__ == "__main__":
>> root = tServer()
>> root.putChild(b"static", File("static"))
>>
>> site = server.Site(root)
>> endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
>> endpoint.listen(site)
>>
>> reactor.run()
>> print("Shutting down!")
>>
>> It's run with the command 'python tserver.py'.  The expectation is that
>> what is inside the custom  will be red.
>>
>> In the same dir as the script is a subdir 'static' with the css file
>> inside it.
>>
>> If I replace 'root' with root = Resource() then / doesn't serve up
>> anything, but /static is a directory listing of the static directory.
>>
>> The dynamic server is basically a copy of several tutorials cooked down
>> to something that I could use to demonstrate the problem.
>>
>> What am I missing here? /headscratch
>>
>> Regards,
>>
>> Jeff
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web with dynamic + static content

[Donal McMullan]

Try replacing:
isLeaf = True
with
isLeaf = False


On Tue, 30 Oct 2018 at 21:32, Jeff Grimmett  wrote:

> I'm sure I'm overlooking something obvious here but I just can't get my
> head around it.
>
> Here's the setup: twisted.web server that generates dynamic content. Child
> that serves up static content, e.g. css and favoicon.  However, the static
> content isn't making it. Instead, any hit to localhost/static actually
> yields up a copy of / again.
>
> Here's the server code
>
> import sys
>
> from twisted.internet import reactor, endpoints
> from twisted.web import server
> from twisted.web.resource import Resource
> from twisted.web.static import File
>
> sys.path.append('lib')
>
> content = """
> 
> 
> 
> 
> 
> 
> 
> This is a test
> 
> 
> """
>
>
> class tServer(Resource):
> isLeaf = True
>
> def render_GET(self, request):
> return bytes(content, "utf-8")
>
>
> if __name__ == "__main__":
> root = tServer()
> root.putChild(b"static", File("static"))
>
> site = server.Site(root)
> endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
> endpoint.listen(site)
>
> reactor.run()
> print("Shutting down!")
>
> It's run with the command 'python tserver.py'.  The expectation is that
> what is inside the custom  will be red.
>
> In the same dir as the script is a subdir 'static' with the css file
> inside it.
>
> If I replace 'root' with root = Resource() then / doesn't serve up
> anything, but /static is a directory listing of the static directory.
>
> The dynamic server is basically a copy of several tutorials cooked down to
> something that I could use to demonstrate the problem.
>
> What am I missing here? /headscratch
>
> Regards,
>
> Jeff
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] twisted.web with dynamic + static content

[Jeff Grimmett]

I'm sure I'm overlooking something obvious here but I just can't get my
head around it.

Here's the setup: twisted.web server that generates dynamic content. Child
that serves up static content, e.g. css and favoicon.  However, the static
content isn't making it. Instead, any hit to localhost/static actually
yields up a copy of / again.

Here's the server code

import sys

from twisted.internet import reactor, endpoints
from twisted.web import server
from twisted.web.resource import Resource
from twisted.web.static import File

sys.path.append('lib')

content = """







This is a test


"""


class tServer(Resource):
isLeaf = True

def render_GET(self, request):
return bytes(content, "utf-8")


if __name__ == "__main__":
root = tServer()
root.putChild(b"static", File("static"))

site = server.Site(root)
endpoint = endpoints.TCP4ServerEndpoint(reactor, 8080)
endpoint.listen(site)

reactor.run()
print("Shutting down!")

It's run with the command 'python tserver.py'.  The expectation is that
what is inside the custom  will be red.

In the same dir as the script is a subdir 'static' with the css file inside
it.

If I replace 'root' with root = Resource() then / doesn't serve up
anything, but /static is a directory listing of the static directory.

The dynamic server is basically a copy of several tutorials cooked down to
something that I could use to demonstrate the problem.

What am I missing here? /headscratch

Regards,

Jeff
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python