[kamaelia-list] Re: Small mod to console example doesn't work?
Ah, I forgot the original context you were working in!
All the suggestions I've made previously could be considered as trying to
this problem: that the ConsoleReader component not only reads user input,
but also generates a prompt for it (which it directly writes to the
console (standard output).
It sounds like what you actually are after building is a simple chat
protocol - abstracted away from having to know about where its input is
coming from or where its output is going to. The ConsoleReader component
mixes a bit of both - it handles the sourcing of input, but also generates
some output itself (the prompt) and worse still, it puts this output onto
the console directly! :-)
So what you're actually after is what you originally had:
Pipeline(
SourceOfInput(),
MyComponent(),
DestinationForOutput()
)
So for testing your chat protocol component MyComponent - we can still use
ConsoleReader and ConsoleEchoer. We just have to ignore the '>>>' prompt
generated by the ConsoleReader and pretend its not there.
When you plug Mycomponent into the ConnectedServer (also known as
SimpleServer in more recent subversion copies of Kamaelia iirc) as its
protocol, it gets wired up in much the same way as above - data coming
from the client will get send to its inbox and data send out of its outbox
will be sent back to the client.
So all of the Seq stuff is not strictly necessary. All you need to do is
encode the sequential behaviour into the main() generator in MyComponent.
For example something along the lines of this (note the shutdown handling
makes it a little messier that it might otherwise be):
def main(self):
mustStop = false;
# 1) generate login prompt
self.send("Enter username:", "outbox")
# 2) wait for response
mustStop = self.doShutdown()
while not mustStop and not self.dataReady("inbox"):
self.pause()
yield 1
mustStop = self.doShutdown()
if mustStop:
return
username = self.recv("inbox")
# 3) welcome
self.send("Welcome "+username+"! Begin chatting!")
# 4) do chatting
...
One other thing to consider is that the ConsoleReader specifically buffers
input until a whole line is received from the user, before outputting that
as a single string message to its "outbox" outbox. Whereas the socket
connection may send fragments of strings before a whole line has been
entered. A component such as this one can sort that out for you:
http://www.kamaelia.org/Components/pydoc/Kamaelia.Visualisation.PhysicsGraph.chunks_to_lines.html
Simply pipeline it with MyComponent and make that the protocol thats given
to the ConnectedServer:
def myProtocol():
return Pipeline(
chunks_to_lines(),
MyComponent()
)
...
ConnectedServer(protocol=myProtocol, port=WHATEVER).run()
Matt
On Tue, 03 Mar 2009 04:35:07 -, Gloria W wrote:
> I got the carousel example working, but it sends/receives from/to the
> original console, not any new client which connects. I realized I can
> reuse the ConsoleEchoer, but somehow make it send a string to a client
> upon connect, and have the server process the string instead of echoing
> it to the server's console.
> I tried this, but all proposed solutions so far write to the console,
> not a newly connected client. Maybe I am missing something, btu I think
> I am close.
>
> Aside from this, I was wondering if a carousel or seq will be
> appropriate to solve my problem.
>
> I am trying to simulate what happens in crude chat systems such as IRC.
> For simplicity, in this example the user connects, is prompted for a
> name, and then can chat freely.
> If I have two pipelines: one detects client connect and is prompting for
> a name, receiving the response, and then handing control over to another
> pipeline; doesn't this require a disconnect/reconnect with the client?
> If so, this would be bad, because the user would have to re-telnet in,
> would not be "logged in", and would have to start again.
>
> So I guess if I get the data flow working, I can write this logic in one
> pipeline, and reuse the same connection, assuming the first thing a user
> types after connecting would be the name. Or maybe there is a clever way
> to hand off a connected pipline to another utility, so that my flow
> control is cleaner? (not running an "if name is set" check upon each
> message).
>
> Thanks again for your help,
> Gloria
>
>
>> This is really cool, and exactly what I need. I will get some time
>> later to implement this and try it out, and I am sure I'll have more
>> questions and comments.
>> Thanks again for this.
>> Gloria
>>> On Mar 2, 8:36 pm, Gloria W wrote:
>>>
Matt, thanks for this response. It took me a while to get through it
and
make sure I understood each point. I app
[kamaelia-list] Re: Small mod to console example doesn't work?
I got the carousel example working, but it sends/receives from/to the
original console, not any new client which connects. I realized I can
reuse the ConsoleEchoer, but somehow make it send a string to a client
upon connect, and have the server process the string instead of echoing
it to the server's console.
I tried this, but all proposed solutions so far write to the console,
not a newly connected client. Maybe I am missing something, btu I think
I am close.
Aside from this, I was wondering if a carousel or seq will be
appropriate to solve my problem.
I am trying to simulate what happens in crude chat systems such as IRC.
For simplicity, in this example the user connects, is prompted for a
name, and then can chat freely.
If I have two pipelines: one detects client connect and is prompting for
a name, receiving the response, and then handing control over to another
pipeline; doesn't this require a disconnect/reconnect with the client?
If so, this would be bad, because the user would have to re-telnet in,
would not be "logged in", and would have to start again.
So I guess if I get the data flow working, I can write this logic in one
pipeline, and reuse the same connection, assuming the first thing a user
types after connecting would be the name. Or maybe there is a clever way
to hand off a connected pipline to another utility, so that my flow
control is cleaner? (not running an "if name is set" check upon each
message).
Thanks again for your help,
Gloria
> This is really cool, and exactly what I need. I will get some time
> later to implement this and try it out, and I am sure I'll have more
> questions and comments.
> Thanks again for this.
> Gloria
>> On Mar 2, 8:36 pm, Gloria W wrote:
>>
>>> Matt, thanks for this response. It took me a while to get through it and
>>> make sure I understood each point. I appreciate it, comments are below.
>>>
>>
>> No problem.
>>
>> I've just realised another way to achieve the same effect that is
>> probably simpler!
>>
>> There *is* a component that is all about getting things done in an
>> order: the Seq component (The name is a homage to the OCCAM language):
>>
>> http://www.kamaelia.org/Components/pydoc/Kamaelia.Chassis.Seq.html
>>
>> With Seq, you provide a list of components. Seq goes through the list,
>> starting with the first component. It activates it and wires up its
>> inboxes and outboxes so that they're mapped to the Seq component's
>> inboxes and outboxes (just like the Carousel does). When that
>> component finishes, it then unwires it and moves onto the next in the
>> list. It therefore effectively runs them *in sequence* one after the
>> other.
>>
>> We've got two steps: displaying the initial message, then taking user
>> input. Both steps need their output to be displayed. So we'll create a
>> Seq componet, Pipeline'd into a console echoer:
>>
>> from Kamaelia.Chassis.Seq import Seq
>>
>> Pipeline(
>> Seq( ... args to be determined! ... ),
>> ConsoleEchoer()
>> ).run()
>>
>> We can use a OneShot component (see
>> http://www.kamaelia.org/Components/pydoc/Kamaelia.Util.OneShot.html
>> ) to send that initial message, so that it gets sent to the
>> ConsoleEchoer and displayed. OneShot, rather conveniently, then
>> immediately terminates; so the Seq component will move onto the next
>> component in its list. That next component can be the Pipeline of a
>> ConsoleReader, sending its output into MyComponent.
>>
>> So what we have is:
>>
>> from Kamaelia.Chassis.Seq import Seq
>> from Kamaelia.Util.OneShot import OneShot
>>
>> Pipeline(
>> Seq(
>> OneShot("initial message to be displayed"),
>> Pipeline(
>> ConsoleReader(),
>> MyComponent()
>> )
>> ),
>> ConsoleEchoer()
>> ).run()
>>
>> In this case, MyComponent doesn't need to bother to send the initial
>> message itself, as it did in the suggestion I previously made.
>>
>> So what happens here? Well, when the system starts running, initially
>> the Seq component selects the first item in its list - the OneShot
>> component. So, what actually ends up getting wired up is something
>> like this:
>>
>> Pipeline(
>> OneShot("initial message to be displayed"),
>> ConsoleEchoer()
>> )
>>
>> Its not actually exactly like that, because the OneShot is still
>> contained within the Seq component - but because all the inboxes and
>> outboxes are mapped to that of the Seq component, it is roughly
>> equivalent.
>>
>> Then when the OneShot component has sent its message and terminated,
>> the Seq component swaps it out and replaces it with the next thing in
>> its list - the pipeline of a ConsoleReader and MyComponent. So the way
>> the system is wired up suddenly switches to something akin to this:
>>
>> Pipeline(
>> Pipeline(
>> ConsoleReader(),
>> MyComponent()
>> ),
>>
[kamaelia-list] Re: Small mod to console example doesn't work?
This is really cool, and exactly what I need. I will get some time later
to implement this and try it out, and I am sure I'll have more questions
and comments.
Thanks again for this.
Gloria
> On Mar 2, 8:36 pm, Gloria W wrote:
>
>> Matt, thanks for this response. It took me a while to get through it and
>> make sure I understood each point. I appreciate it, comments are below.
>>
>
> No problem.
>
> I've just realised another way to achieve the same effect that is
> probably simpler!
>
> There *is* a component that is all about getting things done in an
> order: the Seq component (The name is a homage to the OCCAM language):
>
> http://www.kamaelia.org/Components/pydoc/Kamaelia.Chassis.Seq.html
>
> With Seq, you provide a list of components. Seq goes through the list,
> starting with the first component. It activates it and wires up its
> inboxes and outboxes so that they're mapped to the Seq component's
> inboxes and outboxes (just like the Carousel does). When that
> component finishes, it then unwires it and moves onto the next in the
> list. It therefore effectively runs them *in sequence* one after the
> other.
>
> We've got two steps: displaying the initial message, then taking user
> input. Both steps need their output to be displayed. So we'll create a
> Seq componet, Pipeline'd into a console echoer:
>
> from Kamaelia.Chassis.Seq import Seq
>
> Pipeline(
> Seq( ... args to be determined! ... ),
> ConsoleEchoer()
> ).run()
>
> We can use a OneShot component (see
> http://www.kamaelia.org/Components/pydoc/Kamaelia.Util.OneShot.html
> ) to send that initial message, so that it gets sent to the
> ConsoleEchoer and displayed. OneShot, rather conveniently, then
> immediately terminates; so the Seq component will move onto the next
> component in its list. That next component can be the Pipeline of a
> ConsoleReader, sending its output into MyComponent.
>
> So what we have is:
>
> from Kamaelia.Chassis.Seq import Seq
> from Kamaelia.Util.OneShot import OneShot
>
> Pipeline(
> Seq(
> OneShot("initial message to be displayed"),
> Pipeline(
> ConsoleReader(),
> MyComponent()
> )
> ),
> ConsoleEchoer()
> ).run()
>
> In this case, MyComponent doesn't need to bother to send the initial
> message itself, as it did in the suggestion I previously made.
>
> So what happens here? Well, when the system starts running, initially
> the Seq component selects the first item in its list - the OneShot
> component. So, what actually ends up getting wired up is something
> like this:
>
> Pipeline(
> OneShot("initial message to be displayed"),
> ConsoleEchoer()
> )
>
> Its not actually exactly like that, because the OneShot is still
> contained within the Seq component - but because all the inboxes and
> outboxes are mapped to that of the Seq component, it is roughly
> equivalent.
>
> Then when the OneShot component has sent its message and terminated,
> the Seq component swaps it out and replaces it with the next thing in
> its list - the pipeline of a ConsoleReader and MyComponent. So the way
> the system is wired up suddenly switches to something akin to this:
>
> Pipeline(
> Pipeline(
> ConsoleReader(),
> MyComponent()
> ),
> ConsoleEchoer()
> )
>
> Again, the inner pipeline is, in reality, contained within the Seq
> component, but it behaves roughly like it is shown.
>
>
>
>
> Matt
>
>
>
> >
>
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to kamaelia@googlegroups.com
To unsubscribe from this group, send email to
kamaelia+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~--~~~~--~~--~--~---
[kamaelia-list] Re: Bug in SingleShotHTTPClient
Michael, Thank you again for looking into this. Also, I don't have a lot of network code experience in python, so please take all this with ample salt. > I'm beginning to think I ought to start up a windows VM and see if I can > reproduce this there. I would like to see big K be well supported on windows. And I'd be happy to help in anyway I could. I can't donate a microsoft license to you, but I can test locally. I bet you could get MS to donate a license. But if you need any help setting up a VM, let me know. I use vmware and virtualbox. > Now there's several ways that I could go down this, but I can see that > probably the simplest would be to add a connection timeout this way: > > class TCPClient(Axon.Component.component): >def __init__(self,host,port,delay=0,connect_timeout=60): > self.connect_timeout = connect_timeout > ... >connect_start = time.time() >while not self.safeConnect(sock,(self.host, self.port)): > if self.shutdown(): > return > if ( time.time() - connect_start ) > self.connect_timeout: > self.howDied = "timeout" > raise Finality > yield 1 I am +1 to the idea of including a timeout parameter. In fact, I think every network operation call inside Kamaelia should expose a defaulted timeout parameter. That said, my gut feeling is that the timeout should be handled at the lowest level possible and then exposed all the way up the call tree. For example: > connection, it would hit this logic path in safeConnect: > try: > sock.connect(*sockArgsList); # Expect socket.error: (115, 'Operation > now in progress') > > except socket.error, socket.msg: > (errorno, errmsg) = socket.msg.args > if errorno==errno.EALREADY: > # The socket is non-blocking and a previous connection attempt > has not yet been completed > # We handle this by allowing the code to come back and > repeatedly retry > # connecting. This is a valid, if brute force approach. > assert(self.connecting==1) > return False > elif errorno==errno.EINPROGRESS or errorno==errno.EWOULDBLOCK: > #The socket is non-blocking and the connection cannot be > completed immediately. > # We handle this by allowing the code to come back and > repeatedly retry > # connecting. Rather brute force. > self.connecting=1 > return False # Not connected should retry until no error Here we have EALREADY, EINPROGRESS and EWOULDBLOCK. I think there needs to be a way to timeout these connection attempts rather than simply not starting another attempt after some timeout period. Why would we want to keep retrying the connection during the timeout period? I think it should only make 1 single connection attempt and wait at most timeout period for success. On an only slightly related note, I remember reading a posting once by Glyph where he was promoting one of the values for using twisted. He said that they had spent considerable time testing and debugging the frustratingly disparate socket behaviors on the 3 major platforms and only twisted really "did the right thing" while still exposing a uniform framework interface. My brain doesn't fit twisted, but now I'm starting to appreciate what he was talking about. Thanks, Steve --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "kamaelia" group. To post to this group, send email to kamaelia@googlegroups.com To unsubscribe from this group, send email to kamaelia+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/kamaelia?hl=en -~--~~~~--~~--~--~---
[kamaelia-list] Re: Small mod to console example doesn't work?
On Mar 2, 8:36 pm, Gloria W wrote:
> Matt, thanks for this response. It took me a while to get through it and
> make sure I understood each point. I appreciate it, comments are below.
No problem.
I've just realised another way to achieve the same effect that is
probably simpler!
There *is* a component that is all about getting things done in an
order: the Seq component (The name is a homage to the OCCAM language):
http://www.kamaelia.org/Components/pydoc/Kamaelia.Chassis.Seq.html
With Seq, you provide a list of components. Seq goes through the list,
starting with the first component. It activates it and wires up its
inboxes and outboxes so that they're mapped to the Seq component's
inboxes and outboxes (just like the Carousel does). When that
component finishes, it then unwires it and moves onto the next in the
list. It therefore effectively runs them *in sequence* one after the
other.
We've got two steps: displaying the initial message, then taking user
input. Both steps need their output to be displayed. So we'll create a
Seq componet, Pipeline'd into a console echoer:
from Kamaelia.Chassis.Seq import Seq
Pipeline(
Seq( ... args to be determined! ... ),
ConsoleEchoer()
).run()
We can use a OneShot component (see
http://www.kamaelia.org/Components/pydoc/Kamaelia.Util.OneShot.html
) to send that initial message, so that it gets sent to the
ConsoleEchoer and displayed. OneShot, rather conveniently, then
immediately terminates; so the Seq component will move onto the next
component in its list. That next component can be the Pipeline of a
ConsoleReader, sending its output into MyComponent.
So what we have is:
from Kamaelia.Chassis.Seq import Seq
from Kamaelia.Util.OneShot import OneShot
Pipeline(
Seq(
OneShot("initial message to be displayed"),
Pipeline(
ConsoleReader(),
MyComponent()
)
),
ConsoleEchoer()
).run()
In this case, MyComponent doesn't need to bother to send the initial
message itself, as it did in the suggestion I previously made.
So what happens here? Well, when the system starts running, initially
the Seq component selects the first item in its list - the OneShot
component. So, what actually ends up getting wired up is something
like this:
Pipeline(
OneShot("initial message to be displayed"),
ConsoleEchoer()
)
Its not actually exactly like that, because the OneShot is still
contained within the Seq component - but because all the inboxes and
outboxes are mapped to that of the Seq component, it is roughly
equivalent.
Then when the OneShot component has sent its message and terminated,
the Seq component swaps it out and replaces it with the next thing in
its list - the pipeline of a ConsoleReader and MyComponent. So the way
the system is wired up suddenly switches to something akin to this:
Pipeline(
Pipeline(
ConsoleReader(),
MyComponent()
),
ConsoleEchoer()
)
Again, the inner pipeline is, in reality, contained within the Seq
component, but it behaves roughly like it is shown.
Matt
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to kamaelia@googlegroups.com
To unsubscribe from this group, send email to
kamaelia+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~--~~~~--~~--~--~---
[kamaelia-list] Re: Bug in SingleShotHTTPClient
Hi Steve,
This email may read a little odd. I've been writing this whilst reading and
trying things out, saw your update, and having had a thought. As a result
the train of thought changes as I go through this, but I've left that in
since it may be of use.
On Monday 02 March 2009 21:11:53 Steve wrote:
..
> TCPClient('127.0.0.1', 80).run()
>
> I have no server running on port 80 so it should timeout and return.
> Instead it tries (in a seemingly infinite loop) to make the connection. My
> software firewall (set to allow all) reports around 1 connection attempt
> every second.
That's rather odd. I've just tried the same thing here, and I don't see any
looping attempt to connect - it fails to connect and exits straight away:
~> python
Python 2.5.1 (r251:54863, Jan 10 2008, 18:01:57)
[GCC 4.2.1 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from Kamaelia.Internet.TCPClient import TCPClient
>>> TCPClient("127.0.0.1", 81).run()
>>>
That said, I'm not running a firewall though...
I've also checked that it's passing on shutdown correctly. First the case
that doesn't exit, until control-c :
>>> from Kamaelia.Chassis.Pipeline import Pipeline
>>> from Kamaelia.Util.Console import ConsoleEchoer
>>> Pipeline( ConsoleEchoer() ).run()
And then a version that does exit because the TCPClient passes on it's
shutdown on error:
~> python
Python 2.5.1 (r251:54863, Jan 10 2008, 18:01:57)
[GCC 4.2.1 (SUSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from Kamaelia.Chassis.Pipeline import Pipeline
>>> from Kamaelia.Util.Console import ConsoleEchoer
>>> from Kamaelia.Internet.TCPClient import TCPClient
>>> Pipeline( TCPClient("127.0.0.1", 81), ConsoleEchoer() ).run()
>>>
All this tells me is that the situation I've tested it in and mainly used
it in still works as expected. This isn't the same as your situation.
As a result, I'm no real information up - I'm not reproducing your
behaviour/environment correctly yet.
I'm beginning to think I ought to start up a windows VM and see if I can
reproduce this there.
> The end result is that I believe Kamaelia (or the socket layer) is not
> properly handling a silently refused connection
I can see this is possible, simply because a filtered connection that
doesn't send a TCP RESET won't cause an error at the socket layer
(it just won't connect). That leads to this hanging:
>>> Pipeline( TCPClient("www.google.com", 8000), ConsoleEchoer() ).run()
...
Which is (part of) why you're after timeouts etc, and go after that root
cause.
> Michael, I'm starting to think that this whole TTL component that I've
> made might be completely unneeded. I think I was just reacting to the
> bugs that I've been describing in other threads. If we could identify
> why refused connections are infinite looping on vista and why
> unconnected udp peers are refusing to shutdown, there might be no need
> for this kind of time terminating component
I agree that going after the root cause is a good idea. (That said, the TTL
component is sufficiently generic to be useful beyond this issue, even if
we deal with it differently)
I think you're hitting a combination of things, in the main code, we have:
while not self.safeConnect(sock,(self.host, self.port)):
if self.shutdown():
return
yield 1
This in combination with the other parts is probably why you're seeing a
looping connect attempt.
This goes into safeConnect, and if you're connecting to a filtered
connection, it would hit this logic path in safeConnect:
try:
sock.connect(*sockArgsList); # Expect socket.error: (115, 'Operation
now in progress')
except socket.error, socket.msg:
(errorno, errmsg) = socket.msg.args
if errorno==errno.EALREADY:
# The socket is non-blocking and a previous connection attempt has
not yet been completed
# We handle this by allowing the code to come back and repeatedly
retry
# connecting. This is a valid, if brute force approach.
assert(self.connecting==1)
return False
elif errorno==errno.EINPROGRESS or errorno==errno.EWOULDBLOCK:
#The socket is non-blocking and the connection cannot be completed
immediately.
# We handle this by allowing the code to come back and repeatedly
retry
# connecting. Rather brute force.
self.connecting=1
return False # Not connected should retry until no error
ie it hits one of these three conditions.
For what it's worth a non-filtered not connected socket hits this path:
try:
sock.connect(*sockArgsList); # Expect socket.error: (115, 'Operation
now in progress')
except socket.error, socket.msg:
(errorno, errmsg) = socket.msg.args
if errorno==errno.EALREADY:
# Anything else is an erro
[kamaelia-list] Bug with Unconnected UDP Peers
I'm running on Vista, so again maybe this is a windows specific problem. I have two different application codes that work similarly. One is a server that rotates through a range of ports. It creates a UDP Peer on a port, waits for data, and after a time sends a shutdownMicroprocess and rotates to the next port. The problem is that my software firewall shows these local ports are never relinquished. In in fact, if I wait long enough the rotate algorithm goes through 1000 ports and starts back over. When that happens I get an exception that the requested port is already bound. In the server code I used UDP_ng.TargettedPeer, but I think it's a problem in the basic peer code. The second program is a client side version. It does the same thing except it binds to a port, sends data, waits for a response, rotates, etc. The result is the same thing. The ports are never relniquished. In the client side code I used UDP_ng.SimplePeer, but again I think it's in the base code. At one point I added an extra ._isStopped() check (it wasn't stopped) and then called the .stop() method. This indeed stopped the microprocess but still left the port bound. Any ideas? Could this be related to the TCP ignored connection bug that I was describing in another thread? Thanks, Steve --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "kamaelia" group. To post to this group, send email to kamaelia@googlegroups.com To unsubscribe from this group, send email to kamaelia+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/kamaelia?hl=en -~--~~~~--~~--~--~---
[kamaelia-list] Re: Request small Code Review
On Mar 1, 3:43 pm, I wrote: > Yes, that is indeed unusual. I was struggling a lot with getting > components to shutdown when I tell them. The cascading conditionals Michael, I'm starting to think that this whole TTL component that I've made might be completely unneeded. I think I was just reacting to the bugs that I've been describing in other threads. If we could identify why refused connections are infinite looping on vista and why unconnected udp peers are refusing to shutdown, there might be no need for this kind of time terminating component. --Steve --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "kamaelia" group. To post to this group, send email to kamaelia@googlegroups.com To unsubscribe from this group, send email to kamaelia+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/kamaelia?hl=en -~--~~~~--~~--~--~---
[kamaelia-list] Re: Bug in SingleShotHTTPClient
Michael,
Thank you for taking a look at this. I slimmed down my test case to
just:
from Kamaelia.Internet.TCPClient import TCPClient
if __name__ == '__main__':
TCPClient('www.google.com', 80).run()
I also tested with this line instead:
TCPClient('127.0.0.1', 80).run()
I was thrown off by the UDP traffic generated my the name resolution
of localhost, but the real problem is the handling of a connection
that is refused. I have no server running on port 80 so it should
timeout and return. Instead it tries (in a seemingly infinite loop)
to make the connection. My software firewall (set to allow all)
reports around 1 connection attempt every second. The python process
also consumes ~85% of one of my cores during this time.
Things get even weireder when I test with:
TCPClient('localhost', 80).run()
Here my firewall reports the same TCP connection attempts as the 127
test, but it also reports thousands of UDP packets as well. I suspect
every second it is doing some name resolution which is generating the
UDP traffic.
The end result is that I believe Kamaelia (or the socket layer) is not
properly handling a silently refused connection. There may be a
similar problem with UDP sockets because I was about to detail an
inability to shutdown unconnected UDP peers.
Thanks,
Steve
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to kamaelia@googlegroups.com
To unsubscribe from this group, send email to
kamaelia+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~--~~~~--~~--~--~---
[kamaelia-list] Re: Small mod to console example doesn't work?
Matt, thanks for this response. It took me a while to get through it and
make sure I understood each point. I appreciate it, comments are below.
>> This is true, thanks. Now let's say I wanted to write to the client's
>> console before reading anything from their console.
>>
>> With no other changes, this Pipeline setting doesn't work:
>> Pipeline(
>> MyComponent(),
>> ConsoleReader(),
>> ).run()
>>
>> but I am expecting it to work. What am I missing?
>>
>
> "Pipeline" isn't about saying what order to run things in - its more like
> expressing how to plug things together: The declaration above is saying:
> """ Make two components - an instance of MyComponent and an instance of
> ConsoleReader; now plug them together so that the "outbox" of
> "MyComponent" is joined to the "inbox" of "ConsoleReader" """
>
I was assuming an order of operations here, Thanks for the correction.
It makes sense.
>
>> This is true, thanks. Now let's say I wanted to write to the client's
>> console before reading anything from their console.
>>
>
> So you want a program where the flow of data is as follows:
>
> input from console ---> MyComponent > output to console
>
I guess it would be more like:
send a one-time prompt for input to the client's console --->
read client's response and send to MyComponent
then create the flow you specified:
input from console ---> MyComponent > output to console
Imagine, for example, taking the SimpleChatServer example, and sending
an "Enter your chat name" prompt, getting a response, and THEN
connecting the user to the server.
[snip]
> Thats the simple version. Now a little more complexity! ...
>
> The above code won't guarantee is whether the "initial message for the
> console" is displayed before the prompt that is displayed by the
> ConsoleReader. Why is this? Because the Pipeline declaration effectively
> says "make all these and start them all at teh same time".
>
Got it. Thanks. This helped greatly.
> If you want explicit behaviour where a component isn't created or
> activated, or wired up until a later point in the execution (ie. you want
> to control the order things happen in) then you need to express it
> explicitly.
>
> Here's *a* way to do it:
>
> We can use the Carousel component do delay the creation and wiring up of
> the ConsoleReader component until MyComponent sends a message telling it
> to do so. There's some explanation of Carousel here:
> http://www.kamaelia.org/Cookbook/Carousels if it helps.
>
> In short, Carousel is designed to contain another component. When it
> receives a message, it calls a helper function (that you supply) to create
> a component. It then takes that component and wires it up so that its
> "outbox" and "inbox" are mapped to the "outbox" and "inbox" of the
> Carousel component itself. We'll declare this helper globally for now:
>
> def makeConsoleReader(message): # we ignore the parameter
> return ConsoleReader()
>
>
> So that MyComponent can send a message to the Carousel, we'll need to add
> an extra outbox to MyComponent, and get it to send that signal:
>
> Class MyComponent(component):
> Outboxes = {"outbox" : "some data out",
> "signal" : "Shutdown signal",
> "ready" : "Sends a message when initialisation
> is complete",
> }
>
> ...
>
> def main(self):
> self.send("Initial message for the console", "outbox")
>
> yield 1
>
> self.send("OK", "ready")
>
> while not self.doShutdown():
> if self.dataReady("inbox"):
> data = self.recv("inbox")
> # let's echo what we received...
> self.send(data, 'outbox')
>
> if not self.anyReady():
> self.pause()
>
> yield 1
>
> Note the 'yield' statements - which give other components a chance to run
> - giving the ConsoleEchoer a chance to display that initial message.
>
Makes sense, thanks!
> Remember that we don't control the precise order in which components
> execute: if we sent the initial message and the ready message without a
> yield in between, we might be unlucky, and the Carousel might get executed
> first!
>
> We cannot use a simple pipeline now, since we need to also wire up the
> "ready" outbox of MyComponent to the Carousel's "next" inbox. A Pipeline
> only wires up the standard "inbox" and "control" inboxes and "outbox" and
> "signal" outboxes. So we'll use a Graphline instead, where we specify each
> linkage explicitly:
>
> from Kamaelia.Chassis.Carousel import Carousel
> from Kamaelia.Chassis.Graphline import Graphline
>
>
>
> Graphline(
> READER = Carousel(makeConsoleReader),
> MYCOMP = MyComponent(),
> OUTPUT = ConsoleEchoer(),
> lin
[kamaelia-list] Re: Installation problem under Windows
Michael -
Thanks for taking a look at this. All in all I'm very impressed by
Kamaelia and look forward to working with it.
Miles
On Sun, Mar 1, 2009 at 5:07 PM, Michael Sparks wrote:
>
> Hi Miles,
>
>
> On Mar 1, 3:50 pm, Miles wrote:
>> Hi All -
>>
>> I had a bit of a problem installing Kamaelia 0.6.0 under Windows, and
>> I thought I'd pass along my experience.
>
> Many thanks for this, it's really useful - I personally tend to only
> use Kamaelia on Linux or Mac OS X[1], so reports like this are very
> welcome.
> [1] the former more than the latter
>
>> I'm using Python 2.6.1 on Windows XP Service Pack 3.
>>
>> * Kamaelia appears to require setuptools to install. Setuptools
>> doesn't have a Windows installer for 2.6, so I installed from source
>> using python setup.py install. Not a big deal, but still a touch
>> clumsy.
>
> This was a decision taken after Pycon UK last year when there were
> comments
> that it would be nice to use easy_install to install Kamaelia. I
> personally
> tend prefer a "python setup.py install" approach, but it seemed a
> relatively
> simple change. I wasn't aware at the time of this issue with
> setuptools on
> windows.
>
>> * Once setuptools was installed, I tried "easy_install Kamaelia".
>> This resulted in the following stack trace:
> ...
>> dir = convert_path(f[0])
>> File "c:\Python26\lib\distutils\util.py", line 164, in convert_path
>> raise ValueError, "path '%s' cannot be absolute" % pathname
>> ValueError: path '/usr/local/share/kamaelia' cannot be absolute
>
> This was recently raised by another person on the list, but I hadn't
> had
> a chance to look into resolving the issue (cf note above).
>
>> Installing Kamaelia from source (python setup.py install) gave me the
>> same error.
>
> Indeed.
>
>> The problem appears to be the following line in setup.py:
>>
>> data_files=[ ('/usr/local/share/kamaelia', ['App/
>> kamaelia_logo.png']) ],
>>
>> I changed this to:
>>
>> data_files=[ ('share/kamaelia', ['App/kamaelia_logo.png']) ],
>>
>> and it seemed to work (it place the logo under the egg directory in
>> share/kamaelia).
>
> Interesting. I'll explore how that works on Linux/Mac OS X and find
> out
> where things get put (or rather how apps can find them). This had been
> discussed on IRC in the past couple of days as well, and I was
> pondering
> how to resolve this, and this looks like a potential solution -
> thanks! :)
>
>> Is anyone using Kamaelia under Windows in production? I'm evaluating
>> it for this purpose and any feedback would be helpful. Thanks!
>
> I don't know of anyone using Kamaelia under windows in production.
> However, we have put fixes in for windows in the past when issues have
> been discovered (for example different errno's for socket errors from
> Unix), and significant chunks of code should just work.
>
> My suggestion would be to play with the system to see if it fits your
> usecase, and to let us know if you encounter any issues. My
> expectation is
> that thinks should generally just work, depending on what you're
> aiming to
> do.
>
> However I won't come outright and say "This will definitely be
> perfect" for
> the simple reason I don't personally use Kamaelia on windows enough to
> feel
> comfortable making that claim (I tend to be cautious :) .
>
> Regards,
>
>
> Michael.
> --
> http://yeoldeclue.com/blog
> http://twitter.com/kamaelian
> http://www.kamaelia.org/Home
>
> >
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to kamaelia@googlegroups.com
To unsubscribe from this group, send email to
kamaelia+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~--~~~~--~~--~--~---
[kamaelia-list] Re: Submitting my own console writer for review.
This helped tremendously, thank you!
Gloria
> Gloria,
>
> I'm very new to Kamaelia, but I'm going to take a stab at this:
>
> 1) If you don't inherit from threadedcomponent then your component's
> main function needs to be a generator. Generators use yield
> statements to temporarily suspend execution and send a value back to
> the calling context. Later, the generator is resumed in place where
> it previously yielded.
>
> 2) You need to yield a non-zero value for the component's microprocess
> to be rescheduled for execution.
>
> 3) Pausing is a good idea, but it doesn't happen until you yield, so
> looping on a pause doesn't pause.
>
> 4) Looping on waiting for data in the inbox won't detect a shutdown
> request, so you need to loop on the shutdown request and simlpy check
> for dataReady in the inbox (not an inner loop).
>
> 5) Received data is returned from recv('boxname') not assigned to a
> new parameter.
>
> 6) There is no need to have an explicit __init__ method unless you are
> going to be initializing some instance variables.
>
> 7) No line continuation \ is needed inside a multiline """ comment
> """.
>
> With all those changes, you have: http://pastebin.com/m1fdf518e
>
> Good Luck!
>
> --Steve
>
>
> On Mar 1, 8:31 pm, Gloria W wrote:
>
>> Hi all,
>>
>> I copied snippets from existing console code in Console.py to try to
>> make a functional remote console writer, to, say, prompt for a user name
>> and info before letting anyone into a chat session, for example. Code
>> is here:
>>
>> http://pastebin.com/m6f29ad06
>>
>> And it seems to hang when I use it. I'll keep hacking away at it, but
>> please take a glance and let me know if anything is blatantly wrong.
>> My class inherits from component, and is therefore not threaded. It is
>> meant to be a one-off: wait for a string, return it when the client
>> types it.
>>
>> Thank you!
>> Gloria
>>
> >
>
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to kamaelia@googlegroups.com
To unsubscribe from this group, send email to
kamaelia+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~--~~~~--~~--~--~---
[kamaelia-list] Re: Bug in SingleShotHTTPClient
On Monday 02 March 2009 09:49:36 Steve wrote:
> Oh heck, this bug is in the underlying TCPClient! After spending days
> developing against localhost, I now find that I can't go live without
> having to do manual name resolution. :(
I've done some digging. There's definitely something odd going on here.
I got suspicious that there was a bug in TCPClient. Specifically, if you do:
TCPClient("www.google.com", 80)
This results in a call inside TCPClient of:
self.safeConnect(sock, ("www.google.com", 80))
Which results in (fundamentally) :
sock.connect(("www.google.com", 80))
This has worked (correctly) for the past several years, hence the surprise.
However, your comment made me wonder if this should result in on some
platforms:
IP = sock.gethostbyname("www.google.com")
sock.connect((IP, 80))
Since that's what you'd normally do with the C sockets API.
However, re-reading both the docs for the socket module *AND* looking at the
source for Python-2.6.1/Modules/socketmodule.c , I note that the comments in
the socketmodule say:
/* Convert a string specifying a host name or one of a few symbolic
names to a numeric IP address. This usually calls gethostbyname()
to do the work; the names "" and "" are special.
Return the length (IPv4 should be 4 bytes), or negative if
an error occurred; then an exception is raised. */
Specifically this means that socket.connect itself decodes (for example) the
IP address "66.102.7.99" - for example from:
SingleShotHTTPClient('http://66.102.7.99:8000/') => OK
Using the same function, which through one path checks for something looking
like an IP and parses it, and through the other path, ends up calling
gethostbyname (in gettaddrinfo.c) UNLESS you have IPV6 enabled, in
which case it ends up calling getipnodebyname .
As a result I'm rather puzzled as to what's causing your problem here...
Michael.
--
http://yeoldeclue.com/blog
http://twitter.com/kamaelian
http://www.kamaelia.org/Home
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to kamaelia@googlegroups.com
To unsubscribe from this group, send email to
kamaelia+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~--~~~~--~~--~--~---
[kamaelia-list] Re: Bug in SingleShotHTTPClient
On Monday 02 March 2009 09:39:02 Steve wrote:
> SingleShotHTTPClient on windows vista goes nuts opening thousands of
> ports when making a connection to an address which requires name
> resolution and which includes a port number.
1 > SingleShotHTTPClient('http://www.google.com/') => OK
Google is listening on port 80.
2 > SingleShotHTTPClient('http://www.google.com:8000/') => Kaboom
www.google.com is not listening on port 8000
BUT they're filtering it, so rather than their system sending back a TCP
RESET, it just doesn't respond. Behaviour of Google with telnet:
~/> telnet www.google.com 8000
Trying 209.85.229.147...
[hang]
Behaviour with a system that's not listening on port 8000 AND not filtering
(meaning the TCP stack responds with a TCP RESET) :
~/code.google/kamaelia/trunk/Code/Python/Kamaelia/Examples> telnet 192.168.2.1
8001
Trying 192.168.2.1...
telnet: connect to address 192.168.2.1: Connection refused
3 > SingleShotHTTPClient('http://66.102.7.99:8000/') => OK
66.102.7.99 8000 is not listening on port 8000
(checked, it isn't - and like www.google.com it's also filtering rather than
just resetting)
4 > SingleShotHTTPClient('http://localhost:8000/') => OK
No idea :)
2 & 3 should result in the same behaviour on your system, and I'm confused as
to why it doesn't at present. I think I need slightly more detail here to
help out fixing whatever is the issue.
Michael.
--
http://yeoldeclue.com/blog
http://twitter.com/kamaelian
http://www.kamaelia.org/Home
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to kamaelia@googlegroups.com
To unsubscribe from this group, send email to
kamaelia+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~--~~~~--~~--~--~---
[kamaelia-list] Re: Bug in SingleShotHTTPClient
On Monday 02 March 2009 09:49:36 Steve wrote: > Oh heck, this bug is in the underlying TCPClient! After spending days > developing against localhost, I now find that I can't go live without > having to do manual name resolution. That really should not be a problem (ie I've not seen that problem before). Can you give a minimal example using TCPClient that doesn't work for you? What platform are you under ? I've not needed to change TCPClient with regard to basic functionality in several years which is why I'm asking this. I've got a feeling it's a windows vs linux/Mac OS X thing... Michael. -- http://yeoldeclue.com/blog http://twitter.com/kamaelian http://www.kamaelia.org/Home --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "kamaelia" group. To post to this group, send email to kamaelia@googlegroups.com To unsubscribe from this group, send email to kamaelia+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/kamaelia?hl=en -~--~~~~--~~--~--~---
[kamaelia-list] Re: Small mod to console example doesn't work?
> This is true, thanks. Now let's say I wanted to write to the client's
> console before reading anything from their console.
>
> With no other changes, this Pipeline setting doesn't work:
> Pipeline(
> MyComponent(),
> ConsoleReader(),
> ).run()
>
> but I am expecting it to work. What am I missing?
"Pipeline" isn't about saying what order to run things in - its more like
expressing how to plug things together: The declaration above is saying:
""" Make two components - an instance of MyComponent and an instance of
ConsoleReader; now plug them together so that the "outbox" of
"MyComponent" is joined to the "inbox" of "ConsoleReader" """
You could think of this as being like wiring your video recorder
(MyComponent) up to your television set (ConsoleReader). Both are turned
on and run at the same time. It just so happens that one sends stuff to
the other.
In the Kamaelia system, the MyComponent places data into its "outbox"
outbox. Because it is in this pipeline, that data is automatically put
straight into the "inbox" inbox of the ConsoleReader, ready to be
collected.
This is all about the flow of data. The order in which things happen is
defined by the behaviour of each component.
So going back to your original aim:
> This is true, thanks. Now let's say I wanted to write to the client's
> console before reading anything from their console.
So you want a program where the flow of data is as follows:
input from console ---> MyComponent > output to console
I'm guessing there are two possibilities for what you're after achieving.
One is easier than the other :-)
In the simplest case, you want a pipeline like this:
Pipeline(
ConsoleReader(),
MyComponent(),
ConsoleEchoer()
).run()
What is actually being wired up here? These two diagrams from
http://www.kamaelia.org/Cookbook/Pipelines may help. The first shows the
'intention' you express when you write a Pipeline. The second shows the
detail of what inboxes and outboxes actually get wired up.
http://www.kamaelia.org/images/pipeline1_intention.gif
http://www.kamaelia.org/images/pipeline1_inside.gif
Now the main() generator in MyComponent is where we can control the actual
behaviour, so we'll do something like this:
def main(self):
self.send("Initial message for the console", "outbox")
while not self.doShutdown():
if self.dataReady("inbox"):
data = self.recv("inbox")
# let's echo what we received...
self.send(data, 'outbox')
if not self.anyReady():
self.pause()
yield 1
Thats the simple version. Now a little more complexity! ...
The above code won't guarantee is whether the "initial message for the
console" is displayed before the prompt that is displayed by the
ConsoleReader. Why is this? Because the Pipeline declaration effectively
says "make all these and start them all at teh same time".
If you want explicit behaviour where a component isn't created or
activated, or wired up until a later point in the execution (ie. you want
to control the order things happen in) then you need to express it
explicitly.
Here's *a* way to do it:
We can use the Carousel component do delay the creation and wiring up of
the ConsoleReader component until MyComponent sends a message telling it
to do so. There's some explanation of Carousel here:
http://www.kamaelia.org/Cookbook/Carousels if it helps.
In short, Carousel is designed to contain another component. When it
receives a message, it calls a helper function (that you supply) to create
a component. It then takes that component and wires it up so that its
"outbox" and "inbox" are mapped to the "outbox" and "inbox" of the
Carousel component itself. We'll declare this helper globally for now:
def makeConsoleReader(message): # we ignore the parameter
return ConsoleReader()
So that MyComponent can send a message to the Carousel, we'll need to add
an extra outbox to MyComponent, and get it to send that signal:
Class MyComponent(component):
Outboxes = {"outbox" : "some data out",
"signal" : "Shutdown signal",
"ready" : "Sends a message when initialisation
is complete",
}
...
def main(self):
self.send("Initial message for the console", "outbox")
yield 1
self.send("OK", "ready")
while not self.doShutdown():
if self.dataReady("inbox"):
data = self.recv("inbox")
# let's echo what we received...
self.send(data, 'outbox')
if not self.anyReady():
self.pause()
yield 1
Note the 'yield' statements - which give other components a chance to run
- giving the ConsoleEchoer a chance to display that in
[kamaelia-list] Re: Bug in SingleShotHTTPClient
Oh heck, this bug is in the underlying TCPClient! After spending days
developing against localhost, I now find that I can't go live without
having to do manual name resolution. :(
--Steve
On Mar 2, 1:39 am, Steve wrote:
> FYI
>
> SingleShotHTTPClient on windows vista goes nuts opening thousands of
> ports when making a connection to an address which requires name
> resolution and which includes a port number.
>
> SingleShotHTTPClient('http://www.google.com/') => OK
> SingleShotHTTPClient('http://www.google.com:8000/') => Kaboom
> SingleShotHTTPClient('http://66.102.7.99:8000/') => OK
> SingleShotHTTPClient('http://localhost:8000/') => OK
>
> Cheers,
> Steve
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to kamaelia@googlegroups.com
To unsubscribe from this group, send email to
kamaelia+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~--~~~~--~~--~--~---
[kamaelia-list] Bug in SingleShotHTTPClient
FYI
SingleShotHTTPClient on windows vista goes nuts opening thousands of
ports when making a connection to an address which requires name
resolution and which includes a port number.
SingleShotHTTPClient('http://www.google.com/') => OK
SingleShotHTTPClient('http://www.google.com:8000/') => Kaboom
SingleShotHTTPClient('http://66.102.7.99:8000/') => OK
SingleShotHTTPClient('http://localhost:8000/') => OK
Cheers,
Steve
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"kamaelia" group.
To post to this group, send email to kamaelia@googlegroups.com
To unsubscribe from this group, send email to
kamaelia+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/kamaelia?hl=en
-~--~~~~--~~--~--~---
[kamaelia-list] Re: Submitting my own console writer for review.
On Mon, 02 Mar 2009 06:30:09 -, Steve wrote: > 7) No line continuation \ is needed inside a multiline """ comment > """. You are correct - it is not necessary - but it helps with the parsing done for the automatically generated component reference documentation on the website. Iirc, without it you effectively get a blank, empty line at the start of the doc string. The currently employed algorithm for stripping the indentation gets confused by it. Hmm, thats a bit rubbish really - I ought to sort it out :-) Matt -- | Matt Hammond | Research Engineer, FM&T, BBC, Kingswood Warren, Tadworth, Surrey, UK | http://www.bbc.co.uk/rd/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "kamaelia" group. To post to this group, send email to kamaelia@googlegroups.com To unsubscribe from this group, send email to kamaelia+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/kamaelia?hl=en -~--~~~~--~~--~--~---
