Re: Fwd: Python Signal/Slot + QThred code analysis
On 11/28/2014 06:06 AM, Juan Christian wrote: > Which one would be better in performance, having a single 'Worker' to call > all URLs, having inside this worker functions for each stuff, or having 3~5 > different workers doing different stuff. In the end I'll only print the > data when I have them all. > > The problem is that I would have a long new_value = Signal(QThread, str, > str), with more then 10 values, is it possible to pass objects here > like new_value = Signal(QThread, ObjA, ObjB, ...) so that in the > 'create_topic' I would get the objects and call methods in order to get all > the data? Yes you can pass objects through signals. You could pass a dict, or an instance of one of your own classes. You could also go back to your first architecture and pass in an Outpost class instance to the worker thread when instantiating it, and then the worker could use the outpost instance to perform the work, then pass it back to your handler through the signal. > This approach would be better, because I may be adding more stuff and I > wouldn't need to add tons and tons of new stuff in my signal, only one > object. > > You don't need to redo the code, I just need to know if it's a 'good > approach' and if it's possible, from here I can work my way. Yeah sounds alright. I'm sure there are multiple ways of doing this that would work. -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On Fri Nov 28 2014 at 2:07:32 AM Michael Torrie wrote: Okay, here's a reworking of the code that invokes a new QThread instance each time. Note the QThread instance has to be bound to the MainWindow so that it won't be destroyed when it goes out of scope. Also the Worker thread sends a signal with the data, so there's no need to check worker.trades and risk it being invalid data. Perhaps this would be more what you had in mind. You may want to look into asynchronous I/O as well. That does not require threads at all. Fire off a request to load a url, and then get a callback when it's done. Now it's working like a charm without any freezes. I'll reproduce this 'Worker' for other stuffs that I need to call the web, but first I have a question. Which one would be better in performance, having a single 'Worker' to call all URLs, having inside this worker functions for each stuff, or having 3~5 different workers doing different stuff. In the end I'll only print the data when I have them all. The problem is that I would have a long new_value = Signal(QThread, str, str), with more then 10 values, is it possible to pass objects here like new_value = Signal(QThread, ObjA, ObjB, ...) so that in the 'create_topic' I would get the objects and call methods in order to get all the data? This approach would be better, because I may be adding more stuff and I wouldn't need to add tons and tons of new stuff in my signal, only one object. You don't need to redo the code, I just need to know if it's a 'good approach' and if it's possible, from here I can work my way. -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On 11/27/2014 04:29 PM, Juan Christian wrote:
> Is that right? But I don't think that always calling the site this way is
> good for them and for me, sometimes I may not get new users for like 10~15
> minutes, so I would have wasted a lot of calls for nothing. That's why I
> only instantiate/call the site when I get new users.
Okay, here's a reworking of the code that invokes a new QThread instance
each time. Note the QThread instance has to be bound to the MainWindow
so that it won't be destroyed when it goes out of scope. Also the
Worker thread sends a signal with the data, so there's no need to check
worker.trades and risk it being invalid data.
Perhaps this would be more what you had in mind.
You may want to look into asynchronous I/O as well. That does not
require threads at all. Fire off a request to load a url, and then get
a callback when it's done.
Anyway, here's the code (too lazy to post it to pastebin):
import time
import random
from PySide.QtCore import QObject, QThread, Signal, QFile, QTimer, Qt
from PySide.QtGui import QGroupBox, QGridLayout, QTextEdit, QApplication
from PySide.QtUiTools import QUiLoader
class Worker(QThread):
# This is the signal that we'll emit every time
# we load a new set of values from the url
new_value = Signal(QThread, str,str)
def __init__(self, url, *args, **kwargs):
super(Worker, self).__init__(*args, **kwargs)
self.url = url
self.start()
def run(self):
# Doesn't matter how long this takes because it only
# blocks the thread's event loop, not the GUI
print ('Starting work...')
time.sleep(random.random() * 2)
trades = str(random.random() * 25)
posts = str(random.random() * 100)
print ('Finished work.')
# communicate the values via the signal
# we have to pass self so that the callback
# can properly clean up after the thread.
self.new_value.emit(self, trades, posts)
def loadui(file_name):
loader = QUiLoader()
uifile = QFile(file_name)
uifile.open(QFile.ReadOnly)
ui = loader.load(uifile)
uifile.close()
return ui
if __name__ == "__main__":
import sys
# Putting this in here because it requires access to MainWindow
# which only exists if this part of the code runs.
def create_topic(worker, trades, posts):
box = QGroupBox()
grid = QGridLayout()
nickname = QTextEdit()
box.setFixedHeight(200)
nickname.setText("%s : %s" % (trades, posts))
grid.addWidget(nickname)
box.setLayout(grid)
# Access global variable MainWindow, defined below
MainWindow.vlay.addWidget(box)
# clean up the worker to prevent resource leak.
worker.quit()
worker.wait()
def output_slot():
# in order for the thread to stay running, we have to bind it to
the
# Main Window, which we
worker = Worker('http://www.tf2outpost.com/user /' + id64,
MainWindow)
worker.new_value.connect(create_topic)
app = QApplication(sys.argv)
MainWindow = loadui("main.ui")
MainWindow.vlay.setAlignment(Qt.AlignTop)
timer = QTimer()
timer.start(5000)
timer.timeout.connect(output_slot)
id64 = '123'
MainWindow.show()
app.exec_()
--
https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On 11/27/2014 04:29 PM, Juan Christian wrote: > Is that right? But I don't think that always calling the site this way is > good for them and for me, sometimes I may not get new users for like 10~15 > minutes, so I would have wasted a lot of calls for nothing. That's why I > only instantiate/call the site when I get new users. So the real problem with instantiating a worker thread for every request is that you have to hold onto a reference to the worker long enough for the work to be done. Your original code was not doing this, so had it worked, the thread would have been aborted before it even started its work, as soon as the worker instance went out of scope. -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On 11/27/2014 04:29 PM, Juan Christian wrote:
> So, instantly I found one issue, you said that this code won't block the
> GUI, only the thread event loop, but if we keep moving the window while
> it's open, every time new info is printed the GUI freezes for like 1-2
> seconds.
Correct. The thread does not block the GUI. I cannot replicate your
problem on my machine here. Maybe we're finding a problem with Windows
and Qt. I can drag the window around and it keeps on a going.
> And why this approach of a single instance is better? I mean, I'll have to
> call Outpost every time I get new data from my other API, because there
> will be different users, so I need to create different instances so I call
> the site with a new ID, don't I?
Just seems wasteful, that's all. Why not instantiate Outpost from the
worker thread? Especially if your thread only does one thing every 5
seconds. Starting up a new thread and destroying seems like a waste.
But it certainly can be done that way.
> Maybe the answer for that question is that you using a timer that is ALWAYS
> active and ALWAYS calling the the outpost site, so I'll have something like:
>
> var = Outpost('12345')
> var.posts -> 100
> var.setID('67893')
> var.posts -> 35
>
> Is that right? But I don't think that always calling the site this way is
> good for them and for me, sometimes I may not get new users for like 10~15
> minutes, so I would have wasted a lot of calls for nothing. That's why I
> only instantiate/call the site when I get new users.
I was simply using the logic you originally provided. I have no idea
what Outpost does or how you use it. Your original code just grabbed
some data every 5 seconds, so that's what I got it to do in the most
efficient way. You'll have to adapt it to your real needs.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On Thu Nov 27 2014 at 9:16:38 PM Juan Christian
wrote:
I'll read the code thoroughly and reply to you if I find something strange,
many thanks!
So, instantly I found one issue, you said that this code won't block the
GUI, only the thread event loop, but if we keep moving the window while
it's open, every time new info is printed the GUI freezes for like 1-2
seconds.
And why this approach of a single instance is better? I mean, I'll have to
call Outpost every time I get new data from my other API, because there
will be different users, so I need to create different instances so I call
the site with a new ID, don't I?
Maybe the answer for that question is that you using a timer that is ALWAYS
active and ALWAYS calling the the outpost site, so I'll have something like:
var = Outpost('12345')
var.posts -> 100
var.setID('67893')
var.posts -> 35
Is that right? But I don't think that always calling the site this way is
good for them and for me, sometimes I may not get new users for like 10~15
minutes, so I would have wasted a lot of calls for nothing. That's why I
only instantiate/call the site when I get new users.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On Thu Nov 27 2014 at 8:53:16 PM Michael Torrie wrote: Hope this helps. Here's complete working code, minus the .ui file: http://pastebin.com/VhmSFX2t Thanks, I'll just repost the code on pastebin with a NEVER expire time and UNLISTED, so that it can be of some help for others here in the mailist: http://pastebin.com/9Jrmw4VL I'll read the code thoroughly and reply to you if I find something strange, many thanks! -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On 11/27/2014 04:58 AM, Juan Christian wrote:
> What I have in mind is simple (I think), have the Outpost get the data
> using QThread, and when it got everything needed emit a signal. So when I
> have something like 'var = Outpost('123')', when I get the signal I'll know
> that I can call 'var.trades, var.posts' and it won't give me NullErrors or
> something like that.
So there are a number of logic problems in your code. The main one is
that the call to create your worker class returns immediately. It's not
part of the thread itself. So your signal logic will never work.
This was a good excuse for me to dive into Qt a bit more. So it turns
out that QThread objects can have their own event loops. This turns out
to be quite ideal for what you want. So here's a Worker class does
everything you want, but with only one instance, and uses signals properly:
class Worker(QThread):
# This is the signal that we'll emit every time
# we load a new set of values from the url
new_value = Signal(str,str)
def __init__(self, url, *args, **kwargs):
super(Worker, self).__init__(*args, **kwargs)
self.url = url
self.start()
def run(self):
# Since we're going to run a mini event loop in the thread
# we can create and use a timer here
self.timer = QTimer()
self.timer.start(5000)
self.timer.timeout.connect(self.fetch_url)
# Start the thread's event loop
self.exec_()
def fetch_url(self):
# Doesn't matter how long this takes because it only
# blocks the thread's event loop, not the GUI
time.sleep(random.random() * 2)
trades = str(random.random() * 25)
posts = str(random.random() * 100)
# communicate the values via the signal
self.new_value.emit(trades, posts)
So with this, you can instantiate it once, then tie the new_value signal
to a callback in the main loop. So no need for the Outlook object, and
no intermediate callbacks. The thread signals directly. And the
callback receives the data directly, so no check for NODATA values and
such:
def create_topic(trades, posts):
box = QGroupBox()
grid = QGridLayout()
nickname = QTextEdit()
box.setFixedHeight(200)
nickname.setText("%s : %s" % (trades, posts))
grid.addWidget(nickname)
box.setLayout(grid)
# Access global variable MainWindow, defined below
MainWindow.vlay.addWidget(box)
Hope this helps. Here's complete working code, minus the .ui file:
http://pastebin.com/VhmSFX2t
--
https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On Thu Nov 27 2014 at 2:12:40 AM Michael Torrie wrote:
Hmm, I hit a wall. There's no main.ui file. Can you rework your code
so that you have a single file (plus a separate ui file that's okay),
that simulates the url request, that I can execute easily.
As you asked, here it's, everything on one module and without network
calls, http://pastebin.com/ibS9FSSd
The main.ui is a XML, http://pastebin.com/Eq9hNMBX , just save it as
main.ui and it will work. I think now you have everything in order to test
the app.
What I have in mind is simple (I think), have the Outpost get the data
using QThread, and when it got everything needed emit a signal. So when I
have something like 'var = Outpost('123')', when I get the signal I'll know
that I can call 'var.trades, var.posts' and it won't give me NullErrors or
something like that.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On 11/26/2014 08:57 PM, Michael Torrie wrote: > On 11/26/2014 02:55 PM, Juan Christian wrote: >> On Wed Nov 26 2014 at 1:16:11 AM Michael Torrie wrote: >> You're going to have to post a complete, but small, code example, I >> think. Working with fragments of code is very difficult if not >> impossible to assist with, resulting in obtuse, obvious replies from folks. >> >> As asked, here is all the code: >> >> outpost module: http://pastebin.com/H3K9UUWi >> main module: http://pastebin.com/dFzums9W >> >> I was trying to do other things there but I ended up screwing everything. > > I'm sorry I can't run the code you posted. I don't seem to have any > modules named "request" on Python 3 or Python 2 and your Trader.py seems > to require it. > > You might want to remove the url downloading and Beautiful Soup stuff > and put in some kind of nop loop instead to simulate it. For example, > in the Trader module you can remove everything in run() and just have it > return dummy values, maybe putting in a delay to simulate the work of > downloading and parsing. > > Just an observation from looking at the main module code. > You only need to attach a callback to a signal once. output_slot() > seems to attach the callback each and every time the timer fires. This > is surely wrong. > > I'll give it some further study this evening. Hmm, I hit a wall. There's no main.ui file. Can you rework your code so that you have a single file (plus a separate ui file that's okay), that simulates the url request, that I can execute easily. Here's how you can simulate the web page load: from PySide.QtCore import QObject, QThread, Signal import random import time class Worker(QThread): def __init__(self, url, *args, **kwargs): super().__init__(*args, **kwargs) self.trades = 'NODATA' self.posts = 'NODATA' self.url = url self.start() def run(self): # simulate a web request by sleeping a random time and returning # random strings. time.sleep(random.random() * 2) self.trades = str(random.random() * 25) self.posts = str(random.random() * 100) return self -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On 11/26/2014 02:55 PM, Juan Christian wrote: > On Wed Nov 26 2014 at 1:16:11 AM Michael Torrie wrote: > You're going to have to post a complete, but small, code example, I > think. Working with fragments of code is very difficult if not > impossible to assist with, resulting in obtuse, obvious replies from folks. > > As asked, here is all the code: > > outpost module: http://pastebin.com/H3K9UUWi > main module: http://pastebin.com/dFzums9W > > I was trying to do other things there but I ended up screwing everything. I'm sorry I can't run the code you posted. I don't seem to have any modules named "request" on Python 3 or Python 2 and your Trader.py seems to require it. You might want to remove the url downloading and Beautiful Soup stuff and put in some kind of nop loop instead to simulate it. For example, in the Trader module you can remove everything in run() and just have it return dummy values, maybe putting in a delay to simulate the work of downloading and parsing. Just an observation from looking at the main module code. You only need to attach a callback to a signal once. output_slot() seems to attach the callback each and every time the timer fires. This is surely wrong. I'll give it some further study this evening. -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On Wed Nov 26 2014 at 1:16:11 AM Michael Torrie wrote: You're going to have to post a complete, but small, code example, I think. Working with fragments of code is very difficult if not impossible to assist with, resulting in obtuse, obvious replies from folks. As asked, here is all the code: outpost module: http://pastebin.com/H3K9UUWi main module: http://pastebin.com/dFzums9W I was trying to do other things there but I ended up screwing everything. -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On 11/25/2014 02:36 PM, Juan Christian wrote:
> So guys, I had to change to approach, I read that using Qt I can't do
> multiple inheritance. So my Outpost class can't be like 'Outpost(QObject,
> QThred)'. I had to change the code a bit:
>
> So, let's see the problems:
>
> Traceback (most recent call last):
> File "D:/.../main.py", line 44, in output_slot
> outpost = Outpost('12345')
> File "D:\...\outpost.py", line 27, in __init__
> self.received.emit(worker)
> TypeError: received() only accepts 0 arguments, 2 given!
You're going to have to post a complete, but small, code example, I
think. Working with fragments of code is very difficult if not
impossible to assist with, resulting in obtuse, obvious replies from folks.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
So guys, I had to change to approach, I read that using Qt I can't do
multiple inheritance. So my Outpost class can't be like 'Outpost(QObject,
QThred)'. I had to change the code a bit:
from PySide.QtCore import QObject, QThread, Signal
import requests
import bs4
class Worker(QThread):
def __init__(self, url, *args, **kwargs):
super().__init__(*args, **kwargs)
self.trades = None
self.posts = None
self.url = url
self.start()
def run(self):
soup = bs4.BeautifulSoup(requests.get(self.url).text)
self.trades = soup.select('div.stat_box
div.value')[0].get_text().replace(',', '')
self.posts = soup.select('div.stat_box div.value')[1].get_text()
return self
class Outpost(QObject):
received = Signal()
def __init__(self, id64, *args, **kwargs):
super().__init__(*args, **kwargs)
worker = Worker('http://www.myurl.com/' + id64)
self.received.emit(worker)
So, what I see people doing is creating a 'Worker' class for thread related
stuff and another separate class for the main stuff.
So, let's see the problems:
Traceback (most recent call last):
File "D:/.../main.py", line 44, in output_slot
outpost = Outpost('12345')
File "D:\...\outpost.py", line 27, in __init__
self.received.emit(worker)
TypeError: received() only accepts 0 arguments, 2 given!
The URL is OK, I externally checked, and return the expected.
So, what I'm trying to do here is call my Worker when outpost is
instantiated, then the Worker call the site in the run function, and then
set the variables and return itself.
Back to the main class, I have the init where I instantiate the Worker and
return it in the emit().
If I got it correctly, when I call the 'outpost.received.connect(IPSUM)',
IPSUM will be execute when, and only when I have all values set, because in
my Outpost class it will emit when it has everything done, did I get it
right?
OFF-TOPIC: You guys said that my emails had some trash HTML and strange
stuffs, is it OK now?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Python Signal/Slot + QThred code analysis
On 2014-11-25 15:48, Juan Christian wrote:
On Tue Nov 25 2014 at 1:42:24 PM MRAB mailto:pyt...@mrabarnett.plus.com>> wrote:
I think that the problem there is that strings don't have an __exit__
method.
I don't understand what you said, what you mean by that? =/
The traceback says:
Traceback (most recent call last):
File "D:\Documents\PyCharm\Trader\Trader\core\outpost.py", line 18,
in run
with soup.select('div.stat_box
div.value')[0].get_text().replace(',', '') as trades, \
AttributeError: __exit__
As you can see, it does:
soup.select('div.stat_box div.value')[0].get_text()
which, presumably, returns a string, and then it calls the .replace
method on that string, which returns a string.
That string is then bound to 'trades'.
The problem is that this occurs in the expression part of a 'with'
statement.
If you look at this simpler example:
>>> with 'some string' as a_name:
... pass
...
Traceback (most recent call last):
File "", line 1, in
AttributeError: __exit__
An even simpler example:
>>> with 'somestring':
... pass
...
Traceback (most recent call last):
File "", line 1, in
AttributeError: __exit__
--
https://mail.python.org/mailman/listinfo/python-list
