Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-10-05 Thread René J . V . Bertin
Konrad Rosenbaum wrote:


>> And one cannot even rely on mutexes to "fix" or prevent that?
> 
> If you use Mutexes right you can rely on them to serialize the code that
> blocks on the same Mutex, i.e. only one blocking code block is executed at
> the same time. 
> Depending on hardware timings and bus congestion these
> threads may race each other.

Hmm, ok. Nothing new here in fact, but thanks :)

>> Sounds like the situation that ObjC/Apple addressed by using refcounting
> 
> If by "problem" you mean programmer stupidity (or if you prefer: naturally
> limited cognitive capacity): yes. ;-)

Heh. Let's go for the latter, with capacity determined over all brains working 
on a given project. 


> If by "acceptable" you mean "fast enough for a human being", then yes -
> QueuedConnection will do this for you reliably enough.

Careful, "for the task at hand" always implies "for a human being" (at least 
the 
one who defined the task) but that doesn't mean a human could resolve the 
difference between just enough and just not enough (directly, that is) ;)

> Do not make any assumptions about the order of signals or slots. Try to
> avoid the assumption that a signal only returns after the slot has been
> executed (or that it returns immediately)

That one is easy - I've never seen any user-written code that corresponds to 
the 
signal function, so I've always thought of it as raising a system signal.
R.

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-10-05 Thread Konrad Rosenbaum
On Mon, October 2, 2017 11:54, René J. V. Bertin wrote:
> Konrad Rosenbaum wrote:
>> Whenever a QueuedConnection triggers the sending object generates an
>> event
>> and delivers it to the event queue of the thread that owns the target
>> slot's
>
> So the sending object knows that the nature of each connection to it?

The code behind the signal knows about it and how to deliver itself.

>>> Can I assume
>>> there's some kind of fifo that ensures signals are delivered in order
>>> of
>>
>> Never assume order.
>
> Ok. From your other reactions I infer there is something like a
> mostly-fifo with
> a mutex that protects pushing and popping :)

More or less: yes. The input is thread safe and FIFO (or as FIFO as you
can get in a multi-threaded environment), the output has priorities and
filters.

>> deterministic or in order. Threads execute in a (seemingly) random order
>> and
>> can interrupt or race each other.
>
> And one cannot even rely on mutexes to "fix" or prevent that?

If you use Mutexes right you can rely on them to serialize the code that
blocks on the same Mutex, i.e. only one blocking code block is executed at
the same time. You cannot predict which thread gets the Mutex first or
last. Since Mutexes are a special case of semaphores, the same is true for
semaphores  (again: if you use them right).

The OS can interrupt your thread at any time and give the CPU to another
thread or process that is ready to run (a thread waiting for a blocked
Mutex is not ready). It may also interrupt your thread and run a signal
handler in the same thread! On the other hand if your machine has multiple
CPU cores the OS may let several threads run happily at the same time on
different cores. Depending on hardware timings and bus congestion these
threads may race each other.

>> important or less important than others - e.g. the deleteLater event is
>> normally only handled at the top level of the event loop to prevent
>> crashes
>> caused by less then absolutely ingenious programmers using
>> QueuedConnection,
>> deleteLater and QEventLoop (that fact saved my bacon a few times).
>
> Sounds like the situation that ObjC/Apple addressed by using refcounting
> instead
> of direct deletion.

If by "problem" you mean programmer stupidity (or if you prefer: naturally
limited cognitive capacity): yes. ;-)

> Anyway, order isn't crucial in the particular application I'm working on
> here.
> The only thing that matters is that signals are delivered in an acceptable
> time-
> frame after having been sent. And that should be the case (my initial
> reason for
> getting involved with this was to keep the main event loop from being
> blocked).

If by "acceptable" you mean "fast enough for a human being", then yes -
QueuedConnection will do this for you reliably enough.

With Qt there are some simple rules:

If there is the slightest chance that a QObject is used/called across
threads - use deleteLater instead of delete.

For calls across threads use QueuedConnection explicitly. (In most cases
Qt detects this automatically, but there are some special cases usually
involving moveToThread().)

Do not feed pointers or references into signal-slot-connections. Avoid
sharing pointers/references between threads.

Do not make any assumptions about the order of signals or slots. Try to
avoid the assumption that a signal only returns after the slot has been
executed (or that it returns immediately) - you'll mess with connection
settings soon enough after you forgot that you rely on it. Regard signals
as fire-and-forget-but-maybe-take-some-time-with-unknown-effects.

The combination of these will ensure that your application behaves in an
acceptable manner.

QueuedConnection and deleteLater use the same event queue, so Qt is able
to detect if an object disappears while delivering a signal.

If you absolutely, positively have to share pointers between threads:
a) try to refactor, if that fails:
b) make 100% positively sure that the object lives longer than the
sharing; use QPointer (makes debugging easier); write twice as many lines
of comments as you write code so you cannot forget.

If you have to block on signals: try to use multiple signals instead of a
hard block; document!



   Konrad

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-10-02 Thread René J . V . Bertin
On Monday October 02 2017 11:12:35 Sergio Martins wrote:

Hi,

> You were probably capturing a local variable by reference inside the 
> lambda, which then went out of scope.

That sounds likely (I used '[&]' as the other lambdas do in the same method. I 
don't think I considered scope when trying to understand the crash (stupid...).

> See 
> https://github.com/KDE/clazy/blob/master/src/checks/level0/README-lambda-in-connect.md
>  
> for how to catch it at compile time.

Thanks, I'll keep that in mind for future reference.

R.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-10-02 Thread Sergio Martins

On 2017-09-30 13:54, René J. V.  Bertin wrote:

Apart from this I'd suspect you will still get the SEGV if you do not 
block

- even if the frequency changes. This points to you using some kind of
pointer that is not properly controlled (e.g. sending a signal to a 
QFSW

that is already deleted).


I think it was false alarm. I used a lambda expression as slot and must 
have
misunderstood why the other lambdas in the same function can use '[&]' 
without

ending up with invalid references in the body. I only need the watcher
instance,
so I'm just passing the reference to it explicitly. No more crashing 
and less

ambiguous anyway :)


You were probably capturing a local variable by reference inside the 
lambda, which then went out of scope.


See 
https://github.com/KDE/clazy/blob/master/src/checks/level0/README-lambda-in-connect.md 
for how to catch it at compile time.



Regards,
--
Sérgio Martins | sergio.mart...@kdab.com | Senior Software Engineer
Klarälvdalens Datakonsult AB, a KDAB Group company
Tel: Sweden (HQ) +46-563-540090, USA +1-866-777-KDAB(5322)
KDAB - The Qt, C++ and OpenGL Experts
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-10-02 Thread René J . V . Bertin
Konrad Rosenbaum wrote:

Hi,

I see I'm not the only one accepting the fact that when dealing with event-
driven GUIs having a human-in-the-loop one rather quickly reaches a stage where 
it becomes hard to guarantee anything (with all that implies for writing code 
that handles unforeseen situations *gracefully*).

> crashes. When dealing with threads and signals between them determinism
> flies out the window and the tiniest change in timing can change behavior
> radically.

No kidding (I'm used to working on/with older hardware or simply stressing it 
and seeing all kinds of things no one else can apparently reproduce.)

As I said, the crashes I saw were not to do inappropriate signal/slot 
programming but to me not understanding something in "wildcard" capturing for 
lambdas. IOW, the crash frequency has been 0 ever since I fixed that (the 
capture, not my understanding :))

> Whenever a QueuedConnection triggers the sending object generates an event
> and delivers it to the event queue of the thread that owns the target slot's

So the sending object knows that the nature of each connection to it?

>> Can I assume
>> there's some kind of fifo that ensures signals are delivered in order of
> 
> Never assume order.

Ok. From your other reactions I infer there is something like a mostly-fifo 
with 
a mutex that protects pushing and popping :)

> deterministic or in order. Threads execute in a (seemingly) random order and
> can interrupt or race each other.

And one cannot even rely on mutexes to "fix" or prevent that?

> important or less important than others - e.g. the deleteLater event is
> normally only handled at the top level of the event loop to prevent crashes
> caused by less then absolutely ingenious programmers using QueuedConnection,
> deleteLater and QEventLoop (that fact saved my bacon a few times).

Sounds like the situation that ObjC/Apple addressed by using refcounting 
instead 
of direct deletion.

Anyway, order isn't crucial in the particular application I'm working on here. 
The only thing that matters is that signals are delivered in an acceptable time-
frame after having been sent. And that should be the case (my initial reason 
for 
getting involved with this was to keep the main event loop from being blocked).

R.


___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-10-01 Thread Thiago Macieira
On Saturday, 30 September 2017 02:31:13 PDT René J. V. Bertin wrote:
> > You cannot, because neither the kqueue, inotify or poll backends do that.
> > Those three have no mutexes at all. You have to block the event loop of
> > the
> > thread they were created on.
> 
> That all applies to multithreaded usage, right? What happens when a change
> is signalled while an item is being added *on the same thread*? At least
> the kqueue and inotify backends are written around an API that sends
> asynchronous notifications, no?

There's no such thing that you're describing. Like Konrad said, inotify and 
kquee notify by way of file descriptors and standard Unix event looping. Since 
everything is happening in the same thread, then a change can't be signalled 
*while* an item is being added. Things only happen serially.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-30 Thread Konrad Rosenbaum
Hi,

On Saturday 30 September 2017 16:24:57 René J. V. Bertin wrote:
> Konrad Rosenbaum wrote:
> > Apart from this I'd suspect you will still get the SEGV if you do not
> > block - even if the frequency changes.
> 
> As in when emitting the signal too frequently from multiple threads?

No. as in how many times out of the total signals delivered it actually 
crashes. When dealing with threads and signals between them determinism 
flies out the window and the tiniest change in timing can change behavior 
radically.

> For my personal education, what happens behind the scenes when a signal is
> sent from one thread to a slot in a different thread?

Whenever a QueuedConnection triggers the sending object generates an event 
and delivers it to the event queue of the thread that owns the target slot's 
object. Whenever that thread's event loop is called again it will start to 
work on those events

> Can I assume
> there's some kind of fifo that ensures signals are delivered in order of

Never assume order.

> being sent and such that producer and consumer don't try to modify
> (access) the queue concurrently?

Kind of.

The receiving end of the event queue is of course programmed in a thread 
safe way. And new events are usually queued towards the end of that queue.

BUT.

There are complications: for one, and this cannot be stressed enough, if you 
are dealing with threads you cannot assume anything to be completely 
deterministic or in order. Threads execute in a (seemingly) random order and 
can interrupt or race each other. For another: some events are more 
important or less important than others - e.g. the deleteLater event is 
normally only handled at the top level of the event loop to prevent crashes 
caused by less then absolutely ingenious programmers using QueuedConnection, 
deleteLater and QEventLoop (that fact saved my bacon a few times).

So, the order of events is not 100% guaranteed. Most of the time events will 
be received and handled in the order they were sent, but there are 
exceptions.


Konrad

signature.asc
Description: This is a digitally signed message part.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-30 Thread Adam Treat
Simon just gave a talk about signals and slots on different threads and 
the internals of how Qt handles this at CppCon in Seattle yesterday. I 
would suggest you have a look at the video of his presentation when it 
comes out. You could also look at the documentation: 
http://doc.qt.io/qt-5/threads-qobject.html with particular attention to 
"Signals and Slots Across Threads"


On 09/30/2017 10:24 AM, René J. V. Bertin wrote:

Konrad Rosenbaum wrote:


Apart from this I'd suspect you will still get the SEGV if you do not block
- even if the frequency changes.

As in when emitting the signal too frequently from multiple threads?

For my personal education, what happens behind the scenes when a signal is sent
from one thread to a slot in a different thread? Can I assume there's some kind
of fifo that ensures signals are delivered in order of being sent and such that
producer and consumer don't try to modify (access) the queue concurrently?

R.

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-30 Thread René J . V . Bertin
Konrad Rosenbaum wrote:

> Apart from this I'd suspect you will still get the SEGV if you do not block
> - even if the frequency changes.

As in when emitting the signal too frequently from multiple threads?

For my personal education, what happens behind the scenes when a signal is sent 
from one thread to a slot in a different thread? Can I assume there's some kind 
of fifo that ensures signals are delivered in order of being sent and such that 
producer and consumer don't try to modify (access) the queue concurrently?

R.

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-30 Thread René J . V . Bertin
Konrad Rosenbaum wrote:

> You do know what a thread is - right? ;-)

Yeah, my momma taught me long ago ;)

> Sorry, I'm too lazy/busy to read the source right now to make sure that no
> signal handlers are used here.

No problem, it was just curiosity.

> Apart from this I'd suspect you will still get the SEGV if you do not block
> - even if the frequency changes. This points to you using some kind of
> pointer that is not properly controlled (e.g. sending a signal to a QFSW
> that is already deleted).

I think it was false alarm. I used a lambda expression as slot and must have 
misunderstood why the other lambdas in the same function can use '[&]' without 
ending up with invalid references in the body. I only need the watcher 
instance, 
so I'm just passing the reference to it explicitly. No more crashing and less 
ambiguous anyway :)

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-30 Thread Konrad Rosenbaum
On Saturday 30 September 2017 11:31:13 René J. V. Bertin wrote:
> Thiago Macieira wrote:
[about internal mutex]
> > You cannot, because neither the kqueue, inotify or poll backends do
> > that.
> > Those three have no mutexes at all. You have to block the event loop of
> > the thread they were created on.
> 
> That all applies to multithreaded usage, right? What happens when a change
> is signalled while an item is being added *on the same thread*? At least
> the kqueue and inotify backends are written around an API that sends
> asynchronous notifications, no?

You do know what a thread is - right? ;-)

A quick google search tells me that inotify and kqueue work by signaling 
file events through the normal mechanism of the event loop. This means the 
change is appended to the event queue and handled the next time this thread 
is idle or explicitly calls the event loop. 

It is impossible for the change event to interrupt the add method or vice 
versa. The only mechanism that can do this in the same thread is a signal 
handler.

Sorry, I'm too lazy/busy to read the source right now to make sure that no 
signal handlers are used here.

> I'm now investigating the idea of sending a signal from the dirlister
> thread, connected to a slot in the main thread with Qt::QueuedConnection
> . After some initial success I'm now getting a SEGV that I don't yet
> understand. I'm beginning to wonder if one shouldn't use a blocking
> QueuedConnection...

BlockingQueuedConnections should only be used if you really need the result, 
there is no asynchronous way of transmitting the result and you can make 
sure that there will not be a deadlock.

Apart from this I'd suspect you will still get the SEGV if you do not block 
- even if the frequency changes. This points to you using some kind of 
pointer that is not properly controlled (e.g. sending a signal to a QFSW 
that is already deleted).


Konrad

signature.asc
Description: This is a digitally signed message part.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-30 Thread René J . V . Bertin
Thiago Macieira wrote:

>> >> How does QFSW handle a situation in which an already watched directory is
>> >> changed while another directory is being added? SNIP
>> > 
>> > The Darwin implementation has an internal mutex so that the dispatcher
>> > thread (I'm assuming) is able to get to the current state.
>> 
>> Can I assume that the other platforms have a similar feature that blocks
>> access to the current state while entries are being added or removed (and
>> also defers adding/removing entries while sending out change
>> notifications)?
> 
> You cannot, because neither the kqueue, inotify or poll backends do that.
> Those three have no mutexes at all. You have to block the event loop of the
> thread they were created on.

That all applies to multithreaded usage, right? What happens when a change is 
signalled while an item is being added *on the same thread*? At least the 
kqueue 
and inotify backends are written around an API that sends asynchronous 
notifications, no?

I'm now investigating the idea of sending a signal from the dirlister thread, 
connected to a slot in the main thread with Qt::QueuedConnection . After some 
initial success I'm now getting a SEGV that I don't yet understand. I'm 
beginning to wonder if one shouldn't use a blocking QueuedConnection...

R.

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-29 Thread Thiago Macieira
On sexta-feira, 29 de setembro de 2017 14:47:29 PDT René J. V. Bertin wrote:
> Thiago Macieira wrote:
> >> How does QFSW handle a situation in which an already watched directory is
> >> changed while another directory is being added? Would something else
> >> happen
> >> when adding directories happens on a background thread while change
> >> signals
> >> are connected to slots on the main thread?
> > 
> > The Darwin implementation has an internal mutex so that the dispatcher
> > thread (I'm assuming) is able to get to the current state.
> 
> Can I assume that the other platforms have a similar feature that blocks
> access to the current state while entries are being added or removed (and
> also defers adding/removing entries while sending out change
> notifications)?

You cannot, because neither the kqueue, inotify or poll backends do that. 
Those three have no mutexes at all. You have to block the event loop of the 
thread they were created on.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-29 Thread René J . V . Bertin
Thiago Macieira wrote:

>> How does QFSW handle a situation in which an already watched directory is
>> changed while another directory is being added? Would something else happen
>> when adding directories happens on a background thread while change signals
>> are connected to slots on the main thread?
> 
> The Darwin implementation has an internal mutex so that the dispatcher thread
> (I'm assuming) is able to get to the current state.

Can I assume that the other platforms have a similar feature that blocks access 
to the current state while entries are being added or removed (and also defers 
adding/removing entries while sending out change notifications)?

R

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-29 Thread Thiago Macieira
On sexta-feira, 29 de setembro de 2017 12:50:07 PDT René J. V. Bertin wrote:
> Thiago Macieira wrote:
> > We're not claiming that you have no problem. We're saying the problem is
> > probably in your own code.
> 
> It's a kind of chicken-or-egg question. I'm indeed doing something that
> triggers a problem which is clearly deep inside Qt code. What I want to
> know is whether it's acceptable/foreseeable that that problem gets
> triggered or if it shouldn't be triggered.
> 
> For now I'm going to take away the conclusion that the mutex indeed has to
> be in my code.

Even then we won't support you. QFSW, like almost all QObjects, must only be 
touched by one thread. Anything else you do is at your own risk.

> How does QFSW handle a situation in which an already watched directory is
> changed while another directory is being added? Would something else happen
> when adding directories happens on a background thread while change signals
> are connected to slots on the main thread?

The Darwin implementation has an internal mutex so that the dispatcher thread 
(I'm assuming) is able to get to the current state.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-29 Thread René J . V . Bertin
Thiago Macieira wrote:


> We're not claiming that you have no problem. We're saying the problem is
> probably in your own code.

It's a kind of chicken-or-egg question. I'm indeed doing something that 
triggers 
a problem which is clearly deep inside Qt code. What I want to know is whether 
it's acceptable/foreseeable that that problem gets triggered or if it shouldn't 
be triggered.

For now I'm going to take away the conclusion that the mutex indeed has to be 
in 
my code.

> In the Qt documentation (and most other sane docu) "reentrant" means the
> class can be used on different threads, but each instance must be
> restricted to a single thread.

So I didn't entirely mix up reentrant and thread-safe after all.
I may end up trying to write a demonstrator after all to see if I can reproduce 
the lock-ups on Mac with a minimal example.

>  you ensured that the instance was only used by one instance at a time.

Probably even more than that: only a single QFSW instance used at any given 
moment.

How does QFSW handle a situation in which an already watched directory is 
changed while another directory is being added? Would something else happen 
when 
adding directories happens on a background thread while change signals are 
connected to slots on the main thread?

> Another way is to explicitly use Qt::QueuedConnection to feed the QFSW 

A signal from the background dir reader thread connected to a slot on the main 
thread? That could be an interesting alternative to investigate indeed.

R.

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-29 Thread Thiago Macieira
On sexta-feira, 29 de setembro de 2017 05:54:53 PDT René J.V. Bertin wrote:
> On Friday September 29 2017 14:27:37 Milian Wolff wrote:
> > behavior you are supposedly seeing.
> 
> So you still haven't bothered trying to reproduce this or prove me wrong,
> aside implying I'm making this all up? In that case I'm not sure why *you*
> come trolling on a post that intends to find out if QFSW is being used in a
> way it's not supposed to be used - in which case there is no point in
> wasting time with upstream demonstrators and patches.

We're not claiming that you have no problem. We're saying the problem is 
probably in your own code.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-29 Thread Thiago Macieira
On sexta-feira, 29 de setembro de 2017 02:49:32 PDT Konrad Rosenbaum wrote:
> This is why you were able to solve the problem with a Mutex: you ensured
> that the instance was only used by one instance at a time.

That is not a full solution. You may be able to serialise all *your* access 
with a mutex, but since QFSW is doing some form of I/O, it may still be woken 
up by another thread and do work there. You need to block that thread's event 
loop (and you need to know which thread that is yourself) if you want to 
ensure full serialisation.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-29 Thread Milian Wolff
On Freitag, 29. September 2017 11:35:40 CEST René J.V. Bertin wrote:
> Hi,
> 
> I've been running into issues using adding and removing entries from QFSW in
> concurrent threads. This is in KDevelop while adding all directories of
> multiple projects (source trees) to a single QFSW instance per project. The
> app is somewhat complex so I think but am not entirely certain that the
> source trees are read on a single background thread; if I'm correct each
> QFSW instance is fed from a single thread only. (The QFSW instances are
> created and "slotted" on the main thread.)
> 
> The issues range from more or less severe lock-ups on Mac (the hard lock-ups
> occur in the FSEvents backend) to double-free crashes on Linux. I can only
> avoid them by using a single mutex that ensures that only 1 thread can use
> the class at a time.
> 
> The QFSW documentation only mentions the class is reentrant. Is QFSW
> supposed to be thread-safe (at least at the class level), on all platforms?
> Or do we have to provide our own protection mechanisms if we want to use
> the class this way?

Please Rene, before you waste more time by getting yet more people involved. 
Build a MWE, i.e. a trivial single-file Qt application that exhibits the 
behavior you are supposedly seeing. Then when you think you are sure what is 
going on we can talk. Don't put up open hypotheses on random channels - it's 
really draining everyones energy for no good gain. And don't come again with 
the "I don't have time" - it's less time to create a MWE than to write dozens 
of mails that then have no outcome.

-- 
Milian Wolff | milian.wo...@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH KG, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt Experts


___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-29 Thread René J . V . Bertin
On Friday September 29 2017 14:27:37 Milian Wolff wrote:

> behavior you are supposedly seeing.

So you still haven't bothered trying to reproduce this or prove me wrong, aside 
implying I'm making this all up? In that case I'm not sure why *you* come 
trolling on a post that intends to find out if QFSW is being used in a way it's 
not supposed to be used - in which case there is no point in wasting time with 
upstream demonstrators and patches.


___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Should QFileSystemWatcher be thread-safe? (Qt 5.8.0)

2017-09-29 Thread Konrad Rosenbaum
Hi,

On Fri, September 29, 2017 11:35, René J.V. Bertin wrote:
> I've been running into issues using adding and removing entries from QFSW
> in concurrent threads.
[cut]
> The QFSW documentation only mentions the class is reentrant. Is QFSW
> supposed to be thread-safe (at least at the class level), on all
> platforms? Or do we have to provide our own protection mechanisms if we
> want to use the class this way?

In the Qt documentation (and most other sane docu) "reentrant" means the
class can be used on different threads, but each instance must be
restricted to a single thread. You can have different instances on
different threads.

Actually documenting it as "thread safe" would imply that even instances
can be shared across threads. This is much harder to implement for (I
believe) obvious reasons.

This is why you were able to solve the problem with a Mutex: you ensured
that the instance was only used by one instance at a time.

Another way is to explicitly use Qt::QueuedConnection to feed the QFSW or
to untangle your code to ensure each QSFW is only visible from one thread.


   Konrad

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development