Re: [Geany-Devel] Plugins sharing resources

[Dimitar Zhekov]

On Sat, 13 Apr 2013 09:46:53 +1000
Lex Trotman  wrote:

> On 13 April 2013 03:19, Dimitar Zhekov  wrote:
> 
> > Let me confirm. [...]
> > you expect that a plugin will allocate a resource the first time it
> > needs it, and release the resource as soon as it doesn't need it any
> > more (as opposed to, say, reserving a resource based on the file
> > type). Thus it's reasonable to expect that per-sci allocation will
> > considerably the improve resource usage. Am I correct?..
> >
> > For the record: [max usage 16/23 markers, 0/28? indicators]
> 
> Nothing wrong with your math, but the suggestion was started by talking to
> someone who is writing *another* plugin that wants to use markers, and the
> next one etc.

The math was for the record only; I really wanted to know if we expect
the resource management described above. It looks nice, and certainly
matches numberedbookmarks, but is not convinient for scope (and
probably debugger).

> Certainly we can be authoritative and assign a range to the current plugins
> that are in G-P, but thats not all plugins, and how do we manage it?

I suggested that as a fastest resolution only. Resource management
may be required in the future, but the ability to allocate markers
globally will be helpful. Or at least some resource ID manager,
otherwise each plugin has to attach it's own key/data pairs for each
sci, like William did.

-- 
E-gards: Jimmy
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] Plugins sharing resources

[Lex Trotman]

On 13 April 2013 03:19, Dimitar Zhekov  wrote:

> On Thu, 11 Apr 2013 07:49:41 +1000
> Lex Trotman  wrote:
>
> [...]
>
> On Thu, 11 Apr 2013 12:56:22 +0100
> William Fraser  wrote:
>
> [...]
>
> Let me confirm. You haven't checked the total resource usage, because
> you expect that a plugin will allocate a resource the first time it
> needs it, and release the resource as soon as it doesn't need it any
> more (as opposed to, say, reserving a resource based on the file
> type). Thus it's reasonable to expect that per-sci allocation will
> considerably the improve resource usage. Am I correct?..
>
> --
>
> For the record:
>
> marker usage:
>   scintilla:25..31, folding
>   geany highlighting:   0 (goto line), 1 (bookmark)
>   debugger: 12..17
>   numbered bookmarks:   dynamic #s, 10 markers
>   scope:17..19, configurable
>
> maximum marker usage:   25 / 22 (debugger / scope)
> geany/scintilla:9
> available to plugins:   23
> used by plugins:16 / 13 (70% or 55%)
> fully available:7 / 10
> fastest resolution: numberedbookmarks -> 2..11, scope -> 12..14
>
> [debugger and scope will never be used together]
>
> indicators:
>   scintilla:   none (0..2 defined)
>   geany:   0 (GEANY_INDICATOR_ERROR), 8 (GEANY_INDICATOR_SEARCH)
>   spellcheck:  uses GEANY_INDICATOR_ERROR
>
> Either I'm missing something big, or we'll never out of these.
>

Nothing wrong with your math, but the suggestion was started by talking to
someone who is writing *another* plugin that wants to use markers, and the
next one etc.

Certainly we can be authoritative and assign a range to the current plugins
that are in G-P, but thats not all plugins, and how do we manage it?  If
someone comes and says "for new_plugin I need 10 markers", we havn't got
them if bookmarks and debugger are allowed for.  So we would have to say to
users, sorry, you can't run new_plugin and debugger together, or you can't
run new_plugin and bookmarks together, and who is going to decide which
combinations are allowed?  And anyway which user is going to RTFM where we
say what combinations are legal?  So fixed allocations is not the answer.

Cheers
Lex



>
> Other resource types: havent't checked.
>
> --
> E-gards: Jimmy
> ___
> Devel mailing list
> Devel@lists.geany.org
> https://lists.geany.org/cgi-bin/mailman/listinfo/devel
>
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] Plugins sharing resources

[Dimitar Zhekov]

On Thu, 11 Apr 2013 07:49:41 +1000
Lex Trotman  wrote:

[...]

On Thu, 11 Apr 2013 12:56:22 +0100
William Fraser  wrote:

[...]

Let me confirm. You haven't checked the total resource usage, because
you expect that a plugin will allocate a resource the first time it
needs it, and release the resource as soon as it doesn't need it any
more (as opposed to, say, reserving a resource based on the file
type). Thus it's reasonable to expect that per-sci allocation will
considerably the improve resource usage. Am I correct?..

--

For the record:

marker usage:
  scintilla:25..31, folding
  geany highlighting:   0 (goto line), 1 (bookmark)
  debugger: 12..17
  numbered bookmarks:   dynamic #s, 10 markers
  scope:17..19, configurable

maximum marker usage:   25 / 22 (debugger / scope)
geany/scintilla:9
available to plugins:   23
used by plugins:16 / 13 (70% or 55%)
fully available:7 / 10
fastest resolution: numberedbookmarks -> 2..11, scope -> 12..14

[debugger and scope will never be used together]

indicators:
  scintilla:   none (0..2 defined)
  geany:   0 (GEANY_INDICATOR_ERROR), 8 (GEANY_INDICATOR_SEARCH)
  spellcheck:  uses GEANY_INDICATOR_ERROR

Either I'm missing something big, or we'll never out of these.

Other resource types: havent't checked.

-- 
E-gards: Jimmy
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] Plugins sharing resources

[William Fraser]

On 10/04/2013 22:49, Lex Trotman wrote:
> It makes sense when the resource is only available per sci.
>
> There is no "global" set of markers, just that Geany always set up the same
> set for its own use in each sci.
>
> Even if a plugin uses the same markers in each sci, regardless of language
> or other settings, it still has to set them up and release them on each
> sci.  It is more likely though that a plugin will only use markers on a few
> sci.
>
> So the plugin will have to be allocating the resource per sci anyway, and
> it will need to track it, yes.

This is the way I have done it in numberedbookmarks. I use the
g_object_set_data() and g_object_get_data() functions passing
G_OBJECT(sci) as the first argument, thus attaching a pointer to a
structure that can be used to keep track of resource use on a per sci
object basis. Other than having to watch out for memory leaks of the
structures when shutting down, I think that this is best option to track
resource use. What's more, I'm no GTK expert and have not tried it, but
but using g_object_set_data_full() would probably be an easy way to
prevent memory leaks.
>> 2. If they are above the limit, check how much per-sci will improve
>> the things. If it's less than, say, 15%, per-sci makes no sense.
>>
>> 3. If per-sci still seems reasonable, provide some logical to physical
>> mapping, so that the plugins woudn't need to remember each allocated
>> resource separately for each sci.
>>
> Unless they are set and forget uses, the plugin still has to remember them
> to be able to use them.
If a you're going to create a structure holding handles to the resources
used on a per sci object basis to ensure no clashes in resource use
anyway, then it would be a simple matter to write a function returning
the handle of a numbered resource. A plugin would not then have to
remember which ones were allocated, but could simple call the function
with the resource number and sci object, with the knowlage that if null
were returned then the resource was not in use.

However I feel that this slightly goes against the ethos of geany and
geany-plugins being as light-weight as possible by adding another
function to the geany-plugins api that is not essential. A plugin writer
would have insight into if a plugin needed to keep track of resource use
or not (geany-plugins would be keeping track of which resources were in
use or not anyway, so there is no need to replicate this if a plugin
didn't need to remember this information once a resource had been
established), and it would be a relatively simple matter of adding a
simple way of keeping track of which resources a plugin uses. On the
other hand, having such a simple function would keep plugins simpler,
and thus less likely to contain any bugs.

William
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] Plugins sharing resources

[Lex Trotman]

On 11 April 2013 02:25, Dimitar Zhekov  wrote:

> On Fri, 5 Apr 2013 20:20:55 +1100
> Lex Trotman  wrote:
>
> > Matthew has pointed out on IRC that the markers and indicators are per
> > scintilla object, not global.  He has a suggested alternative API which
> he
> > will post after its tested.
>
> On Tue, 9 Apr 2013 23:20:16 +0100
> WILLIAM FRASER  wrote:
>
> > What do people think about these options? Does anyone have any other
> ideas?
>
> This was discussed before. At that time Lex was agains global resource
> mapping and wanted per-sci resource allocation instead. I will only
> repeat my suggestions:
>
> 1. Check how many resources we are using. If they are below the global
> limit or slighly above it, per-sci makes no sense.
>

It makes sense when the resource is only available per sci.

There is no "global" set of markers, just that Geany always set up the same
set for its own use in each sci.

Even if a plugin uses the same markers in each sci, regardless of language
or other settings, it still has to set them up and release them on each
sci.  It is more likely though that a plugin will only use markers on a few
sci.

So the plugin will have to be allocating the resource per sci anyway, and
it will need to track it, yes.


>
> 2. If they are above the limit, check how much per-sci will improve
> the things. If it's less than, say, 15%, per-sci makes no sense.
>
> 3. If per-sci still seems reasonable, provide some logical to physical
> mapping, so that the plugins woudn't need to remember each allocated
> resource separately for each sci.
>

Unless they are set and forget uses, the plugin still has to remember them
to be able to use them.

Cheers
Lex




>
> --
> E-gards: Jimmy
> ___
> Devel mailing list
> Devel@lists.geany.org
> https://lists.geany.org/cgi-bin/mailman/listinfo/devel
>
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] Plugins sharing resources

[Dimitar Zhekov]

On Fri, 5 Apr 2013 20:20:55 +1100
Lex Trotman  wrote:

> Matthew has pointed out on IRC that the markers and indicators are per
> scintilla object, not global.  He has a suggested alternative API which he
> will post after its tested.

On Tue, 9 Apr 2013 23:20:16 +0100
WILLIAM FRASER  wrote:

> What do people think about these options? Does anyone have any other ideas?

This was discussed before. At that time Lex was agains global resource
mapping and wanted per-sci resource allocation instead. I will only
repeat my suggestions:

1. Check how many resources we are using. If they are below the global
limit or slighly above it, per-sci makes no sense.

2. If they are above the limit, check how much per-sci will improve
the things. If it's less than, say, 15%, per-sci makes no sense.

3. If per-sci still seems reasonable, provide some logical to physical
mapping, so that the plugins woudn't need to remember each allocated
resource separately for each sci.

-- 
E-gards: Jimmy
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] Plugins sharing resources

[Matthew Brush]

On 13-04-09 03:20 PM, WILLIAM FRASER wrote:

On 06/04/2013, Lex Trotman  wrote:

On 6 April 2013 00:02, William Fraser  wrote:


I attached an int to each scintilla object to keep track of what's been
assigned in my markers plugin. The only problem was that some markers are
used by scintilla, so any plugin would have to make note of which ones
were
being used by the editor (with the problem that you would have to update
plugins if editor changed) or, you would have to submit code to scintilla
so that it also used the uniform resource allocation.



Hi William,

Which plugin is that?  And how do you know that say the bookmarks plugin
isn't using that marker already?  Thats the sort of problem this is to
solve.

Cheers
Lex


Hello Lex,

It is the GeanyNumberedBookmarks plugin that has to manage sharing
markers with other plugins and the editor. I had to re-write it as
some of the markers were clashing with another plugin (I forget which
one).

The functions that handle keeping track of markers are
GetMarkersUsed(), NextFreeMarker(), SetMarker(), and DeleteMarker().

To get around the problem of different scintilla editor instances,
each of which could have more than one document open in them, I used
the g_object_set_data() and g_object_get_data() functions to attach a
pointer to an allocated piece of memory holding data on which markers
I'm using per Scintilla editor object (I see if there is one allocated
already and if not add one, thus ensuring only one per editor
instance). As there are 32 markers I used a 32 bit int to hold which
markers were being used.

Unfortunately it's a little more complex as other pluggins can also
use the markers, as can the editor (some are documented as being in
use:0 &1 for bookmarks & errors, 25 onwards for folds). So the
NextFreeMarker() function in GeanyNumberedBookmarks thus only checks
between 2 and 24 for free markers. I have assumed that anyone using a
marker will likely set it to something other than the default of
SC_MARK_CIRCLE, or at least followed documented good practice of
resetting it to SC_MARK_AVAILABLE if they no longer need it. This is
how I find unused markers.

However there are potential problems with this due to the assumptions.
Problems that could arise include: Markers being used by another
plugin (or the editor if it's use of markers changes) after
GeanyNumberedBookmarks has started using them without checking to see
if they are already in use, and not setting the marker to something
other than SC_MARK_CIRCLE.

To get round these problem I can see only 3 options:
1) Each plugin manually setting their own markers as we do at the
moment but with better co-ordination to avoid clashes, or to enable
the user to set them manually in the pluggin settings. This is sort of
what we have at the moment and is messy, and makes it difficult to
maintain plugins, or develop new ones without clashing with existing
plugins. It also has the downside of needing to check all pluggins if
scintilla changes which markers it uses.

2) Submit a patch to Scintilla that adds functions similar to
GetMarkersUsed(), NextFreeMarker(), SetMarker(), and DeleteMarker()
that the editor, and any plugins would use. The downside of this is
that they might not accept the patch, and that it would then mean that
plugins using markers would only work with the version of Scintilla
where the patch was accepted or later versions. It would be the
neatest, and most robust solution.

3) Have plugin functions similar to GetMarkersUsed(),
NextFreeMarker(), SetMarker(), and DeleteMarker() that any plugin has
to call to use markers as part of the geany-plugins standard function
calls. We would have to monitor Scintilla for changes to markers but
would then mean that our plugins would play nicely with each other,
and we could update/modify these functions independent of Scintilla.
Also we would only have to changed the geany-plugins standard function
if Scintilla's use of markers changed rather than having to re-write
all the plugins. We also would not have to wait for any changes to
Scintilla to propogate through the versions. This would probably be
the fastest way of doing things.

There is no reason why we could not use a combined approach: submit a
patch to Scintilla and in the meantime have our own functions until
the changes to Scintilla were adopted.

Any plugins wide, or editor marker management functions would have to
be thread safe to avoid clashes between plugins or the editor.



I don't think they'd need to be thread-safe, nothing else in Geany or 
the plugin API is.



What do people think about these options? Does anyone have any other ideas?



Here's the ideas I was working on:
https://gist.github.com/codebrainz/a72b0c18669649ee7939

I still haven't tested it too much but I think the idea is pretty sound 
and flexible. It's quite like your #3 but a bit more generic and 
includes also a signal plugins can connect to in order to notify them 
when resources become available.


If you don'

Re: [Geany-Devel] Plugins sharing resources

[WILLIAM FRASER]

On 06/04/2013, Lex Trotman  wrote:
> On 6 April 2013 00:02, William Fraser  wrote:
>
>> I attached an int to each scintilla object to keep track of what's been
>> assigned in my markers plugin. The only problem was that some markers are
>> used by scintilla, so any plugin would have to make note of which ones
>> were
>> being used by the editor (with the problem that you would have to update
>> plugins if editor changed) or, you would have to submit code to scintilla
>> so that it also used the uniform resource allocation.
>>
>
> Hi William,
>
> Which plugin is that?  And how do you know that say the bookmarks plugin
> isn't using that marker already?  Thats the sort of problem this is to
> solve.
>
> Cheers
> Lex

Hello Lex,

It is the GeanyNumberedBookmarks plugin that has to manage sharing
markers with other plugins and the editor. I had to re-write it as
some of the markers were clashing with another plugin (I forget which
one).

The functions that handle keeping track of markers are
GetMarkersUsed(), NextFreeMarker(), SetMarker(), and DeleteMarker().

To get around the problem of different scintilla editor instances,
each of which could have more than one document open in them, I used
the g_object_set_data() and g_object_get_data() functions to attach a
pointer to an allocated piece of memory holding data on which markers
I'm using per Scintilla editor object (I see if there is one allocated
already and if not add one, thus ensuring only one per editor
instance). As there are 32 markers I used a 32 bit int to hold which
markers were being used.

Unfortunately it's a little more complex as other pluggins can also
use the markers, as can the editor (some are documented as being in
use:0 &1 for bookmarks & errors, 25 onwards for folds). So the
NextFreeMarker() function in GeanyNumberedBookmarks thus only checks
between 2 and 24 for free markers. I have assumed that anyone using a
marker will likely set it to something other than the default of
SC_MARK_CIRCLE, or at least followed documented good practice of
resetting it to SC_MARK_AVAILABLE if they no longer need it. This is
how I find unused markers.

However there are potential problems with this due to the assumptions.
Problems that could arise include: Markers being used by another
plugin (or the editor if it's use of markers changes) after
GeanyNumberedBookmarks has started using them without checking to see
if they are already in use, and not setting the marker to something
other than SC_MARK_CIRCLE.

To get round these problem I can see only 3 options:
1) Each plugin manually setting their own markers as we do at the
moment but with better co-ordination to avoid clashes, or to enable
the user to set them manually in the pluggin settings. This is sort of
what we have at the moment and is messy, and makes it difficult to
maintain plugins, or develop new ones without clashing with existing
plugins. It also has the downside of needing to check all pluggins if
scintilla changes which markers it uses.

2) Submit a patch to Scintilla that adds functions similar to
GetMarkersUsed(), NextFreeMarker(), SetMarker(), and DeleteMarker()
that the editor, and any plugins would use. The downside of this is
that they might not accept the patch, and that it would then mean that
plugins using markers would only work with the version of Scintilla
where the patch was accepted or later versions. It would be the
neatest, and most robust solution.

3) Have plugin functions similar to GetMarkersUsed(),
NextFreeMarker(), SetMarker(), and DeleteMarker() that any plugin has
to call to use markers as part of the geany-plugins standard function
calls. We would have to monitor Scintilla for changes to markers but
would then mean that our plugins would play nicely with each other,
and we could update/modify these functions independent of Scintilla.
Also we would only have to changed the geany-plugins standard function
if Scintilla's use of markers changed rather than having to re-write
all the plugins. We also would not have to wait for any changes to
Scintilla to propogate through the versions. This would probably be
the fastest way of doing things.

There is no reason why we could not use a combined approach: submit a
patch to Scintilla and in the meantime have our own functions until
the changes to Scintilla were adopted.

Any plugins wide, or editor marker management functions would have to
be thread safe to avoid clashes between plugins or the editor.

What do people think about these options? Does anyone have any other ideas?

William

P.S. sorry for delay in getting back, and for not doing much with my
pluggins of late, but life is beyond busy at the moment & for the
foreseeable future.
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] Plugins sharing resources

[Lex Trotman]

On 6 April 2013 00:02, William Fraser  wrote:

> I attached an int to each scintilla object to keep track of what's been
> assigned in my markers plugin. The only problem was that some markers are
> used by scintilla, so any plugin would have to make note of which ones were
> being used by the editor (with the problem that you would have to update
> plugins if editor changed) or, you would have to submit code to scintilla
> so that it also used the uniform resource allocation.
>

Hi William,

Which plugin is that?  And how do you know that say the bookmarks plugin
isn't using that marker already?  Thats the sort of problem this is to
solve.

Cheers
Lex



>
>
>  Original Message 
>
> Lex Trotman  wrote:
>
> Update:
>
> Matthew has pointed out on IRC that the markers and indicators are per
> scintilla object, not global.  He has a suggested alternative API which he
> will post after its tested.
>
> Cheers
> Lex
>
>
> On 5 April 2013 13:09, Lex Trotman  wrote:
>
>> There are several plugins that share limited resources, such as scintilla
>> markers and indicators.
>>
>> At the moment there is no coordination for using those resources so
>> plugins can interfere with one another and possibly with Geany.
>>
>> The only common thing between plugins and Geany is Geany, so this is a
>> proposal to add a resources management interface to Geany.
>>
>> The proposed interface is below, I have tried to make it so it is easy to
>> expand to resources that are not integers (markers and indicators are all
>> integers)
>>
>> enum resource_type { GEANY_RESOURCE_MARKER, GEANY_RESOURCE_INDICATOR };
>> gboolean alloc_int_resource( enum resource_type, int* resource_num );
>> void free_int_resource( int resource_num );
>>
>> This allows extra resources to be added without ABI changes (so long as
>> they are added to the end of the enum).
>>
>> If (when) other types of resource need managing then extra functions can
>> be added again without breaking the ABI.
>>
>> Automatic release is not suggested because that prevents use of RAII and
>> similar management schemes in C++ or GC for other plugin languages.  For
>> plain C plugins just call free_int_resource in the plugin_cleanup function.
>>
>> Any simpler, more flexible suggestions?
>>
>> Cheers
>> Lex
>>
>
>
> ___
> Devel mailing list
> Devel@lists.geany.org
> https://lists.geany.org/cgi-bin/mailman/listinfo/devel
>
>
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] Plugins sharing resources

[William Fraser]

I attached an int to each scintilla object to keep track of what's been assigned in my markers plugin. The only problem was that some markers are used by scintilla, so any plugin would have to make note of which ones were being used by the editor (with the problem that you would have to update plugins if editor changed) or, you would have to submit code to scintilla so that it also used the uniform resource allocation. Original Message Lex Trotman  wrote:
Update:Matthew has pointed out on IRC that the markers and indicators are per scintilla object, not global.  He has a suggested alternative API which he will post after its tested.
CheersLexOn 5 April 2013 13:09, Lex Trotman  wrote:
There are several plugins that share limited resources, such as scintilla markers and indicators.
At the moment there is no coordination for using those resources so plugins can interfere with one another and possibly with Geany.
The only common thing between plugins and Geany is Geany, so this is a proposal to add a resources management interface to Geany.The proposed interface is below, I have tried to make it so it is easy to expand to resources that are not integers (markers and indicators are all integers)

enum resource_type { GEANY_RESOURCE_MARKER, GEANY_RESOURCE_INDICATOR };gboolean alloc_int_resource( enum resource_type, int* resource_num );void free_int_resource( int resource_num );

This allows extra resources to be added without ABI changes (so long as they are added to the end of the enum).If (when) other types of resource need managing then extra functions can be added again without breaking the ABI.

Automatic release is not suggested because that prevents use of RAII and similar management schemes in C++ or GC for other plugin languages.  For plain C plugins just call free_int_resource in the plugin_cleanup function.

Any simpler, more flexible suggestions?CheersLex


___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] Plugins sharing resources

[Lex Trotman]

Update:

Matthew has pointed out on IRC that the markers and indicators are per
scintilla object, not global.  He has a suggested alternative API which he
will post after its tested.

Cheers
Lex


On 5 April 2013 13:09, Lex Trotman  wrote:

> There are several plugins that share limited resources, such as scintilla
> markers and indicators.
>
> At the moment there is no coordination for using those resources so
> plugins can interfere with one another and possibly with Geany.
>
> The only common thing between plugins and Geany is Geany, so this is a
> proposal to add a resources management interface to Geany.
>
> The proposed interface is below, I have tried to make it so it is easy to
> expand to resources that are not integers (markers and indicators are all
> integers)
>
> enum resource_type { GEANY_RESOURCE_MARKER, GEANY_RESOURCE_INDICATOR };
> gboolean alloc_int_resource( enum resource_type, int* resource_num );
> void free_int_resource( int resource_num );
>
> This allows extra resources to be added without ABI changes (so long as
> they are added to the end of the enum).
>
> If (when) other types of resource need managing then extra functions can
> be added again without breaking the ABI.
>
> Automatic release is not suggested because that prevents use of RAII and
> similar management schemes in C++ or GC for other plugin languages.  For
> plain C plugins just call free_int_resource in the plugin_cleanup function.
>
> Any simpler, more flexible suggestions?
>
> Cheers
> Lex
>
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel