Re: [Bf-committers] Some Ideas for a Blender Plugin System (hewi)

[hewi jupama]

 Reply

>> What's wrong with something depending on string.h?

It's GHOST, the engine of blender, the contact between OpenGL and extension 
wrappers and all, if I am correct.  
It means the same as your car engine will not run unless you have plugged in 
the right hand side back seat passengers seat belt.  mmmh ... not really


>> A black box that never fails, no matter how strange the code is that
>> links into it, I think that can't exist. 

Never fails is a utopia, but it should be restricted to the minimum (failing) 
and when it does, it should be up and running in the blink of an eye.

Story of Hamilton's steering wheel software (SWS).  of the 100's of races and 
test, his steering wheel software broke down once, indeed, once.  He pushed the 
reset button, in the blink of an eye the car was controllable again (black box 
with core functionality).  He drove on.  While all plugins were loaded, the 
engineers found the speed control in pit lane plugin was erroneous.  He did not 
push that button again, at least not during this race.  I think he even won the 
race ...

>> Of course we all aim at rock solid software, but I don't see how
>> third-party plugins will help with that.

That's why I tried to convince you with the tank example, apparently I failed

> An other nice example of a good plug-in would be a browser.
> Wouldn't that be cool to be able to browse inside Blender :)

>> I desperately don't want that.

Well then don't plug it in :)  If I do not want a passenger seat in my race 
car, than I don't put one in either


>>  I don't share the vision that a plugin system is the only approach to this, 
>> or that
>> will make things easier to develop & fix.

Which approach would you suggest?


>>Recently at blender conference there was a presentation showing custom
>> cloth solver using numpy:
>> https://www.blender.org/conference/2015/presentations/178

way too complicated,  I still want to start with the array modifier plugin.

As many of you suggested and ventilated many times before, we have enough 
people who do the blah blah, but not enough really doing the boem boem.  For a 
while I have looked and tried to find a way into the coding of Blender, but I 
wanted to start at the core, that is hard.  This is a good opportunity for me 
to do some Boom-Boom ;)

So, on that bomb-shell, I am looking to touch base with Mr Martin Felke.  
Unfortunatelly, I am not allowed any more to look at the subscribers list of 
this mail thread.  So I cannot mail him directly.  Any help with getting this 
'project' started would be most welcome.

Kindest regards

Hewi
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System (hewi)

[Sybren A . Stüvel]

Hey Hewi

On Tue, Nov 10, 2015 at 07:34:20AM +, hewi jupama wrote:
> In an ideal world, you should (re)write Blender's core and dived all
> the rest into plugins, unfortunately that is not possible.  Way too
> many circular dependencies, even way down to the core (GHOST
> depending on , what the F**).

What's wrong with something depending on string.h?

> eventually you are left with a solid black box, which will never
> fail

A black box that never fails, no matter how strange the code is that
links into it, I think that can't exist. It would need so many checks
on that linked-in code, that I'm fairly sure the problem can be
reduced to the halting problem [1].

Of course we all aim at rock solid software, but I don't see how
third-party plugins will help with that.

> An other nice example of a good plug-in would be a browser.
> Wouldn't that be cool to be able to browse inside Blender :)

I desperately don't want that.


Don't get me wrong, I really like that we're discussing possible forms
of plugin structures, as it provides us with different views on the
subject. I fully agree that simplifying dependencies and interfaces
between Blender's components is a good thing. However, I don't share
the vision that a plugin system is the only approach to this, or that
will make things easier to develop & fix.


[1] https://en.wikipedia.org/wiki/Halting_problem

-- 
Sybren A. Stüvel

http://stuvelfoto.nl/
http://stuvel.eu/


signature.asc
Description: Digital signature
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System (hewi)

[Cezary Kopias]

I imagine that some kind of plug in system could be used in node based work
flow allowing for efficient custom physics solvers.

Would this be a good use case?

Recently at blender conference there was a presentation showing custom
cloth solver using numpy:
https://www.blender.org/conference/2015/presentations/178

... so maybe in my example case this is a better way to go?

Cezary Kopias - user :)

On 10 November 2015 at 11:24, Sybren A. Stüvel  wrote:

> Hey Hewi
>
> On Tue, Nov 10, 2015 at 07:34:20AM +, hewi jupama wrote:
> > In an ideal world, you should (re)write Blender's core and dived all
> > the rest into plugins, unfortunately that is not possible.  Way too
> > many circular dependencies, even way down to the core (GHOST
> > depending on , what the F**).
>
> What's wrong with something depending on string.h?
>
> > eventually you are left with a solid black box, which will never
> > fail
>
> A black box that never fails, no matter how strange the code is that
> links into it, I think that can't exist. It would need so many checks
> on that linked-in code, that I'm fairly sure the problem can be
> reduced to the halting problem [1].
>
> Of course we all aim at rock solid software, but I don't see how
> third-party plugins will help with that.
>
> > An other nice example of a good plug-in would be a browser.
> > Wouldn't that be cool to be able to browse inside Blender :)
>
> I desperately don't want that.
>
>
> Don't get me wrong, I really like that we're discussing possible forms
> of plugin structures, as it provides us with different views on the
> subject. I fully agree that simplifying dependencies and interfaces
> between Blender's components is a good thing. However, I don't share
> the vision that a plugin system is the only approach to this, or that
> will make things easier to develop & fix.
>
>
> [1] https://en.wikipedia.org/wiki/Halting_problem
>
> --
> Sybren A. Stüvel
>
> http://stuvelfoto.nl/
> http://stuvel.eu/
>
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>
>
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System (hewi)

[hewi jupama]

tions andd do static tests of all internal 
functions and stuff
 3 the runtime tests, to check if the lama still has furr and eyes after 
rendering ;)


--

Message: 3
Date: Thu, 5 Nov 2015 15:16:07 +0100
From: Fran?ois T. <francoistarl...@gmail.com>
Subject: Re: [Bf-committers] Some Ideas for a Blender Plugin System
To: bf-blender developers <bf-committers@blender.org>
Message-ID:
<cabe+ddbk135phpgs165unfmu4gc8oldhr5ud6rchbxabqs7...@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

- Cython does not have as much performance as pure C/C++
- Python is still the best choice for prototyping and pipeline stuff
- C/C++ API can be used for better performance without having the nightmare
to dive into Blender core code to just add one modifier

Most of the modifier we are developping in Maya are first done in Python,
but then it is port to C++ for performance matter. Same approach would be
nice in Blender.

Of course we can modify Blender code and recompile it but :
- this is low level coding compare to the use of an API
- can make future merge more complicated for a studio

And in some cases like sharing code for importer/exporter in alembic, fbx
would be more easier if it was done as a plugin rather than being embeded
in our blender build

2015-11-04 14:44 GMT+01:00 Diego Gangl <dnico...@gmail.com>:

> Hi all,
>
> What would be the benefit of doing this, instead of expanding the python
> api?
>
> The only case I can think of is doing really complex computations or
> working with large sets of data, and you already have three options for
> that:
>
> - Use numpy (for some things)
> - Use Cython
> - Start a process from python and communicate with pipes
>
> A plugin/module system might sound great on paper, but I think it would be
> a nightmare of corner cases.
>
>
>
>
>
>
>
>
>
>
>
>
> 2015-11-04 4:58 GMT-03:00 Sybren A. St?vel <syb...@stuvel.eu>:
>
> > On Tue, Nov 03, 2015 at 08:04:03PM +0100, matmenu wrote:
> > > 2) Do you have a proof for this claim? I mean, modifier/operator
> > > combinations already lead to bugs, I don't see why having them as
> > > plugins/modules would make those problems arise more often?
> >
> > I don't understand your reasoning. To me it reads like "we get bugs
> > now, so why would we get more when we complicate the software?" In the
> > current situation there is a fixed set of functionality in C++
> > (ignoring precompiler directives). In the case of a plugin structure,
> > not only do we have the same set of functionality, but also situations
> > where that functionality is only partially loaded. More different
> > possible situations -> more complex -> more bugs.
> >
> > > 3) That's true, but then the user decide to install this module, so
> > > he will report the bug to the original dev
> >
> > This is only true if the bug is trivial to to identify by end-users.
> > Plenty of bugs won't be. Think about double-free bugs, where the first
> > free is performed in the plugin. A crash may happen when Blender
> > itself frees the pointer, and thus any traceback (if that's shown at
> > all) will indicate a problem in Blender, not the plugin.
> >
> > > 4) Also true, but we now have a buildbot that can make builds on
> > > demand for all supported OS. It could be used further by
> > > module/plugin devs and avoid having to ship a compiler with Blender.
> >
> > AFAIK FreeBSD is not built by this system. Furthermore, allowing
> > arbitrary third party code to be run on a BI/BF-hosted machine is an
> > invitation for trouble. This already happens, but through the Git
> > repository, so at least it's only done by people that have been vetted
> > by receiving commit access.
> >
> > --
> > Sybren A. St?vel
> >
> > http://stuvelfoto.nl/
> > http://stuvel.eu/
> >
> > ___
> > Bf-committers mailing list
> > Bf-committers@blender.org
> > http://lists.blender.org/mailman/listinfo/bf-committers
> >
> >
>
>
> --
> --
> Diego Gangl - sinestesia.co
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>



--

Fran?ois Tarlier
www.francois-tarlier.com
www.linkedin.com/in/francoistarlier


--

___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[Martijn Berger]

Hi all,

If we want to go for speed.

On Thu, Nov 5, 2015 at 3:16 PM, François T. 
wrote:

> - Cython does not have as much performance as pure C/C++
>

While this is true even making the RNA api accessible through cython would
be a step up compared to current situation


> - Python is still the best choice for prototyping and pipeline stuff
> - C/C++ API can be used for better performance without having the nightmare
> to dive into Blender core code to just add one modifier
>

Main problem I see here is that keeping such an API stable is a non trivial
job

>
> Most of the modifier we are developping in Maya are first done in Python,
> but then it is port to C++ for performance matter. Same approach would be
> nice in Blender.
>


>
> And in some cases like sharing code for importer/exporter in alembic, fbx
> would be more easier if it was done as a plugin rather than being embeded
> in our blender build
>

As we discussed on the blender conference I think your Alembic work might
actually be useful by attempting to port it to our existing RNA C++-API and
see what work needs to be done on that API in that way.
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[François T .]

- Cython does not have as much performance as pure C/C++
- Python is still the best choice for prototyping and pipeline stuff
- C/C++ API can be used for better performance without having the nightmare
to dive into Blender core code to just add one modifier

Most of the modifier we are developping in Maya are first done in Python,
but then it is port to C++ for performance matter. Same approach would be
nice in Blender.

Of course we can modify Blender code and recompile it but :
- this is low level coding compare to the use of an API
- can make future merge more complicated for a studio

And in some cases like sharing code for importer/exporter in alembic, fbx
would be more easier if it was done as a plugin rather than being embeded
in our blender build

2015-11-04 14:44 GMT+01:00 Diego Gangl :

> Hi all,
>
> What would be the benefit of doing this, instead of expanding the python
> api?
>
> The only case I can think of is doing really complex computations or
> working with large sets of data, and you already have three options for
> that:
>
> - Use numpy (for some things)
> - Use Cython
> - Start a process from python and communicate with pipes
>
> A plugin/module system might sound great on paper, but I think it would be
> a nightmare of corner cases.
>
>
>
>
>
>
>
>
>
>
>
>
> 2015-11-04 4:58 GMT-03:00 Sybren A. Stüvel :
>
> > On Tue, Nov 03, 2015 at 08:04:03PM +0100, matmenu wrote:
> > > 2) Do you have a proof for this claim? I mean, modifier/operator
> > > combinations already lead to bugs, I don't see why having them as
> > > plugins/modules would make those problems arise more often?
> >
> > I don't understand your reasoning. To me it reads like "we get bugs
> > now, so why would we get more when we complicate the software?" In the
> > current situation there is a fixed set of functionality in C++
> > (ignoring precompiler directives). In the case of a plugin structure,
> > not only do we have the same set of functionality, but also situations
> > where that functionality is only partially loaded. More different
> > possible situations -> more complex -> more bugs.
> >
> > > 3) That's true, but then the user decide to install this module, so
> > > he will report the bug to the original dev
> >
> > This is only true if the bug is trivial to to identify by end-users.
> > Plenty of bugs won't be. Think about double-free bugs, where the first
> > free is performed in the plugin. A crash may happen when Blender
> > itself frees the pointer, and thus any traceback (if that's shown at
> > all) will indicate a problem in Blender, not the plugin.
> >
> > > 4) Also true, but we now have a buildbot that can make builds on
> > > demand for all supported OS. It could be used further by
> > > module/plugin devs and avoid having to ship a compiler with Blender.
> >
> > AFAIK FreeBSD is not built by this system. Furthermore, allowing
> > arbitrary third party code to be run on a BI/BF-hosted machine is an
> > invitation for trouble. This already happens, but through the Git
> > repository, so at least it's only done by people that have been vetted
> > by receiving commit access.
> >
> > --
> > Sybren A. Stüvel
> >
> > http://stuvelfoto.nl/
> > http://stuvel.eu/
> >
> > ___
> > Bf-committers mailing list
> > Bf-committers@blender.org
> > http://lists.blender.org/mailman/listinfo/bf-committers
> >
> >
>
>
> --
> --
> Diego Gangl - sinestesia.co
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>



-- 

François Tarlier
www.francois-tarlier.com
www.linkedin.com/in/francoistarlier
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[Diego Gangl]

Hi all,

What would be the benefit of doing this, instead of expanding the python
api?

The only case I can think of is doing really complex computations or
working with large sets of data, and you already have three options for
that:

- Use numpy (for some things)
- Use Cython
- Start a process from python and communicate with pipes

A plugin/module system might sound great on paper, but I think it would be
a nightmare of corner cases.












2015-11-04 4:58 GMT-03:00 Sybren A. Stüvel :

> On Tue, Nov 03, 2015 at 08:04:03PM +0100, matmenu wrote:
> > 2) Do you have a proof for this claim? I mean, modifier/operator
> > combinations already lead to bugs, I don't see why having them as
> > plugins/modules would make those problems arise more often?
>
> I don't understand your reasoning. To me it reads like "we get bugs
> now, so why would we get more when we complicate the software?" In the
> current situation there is a fixed set of functionality in C++
> (ignoring precompiler directives). In the case of a plugin structure,
> not only do we have the same set of functionality, but also situations
> where that functionality is only partially loaded. More different
> possible situations -> more complex -> more bugs.
>
> > 3) That's true, but then the user decide to install this module, so
> > he will report the bug to the original dev
>
> This is only true if the bug is trivial to to identify by end-users.
> Plenty of bugs won't be. Think about double-free bugs, where the first
> free is performed in the plugin. A crash may happen when Blender
> itself frees the pointer, and thus any traceback (if that's shown at
> all) will indicate a problem in Blender, not the plugin.
>
> > 4) Also true, but we now have a buildbot that can make builds on
> > demand for all supported OS. It could be used further by
> > module/plugin devs and avoid having to ship a compiler with Blender.
>
> AFAIK FreeBSD is not built by this system. Furthermore, allowing
> arbitrary third party code to be run on a BI/BF-hosted machine is an
> invitation for trouble. This already happens, but through the Git
> repository, so at least it's only done by people that have been vetted
> by receiving commit access.
>
> --
> Sybren A. Stüvel
>
> http://stuvelfoto.nl/
> http://stuvel.eu/
>
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>
>


-- 
--
Diego Gangl - sinestesia.co
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[Dalai Felinto]

Hi,

I'm missing the main point here which would be, what is the advantage
of a C/C++ plugin system over the current Python addon interface?

I'm currently developing an addon which relies on an external C++ SDK.
I got things working with ctypes and C API. It works pretty fine.

Cheers,
Dalai
--
blendernetwork.org/dalai-felinto
www.dalaifelinto.com


2015-11-03 10:59 GMT-02:00 Martin Felke :
> Hi, today i wrote down a couple of more thoughts regarding a plugin
> system. As said earlier, those serve only as discussion starting points
> and are not meant yet as design principles made out of cement. :)
>
> Fat VS Slim Core
> 
> - currently we have a fat core which contains all functionality
>   as monolith
> - hard to extend and to maintain because it might be everything depends
>   on everything else
> - idea: a slim, "bootloader like" core, which handles modules and
>   plugins registration and deregistration and serves as API Hub (of
>   existing modules)
> - modules are not standalone, they communicate via a central core API
>   Hub so they need the core as well
> - this way the core can track module APIs and provide a fallback (as
>   in exception handler) for missing code / functions (if module or
>   plugin is not present)
>
> Modules VS Plugins
> --
> - need to distinguish between logical code separation (module) and
>   "physical" separation (plugin, as in shared library or dll)
> - group existing core functionality to modules (like, an editor could
>   be a module, or even a modifier)
> - base modules can be provided by "base" plugins, and extension modules
>   by external plugins
> - plugins can provide parts of modules, entire modules or multiple
>   modules (like nodes maybe, modifiers, new editors, or extensions to
>   editors)
>
> Dependencies
> 
> - do we want to have inter-plugin dependencies ? For a slim core
>   approach this might be necessary, if plugin A provides a module or
>   part where plugin B 's code depends on
> - else base modules should be moved / kept in the core, so everyone has
>   a minimum set of functionality without needing to use plugins
> - but in general, should plugin code only use Core API (the Hub ?)
>   would be better than "directly" accessing other plugins code, because
>   the core might provide the error fallback in case a plugin isnt
>   present
>
> Plugin properties
> -
> - should be "hotpluggable", addable, removable during runtime
> - core needs to take care of disabling all related functionality when
>   module is unloaded (closing editor, saving for example)
> - each plugin needs an unique identifier of some kind, and versioning
>   info
> - if basis modules are in a plugin, this plugin needs to be flagged as
>   important or official or so
> - plugins could be classified by their purpose, like provides Module X,
>   extends Module Y, replaces Module Z, removes Module W
>
>
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[matmenu]

Hi all,
Am 03/11/2015 um 15:14 schrieb Sybren A. Stüvel:
> All in all I can see a few cons: 1) Blender will become more complex, 
> as not only its functionality needs to be maintained, but also the 
> "plumbing" required for the plugin system.2)  Development will become 
> more complex, as different permutations of loaded/unloaded plugins 
> have to be tested. 3) Mistakes in a C/C++ plugin can cause all kinds 
> of crashes of Blender itself. In contrast, an exception in a Python 
> script is relatively easy to catch & handle without sacrificing 
> Blender's stability. 4) Compilation of the code into libraries is 
> difficult, as it is very unlikely that all plugin developers will have 
> access to all OSses that Blender runs on. Sorry for this depressing 
> post, but I think we should discuss whether we want such a plugin 
> system at all, before diving into the specifics. Cheers, 
1) The point is to have a better use of the community ecosystem to get a 
faster and wider (for all the diferent publics Blender now has with 
animators, sculptors, 3D Games devs, architects, science, etc...) 
development. So I think everybody will be happy if functionnality 
integration is done better the old way (more reviews leading to more 
integration of community work). The core devs are here the best to 
decide what is the best for them (code and maintain a C API or review, 
commit and maintain master code)
2) Do you have a proof for this claim? I mean, modifier/operator 
combinations already lead to bugs, I don't see why having them as 
plugins/modules would make those problems arise more often?
3) That's true, but then the user decide to install this module, so he 
will report the bug to the original dev, making less work for the core 
ones. Only the problems coming from the core will be transfered, but 
that's already the case with the Python API, so it doesn't lead to more 
workload here, maybe even less or it will compensate for the overhead of 
maintaining the new API.
4) Also true, but we now have a buildbot that can make builds on demand 
for all supported OS. It could be used further by module/plugin devs and 
avoid having to ship a compiler with Blender.
Cheers
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[matmenu]

Hi,
What he said. Ask the Luxcore devs doing the integration with Blender. 
Of course, if you manage to create a proog like a very basic array 
modifier (only make duplicates of selected mesh every 1 BU on the x axis 
with the number given by the user, should be easy to program) that work 
as efficiently memory-wise and as fast as the one shipped with Blender 
and with the methode you discribed would be a great proof that C/C++ API 
is not needed.
Cheers,
Mat

Am 03/11/2015 um 14:50 schrieb Yury Baranov:
> Dalai, I think addon system has pretty much limited functionality.
> 1. No modifiers
> 2. No procedural objects
> 3. No animation controllers
> Every place where addon can't be used and where plugin needs to be used
> have a lot of calculations and a lot of "data sending/recieving/sending
> back" as a task. I think that's the main reason.
> For example, Sapling will be better as a interactive procedural object. But
> it's an addon, with all limitations and problems on the board.
> And of course need to mention that every plugin basically need 4 versions
> (because of its binary nature) - Linux, FreeBSD, Windows(32/64), MacOS X.
>
> 2015-11-03 16:14 GMT+03:00 Dalai Felinto :
>
>> Hi,
>>
>> I'm missing the main point here which would be, what is the advantage
>> of a C/C++ plugin system over the current Python addon interface?
>>
>> I'm currently developing an addon which relies on an external C++ SDK.
>> I got things working with ctypes and C API. It works pretty fine.
>>
>> Cheers,
>> Dalai
>> --
>> blendernetwork.org/dalai-felinto
>> www.dalaifelinto.com
>>
>>
>> 2015-11-03 10:59 GMT-02:00 Martin Felke :
>>> Hi, today i wrote down a couple of more thoughts regarding a plugin
>>> system. As said earlier, those serve only as discussion starting points
>>> and are not meant yet as design principles made out of cement. :)
>>>
>>> Fat VS Slim Core
>>> 
>>> - currently we have a fat core which contains all functionality
>>>as monolith
>>> - hard to extend and to maintain because it might be everything depends
>>>on everything else
>>> - idea: a slim, "bootloader like" core, which handles modules and
>>>plugins registration and deregistration and serves as API Hub (of
>>>existing modules)
>>> - modules are not standalone, they communicate via a central core API
>>>Hub so they need the core as well
>>> - this way the core can track module APIs and provide a fallback (as
>>>in exception handler) for missing code / functions (if module or
>>>plugin is not present)
>>>
>>> Modules VS Plugins
>>> --
>>> - need to distinguish between logical code separation (module) and
>>>"physical" separation (plugin, as in shared library or dll)
>>> - group existing core functionality to modules (like, an editor could
>>>be a module, or even a modifier)
>>> - base modules can be provided by "base" plugins, and extension modules
>>>by external plugins
>>> - plugins can provide parts of modules, entire modules or multiple
>>>modules (like nodes maybe, modifiers, new editors, or extensions to
>>>editors)
>>>
>>> Dependencies
>>> 
>>> - do we want to have inter-plugin dependencies ? For a slim core
>>>approach this might be necessary, if plugin A provides a module or
>>>part where plugin B 's code depends on
>>> - else base modules should be moved / kept in the core, so everyone has
>>>a minimum set of functionality without needing to use plugins
>>> - but in general, should plugin code only use Core API (the Hub ?)
>>>would be better than "directly" accessing other plugins code, because
>>>the core might provide the error fallback in case a plugin isnt
>>>present
>>>
>>> Plugin properties
>>> -
>>> - should be "hotpluggable", addable, removable during runtime
>>> - core needs to take care of disabling all related functionality when
>>>module is unloaded (closing editor, saving for example)
>>> - each plugin needs an unique identifier of some kind, and versioning
>>>info
>>> - if basis modules are in a plugin, this plugin needs to be flagged as
>>>important or official or so
>>> - plugins could be classified by their purpose, like provides Module X,
>>>extends Module Y, replaces Module Z, removes Module W
>>>
>>>
>>> ___
>>> Bf-committers mailing list
>>> Bf-committers@blender.org
>>> http://lists.blender.org/mailman/listinfo/bf-committers
>> ___
>> Bf-committers mailing list
>> Bf-committers@blender.org
>> http://lists.blender.org/mailman/listinfo/bf-committers
>>
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers

___
Bf-committers mailing list
Bf-committers@blender.org

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[Piotr Arlukowicz]

What about thinking about Blender as about a system composed in the idea of
different API joined together?
It has been said during Bcon15 that an asset system needs its own api. This
is drawing an idea to have Blender internally divided to different parts
(bmesh, assets, depsgraph, etc) joined thru APIs which are a bit similar to
modules mentioned here as 'plugins'. The whole Blender as a C-plugins
conglomerate is a bit scary and graphic, but we have already some kind of
that separation (I'm not very aware of Blender internals yet). Things done
in a proper way could give us a nicer separation between different parts of
the code, eliminate some spaghetti and old code and allow us to work on
some parts without the fear of breaking things here and there. So I would
opt for moving towards multicomponent system composed of smaller blackboxes
instead of current partially monolithic system if we are going to 2.8 and
speaking about C plugins will be easier when we will have nice Blender
multicomponent design.

cheers
pio

​Piotr
​ Arlukowicz, BFCT​

​*YT: /user/piotao?feature=guide*
 *FB:* */polskikursblendera* *TW:*
*/piotao*
*Blender Network:* *https://www.blendernetwork.org/piotr-arlukowicz
*
*Polski Kurs Blendera:* http://polskikursblendera.pl




2015-11-03 15:29 GMT+01:00 Gaia Clary :

> One of the benefits of a Blender PLugin System (bps) that i can think of
> is:
> The Collada module could be extracted into a separate plugin.
>
> But this can only work well if the bps does not force us to rebuild a
> plugin for each blender release. Otherwise maintenance would become a
> nightmare, especially because plugins need to be built for multiple
> platforms.
>
> cheers,
> Gaia
>
> On 03.11.2015 14:14, Dalai Felinto wrote:
> > Hi,
> >
> > I'm missing the main point here which would be, what is the advantage
> > of a C/C++ plugin system over the current Python addon interface?
> >
> > I'm currently developing an addon which relies on an external C++ SDK.
> > I got things working with ctypes and C API. It works pretty fine.
> >
> > Cheers,
> > Dalai
> > --
> > blendernetwork.org/dalai-felinto
> > www.dalaifelinto.com
> >
> >
> > 2015-11-03 10:59 GMT-02:00 Martin Felke :
> >> Hi, today i wrote down a couple of more thoughts regarding a plugin
> >> system. As said earlier, those serve only as discussion starting points
> >> and are not meant yet as design principles made out of cement. :)
> >>
> >> Fat VS Slim Core
> >> 
> >> - currently we have a fat core which contains all functionality
> >>as monolith
> >> - hard to extend and to maintain because it might be everything depends
> >>on everything else
> >> - idea: a slim, "bootloader like" core, which handles modules and
> >>plugins registration and deregistration and serves as API Hub (of
> >>existing modules)
> >> - modules are not standalone, they communicate via a central core API
> >>Hub so they need the core as well
> >> - this way the core can track module APIs and provide a fallback (as
> >>in exception handler) for missing code / functions (if module or
> >>plugin is not present)
> >>
> >> Modules VS Plugins
> >> --
> >> - need to distinguish between logical code separation (module) and
> >>"physical" separation (plugin, as in shared library or dll)
> >> - group existing core functionality to modules (like, an editor could
> >>be a module, or even a modifier)
> >> - base modules can be provided by "base" plugins, and extension modules
> >>by external plugins
> >> - plugins can provide parts of modules, entire modules or multiple
> >>modules (like nodes maybe, modifiers, new editors, or extensions to
> >>editors)
> >>
> >> Dependencies
> >> 
> >> - do we want to have inter-plugin dependencies ? For a slim core
> >>approach this might be necessary, if plugin A provides a module or
> >>part where plugin B 's code depends on
> >> - else base modules should be moved / kept in the core, so everyone has
> >>a minimum set of functionality without needing to use plugins
> >> - but in general, should plugin code only use Core API (the Hub ?)
> >>would be better than "directly" accessing other plugins code, because
> >>the core might provide the error fallback in case a plugin isnt
> >>present
> >>
> >> Plugin properties
> >> -
> >> - should be "hotpluggable", addable, removable during runtime
> >> - core needs to take care of disabling all related functionality when
> >>module is unloaded (closing editor, saving for example)
> >> - each plugin needs an unique identifier of some kind, and versioning
> >>info
> >> - if basis modules are in a plugin, this plugin needs to be flagged as
> >>important or official or so
> >> - plugins could be classified by their purpose, like provides Module X,
> >>extends Module Y, replaces 

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[Sybren A . Stüvel]

On Tue, Nov 03, 2015 at 08:04:03PM +0100, matmenu wrote:
> 2) Do you have a proof for this claim? I mean, modifier/operator
> combinations already lead to bugs, I don't see why having them as
> plugins/modules would make those problems arise more often?

I don't understand your reasoning. To me it reads like "we get bugs
now, so why would we get more when we complicate the software?" In the
current situation there is a fixed set of functionality in C++
(ignoring precompiler directives). In the case of a plugin structure,
not only do we have the same set of functionality, but also situations
where that functionality is only partially loaded. More different
possible situations -> more complex -> more bugs.

> 3) That's true, but then the user decide to install this module, so
> he will report the bug to the original dev

This is only true if the bug is trivial to to identify by end-users.
Plenty of bugs won't be. Think about double-free bugs, where the first
free is performed in the plugin. A crash may happen when Blender
itself frees the pointer, and thus any traceback (if that's shown at
all) will indicate a problem in Blender, not the plugin.

> 4) Also true, but we now have a buildbot that can make builds on
> demand for all supported OS. It could be used further by
> module/plugin devs and avoid having to ship a compiler with Blender.

AFAIK FreeBSD is not built by this system. Furthermore, allowing
arbitrary third party code to be run on a BI/BF-hosted machine is an
invitation for trouble. This already happens, but through the Git
repository, so at least it's only done by people that have been vetted
by receiving commit access.

-- 
Sybren A. Stüvel

http://stuvelfoto.nl/
http://stuvel.eu/


signature.asc
Description: Digital signature
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[Sybren A . Stüvel]

Hi all,

On Sun, Nov 01, 2015 at 04:11:54PM +0100, Martin Felke wrote:
> Hi, today i was thinking a bit about possible approaches on how to
> add a C / C++ Plugin system to blender.

There seems to be a thing missing: a good overview of the pros and
cons of having a C/C++ plugin system at all.

> - dynamic compilation of module sources via Python / CFFI or LLVM,
> as in OSL Nodes

Help me understand - how is this going to work, when most 3D artists
don't have a compiler installed? Are we going to ship Blender with a
compiler?

> - source distribution of plugins is C / C++  + some py Wrapper for
> autocompile or binary distribution in case no compiler is present

So you want to ship binaries as well? Then why bother with
autocompilation? Or is that only used while developing plugins?


On Mon, Nov 02, 2015 at 10:06:08PM +0100, matmenu wrote:
> Sounds really good and needed as reality showed that being GPL is
> not enough to make adding functionality easy. Looks also like all
> devs loose to much time reviewing, so c/c++ plugins are a really
> good solution to avoid so many modifiers die in the patch tracker or
> be limited to some custom builds like in past year.

Then again, creating a good plugin system is not easy, so that too
will take up developers time, which would otherwise be put into
feature development and bug fixing. Furthermore, creating and
maintaining an accurate dependency graph of the plugins will also take
time (and we'll forget to maintain those, causing bugs).

All in all I can see a few cons:

- Blender will become more complex, as not only its functionality
  needs to be maintained, but also the "plumbing" required for the
  plugin system.

- Development will become more complex, as different permutations of
  loaded/unloaded plugins have to be tested.

- Mistakes in a C/C++ plugin can cause all kinds of crashes of Blender
  itself. In contrast, an exception in a Python script is relatively
  easy to catch & handle without sacrificing Blender's stability.

- Compilation of the code into libraries is difficult, as it is very
  unlikely that all plugin developers will have access to all OSses
  that Blender runs on.

Sorry for this depressing post, but I think we should discuss whether
we want such a plugin system at all, before diving into the specifics.

Cheers,
-- 
Sybren A. Stüvel

http://stuvelfoto.nl/
http://stuvel.eu/
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[Yury Baranov]

Dalai, I think addon system has pretty much limited functionality.
1. No modifiers
2. No procedural objects
3. No animation controllers
Every place where addon can't be used and where plugin needs to be used
have a lot of calculations and a lot of "data sending/recieving/sending
back" as a task. I think that's the main reason.
For example, Sapling will be better as a interactive procedural object. But
it's an addon, with all limitations and problems on the board.
And of course need to mention that every plugin basically need 4 versions
(because of its binary nature) - Linux, FreeBSD, Windows(32/64), MacOS X.

2015-11-03 16:14 GMT+03:00 Dalai Felinto :

> Hi,
>
> I'm missing the main point here which would be, what is the advantage
> of a C/C++ plugin system over the current Python addon interface?
>
> I'm currently developing an addon which relies on an external C++ SDK.
> I got things working with ctypes and C API. It works pretty fine.
>
> Cheers,
> Dalai
> --
> blendernetwork.org/dalai-felinto
> www.dalaifelinto.com
>
>
> 2015-11-03 10:59 GMT-02:00 Martin Felke :
> > Hi, today i wrote down a couple of more thoughts regarding a plugin
> > system. As said earlier, those serve only as discussion starting points
> > and are not meant yet as design principles made out of cement. :)
> >
> > Fat VS Slim Core
> > 
> > - currently we have a fat core which contains all functionality
> >   as monolith
> > - hard to extend and to maintain because it might be everything depends
> >   on everything else
> > - idea: a slim, "bootloader like" core, which handles modules and
> >   plugins registration and deregistration and serves as API Hub (of
> >   existing modules)
> > - modules are not standalone, they communicate via a central core API
> >   Hub so they need the core as well
> > - this way the core can track module APIs and provide a fallback (as
> >   in exception handler) for missing code / functions (if module or
> >   plugin is not present)
> >
> > Modules VS Plugins
> > --
> > - need to distinguish between logical code separation (module) and
> >   "physical" separation (plugin, as in shared library or dll)
> > - group existing core functionality to modules (like, an editor could
> >   be a module, or even a modifier)
> > - base modules can be provided by "base" plugins, and extension modules
> >   by external plugins
> > - plugins can provide parts of modules, entire modules or multiple
> >   modules (like nodes maybe, modifiers, new editors, or extensions to
> >   editors)
> >
> > Dependencies
> > 
> > - do we want to have inter-plugin dependencies ? For a slim core
> >   approach this might be necessary, if plugin A provides a module or
> >   part where plugin B 's code depends on
> > - else base modules should be moved / kept in the core, so everyone has
> >   a minimum set of functionality without needing to use plugins
> > - but in general, should plugin code only use Core API (the Hub ?)
> >   would be better than "directly" accessing other plugins code, because
> >   the core might provide the error fallback in case a plugin isnt
> >   present
> >
> > Plugin properties
> > -
> > - should be "hotpluggable", addable, removable during runtime
> > - core needs to take care of disabling all related functionality when
> >   module is unloaded (closing editor, saving for example)
> > - each plugin needs an unique identifier of some kind, and versioning
> >   info
> > - if basis modules are in a plugin, this plugin needs to be flagged as
> >   important or official or so
> > - plugins could be classified by their purpose, like provides Module X,
> >   extends Module Y, replaces Module Z, removes Module W
> >
> >
> > ___
> > Bf-committers mailing list
> > Bf-committers@blender.org
> > http://lists.blender.org/mailman/listinfo/bf-committers
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[Vicente Carro]

Hi,

From a pipeline td point of view I would say keeping (and
extending/improving) the current python API is the best option.
When you need to do something during production usually you almost
have no time and for those situations python is the way to go. You
just do a prototype on the fly then you clean it a little bit and
ready to go. If you try to do that with C/C++ well, in the best case
scenario, is going to take way more time. And that if you are lucky.
For us is way more usable something that is fast to develop and slow
to execute than the reverse.

Regards

Vicente



On 3 November 2015 at 14:14, Sybren A. Stüvel  wrote:
> Hi all,
>
> On Sun, Nov 01, 2015 at 04:11:54PM +0100, Martin Felke wrote:
>> Hi, today i was thinking a bit about possible approaches on how to
>> add a C / C++ Plugin system to blender.
>
> There seems to be a thing missing: a good overview of the pros and
> cons of having a C/C++ plugin system at all.
>
>> - dynamic compilation of module sources via Python / CFFI or LLVM,
>> as in OSL Nodes
>
> Help me understand - how is this going to work, when most 3D artists
> don't have a compiler installed? Are we going to ship Blender with a
> compiler?
>
>> - source distribution of plugins is C / C++  + some py Wrapper for
>> autocompile or binary distribution in case no compiler is present
>
> So you want to ship binaries as well? Then why bother with
> autocompilation? Or is that only used while developing plugins?
>
>
> On Mon, Nov 02, 2015 at 10:06:08PM +0100, matmenu wrote:
>> Sounds really good and needed as reality showed that being GPL is
>> not enough to make adding functionality easy. Looks also like all
>> devs loose to much time reviewing, so c/c++ plugins are a really
>> good solution to avoid so many modifiers die in the patch tracker or
>> be limited to some custom builds like in past year.
>
> Then again, creating a good plugin system is not easy, so that too
> will take up developers time, which would otherwise be put into
> feature development and bug fixing. Furthermore, creating and
> maintaining an accurate dependency graph of the plugins will also take
> time (and we'll forget to maintain those, causing bugs).
>
> All in all I can see a few cons:
>
> - Blender will become more complex, as not only its functionality
>   needs to be maintained, but also the "plumbing" required for the
>   plugin system.
>
> - Development will become more complex, as different permutations of
>   loaded/unloaded plugins have to be tested.
>
> - Mistakes in a C/C++ plugin can cause all kinds of crashes of Blender
>   itself. In contrast, an exception in a Python script is relatively
>   easy to catch & handle without sacrificing Blender's stability.
>
> - Compilation of the code into libraries is difficult, as it is very
>   unlikely that all plugin developers will have access to all OSses
>   that Blender runs on.
>
> Sorry for this depressing post, but I think we should discuss whether
> we want such a plugin system at all, before diving into the specifics.
>
> Cheers,
> --
> Sybren A. Stüvel
>
> http://stuvelfoto.nl/
> http://stuvel.eu/
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[Gaia Clary]

One of the benefits of a Blender PLugin System (bps) that i can think of is:
The Collada module could be extracted into a separate plugin.

But this can only work well if the bps does not force us to rebuild a 
plugin for each blender release. Otherwise maintenance would become a 
nightmare, especially because plugins need to be built for multiple 
platforms.

cheers,
Gaia

On 03.11.2015 14:14, Dalai Felinto wrote:
> Hi,
>
> I'm missing the main point here which would be, what is the advantage
> of a C/C++ plugin system over the current Python addon interface?
>
> I'm currently developing an addon which relies on an external C++ SDK.
> I got things working with ctypes and C API. It works pretty fine.
>
> Cheers,
> Dalai
> --
> blendernetwork.org/dalai-felinto
> www.dalaifelinto.com
>
>
> 2015-11-03 10:59 GMT-02:00 Martin Felke :
>> Hi, today i wrote down a couple of more thoughts regarding a plugin
>> system. As said earlier, those serve only as discussion starting points
>> and are not meant yet as design principles made out of cement. :)
>>
>> Fat VS Slim Core
>> 
>> - currently we have a fat core which contains all functionality
>>as monolith
>> - hard to extend and to maintain because it might be everything depends
>>on everything else
>> - idea: a slim, "bootloader like" core, which handles modules and
>>plugins registration and deregistration and serves as API Hub (of
>>existing modules)
>> - modules are not standalone, they communicate via a central core API
>>Hub so they need the core as well
>> - this way the core can track module APIs and provide a fallback (as
>>in exception handler) for missing code / functions (if module or
>>plugin is not present)
>>
>> Modules VS Plugins
>> --
>> - need to distinguish between logical code separation (module) and
>>"physical" separation (plugin, as in shared library or dll)
>> - group existing core functionality to modules (like, an editor could
>>be a module, or even a modifier)
>> - base modules can be provided by "base" plugins, and extension modules
>>by external plugins
>> - plugins can provide parts of modules, entire modules or multiple
>>modules (like nodes maybe, modifiers, new editors, or extensions to
>>editors)
>>
>> Dependencies
>> 
>> - do we want to have inter-plugin dependencies ? For a slim core
>>approach this might be necessary, if plugin A provides a module or
>>part where plugin B 's code depends on
>> - else base modules should be moved / kept in the core, so everyone has
>>a minimum set of functionality without needing to use plugins
>> - but in general, should plugin code only use Core API (the Hub ?)
>>would be better than "directly" accessing other plugins code, because
>>the core might provide the error fallback in case a plugin isnt
>>present
>>
>> Plugin properties
>> -
>> - should be "hotpluggable", addable, removable during runtime
>> - core needs to take care of disabling all related functionality when
>>module is unloaded (closing editor, saving for example)
>> - each plugin needs an unique identifier of some kind, and versioning
>>info
>> - if basis modules are in a plugin, this plugin needs to be flagged as
>>important or official or so
>> - plugins could be classified by their purpose, like provides Module X,
>>extends Module Y, replaces Module Z, removes Module W
>>
>>
>> ___
>> Bf-committers mailing list
>> Bf-committers@blender.org
>> http://lists.blender.org/mailman/listinfo/bf-committers
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers

___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

Re: [Bf-committers] Some Ideas for a Blender Plugin System

[matmenu]

Hi Martin,

Sounds really good and needed as reality showed that being GPL is not 
enough to make adding functionality easy. Looks also like all devs loose 
to much time reviewing, so c/c++ plugins are a really good solution to 
avoid so many modifiers die in the patch tracker or be limited to some 
custom builds like in past year. I hope you get some help to make it a 
reality!

Regards,
Mat

Am 01/11/2015 um 16:11 schrieb Martin Felke:
> Hi, today i was thinking a bit about possible approaches on how to
> add a C / C++ Plugin system to blender.
> This is a short (a bit unsorted) list of my thoughts and might serve as
> discussion start point perhaps.
>
> Blender Plugin System
> -
>   
> - Idea: Have a modularized approach, add C/C++ to core without
>modifying it later
> - track presence of modules and status (OK, Not Functional, Changed)
> - store Plugin Data separately from Blend (e.g. FM simulation /
>shard data), so core can start with basic blend only if plugin is
>absent can try to load plugin while loading data
>  
> - dynamic compilation of module sources via Python / CFFI or LLVM,
>as in OSL Nodes
> - access core api thru bpy or directly thru RNA or even DNA
> - source distribution of plugins is C / C++  + some py Wrapper
>for autocompile or binary distribution in case no compiler is present
> - plugins have certain structure and versions for all OS;
>Hash, Version, source, binaries
> - should be installable and activatable like addons
>   
> - pack CFFI and distutils to python distro of blender
> - windows fix: declspec, symbol export so linking will work
>   
> - plugin development addon: provide DNA / RNA headers
>(and C sources or precompiled stuff) + project generators
>for standard IDEs, to give a starting point for plugin devs
>   
> - core support: - Plugin Management system which handles plugins
>  - modify ID and Non-ID base structures to include
> info about plugin origin
>  - save,load data per plugin / chunked
> (and bundle differently together)
>  - plugin crash handler
> (so not the entire core goes down with a
> failing plugin)
>
> - plugins should expose py api too, so addons can build upon them too
> - plugin repos and updating / automatic compilation system
>(after update) ?
>
> ___
> Bf-committers mailing list
> Bf-committers@blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers

___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers

[Bf-committers] Some Ideas for a Blender Plugin System

[Martin Felke]

Hi, today i was thinking a bit about possible approaches on how to 
add a C / C++ Plugin system to blender.
This is a short (a bit unsorted) list of my thoughts and might serve as
discussion start point perhaps.

Blender Plugin System 
-
 
- Idea: Have a modularized approach, add C/C++ to core without
  modifying it later
- track presence of modules and status (OK, Not Functional, Changed)
- store Plugin Data separately from Blend (e.g. FM simulation / 
  shard data), so core can start with basic blend only if plugin is 
  absent can try to load plugin while loading data

- dynamic compilation of module sources via Python / CFFI or LLVM, 
  as in OSL Nodes
- access core api thru bpy or directly thru RNA or even DNA
- source distribution of plugins is C / C++  + some py Wrapper 
  for autocompile or binary distribution in case no compiler is present
- plugins have certain structure and versions for all OS; 
  Hash, Version, source, binaries
- should be installable and activatable like addons
 
- pack CFFI and distutils to python distro of blender
- windows fix: declspec, symbol export so linking will work
 
- plugin development addon: provide DNA / RNA headers 
  (and C sources or precompiled stuff) + project generators
  for standard IDEs, to give a starting point for plugin devs
 
- core support: - Plugin Management system which handles plugins
- modify ID and Non-ID base structures to include 
  info about plugin origin
- save,load data per plugin / chunked 
  (and bundle differently together)
- plugin crash handler 
  (so not the entire core goes down with a 
  failing plugin)

- plugins should expose py api too, so addons can build upon them too
- plugin repos and updating / automatic compilation system 
  (after update) ?   

___
Bf-committers mailing list
Bf-committers@blender.org
http://lists.blender.org/mailman/listinfo/bf-committers