Re: [Pharo-users] UFFI and Fortran

[Todd Blanchard]

I keep running into issues on 64 bit FFI.

Compiling 32 bit libraries is atypical and requires a lot of fiddling of 
unfamiliar build systems for various libraries so I'm just going to work in 64 
bit land on Mac.

> On Nov 6, 2017, at 7:25 PM, horrido  wrote:
> 
> That would be phenomenal! Let me know if and when it's done and I shall
> scream it out to all of social media!
> 
> 
> 
> tblanchard wrote
>> Yes - the reason I've been trying to learn FFI is specifically to get
>> TensorFlow integration.
>> 
>> I want a better ML workbench.
>> 
>>> On Nov 2, 2017, at 9:08 PM, Ben Coman <
> 
>> btc@
> 
>> > wrote:
>>> 
>>> 
>>> 
>>> On Fri, Nov 3, 2017 at 11:10 AM, horrido <
> 
>> horrido.hobbies@
> 
>>  
>> horrido.hobbies@
> 
>> >> wrote:
>>> Sounds like it's possible to call into TensorFlow (Python) from Pharo.
>>> That
>>> would give Pharo a tremendous boost for machine learning applications.
>>> 
>>> Am I right?
>>> 
>>> Tensorflow has a C API for FFI from other languages.
>>> * https://www.tensorflow.org/install/install_c
>>> ;
>>> * https://www.tensorflow.org/extend/language_bindings
>>> ;
>>> 
>>> cheers -ben
> 
> 
> 
> 
> 
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
> 

Re: [Pharo-users] UFFI and Fortran

[horrido]

That would be phenomenal! Let me know if and when it's done and I shall
scream it out to all of social media!



tblanchard wrote
> Yes - the reason I've been trying to learn FFI is specifically to get
> TensorFlow integration.
> 
> I want a better ML workbench.
> 
>> On Nov 2, 2017, at 9:08 PM, Ben Coman <

> btc@

> > wrote:
>> 
>> 
>> 
>> On Fri, Nov 3, 2017 at 11:10 AM, horrido <

> horrido.hobbies@

>   horrido.hobbies@

> >> wrote:
>> Sounds like it's possible to call into TensorFlow (Python) from Pharo.
>> That
>> would give Pharo a tremendous boost for machine learning applications.
>> 
>> Am I right?
>> 
>> Tensorflow has a C API for FFI from other languages.
>> * https://www.tensorflow.org/install/install_c
>> ;
>> * https://www.tensorflow.org/extend/language_bindings
>> ;
>> 
>> cheers -ben





--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] UFFI and Fortran

[Dimitris Chloupis]

Yes Atlas , a library I made that is available via Package Browser from
inside the Pharo image, makes Pharo capable of using any cpython library
from inside Pharo, with Pharo syntax, live coding and debugging again
inside Pharo. The Pharo syntax is limited to just calling methods and
variable assignments. The library also has inlining python code (pass path
code as a string from inside your pharo code) that gives access to full
python capabilities (creating Python classes, list comprehensions,
decorators, threading, multiprocessing etc).

The advantage of using UFFI over Atlas is that it gives far better
performance. Plus gives access to C libraries not wrapped for Python.

The advantage of Atlas is that allows for dynamic execution of the library
without the need for the creation of wrappers.  Plus it allows for remote
execution, meaning the python library can be in another computer or server
anywhere in the world. So its much more than a FFI.

Of course both approaches will require from you to map C or Python data to
Pharo objects.

At some point I will update Atlas to be as fast as UFFI , adding to the
socket functionality also shared memory that is vastly faster. For now this
is a low priority for me.

On Fri, Nov 3, 2017 at 5:10 AM horrido  wrote:

> Sounds like it's possible to call into TensorFlow (Python) from Pharo. That
> would give Pharo a tremendous boost for machine learning applications.
>
> Am I right?
>
>
> kilon.alios wrote
> > Maybe you talk about TalkFFI which aumatically wrapped C libraries for
> > Pharo and i think Squeak as well but used Nativeboos, i think
> >
> >
> http://forum.world.st/TalkFFI-automatic-FFI-generation-for-Pharo-td4662239.html
> >
> > On the subject of Fortran yes you can use UFFI if Fortran code is
> compiled
> > as a DLL (or equivelant non Windoom OSes)
> >
> > https://software.intel.com/en-us/node/535305
> >
> > Dynamic Library format are not exclusive to C for generation , many
> > languages can generate them so even though UFFI is used predominatly for
> C
> > any language that can generate a DLL without any name mangling should be
> > accessible via UFFI.
> >
> > if DLL generation is not possible, then you can drop down to my solution
> > of
> > Shared Memory Mapped Files. Which means you make an executable in Frotran
> > (or any other language)  that contains the code you want to access and
> > creates a shared memory region and you can access that region from Pharo
> > via UFFI.
> >
> > This is how I built my CPPBridge project. Which one can use as a template
> > for generating similar bridge for Fortran or any other language where the
> > generation of DLLs is not practical or possible.
> >
> > The shared memory region can be used also as a means of communication
> > additional to sharing live state, memory mapped files automatically save
> > the live state so next time you open the file it behaves as you would
> > expect a Pharo image to behave by restoring the live state. A means to
> > extend Pharo image to include memory not managed by the VM.
> >
> > Because memory mapped files is mechanism of the OS Kernel and is also the
> > mechanism used for the loading of dynamic libraries like DLLs there is
> > also
> > no loss of performance and is the fastest way of communication between
> > languages, libraries and applications.
> >
> > So yes using Fortran code is possible via UFFI in more than one way.
> >
> > But in the end it should be noted here that because IPC mechanisms
> (common
> > way of using libraries from other languages) are based on core OS
> > functionality like , memory managment, sockets , pipes, etc its not hard
> > to
> > use libraries from any language from inside any language.
> >
> > So Pharo can use Fortran libraries and Fortran can use Pharo libraries,
> > its
> > also possible to retain live coding even when executing code written in
> > another language through various means. Recently I was succesful in
> making
> > Python into a basic live coding enviroment , meaning code that when
> > changed
> > in the source file it updates also existing live intances as you expect
> in
> > Pharo. Python also supports memory mapped files so its possible to create
> > a
> > joined live coding enviroment and live image that containes both Pharo
> and
> > Python bytecode. Of course without touching VM source code.
> >
> > Even though live state is not tricky to retain for compiled languages,
> > live
> > code can be a bit trickier to do but still not impossible or that hard as
> > most would assume.
> >
> > Sky is the limit.
> >
> > Bottom line is that if you have a favorite library in any language you
> can
> > use it from inside Pharo with at worst a few hundrend lines of setup code
> > you can create as a library and of coure reuse next time you want to use
> > again a library from another language without having to write a line of
> > additional code. The technology is available is just a matter of learning
> > how to use it.  Especially if its

Re: [Pharo-users] UFFI and Fortran

[Todd Blanchard]

Yes - the reason I've been trying to learn FFI is specifically to get 
TensorFlow integration.

I want a better ML workbench.

> On Nov 2, 2017, at 9:08 PM, Ben Coman  wrote:
> 
> 
> 
> On Fri, Nov 3, 2017 at 11:10 AM, horrido  > wrote:
> Sounds like it's possible to call into TensorFlow (Python) from Pharo. That
> would give Pharo a tremendous boost for machine learning applications.
> 
> Am I right?
> 
> Tensorflow has a C API for FFI from other languages.
> * https://www.tensorflow.org/install/install_c 
> 
> * https://www.tensorflow.org/extend/language_bindings 
> 
> 
> cheers -ben
> 
> 
> 
> kilon.alios wrote
> > Maybe you talk about TalkFFI which aumatically wrapped C libraries for
> > Pharo and i think Squeak as well but used Nativeboos, i think
> >
> > http://forum.world.st/TalkFFI-automatic-FFI-generation-for-Pharo-td4662239.html
> >  
> > 
> >
> > On the subject of Fortran yes you can use UFFI if Fortran code is compiled
> > as a DLL (or equivelant non Windoom OSes)
> >
> > https://software.intel.com/en-us/node/535305 
> > 
> >
> > Dynamic Library format are not exclusive to C for generation , many
> > languages can generate them so even though UFFI is used predominatly for C
> > any language that can generate a DLL without any name mangling should be
> > accessible via UFFI.
> >
> > if DLL generation is not possible, then you can drop down to my solution
> > of
> > Shared Memory Mapped Files. Which means you make an executable in Frotran
> > (or any other language)  that contains the code you want to access and
> > creates a shared memory region and you can access that region from Pharo
> > via UFFI.
> >
> > This is how I built my CPPBridge project. Which one can use as a template
> > for generating similar bridge for Fortran or any other language where the
> > generation of DLLs is not practical or possible.
> >
> > The shared memory region can be used also as a means of communication
> > additional to sharing live state, memory mapped files automatically save
> > the live state so next time you open the file it behaves as you would
> > expect a Pharo image to behave by restoring the live state. A means to
> > extend Pharo image to include memory not managed by the VM.
> >
> > Because memory mapped files is mechanism of the OS Kernel and is also the
> > mechanism used for the loading of dynamic libraries like DLLs there is
> > also
> > no loss of performance and is the fastest way of communication between
> > languages, libraries and applications.
> >
> > So yes using Fortran code is possible via UFFI in more than one way.
> >
> > But in the end it should be noted here that because IPC mechanisms (common
> > way of using libraries from other languages) are based on core OS
> > functionality like , memory managment, sockets , pipes, etc its not hard
> > to
> > use libraries from any language from inside any language.
> >
> > So Pharo can use Fortran libraries and Fortran can use Pharo libraries,
> > its
> > also possible to retain live coding even when executing code written in
> > another language through various means. Recently I was succesful in making
> > Python into a basic live coding enviroment , meaning code that when
> > changed
> > in the source file it updates also existing live intances as you expect in
> > Pharo. Python also supports memory mapped files so its possible to create
> > a
> > joined live coding enviroment and live image that containes both Pharo and
> > Python bytecode. Of course without touching VM source code.
> >
> > Even though live state is not tricky to retain for compiled languages,
> > live
> > code can be a bit trickier to do but still not impossible or that hard as
> > most would assume.
> >
> > Sky is the limit.
> >
> > Bottom line is that if you have a favorite library in any language you can
> > use it from inside Pharo with at worst a few hundrend lines of setup code
> > you can create as a library and of coure reuse next time you want to use
> > again a library from another language without having to write a line of
> > additional code. The technology is available is just a matter of learning
> > how to use it.  Especially if its a very large libraries it will be
> > thousands times easier than trying to replicate the functionality in
> > Pharo.
> >
> >
> >
> > On Fri, Oct 27, 2017 at 6:26 PM Sean P. DeNigris <
> 
> > sean@
> 
> > >
> > wrote:
> >
> >> Ben Coman wrote
> >> > it seems to hint how to do it from Pharo UFFI.
> >>
> >> Slightly OT: I remember years ago, someone (Dave Mason?) demoed a library
> >> which automatically created FFI wrappers for C libs. I never heard
> >> anything
> >> about it after that, which is sad, because I was amazed and wanted to use
> >> it!
> >>
> >>
> >>
> >> -

Re: [Pharo-users] UFFI and Fortran

[Ben Coman]

On Fri, Nov 3, 2017 at 11:10 AM, horrido  wrote:

> Sounds like it's possible to call into TensorFlow (Python) from Pharo. That
> would give Pharo a tremendous boost for machine learning applications.
>
> Am I right?
>

Tensorflow has a C API for FFI from other languages.
* https://www.tensorflow.org/install/install_c
* https://www.tensorflow.org/extend/language_bindings

cheers -ben


>
> kilon.alios wrote
> > Maybe you talk about TalkFFI which aumatically wrapped C libraries for
> > Pharo and i think Squeak as well but used Nativeboos, i think
> >
> > http://forum.world.st/TalkFFI-automatic-FFI-generation-for-
> Pharo-td4662239.html
> >
> > On the subject of Fortran yes you can use UFFI if Fortran code is
> compiled
> > as a DLL (or equivelant non Windoom OSes)
> >
> > https://software.intel.com/en-us/node/535305
> >
> > Dynamic Library format are not exclusive to C for generation , many
> > languages can generate them so even though UFFI is used predominatly for
> C
> > any language that can generate a DLL without any name mangling should be
> > accessible via UFFI.
> >
> > if DLL generation is not possible, then you can drop down to my solution
> > of
> > Shared Memory Mapped Files. Which means you make an executable in Frotran
> > (or any other language)  that contains the code you want to access and
> > creates a shared memory region and you can access that region from Pharo
> > via UFFI.
> >
> > This is how I built my CPPBridge project. Which one can use as a template
> > for generating similar bridge for Fortran or any other language where the
> > generation of DLLs is not practical or possible.
> >
> > The shared memory region can be used also as a means of communication
> > additional to sharing live state, memory mapped files automatically save
> > the live state so next time you open the file it behaves as you would
> > expect a Pharo image to behave by restoring the live state. A means to
> > extend Pharo image to include memory not managed by the VM.
> >
> > Because memory mapped files is mechanism of the OS Kernel and is also the
> > mechanism used for the loading of dynamic libraries like DLLs there is
> > also
> > no loss of performance and is the fastest way of communication between
> > languages, libraries and applications.
> >
> > So yes using Fortran code is possible via UFFI in more than one way.
> >
> > But in the end it should be noted here that because IPC mechanisms
> (common
> > way of using libraries from other languages) are based on core OS
> > functionality like , memory managment, sockets , pipes, etc its not hard
> > to
> > use libraries from any language from inside any language.
> >
> > So Pharo can use Fortran libraries and Fortran can use Pharo libraries,
> > its
> > also possible to retain live coding even when executing code written in
> > another language through various means. Recently I was succesful in
> making
> > Python into a basic live coding enviroment , meaning code that when
> > changed
> > in the source file it updates also existing live intances as you expect
> in
> > Pharo. Python also supports memory mapped files so its possible to create
> > a
> > joined live coding enviroment and live image that containes both Pharo
> and
> > Python bytecode. Of course without touching VM source code.
> >
> > Even though live state is not tricky to retain for compiled languages,
> > live
> > code can be a bit trickier to do but still not impossible or that hard as
> > most would assume.
> >
> > Sky is the limit.
> >
> > Bottom line is that if you have a favorite library in any language you
> can
> > use it from inside Pharo with at worst a few hundrend lines of setup code
> > you can create as a library and of coure reuse next time you want to use
> > again a library from another language without having to write a line of
> > additional code. The technology is available is just a matter of learning
> > how to use it.  Especially if its a very large libraries it will be
> > thousands times easier than trying to replicate the functionality in
> > Pharo.
> >
> >
> >
> > On Fri, Oct 27, 2017 at 6:26 PM Sean P. DeNigris <
>
> > sean@
>
> > >
> > wrote:
> >
> >> Ben Coman wrote
> >> > it seems to hint how to do it from Pharo UFFI.
> >>
> >> Slightly OT: I remember years ago, someone (Dave Mason?) demoed a
> library
> >> which automatically created FFI wrappers for C libs. I never heard
> >> anything
> >> about it after that, which is sad, because I was amazed and wanted to
> use
> >> it!
> >>
> >>
> >>
> >> -
> >> Cheers,
> >> Sean
> >> --
> >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
> >>
> >>
>
>
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>
>

Re: [Pharo-users] UFFI and Fortran

[horrido]

Sounds like it's possible to call into TensorFlow (Python) from Pharo. That
would give Pharo a tremendous boost for machine learning applications.

Am I right?


kilon.alios wrote
> Maybe you talk about TalkFFI which aumatically wrapped C libraries for
> Pharo and i think Squeak as well but used Nativeboos, i think
> 
> http://forum.world.st/TalkFFI-automatic-FFI-generation-for-Pharo-td4662239.html
> 
> On the subject of Fortran yes you can use UFFI if Fortran code is compiled
> as a DLL (or equivelant non Windoom OSes)
> 
> https://software.intel.com/en-us/node/535305
> 
> Dynamic Library format are not exclusive to C for generation , many
> languages can generate them so even though UFFI is used predominatly for C
> any language that can generate a DLL without any name mangling should be
> accessible via UFFI.
> 
> if DLL generation is not possible, then you can drop down to my solution
> of
> Shared Memory Mapped Files. Which means you make an executable in Frotran
> (or any other language)  that contains the code you want to access and
> creates a shared memory region and you can access that region from Pharo
> via UFFI.
> 
> This is how I built my CPPBridge project. Which one can use as a template
> for generating similar bridge for Fortran or any other language where the
> generation of DLLs is not practical or possible.
> 
> The shared memory region can be used also as a means of communication
> additional to sharing live state, memory mapped files automatically save
> the live state so next time you open the file it behaves as you would
> expect a Pharo image to behave by restoring the live state. A means to
> extend Pharo image to include memory not managed by the VM.
> 
> Because memory mapped files is mechanism of the OS Kernel and is also the
> mechanism used for the loading of dynamic libraries like DLLs there is
> also
> no loss of performance and is the fastest way of communication between
> languages, libraries and applications.
> 
> So yes using Fortran code is possible via UFFI in more than one way.
> 
> But in the end it should be noted here that because IPC mechanisms (common
> way of using libraries from other languages) are based on core OS
> functionality like , memory managment, sockets , pipes, etc its not hard
> to
> use libraries from any language from inside any language.
> 
> So Pharo can use Fortran libraries and Fortran can use Pharo libraries,
> its
> also possible to retain live coding even when executing code written in
> another language through various means. Recently I was succesful in making
> Python into a basic live coding enviroment , meaning code that when
> changed
> in the source file it updates also existing live intances as you expect in
> Pharo. Python also supports memory mapped files so its possible to create
> a
> joined live coding enviroment and live image that containes both Pharo and
> Python bytecode. Of course without touching VM source code.
> 
> Even though live state is not tricky to retain for compiled languages,
> live
> code can be a bit trickier to do but still not impossible or that hard as
> most would assume.
> 
> Sky is the limit.
> 
> Bottom line is that if you have a favorite library in any language you can
> use it from inside Pharo with at worst a few hundrend lines of setup code
> you can create as a library and of coure reuse next time you want to use
> again a library from another language without having to write a line of
> additional code. The technology is available is just a matter of learning
> how to use it.  Especially if its a very large libraries it will be
> thousands times easier than trying to replicate the functionality in
> Pharo.
> 
> 
> 
> On Fri, Oct 27, 2017 at 6:26 PM Sean P. DeNigris <

> sean@

> >
> wrote:
> 
>> Ben Coman wrote
>> > it seems to hint how to do it from Pharo UFFI.
>>
>> Slightly OT: I remember years ago, someone (Dave Mason?) demoed a library
>> which automatically created FFI wrappers for C libs. I never heard
>> anything
>> about it after that, which is sad, because I was amazed and wanted to use
>> it!
>>
>>
>>
>> -
>> Cheers,
>> Sean
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>
>>





--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] UFFI and Fortran

[Todd Blanchard]

SmalltalkAgents used to be able to point to any shared library and import all 
the functions.

> On Oct 27, 2017, at 8:26 AM, Sean P. DeNigris  wrote:
> 
> Ben Coman wrote
>> it seems to hint how to do it from Pharo UFFI.
> 
> Slightly OT: I remember years ago, someone (Dave Mason?) demoed a library
> which automatically created FFI wrappers for C libs. I never heard anything
> about it after that, which is sad, because I was amazed and wanted to use
> it!
> 
> 
> 
> -
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
> 

Re: [Pharo-users] UFFI and Fortran

[Dimitris Chloupis]

Maybe you talk about TalkFFI which aumatically wrapped C libraries for
Pharo and i think Squeak as well but used Nativeboos, i think

http://forum.world.st/TalkFFI-automatic-FFI-generation-for-Pharo-td4662239.html

On the subject of Fortran yes you can use UFFI if Fortran code is compiled
as a DLL (or equivelant non Windoom OSes)

https://software.intel.com/en-us/node/535305

Dynamic Library format are not exclusive to C for generation , many
languages can generate them so even though UFFI is used predominatly for C
any language that can generate a DLL without any name mangling should be
accessible via UFFI.

if DLL generation is not possible, then you can drop down to my solution of
Shared Memory Mapped Files. Which means you make an executable in Frotran
(or any other language)  that contains the code you want to access and
creates a shared memory region and you can access that region from Pharo
via UFFI.

This is how I built my CPPBridge project. Which one can use as a template
for generating similar bridge for Fortran or any other language where the
generation of DLLs is not practical or possible.

The shared memory region can be used also as a means of communication
additional to sharing live state, memory mapped files automatically save
the live state so next time you open the file it behaves as you would
expect a Pharo image to behave by restoring the live state. A means to
extend Pharo image to include memory not managed by the VM.

Because memory mapped files is mechanism of the OS Kernel and is also the
mechanism used for the loading of dynamic libraries like DLLs there is also
no loss of performance and is the fastest way of communication between
languages, libraries and applications.

So yes using Fortran code is possible via UFFI in more than one way.

But in the end it should be noted here that because IPC mechanisms (common
way of using libraries from other languages) are based on core OS
functionality like , memory managment, sockets , pipes, etc its not hard to
use libraries from any language from inside any language.

So Pharo can use Fortran libraries and Fortran can use Pharo libraries, its
also possible to retain live coding even when executing code written in
another language through various means. Recently I was succesful in making
Python into a basic live coding enviroment , meaning code that when changed
in the source file it updates also existing live intances as you expect in
Pharo. Python also supports memory mapped files so its possible to create a
joined live coding enviroment and live image that containes both Pharo and
Python bytecode. Of course without touching VM source code.

Even though live state is not tricky to retain for compiled languages, live
code can be a bit trickier to do but still not impossible or that hard as
most would assume.

Sky is the limit.

Bottom line is that if you have a favorite library in any language you can
use it from inside Pharo with at worst a few hundrend lines of setup code
you can create as a library and of coure reuse next time you want to use
again a library from another language without having to write a line of
additional code. The technology is available is just a matter of learning
how to use it.  Especially if its a very large libraries it will be
thousands times easier than trying to replicate the functionality in Pharo.



On Fri, Oct 27, 2017 at 6:26 PM Sean P. DeNigris 
wrote:

> Ben Coman wrote
> > it seems to hint how to do it from Pharo UFFI.
>
> Slightly OT: I remember years ago, someone (Dave Mason?) demoed a library
> which automatically created FFI wrappers for C libs. I never heard anything
> about it after that, which is sad, because I was amazed and wanted to use
> it!
>
>
>
> -
> Cheers,
> Sean
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>
>

Re: [Pharo-users] UFFI and Fortran

[Sean P. DeNigris]

Ben Coman wrote
> it seems to hint how to do it from Pharo UFFI.

Slightly OT: I remember years ago, someone (Dave Mason?) demoed a library
which automatically created FFI wrappers for C libs. I never heard anything
about it after that, which is sad, because I was amazed and wanted to use
it!



-
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] UFFI and Fortran

[Ben Coman]

Always starting by assuming someone in another language wants to do the
same ;)
googling... fortran ffi python
second result...
https://maurow.bitbucket.io/notes/calling_fortran_from_misc.html

I only skimmed quickly, but it seems to hint how to do it from Pharo UFFI.

cheers -ben

P.S. Interesting concept.  All too often its C libraries used as an example,
but IIUC, Fortran still provides the highest performance of all the
numerical libraries, with a many-year pedigree of reliability.