I was afraid it will come to that :) I will try to make something tonight, either it will help you fix it, or I will find out what I did wrong
On 8 October 2017 at 09:49, Daniel Dunbar <daniel_dun...@apple.com> wrote: > Is it possible for you to make a small test package that shows the > problem, and file a bug in bugs.swift.org? There may be something we need > to fix in SwiftPM before this can work (because of our linking model). > > - Daniel > > > On Oct 7, 2017, at 10:42 PM, Ján Kosa via swift-users < > swift-users@swift.org> wrote: > > That is exactly what I did. The only package that depends on the protobuf > is the PluginInterface. Both MyPlugin and and PluginConsumer depend on the > PluginInterface and not on the protobuf itself. I had to shuffle around my > dependencies a bit, which resulted in smaller number of dependencies but > they don't make much sense now (as in, some target had to depend on > PluginInterface even if they don't need to, just to get access to > protobuf). I could live with that if it solved the issue, but it didn't. > > I am adding my Package.swift files in case I missed something: > > PluginInterface: > > ```swift > let package = Package( > > name: "PluginInterface", > > products: [ .library(name: "PluginInterface", type: .dynamic, targets: [ > "PluginInterface"]) ], > > dependencies: [ .package(url: "https://github.com/apple/swift-protobuf.git > ", from: "0.0.0") ], > > targets: [ .target(name: "PluginInterface", dependencies: ["SwiftProtobuf"]) > ] > > )``` > > > MyPlugin: > > ```swift > > let package = Package( > > name: "MyPlugin", > > products: [ .library(name: "MyPlugin", type: .dynamic, targets: [ > "PluginImpl"]) ], > > dependencies: [ > > .package(url: "path/to/PluginInterface.git", from: "0.0.0"), > > ], > > targets: [ > > .target(name: "PluginImpl", dependencies: ["ProtoBufMessages"]), > > .target(name: "ProtoBufMessages", dependencies: ["PluginInterface"]) > > ] > > )``` > > > PluginConsumer: > > ```swift > > let package = Package( > > name: "PluginConsumer", > > dependencies: [ > > .package(url: "https://github.com/PerfectlySoft/Perfect-WebSockets.git";, > from: "3.0.0"), > > .package(url: "https://github.com/PerfectlySoft/Perfect-HTTPServer.git";, > from: "3.0.0"), > > .package(url: "path/to/PluginInterface", from: "0.0.0"), > > .package(url: "https://github.com/krzyzanowskim/CryptoSwift.git";, from: > "0.0.0") > > ], > > targets: [ > > .target(name: "AppMaster", dependencies: ["Shared", "CryptoSwift"]), > > .target(name: "PluginConsumer", dependencies: ["Shared", "CryptoSwift"]), > > .target(name: "Shared", dependencies: ["ProtoBufMessages", > "PerfectHTTPServer", "PerfectWebSockets"]), > > .target(name: "ProtoBufMessages", dependencies: ["PluginInterface"]) > > ] > > )``` > > > App master is separate executable that shares some functionality with > PluginConsumer, but it doesn't link against it in any way. I guess it could > be omitted, but I wanted to give you whole thing as it is > > On 7 October 2017 at 18:33, Geordie Jay <geo...@gmail.com> wrote: > >> >> Ján Kosa <lope...@gmail.com> schrieb am Sa. 7. Okt. 2017 um 15:27: >> >>> I tried to use @_exported and it helped somewhat. While I still have >>> same warnings, size of the PluginInterface library went down by 6mb (to >>> 120kb) so it looks like Protobuf is no longer statically linked to it. >>> However, size of PluginConsumer executable went up by same 6mb, it looks >>> like it is linked there twice now. >>> >> >> To be clear: take protobuf out of the PluginConsumer dependencies. >> Actually, I’m not sure which is which, but protobuf should only be listed >> as a dependency of one package, where it is imported as @_exported. After >> that, your other modules depend on the package that imports / exports >> protobuf. >> >> I also noticed interesting thing. If I build executable using `swift >>> build` the size is around 17mb, when I generate xcode project and build it >>> using that, size is around 200kb, but I get same warnings using both >>> approaches >>> >>> On 7 October 2017 at 15:44, Geordie Jay <geo...@gmail.com> wrote: >>> >>>> >>>> Ján Kosa <lope...@gmail.com> schrieb am Sa. 7. Okt. 2017 um 13:34: >>>> >>>>> I tried swift package clean, but it didn't help >>>>> >>>>> "Try to ensure the plugin provider module (libA) is (only) being >>>>> compiled into its standalone shared library file." >>>>> How do I go about this? It is 3rd party module, it doesn't define any >>>>> products (https://github.com/apple/swift-protobuf.git). Is there >>>>> something I can do in my Package files to make sure it is loaded >>>>> dynamically? >>>>> >>>> >>>> When you compile a package depending on protobuf, all the relevant >>>> symbols end up in your package’s library file. So here’s something you >>>> might try: >>>> >>>> import protobuf into your own “PluginProvider” module (package), which >>>> has a shared library product like this: ‘@_exported import Protobuf’ in >>>> some compiled swift file. Then from the other dependent modules “import >>>> PluginProvider” - the protobuf symbols should be available, and all from >>>> one (nonconflicting) source. >>>> >>>> Geordie >>>> >>>> >>>> >>>>> On 6 October 2017 at 22:52, Geordie Jay <geo...@gmail.com> wrote: >>>>> >>>>>> I think SwiftPM is (incorrectly) compiling A.XYZ <http://a.xyz/> >>>>>> into each of the modules that depend on it, as well as into your intended >>>>>> libA.so file. >>>>>> >>>>>> Try to ensure the plugin provider module (libA) is (only) being >>>>>> compiled into its standalone shared library file. Try cleaning the >>>>>> swiftpm >>>>>> build for one (swift package clean) and ensure the Package.swift files >>>>>> are >>>>>> correctly set up to output the shared library. >>>>>> >>>>>> Sorry I can’t be more specific, I’ve had these same kinds of issues >>>>>> before but I’m not 100% what they were. >>>>>> >>>>>> Geordie >>>>>> >>>>>> >>>>>> Ján Kosa via swift-users <swift-users@swift.org> schrieb am Fr. 6. >>>>>> Okt. 2017 um 14:41: >>>>>> >>>>>>> It worked! Took me a while to iron out details, but it is working >>>>>>> now. Huge thanks sir, I will name my firstborn after you. >>>>>>> Thanks for the @_cdecl("initializePlugin") tip as well, I didn't >>>>>>> know about it and it will be very useful. >>>>>>> >>>>>>> I am having slightly related problem now (it was there before, but I >>>>>>> ignored it for the time being), not sure if I should start a new thread? >>>>>>> >>>>>>> The PluginInterface module has one external dependency on module A, >>>>>>> PluginConsumer has the dependency on module B which has dependency on >>>>>>> same >>>>>>> module A that the PluginInterface uses. When I load the plugin library, >>>>>>> I >>>>>>> get bunch of errors like: >>>>>>> >>>>>>> Class A.XYZ <http://a.xyz/> is implemented in >>>>>>> both libPluginInterface.dylib and libMyPlugin.dylib >>>>>>> >>>>>>> I know why it is there, but I don't know how to get rid of it. I >>>>>>> can't just remove dependency from PluginConsumer and use the one from >>>>>>> PluginInterface (if that would even work?) because PluginConsumer does >>>>>>> not >>>>>>> depend on it directly, but it is going through module B first >>>>>>> >>>>>>> Cheers, >>>>>>> Lope >>>>>>> >>>>>>> On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dun...@apple.com> >>>>>>> wrote: >>>>>>> >>>>>>>> The way that I have done this in the past is pass a protocol as an >>>>>>>> unsafe pointer to an exposed entry point: >>>>>>>> ```swift >>>>>>>> let entryPoint = dlsym(handle, “initializePlugin”) >>>>>>>> guard entryPoint != nil else { >>>>>>>> fatalError("missing plugin entry point: >>>>>>>> \(pluginPath)") >>>>>>>> } >>>>>>>> typealias PluginInitializationFunc = @convention(c) >>>>>>>> (UnsafeRawPointer) -> () >>>>>>>> let f = unsafeBitCast(entryPoint, to: >>>>>>>> PluginInitializationFunc.self) >>>>>>>> f(Unmanaged.passUnretained(self).toOpaque()) >>>>>>>> ``` >>>>>>>> >>>>>>>> and then in the plugin convert back to the appropriate type: >>>>>>>> >>>>>>>> ``` >>>>>>>> @_cdecl("initializePlugin") >>>>>>>> public func initializePlugin(_ ptr: UnsafeRawPointer) { >>>>>>>> let manager = Unmanaged<PluginManager>.fromO >>>>>>>> paque(ptr).takeUnretainedValue() >>>>>>>> ``` >>>>>>>> >>>>>>>> HTH, >>>>>>>> - Daniel >>>>>>>> >>>>>>>> On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users < >>>>>>>> swift-users@swift.org> wrote: >>>>>>>> >>>>>>>> Hello folks, >>>>>>>> >>>>>>>> I have been toying with dynamic libraries, trying to implement >>>>>>>> plugin functionality. I was able to get to the point where I can call >>>>>>>> simple function in loaded library, but I am having troubles starting >>>>>>>> more >>>>>>>> sophisticated communication channel. >>>>>>>> >>>>>>>> There are 3 projects >>>>>>>> - PluginConsumer is an app that loads plugin libraries >>>>>>>> - MyPlugin is a plugin implementation, output is dynamic library >>>>>>>> that PluginConsumer loads >>>>>>>> - PluginInterface is common interface that both MyPlugin and >>>>>>>> PluginConsumer use, so that they know how to communicate >>>>>>>> >>>>>>>> My first idea was to have PluginInterface be a simple SPM project >>>>>>>> with single file where the bare-bones PluginInterface class would be: >>>>>>>> >>>>>>>> >>>>>>>> open class PluginInterface { >>>>>>>> >>>>>>>> open func sayHi() >>>>>>>> >>>>>>>> } >>>>>>>> >>>>>>>> >>>>>>>> Package.swift file: >>>>>>>> >>>>>>>> >>>>>>>> // swift-tools-version:4.0 >>>>>>>> >>>>>>>> import PackageDescription >>>>>>>> >>>>>>>> let package = Package( >>>>>>>> >>>>>>>> name: "PluginInterface", >>>>>>>> >>>>>>>> products: [ .library(name: "PluginInterface", type: .dynamic, >>>>>>>> targets: ["PluginInterface"]) ], >>>>>>>> >>>>>>>> targets: [ .target(name: "PluginInterface") ] >>>>>>>> >>>>>>>> ) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> UserPlugin is also very simple project containing only one file: >>>>>>>> >>>>>>>> >>>>>>>> public func getPlugin() -> AnyObject { >>>>>>>> >>>>>>>> return MyPlugin() >>>>>>>> >>>>>>>> } >>>>>>>> >>>>>>>> >>>>>>>> class MyPlugin: PluginInterface { >>>>>>>> >>>>>>>> override func sayHi() { >>>>>>>> >>>>>>>> print("Hi from my plugin") >>>>>>>> >>>>>>>> } >>>>>>>> >>>>>>>> } >>>>>>>> >>>>>>>> Package.swift: >>>>>>>> >>>>>>>> >>>>>>>> // swift-tools-version:4.0 >>>>>>>> >>>>>>>> import PackageDescription >>>>>>>> >>>>>>>> let package = Package( >>>>>>>> >>>>>>>> name: "MyPlugin", >>>>>>>> >>>>>>>> products: [ .library(name: "MyPlugin", type: .dynamic, >>>>>>>> targets: ["MyPlugin"]) ], >>>>>>>> >>>>>>>> dependencies: [ .package(url: "url_to_PluginInterface", from: >>>>>>>> "0.0.0"), ], >>>>>>>> >>>>>>>> targets: [ >>>>>>>> >>>>>>>> .target(name: "PluginInterface", dependencies: [ >>>>>>>> "PluginInterface"]), >>>>>>>> >>>>>>>> .target(name: "MyPlugin", dependencies: ["PluginInterface" >>>>>>>> ]), >>>>>>>> >>>>>>>> ] >>>>>>>> >>>>>>>> ) >>>>>>>> >>>>>>>> >>>>>>>> The PluginConsumer is bit more complicated, but here is relevant >>>>>>>> part (lib loading and function calling): >>>>>>>> >>>>>>>> >>>>>>>> typealias InitFunction = @convention(c) () -> AnyObject >>>>>>>> >>>>>>>> >>>>>>>> let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL) >>>>>>>> >>>>>>>> if openRes != nil { >>>>>>>> >>>>>>>> defer { >>>>>>>> >>>>>>>> dlclose(openRes) >>>>>>>> >>>>>>>> } >>>>>>>> >>>>>>>> let symbolName = "mangled_symbol_name" >>>>>>>> >>>>>>>> let sym = dlsym(openRes, symbolName) >>>>>>>> >>>>>>>> >>>>>>>> if sym != nil { >>>>>>>> >>>>>>>> let f: InitFunction = unsafeBitCast(sym, to: InitFunction. >>>>>>>> self) >>>>>>>> >>>>>>>> let plugin = f() as? PluginInterface >>>>>>>> >>>>>>>> } >>>>>>>> >>>>>>>> } >>>>>>>> >>>>>>>> Package.swift file: >>>>>>>> >>>>>>>> // swift-tools-version:4.0 >>>>>>>> >>>>>>>> import PackageDescription >>>>>>>> >>>>>>>> let package = Package( >>>>>>>> >>>>>>>> name: "PluginConsumer", >>>>>>>> >>>>>>>> dependencies: [ .package(url: "path_to_plugin_interface", >>>>>>>> from: "0.0.0") ], >>>>>>>> >>>>>>>> targets: [ .target(name: "PluginConsumer", dependencies: [ >>>>>>>> "PluginConsumer"]) ] >>>>>>>> >>>>>>>> ) >>>>>>>> >>>>>>>> >>>>>>>> This all compiles nicely, MyPlugin project creates dylib file that >>>>>>>> executable created by PluginConsumer can load, but the problem is with >>>>>>>> following line: >>>>>>>> >>>>>>>> let plugin = f() as? PluginInterface >>>>>>>> >>>>>>>> Type of the plugin is MyPlugin, but from the consumer's view, it >>>>>>>> doesn't inherit from PluginInterface so I can't call sayHi() method. I >>>>>>>> assume this is because there is no relation between PluginInterface >>>>>>>> class >>>>>>>> that compiler uses for MyPlugin project one that it uses for >>>>>>>> PluginConsumer >>>>>>>> project. After library is loaded, they are two completely different >>>>>>>> classes >>>>>>>> that happen to share same name. Is my assumption correct and how do I >>>>>>>> go >>>>>>>> about fixing it? >>>>>>>> >>>>>>>> I had an idea I could make PluginInterface emit dynamic library >>>>>>>> that would be dynamically linked by both MyPlugin and PluginConsumer, >>>>>>>> thus >>>>>>>> making them share same PluginInterface class, but I can't figure out >>>>>>>> how to >>>>>>>> do that (or if it's right way of doing this). >>>>>>>> >>>>>>>> >>>>>>>> Any help appreciated :) >>>>>>>> >>>>>>>> Lope >>>>>>>> _______________________________________________ >>>>>>>> swift-users mailing list >>>>>>>> swift-users@swift.org >>>>>>>> https://lists.swift.org/mailman/listinfo/swift-users >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> swift-users mailing list >>>>>>> swift-users@swift.org >>>>>>> https://lists.swift.org/mailman/listinfo/swift-users >>>>>>> >>>>>> >>>>> >>> > _______________________________________________ > swift-users mailing list > swift-users@swift.org > https://lists.swift.org/mailman/listinfo/swift-users > > >
_______________________________________________ swift-users mailing list swift-users@swift.org https://lists.swift.org/mailman/listinfo/swift-users
