Is it possible for you to make a small test package that shows the problem, and file a bug in bugs.swift.org? There may be something we need to fix in SwiftPM before this can work (because of our linking model).
- Daniel > On Oct 7, 2017, at 10:42 PM, Ján Kosa via swift-users <swift-users@swift.org> > wrote: > > That is exactly what I did. The only package that depends on the protobuf is > the PluginInterface. Both MyPlugin and and PluginConsumer depend on the > PluginInterface and not on the protobuf itself. I had to shuffle around my > dependencies a bit, which resulted in smaller number of dependencies but they > don't make much sense now (as in, some target had to depend on > PluginInterface even if they don't need to, just to get access to protobuf). > I could live with that if it solved the issue, but it didn't. > > I am adding my Package.swift files in case I missed something: > > PluginInterface: > > ```swift > let package = Package( > name: "PluginInterface", > > products: [ .library(name: "PluginInterface", type: .dynamic, targets: > ["PluginInterface"]) ], > > dependencies: [ .package(url: "https://github.com/apple/swift-protobuf.git > <https://github.com/apple/swift-protobuf.git>", from: "0.0.0") ], > > targets: [ .target(name: "PluginInterface", dependencies: ["SwiftProtobuf"]) ] > > )``` > > > > MyPlugin: > > ```swift > > let package = Package( > > name: "MyPlugin", > > products: [ .library(name: "MyPlugin", type: .dynamic, targets: > ["PluginImpl"]) ], > > dependencies: [ > > .package(url: "path/to/PluginInterface.git", from: "0.0.0"), > > ], > > targets: [ > > .target(name: "PluginImpl", dependencies: ["ProtoBufMessages"]), > > .target(name: "ProtoBufMessages", dependencies: ["PluginInterface"]) > > ] > > > )``` > > > > PluginConsumer: > > ```swift > > let package = Package( > > name: "PluginConsumer", > > dependencies: [ > > .package(url: "https://github.com/PerfectlySoft/Perfect-WebSockets.git > <https://github.com/PerfectlySoft/Perfect-WebSockets.git>", from: "3.0.0"), > > .package(url: "https://github.com/PerfectlySoft/Perfect-HTTPServer.git > <https://github.com/PerfectlySoft/Perfect-HTTPServer.git>", from: "3.0.0"), > > .package(url: "path/to/PluginInterface", from: "0.0.0"), > > .package(url: "https://github.com/krzyzanowskim/CryptoSwift.git > <https://github.com/krzyzanowskim/CryptoSwift.git>", from: "0.0.0") > > ], > > targets: [ > > .target(name: "AppMaster", dependencies: ["Shared", "CryptoSwift"]), > > .target(name: "PluginConsumer", dependencies: ["Shared", "CryptoSwift"]), > > .target(name: "Shared", dependencies: ["ProtoBufMessages", > "PerfectHTTPServer", "PerfectWebSockets"]), > > .target(name: "ProtoBufMessages", dependencies: ["PluginInterface"]) > > ] > > > )``` > > > > App master is separate executable that shares some functionality with > PluginConsumer, but it doesn't link against it in any way. I guess it could > be omitted, but I wanted to give you whole thing as it is > > > On 7 October 2017 at 18:33, Geordie Jay <geo...@gmail.com > <mailto:geo...@gmail.com>> wrote: > > Ján Kosa <lope...@gmail.com <mailto:lope...@gmail.com>> schrieb am Sa. 7. > Okt. 2017 um 15:27: > I tried to use @_exported and it helped somewhat. While I still have same > warnings, size of the PluginInterface library went down by 6mb (to 120kb) so > it looks like Protobuf is no longer statically linked to it. However, size of > PluginConsumer executable went up by same 6mb, it looks like it is linked > there twice now. > > To be clear: take protobuf out of the PluginConsumer dependencies. Actually, > I’m not sure which is which, but protobuf should only be listed as a > dependency of one package, where it is imported as @_exported. After that, > your other modules depend on the package that imports / exports protobuf. > > I also noticed interesting thing. If I build executable using `swift build` > the size is around 17mb, when I generate xcode project and build it using > that, size is around 200kb, but I get same warnings using both approaches > > On 7 October 2017 at 15:44, Geordie Jay <geo...@gmail.com > <mailto:geo...@gmail.com>> wrote: > > Ján Kosa <lope...@gmail.com <mailto:lope...@gmail.com>> schrieb am Sa. 7. > Okt. 2017 um 13:34: > I tried swift package clean, but it didn't help > > "Try to ensure the plugin provider module (libA) is (only) being compiled > into its standalone shared library file." > How do I go about this? It is 3rd party module, it doesn't define any > products (https://github.com/apple/swift-protobuf.git > <https://github.com/apple/swift-protobuf.git>). Is there something I can do > in my Package files to make sure it is loaded dynamically? > > When you compile a package depending on protobuf, all the relevant symbols > end up in your package’s library file. So here’s something you might try: > > import protobuf into your own “PluginProvider” module (package), which has a > shared library product like this: ‘@_exported import Protobuf’ in some > compiled swift file. Then from the other dependent modules “import > PluginProvider” - the protobuf symbols should be available, and all from one > (nonconflicting) source. > > Geordie > > > > On 6 October 2017 at 22:52, Geordie Jay <geo...@gmail.com > <mailto:geo...@gmail.com>> wrote: > I think SwiftPM is (incorrectly) compiling A.XYZ <http://a.xyz/> into each of > the modules that depend on it, as well as into your intended libA.so file. > > Try to ensure the plugin provider module (libA) is (only) being compiled into > its standalone shared library file. Try cleaning the swiftpm build for one > (swift package clean) and ensure the Package.swift files are correctly set up > to output the shared library. > > Sorry I can’t be more specific, I’ve had these same kinds of issues before > but I’m not 100% what they were. > > Geordie > > > Ján Kosa via swift-users <swift-users@swift.org > <mailto:swift-users@swift.org>> schrieb am Fr. 6. Okt. 2017 um 14:41: > It worked! Took me a while to iron out details, but it is working now. Huge > thanks sir, I will name my firstborn after you. > Thanks for the @_cdecl("initializePlugin") tip as well, I didn't know about > it and it will be very useful. > > I am having slightly related problem now (it was there before, but I ignored > it for the time being), not sure if I should start a new thread? > > The PluginInterface module has one external dependency on module A, > PluginConsumer has the dependency on module B which has dependency on same > module A that the PluginInterface uses. When I load the plugin library, I get > bunch of errors like: > > Class A.XYZ <http://a.xyz/> is implemented in both libPluginInterface.dylib > and libMyPlugin.dylib > > I know why it is there, but I don't know how to get rid of it. I can't just > remove dependency from PluginConsumer and use the one from PluginInterface > (if that would even work?) because PluginConsumer does not depend on it > directly, but it is going through module B first > > Cheers, > Lope > > On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dun...@apple.com > <mailto:daniel_dun...@apple.com>> wrote: > The way that I have done this in the past is pass a protocol as an unsafe > pointer to an exposed entry point: > ```swift > let entryPoint = dlsym(handle, “initializePlugin”) > guard entryPoint != nil else { > fatalError("missing plugin entry point: \(pluginPath)") > } > typealias PluginInitializationFunc = @convention(c) > (UnsafeRawPointer) -> () > let f = unsafeBitCast(entryPoint, to: > PluginInitializationFunc.self) > f(Unmanaged.passUnretained(self).toOpaque()) > ``` > > and then in the plugin convert back to the appropriate type: > > ``` > @_cdecl("initializePlugin") > public func initializePlugin(_ ptr: UnsafeRawPointer) { > let manager = > Unmanaged<PluginManager>.fromOpaque(ptr).takeUnretainedValue() > ``` > > HTH, > - Daniel > >> On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users <swift-users@swift.org >> <mailto:swift-users@swift.org>> wrote: >> >> Hello folks, >> >> I have been toying with dynamic libraries, trying to implement plugin >> functionality. I was able to get to the point where I can call simple >> function in loaded library, but I am having troubles starting more >> sophisticated communication channel. >> >> There are 3 projects >> - PluginConsumer is an app that loads plugin libraries >> - MyPlugin is a plugin implementation, output is dynamic library that >> PluginConsumer loads >> - PluginInterface is common interface that both MyPlugin and PluginConsumer >> use, so that they know how to communicate >> >> My first idea was to have PluginInterface be a simple SPM project with >> single file where the bare-bones PluginInterface class would be: >> >> >> open class PluginInterface { >> >> open func sayHi() >> >> } >> >> >> >> Package.swift file: >> >> >> >> // swift-tools-version:4.0 >> >> import PackageDescription >> >> let package = Package( >> >> name: "PluginInterface", >> >> products: [ .library(name: "PluginInterface", type: .dynamic, targets: >> ["PluginInterface"]) ], >> >> targets: [ .target(name: "PluginInterface") ] >> >> >> ) >> >> >> >> >> >> UserPlugin is also very simple project containing only one file: >> >> >> >> public func getPlugin() -> AnyObject { >> >> return MyPlugin() >> >> >> } >> >> >> >> class MyPlugin: PluginInterface { >> >> override func sayHi() { >> >> print("Hi from my plugin") >> >> } >> >> } >> >> Package.swift: >> >> >> >> // swift-tools-version:4.0 >> >> import PackageDescription >> >> let package = Package( >> >> name: "MyPlugin", >> >> products: [ .library(name: "MyPlugin", type: .dynamic, targets: >> ["MyPlugin"]) ], >> >> dependencies: [ .package(url: "url_to_PluginInterface", from: "0.0.0"), >> ], >> >> targets: [ >> >> .target(name: "PluginInterface", dependencies: ["PluginInterface"]), >> >> .target(name: "MyPlugin", dependencies: ["PluginInterface"]), >> >> ] >> >> >> ) >> >> >> >> The PluginConsumer is bit more complicated, but here is relevant part (lib >> loading and function calling): >> >> >> >> typealias InitFunction = @convention(c) () -> AnyObject >> >> >> >> let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL) >> >> if openRes != nil { >> >> defer { >> >> dlclose(openRes) >> >> } >> >> let symbolName = "mangled_symbol_name" >> >> let sym = dlsym(openRes, symbolName) >> >> >> >> if sym != nil { >> >> let f: InitFunction = unsafeBitCast(sym, to: InitFunction.self) >> >> let plugin = f() as? PluginInterface >> >> } >> >> >> } >> >> Package.swift file: >> >> // swift-tools-version:4.0 >> >> import PackageDescription >> >> let package = Package( >> >> name: "PluginConsumer", >> >> dependencies: [ .package(url: "path_to_plugin_interface", from: "0.0.0") >> ], >> >> targets: [ .target(name: "PluginConsumer", dependencies: >> ["PluginConsumer"]) ] >> >> >> ) >> >> >> >> This all compiles nicely, MyPlugin project creates dylib file that >> executable created by PluginConsumer can load, but the problem is with >> following line: >> >> let plugin = f() as? PluginInterface >> >> Type of the plugin is MyPlugin, but from the consumer's view, it doesn't >> inherit from PluginInterface so I can't call sayHi() method. I assume this >> is because there is no relation between PluginInterface class that compiler >> uses for MyPlugin project one that it uses for PluginConsumer project. After >> library is loaded, they are two completely different classes that happen to >> share same name. Is my assumption correct and how do I go about fixing it? >> >> I had an idea I could make PluginInterface emit dynamic library that would >> be dynamically linked by both MyPlugin and PluginConsumer, thus making them >> share same PluginInterface class, but I can't figure out how to do that (or >> if it's right way of doing this). >> >> >> >> Any help appreciated :) >> >> Lope >> >> _______________________________________________ >> swift-users mailing list >> swift-users@swift.org <mailto:swift-users@swift.org> >> https://lists.swift.org/mailman/listinfo/swift-users >> <https://lists.swift.org/mailman/listinfo/swift-users> > > > _______________________________________________ > swift-users mailing list > swift-users@swift.org <mailto:swift-users@swift.org> > https://lists.swift.org/mailman/listinfo/swift-users > <https://lists.swift.org/mailman/listinfo/swift-users> > > > > _______________________________________________ > swift-users mailing list > swift-users@swift.org > https://lists.swift.org/mailman/listinfo/swift-users
_______________________________________________ swift-users mailing list swift-users@swift.org https://lists.swift.org/mailman/listinfo/swift-users
