Hi, On 2018-01-27 22:06:59 -0800, Jeff Davis wrote: > > infeasible because quite freuquently both non-JITed code and JITed code > > need adjustments. That'd solve your concern about > > Can you explain further?
There's already a *lot* of integration points in the patchseries. Error handling needs to happen in parts of code we do not want to make extensible, the defintion of expression steps has to exactly match, the core code needs to emit the right types for syncing, the core code needs to define the right FIELDNO accessors, there needs to be planner integrations. Many of those aren't doable with even remotely the same effort, both initial and continual, from non-core code.... I think those alone make it bad, but there'll be more. Short-Medium term expression evaluation needs to evolve further to make JITing cachable: http://archives.postgresql.org/message-id/20180124203616.3gx4vm45hpoijpw3%40alap3.anarazel.de which again definitely has to be happen in core and will require corresponding changes on the JIT side very step. Then we'll need to introduce something like plancache (or something similar?) support for JITing to reuse JITed functions. Then there's also a significant difference in how large the adoption's going to be, and how all the core code that'd need to be added is supposed to be testable without the JIT emitting side in core. > > I think it's a fools errand to try to keep in sync with core changes on > > the expression evaluation and struct definition side of things. There's > > planner integration, error handling integration and similar related > > things too, all of which require core changes. Therefore I don't think > > there's a reasonable chance of success of doing this outside of core > > postgres. > > I wasn't suggesting the entire patch be done outside of core. Core > will certainly need to know about JIT compilation, but I am not > convinced that it needs to know about the details of LLVM. All the > references to the LLVM library itself are contained in a few files, so > you've already got it well organized. What's stopping us from putting > that code into a "jit provider" extension that implements the proper > interfaces? The above hopefully answers that? What we could do, imo somewhat realistically, is to put most of the provider into a dynamically loaded shared library that lives in core (similar to how we build the pgoutput output plugin shared library as part of core). But that still would end up hard coding things like LLVM specific error handling etc, which we currently do *NOT* want to be extensible. > > Well, but doing this outside of core would pretty much prohibit doing so > > forever, no? > > First of all, building .bc files at build time is much less invasive > than linking to the LLVM library. Could you expand on that, I don't understand why that'd be the case? > Any version of clang will produce bitcode that can be read by any LLVM > library or tool later (more or less). Well, forward portable, not backward portable. > Second, we could change our minds later. Mark any extension APIs as > experimental, and decide we want to move LLVM into postgres whenever > it is needed. > > Third, there's lots of cool stuff we can do here: > * put the source in the catalog > * an extension could have its own catalog and build the source into > bitcode and cache it there > * the source for functions would flow to replicas, etc. > * security-conscious environments might even choose to run some of > the C code in a safe C interpreter rather than machine code I agree, but what does that have to do with the llvmjit stuff being an extension or not? Greetings, Andres Freund