#general
@ita.pai: @ita.pai has joined the channel
#random
@ita.pai: @ita.pai has joined the channel
#feat-text-search
@adilsonbna: @adilsonbna has joined the channel
@tsjagan: @tsjagan has joined the channel
#feat-presto-connector
@adilsonbna: @adilsonbna has joined the channel
@tsjagan: @tsjagan has joined the channel
#troubleshooting
@ricardobordon: Hi folks! QQ, by reading
@jackie.jxt: Thanks for reporting this. We will update the links
@jackie.jxt: Here is the up-to-date contribution guidelines:
@ricardobordon: Thank you! Very appreciated!
@gergely.lendvai93: Hi everyone, I was trying to add custom udfs to Pinot and it seems they can be used (at least I was able to use it from query console), however when I was checking the logs I saw that none of the plugins under the plugins folder were managed to load during initialization. I also tried it without my custom jar but it resulted in the same. Here are some examples errors:
@gergely.lendvai93: ```2021/02/24 15:56:42.896 ERROR [PluginManager] [main] Failed to load plugin [pinot-thrift] from dir [/opt/pinot/plugins/pinot-input-format/pinot-thrift] java.lang.IllegalArgumentException: object is not an instance of declaring class at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.apache.pinot.spi.plugin.PluginClassLoader.<init>(PluginClassLoader.java:50) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.createClassLoader(PluginManager.java:196) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.load(PluginManager.java:187) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.init(PluginManager.java:157) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.init(PluginManager.java:123) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.<init>(PluginManager.java:104) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.<clinit>(PluginManager.java:46) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:182) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] 2021/02/24 15:56:42.897 ERROR [PluginManager] [main] Failed to load plugin [pinot-kafka-2.0] from dir [/opt/pinot/plugins/pinot-stream-ingestion/pinot-kafka-2.0] java.lang.IllegalArgumentException: object is not an instance of declaring class at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.apache.pinot.spi.plugin.PluginClassLoader.<init>(PluginClassLoader.java:50) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.createClassLoader(PluginManager.java:196) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.load(PluginManager.java:187) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.init(PluginManager.java:157) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.init(PluginManager.java:123) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.<init>(PluginManager.java:104) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.spi.plugin.PluginManager.<clinit>(PluginManager.java:46) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df] at org.apache.pinot.tools.admin.PinotAdministrator.main(PinotAdministrator.java:182) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-d74224a14b07bbe3e5c4909b51856886e29510df]```
@gergely.lendvai93: I saw this
@jackie.jxt: @fx19880617 Any idea on this issue?
@fx19880617: I think for java-11 pinot by default import all the libs from plugins dir
@fx19880617: you can skip the env var `-Dplugins.dir=/opt/pinot/plugins -Dplugins.include=pinot-s3`
@gergely.lendvai93: hey, sorry my message was split into 3 parts… These are the first 2 parts of it:
@fx19880617: for java 11, can you just try to drop the lib under a directory of `plugins`
@fx19880617: then it should be auto imported
@gergely.lendvai93: yeah, exactly what I did, I placed the lib under `plugins/udfs/` and I could use them from the controller query console. It is just weird that in spite of it working I get the error messages during the startup of the components.
@bowlesns: Running a query and I get different results for one field each time I run it. Any explanation for why this might happen? ```SELECT id, week(fromDateTime(datefield, 'yyyy-MM-dd')) as week, SUM(f1 * f2 * f3) AS "ftotal" ****THIS IS THE FIELD THAT RETURNS DIFFERENT RESULTS**** FROM mytable WHERE year(fromDateTime(datefield, 'yyyy-MM-dd')) >= cast(year(now())-2 as long) AND id in (123, 1234, 12345) AND f4 = 'blah' AND f5 in ('1-2', '3-4') AND f6 in ('foo', 'bar') GROUP BY 1, 2```
@jackie.jxt: Different result for the same group or it returns different groups?
@bowlesns: Actually you’re right, it is returning different groups based off the second column. It looks like it is limiting the results to 10 (I am using the query console) but I don’t have a limit in my query
@bowlesns: maybe the console limits it to 10 by default?
@jackie.jxt: Yes, the limit is 10 by default if not specified
@bowlesns: whoops, PEBKAC haha. Thanks Jackie!
@bowlesns: The table says the state is “BAD” although the number of reported segments are 100%. Recieved a few broken connection errors when ingesting so running again, but I’m not sure if that’s related to the different query results.
@ita.pai: @ita.pai has joined the channel
#pinot-dev
@adilsonbna: @adilsonbna has joined the channel
@tsjagan: @tsjagan has joined the channel
#segment-write-api
@yupeng: do you think we can make it in 1.0 release?
@fx19880617: segment writer api ? I don’t see any issue to prevent this
@npawar: when is 1.0?
@yupeng: the graduation in april?
@yupeng: cool. coz the flink connector needs to depend on it
@npawar: wait when was graduation in April decided :joy: i didnt know!
@yupeng: i just took the words from the PQL deprecation thread…
@yupeng: from Mayank
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
