Apache Pinot Daily Email Digest (2021-03-10)

Pinot Slack Email Digest Wed, 10 Mar 2021 18:00:52 -0800

#general

@james.wes.taylor: @james.wes.taylor has joined the channel
@contactvivekjain: @contactvivekjain has joined the channel
@carlosmanzueta: @carlosmanzueta has joined the channel
@dileepkumarv.allam: @dileepkumarv.allam has joined the channel
@joshhighley: what's the method for stopping a Pinot server? "StopServer" isn't a valid sub-command, do I just kill the process?
@g.kishore: yes, there is a shutdown hook that gets triggered
@manish.bhoge: @manish.bhoge has joined the channel
@manish.bhoge: I'm trying to set up the docker image of Pinot, and to set up this I'm doing the maven build : ```# Build Pinot $ mvn clean install -DskipTests -Pbin-dist``` But, it is failing with an error, any idea on this below error: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project pinot-yammer: Execution default of goal org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade failed: Plugin org.apache.maven.plugins:maven-shade-plugin:3.2.1 or one of its dependencies could not be resolved: The following artifacts could not be resolved: org.apache.maven.shared:maven-artifact-transfer:jar:0.10.0, org.ow2.asm:asm:jar:7.0: Could not transfer artifact org.apache.maven.shared:maven-artifact-transfer:jar:0.10.0 from/to central (): Connect to [] failed: Connection timed out (Connection timed out) -> [Help 1]
@g.kishore: might be a transient error, can you try again
@manish.bhoge: Yes tried couple of times ..but still no luck :(
@g.kishore: latest code?
@manish.bhoge: Its latest from the git repo... I assume that's your question
@g.kishore: yes
@g.kishore: delete the jar in m2 and try again
@g.kishore:
@g.kishore: Try increasing the timeout value
@joshhighley: what's the difference between ```bin/pinot-admin.sh StartServer``` and ```bin/start-server.sh``` ? Which way should be used?
@g.kishore: both of them trigger the same command. use pinot-admin as the entry point for everything
@joshhighley: if I use start-server.sh it doesn't load our existing segments. If I use pinot-admin.sh there's no logging to pinotServer.log
@g.kishore: see the tab "Using launcher scripts"
@joshhighley: we were using that method but it wasn't logging to pinotServer.log
@joshhighley: pinot-admin appears to use pinot-admin-log4j2.xml but I don't see it actually logging to the component log files
@g.kishore: Yeah, I noticed that as well
@karinwolok1: :thinking_face: Interested in learning more about the TrinoDB (formally, PrestoSQL) and the Apache Pinot :wine_glass: connector? :film_projector: Join @fx19880617, @elon.azoulay next week for a live broadcast with the TrinoDB team!
@santosh.rudra: @santosh.rudra has joined the channel
@xulinnankai: @xulinnankai has joined the channel
@g.kishore: Some of you might be waiting for updates the next release, we created a channel <#C01QQTV06MB|releases> for further updates. Please join the channel if you want to hear more. You can also always stay up to date by subscribing to
@morzaria: @morzaria has joined the channel
@suresh.k.kode: @suresh.k.kode has joined the channel

#random

#troubleshooting

@james.wes.taylor: @james.wes.taylor has joined the channel
@girishbhat.m7: In the class org.apache.pinot.plugin.ingestion.batch.standalone.SegmentGenerationJobRunner.init(SegmentGenerationJobSpec) method below statements throws error ``` _inputDirFS = PinotFSFactory.create(_inputDirURI.getScheme());``` if the scheme is not loaded and in the run method schemes are registered . Isn't it good way to add method like below in the init method of the Jobrunner ? ```void registerSchemsIfNotRegistered(_spec)```
@g.kishore: @fx19880617 what do you think?
@fx19880617: this should be auto loaded when you configure the fs properties
@fx19880617: the pinot fs and corresponded scheme should be registered in the `PinotFSSpec`
@fx19880617: this is one example:
@contactvivekjain: @contactvivekjain has joined the channel
@carlosmanzueta: @carlosmanzueta has joined the channel
@dileepkumarv.allam: @dileepkumarv.allam has joined the channel
@manish.bhoge: @manish.bhoge has joined the channel
@manish.bhoge: I'm trying to set up the docker image of Pinot, and to set up this I'm doing the maven build : ```# Build Pinot $ mvn clean install -DskipTests -Pbin-dist``` But, it is failing with an error, any idea on this below error: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (default) on project pinot-yammer: Execution default of goal org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade failed: Plugin org.apache.maven.plugins:maven-shade-plugin:3.2.1 or one of its dependencies could not be resolved: The following artifacts could not be resolved: org.apache.maven.shared:maven-artifact-transfer:jar:0.10.0, org.ow2.asm:asm:jar:7.0: Could not transfer artifact org.apache.maven.shared:maven-artifact-transfer:jar:0.10.0 from/to central (): Connect to [] failed: Connection timed out (Connection timed out) -> [Help 1]
@ken: This looks like a typical network problem trying to talk to the Maven central repo (nothing specific to Pinot)
@manish.bhoge: I thought so and tried couple of times ..but still same error
@ken: `Could not transfer artifact org.apache.maven.shared:maven-artifact-transfer:jar:0.10.0 from/to central (): Connect to [] failed: Connection timed out (Connection timed out)`` By any chance are your other Maven builds set up to use a proxy server for the repo?
@joshhighley: Using cli pinot-admin.sh to start the components, how do I get it to log to the individual log files -- pinotServer, pinotBroker, etc ? I see the files defined in the admin log4j config, but they don't appear to be used
@mayanks: It might be written in a different dir (say repo root)?
@santosh.rudra: @santosh.rudra has joined the channel
@xulinnankai: @xulinnankai has joined the channel
@morzaria: @morzaria has joined the channel
@suresh.k.kode: @suresh.k.kode has joined the channel

#segment-write-api

@npawar: @yupeng fyi, Kishore suggested we have separate interfaces for creation and upload. Please look at the PR again
@yupeng: sure . will take a look

#releases

@fx19880617: @fx19880617 has joined the channel
@fx19880617: @fx19880617 set the channel purpose: Any thing related to releases
@g.kishore: @g.kishore has joined the channel
@yupeng: @yupeng has joined the channel
@chinmay.cerebro: @chinmay.cerebro has joined the channel
@fx19880617: @fx19880617 set the channel topic: Team is on releasing Pinot 0.7.0. Release Manager: @yupeng
@g.kishore: let me share this in general
@fx19880617: Also put this in the welcome message/general guide. So when new users join, this channel should show up.
@g.kishore: yes
@grace.walkuski: @grace.walkuski has joined the channel
@g.kishore: @yupeng @fx19880617 can you please set topic to tentative timeline
@fx19880617: @fx19880617 set the channel topic: Team is on releasing Pinot 0.7.0. Release Manager: @yupeng, ETA: Mar 24
@fx19880617: @yupeng I put the tentative timeline to be 2 weeks after, feel free to move it
@mailtobuchi: @mailtobuchi has joined the channel
@yupeng: thats reasonable
@yupeng: will send out rc1 for vote today
@ujwala.tulshigiri: @ujwala.tulshigiri has joined the channel
@gokulrk2696: @gokulrk2696 has joined the channel
@yupeng: the vote is out for rc1
@amrish.k.lal: @amrish.k.lal has joined the channel
@jlli: @jlli has joined the channel
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]