#general
@wentjin: @wentjin has joined the channel
@syedakram93: Do we have any Doc to use Hdfs as deepstore?
@xiangfu0:
@31cswarszawa: @31cswarszawa has joined the channel
@karinwolok1: Starting in 15 minutes!
@fangsheng.ferris.li: @fangsheng.ferris.li has joined the channel
@elon.azoulay: Thanks to @jackie.jxt and @ssubrama for the great presentation - this was so informative!
@ssubrama: you are welcome Elon, glad we could share our ideas. A silver lining in the pandemic is that we can do so from home, with zero commute :slightly_smiling_face:
@karinwolok1: Great job today, @ssubrama and @jackie.jxt!!! Thank you for taking the time!
@chxing: Hi all I found pinot 0.8.0 has been released, Do we have released package for download? thx
@kulbir.nijjer: @chxing did you see a release announcement somewhere ? I only see 0.7.x listed here as latest:
@mayanks: I think it was still under vote until yesterday
@chxing: Ok thx, I only found the released version from github
#random
@wentjin: @wentjin has joined the channel
@31cswarszawa: @31cswarszawa has joined the channel
@fangsheng.ferris.li: @fangsheng.ferris.li has joined the channel
#troubleshooting
@wentjin: @wentjin has joined the channel
@31cswarszawa: @31cswarszawa has joined the channel
@31cswarszawa: Hey everyone! I’m trying to build a streaming architecture that consists of kafka and Pinot. Here’s my `docker-compose`: ```services: web: image: apachepinot/pinot:latest command: QuickStart -type hybrid container_name: "pinot" volumes: - ./config:/config ports: - "9000:9000" # Controller - "9001:8000" # Broker - "9002:7000" # Server``` Is `kafka` included as a part of this image? If so, how can I reach it (which port)? Finally, how can I create a topic and publish/subscribe to it? thanks for your help!
@dlavoie: A while ago I blogged about a monitoring setup for Apache Pinot. The provided example contains a fully executable docker compose setup with Kafka and Pinot and even a smoke test app
@dlavoie:
@31cswarszawa: @dlavoie Thanks! Looking at the `docker-compose` file in the repo you shared, I can see that you separate all Pinot pieces + zookeeper + kafka. Hovewer, in the compose file I shared, they are a part of one image. My question is: which solution is officially suggested by Pinot devs?
@dlavoie: Kafka is not packaged in the Pinot image
@dlavoie: Quickstart mode is good for local testing. Standalone component is more scalable and a better production practice.
@31cswarszawa: So do I understand correctly that the best practice for testing locally (kafka + pinot) would be not to use Quickstart mode and follow your tutorial?
@dlavoie: Just use whatever works for you :slightly_smiling_face:
@dlavoie: The example I shared has details to setup kafka in your docker compose
@dlavoie: You can take the kafka stuff and integrate it in your quickstart docker compose
@dlavoie: Kafka will be recheable locally from `localhost:9092`, and from `kafka:19092` within the pinot container.
@31cswarszawa: Great, thanks a lot for your help!
@syedakram93: I've table created with 1 segment locally configured controller... after that i configured deepstore as HDFS and moved controller data of that table 'T' to HDFS deepstore and restarted controller & server... after that query is not serving for that table.
@syedakram93: Table status is Good
@syedakram93: and it show proper metadata and other things
@syedakram93: @xiangfu0
@syedakram93: only while querying for the migrated controller data from local to hdfs has a problem...
@syedakram93: can someone help?
@syedakram93: Table - gitHubEvents
@xiangfu0: what’s segment metadata?
@xiangfu0: can you check download URL and modify segment metadata to change it to hdfs
@syedakram93: How to check download url
@xiangfu0: click on segment name
@xiangfu0: from ui
@xiangfu0: download uri won’t change when you change the deep store, it’s only for new uploaded segments
@xiangfu0: for existing segments, you need to modify the download uri right now
@xiangfu0: We have one issue tracking this:
@xiangfu0: with all the detailed steps
@fangsheng.ferris.li: @fangsheng.ferris.li has joined the channel
@jai.patel856: We’re currently on Pinot 0.6 and we’re getting a ton of warnings when ingesting a new PInot Upsert table. I haven’t seen this particular error before. Is there something I should be looking for? ```2021/08/18 00:35:45.093 WARN [LLRealtimeSegmentDataManager_enriched_station_orders_v1_14_rt_upsert_v2_1__7__213__20210818T0025Z] [enriched_customer_orders_v1_15_1_upsert__6__26__20210817T0050Z] Commit failed with response {"offset":-1,"streamPartitionMsgOffset":null,"status":"FAILED","isSplitCommitType":false,"buildTimeSec":-1} 2021/08/18 00:35:46.258 WARN [LLRealtimeSegmentDataManager_enriched_station_orders_v1_14_rt_upsert_v2_1__7__213__20210818T0025Z] [enriched_station_orders_v1_14_rt_upsert_v2_1__0__219__20210816T2149Z] Commit failed with response {"offset":-1,"streamPartitionMsgOffset":null,"status":"FAILED","isSplitCommitType":false,"buildTimeSec":-1} 2021/08/18 00:35:46.311 WARN [LLRealtimeSegmentDataManager_enriched_station_orders_v1_14_rt_upsert_v2_1__7__213__20210818T0025Z] [enriched_station_orders_v1_14_rt_upsert_v2_1__10__218__20210817T0010Z] Commit failed with response {"offset":-1,"streamPartitionMsgOffset":null,"status":"FAILED","isSplitCommitType":false,"buildTimeSec":-1}``` enriched_station_orders_v1_15_1_upsert is a new table, but it seems to be impacting older tables as well.
@ssubrama: can u look for the segment name in the controller logs?
@jai.patel856: Hrm….
@jai.patel856: ```2021/08/18 00:53:29.740 ERROR [LLCSegmentCompletionHandlers] [grizzly-http-server-4] Caught exception while committing segment: enriched_customer_orders_v1_15_1_upsert__5__16__20210817T0128Z from instance: Server_pinot-upsert-server-zonal-2.pinot-upsert-server-headless.pinot-dev.svc.cluster.local_8098 java.io.IOException: No space left on device at sun.nio.ch.FileDispatcherImpl.write0(Native Method) ~[?:?] at sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:62) ~[?:?] at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113) ~[?:?] at sun.nio.ch.IOUtil.write(IOUtil.java:79) ~[?:?] at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280) ~[?:?] at java.nio.channels.Channels.writeFullyImpl(Channels.java:74) ~[?:?] at java.nio.channels.Channels.writeFully(Channels.java:97) ~[?:?] at java.nio.channels.Channels$1.write(Channels.java:172) ~[?:?] at java.io.InputStream.transferTo(InputStream.java:705) ~[?:?] at java.nio.file.Files.copy(Files.java:3078) ~[?:?] at org.apache.pinot.controller.api.resources.LLCSegmentCompletionHandlers.extractSegmentFromFormToLocalTempFile(LLCSegmentCompletionHandlers.java:445) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.apache.pinot.controller.api.resources.LLCSegmentCompletionHandlers.segmentCommit(LLCSegmentCompletionHandlers.java:289) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at jdk.internal.reflect.GeneratedMethodAccessor330.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:679) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:353) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549) ~[pinot-all-0.6.0-jar-with-dependencies.jar:0.6.0-8f7da20c24e77e292cdf89cef7f1aa653c9a35f3] at java.lang.Thread.run(Thread.java:834) [?:?]```
@jai.patel856: is this out of disk space on the controller or on a server?
@ssubrama: out of space on the controller. As segments complete, they are deposited on the controller. I think you may be running out of temporary space, or (depending on how you have your segment store configured) segment storage space
@ssubrama: I think it is local temporary storage space
@ssubrama: on the controller.
@jai.patel856: okay, thanks for the pointer, I’ll dig in more there.
#pinot-dev
@mayanks: Hi Team, do we have any talks scheduled (or interest for talks) on Pinot scheduled for ApacheCon happening in Sept?
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
