#general
@karinwolok1: Hey All! Get ready for some incredible speakers coming your way at
@xiangfu0: set the channel topic: Welcome to the community! Please show your support by starring our project repository on GitHub
@karinwolok1: Meetup next week! :smile: By LinkedIn engineers on the Apache Pinot team! @snlee (PMC) and @jiapengtao0
@weixiang.sun: Hi Pinot team, I am trying to create a realtime Pinot table ingesting the data from Kafka topic. ```1. The Kafka stream data has two time columns: processed_at and created_at. 2. The processed_at column is in-order inside Kafka stream. 3. The created_at is out-of-order inside Kafka stream``` The retention of realtime pinot table is depending on created_at. If we want to use created_at as timeColumnName, since created_at can be very old, a lot of stale segments can be created. If we want to use processed_at as timeColumnName, a lot of old orders can live in the realtime table. Do you guys have any suggestion about which one to choose as timeColumnName?
#pinot-power-bi
@salkadam: @salkadam has joined the channel
#troubleshooting
@nageshblore: I am trying to get Pinot running on Kubernetes. After installing with Helm, there is this note. *Where* is the `StorageClass` specified? This is on bare metal, btw. > Please specify *StorageClass* based on your cloud vendor.
@xiangfu0: For your bare metal, do you internally have any storage class support for remote data store ?
@xiangfu0: Or do you have deep store like hdfs etc?
@mapshen: When we run Trino with Pinot 0.8.0, we get the error `unsupported data table version: 3`. It seems Trino still uses Pinot client 0.6.0. Do we have plans on upgrade it to 0.8.0? If it is as simple as bumping up the version, I can help raise a PR.
@mayanks: @elon.azoulay is it already in the works?
@elon.azoulay: Yes, we already have a pr being actively reviewed and we deployed it. Also we have support for a lot of the newer features: lookup join, groovy functions, grpc streaming, passing query parameters, dimension table support (insert and create table) - will document all the features.
@mapshen: thats great to hear! Can you point me to the PR? Any ETA on get that merged and released?
@bowenwan: Hi, I am trying to do batch ingestion via Spark to a OFFLINE table. It often crashed the broker when the data is large enough. From what I can tell, larger memory allocated to broker can help. But Is there any documents about how to calculate best settings about what should be right memory allocation, disk size, etc for different type of nodes ? For example for 1T data ?
#aggregators
@salkadam: @salkadam has joined the channel
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pinot.apache.org For additional commands, e-mail: dev-h...@pinot.apache.org
