#general
@punit.kj: @punit.kj has joined the channel
@chris.lu: @chris.lu has joined the channel
@karinwolok1: :tada: Today! Catch @fx19880617 and @g.kishore’s presentations & panel talks at PrestoCon! (conference is FREE!)
@kvndesign.mail: @kvndesign.mail has joined the channel
#random
@punit.kj: @punit.kj has joined the channel
@chris.lu: @chris.lu has joined the channel
@kvndesign.mail: @kvndesign.mail has joined the channel
#feat-presto-connector
@punit.kj: @punit.kj has joined the channel
#troubleshooting
@punit.kj: @punit.kj has joined the channel
@phuchdh: Hi guys, i need to set group.id in LLC, but i cannot find any configuration for that. In my company, we need permission for specify group.id to consume message. ```2021/03/24 06:13:18.879 WARN [ConsumerCoordinator] [RuleLogsQC__2__1__20210319T0728Z] [Consumer clientId=consumer-12735, groupId=] Synchronous auto-commit of offsets {c1.elk.db-gamification-consumer-log.qc-2=OffsetAndMetadata{offset=17073, metadata=''}} failed: Not authorized to access group:```
@chris.lu: @chris.lu has joined the channel
@ravi.maddi: Hi All,
@ravi.maddi: Hi All, I am using JSON Indexing, I have a scenario, In my persons json content, a column 'addresses' contains a list of addresses. Like this: Need Help :slightly_smiling_face: ``` { "personId": "9878", "addresses": [ { "doorNum": "45456", "Street": "Washington Road", "area": "sector-1" }, { "doorNum": "676756", "Street": "Washington Road", "area": "sector-2" }, { "doorNum": "768768", "Street": "Washington Road", "area": "sector-4" } ] } { "personId": "68768", "addresses": [ { "doorNum": "45456", "Street": "Washington Road", "area": "sector-1" }, { "doorNum": "676756", "Street": "Washington Road", "area": "sector-2" }, { "doorNum": "768768", "Street": "Washington Road", "area": "sector-4" } ] }``` In Schema config file I mentioned like this: ``` { "name": "addresses", "dataType": "STRING", "maxLength": 2147483647, "singleValueField": false },``` In Table Config file I mentioned like this: ``` "jsonIndexColumns": [ "addresses" ],``` But I am not able to find data in Pinot Query Console, I am not able to find any error in any log. Need Help
@tanmay.movva: Hello, We have setup a monitoring dashboard on grafana for pinot. We are currently using the image with `latest` tag and we observe that few metrics related to query response time have been coming as `0` . We didn’t change any config on pinot or the grafana query, so wanted to know if there was any breaking change merged recently? Here is a screen shot of the dashboard.
@tanmay.movva: Affected Metrics - ```pinot_broker_queryExecution_* pinot_server_freshnessLagMs_* pinot_server_totalQueryTime_* pinot_server_queryProcessing_*```
@g.kishore: Can you check the JMX metric names. There was a change in the metric name
@g.kishore: @fx19880617
@fx19880617: I will check, could be related to the recent metrics plugin extraction @jlli just fyi
@fx19880617: @tanmay.movva do you know which image you are using previously? Or when you restarted your deployment before this restart ?
@tanmay.movva: We are/were using the image with `latest` tag only. From the metrics dashboard, we are not able to see the metrics after 17th March(We restarted on that day).
@fx19880617: I will check on that
@tanmay.movva: > Can you check the JMX metric names. There was a change in the metric name Checked for alternate metrics for those but didn’t find any.
@fx19880617: did you see anything when you run `curl localhost:8080/jmx` in the container?
@fx19880617: also are the controller metrics there?
@tanmay.movva: > also are the controller metrics there? Yes.
@fx19880617: hmm, controller are also restarted?
@tanmay.movva: They were restarted, but not sure if the image was pulled for them. Will force a image pull for all the services once.
@jlli: I don’t think it’s related to metrics plugin extraction. The latest official release was cut based on the commit hash on 02/24 while metric plugin extraction was merged 03/08
@tanmay.movva: > hmm, controller are also restarted? Just did this, don’t think controller metrics are affected. I randomly checked few metrics and they were coming in. Any specific metrics to check for?
@tanmay.movva: > he latest official release was cut based on the commit hash on 02/24 while metric plugin extraction was merged 03/08 We are not using the officially released tag yet. We are using the `latest` tag for pinot image. Is `0.7.0` tag released?
@jlli: So the table query latency should be emitted from pinot-broker right? I think you should take a look at the health of broker metric
@tanmay.movva: I can see the QPS metrics from broker but not query latency metrics.
@tanmay.movva: And helix_connected_value, Status of broker on UI all are OK. Queries are also being processed.
@fx19880617: can u list the metrics missing, then I can check
@tanmay.movva: Hmm..Will try to get list of all metrics, but from the panels I setup in the dashboard, these are the metrics missing currently.
@fx19880617: hmm, I tried latest image and found that those metrics are there: ```root@pinot-server-0:/opt/pinot# curl localhost:8080/jmx|grep freshnessLagMs pinot_server_freshnessLagMs_MeanRate Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>MeanRate) # TYPE pinot_server_freshnessLagMs_MeanRate untyped pinot_server_freshnessLagMs_MeanRate{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_MeanRate{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_OneMinuteRate Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>OneMinuteRate) # TYPE pinot_server_freshnessLagMs_OneMinuteRate untyped pinot_server_freshnessLagMs_OneMinuteRate{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_OneMinuteRate{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_StdDev Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>StdDev) # TYPE pinot_server_freshnessLagMs_StdDev untyped pinot_server_freshnessLagMs_StdDev{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_StdDev{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_95thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>95thPercentile) # TYPE pinot_server_freshnessLagMs_95thPercentile untyped pinot_server_freshnessLagMs_95thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_95thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_99thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>99thPercentile) # TYPE pinot_server_freshnessLagMs_99thPercentile untyped pinot_server_freshnessLagMs_99thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_99thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_Mean Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>Mean) # TYPE pinot_server_freshnessLagMs_Mean untyped pinot_server_freshnessLagMs_Mean{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_Mean{table="airlineStats",tableType="REALTIME",} 0.0 100 180k 100 180k 0 0 204k # HELP pinot_server_freshnessLagMs_FifteenMinuteRate Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>FifteenMinuteRate) # TYPE pinot_server_freshnessLagMs_FifteenMinuteRate untyped 0 pinot_server_freshnessLagMs_FifteenMinuteRate{table="airlineStatsAvro",tableType="REALTIME",} 0.0 -pinot_server_freshnessLagMs_FifteenMinuteRate{table="airlineStats",tableType="REALTIME",} 0.0 -# HELP pinot_server_freshnessLagMs_Min Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>Min) :-# TYPE pinot_server_freshnessLagMs_Min untyped -:pinot_server_freshnessLagMs_Min{table="airlineStatsAvro",tableType="REALTIME",} 0.0 -pinot_server_freshnessLagMs_Min{table="airlineStats",tableType="REALTIME",} 0.0 - -# HELP pinot_server_freshnessLagMs_Count Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>Count) -:--:-- --:# TYPE pinot_server_freshnessLagMs_Count untyped --pinot_server_freshnessLagMs_Count{table="airlineStatsAvro",tableType="REALTIME",} 0.0 :pinot_server_freshnessLagMs_Count{table="airlineStats",tableType="REALTIME",} 0.0 -- # HELP pinot_server_freshnessLagMs_999thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>999thPercentile) 2# TYPE pinot_server_freshnessLagMs_999thPercentile untyped 04pinot_server_freshnessLagMs_999thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 k pinot_server_freshnessLagMs_999thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_50thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>50thPercentile) # TYPE pinot_server_freshnessLagMs_50thPercentile untyped pinot_server_freshnessLagMs_50thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_50thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_75thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>75thPercentile) # TYPE pinot_server_freshnessLagMs_75thPercentile untyped pinot_server_freshnessLagMs_75thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_75thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_FiveMinuteRate Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>FiveMinuteRate) # TYPE pinot_server_freshnessLagMs_FiveMinuteRate untyped pinot_server_freshnessLagMs_FiveMinuteRate{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_FiveMinuteRate{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_98thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>98thPercentile) # TYPE pinot_server_freshnessLagMs_98thPercentile untyped pinot_server_freshnessLagMs_98thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_98thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 # HELP pinot_server_freshnessLagMs_Max Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.airlineStatsAvro_REALTIME.freshnessLagMs"><>Max) # TYPE pinot_server_freshnessLagMs_Max untyped pinot_server_freshnessLagMs_Max{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_freshnessLagMs_Max{table="airlineStats",tableType="REALTIME",} 0.0```
@fx19880617: ```root@pinot-server-0:/opt/pinot# curl localhost:8080/jmx|grep totalQueryTime pinot_server_totalQueryTime_Count Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>Count) # TYPE pinot_server_totalQueryTime_Count untyped pinot_server_totalQueryTime_Count{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Count{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Count{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Count{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_Count{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_Count{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_Count{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_FifteenMinuteRate Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>FifteenMinuteRate) # TYPE pinot_server_totalQueryTime_FifteenMinuteRate untyped pinot_server_totalQueryTime_FifteenMinuteRate{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_FifteenMinuteRate{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_FifteenMinuteRate{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_FifteenMinuteRate{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_FifteenMinuteRate{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_FifteenMinuteRate{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_FifteenMinuteRate{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_Max Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>Max) # TYPE pinot_server_totalQueryTime_Max untyped pinot_server_totalQueryTime_Max{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Max{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Max{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Max{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_Max{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_Max{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_Max{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_50thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>50thPercentile) # TYPE pinot_server_totalQueryTime_50thPercentile untyped pinot_server_totalQueryTime_50thPercentile{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_50thPercentile{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_50thPercentile{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_50thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_50thPercentile{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_50thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_50thPercentile{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_98thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>98thPercentile) # TYPE pinot_server_totalQueryTime_98thPercentile untyped pinot_server_totalQueryTime_98thPercentile{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_98thPercentile{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_98thPercentile{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_98thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_98thPercentile{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_98thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_98thPercentile{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_75thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>75thPercentile) # TYPE pinot_server_totalQueryTime_75thPercentile untyped pinot_server_totalQueryTime_75thPercentile{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_75thPercentile{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_75thPercentile{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_75thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_75thPercentile{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_75thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_75thPercentile{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_FiveMinuteRate Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>FiveMinuteRate) # TYPE pinot_server_totalQueryTime_FiveMinuteRate untyped pinot_server_totalQueryTime_FiveMinuteRate{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_FiveMinuteRate{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_FiveMinuteRate{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_FiveMinuteRate{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_FiveMinuteRate{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_FiveMinuteRate{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_FiveMinuteRate{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_StdDev Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>StdDev) # TYPE pinot_server_totalQueryTime_StdDev untyped pinot_server_totalQueryTime_StdDev{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_StdDev{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_StdDev{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_StdDev{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_StdDev{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_StdDev{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_StdDev{table="complexWebsite",tableType="OFFLINE",} 0.0 100 180k 100 180k 0 0 204k 0 --:--:-- --:--:-- --:--:-- 204k # HELP pinot_server_totalQueryTime_OneMinuteRate Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>OneMinuteRate) # TYPE pinot_server_totalQueryTime_OneMinuteRate untyped pinot_server_totalQueryTime_OneMinuteRate{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_OneMinuteRate{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_OneMinuteRate{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_OneMinuteRate{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_OneMinuteRate{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_OneMinuteRate{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_OneMinuteRate{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_Min Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>Min) # TYPE pinot_server_totalQueryTime_Min untyped pinot_server_totalQueryTime_Min{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Min{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Min{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Min{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_Min{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_Min{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_Min{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_Mean Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>Mean) # TYPE pinot_server_totalQueryTime_Mean untyped pinot_server_totalQueryTime_Mean{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Mean{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Mean{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_Mean{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_Mean{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_Mean{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_Mean{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_MeanRate Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>MeanRate) # TYPE pinot_server_totalQueryTime_MeanRate untyped pinot_server_totalQueryTime_MeanRate{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_MeanRate{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_MeanRate{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_MeanRate{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_MeanRate{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_MeanRate{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_MeanRate{table="complexWebsite",tableType="OFFLINE",} 0.0 # HELP pinot_server_totalQueryTime_95thPercentile Attribute exposed for management ("org.apache.pinot.common.metrics"<type="ServerMetrics", name="pinot.server.simpleWebsite_OFFLINE.totalQueryTime"><>95thPercentile) # TYPE pinot_server_totalQueryTime_95thPercentile untyped pinot_server_totalQueryTime_95thPercentile{table="simpleWebsite",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_95thPercentile{table="covid19_confirmed_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_95thPercentile{table="covid19_deaths_us",tableType="OFFLINE",} 0.0 pinot_server_totalQueryTime_95thPercentile{table="airlineStatsAvro",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_95thPercentile{table="covid19_confirmed",tableType="us",} 0.0 pinot_server_totalQueryTime_95thPercentile{table="airlineStats",tableType="REALTIME",} 0.0 pinot_server_totalQueryTime_95thPercentile{table="complexWebsite",tableType="OFFLINE",} 0.0```
@fx19880617: can you check if the jmx server are reporting those metrics?
@fx19880617: also check if your prometheus metrics exporter are deployed
@tanmay.movva: Those metrics are coming in. But they are coming in as `0` . They aren’t being updated. Even the metrics you shared have zero value. But when queries are running then execution Time shouldn’t be zero no?
@tanmay.movva: My problem was not that they were missing, but the value of those metrics is always zero. Which wasn’t the case earlier.
@fx19880617: i see, i will double check
@chxing: Hi All, I got an error Caused by:
@dlavoie: the prometheus exporter is not a client. It’s a server. so when you define `jmx_prometheus_javaagent-0.12.0.jar=9066`, it opens a port 9066 on your Pinot process for Prometheus to scrape metrics.
@dlavoie: This configuration is not for your prometheus server port. You need a distinct port. and configure a prometheus scrape config that reference this pinot host and port.
@chxing: Thx, I can config different ports in prometheus to collect those metrics
@dlavoie: Yes
@dlavoie: It,s all part of your prometheus scrape config
@dlavoie: If everything is running on one node, you will need distinct port on the javaagent config for each pinot component
@chxing: yes I just changed <
@prshnt.1314: Hi all, I'm trying to connect presto to pinot cluster using connector. But I'm facing this null pointer exception. Has anyone else faced this or give some hints to solve this.
@joshhighley: When ingesting json data, do the json objects have to be flat? I can't find docs about parsing json object trees, but I see the github event example is a tree
@mayanks: It depends on what you are trying to do: ```1. If you want to store data as JSON blob in Pinot and use JSON indexing:
@joshhighley: #2. The row data is several levels deep, and it's not an array but rather a list of id properties { node1: { node2: { node3: { id1: { col1: value1, col2: value2 }, id2: { col1: value1, col2: value2 } } } }
@mayanks: You can write a transform to extract that data and map to a column in Pinot schema
@mayanks: List can map to multi-valued columns
@joshhighley: sorry, hit send before my example was finished. id1 is a row, id2 is a row, etc
@mayanks: You should still be able to write per column transform (would be a little inefficient to do it once per column)?
@joshhighley: How does Pinot decide what the row level object is? ex: id1, id2, etc vs some other object
@mayanks: Your transform function would need to decide that
@kvndesign.mail: @kvndesign.mail has joined the channel
@joshhighley: The server hosting our Pinot got rebooted and when we restarted the components, we have no segments, no tables and no schemas. I see in the server logs that it's looking in the correct directories for instance data and instance segment, and I've confirmed there's data in those directories. Pinot server isn't loading anything though?
@mayanks: What do you see in the ideal-state? Does it claim that the server should host data?
@joshhighley: where do I check that?
@mayanks: In the cluster manager or in zookeeper (via query console)
@mayanks: Ideal-state reflects which segments are "expected" to be on which servers.
@mayanks: External-view reflects the current state of the system.
@mayanks: Let's first see what Ideal-State says. If it claims that the server should host segments, then we check external view.
@joshhighley: those only have 'brokerResource' and 'leadControllerResource'
@mayanks: Hmm, nothing for the table?
@joshhighley: under idealStates? no
@mayanks: Does cluster manager show any tables tab?
@mayanks: or the rest-api `<controllerHost>:<controllerPort>/tables`?
@mayanks: If none of them show the table, then there is no table :grimacing:
@joshhighley: ugh. pinot-admin.sh doesn't log correctly so we don't even have log files
@mayanks: I think it should log correctly, unless you have overwritten log4j configs?
@mayanks: If you still feel it is not doing so, please file an issue and we will address it.
@jackie.jxt: If only these 2 records exist in ideal state, you need to check if zookeeper somehow gets all the contents deleted during the reboot
#pinot-dev
@punit.kj: @punit.kj has joined the channel
#pinot-startup
@ravi.maddi: Hi All, I am using JSON Indexing, I have a scenario, In my persons json content, a column 'addresses' contains a list of addresses. Like this: Need Help :slightly_smiling_face: ``` { "personId": "9878", "addresses": [ { "doorNum": "45456", "Street": "Washington Road", "area": "sector-1" }, { "doorNum": "676756", "Street": "Washington Road", "area": "sector-2" }, { "doorNum": "768768", "Street": "Washington Road", "area": "sector-4" } ] } { "personId": "68768", "addresses": [ { "doorNum": "45456", "Street": "Washington Road", "area": "sector-1" }, { "doorNum": "676756", "Street": "Washington Road", "area": "sector-2" }, { "doorNum": "768768", "Street": "Washington Road", "area": "sector-4" } ] }``` In Schema config file I mentioned like this: ``` { "name": "addresses", "dataType": "STRING", "maxLength": 2147483647, "singleValueField": false },``` In Table Config file I mentioned like this: ``` "jsonIndexColumns": [ "addresses" ],``` But I am not able to find data in Pinot Query Console, I am not able to find any error in any log. Need Help @mayanks Can you help me.
@mayanks: Yes will look at in shortly
@mayanks: What about select * ? Does it return anything
@mayanks: If not what is the error you get
@ravi.maddi: select * giving zero records.
@mayanks: no errors?
@ravi.maddi: no errors
@mayanks: also can you check external view from console to see if there are any segments?
@mayanks: We want to check if this is an issue because segment is not loaded in Pinot, or if the segment is not generated correctly
@mayanks: If you have access to the segment directory, can you post the metadata.properties file inside of the segment dir?
@ravi.maddi: Ok, I will check
@ravi.maddi: I am using quick start streaming to start my cluster.
@mayanks: can we do a zoom call now?
@ravi.maddi: I am not able to find any file with name metadata.properties
@mayanks: easier to debug that way
@ravi.maddi: Yes Mayank
@mayanks: let me send a link
@ravi.maddi: ok
@mayanks: ```Mayank Shrivastava is inviting you to a scheduled Zoom meeting. Topic: Mayank Shrivastava's Personal Meeting Room Join Zoom Meeting
@mayanks: ok i have logged in ^^
@mayanks: are you joining @ravi.maddi?
@ravi.maddi: joined
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pinot.apache.org For additional commands, e-mail: dev-h...@pinot.apache.org