codecov-commenter commented on PR #11782:
URL: https://github.com/apache/pinot/pull/11782#issuecomment-1756957767
##
[Codecov](https://app.codecov.io/gh/apache/pinot/pull/11782?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
R
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 10055ffc39 free up more disk for trivy job (#117
xiangfu0 merged PR #11780:
URL: https://github.com/apache/pinot/pull/11780
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apa
xiangfu0 opened a new issue, #11783:
URL: https://github.com/apache/pinot/issues/11783
https://github.com/apache/pinot/actions/runs/6478269923/job/17589792539?pr=11780
```
2023-10-11T05:42:25.8353888Z [ERROR]
org.apache.pinot.controller.api.TableViewsTest.setUp -- Time elapsed: 7.787
piby180 commented on issue #11761:
URL: https://github.com/apache/pinot/issues/11761#issuecomment-1756909824
> > What do you think about downgrading the AWS-SDK version? (only in the
environment with the problem, not in the official release of course)
>
> Just trying to understand the
saurabhd336 commented on PR #11729:
URL: https://github.com/apache/pinot/pull/11729#issuecomment-1756851682
@Jackie-Jiang I added some code example on how to use these new APIs in the
description
--
This is an automated message from the Apache Git Service.
To respond to the message, pleas
saurabhd336 commented on code in PR #11729:
URL: https://github.com/apache/pinot/pull/11729#discussion_r1354125407
##
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/reader/ForwardIndexReader.java:
##
@@ -916,4 +919,67 @@ default byte[][] getBytesMV(int docId,
xiangfu0 opened a new pull request, #11782:
URL: https://github.com/apache/pinot/pull/11782
- Add new API in ZKMetadataProvider for
`setTableConfig(ZkHelixPropertyStore propertyStore, String
tableNameWithType, TableConfig tableConfig, int expectVersion)` with previous
version.
- Update
saurabhd336 commented on code in PR #11729:
URL: https://github.com/apache/pinot/pull/11729#discussion_r1354080442
##
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/forward/FixedByteChunkMVForwardIndexReader.java:
##
@@ -195,6 +220,38 @@ p
saurabhd336 commented on code in PR #11729:
URL: https://github.com/apache/pinot/pull/11729#discussion_r1353988984
##
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/reader/ForwardIndexReader.java:
##
@@ -916,4 +919,67 @@ default byte[][] getBytesMV(int docId,
saurabhd336 commented on code in PR #11729:
URL: https://github.com/apache/pinot/pull/11729#discussion_r1353988772
##
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/reader/ForwardIndexReader.java:
##
@@ -916,4 +919,67 @@ default byte[][] getBytesMV(int docId,
itschrispeck commented on code in PR #11739:
URL: https://github.com/apache/pinot/pull/11739#discussion_r1353978315
##
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/json/ImmutableJsonIndexReader.java:
##
@@ -76,6 +80,67 @@ public Immutabl
haowanv commented on issue #11764:
URL: https://github.com/apache/pinot/issues/11764#issuecomment-1756620727
I deployed pinot by helmfile sync command, and all pods are as below:
```
pinot-broker-0 1/1 Running 0
19m
pinot-controller
JasonLeeWeiHern commented on issue #11772:
URL: https://github.com/apache/pinot/issues/11772#issuecomment-1756602361
Yes okay.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific commen
Jackie-Jiang commented on issue #11658:
URL: https://github.com/apache/pinot/issues/11658#issuecomment-1756553220
In that case how much code do you think can be reused? Currently
`ConcurrentMapPartitionUpsertMetadataManager` heavily rely on
`ConcurrentHashMap.compute()`, and TBH I don't thi
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pinot.git
The following commit(s) were added to refs/heads/master by this push:
new 169aba9fc9 Fix schema name in table config durin
xiangfu0 merged PR #11574:
URL: https://github.com/apache/pinot/pull/11574
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apa
Jackie-Jiang commented on issue #11772:
URL: https://github.com/apache/pinot/issues/11772#issuecomment-1756543081
Do you want to help contribute this feature?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL abo
Jackie-Jiang commented on issue #11764:
URL: https://github.com/apache/pinot/issues/11764#issuecomment-1756542167
I don't think the red box is related because all the exceptions are caught
here. From the above log, seems like controller is actually shut down
gracefully (i.e. not crash). Can
This is an automated email from the ASF dual-hosted git repository.
snlee pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pinot.git
The following commit(s) were added to refs/heads/master by this push:
new cdd16fef4d Prevent NPE when attempt to fetch parti
snleee merged PR #11769:
URL: https://github.com/apache/pinot/pull/11769
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apach
Jackie-Jiang commented on code in PR #11574:
URL: https://github.com/apache/pinot/pull/11574#discussion_r1353669798
##
pinot-common/src/main/java/org/apache/pinot/common/metadata/ZKMetadataProvider.java:
##
@@ -79,6 +80,26 @@ public static void
setTableConfig(ZkHelixPropertySto
Jackie-Jiang commented on code in PR #11780:
URL: https://github.com/apache/pinot/pull/11780#discussion_r1353664741
##
.github/workflows/scripts/.pinot_vuln_check.sh:
##
@@ -39,3 +39,5 @@ docker build \
.
docker image ls
+docker image prune --all --filter "until=1h" -f
dang-stripe opened a new issue, #11781:
URL: https://github.com/apache/pinot/issues/11781
Per https://docs.pinot.apache.org/users/tutorials/schema-evolution, most
schema changes aside from adding a column are not supported. In order to be
able to rollback to a previous schema, we want to be
codecov-commenter commented on PR #11780:
URL: https://github.com/apache/pinot/pull/11780#issuecomment-1756432521
##
[Codecov](https://app.codecov.io/gh/apache/pinot/pull/11780?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
R
walterddr commented on code in PR #11746:
URL: https://github.com/apache/pinot/pull/11746#discussion_r1353601990
##
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MailboxReceiveOperator.java:
##
@@ -42,6 +42,10 @@ public String toExplainString() {
walterddr commented on code in PR #11746:
URL: https://github.com/apache/pinot/pull/11746#discussion_r1353597355
##
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/exchange/BlockExchange.java:
##
@@ -68,25 +67,29 @@ protected BlockExchange(List
sending
itschrispeck commented on PR #11739:
URL: https://github.com/apache/pinot/pull/11739#issuecomment-1756396539
Thanks for the review and suggestions! Will update w/ perf numbers.
For `json_match` filter are you referring to some query like `where
json_match(col, '$.keyA = ''val''') AND
itschrispeck commented on code in PR #11739:
URL: https://github.com/apache/pinot/pull/11739#discussion_r1353529541
##
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/reader/JsonIndexReader.java:
##
@@ -31,4 +33,13 @@ public interface JsonIndexReader extends I
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pinot.git
from 3a9f95f469 Fix PeerServerSegmentFinder not respecting HTTPS port
(#11752)
add 17ffa3e305 Capture build scans on
xiangfu0 merged PR #11767:
URL: https://github.com/apache/pinot/pull/11767
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apa
Jackie-Jiang commented on PR #11739:
URL: https://github.com/apache/pinot/pull/11739#issuecomment-1756348232
Can you share some new perf numbers after the optimization? In the query,
let's add the `json_match` in the filter
--
This is an automated message from the Apache Git Service.
To r
Jackie-Jiang commented on code in PR #11739:
URL: https://github.com/apache/pinot/pull/11739#discussion_r1353494155
##
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/json/ImmutableJsonIndexReader.java:
##
@@ -76,6 +80,67 @@ public Immutabl
xiangfu0 opened a new pull request, #11780:
URL: https://github.com/apache/pinot/pull/11780
Trying to free up more disk space for trivy job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the spe
snleee commented on issue #11580:
URL: https://github.com/apache/pinot/issues/11580#issuecomment-1756325341
1. `$partitionValues` are the ones partition id of the segment. This
information is already getting populated to segment metadata as
`partitionValues` when the customer partition func
jadami10 commented on PR #11768:
URL: https://github.com/apache/pinot/pull/11768#issuecomment-1756257942
> Do we return the job id when the job finishes immediately (IMO that is not
really needed)? If we want to track all the jobs, we also need to update the
status for dry-run etc (basicall
Jackie-Jiang commented on PR #11771:
URL: https://github.com/apache/pinot/pull/11771#issuecomment-1756239578
Please fix the import
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific com
Jackie-Jiang commented on code in PR #11729:
URL: https://github.com/apache/pinot/pull/11729#discussion_r1353373261
##
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/reader/ForwardIndexReader.java:
##
@@ -916,4 +919,67 @@ default byte[][] getBytesMV(int docId
Jackie-Jiang commented on PR #11768:
URL: https://github.com/apache/pinot/pull/11768#issuecomment-1756173067
cc @klsince to also take a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the sp
Jackie-Jiang commented on PR #11768:
URL: https://github.com/apache/pinot/pull/11768#issuecomment-1756172808
Currently we only track the rebalance status when it returns `IN_PROGRESS`
(execute in background), and not track the rebalance status when it finishes
immediately (`NO_OP` or `DONE`
davizucon commented on issue #11761:
URL: https://github.com/apache/pinot/issues/11761#issuecomment-1756130086
Thank you for your help !
Sure, in the tests I took, the credential comes from here:
awsCredentialsProvider = DefaultCredentialsProvider.create(); not sure if I set
AccessKey
swaminathanmanish commented on issue #11761:
URL: https://github.com/apache/pinot/issues/11761#issuecomment-1756074153
> What do you think about downgrading the AWS-SDK version? (only in the
environment with the problem, not in the official release of course)
Just to understand the sc
klsince commented on code in PR #11740:
URL: https://github.com/apache/pinot/pull/11740#discussion_r1353180549
##
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/rebalance/RebalanceConfig.java:
##
@@ -84,6 +84,24 @@ public class RebalanceConfig {
@ApiMod
itschrispeck commented on code in PR #11739:
URL: https://github.com/apache/pinot/pull/11739#discussion_r1353133600
##
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/json/ImmutableJsonIndexReader.java:
##
@@ -76,6 +77,63 @@ public Immutabl
davizucon commented on issue #11761:
URL: https://github.com/apache/pinot/issues/11761#issuecomment-1756001216
I got this error too, what you think to downgrade aws sdk to work around
this error?
--
This is an automated message from the Apache Git Service.
To respond to the message, plea
rohityadav1993 commented on PR #11584:
URL: https://github.com/apache/pinot/pull/11584#issuecomment-1755999768
> IIUC, `LazyRow` is introduced to avoid reading the same value from the
segment multiple times. I think it is a good abstraction (suggestion making it
a wrapper over `IndexSegment
cbalci commented on code in PR #11774:
URL: https://github.com/apache/pinot/pull/11774#discussion_r1353024718
##
pinot-core/src/main/java/org/apache/pinot/core/common/ObjectSerDeUtils.java:
##
@@ -515,23 +518,23 @@ public VarianceTuple deserialize(ByteBuffer byteBuffer) {
}
t0mpere commented on code in PR #11744:
URL: https://github.com/apache/pinot/pull/11744#discussion_r1352891714
##
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/StarTreeIndexConfig.java:
##
@@ -34,6 +35,8 @@ public class StarTreeIndexConfig extends BaseJsonConfig {
rajagopr commented on code in PR #11769:
URL: https://github.com/apache/pinot/pull/11769#discussion_r1353051133
##
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaStreamMetadataProvider.java:
##
@@ -57,7 +60,11 @@ pu
codecov-commenter commented on PR #11779:
URL: https://github.com/apache/pinot/pull/11779#issuecomment-1755927451
##
[Codecov](https://app.codecov.io/gh/apache/pinot/pull/11779?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
R
Jackie-Jiang commented on code in PR #11574:
URL: https://github.com/apache/pinot/pull/11574#discussion_r1353040724
##
pinot-controller/src/main/java/org/apache/pinot/controller/BaseControllerStarter.java:
##
@@ -549,6 +556,83 @@ protected void configure() {
_serviceStatus
codecov-commenter commented on PR #11778:
URL: https://github.com/apache/pinot/pull/11778#issuecomment-1755916275
##
[Codecov](https://app.codecov.io/gh/apache/pinot/pull/11778?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
R
Jackie-Jiang commented on PR #11584:
URL: https://github.com/apache/pinot/pull/11584#issuecomment-1755887036
IIUC, `LazyRow` is introduced to avoid reading the same value from the
segment multiple times. I think it is a good abstraction (suggestion making it
a wrapper over `IndexSegment` an
tibrewalpratik17 opened a new pull request, #11779:
URL: https://github.com/apache/pinot/pull/11779
labels:
- feature
Created this draft PR to just give a design view of supporting retention for
deleted upsert primary keys which we are discussing in #11736.
TODO:
- Add co
Jackie-Jiang merged PR #11752:
URL: https://github.com/apache/pinot/pull/11752
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot
This is an automated email from the ASF dual-hosted git repository.
jackie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/pinot.git
from 24af80d3bc more geospatial v2 integration tests (#11741)
add 3a9f95f469 Fix PeerServerSegmentFinder not respecting
soumitra-st opened a new pull request, #11778:
URL: https://github.com/apache/pinot/pull/11778
GenerateData command is an existing utility to generate random dataset based
on a schema. This PR adds support for writing data in JSON format in addition
to AVRO and CSV.
`feature`
--
T
t0mpere commented on code in PR #11744:
URL: https://github.com/apache/pinot/pull/11744#discussion_r1352891714
##
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/StarTreeIndexConfig.java:
##
@@ -34,6 +35,8 @@ public class StarTreeIndexConfig extends BaseJsonConfig {
snleee commented on code in PR #11769:
URL: https://github.com/apache/pinot/pull/11769#discussion_r1352882744
##
pinot-plugins/pinot-stream-ingestion/pinot-kafka-2.0/src/main/java/org/apache/pinot/plugin/stream/kafka20/KafkaStreamMetadataProvider.java:
##
@@ -57,7 +60,11 @@ publ
walterddr commented on code in PR #11763:
URL: https://github.com/apache/pinot/pull/11763#discussion_r1352797541
##
pinot-spi/src/main/java/org/apache/pinot/spi/utils/BooleanUtils.java:
##
@@ -64,6 +64,13 @@ public static int toInt(String booleanString) {
return toBoolean(b
codecov-commenter commented on PR #11776:
URL: https://github.com/apache/pinot/pull/11776#issuecomment-1755710347
##
[Codecov](https://app.codecov.io/gh/apache/pinot/pull/11776?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
R
codecov-commenter commented on PR #11774:
URL: https://github.com/apache/pinot/pull/11774#issuecomment-1755683900
##
[Codecov](https://app.codecov.io/gh/apache/pinot/pull/11774?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
R
walterddr commented on issue #10381:
URL: https://github.com/apache/pinot/issues/10381#issuecomment-1755645902
we can definitely discuss the ingestion level nullability however regardless
of whether we use `column level nullability` as a source of truth for
ingestion. the solution to the qu
piby180 commented on issue #11761:
URL: https://github.com/apache/pinot/issues/11761#issuecomment-1755643203
@swaminathanmanish The issue seems to be somewhat related to the comment
[here](https://github.com/aws/aws-sdk-java-v2/issues/4386#issuecomment-1712361944)
> WebIdentityTokenF
mgranderath opened a new issue, #11777:
URL: https://github.com/apache/pinot/issues/11777
We have a requirement that all communication needs to be secured using
mutual TLS, also Pinot internal communication. From our investigation we found
that both GRPC servers for the multi-stage query en
swaminathanmanish commented on issue #11761:
URL: https://github.com/apache/pinot/issues/11761#issuecomment-1755628115
> Thanks for reporting the issue. @snleee @swaminathanmanish Can you help
take a look?
Thanks @Jackie-Jiang . Will take a look.
--
This is an automated message fro
ege-st opened a new pull request, #11776:
URL: https://github.com/apache/pinot/pull/11776
This PR updates the Realtime Segment Builder to remove an unnecessary and
costly transformation: Pinot would transpose the column major table to a row
major table and then transform it back to a column
alexch2000 opened a new issue, #11775:
URL: https://github.com/apache/pinot/issues/11775
# Create a Window HOP function similar to DATETIMECONVERT.
The DATETIMECONVERTWINDOWHOP converts the value from a column that contains
an epoch timestamp into another time unit and buckets based
davecromberge commented on issue #6420:
URL: https://github.com/apache/pinot/issues/6420#issuecomment-1755579214
I've added support for the CPC sketch in this PR which is still being tested:
https://github.com/apache/pinot/pull/11774
The name of the query aggregation function has be
davecromberge opened a new pull request, #11774:
URL: https://github.com/apache/pinot/pull/11774
The [CPC sketch](https://arxiv.org/abs/1708.06839) is a space-efficient
cardinality estimation algorithm. More detail can be found on the Apache
Datasketches website documentation: https://datas
alexch2000 opened a new pull request, #11773:
URL: https://github.com/apache/pinot/pull/11773
Adding DATETIMECONVERTWINDOWHOP function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
gortiz commented on issue #10381:
URL: https://github.com/apache/pinot/issues/10381#issuecomment-1755446477
> IMO this is not a good idea, see my table above. we dont want to couple
ingestion time config with query time config
I think that is where we see things differently. I think c
jddsc commented on issue #11407:
URL: https://github.com/apache/pinot/issues/11407#issuecomment-1755258113
Hi team, same issue as well.
Could you share your workaround (if any)? @rodolphogarrido
--
This is an automated message from the Apache Git Service.
To respond to the message,
Aravind-Suresh commented on issue #11658:
URL: https://github.com/apache/pinot/issues/11658#issuecomment-1755058392
Yes you are right @Jackie-Jiang. For our experiments, we did a small
refactor to RecordLocation to hold the segment name instead of IndexSegment and
derived the index segment
CA1087 commented on issue #11407:
URL: https://github.com/apache/pinot/issues/11407#issuecomment-1754930854
Hi Team,
We are facing the same issue as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
JasonLeeWeiHern opened a new issue, #11772:
URL: https://github.com/apache/pinot/issues/11772
Then query runner tool doesn't accept the header as a parameter, making the
tool unusable for the pinot cluster with access control as we can't pass the
token.
--
This is an automated message fr
76 matches
Mail list logo
