#general
@krishnaraoveera1294: @krishnaraoveera1294 has joined the channel
@vanduc.dn: @vanduc.dn has joined the channel
#random
@krishnaraoveera1294: @krishnaraoveera1294 has joined the channel
@vanduc.dn: @vanduc.dn has joined the channel
#troubleshooting
@diogo.baeder: Hi folks! How can I use the `/ingestFromURI` endpoint from the Controller API to ingest a file as a segment, but defining the segment name myself? I tried passing `segment.name` to the `batchConfigMapStr` parameter JSON, but it didn't work, the Controller ends up creating the segment name by itself. I'd like to have more control over this, because I want to be able to more easily replace segments.
@diogo.baeder: Just for reference, this is my full URI: ```/ingestFromURI?tableNameWithType=cases_OFFLINE&batchConfigMapStr={\"inputFormat\":\"json\",\"input.fs.className\":\"org.apache.pinot.spi.filesystem.LocalPinotFS\",\"segment.name\":\"cases_br_2015_07\"}&sourceURIStr=file:///sensitive-data/outputs/cases/br/2015_07.json```
@mayanks: You can use batch ingestion that allows you full control:
@mayanks: There’s segment name generator spec:
@diogo.baeder: Ah, it's `segmentNameGeneratorSpec` then... I don't want to use the usual batch ingestion approach because I want to be able to manually upload segments, to be able to replace them later on demand. Batch ingestion doesn't ingest files that have already been marked as processed, right?
@diogo.baeder: Worked now! I just used `segmentNameGenerator.type` and `segmentNameGenerator.configs.segment.name` (with the literal dots) as keys for the `batchConfigMapStr` JSON and it's now generating segments as I want :slightly_smiling_face:
@diogo.baeder: Aaaand I could also replace the segment by just ingesting the same segment again, with changed data! Yay!
@mark.needham:
@mark.needham: It sounds like you did something similar to what I did in this one
@mark.needham: maybe I can update it to show how to use the HTTP API
@diogo.baeder: Yeah, I used the API just because I didn't want to depend on the admin CLI on the server that will be managing the loading of data into Pinot. In any case, it worked quite well for me, after I finally understood how to properly configure the segment name.
@krishnaraoveera1294: @krishnaraoveera1294 has joined the channel
@vanduc.dn: @vanduc.dn has joined the channel
#getting-started
@krishnaraoveera1294: @krishnaraoveera1294 has joined the channel
@vanduc.dn: @vanduc.dn has joined the channel
#introductions
@krishnaraoveera1294: @krishnaraoveera1294 has joined the channel
@vanduc.dn: @vanduc.dn has joined the channel
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pinot.apache.org For additional commands, e-mail: dev-h...@pinot.apache.org
