#general
@zineb.raiiss: Hello friends, This is my first time on this platform. I have a problem with a table on Pinot. I created Schéma and the table on Pinot but I can't seem to populate the table with the data I have on the csv file. I read the message I got during segmentation and found the date column field on Schema is not declare well, Date field format on csv file does not match what is declare on diagram. The Date field on File: 08/01/2018 08:00:00 Statement on Diagram: "dateTimeFieldSpecs": [{ "name": "Date_measure", "dataType": "STRING", "format": "1: DAYS: SIMPLE_DATE_FORMAT: dd / MM / YYY HH: mm", "granularity": "5: MINUTES"
@kharekartik: Can you try with format "format":"1:DAYS:SIMPLE_DATE_FORMAT:dd/MM/yyyy HH:mm:ss"
@xiangfu0: i think you have extra spaces in `dd / MM / YYY HH: mm`
@zineb.raiiss: *
@zineb.raiiss: *
@zineb.raiiss: This is result of Segment
@kharekartik: Hi, I think just removing the ss from the format I suggested should work then
@zineb.raiiss: Hi
@zineb.raiiss: yes, I have deleted ss since I have no second in the data. But still, walk no longer
@zineb.raiiss: here is format I used in declaration"format":"1:DAYS:SIMPLE_DATE_FORMAT:dd/MM/yyyy HH:mm"
@zineb.raiiss: if you have any idea about the fault I put
@xiangfu0: Dear Community, TL;DR, Pinot will remove
@zineb.raiiss: Hi,
@zineb.raiiss: after the execution of the segment in order to fill the table here is the error message that I have
@syed.hadi: @syed.hadi has joined the channel
@prtk.ngm: Hi Team, We need to some preprocessing of the record before running ingestion job - create pinot segments outside and push to Pinot data store, what is the way in Pinot we can run some spark job (preprocessing) before running actual ingestion job for pinot segments.
@mayanks: What’s the preprocessing you need to do? Currently the spark ingestion job in Pinot does not support custom preprocessing
@prtk.ngm: we already have on preprocessing spark job - This spark job o/p goes to HDFS
@prtk.ngm: We have ingestion job read HDFS create segment and push to pinot data store.
@prtk.ngm: we want to create pipeline process data after that create segment and push to pinot
@mayanks: If you already have input for create segment you can just use ingestion job to create segment and push to Pinot right?
@mayanks: @prtk.ngm
@lxy1995seu: @lxy1995seu has joined the channel
#random
@syed.hadi: @syed.hadi has joined the channel
@lxy1995seu: @lxy1995seu has joined the channel
#feat-better-schema-evolution
@mattk: @mattk has joined the channel
#pinot-website
@mattk: @mattk has joined the channel
#troubleshooting
@syed.hadi: @syed.hadi has joined the channel
@lxy1995seu: @lxy1995seu has joined the channel
@bowenwan: Hi, it seems that `bin/pinot-ingestion-job.sh` does not accept yaml ingestion job config. It does not work with the example command (`bin/pinot-ingestion-job.sh examples/batch/airlineStats/ingestionJobSpec.yaml`) I copy from
#feat-geo-spatial-index
@mattk: @mattk has joined the channel
#pinot-dev
@zineb.raiiss: @zineb.raiiss has joined the channel
@mattk: @mattk has joined the channel
@suresh.intuit: @suresh.intuit has joined the channel
@weixiang.sun: @weixiang.sun has joined the channel
#s3-multiple-buckets
@mattk: @mattk has joined the channel
#pinot-realtime-table-rebalance
@mattk: @mattk has joined the channel
#getting-started
@zineb.raiiss: @zineb.raiiss has joined the channel
@zineb.raiiss: Hello, This is my first time on this platform. I have a problem with a table on Pinot. I created Schéma and the table on Pinot but I can't seem to populate the table with the data I have on the CSV file. I read the message I got during segmentation and found the date column field on Schema is not declare well, Date field format on CSV file does not match what is declare on the diagram. The Date field on File: 08/01/2018 08:00:00 Statement on Diagram: "dateTimeFieldSpecs": [{ "name": "Date_measure", "dataType": "STRING", "format": "1: DAYS: SIMPLE_DATE_FORMAT: dd / MM / YYY HH: mm", "granularity": "5: MINUTES"
@jackie.jxt: Can you please try `dd/MM/yyyy HH:mm` and see if it works?
@jackie.jxt: I do recommend transforming the time format when ingesting the data using ingestion transform:
@jackie.jxt: `/` is an invalid character in segment name fyi
@zineb.raiiss: I corrected the Format- but doesn't work
@zineb.raiiss:
@zineb.raiiss: Please help
@suresh.intuit: @suresh.intuit has joined the channel
@dadelcas: Hey, is there a way to tag servers upon deployment/start up? I.e. via config files inatead of using the API?
@g.kishore: No, you can only tag them using API.. we avoid config files as much as possible.. it’s makes everything static
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@pinot.apache.org For additional commands, e-mail: dev-h...@pinot.apache.org