havannavar opened a new issue #6920: Kafka indexing service - unable to create datasource URL: https://github.com/apache/incubator-druid/issues/6920 I'm using druid 0.13.0-incubating, Kafka Indexing service, amazon s3 and Postgres, Though topic is created in Kafka but unable to create datasource. No exception or Fails messages in the Logs of coordinator and overload and FAILED logs. Also, I can see records in druid_tasks, druid_supervisor tables and I don't see any records in **druid_datasource**. Can someone help me, going clueless on this issue. Below is my **common.runtime. properties** ```` druid.extensions.loadList=[ "druid-kafka-indexing-service", "druid-s3-extensions", "postgresql-metadata-storage"] # # Zookeeper # druid.zk.service.host=<ipaddress> druid.zk.paths.base=/druid # For PostgreSQL (make sure to additionally include the Postgres extension): druid.metadata.storage.type=postgresql druid.metadata.storage.connector.connectURI=jdbc:postgresql://<ip>:<port>/druid druid.metadata.storage.connector.user=druid druid.metadata.storage.connector.password=<pwd> # # Deep storage # # For S3: druid.storage.type=s3 druid.storage.bucket= myBucket druid.storage.baseKey=druid/segments druid.s3.accessKey=key druid.s3.secretKey=secreteKey # # Indexing service logs # # For S3: druid.indexer.logs.type=s3 druid.indexer.logs.s3Bucket=myBucket druid.indexer.logs.s3Prefix=druid/indexing-logs # # Service discovery # druid.selectors.indexing.serviceName=druid/overlord druid.selectors.coordinator.serviceName=druid/coordinator # # Monitoring # druid.monitoring.monitors=["org.apache.druid.java.util.metrics.JvmMonitor"] druid.emitter=logging druid.emitter.logging.logLevel=info # Storage type of double columns # ommiting this will lead to index double as float at the storage layer druid.indexing.doubleStorage=double ``` ` And Below is my datasource ``` { "type": "kafka", "dataSchema": { "dataSource": "4pm_27thjan", "parser": { "type": "string", "parseSpec": { "format": "json", "timestampSpec": { "column": "time", "format": "auto" }, "dimensionsSpec": { "dimensions": [ "channel", "cityName", "comment", "countryIsoCode", "countryName", "isAnonymous", "isMinor", "isNew", "isRobot", "isUnpatrolled", "metroCode", "namespace", "page", "regionIsoCode", "regionName", "user", { "name": "added", "type": "long" }, { "name": "deleted", "type": "long" }, { "name": "delta", "type": "long" } ] } } }, "metricsSpec" : [], "granularitySpec": { "type": "uniform", "segmentGranularity": "HOUR", "queryGranularity": "NONE", "rollup": false } }, "tuningConfig": { "type": "kafka", "reportParseExceptions": false }, "ioConfig": { "topic": "4pm_27thjan_topic", "replicas": 1, "taskDuration": "PT1H", "completionTimeout": "PT2H", "consumerProperties": { "bootstrap.servers": "<broker1>:9092,<broker2>:9096,<broker3>:9010" } } } ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
