Re: Metron Writes partial JSON to HDFS

2019-04-15 Thread Michael Miklavcic
Hi, this looks like you may be getting failures when writing to HDFS. For example, if there's a problem with a batch, it's possible that it will partially be written to HDFS. I would expect that in instances like this you will see duplicate entries in your HDFS records. The reason for this is that

Re: Load_tool.sh issues

2019-04-15 Thread Michael Miklavcic
Thiago, have you had any luck with this? I haven't seen this particular issue before. Is this happening for all of your topologies, or just a specific set? Which ones? Also, can you check for any errors in the kafka broker logs along with your Storm topology logs when the hang occurs? One other thi