Writing to a state in storm in not atomic. Storm guarantees that once the batch completes that the data is written out to all of the states that expect to receive it. The HDFS state guarantees that the dat will have been flushed out the the data nodes when a batch completes and if the topology keeps running eventually the files will be rotated and made available for others to process, but there is no guarantee that the files will rotate at the same time or anything like that.
- Bobby On Thursday, August 3, 2017, 1:17:39 PM CDT, Geoffrey Holmes <[email protected]> wrote: <!--#yiv9469310365 _filtered #yiv9469310365 {font-family:Arial;panose-1:2 11 6 4 2 2 2 2 2 4;} _filtered #yiv9469310365 {font-family:"Cambria Math";panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv9469310365 {font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}#yiv9469310365 #yiv9469310365 p.yiv9469310365MsoNormal, #yiv9469310365 li.yiv9469310365MsoNormal, #yiv9469310365 div.yiv9469310365MsoNormal {margin:0in;margin-bottom:.0001pt;font-size:12.0pt;font-family:"Calibri", sans-serif;}#yiv9469310365 a:link, #yiv9469310365 span.yiv9469310365MsoHyperlink {color:#0563C1;text-decoration:underline;}#yiv9469310365 a:visited, #yiv9469310365 span.yiv9469310365MsoHyperlinkFollowed {color:#954F72;text-decoration:underline;}#yiv9469310365 span.yiv9469310365EmailStyle17 {font-family:"Arial", sans-serif;color:windowtext;font-weight:normal;font-style:normal;}#yiv9469310365 span.yiv9469310365msoIns {text-decoration:underline;color:teal;}#yiv9469310365 .yiv9469310365MsoChpDefault {font-family:"Calibri", sans-serif;} _filtered #yiv9469310365 {margin:1.0in 1.0in 1.0in 1.0in;}#yiv9469310365 div.yiv9469310365WordSection1 {}--> I read STORM-837 (https://issues.apache.org/jira/browse/STORM-837) and have a question. How does this work if I have more than one HDFS state in my Trident topology? Can I ensure that a record ends up written to both HDFS states or none but not just one or the other?
