As a follow up, I've modified my data such that there are 2x static states that are identical and modified one of the Queries to use the copy, this restored the topology to the "working as expected" state. So there is certainly something problematic with double querying a TridentState in a single topology.
For more information, there may have been a few other manifestations of this bug even though the most common was a hang: Occasionally there would be a null exception in StateQueryProcessor#execute because processorContext.state[_context.getStateIndex()] was null Occasionally the "Received unexpected tuple ..." RuntimeException would trip in SubtopologyBolt#execute Neither of these symptoms has resurfaced since I made the twin TridentStates (but it's possible they are just rare and caused by something else). -----Original Message----- From: Bart Wyatt [mailto:[email protected]] Sent: Wednesday, October 15, 2014 9:19 AM To: [email protected] Subject: Trident stateQuery(stateA)->... stateQuery(stateA) hangs? Hello all, I started with a working trident topology in the form of: stream .each(new Fields("type", "session", "data"), new CounterKeyGen(), new Fields("counterKey")) .groupBy(new Fields("counterKey")) .persistentAggregate(counterFactory, new Count(), new Fields("count")) .newValuesStream() .each(new Fields("counterKey", "count"), new CountPostProcessing(), new Fields()); Where CounterKeyGen is a Function that maps incoming tuples to 0 or more keys for aggregation and CountPostProcessing does some stuff based on the final counts. Inside these Functions were tweakable constants that I wanted to update "on the fly" using a global config stored in (static) trident state. Resulting in: stream .stateQuery(config, new SnapshotGet(), new Fields("__CONFIG")) .each(new Fields("type", "session", "data ", "__CONFIG"), new CounterKeyGen(), new Fields("counterKey")) .groupBy(new Fields("counterKey")) .persistentAggregate(counterFactory, new Count(), new Fields("count")) .newValuesStream() .stateQuery(config, new SnapshotGet(), new Fields("__CONFIG")) .each(new Fields("counterKey", "count", "__CONFIG "), new CountPostProcessing(), new Fields()); This change hangs the topology, it seems the initial stateQuery is waiting for a commit tuple that never arrives. From debugging I can tell that the partial topology continues to work up until the second stateQuery is added. Given that newValuesStream() outputs "counterKey" and "count" I need to re-fetch the config or concatenate it with the key in order to have it in the second phase of the topology. Neither is ideal, but since I need to have these counts persist across config minor config changes, I cannot concatenate the key and config naively. I would need to modify my backing map to intelligently discard the config for the persistence but maintain it otherwise. Is there a viable workaround for the double-stateQuery issue? Better still, is there a more efficient way to have batch level read only parameters than to inject them into every tuple? -Bart ________________________________ This e-mail may contain CONFIDENTIAL AND PROPRIETARY INFORMATION and/or PRIVILEGED AND CONFIDENTIAL COMMUNICATION intended solely for the recipient and, therefore, may not be retransmitted to any party outside of the recipient's organization without the prior written consent of the sender. If you have received this e-mail in error please notify the sender immediately by telephone or reply e-mail and destroy the original message without making a copy. Deep Silver Volition, LLC accepts no liability for any losses or damages resulting from infected e-mail transmissions and viruses in e-mail attachment. ________________________________ This e-mail may contain CONFIDENTIAL AND PROPRIETARY INFORMATION and/or PRIVILEGED AND CONFIDENTIAL COMMUNICATION intended solely for the recipient and, therefore, may not be retransmitted to any party outside of the recipient's organization without the prior written consent of the sender. If you have received this e-mail in error please notify the sender immediately by telephone or reply e-mail and destroy the original message without making a copy. Deep Silver Volition, LLC accepts no liability for any losses or damages resulting from infected e-mail transmissions and viruses in e-mail attachment.
