Jozef Vilcek created BEAM-4672: ---------------------------------- Summary: Dynamic destination write to HDFS fails with IntervalWindow in shard template Key: BEAM-4672 URL: https://issues.apache.org/jira/browse/BEAM-4672 Project: Beam Issue Type: Bug Components: io-java-hadoop Affects Versions: 2.4.0 Reporter: Jozef Vilcek Assignee: Chamikara Jayalath
I am doing a windowed write with dynamic destinations and it is failing with error message : {noformat} java.lang.IllegalArgumentException: Expect srcResourceIds and destResourceIds have the same scheme, but received hdfs, window-1970-01-01T00{noformat} I am writing from IntervalWindow, for which target file name created by DefaultFilenamePolicy is going to look something like this: {noformat} window-1970-01-01T00:00:00.000Z-1970-01-01T01:00:00.000Z-pane-0-last-00000-of-00001.avro{noformat} When resolving the path, HadoopResourceId is used and this one is backed by java URI. The filename like above is parsed incorrectly because of ':' character introduced by window-to-string. -- This message was sent by Atlassian JIRA (v7.6.3#76005)