[jira] [Work logged] (BEAM-9687) Names of temporary files created by interactive runner include characters invalid on some platforms.
[ https://issues.apache.org/jira/browse/BEAM-9687?focusedWorklogId=415774&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415774 ] ASF GitHub Bot logged work on BEAM-9687: Author: ASF GitHub Bot Created on: 03/Apr/20 22:05 Start Date: 03/Apr/20 22:05 Worklog Time Spent: 10m Work Description: aaltay commented on pull request #11306: [BEAM-9687] Change delimeter to a dash as the pipe is a reserved symbol in Windows in file names URL: https://github.com/apache/beam/pull/11306 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415774) Time Spent: 40m (was: 0.5h) > Names of temporary files created by interactive runner include characters > invalid on some platforms. > > > Key: BEAM-9687 > URL: https://issues.apache.org/jira/browse/BEAM-9687 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Valentyn Tymofieiev >Assignee: Sam Rohde >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Temporary files created by interactive runner in streaming scenarios include > pipe '|' characters, which are not allowed in filenames on Windows platform. > This causes test failures on a Windows platform: > > python setup.py nosetests --tests > apache_beam/runners/interactive/pipeline_instrument_test.py:PipelineInstrumentTest.test_instrument_example_unbounded_pipeline_to_multiple_read_cache > == > ERROR: Tests that the instrumenter works for multiple unbounded sources. > -- > Traceback (most recent call last): > File > "C:\projects\apache_beam\runners\interactive\pipeline_instrument_test.py", > line 698, in test_instrument_example_unbounded_pipeline_to_multiple_read_cache > self._mock_write_cache([b''], cache_key) > File > "C:\projects\apache_beam\runners\interactive\pipeline_instrument_test.py", > line 227, in _mock_write_cache > ie.current_env().cache_manager().write(values, *labels) > File > "C:\projects\apache_beam\runners\interactive\caching\streaming_cache.py", > line 323, in write > with open(filepath, 'ab') as f: > IOError: [Errno 22] invalid mode ('ab') or filename: > 'c:\\users\\deft-t~1\\appdata\\local\\temp\\2\\interactive-temp-xwg5qi\\full\\pcoll_1|149781752|149781920|1 > 49231600' > > [1] > https://github.com/apache/beam/blob/e6b37c44d542969b6104fc97ee6f25b6f7d2ddba/sdks/python/apache_beam/runners/interactive/caching/streaming_cache.py#L323 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9687) Names of temporary files created by interactive runner include characters invalid on some platforms.
[ https://issues.apache.org/jira/browse/BEAM-9687?focusedWorklogId=415721&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415721 ] ASF GitHub Bot logged work on BEAM-9687: Author: ASF GitHub Bot Created on: 03/Apr/20 20:31 Start Date: 03/Apr/20 20:31 Worklog Time Spent: 10m Work Description: rohdesamuel commented on issue #11306: [BEAM-9687] Change delimeter to a dash as the pipe is a reserved symbol in Windows in file names URL: https://github.com/apache/beam/pull/11306#issuecomment-608644347 Thanks Ahmet! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415721) Time Spent: 0.5h (was: 20m) > Names of temporary files created by interactive runner include characters > invalid on some platforms. > > > Key: BEAM-9687 > URL: https://issues.apache.org/jira/browse/BEAM-9687 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Valentyn Tymofieiev >Assignee: Sam Rohde >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Temporary files created by interactive runner in streaming scenarios include > pipe '|' characters, which are not allowed in filenames on Windows platform. > This causes test failures on a Windows platform: > > python setup.py nosetests --tests > apache_beam/runners/interactive/pipeline_instrument_test.py:PipelineInstrumentTest.test_instrument_example_unbounded_pipeline_to_multiple_read_cache > == > ERROR: Tests that the instrumenter works for multiple unbounded sources. > -- > Traceback (most recent call last): > File > "C:\projects\apache_beam\runners\interactive\pipeline_instrument_test.py", > line 698, in test_instrument_example_unbounded_pipeline_to_multiple_read_cache > self._mock_write_cache([b''], cache_key) > File > "C:\projects\apache_beam\runners\interactive\pipeline_instrument_test.py", > line 227, in _mock_write_cache > ie.current_env().cache_manager().write(values, *labels) > File > "C:\projects\apache_beam\runners\interactive\caching\streaming_cache.py", > line 323, in write > with open(filepath, 'ab') as f: > IOError: [Errno 22] invalid mode ('ab') or filename: > 'c:\\users\\deft-t~1\\appdata\\local\\temp\\2\\interactive-temp-xwg5qi\\full\\pcoll_1|149781752|149781920|1 > 49231600' > > [1] > https://github.com/apache/beam/blob/e6b37c44d542969b6104fc97ee6f25b6f7d2ddba/sdks/python/apache_beam/runners/interactive/caching/streaming_cache.py#L323 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9687) Names of temporary files created by interactive runner include characters invalid on some platforms.
[ https://issues.apache.org/jira/browse/BEAM-9687?focusedWorklogId=415625&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415625 ] ASF GitHub Bot logged work on BEAM-9687: Author: ASF GitHub Bot Created on: 03/Apr/20 17:45 Start Date: 03/Apr/20 17:45 Worklog Time Spent: 10m Work Description: aaltay commented on issue #11306: [BEAM-9687] Change delimeter to a dash as the pipe is a reserved symbol in Windows in file names URL: https://github.com/apache/beam/pull/11306#issuecomment-608573647 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415625) Time Spent: 20m (was: 10m) > Names of temporary files created by interactive runner include characters > invalid on some platforms. > > > Key: BEAM-9687 > URL: https://issues.apache.org/jira/browse/BEAM-9687 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Valentyn Tymofieiev >Assignee: Sam Rohde >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Temporary files created by interactive runner in streaming scenarios include > pipe '|' characters, which are not allowed in filenames on Windows platform. > This causes test failures on a Windows platform: > > python setup.py nosetests --tests > apache_beam/runners/interactive/pipeline_instrument_test.py:PipelineInstrumentTest.test_instrument_example_unbounded_pipeline_to_multiple_read_cache > == > ERROR: Tests that the instrumenter works for multiple unbounded sources. > -- > Traceback (most recent call last): > File > "C:\projects\apache_beam\runners\interactive\pipeline_instrument_test.py", > line 698, in test_instrument_example_unbounded_pipeline_to_multiple_read_cache > self._mock_write_cache([b''], cache_key) > File > "C:\projects\apache_beam\runners\interactive\pipeline_instrument_test.py", > line 227, in _mock_write_cache > ie.current_env().cache_manager().write(values, *labels) > File > "C:\projects\apache_beam\runners\interactive\caching\streaming_cache.py", > line 323, in write > with open(filepath, 'ab') as f: > IOError: [Errno 22] invalid mode ('ab') or filename: > 'c:\\users\\deft-t~1\\appdata\\local\\temp\\2\\interactive-temp-xwg5qi\\full\\pcoll_1|149781752|149781920|1 > 49231600' > > [1] > https://github.com/apache/beam/blob/e6b37c44d542969b6104fc97ee6f25b6f7d2ddba/sdks/python/apache_beam/runners/interactive/caching/streaming_cache.py#L323 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9687) Names of temporary files created by interactive runner include characters invalid on some platforms.
[ https://issues.apache.org/jira/browse/BEAM-9687?focusedWorklogId=415612&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415612 ] ASF GitHub Bot logged work on BEAM-9687: Author: ASF GitHub Bot Created on: 03/Apr/20 17:40 Start Date: 03/Apr/20 17:40 Worklog Time Spent: 10m Work Description: rohdesamuel commented on pull request #11306: [BEAM-9687] Change delimeter to a dash as the pipe is a reserved symbol in Windows in file names URL: https://github.com/apache/beam/pull/11306 Change-Id: I96853c57f421f60ddbee24c1b4f94ec5b80d948c Windows reserves the use of the '|' symbol in file names. Change the cache key delimiter to a dash to fix this. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[![Build Status](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[![Build Status](https://builds.apache.org/jo