[jira] [Commented] (BEAM-6354) Hanging SplittableDoFnTest#testLateData
[
https://issues.apache.org/jira/browse/BEAM-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749552#comment-16749552
]
Kenneth Knowles commented on BEAM-6354:
---
Bisect says {{[d13663786cce2851fb83c4e0dee315435518f471] [BEAM-2939]
SplittableDoFn Java SDK API Changes (#6969)}} which sounds about right.
> Hanging SplittableDoFnTest#testLateData
> ---
>
> Key: BEAM-6354
> URL: https://issues.apache.org/jira/browse/BEAM-6354
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Gleb Kanterov
>Assignee: Kenneth Knowles
>Priority: Major
> Fix For: 2.10.0
>
>
> It seems that they have a similar root cause because both of them use
> unbounded streams.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5442) PortableRunner swallows custom options for Runner
[ https://issues.apache.org/jira/browse/BEAM-5442?focusedWorklogId=188653=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188653 ] ASF GitHub Bot logged work on BEAM-5442: Author: ASF GitHub Bot Created on: 23/Jan/19 05:39 Start Date: 23/Jan/19 05:39 Worklog Time Spent: 10m Work Description: tweise commented on pull request #7597: [WIP] [BEAM-5442] Retrieve valid runner options from JobService URL: https://github.com/apache/beam/pull/7597 So far just a sketch of the service side and an example output is here: https://gist.github.com/tweise/093a4b2d0384cbb65fcf558c600adf63 Next step is to wire it in Py PortableRunner Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it. Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | --- | --- | --- | --- Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | --- | --- | --- | --- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188653) Time Spent: 13h 10m (was: 13h) > PortableRunner swallows custom options for Runner > - > >
[jira] [Updated] (BEAM-6492) BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0
[ https://issues.apache.org/jira/browse/BEAM-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-6492: -- Labels: sickbay (was: ) > BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0 > --- > > Key: BEAM-6492 > URL: https://issues.apache.org/jira/browse/BEAM-6492 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Affects Versions: 2.8.0, 2.9.0 >Reporter: Gleb Kanterov >Priority: Major > Labels: sickbay > > It seems that they have a similar root cause because both of them use > unbounded streams. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-6354) Hanging SplittableDoFnTest#testLateData
[ https://issues.apache.org/jira/browse/BEAM-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-6354: -- Summary: Hanging SplittableDoFnTest#testLateData (was: Hanging BoundedReadFromUnboundedSourceTest#testTimeBound and SplittableDoFnTest#testLateData) > Hanging SplittableDoFnTest#testLateData > --- > > Key: BEAM-6354 > URL: https://issues.apache.org/jira/browse/BEAM-6354 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Gleb Kanterov >Assignee: Kenneth Knowles >Priority: Major > Fix For: 2.10.0 > > > It seems that they have a similar root cause because both of them use > unbounded streams. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-6479) org.apache.beam.sdk.io.AvroIO.RecordFormatter should use IndexedRecord and not GenericRecord
[ https://issues.apache.org/jira/browse/BEAM-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-6479: -- Component/s: (was: sdk-java-core) io-java-avro > org.apache.beam.sdk.io.AvroIO.RecordFormatter should use IndexedRecord and > not GenericRecord > > > Key: BEAM-6479 > URL: https://issues.apache.org/jira/browse/BEAM-6479 > Project: Beam > Issue Type: Task > Components: io-java-avro >Affects Versions: 2.9.0 >Reporter: Romain Manni-Bucau >Priority: Major > > GenericRecord being an indexedRecord and the schema being provided this can > only makes the supported use cases bigger -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-6492) BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0
[ https://issues.apache.org/jira/browse/BEAM-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-6492: -- Fix Version/s: (was: 2.10.0) > BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0 > --- > > Key: BEAM-6492 > URL: https://issues.apache.org/jira/browse/BEAM-6492 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Gleb Kanterov >Priority: Major > > It seems that they have a similar root cause because both of them use > unbounded streams. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-6492) BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0
[ https://issues.apache.org/jira/browse/BEAM-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-6492: -- Affects Version/s: 2.8.0 2.9.0 > BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0 > --- > > Key: BEAM-6492 > URL: https://issues.apache.org/jira/browse/BEAM-6492 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Affects Versions: 2.8.0, 2.9.0 >Reporter: Gleb Kanterov >Priority: Major > > It seems that they have a similar root cause because both of them use > unbounded streams. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-6492) BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0
[ https://issues.apache.org/jira/browse/BEAM-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles reassigned BEAM-6492: - Assignee: (was: Kenneth Knowles) > BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0 > --- > > Key: BEAM-6492 > URL: https://issues.apache.org/jira/browse/BEAM-6492 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Gleb Kanterov >Priority: Major > Fix For: 2.10.0 > > > It seems that they have a similar root cause because both of them use > unbounded streams. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-6492) BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0
[ https://issues.apache.org/jira/browse/BEAM-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles updated BEAM-6492: -- Summary: BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0 (was: Hanging BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0) > BoundedReadFromUnboundedSourceTest#testTimeBound hangs since at least 2.8.0 > --- > > Key: BEAM-6492 > URL: https://issues.apache.org/jira/browse/BEAM-6492 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Gleb Kanterov >Assignee: Kenneth Knowles >Priority: Major > Fix For: 2.10.0 > > > It seems that they have a similar root cause because both of them use > unbounded streams. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-6461) beam_precommit_Spotless_Cron flake on format violations
[
https://issues.apache.org/jira/browse/BEAM-6461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kenneth Knowles resolved BEAM-6461.
---
Resolution: Fixed
Fix Version/s: Not applicable
Should be healthy at head.
> beam_precommit_Spotless_Cron flake on format violations
> ---
>
> Key: BEAM-6461
> URL: https://issues.apache.org/jira/browse/BEAM-6461
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Tyler Akidau
>Assignee: Kenneth Knowles
>Priority: Major
> Fix For: Not applicable
>
>
> From: https://builds.apache.org/job/beam_PreCommit_Spotless_Cron/302/console
>
> *04:01:32* >
> *Task :beam-sdks-java-core:spotlessJava*
> FAILED*04:01:32* *04:01:32* FAILURE: Build failed with an
> exception.*04:01:32* *04:01:32* * What went wrong:*04:01:32* Execution failed
> for task ':beam-sdks-java-core:spotlessJava'.*04:01:32* > The following files
> had format violations:*04:01:32*
> sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/utils/TestPOJOs.java*04:01:32*
>@@ -820,8 +820,7 @@*04:01:32* *04:01:32*
> ··/**·The·schema·for·\{@link·PojoWithNestedArray}.·*/*04:01:32*
> ··public·static·final·Schema·POJO_WITH_NESTED_ARRAY_SCHEMA·=*04:01:32*
>-··Schema.builder().addArrayField("pojos",*04:01:32*
> -··FieldType.array(*04:01:32*
> -··FieldType.row(SIMPLE_POJO_SCHEMA)))*04:01:32*
> -··.build();*04:01:32* +··Schema.builder()*04:01:32*
>
> +··.addArrayField("pojos",·FieldType.array(FieldType.row(SIMPLE_POJO_SCHEMA)))*04:01:32*
>+··.build();*04:01:32*}*04:01:32*
> sdks/java/core/src/test/java/org/apache/beam/sdk/schemas/JavaFieldSchemaTest.java*04:01:32*
>@@ -466,11 +466,10 @@*04:01:32* *04:01:32*
> SimplePOJO·simplePojo·=·createSimple("string");*04:01:32*
> List·simplePojoList·=·ImmutableList.of(simplePojo,·simplePojo);*04:01:32*
>-List>·simplePojoListOfList·=*04:01:32*
> -ImmutableList.of(simplePojoList,·simplePojoList);*04:01:32*
>
> +List>·simplePojoListOfList·=·ImmutableList.of(simplePojoList,·simplePojoList);*04:01:32*
> *04:01:32*
> -PojoWithNestedArray·converted·=·registry.getFromRowFunction(PojoWithNestedArray.class)*04:01:32*
>-.apply(nestedRow);*04:01:32*
> +PojoWithNestedArray·converted·=*04:01:32*
> +registry.getFromRowFunction(PojoWithNestedArray.class).apply(nestedRow);*04:01:32*
>
> assertEquals(simplePojoListOfList,·converted.pojos);*04:01:32*
> ··}*04:01:32* *04:01:32* @@ -487,11 +486,12 @@*04:01:32* *04:01:32*
>SimplePOJO·simplePojo·=·createSimple("string");*04:01:32*
>
> List·simplePojoList·=·ImmutableList.of(simplePojo,·simplePojo);*04:01:32*
>-List>·simplePojoListOfList·=*04:01:32*
> -ImmutableList.of(simplePojoList,·simplePojoList);*04:01:32*
>
> +List>·simplePojoListOfList·=·ImmutableList.of(simplePojoList,·simplePojoList);*04:01:32*
> *04:01:32*
> -Row·converted·=·registry.getToRowFunction(PojoWithNestedArray.class)*04:01:32*
>
> -.apply(new·PojoWithNestedArray(simplePojoListOfList));*04:01:32*
> +Row·converted·=*04:01:32* +registry*04:01:32*
>+.getToRowFunction(PojoWithNestedArray.class)*04:01:32*
>
> +.apply(new·PojoWithNestedArray(simplePojoListOfList));*04:01:32*
>assertEquals(nestedRow,·converted);*04:01:32*
> ··}*04:01:32*}*04:01:32* Run 'gradlew spotlessApply' to fix
> these violations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-6354) Hanging BoundedReadFromUnboundedSourceTest#testTimeBound and SplittableDoFnTest#testLateData
[ https://issues.apache.org/jira/browse/BEAM-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749530#comment-16749530 ] Kenneth Knowles commented on BEAM-6354: --- I've now confirmed that BoundedReadFromUnboundedSourceTest was broken for 2.9.0 and also 2.8.0. I was mistaken about it working. The tests were not running at that time. I don't think this needs to block the 2.10.0 release. However, SplittableDoFnTest was not broken for 2.8.0 and not broken for 2.9.0. I will bisect that one separately. > Hanging BoundedReadFromUnboundedSourceTest#testTimeBound and > SplittableDoFnTest#testLateData > > > Key: BEAM-6354 > URL: https://issues.apache.org/jira/browse/BEAM-6354 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Gleb Kanterov >Assignee: Kenneth Knowles >Priority: Major > Fix For: 2.10.0 > > > It seems that they have a similar root cause because both of them use > unbounded streams. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-6447) Spotless paddedCell appears to cause unpredictable behavior
[ https://issues.apache.org/jira/browse/BEAM-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles resolved BEAM-6447. --- Resolution: Fixed Fix Version/s: Not applicable > Spotless paddedCell appears to cause unpredictable behavior > --- > > Key: BEAM-6447 > URL: https://issues.apache.org/jira/browse/BEAM-6447 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Kenneth Knowles >Assignee: Kenneth Knowles >Priority: Major > Fix For: Not applicable > > Time Spent: 3h 10m > Remaining Estimate: 0h > > In https://github.com/apache/beam/pull/7505 spotlessCheck passed even though > spotlessApply was not a no-op. > In https://github.com/apache/beam/pull/7523 and > https://github.com/apache/beam/pull/7527 spotlessApply was run and the result > _also_ passed spotlessCheck. > Confirmed that spotlessCheck fails on > https://github.com/apache/beam/pull/7505 if paddedCell is turned off. The use > of paddedCell is to workaround bugs in the underlying formatter (in our case > google-java-format). Currently, there is no known bug affecting us, so we > should not be using paddedCell. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-6479) org.apache.beam.sdk.io.AvroIO.RecordFormatter should use IndexedRecord and not GenericRecord
[ https://issues.apache.org/jira/browse/BEAM-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenneth Knowles reassigned BEAM-6479: - Assignee: (was: Kenneth Knowles) > org.apache.beam.sdk.io.AvroIO.RecordFormatter should use IndexedRecord and > not GenericRecord > > > Key: BEAM-6479 > URL: https://issues.apache.org/jira/browse/BEAM-6479 > Project: Beam > Issue Type: Task > Components: sdk-java-core >Affects Versions: 2.9.0 >Reporter: Romain Manni-Bucau >Priority: Major > > GenericRecord being an indexedRecord and the schema being provided this can > only makes the supported use cases bigger -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-6354) Hanging BoundedReadFromUnboundedSourceTest#testTimeBound and SplittableDoFnTest#testLateData
[
https://issues.apache.org/jira/browse/BEAM-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749510#comment-16749510
]
Kenneth Knowles commented on BEAM-6354:
---
One the BoundedReadFromUnboundedSourceTest I noticed logs filled with
{code}
[direct-runner-worker] WARN org.apache.beam.sdk.util.MutationDetectors - Coder
of type class org.apache.beam.sdk.coders.KvCoder has a #structuralValue method
which does not return true when the encoding of the elements is equal. Element
KV{-1096158736,
Shard{source=org.apache.beam.sdk.io.BoundedReadFromUnboundedSourceTest$TestCountingSource@b310c3,
maxNumRecords=9223372036854775807, maxReadTime=PT0.200S}}
{code}
> Hanging BoundedReadFromUnboundedSourceTest#testTimeBound and
> SplittableDoFnTest#testLateData
>
>
> Key: BEAM-6354
> URL: https://issues.apache.org/jira/browse/BEAM-6354
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Gleb Kanterov
>Assignee: Kenneth Knowles
>Priority: Major
> Fix For: 2.10.0
>
>
> It seems that they have a similar root cause because both of them use
> unbounded streams.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (BEAM-6354) Hanging BoundedReadFromUnboundedSourceTest#testTimeBound and SplittableDoFnTest#testLateData
[ https://issues.apache.org/jira/browse/BEAM-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on BEAM-6354 started by Kenneth Knowles. - > Hanging BoundedReadFromUnboundedSourceTest#testTimeBound and > SplittableDoFnTest#testLateData > > > Key: BEAM-6354 > URL: https://issues.apache.org/jira/browse/BEAM-6354 > Project: Beam > Issue Type: Bug > Components: sdk-java-core >Reporter: Gleb Kanterov >Assignee: Kenneth Knowles >Priority: Major > Fix For: 2.10.0 > > > It seems that they have a similar root cause because both of them use > unbounded streams. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6271) initial support for portable api in samza runner
[ https://issues.apache.org/jira/browse/BEAM-6271?focusedWorklogId=188647=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188647 ] ASF GitHub Bot logged work on BEAM-6271: Author: ASF GitHub Bot Created on: 23/Jan/19 04:13 Start Date: 23/Jan/19 04:13 Worklog Time Spent: 10m Work Description: lhaiesp commented on issue #7321: [BEAM-6271] SamzaRunner: initial support for portable api in samza runner URL: https://github.com/apache/beam/pull/7321#issuecomment-456663737 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188647) Time Spent: 5.5h (was: 5h 20m) > initial support for portable api in samza runner > > > Key: BEAM-6271 > URL: https://issues.apache.org/jira/browse/BEAM-6271 > Project: Beam > Issue Type: Task > Components: runner-samza >Reporter: Hai Lu >Assignee: Hai Lu >Priority: Major > Time Spent: 5.5h > Remaining Estimate: 0h > > Support portable api in Samza runner. > This ticket tracks the initial effort to support portable api in Samza > runner, including job server inside samza runner, config translation for > portable pipeline, transform translation for portable pipeline, refactor of > existing codes to merge logic of portable api and java api as much as > possible, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188644=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188644
]
ASF GitHub Bot logged work on BEAM-6489:
Author: ASF GitHub Bot
Created on: 23/Jan/19 03:57
Start Date: 23/Jan/19 03:57
Worklog Time Spent: 10m
Work Description: charlesccychen commented on pull request #7596:
[BEAM-6489] Remove usages of a deprecated flag and simplify SDK dependencies'
version spec.
URL: https://github.com/apache/beam/pull/7596
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188644)
Time Spent: 1h 40m (was: 1.5h)
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188643=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188643
]
ASF GitHub Bot logged work on BEAM-6489:
Author: ASF GitHub Bot
Created on: 23/Jan/19 03:57
Start Date: 23/Jan/19 03:57
Worklog Time Spent: 10m
Work Description: charlesccychen commented on issue #7596: [BEAM-6489]
Remove usages of a deprecated flag and simplify SDK dependencies' version spec.
URL: https://github.com/apache/beam/pull/7596#issuecomment-456661122
Thank you Valentyn! This is much cleaner and unbreaks precommits.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188643)
Time Spent: 1.5h (was: 1h 20m)
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188645=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188645
]
ASF GitHub Bot logged work on BEAM-6489:
Author: ASF GitHub Bot
Created on: 23/Jan/19 03:57
Start Date: 23/Jan/19 03:57
Worklog Time Spent: 10m
Work Description: charlesccychen commented on issue #7596: [BEAM-6489]
Remove usages of a deprecated flag and simplify SDK dependencies' version spec.
URL: https://github.com/apache/beam/pull/7596#issuecomment-456661122
Thank you Valentyn! LGTM. This is much cleaner and unbreaks precommits.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188645)
Time Spent: 1h 50m (was: 1h 40m)
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6439) Move Python Flink VR tests to PreCommit
[ https://issues.apache.org/jira/browse/BEAM-6439?focusedWorklogId=188634=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188634 ] ASF GitHub Bot logged work on BEAM-6439: Author: ASF GitHub Bot Created on: 23/Jan/19 03:32 Start Date: 23/Jan/19 03:32 Worklog Time Spent: 10m Work Description: tweise commented on pull request #7539: [BEAM-6439] Move Python Validates Runner Flink test to PreCommit URL: https://github.com/apache/beam/pull/7539 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188634) Time Spent: 4h 40m (was: 4.5h) > Move Python Flink VR tests to PreCommit > --- > > Key: BEAM-6439 > URL: https://issues.apache.org/jira/browse/BEAM-6439 > Project: Beam > Issue Type: New Feature > Components: runner-flink, testing >Reporter: Robert Bradshaw >Assignee: Robert Bradshaw >Priority: Major > Fix For: Not applicable > > Attachments: png.png > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Now that they're fast and stable, it would be good to catch changes that > break this earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6439) Move Python Flink VR tests to PreCommit
[ https://issues.apache.org/jira/browse/BEAM-6439?focusedWorklogId=188632=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188632 ] ASF GitHub Bot logged work on BEAM-6439: Author: ASF GitHub Bot Created on: 23/Jan/19 03:29 Start Date: 23/Jan/19 03:29 Worklog Time Spent: 10m Work Description: tweise commented on issue #7539: [BEAM-6439] Move Python Validates Runner Flink test to PreCommit URL: https://github.com/apache/beam/pull/7539#issuecomment-456656153 Hopefully pre-commit check signal will help prevent some of the build breakages we saw lately. But I don't think running all tests of a runners x SDK matrix is feasible going forward. Even just this combination, once it matches the coverage of the Java SDK, is probably no longer suitable for pre-commit. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188632) Time Spent: 4.5h (was: 4h 20m) > Move Python Flink VR tests to PreCommit > --- > > Key: BEAM-6439 > URL: https://issues.apache.org/jira/browse/BEAM-6439 > Project: Beam > Issue Type: New Feature > Components: runner-flink, testing >Reporter: Robert Bradshaw >Assignee: Robert Bradshaw >Priority: Major > Fix For: Not applicable > > Attachments: png.png > > Time Spent: 4.5h > Remaining Estimate: 0h > > Now that they're fast and stable, it would be good to catch changes that > break this earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188629=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188629
]
ASF GitHub Bot logged work on BEAM-6489:
Author: ASF GitHub Bot
Created on: 23/Jan/19 03:19
Start Date: 23/Jan/19 03:19
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #7596: [BEAM-6489] Remove
usages of a deprecated flag and simplify SDK dependencies' version spec.
URL: https://github.com/apache/beam/pull/7596#issuecomment-456654449
CC'ing a few committers - feel free to merge this to unblock the Python
Precommit suite. One review is enough. Thank you.
@mxm @robertwb @aaltay @charlesccychen
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188629)
Time Spent: 1h 20m (was: 1h 10m)
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188619=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188619
]
ASF GitHub Bot logged work on BEAM-6489:
Author: ASF GitHub Bot
Created on: 23/Jan/19 02:44
Start Date: 23/Jan/19 02:44
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #7596: [BEAM-6489] Remove
usages of a deprecated flag and simplify SDK dependencies' version spec.
URL: https://github.com/apache/beam/pull/7596#issuecomment-456647851
Run Portable_Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188619)
Time Spent: 1h (was: 50m)
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
> Time Spent: 1h
> Remaining Estimate: 0h
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188620=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188620
]
ASF GitHub Bot logged work on BEAM-6489:
Author: ASF GitHub Bot
Created on: 23/Jan/19 02:44
Start Date: 23/Jan/19 02:44
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #7596: [BEAM-6489] Remove
usages of a deprecated flag and simplify SDK dependencies' version spec.
URL: https://github.com/apache/beam/pull/7596#issuecomment-456647894
Run Portable_Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188620)
Time Spent: 1h 10m (was: 1h)
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6439) Move Python Flink VR tests to PreCommit
[ https://issues.apache.org/jira/browse/BEAM-6439?focusedWorklogId=188616=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188616 ] ASF GitHub Bot logged work on BEAM-6439: Author: ASF GitHub Bot Created on: 23/Jan/19 02:39 Start Date: 23/Jan/19 02:39 Worklog Time Spent: 10m Work Description: angoenka commented on issue #7539: [BEAM-6439] Move Python Validates Runner Flink test to PreCommit URL: https://github.com/apache/beam/pull/7539#issuecomment-456646785 Now I see what you mean. Thanks! Updated the PR comments and readme to remove reference for the old postcommit job. But I would say that having these run available in post commit and visible in PR gave a lot of confidence in the system. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188616) Time Spent: 4h 20m (was: 4h 10m) > Move Python Flink VR tests to PreCommit > --- > > Key: BEAM-6439 > URL: https://issues.apache.org/jira/browse/BEAM-6439 > Project: Beam > Issue Type: New Feature > Components: runner-flink, testing >Reporter: Robert Bradshaw >Assignee: Robert Bradshaw >Priority: Major > Fix For: Not applicable > > Attachments: png.png > > Time Spent: 4h 20m > Remaining Estimate: 0h > > Now that they're fast and stable, it would be good to catch changes that > break this earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188592=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188592
]
ASF GitHub Bot logged work on BEAM-6489:
Author: ASF GitHub Bot
Created on: 23/Jan/19 01:48
Start Date: 23/Jan/19 01:48
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #7596: [BEAM-6489] Remove
usages of a deprecated flag and simplify SDK dependencies' version spec.
URL: https://github.com/apache/beam/pull/7596#issuecomment-456636422
Run Portable_Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188592)
Time Spent: 50m (was: 40m)
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?focusedWorklogId=188577=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188577 ] ASF GitHub Bot logged work on BEAM-6491: Author: ASF GitHub Bot Created on: 23/Jan/19 01:19 Start Date: 23/Jan/19 01:19 Worklog Time Spent: 10m Work Description: swegner commented on issue #7595: [BEAM-6491] Ignore flaky test: FileIOTest.testMatchWatchForNewFiles URL: https://github.com/apache/beam/pull/7595#issuecomment-456628938 LGTM, will merge when green This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188577) Time Spent: 0.5h (was: 20m) > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: Bug > Components: java-fn-execution, test-failures >Reporter: Alex Amato >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at >
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[ https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188586=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188586 ] ASF GitHub Bot logged work on BEAM-6489: Author: ASF GitHub Bot Created on: 23/Jan/19 01:35 Start Date: 23/Jan/19 01:35 Worklog Time Spent: 10m Work Description: tvalentyn commented on pull request #7596: [BEAM-6489] Cleanup tox.ini URL: https://github.com/apache/beam/pull/7596 This change: - Adds the dependency to newly released dill==0.2.9, which removes the necessity to pass --process-dependency-links to pip, since we no longer need to pick up an unreleased change. - Cleans up tox.ini to use platform-specific dependecy declaration https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-platform-specific-dependencies - Sorts the SDK dependencies alphabetically. Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | --- | --- | --- | --- Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | --- | --- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188586) Time Spent: 40m (was: 0.5h) > Python precommits are failiing due to a pip error: "no such option: > --process-dependency-links" > > > Key: BEAM-6489 > URL: https://issues.apache.org/jira/browse/BEAM-6489 > Project: Beam > Issue Type:
[jira] [Work logged] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?focusedWorklogId=188584=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188584 ] ASF GitHub Bot logged work on BEAM-6491: Author: ASF GitHub Bot Created on: 23/Jan/19 01:31 Start Date: 23/Jan/19 01:31 Worklog Time Spent: 10m Work Description: swegner commented on pull request #7595: [BEAM-6491] Ignore flaky test: FileIOTest.testMatchWatchForNewFiles URL: https://github.com/apache/beam/pull/7595 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188584) Time Spent: 50m (was: 40m) > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: Bug > Components: java-fn-execution, test-failures >Reporter: Alex Amato >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at
[jira] [Work logged] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?focusedWorklogId=188582=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188582 ] ASF GitHub Bot logged work on BEAM-6491: Author: ASF GitHub Bot Created on: 23/Jan/19 01:27 Start Date: 23/Jan/19 01:27 Worklog Time Spent: 10m Work Description: lhaiesp commented on issue #7595: [BEAM-6491] Ignore flaky test: FileIOTest.testMatchWatchForNewFiles URL: https://github.com/apache/beam/pull/7595#issuecomment-456631374 I don't think it's "flaky". It fails consistently at least on the Jenkins hosts. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188582) Time Spent: 40m (was: 0.5h) > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: Bug > Components: java-fn-execution, test-failures >Reporter: Alex Amato >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at >
[jira] [Updated] (BEAM-5732) expose runner mode to user through samza pipeline option
[ https://issues.apache.org/jira/browse/BEAM-5732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Chen updated BEAM-5732: --- Description: We should expose runner mode to user through samza pipeline option so that user can decide whether to start samza job as local mode, clustered mode (YARN) or standalone mode (Zookeeper). This should work consistently in both Java runner and Portable runner. was: We should expose runner mode to user through samza pipeline option so that user can decide whether to start samza job as local mode or remote mode. This should work consistently in both Java runner and Portable runner. > expose runner mode to user through samza pipeline option > > > Key: BEAM-5732 > URL: https://issues.apache.org/jira/browse/BEAM-5732 > Project: Beam > Issue Type: Improvement > Components: runner-samza >Reporter: Hai Lu >Assignee: Daniel Chen >Priority: Major > > We should expose runner mode to user through samza pipeline option so that > user can decide whether to start samza job as local mode, clustered mode > (YARN) or standalone mode (Zookeeper). > This should work consistently in both Java runner and Portable runner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749363#comment-16749363 ] Daniel Oliveira commented on BEAM-6491: --- Just as an update, I made a PR setting the test to be ignored until this flake is fixed. > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: Bug > Components: java-fn-execution, test-failures >Reporter: Alex Amato >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown
[jira] [Resolved] (BEAM-6386) Add named variant of PTransform::compose
[
https://issues.apache.org/jira/browse/BEAM-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Klukas resolved BEAM-6386.
---
Resolution: Fixed
Fix Version/s: 2.11.0
> Add named variant of PTransform::compose
>
>
> Key: BEAM-6386
> URL: https://issues.apache.org/jira/browse/BEAM-6386
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
>Reporter: Jeff Klukas
>Assignee: Jeff Klukas
>Priority: Minor
> Fix For: 2.11.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> BEAM- 5413 introduced PTransform::compose as a concise way of creating a
> composition of transforms as a lambda. We should add a variant to allow
> specifying a name for the returned transform in the same way that {{apply}}
> can take an explicit name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188574=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188574
]
ASF GitHub Bot logged work on BEAM-6489:
Author: ASF GitHub Bot
Created on: 23/Jan/19 00:58
Start Date: 23/Jan/19 00:58
Worklog Time Spent: 10m
Work Description: markflyhigh commented on issue #7594: [BEAM-6489] Ping
pip to 18.1 due to process-dependency-links deprecated in 19.0
URL: https://github.com/apache/beam/pull/7594#issuecomment-456624870
looks like the change should be made in `tox.ini` by adding to dep field of
each env.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188574)
Time Spent: 0.5h (was: 20m)
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188571=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188571
]
ASF GitHub Bot logged work on BEAM-6489:
Author: ASF GitHub Bot
Created on: 23/Jan/19 00:54
Start Date: 23/Jan/19 00:54
Worklog Time Spent: 10m
Work Description: markflyhigh commented on issue #7594: [BEAM-6489] Ping
pip to 18.1 due to process-dependency-links deprecated in 19.0
URL: https://github.com/apache/beam/pull/7594#issuecomment-456624107
This change will not fix the existing `--process-dependency-links` issue
since the pip we should downgrade is installed by tox as part of tox run.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188571)
Time Spent: 20m (was: 10m)
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6161) Add ElementCount MonitoringInfos for the Java SDK
[ https://issues.apache.org/jira/browse/BEAM-6161?focusedWorklogId=188569=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188569 ] ASF GitHub Bot logged work on BEAM-6161: Author: ASF GitHub Bot Created on: 23/Jan/19 00:51 Start Date: 23/Jan/19 00:51 Worklog Time Spent: 10m Work Description: ajamato commented on issue #7272: [BEAM-6161] Introduce PCollectionConsumerRegistry and add ElementCoun… URL: https://github.com/apache/beam/pull/7272#issuecomment-456623343 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188569) Time Spent: 9h 50m (was: 9h 40m) > Add ElementCount MonitoringInfos for the Java SDK > - > > Key: BEAM-6161 > URL: https://issues.apache.org/jira/browse/BEAM-6161 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution, sdk-java-harness >Reporter: Alex Amato >Assignee: Alex Amato >Priority: Major > Time Spent: 9h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6161) Add ElementCount MonitoringInfos for the Java SDK
[ https://issues.apache.org/jira/browse/BEAM-6161?focusedWorklogId=188570=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188570 ] ASF GitHub Bot logged work on BEAM-6161: Author: ASF GitHub Bot Created on: 23/Jan/19 00:51 Start Date: 23/Jan/19 00:51 Worklog Time Spent: 10m Work Description: ajamato commented on issue #7272: [BEAM-6161] Introduce PCollectionConsumerRegistry and add ElementCoun… URL: https://github.com/apache/beam/pull/7272#issuecomment-456623369 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188570) Time Spent: 10h (was: 9h 50m) > Add ElementCount MonitoringInfos for the Java SDK > - > > Key: BEAM-6161 > URL: https://issues.apache.org/jira/browse/BEAM-6161 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution, sdk-java-harness >Reporter: Alex Amato >Assignee: Alex Amato >Priority: Major > Time Spent: 10h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?focusedWorklogId=188568=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188568 ] ASF GitHub Bot logged work on BEAM-6491: Author: ASF GitHub Bot Created on: 23/Jan/19 00:43 Start Date: 23/Jan/19 00:43 Worklog Time Spent: 10m Work Description: youngoli commented on issue #7595: [BEAM-6491] Ignore flaky test: FileIOTest.testMatchWatchForNewFiles URL: https://github.com/apache/beam/pull/7595#issuecomment-456621781 R: @swegner CC: @ajamato This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188568) Time Spent: 20m (was: 10m) > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: Bug > Components: java-fn-execution, test-failures >Reporter: Alex Amato >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at >
[jira] [Work logged] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?focusedWorklogId=188567=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188567 ] ASF GitHub Bot logged work on BEAM-6491: Author: ASF GitHub Bot Created on: 23/Jan/19 00:42 Start Date: 23/Jan/19 00:42 Worklog Time Spent: 10m Work Description: youngoli commented on pull request #7595: [BEAM-6491] Ignore flaky test: FileIOTest.testMatchWatchForNewFiles URL: https://github.com/apache/beam/pull/7595 This test was only recently enabled so I don't think it's a big deal to ignore this until the flakiness is fixed. Follow this checklist to help us incorporate your contribution quickly and easily: - [x] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it. Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | --- | --- | --- | --- Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | --- | --- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188567) Time Spent:
[jira] [Work logged] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[ https://issues.apache.org/jira/browse/BEAM-6489?focusedWorklogId=188565=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188565 ] ASF GitHub Bot logged work on BEAM-6489: Author: ASF GitHub Bot Created on: 23/Jan/19 00:37 Start Date: 23/Jan/19 00:37 Worklog Time Spent: 10m Work Description: markflyhigh commented on pull request #7594: [BEAM-6489] Ping pip to 18.1 due to process-dependency-links deprecated in 19.0 URL: https://github.com/apache/beam/pull/7594 Fix Python tests before we know the better way to handle `--process-dependency-links` flag in 19.0. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it. Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | --- | --- | --- | --- Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | --- | --- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188565) Time Spent:
[jira] [Updated] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Oliveira updated BEAM-6491: -- Issue Type: Bug (was: New Feature) > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: Bug > Components: java-fn-execution, test-failures >Reporter: Alex Amato >Priority: Major > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at
[jira] [Updated] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Oliveira updated BEAM-6491: -- Component/s: test-failures > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution, test-failures >Reporter: Alex Amato >Priority: Major > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at
[jira] [Commented] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749330#comment-16749330
]
Valentyn Tymofieiev commented on BEAM-6489:
---
Yes, that would very well explain the nature of the failure. We had to add a
--process-dependecy-links flag for purposes of Python3 conversion in
https://github.com/apache/beam/pull/7104, to pick up an unreleased change in
dill. On Jan 21, there was a dill 2.9 release, so I believe we no longer need
this flag, and I'll send a PR to clean it up.
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749324#comment-16749324
]
Udi Meiri commented on BEAM-6489:
-
[~markflyhigh] believes this is due to a new version of pip:
https://github.com/pypa/pip/issues/6162
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749322#comment-16749322 ] Daniel Oliveira commented on BEAM-6491: --- I wonder if this is related to BEAM-6352 > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution >Reporter: Alex Amato >Priority: Major > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at
[jira] [Work logged] (BEAM-6017) Support nanosecond precision for DATETIME in Schema
[ https://issues.apache.org/jira/browse/BEAM-6017?focusedWorklogId=188554=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188554 ] ASF GitHub Bot logged work on BEAM-6017: Author: ASF GitHub Bot Created on: 22/Jan/19 23:58 Start Date: 22/Jan/19 23:58 Worklog Time Spent: 10m Work Description: stale[bot] commented on pull request #6991: [BEAM-6017] [WIP] Support nanosecond precision for DateTime field in Row URL: https://github.com/apache/beam/pull/6991 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188554) Time Spent: 2.5h (was: 2h 20m) > Support nanosecond precision for DATETIME in Schema > --- > > Key: BEAM-6017 > URL: https://issues.apache.org/jira/browse/BEAM-6017 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Time Spent: 2.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6017) Support nanosecond precision for DATETIME in Schema
[ https://issues.apache.org/jira/browse/BEAM-6017?focusedWorklogId=188553=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188553 ] ASF GitHub Bot logged work on BEAM-6017: Author: ASF GitHub Bot Created on: 22/Jan/19 23:58 Start Date: 22/Jan/19 23:58 Worklog Time Spent: 10m Work Description: stale[bot] commented on issue #6991: [BEAM-6017] [WIP] Support nanosecond precision for DateTime field in Row URL: https://github.com/apache/beam/pull/6991#issuecomment-456612358 This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188553) Time Spent: 2h 20m (was: 2h 10m) > Support nanosecond precision for DATETIME in Schema > --- > > Key: BEAM-6017 > URL: https://issues.apache.org/jira/browse/BEAM-6017 > Project: Beam > Issue Type: New Feature > Components: sdk-java-core >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749303#comment-16749303 ] Alex Amato commented on BEAM-6491: -- Though, simply moving the writer.join() statement upward, may defeat the purpose of the test, which seems to intend to see what happens while the pipeline is running and new files are written. I am wondering if there is a more clever AssertThat which can be used to add custom code which runs inside the pipeline in order to populate the expected values inside a function? Then it can run after the test finishes and insert its check after the pipeline has finished (and extracting the modified times as well) > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution >Reporter: Alex Amato >Priority: Major > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at >
[jira] [Work logged] (BEAM-6457) bigquery.py is too large, and some tools are better moved elsewhere
[ https://issues.apache.org/jira/browse/BEAM-6457?focusedWorklogId=188552=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188552 ] ASF GitHub Bot logged work on BEAM-6457: Author: ASF GitHub Bot Created on: 22/Jan/19 23:57 Start Date: 22/Jan/19 23:57 Worklog Time Spent: 10m Work Description: pabloem commented on pull request #7542: [BEAM-6457] Refactoring of a few BigQuery classes. URL: https://github.com/apache/beam/pull/7542#discussion_r250008609 ## File path: sdks/python/apache_beam/io/gcp/bigquery.py ## @@ -165,38 +148,6 @@ 'WriteToBigQuery', ] -JSON_COMPLIANCE_ERROR = 'NAN, INF and -INF values are not JSON compliant.' -MAX_RETRIES = 3 - - -def default_encoder(obj): - if isinstance(obj, decimal.Decimal): -return str(obj) - raise TypeError( - "Object of type '%s' is not JSON serializable" % type(obj).__name__) - - -class RowAsDictJsonCoder(coders.Coder): Review comment: I made sure to avoid moving any methods defined in `__all__`. Isn't `__all__` a reasonable implicit include/exclude of backward compatibility guarantees? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188552) Time Spent: 2h (was: 1h 50m) > bigquery.py is too large, and some tools are better moved elsewhere > --- > > Key: BEAM-6457 > URL: https://issues.apache.org/jira/browse/BEAM-6457 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Pablo Estrada >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > Need to do a bit of refactoring of that file -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-6470) python tenacity is breaks the Python validates runner flink tests
[ https://issues.apache.org/jira/browse/BEAM-6470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Goenka closed BEAM-6470. -- Resolution: Won't Fix Fix Version/s: Not applicable > python tenacity is breaks the Python validates runner flink tests > - > > Key: BEAM-6470 > URL: https://issues.apache.org/jira/browse/BEAM-6470 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Fix For: Not applicable > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Python tenacity is only added for test cases. > Beam VR tests executes actual pipelines which do not have tenacity. > We already have apache_beam.util.retry which has the retry functionality so > replacing tenacity with it will solve this problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-6266) Newest AWS SDK causes errors on startup without usage
[
https://issues.apache.org/jira/browse/BEAM-6266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749310#comment-16749310
]
Mike Kaplinskiy commented on BEAM-6266:
---
I could take a stab at it. The solution I'm thinking of is to make
{{this.amazonS3 = null;}} when the right flags aren't provided. We can then
{{Preconditions.checkState}} in all the APIs to give a good error message. Does
that sound reasonable [~iemejia]?
> Newest AWS SDK causes errors on startup without usage
> -
>
> Key: BEAM-6266
> URL: https://issues.apache.org/jira/browse/BEAM-6266
> Project: Beam
> Issue Type: Bug
> Components: io-java-aws
>Affects Versions: 2.6.0, 2.7.0, 2.8.0, 2.9.0
>Reporter: Mike Kaplinskiy
>Assignee: Ismaël Mejía
>Priority: Major
>
> The S3 filesystem implementation in Beam logs a message like this if it is
> linked in:
> {code:java}
> 2018-12-18 22:17:55 INFO S3FileSystem:104 - The AWS S3 Beam extension was
> included in this build, but the awsRegion flag was not specified. If you
> don't plan to use S3, then ignore this message.{code}
> Previous versions of the AWS libraries seemed to allow you to {{ignore this
> message}} , but the newest libraries throw an exception:
> {code:java}
> org.apache.beam.sdk.Pipeline.create Pipeline.java: 145
> org.apache.beam.sdk.PipelineRunner.fromOptions PipelineRunner.java: 47
> org.apache.beam.sdk.io.FileSystems.setDefaultPipelineOptions
> FileSystems.java: 482
> org.apache.beam.sdk.io.FileSystems.verifySchemesAreUnique FileSystems.java:
> 492
> org.apache.beam.sdk.io.aws.s3.S3FileSystemRegistrar.fromOptions
> S3FileSystemRegistrar.java: 39
> org.apache.beam.sdk.io.aws.s3.S3FileSystem. S3FileSystem.java: 108
> org.apache.beam.sdk.io.aws.s3.S3FileSystem.buildAmazonS3Client
> S3FileSystem.java: 122
> com.amazonaws.client.builder.AwsClientBuilder.withRegion
> AwsClientBuilder.java: 245
> com.amazonaws.client.builder.AwsClientBuilder.getRegionObject
> AwsClientBuilder.java: 258
> com.amazonaws.SdkClientException: Could not find region information for
> 'null' in SDK metadata.
> retryable: true{code}
> (this stack trace is from 2.6.0, but master is also affected)
>
> The root cause is this code that gets run unconditionally
> (https://github.com/apache/beam/blob/master/sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/s3/DefaultS3ClientBuilderFactory.java#L42):
> {code:java}
> builder = builder.withRegion(s3Options.getAwsRegion());{code}
> Unfortunately the latest AWS library will throw if you pass {{null}} to
> {{withRegion}}. It previously did nothing. This release of the AWS java sdk
> added the error check:
> [https://github.com/aws/aws-sdk-java/commit/8f07cc35eec9047f7cfdbc7de3abc6d4327b08d0]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (BEAM-6470) python tenacity is breaks the Python validates runner flink tests
[ https://issues.apache.org/jira/browse/BEAM-6470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749306#comment-16749306 ] Ankur Goenka commented on BEAM-6470: Thanks! Will mark it won't fix for now. > python tenacity is breaks the Python validates runner flink tests > - > > Key: BEAM-6470 > URL: https://issues.apache.org/jira/browse/BEAM-6470 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 3h 20m > Remaining Estimate: 0h > > Python tenacity is only added for test cases. > Beam VR tests executes actual pipelines which do not have tenacity. > We already have apache_beam.util.retry which has the retry functionality so > replacing tenacity with it will solve this problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6457) bigquery.py is too large, and some tools are better moved elsewhere
[ https://issues.apache.org/jira/browse/BEAM-6457?focusedWorklogId=188551=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188551 ] ASF GitHub Bot logged work on BEAM-6457: Author: ASF GitHub Bot Created on: 22/Jan/19 23:52 Start Date: 22/Jan/19 23:52 Worklog Time Spent: 10m Work Description: chamikaramj commented on pull request #7542: [BEAM-6457] Refactoring of a few BigQuery classes. URL: https://github.com/apache/beam/pull/7542#discussion_r250007509 ## File path: sdks/python/apache_beam/io/gcp/bigquery.py ## @@ -165,38 +148,6 @@ 'WriteToBigQuery', ] -JSON_COMPLIANCE_ERROR = 'NAN, INF and -INF values are not JSON compliant.' -MAX_RETRIES = 3 - - -def default_encoder(obj): - if isinstance(obj, decimal.Decimal): -return str(obj) - raise TypeError( - "Object of type '%s' is not JSON serializable" % type(obj).__name__) - - -class RowAsDictJsonCoder(coders.Coder): Review comment: Unfortunately I think this (and some of the other changes below) are backwards incompatible changes to the public API. Can we leave stubs to those methods with deprecation warnings so that we don't break backwards compatibility ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188551) Time Spent: 1h 50m (was: 1h 40m) > bigquery.py is too large, and some tools are better moved elsewhere > --- > > Key: BEAM-6457 > URL: https://issues.apache.org/jira/browse/BEAM-6457 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Pablo Estrada >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > > Need to do a bit of refactoring of that file -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749299#comment-16749299 ] Alex Amato commented on BEAM-6491: -- his PR introduces the threading style (no problem yet, since we aren't trying to access the file in the expected check) https://github.com/apache/beam/commit/910d02fb464d8a91c7149f214419964cb834639b However, the check for the last modified time is added in this PR which introduces the race condition https://github.com/apache/beam/commit/410d6c7b5f933dcb0280894553c1e576ee4e4884 > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution >Reporter: Alex Amato >Priority: Major > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at >
[jira] [Commented] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749290#comment-16749290 ] Alex Amato commented on BEAM-6491: -- I don't fully understand the "writer Thread" in testMatchWatchForNewFiles I think that the join in this test should moved up? In this test we are tryiong to access the file and its modified time when populating the "expected" values, but there is no guarantee that the file exists yet, or is done being modified. > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution >Reporter: Alex Amato >Priority: Major > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
[jira] [Updated] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Gryzykhin updated BEAM-6489:
Component/s: test-failures
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Gryzykhin updated BEAM-6489:
Labels: currently-failing (was: )
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
> Labels: currently-failing
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (BEAM-6490) Python precommit fail with no such option: --process-dependency-links
[
https://issues.apache.org/jira/browse/BEAM-6490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Gryzykhin resolved BEAM-6490.
-
Resolution: Duplicate
Fix Version/s: Not applicable
> Python precommit fail with no such option: --process-dependency-links
> -
>
> Key: BEAM-6490
> URL: https://issues.apache.org/jira/browse/BEAM-6490
> Project: Beam
> Issue Type: New Feature
> Components: test-failures
>Reporter: Mikhail Gryzykhin
>Assignee: Mikhail Gryzykhin
>Priority: Major
> Labels: currently-failing
> Fix For: Not applicable
>
>
> Sample job
> [https://builds.apache.org/job/beam_PreCommit_Python_Phrase/152/console]
> Error:
> *14:18:28* GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py*14:18:28*
> py3 create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3*14:18:29*
> py3 installdeps: future==0.16.0*14:18:29* ERROR: invocation failed (exit
> code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/log/py3-1.log*14:18:29*
> ERROR: actionid: py3*14:18:29* msg: getenv*14:18:29* cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links',
> 'future==0.16.0']*14:18:29* *14:18:29* *14:18:29* Usage: *14:18:29* pip
> install [options] [package-index-options]
> ...*14:18:29* pip install [options] -r
> [package-index-options] ...*14:18:29* pip install [options] [-e] project url> ...*14:18:29* pip install [options] [-e]
> ...*14:18:29* pip install [options] ...*14:18:29*
> *14:18:29* no such option: --process-dependency-links*14:18:29* *14:18:29*
> ERROR: could not install deps [future==0.16.0]; v =
> InvocationError('/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/bin/python
>
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/bin/pip
> install --retries 10 --process-dependency-links future==0.16.0 (see
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/log/py3-1.log)',
> 2)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6468) Cannot create empty TestBoundedTable
[ https://issues.apache.org/jira/browse/BEAM-6468?focusedWorklogId=188545=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188545 ] ASF GitHub Bot logged work on BEAM-6468: Author: ASF GitHub Bot Created on: 22/Jan/19 23:34 Start Date: 22/Jan/19 23:34 Worklog Time Spent: 10m Work Description: amaliujia commented on issue #7568: [BEAM-6468] Allow creating empty TestBoundedTable URL: https://github.com/apache/beam/pull/7568#issuecomment-456607222 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188545) Time Spent: 1h 50m (was: 1h 40m) > Cannot create empty TestBoundedTable > > > Key: BEAM-6468 > URL: https://issues.apache.org/jira/browse/BEAM-6468 > Project: Beam > Issue Type: Bug > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749279#comment-16749279 ] Alex Amato commented on BEAM-6491: -- Recent PRs for the test file: https://github.com/apache/beam/commit/2bf5eadff02f9effdd299c867d6e476594ddeb22#diff-9a483e071fce83aa7be513e8b8ffdba1 > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution >Reporter: Alex Amato >Priority: Major > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) at >
[jira] [Commented] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
[ https://issues.apache.org/jira/browse/BEAM-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749286#comment-16749286 ] Alex Amato commented on BEAM-6491: -- There are a lot of sleeps in this test, and its using multiple threads. perhaps it needs to use some tools to coordinate them > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java > presubmit > > > Key: BEAM-6491 > URL: https://issues.apache.org/jira/browse/BEAM-6491 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution >Reporter: Alex Amato >Priority: Major > > I've seen this fail in a few different PRs for different contributors > [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] > java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at > sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) > at > sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) > at > sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) > at java.nio.file.Files.readAttributes(Files.java:1737) at > java.nio.file.Files.getLastModifiedTime(Files.java:2266) at > org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at > org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at > org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at > org.junit.runners.ParentRunner.run(ParentRunner.java:396) at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) > at > org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) at >
[jira] [Created] (BEAM-6490) Python precommit fail with no such option: --process-dependency-links
Mikhail Gryzykhin created BEAM-6490:
---
Summary: Python precommit fail with no such option:
--process-dependency-links
Key: BEAM-6490
URL: https://issues.apache.org/jira/browse/BEAM-6490
Project: Beam
Issue Type: New Feature
Components: test-failures
Reporter: Mikhail Gryzykhin
Assignee: Mikhail Gryzykhin
Sample job
[https://builds.apache.org/job/beam_PreCommit_Python_Phrase/152/console]
Error:
*14:18:28* GLOB sdist-make:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py*14:18:28*
py3 create:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3*14:18:29*
py3 installdeps: future==0.16.0*14:18:29* ERROR: invocation failed (exit code
2), logfile:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/log/py3-1.log*14:18:29*
ERROR: actionid: py3*14:18:29* msg: getenv*14:18:29* cmdargs:
['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/bin/python',
'/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/bin/pip',
'install', '--retries', '10', '--process-dependency-links',
'future==0.16.0']*14:18:29* *14:18:29* *14:18:29* Usage: *14:18:29* pip
install [options] [package-index-options] ...*14:18:29*
pip install [options] -r [package-index-options]
...*14:18:29* pip install [options] [-e] ...*14:18:29*
pip install [options] [-e] ...*14:18:29* pip install
[options] ...*14:18:29* *14:18:29* no such option:
--process-dependency-links*14:18:29* *14:18:29* ERROR: could not install deps
[future==0.16.0]; v =
InvocationError('/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/bin/python
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/bin/pip
install --retries 10 --process-dependency-links future==0.16.0 (see
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py3/log/py3-1.log)',
2)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[
https://issues.apache.org/jira/browse/BEAM-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Valentyn Tymofieiev updated BEAM-6489:
--
Description:
{noformat}
13:35:37 GLOB sdist-make:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
13:35:37 docs create:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
13:35:39 ERROR: invocation failed (exit code 2), logfile:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
13:35:39 ERROR: actionid: docs
13:35:39 msg: getenv
13:35:39 cmdargs:
['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
'/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
'sphinx_rtd_theme==0.2.4']
13:35:39
13:35:39
13:35:39 Usage:
13:35:39 pip install [options]
[package-index-options] ...
13:35:39 pip install [options] -r [package-index-options]
...
13:35:39 pip install [options] [-e] ...
13:35:39 pip install [options] [-e] ...
13:35:39 pip install [options] ...
13:35:39
13:35:39 no such option: --process-dependency-links
{noformat}
was:
13:35:37 GLOB sdist-make:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
13:35:37 docs create:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
13:35:39 ERROR: invocation failed (exit code 2), logfile:
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
13:35:39 ERROR: actionid: docs
13:35:39 msg: getenv
13:35:39 cmdargs:
['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
'/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
'sphinx_rtd_theme==0.2.4']
13:35:39
13:35:39
13:35:39 Usage:
13:35:39 pip install [options]
[package-index-options] ...
13:35:39 pip install [options] -r [package-index-options]
...
13:35:39 pip install [options] [-e] ...
13:35:39 pip install [options] [-e] ...
13:35:39 pip install [options] ...
13:35:39
13:35:39 no such option: --process-dependency-links
> Python precommits are failiing due to a pip error: "no such option:
> --process-dependency-links"
>
>
> Key: BEAM-6489
> URL: https://issues.apache.org/jira/browse/BEAM-6489
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Blocker
>
> {noformat}
> 13:35:37 GLOB sdist-make:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py
> 13:35:37 docs create:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs
> 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4
> 13:35:39 ERROR: invocation failed (exit code 2), logfile:
> /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log
> 13:35:39 ERROR: actionid: docs
> 13:35:39 msg: getenv
> 13:35:39 cmdargs:
> ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python',
>
> '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip',
> 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5',
> 'sphinx_rtd_theme==0.2.4']
> 13:35:39
> 13:35:39
> 13:35:39 Usage:
> 13:35:39 pip install [options]
> [package-index-options] ...
> 13:35:39 pip install [options] -r
> [package-index-options] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] [-e] ...
> 13:35:39 pip install [options] ...
> 13:35:39
> 13:35:39 no such option: --process-dependency-links
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
[ https://issues.apache.org/jira/browse/BEAM-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-6489: -- Description: 13:35:37 GLOB sdist-make: /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py 13:35:37 docs create: /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4 13:35:39 ERROR: invocation failed (exit code 2), logfile: /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log 13:35:39 ERROR: actionid: docs 13:35:39 msg: getenv 13:35:39 cmdargs: ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python', '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip', 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5', 'sphinx_rtd_theme==0.2.4'] 13:35:39 13:35:39 13:35:39 Usage: 13:35:39 pip install [options] [package-index-options] ... 13:35:39 pip install [options] -r [package-index-options] ... 13:35:39 pip install [options] [-e] ... 13:35:39 pip install [options] [-e] ... 13:35:39 pip install [options] ... 13:35:39 13:35:39 no such option: --process-dependency-links was:*14:32:41* cmdargs: ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py27-cython/bin/python', '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py27-cython/bin/pip', 'install', '--retries', '10', '--process-dependency-links', 'cython==0.28.1', 'future==0.16.0']*14:32:41* *14:32:41* *14:32:41* Usage: *14:32:41* pip install [options] [package-index-options] ...*14:32:41* pip install [options] -r [package-index-options] ...*14:32:41* pip install [options] [-e] ...*14:32:41* pip install [options] [-e] ...*14:32:41* pip install [options] ...*14:32:41* *14:32:41* no such option: --process-dependency-links > Python precommits are failiing due to a pip error: "no such option: > --process-dependency-links" > > > Key: BEAM-6489 > URL: https://issues.apache.org/jira/browse/BEAM-6489 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Valentyn Tymofieiev >Assignee: Valentyn Tymofieiev >Priority: Blocker > > 13:35:37 GLOB sdist-make: > /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/setup.py > 13:35:37 docs create: > /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs > 13:35:39 docs installdeps: Sphinx==1.6.5, sphinx_rtd_theme==0.2.4 > 13:35:39 ERROR: invocation failed (exit code 2), logfile: > /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/log/docs-1.log > 13:35:39 ERROR: actionid: docs > 13:35:39 msg: getenv > 13:35:39 cmdargs: > ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/python', > > '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/docs/bin/pip', > 'install', '--retries', '10', '--process-dependency-links', 'Sphinx==1.6.5', > 'sphinx_rtd_theme==0.2.4'] > 13:35:39 > 13:35:39 > 13:35:39 Usage: > 13:35:39 pip install [options] > [package-index-options] ... > 13:35:39 pip install [options] -r > [package-index-options] ... > 13:35:39 pip install [options] [-e] ... > 13:35:39 pip install [options] [-e] ... > 13:35:39 pip install [options] ... > 13:35:39 > 13:35:39 no such option: --process-dependency-links -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-6470) python tenacity is breaks the Python validates runner flink tests
[ https://issues.apache.org/jira/browse/BEAM-6470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749274#comment-16749274 ] Maximilian Michels commented on BEAM-6470: -- We can, though it doesn't seem to be necessary. The main cause of the failure were the missing test dependencies. If you want to fix this, fine with me, otherwise I'd mark this with Won't Fix. > python tenacity is breaks the Python validates runner flink tests > - > > Key: BEAM-6470 > URL: https://issues.apache.org/jira/browse/BEAM-6470 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 3h 20m > Remaining Estimate: 0h > > Python tenacity is only added for test cases. > Beam VR tests executes actual pipelines which do not have tenacity. > We already have apache_beam.util.retry which has the retry functionality so > replacing tenacity with it will solve this problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-6491) org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit
Alex Amato created BEAM-6491: Summary: org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles flakey in java presubmit Key: BEAM-6491 URL: https://issues.apache.org/jira/browse/BEAM-6491 Project: Beam Issue Type: New Feature Components: java-fn-execution Reporter: Alex Amato I've seen this fail in a few different PRs for different contributors [https://builds.apache.org/job/beam_PreCommit_Java_Commit/3688/testReport/org.apache.beam.sdk.io/FileIOTest/testMatchWatchForNewFiles/] java.nio.file.NoSuchFileException: /tmp/junit7417716876899005708/watch/first at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55) at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144) at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) at java.nio.file.Files.readAttributes(Files.java:1737) at java.nio.file.Files.getLastModifiedTime(Files.java:2266) at org.apache.beam.sdk.io.FileIOTest.testMatchWatchForNewFiles(FileIOTest.java:243) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.apache.beam.sdk.testing.TestPipeline$1.evaluate(TestPipeline.java:319) at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:265) at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:349) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:314) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:312) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292) at org.junit.runners.ParentRunner.run(ParentRunner.java:396) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:106) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58) at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38) at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:66) at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:117) at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:155) at
[jira] [Created] (BEAM-6489) Python precommits are failiing due to a pip error: "no such option: --process-dependency-links"
Valentyn Tymofieiev created BEAM-6489: - Summary: Python precommits are failiing due to a pip error: "no such option: --process-dependency-links" Key: BEAM-6489 URL: https://issues.apache.org/jira/browse/BEAM-6489 Project: Beam Issue Type: Bug Components: sdk-py-core Reporter: Valentyn Tymofieiev Assignee: Valentyn Tymofieiev *14:32:41* cmdargs: ['/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py27-cython/bin/python', '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/target/.tox/py27-cython/bin/pip', 'install', '--retries', '10', '--process-dependency-links', 'cython==0.28.1', 'future==0.16.0']*14:32:41* *14:32:41* *14:32:41* Usage: *14:32:41* pip install [options] [package-index-options] ...*14:32:41* pip install [options] -r [package-index-options] ...*14:32:41* pip install [options] [-e] ...*14:32:41* pip install [options] [-e] ...*14:32:41* pip install [options] ...*14:32:41* *14:32:41* no such option: --process-dependency-links -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6269) Support mixed-language pipelines.
[
https://issues.apache.org/jira/browse/BEAM-6269?focusedWorklogId=188534=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188534
]
ASF GitHub Bot logged work on BEAM-6269:
Author: ASF GitHub Bot
Created on: 22/Jan/19 23:05
Start Date: 22/Jan/19 23:05
Worklog Time Spent: 10m
Work Description: kennknowles commented on pull request #7316:
[BEAM-6269] Cross-SDK transform expansion protocol.
URL: https://github.com/apache/beam/pull/7316#discussion_r249995295
##
File path:
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/SdkComponents.java
##
@@ -239,10 +273,10 @@ public String getOnlyEnvironmentId() {
}
private String uniqify(String baseName, Set existing) {
-String name = baseName;
+String name = namespace + baseName;
Review comment:
Unless it is hideously expensive, it might be smart to choose a canonical
meta-format now (that there are three components) and escape the components.
For example slash-separated with slashes internal to the name escaped. This is
mostly a reaction to more strings being concatenated in a way that is starting
to look like it should be real structure.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188534)
Time Spent: 3h 20m (was: 3h 10m)
> Support mixed-language pipelines.
> -
>
> Key: BEAM-6269
> URL: https://issues.apache.org/jira/browse/BEAM-6269
> Project: Beam
> Issue Type: New Feature
> Components: beam-model, runner-core, sdk-java-core, sdk-py-core
>Reporter: Robert Bradshaw
>Assignee: Kenneth Knowles
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6269) Support mixed-language pipelines.
[
https://issues.apache.org/jira/browse/BEAM-6269?focusedWorklogId=188532=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188532
]
ASF GitHub Bot logged work on BEAM-6269:
Author: ASF GitHub Bot
Created on: 22/Jan/19 23:05
Start Date: 22/Jan/19 23:05
Worklog Time Spent: 10m
Work Description: kennknowles commented on pull request #7316:
[BEAM-6269] Cross-SDK transform expansion protocol.
URL: https://github.com/apache/beam/pull/7316#discussion_r249996455
##
File path: model/job-management/src/main/proto/beam_expansion_api.proto
##
@@ -0,0 +1,68 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+/*
+ * Protocol Buffers describing the Expansion API, an api for expanding
+ * transforms in a remote SDK.
+ */
+
+syntax = "proto3";
+
+package org.apache.beam.model.expansion.v1;
+
+option go_package = "construction_v1";
+option java_package = "org.apache.beam.model.expansion.v1";
+option java_outer_classname = "ExpansionApi";
+
+import "beam_runner_api.proto";
+
+message ExpansionRequest {
+ // Set of components needed to interpret the transform, or which
+ // may be useful for its expansion. This includes the input
+ // PCollections (if any) to the to-be-expanded transform, along
+ // with their coders and windowing strategies.
+ org.apache.beam.model.pipeline.v1.Components components = 1;
+
+ // The actual PTransform to be expaneded according to its spec.
+ // Its input should be set, but its subtransforms and outputs
+ // should not be.
+ org.apache.beam.model.pipeline.v1.PTransform transform = 2;
+
+ // A namespace (prefix) to use for the id of any newly created
+ // components.
+ string namespace = 3;
Review comment:
Worth noting that this is not necessary. The whole idea of components is
that they never have meaning outside the message they are contained in, and are
just a way of encoding a graph. It is fair to critique that the copying and
re-allocating ids has computational cost. The last time we relied on ids having
significance outside a single message we hit bugs and migration pain. Those are
expected consequences IMO.
Here, it seems that it is not so bad. IIUC you are just using this namespace
as a way to cheaply reuse components, but not establishing any cross-message
dependence upon them.
But it is worth noting that you can do a role-reversal and save yourself
this field: the expansion service can freely prefix all the _other_ ids with
something to avoid collisions with its chosen namespace.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188532)
Time Spent: 3h (was: 2h 50m)
> Support mixed-language pipelines.
> -
>
> Key: BEAM-6269
> URL: https://issues.apache.org/jira/browse/BEAM-6269
> Project: Beam
> Issue Type: New Feature
> Components: beam-model, runner-core, sdk-java-core, sdk-py-core
>Reporter: Robert Bradshaw
>Assignee: Kenneth Knowles
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6269) Support mixed-language pipelines.
[
https://issues.apache.org/jira/browse/BEAM-6269?focusedWorklogId=188533=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188533
]
ASF GitHub Bot logged work on BEAM-6269:
Author: ASF GitHub Bot
Created on: 22/Jan/19 23:05
Start Date: 22/Jan/19 23:05
Worklog Time Spent: 10m
Work Description: kennknowles commented on pull request #7316:
[BEAM-6269] Cross-SDK transform expansion protocol.
URL: https://github.com/apache/beam/pull/7316#discussion_r249994859
##
File path:
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/SdkComponents.java
##
@@ -81,9 +101,13 @@ public static SdkComponents create(PipelineOptions
options) {
return sdkComponents;
}
- private SdkComponents() {}
+ private SdkComponents(String namespace) {
+this.namespace = namespace;
Review comment:
Document the namespace idea?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188533)
Time Spent: 3h 10m (was: 3h)
> Support mixed-language pipelines.
> -
>
> Key: BEAM-6269
> URL: https://issues.apache.org/jira/browse/BEAM-6269
> Project: Beam
> Issue Type: New Feature
> Components: beam-model, runner-core, sdk-java-core, sdk-py-core
>Reporter: Robert Bradshaw
>Assignee: Kenneth Knowles
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6468) Cannot create empty TestBoundedTable
[ https://issues.apache.org/jira/browse/BEAM-6468?focusedWorklogId=188530=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188530 ] ASF GitHub Bot logged work on BEAM-6468: Author: ASF GitHub Bot Created on: 22/Jan/19 22:59 Start Date: 22/Jan/19 22:59 Worklog Time Spent: 10m Work Description: amaliujia commented on issue #7568: [BEAM-6468] Allow creating empty TestBoundedTable URL: https://github.com/apache/beam/pull/7568#issuecomment-456596488 reset test please This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188530) Time Spent: 1h 40m (was: 1.5h) > Cannot create empty TestBoundedTable > > > Key: BEAM-6468 > URL: https://issues.apache.org/jira/browse/BEAM-6468 > Project: Beam > Issue Type: Bug > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6181) Utilize MetricInfo for reporting user metrics in Portable Dataflow Java Runner.
[ https://issues.apache.org/jira/browse/BEAM-6181?focusedWorklogId=188524=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188524 ] ASF GitHub Bot logged work on BEAM-6181: Author: ASF GitHub Bot Created on: 22/Jan/19 22:43 Start Date: 22/Jan/19 22:43 Worklog Time Spent: 10m Work Description: Ardagan commented on issue #7323: [BEAM-6181] Implemented msec counters support in FnApi world. URL: https://github.com/apache/beam/pull/7323#issuecomment-456592205 Update on failures: I ran ./gradlew :beam-runners-direct-java:needsRunnerTests locally and it passes. Seem to be flake. Looking on how can I run python tests. Running :pythonPreCommit fails installing docs deps. Checking how can I repro the error. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188524) Time Spent: 14h 10m (was: 14h) > Utilize MetricInfo for reporting user metrics in Portable Dataflow Java > Runner. > --- > > Key: BEAM-6181 > URL: https://issues.apache.org/jira/browse/BEAM-6181 > Project: Beam > Issue Type: Bug > Components: java-fn-execution >Reporter: Mikhail Gryzykhin >Assignee: Mikhail Gryzykhin >Priority: Major > Time Spent: 14h 10m > Remaining Estimate: 0h > > New approach to report metrics in FnApi is to utilize MetricInfo structures. > This approach is implemented in Python SDK and work is ongoing in Java SDK. > This tasks includes plumbing User metrics reported via MetricInfos through > Dataflow Java Runner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-6170) NexmarkLauncher stall warning causes benchmark failure
[ https://issues.apache.org/jira/browse/BEAM-6170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Whittle resolved BEAM-6170. --- Resolution: Fixed Fix Version/s: 2.10.0 > NexmarkLauncher stall warning causes benchmark failure > -- > > Key: BEAM-6170 > URL: https://issues.apache.org/jira/browse/BEAM-6170 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: Sam Whittle >Assignee: Sam Whittle >Priority: Trivial > Fix For: 2.10.0 > > Original Estimate: 10m > Time Spent: 1.5h > Remaining Estimate: 0h > > The monitoring in the NexmarkLauncher watches the pipeline results and source > for events. If no events are consumed from the source or published to the > output for over > STUCK_WARNING_DELAY it prints an warning message: > "WARNING: streaming query appears to have been stuck for %d min.", > However it also adds this to the errors list, which causes the pipeline to > terminate by throwing an exception. There is a separate > STUCK_TERMINATE_DELAY which is used to terminate the pipeline. It seems > inconsistent to cause the pipeline to fail with the warning timeout. It is > also not 100% accurrate because the pipeline may be busy processing stages > other than the input or output. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6181) Utilize MetricInfo for reporting user metrics in Portable Dataflow Java Runner.
[ https://issues.apache.org/jira/browse/BEAM-6181?focusedWorklogId=188525=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188525 ] ASF GitHub Bot logged work on BEAM-6181: Author: ASF GitHub Bot Created on: 22/Jan/19 22:45 Start Date: 22/Jan/19 22:45 Worklog Time Spent: 10m Work Description: Ardagan commented on issue #7323: [BEAM-6181] Implemented msec counters support in FnApi world. URL: https://github.com/apache/beam/pull/7323#issuecomment-456592635 Rebased on master. See if this was some merge issue. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188525) Time Spent: 14h 20m (was: 14h 10m) > Utilize MetricInfo for reporting user metrics in Portable Dataflow Java > Runner. > --- > > Key: BEAM-6181 > URL: https://issues.apache.org/jira/browse/BEAM-6181 > Project: Beam > Issue Type: Bug > Components: java-fn-execution >Reporter: Mikhail Gryzykhin >Assignee: Mikhail Gryzykhin >Priority: Major > Time Spent: 14h 20m > Remaining Estimate: 0h > > New approach to report metrics in FnApi is to utilize MetricInfo structures. > This approach is implemented in Python SDK and work is ongoing in Java SDK. > This tasks includes plumbing User metrics reported via MetricInfos through > Dataflow Java Runner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6181) Utilize MetricInfo for reporting user metrics in Portable Dataflow Java Runner.
[ https://issues.apache.org/jira/browse/BEAM-6181?focusedWorklogId=188515=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188515 ] ASF GitHub Bot logged work on BEAM-6181: Author: ASF GitHub Bot Created on: 22/Jan/19 22:29 Start Date: 22/Jan/19 22:29 Worklog Time Spent: 10m Work Description: Ardagan commented on issue #7323: [BEAM-6181] Implemented msec counters support in FnApi world. URL: https://github.com/apache/beam/pull/7323#issuecomment-456583437 run python precommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188515) Time Spent: 13h 50m (was: 13h 40m) > Utilize MetricInfo for reporting user metrics in Portable Dataflow Java > Runner. > --- > > Key: BEAM-6181 > URL: https://issues.apache.org/jira/browse/BEAM-6181 > Project: Beam > Issue Type: Bug > Components: java-fn-execution >Reporter: Mikhail Gryzykhin >Assignee: Mikhail Gryzykhin >Priority: Major > Time Spent: 13h 50m > Remaining Estimate: 0h > > New approach to report metrics in FnApi is to utilize MetricInfo structures. > This approach is implemented in Python SDK and work is ongoing in Java SDK. > This tasks includes plumbing User metrics reported via MetricInfos through > Dataflow Java Runner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3286) Go SDK support for portable side input
[ https://issues.apache.org/jira/browse/BEAM-3286?focusedWorklogId=188513=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188513 ] ASF GitHub Bot logged work on BEAM-3286: Author: ASF GitHub Bot Created on: 22/Jan/19 22:24 Start Date: 22/Jan/19 22:24 Worklog Time Spent: 10m Work Description: lostluck commented on issue #7591: [BEAM-3286] Fix side input window coding, & re-enable tests. URL: https://github.com/apache/beam/pull/7591#issuecomment-456586668 R: @aaltay Please review! Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188513) Time Spent: 3h 50m (was: 3h 40m) > Go SDK support for portable side input > -- > > Key: BEAM-3286 > URL: https://issues.apache.org/jira/browse/BEAM-3286 > Project: Beam > Issue Type: Sub-task > Components: sdk-go >Reporter: Henning Rohde >Assignee: Henning Rohde >Priority: Major > Labels: portability > Fix For: 2.8.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6181) Utilize MetricInfo for reporting user metrics in Portable Dataflow Java Runner.
[ https://issues.apache.org/jira/browse/BEAM-6181?focusedWorklogId=188516=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188516 ] ASF GitHub Bot logged work on BEAM-6181: Author: ASF GitHub Bot Created on: 22/Jan/19 22:29 Start Date: 22/Jan/19 22:29 Worklog Time Spent: 10m Work Description: Ardagan commented on issue #7323: [BEAM-6181] Implemented msec counters support in FnApi world. URL: https://github.com/apache/beam/pull/7323#issuecomment-456583397 run java precommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188516) Time Spent: 14h (was: 13h 50m) > Utilize MetricInfo for reporting user metrics in Portable Dataflow Java > Runner. > --- > > Key: BEAM-6181 > URL: https://issues.apache.org/jira/browse/BEAM-6181 > Project: Beam > Issue Type: Bug > Components: java-fn-execution >Reporter: Mikhail Gryzykhin >Assignee: Mikhail Gryzykhin >Priority: Major > Time Spent: 14h > Remaining Estimate: 0h > > New approach to report metrics in FnApi is to utilize MetricInfo structures. > This approach is implemented in Python SDK and work is ongoing in Java SDK. > This tasks includes plumbing User metrics reported via MetricInfos through > Dataflow Java Runner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3286) Go SDK support for portable side input
[ https://issues.apache.org/jira/browse/BEAM-3286?focusedWorklogId=188517=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188517 ] ASF GitHub Bot logged work on BEAM-3286: Author: ASF GitHub Bot Created on: 22/Jan/19 22:29 Start Date: 22/Jan/19 22:29 Worklog Time Spent: 10m Work Description: aaltay commented on pull request #7591: [BEAM-3286] Fix side input window coding, & re-enable tests. URL: https://github.com/apache/beam/pull/7591 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188517) Time Spent: 4h (was: 3h 50m) > Go SDK support for portable side input > -- > > Key: BEAM-3286 > URL: https://issues.apache.org/jira/browse/BEAM-3286 > Project: Beam > Issue Type: Sub-task > Components: sdk-go >Reporter: Henning Rohde >Assignee: Henning Rohde >Priority: Major > Labels: portability > Fix For: 2.8.0 > > Time Spent: 4h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6181) Utilize MetricInfo for reporting user metrics in Portable Dataflow Java Runner.
[ https://issues.apache.org/jira/browse/BEAM-6181?focusedWorklogId=188514=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188514 ] ASF GitHub Bot logged work on BEAM-6181: Author: ASF GitHub Bot Created on: 22/Jan/19 22:29 Start Date: 22/Jan/19 22:29 Worklog Time Spent: 10m Work Description: Ardagan commented on issue #7323: [BEAM-6181] Implemented msec counters support in FnApi world. URL: https://github.com/apache/beam/pull/7323#issuecomment-456588179 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188514) Time Spent: 13h 40m (was: 13.5h) > Utilize MetricInfo for reporting user metrics in Portable Dataflow Java > Runner. > --- > > Key: BEAM-6181 > URL: https://issues.apache.org/jira/browse/BEAM-6181 > Project: Beam > Issue Type: Bug > Components: java-fn-execution >Reporter: Mikhail Gryzykhin >Assignee: Mikhail Gryzykhin >Priority: Major > Time Spent: 13h 40m > Remaining Estimate: 0h > > New approach to report metrics in FnApi is to utilize MetricInfo structures. > This approach is implemented in Python SDK and work is ongoing in Java SDK. > This tasks includes plumbing User metrics reported via MetricInfos through > Dataflow Java Runner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6439) Move Python Flink VR tests to PreCommit
[ https://issues.apache.org/jira/browse/BEAM-6439?focusedWorklogId=188508=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188508 ] ASF GitHub Bot logged work on BEAM-6439: Author: ASF GitHub Bot Created on: 22/Jan/19 22:14 Start Date: 22/Jan/19 22:14 Worklog Time Spent: 10m Work Description: tweise commented on issue #7539: [BEAM-6439] Move Python Validates Runner Flink test to PreCommit URL: https://github.com/apache/beam/pull/7539#issuecomment-456583943 `git grep 'beam_PostCommit_Python_VR_Flink'` - see earlier comment This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188508) Time Spent: 4h 10m (was: 4h) > Move Python Flink VR tests to PreCommit > --- > > Key: BEAM-6439 > URL: https://issues.apache.org/jira/browse/BEAM-6439 > Project: Beam > Issue Type: New Feature > Components: runner-flink, testing >Reporter: Robert Bradshaw >Assignee: Robert Bradshaw >Priority: Major > Fix For: Not applicable > > Attachments: png.png > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Now that they're fast and stable, it would be good to catch changes that > break this earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6468) Cannot create empty TestBoundedTable
[ https://issues.apache.org/jira/browse/BEAM-6468?focusedWorklogId=188512=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188512 ] ASF GitHub Bot logged work on BEAM-6468: Author: ASF GitHub Bot Created on: 22/Jan/19 22:22 Start Date: 22/Jan/19 22:22 Worklog Time Spent: 10m Work Description: amaliujia commented on issue #7568: [BEAM-6468] Allow creating empty TestBoundedTable URL: https://github.com/apache/beam/pull/7568#issuecomment-456586268 reset tests This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188512) Time Spent: 1.5h (was: 1h 20m) > Cannot create empty TestBoundedTable > > > Key: BEAM-6468 > URL: https://issues.apache.org/jira/browse/BEAM-6468 > Project: Beam > Issue Type: Bug > Components: dsl-sql >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6161) Add ElementCount MonitoringInfos for the Java SDK
[ https://issues.apache.org/jira/browse/BEAM-6161?focusedWorklogId=188509=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188509 ] ASF GitHub Bot logged work on BEAM-6161: Author: ASF GitHub Bot Created on: 22/Jan/19 22:17 Start Date: 22/Jan/19 22:17 Worklog Time Spent: 10m Work Description: ajamato commented on issue #7272: [BEAM-6161] Introduce PCollectionConsumerRegistry and add ElementCoun… URL: https://github.com/apache/beam/pull/7272#issuecomment-456584783 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188509) Time Spent: 9h 40m (was: 9.5h) > Add ElementCount MonitoringInfos for the Java SDK > - > > Key: BEAM-6161 > URL: https://issues.apache.org/jira/browse/BEAM-6161 > Project: Beam > Issue Type: New Feature > Components: java-fn-execution, sdk-java-harness >Reporter: Alex Amato >Assignee: Alex Amato >Priority: Major > Time Spent: 9h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5879) TFRecordio not Py3 compatible
[ https://issues.apache.org/jira/browse/BEAM-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev closed BEAM-5879. - Resolution: Fixed Fix Version/s: 2.11.0 > TFRecordio not Py3 compatible > - > > Key: BEAM-5879 > URL: https://issues.apache.org/jira/browse/BEAM-5879 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core > Environment: python 3.5 >Reporter: Ruoyu Liu >Assignee: Robbe >Priority: Major > Fix For: 2.11.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Error when trying to write to file. Initial issue in line 103 in > io/tfrecordio.py, after making the content to bytes, there will be > segmentation fault when reading. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6181) Utilize MetricInfo for reporting user metrics in Portable Dataflow Java Runner.
[ https://issues.apache.org/jira/browse/BEAM-6181?focusedWorklogId=188506=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188506 ] ASF GitHub Bot logged work on BEAM-6181: Author: ASF GitHub Bot Created on: 22/Jan/19 22:12 Start Date: 22/Jan/19 22:12 Worklog Time Spent: 10m Work Description: Ardagan commented on issue #7323: [BEAM-6181] Implemented msec counters support in FnApi world. URL: https://github.com/apache/beam/pull/7323#issuecomment-456583397 run java precommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188506) Time Spent: 13h 20m (was: 13h 10m) > Utilize MetricInfo for reporting user metrics in Portable Dataflow Java > Runner. > --- > > Key: BEAM-6181 > URL: https://issues.apache.org/jira/browse/BEAM-6181 > Project: Beam > Issue Type: Bug > Components: java-fn-execution >Reporter: Mikhail Gryzykhin >Assignee: Mikhail Gryzykhin >Priority: Major > Time Spent: 13h 20m > Remaining Estimate: 0h > > New approach to report metrics in FnApi is to utilize MetricInfo structures. > This approach is implemented in Python SDK and work is ongoing in Java SDK. > This tasks includes plumbing User metrics reported via MetricInfos through > Dataflow Java Runner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6181) Utilize MetricInfo for reporting user metrics in Portable Dataflow Java Runner.
[ https://issues.apache.org/jira/browse/BEAM-6181?focusedWorklogId=188507=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188507 ] ASF GitHub Bot logged work on BEAM-6181: Author: ASF GitHub Bot Created on: 22/Jan/19 22:12 Start Date: 22/Jan/19 22:12 Worklog Time Spent: 10m Work Description: Ardagan commented on issue #7323: [BEAM-6181] Implemented msec counters support in FnApi world. URL: https://github.com/apache/beam/pull/7323#issuecomment-456583437 run python precommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188507) Time Spent: 13.5h (was: 13h 20m) > Utilize MetricInfo for reporting user metrics in Portable Dataflow Java > Runner. > --- > > Key: BEAM-6181 > URL: https://issues.apache.org/jira/browse/BEAM-6181 > Project: Beam > Issue Type: Bug > Components: java-fn-execution >Reporter: Mikhail Gryzykhin >Assignee: Mikhail Gryzykhin >Priority: Major > Time Spent: 13.5h > Remaining Estimate: 0h > > New approach to report metrics in FnApi is to utilize MetricInfo structures. > This approach is implemented in Python SDK and work is ongoing in Java SDK. > This tasks includes plumbing User metrics reported via MetricInfos through > Dataflow Java Runner. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3286) Go SDK support for portable side input
[ https://issues.apache.org/jira/browse/BEAM-3286?focusedWorklogId=188505=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188505 ] ASF GitHub Bot logged work on BEAM-3286: Author: ASF GitHub Bot Created on: 22/Jan/19 22:08 Start Date: 22/Jan/19 22:08 Worklog Time Spent: 10m Work Description: lostluck commented on issue #7591: [BEAM-3286] Fix side input window coding, & re-enable tests. URL: https://github.com/apache/beam/pull/7591#issuecomment-456582191 Run Go PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188505) Time Spent: 3h 40m (was: 3.5h) > Go SDK support for portable side input > -- > > Key: BEAM-3286 > URL: https://issues.apache.org/jira/browse/BEAM-3286 > Project: Beam > Issue Type: Sub-task > Components: sdk-go >Reporter: Henning Rohde >Assignee: Henning Rohde >Priority: Major > Labels: portability > Fix For: 2.8.0 > > Time Spent: 3h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-3286) Go SDK support for portable side input
[ https://issues.apache.org/jira/browse/BEAM-3286?focusedWorklogId=188504=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188504 ] ASF GitHub Bot logged work on BEAM-3286: Author: ASF GitHub Bot Created on: 22/Jan/19 22:08 Start Date: 22/Jan/19 22:08 Worklog Time Spent: 10m Work Description: lostluck commented on pull request #7591: [BEAM-3286] Fix side input window coding, & re-enable tests. URL: https://github.com/apache/beam/pull/7591 Side input coding requires that a single window is passed to the runner side, but the Go SDK was inadvertently passing a *list* of windows of size 1. This prevented side inputs working correctly on portable runners. Uncommented the tests to validate them on post commit. Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it. Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | --- | --- | --- | --- Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | --- | --- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please
[jira] [Commented] (BEAM-5879) TFRecordio not Py3 compatible
[ https://issues.apache.org/jira/browse/BEAM-5879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749188#comment-16749188 ] Valentyn Tymofieiev commented on BEAM-5879: --- Great, thanks! > TFRecordio not Py3 compatible > - > > Key: BEAM-5879 > URL: https://issues.apache.org/jira/browse/BEAM-5879 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core > Environment: python 3.5 >Reporter: Ruoyu Liu >Assignee: Robbe >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > > Error when trying to write to file. Initial issue in line 103 in > io/tfrecordio.py, after making the content to bytes, there will be > segmentation fault when reading. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6487) BigQuery api is out of date
[ https://issues.apache.org/jira/browse/BEAM-6487?focusedWorklogId=188495=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188495 ] ASF GitHub Bot logged work on BEAM-6487: Author: ASF GitHub Bot Created on: 22/Jan/19 21:34 Start Date: 22/Jan/19 21:34 Worklog Time Spent: 10m Work Description: pabloem commented on issue #7590: [BEAM-6487] Updating BQ API URL: https://github.com/apache/beam/pull/7590#issuecomment-456572238 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188495) Time Spent: 10m Remaining Estimate: 0h > BigQuery api is out of date > --- > > Key: BEAM-6487 > URL: https://issues.apache.org/jira/browse/BEAM-6487 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Pablo Estrada >Assignee: Pablo Estrada >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6470) python tenacity is breaks the Python validates runner flink tests
[ https://issues.apache.org/jira/browse/BEAM-6470?focusedWorklogId=188493=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188493 ] ASF GitHub Bot logged work on BEAM-6470: Author: ASF GitHub Bot Created on: 22/Jan/19 21:32 Start Date: 22/Jan/19 21:32 Worklog Time Spent: 10m Work Description: angoenka commented on pull request #7577: [BEAM-6470] Remove dependency on tenacity URL: https://github.com/apache/beam/pull/7577 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188493) Time Spent: 3h 20m (was: 3h 10m) > python tenacity is breaks the Python validates runner flink tests > - > > Key: BEAM-6470 > URL: https://issues.apache.org/jira/browse/BEAM-6470 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 3h 20m > Remaining Estimate: 0h > > Python tenacity is only added for test cases. > Beam VR tests executes actual pipelines which do not have tenacity. > We already have apache_beam.util.retry which has the retry functionality so > replacing tenacity with it will solve this problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6439) Move Python Flink VR tests to PreCommit
[ https://issues.apache.org/jira/browse/BEAM-6439?focusedWorklogId=188494=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188494 ] ASF GitHub Bot logged work on BEAM-6439: Author: ASF GitHub Bot Created on: 22/Jan/19 21:33 Start Date: 22/Jan/19 21:33 Worklog Time Spent: 10m Work Description: angoenka commented on issue #7539: [BEAM-6439] Move Python Validates Runner Flink test to PreCommit URL: https://github.com/apache/beam/pull/7539#issuecomment-456571685 Ping: @tweise @robertwb This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188494) Time Spent: 4h (was: 3h 50m) > Move Python Flink VR tests to PreCommit > --- > > Key: BEAM-6439 > URL: https://issues.apache.org/jira/browse/BEAM-6439 > Project: Beam > Issue Type: New Feature > Components: runner-flink, testing >Reporter: Robert Bradshaw >Assignee: Robert Bradshaw >Priority: Major > Fix For: Not applicable > > Attachments: png.png > > Time Spent: 4h > Remaining Estimate: 0h > > Now that they're fast and stable, it would be good to catch changes that > break this earlier. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-6486) Cross-language transform API
Chamikara Jayalath created BEAM-6486: Summary: Cross-language transform API Key: BEAM-6486 URL: https://issues.apache.org/jira/browse/BEAM-6486 Project: Beam Issue Type: New Feature Components: beam-model, runner-core, sdk-java-core, sdk-py-core Reporter: Chamikara Jayalath Assignee: Chamikara Jayalath We should be able to refer to a transform defined in SDK X from a different SDK Y. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-6487) BigQuery api is out of date
Pablo Estrada created BEAM-6487: --- Summary: BigQuery api is out of date Key: BEAM-6487 URL: https://issues.apache.org/jira/browse/BEAM-6487 Project: Beam Issue Type: Bug Components: sdk-py-core Reporter: Pablo Estrada Assignee: Pablo Estrada -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6470) python tenacity is breaks the Python validates runner flink tests
[ https://issues.apache.org/jira/browse/BEAM-6470?focusedWorklogId=188492=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188492 ] ASF GitHub Bot logged work on BEAM-6470: Author: ASF GitHub Bot Created on: 22/Jan/19 21:32 Start Date: 22/Jan/19 21:32 Worklog Time Spent: 10m Work Description: angoenka commented on issue #7577: [BEAM-6470] Remove dependency on tenacity URL: https://github.com/apache/beam/pull/7577#issuecomment-456571539 > Looks like we can drop this due to #7576 ? Sounds good! Closing the PR. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 188492) Time Spent: 3h 10m (was: 3h) > python tenacity is breaks the Python validates runner flink tests > - > > Key: BEAM-6470 > URL: https://issues.apache.org/jira/browse/BEAM-6470 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 3h 10m > Remaining Estimate: 0h > > Python tenacity is only added for test cases. > Beam VR tests executes actual pipelines which do not have tenacity. > We already have apache_beam.util.retry which has the retry functionality so > replacing tenacity with it will solve this problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-6386) Add named variant of PTransform::compose
[
https://issues.apache.org/jira/browse/BEAM-6386?focusedWorklogId=188490=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188490
]
ASF GitHub Bot logged work on BEAM-6386:
Author: ASF GitHub Bot
Created on: 22/Jan/19 21:31
Start Date: 22/Jan/19 21:31
Worklog Time Spent: 10m
Work Description: swegner commented on pull request #7437: [BEAM-6386]
Add named variant of PTransform::compose()
URL: https://github.com/apache/beam/pull/7437#discussion_r249967618
##
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/PTransform.java
##
@@ -319,4 +319,16 @@ public OutputT expand(InputT input) {
}
};
}
+
+ /** Like {@link #compose(SerializableFunction)}, but with a custom name. */
+ @Experimental
+ public static
Review comment:
First off-- thanks for taking the time to write up the motivation. I believe
I have a good understanding of the scenario now.
So, have two ways of specifying a composite transform:
1. By defining a class which extends `PTransform` and filling in
the `expand()` method.
2. Using this `PTranform.compose()` helper syntax.
Defining a class gives access to all features of the PTransform spec
(naming, validation, display data, multiple inputs), while the helper syntax
offers a very concise syntax for the simplest cases.
My general thoughts:
a. We shouldn't add verbosity to the shorthand syntax as it removes the
utility of it.
b. We shouldn't add a bunch of complexity to the shorthand but and instead
defer to the full syntax.
In this case, having an overload with a String name is still a useful
shorthand and doesn't violate either of the above. And you've defined a real
scenario where it provides value. So: I'm convinced.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188490)
Time Spent: 50m (was: 40m)
> Add named variant of PTransform::compose
>
>
> Key: BEAM-6386
> URL: https://issues.apache.org/jira/browse/BEAM-6386
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
>Reporter: Jeff Klukas
>Assignee: Jeff Klukas
>Priority: Minor
> Time Spent: 50m
> Remaining Estimate: 0h
>
> BEAM- 5413 introduced PTransform::compose as a concise way of creating a
> composition of transforms as a lambda. We should add a variant to allow
> specifying a name for the returned transform in the same way that {{apply}}
> can take an explicit name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-6386) Add named variant of PTransform::compose
[
https://issues.apache.org/jira/browse/BEAM-6386?focusedWorklogId=188491=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-188491
]
ASF GitHub Bot logged work on BEAM-6386:
Author: ASF GitHub Bot
Created on: 22/Jan/19 21:31
Start Date: 22/Jan/19 21:31
Worklog Time Spent: 10m
Work Description: swegner commented on pull request #7437: [BEAM-6386]
Add named variant of PTransform::compose()
URL: https://github.com/apache/beam/pull/7437
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 188491)
Time Spent: 1h (was: 50m)
> Add named variant of PTransform::compose
>
>
> Key: BEAM-6386
> URL: https://issues.apache.org/jira/browse/BEAM-6386
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
>Reporter: Jeff Klukas
>Assignee: Jeff Klukas
>Priority: Minor
> Time Spent: 1h
> Remaining Estimate: 0h
>
> BEAM- 5413 introduced PTransform::compose as a concise way of creating a
> composition of transforms as a lambda. We should add a variant to allow
> specifying a name for the returned transform in the same way that {{apply}}
> can take an explicit name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (BEAM-6488) Portable Flink runner support for running cross-language transforms
[ https://issues.apache.org/jira/browse/BEAM-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6488: - Description: To support running cross-language transforms, Portable Flink runner needs to support executing pipelines with steps defined to be executed for different environments. I believe this support is already there. If that is the case we should validate that and add any missing tests. If there are missing pieces, we should figure out details and create more JIRAs as needed. CC: [~angoenka] was: To support running cross-language transforms, Flink runner needs to support executing pipelines with steps defined to be executed for different environments. I believe this support is already there. If that is the case we should validate that and add any missing tests. If there are missing pieces, we should figure out details and create more JIRAs as needed. CC: [~angoenka] > Portable Flink runner support for running cross-language transforms > --- > > Key: BEAM-6488 > URL: https://issues.apache.org/jira/browse/BEAM-6488 > Project: Beam > Issue Type: New Feature > Components: beam-model, runner-core, runner-flink, sdk-java-core, > sdk-py-core >Reporter: Chamikara Jayalath >Assignee: Heejong Lee >Priority: Major > > To support running cross-language transforms, Portable Flink runner needs to > support executing pipelines with steps defined to be executed for different > environments. > I believe this support is already there. If that is the case we should > validate that and add any missing tests. > If there are missing pieces, we should figure out details and create more > JIRAs as needed. > CC: [~angoenka] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-6488) Flink runner support for running cross-language transforms
Chamikara Jayalath created BEAM-6488: Summary: Flink runner support for running cross-language transforms Key: BEAM-6488 URL: https://issues.apache.org/jira/browse/BEAM-6488 Project: Beam Issue Type: New Feature Components: beam-model, runner-core, runner-flink, sdk-java-core, sdk-py-core Reporter: Chamikara Jayalath Assignee: Heejong Lee To support running cross-language transforms, Flink runner needs to support executing pipelines with steps defined to be executed for different environments. I believe this support is already there. If that is the case we should validate that and add any missing tests. If there are missing pieces, we should figure out details and create more JIRAs as needed. CC: [~angoenka] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-6488) Portable Flink runner support for running cross-language transforms
[ https://issues.apache.org/jira/browse/BEAM-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath updated BEAM-6488: - Summary: Portable Flink runner support for running cross-language transforms (was: Flink runner support for running cross-language transforms) > Portable Flink runner support for running cross-language transforms > --- > > Key: BEAM-6488 > URL: https://issues.apache.org/jira/browse/BEAM-6488 > Project: Beam > Issue Type: New Feature > Components: beam-model, runner-core, runner-flink, sdk-java-core, > sdk-py-core >Reporter: Chamikara Jayalath >Assignee: Heejong Lee >Priority: Major > > To support running cross-language transforms, Flink runner needs to support > executing pipelines with steps defined to be executed for different > environments. > I believe this support is already there. If that is the case we should > validate that and add any missing tests. > If there are missing pieces, we should figure out details and create more > JIRAs as needed. > CC: [~angoenka] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-6485) Cross-language transform expansion protocol
Chamikara Jayalath created BEAM-6485: Summary: Cross-language transform expansion protocol Key: BEAM-6485 URL: https://issues.apache.org/jira/browse/BEAM-6485 Project: Beam Issue Type: New Feature Components: beam-model, runner-core, sdk-java-core, sdk-py-core Reporter: Chamikara Jayalath Assignee: Robert Bradshaw -- This message was sent by Atlassian JIRA (v7.6.3#76005)
