Re: Python staging file weirdness

Can we filter the cache directory only for the artifacts that we want and not everything that is there? On Wed, Dec 4, 2019 at 6:56 PM Valentyn Tymofieiev wrote: > Luke, I am not sure I understand the question. The caching that happens > here is implemented in the SDK for requirements packages:

Re: Python staging file weirdness

Luke, I am not sure I understand the question. The caching that happens here is implemented in the SDK for requirements packages: https://github.com/apache/beam/blob/438055c95116f4e6e419e5faa9c42f7d329c421c/sdks/python/apache_beam/runners/portability/stager.py#L161 On Wed, Dec 4, 2019 at 6:19 PM

Re: Python staging file weirdness

Is there a way to use a cache on disk that is separate from the set of packages we use as requirements? On Wed, Dec 4, 2019 at 5:58 PM Udi Meiri wrote: > Thanks! > Another reason to periodically referesh workers. > > On Wed, Nov 27, 2019 at 10:37 PM Valentyn Tymofieiev > wrote: > >> Tests job

Re: Python staging file weirdness

Thanks! Another reason to periodically referesh workers. On Wed, Nov 27, 2019 at 10:37 PM Valentyn Tymofieiev wrote: > Tests job specify[1] a requirements.txt file that contains two entries: > pyhamcrest, mock. > > We download[2] sources of packages specified in requirements file, > and

Re: Python interactive runner: test dependencies removed

Thanks for the heads up! I was wondering why the interactive tests are skipped, lol. So we are moving away from the deprecated pytest-runner (with the changes in setup.py) but still sticking to pytest since it's replacing nosetest. Can I add "interactive" as "extras" to testenv "py37-pytest" and

Re: Python interactive runner: test dependencies removed

+Ning Kang +Sam Rohde fyi On Wed, Nov 27, 2019 at 5:09 PM Udi Meiri wrote: > As part of a move to stop using the deprecated (and racey) setup.py > keywords setup_requires and test_require, interactive runner dependencies > have been removed from tests in >

Re: Contributor permission for Beam Jira tickets

Welcome, I have added you as a contributor. On Wed, Dec 4, 2019 at 4:19 PM Esun Kim wrote: > Hi, > > This is Esun Kim from Google. I'm working on GCS connector of beam IO. > Can you add me as a contributor for Beam's Jira issue tracker? My Jira ID > is veblush. > > Regards, > Esun. > >

Contributor permission for Beam Jira tickets

Hi, This is Esun Kim from Google. I'm working on GCS connector of beam IO. Can you add me as a contributor for Beam's Jira issue tracker? My Jira ID is veblush. Regards, Esun.

Re: Version Beam Website Documentation

I agree, having a single website showcase the latest beam versions and encourages users to use the latest Beam version which is very useful. Calling out version limitations are definitely makes users life easier. The usecase I have in mind is more on the lines of best practices and recommended

[RELEASE] Tracking 2.18

Following the release calendar, I plan on cutting the 2.18 release branch today. There are currently 8 release blockers . smime.p7s Description: S/MIME Cryptographic Signature

Re: [Interactive Beam] Changes to what to cache

Thanks Pablo! I've sent a user-geared update too. On Wed, Dec 4, 2019 at 11:16 AM Pablo Estrada wrote: > Thanks for sharing, Ning! > Is this update valuable to users as well? If so, consider sending a > user-geared update to u...@beam.apache.org. > > -P. > > On Wed, Dec 4, 2019 at 11:14 AM Ning

Re: [Interactive Beam] Changes to what to cache

Thanks for sharing, Ning! Is this update valuable to users as well? If so, consider sending a user-geared update to u...@beam.apache.org. -P. On Wed, Dec 4, 2019 at 11:14 AM Ning Kang wrote: > *If you are not an Interactive Beam user, you can ignore this email.* > > Hi everyone, > > Recently,

[Interactive Beam] Changes to what to cache

*If you are not an Interactive Beam user, you can ignore this email.* Hi everyone, Recently, we've been actively developing on top of the existing InteractiveRunner for more Interactive Beam features

Re: Request for review of PR [Beam-8564]

Going with the Registrar/ServiceLoader route would allow for alternative providers for the same compression algorithms so if they don't like one they can always contribute a different one. On Wed, Dec 4, 2019 at 8:22 AM Ismaël Mejía wrote: > (1) seems not to be the issue because it is Apache

Re: Request for review of PR [Beam-8564]

(1) seems not to be the issue because it is Apache licensed. (2) and (3) are the big issues, because it requires a provided huge uber jar that essentially leaks Hadoop classes into core SDK [1] so it is definitely concerning. We discussed at some point during the PR that added ZStandard support

Re: Wiki edit access

Thanks! On Wed, Dec 4, 2019 at 5:07 PM Maximilian Michels wrote: > Done ;) > > On 04.12.19 15:49, Kamil Wasilewski wrote: > > Hi all, > > > > I'm going to make a contribution to documentation pages that describe > > testing framework in Beam. May I get access to edit the Wiki? My > > username

Re: Wiki edit access

Done ;) On 04.12.19 15:49, Kamil Wasilewski wrote: Hi all, I'm going to make a contribution to documentation pages that describe testing framework in Beam. May I get access to edit the Wiki? My username is kamilwu. Kamil

Re: real real-time beam

Hi Kenn, On 12/4/19 5:38 AM, Kenneth Knowles wrote: Jan - let's try to defrag the threads on your time sorting proposal. This thread may have useful ideas but I want to focus on helping Aaron in this thread. You can link to this thread from other threads or from a design doc. Does this seem

Wiki edit access

Hi all, I'm going to make a contribution to documentation pages that describe testing framework in Beam. May I get access to edit the Wiki? My username is kamilwu. Kamil

Re: Version Beam Website Documentation

The API reference docs (Java and Python at least) are versioned, so we have a durable reference there and it's possible to link to particular sections of API docs for particular versions. For the major bits of introductory documentation (like the Beam Programming Guide), I think it's a good thing