Re: Shutting down Perfkit Explorer

[Etienne Chauchot]

Thanks Kamil !

Etienne

On 25/09/2020 16:10, Kamil Wasilewski wrote:

They have been migrated: http://metrics.beam.apache.org

The website doesn't support HTTPS, so if you can't access it, you may 
need to add an exception to your browser extension.


On Fri, Sep 25, 2020 at 3:49 PM Etienne Chauchot > wrote:


Hi all,

I'm coming a bit after the battle, but how should we access the
dashboards now (load tests, nexmark etc...)?

Are the dashboards lost or have they migrated to another environment ?

Thanks

Etienne

On 24/09/2020 13:51, Robert Burke wrote:

LGTM
Good clear message.

On Thu, Sep 24, 2020, 4:47 AM Kamil Wasilewski
mailto:kamil.wasilew...@polidea.com>> wrote:

The message has been updated:
https://apache-beam-testing.appspot.com/

On Thu, Sep 24, 2020 at 12:07 PM Kamil Wasilewski
mailto:kamil.wasilew...@polidea.com>> wrote:

I'm not sure if such a jira issue exists. I'm also not
convinced that we need a new one. New jira means there is
an action to be taken, now or in the future. Our goal is
only to make sure no one would disable the app engine
accidently.

Putting some information instead of "hello world" is a
good idea. I will update it today. Thank you Tobiasz for
the link, it will be very helpful.


On Thu, Sep 24, 2020 at 11:32 AM Tobiasz Kędzierski
mailto:tobiasz.kedzier...@polidea.com>> wrote:

I agree with Robert, putting some information instead
of "hello world" could prevent disabling GAE and
connected problems in the future.
If a relevant Jira issue exists it would be a great
idea to put a link to it.
What do you think about adding a link to GCP
documentation mentioning this dependency [1] ?

BR Tobiasz

[1]

https://cloud.google.com/datastore/docs/reference/libraries#dependency_on_application



On Wed, Sep 23, 2020 at 5:33 PM Robert Burke
mailto:rob...@frantil.com>> wrote:

Perhaps instead of "hello world" the message
could refer to a jira about the Datastore IT tests?
I suspect if we don't we'll just have a repeat of
"we shut down app engine since it was just
running a hello world, and the Datastore tests died".

On Wed, Sep 23, 2020, 8:14 AM Kamil Wasilewski
mailto:kamil.wasilew...@polidea.com>> wrote:

An error message that Udi sent is pretty self
explanatory. Disabling Google App Engine
caused Datastore to be not accessible too.
What's interesting, it doesn't make any
difference if the application on GAE is
actually using Datastore or not. GAE must be
simply turned on and that's the only requirement.

I replaced Perfkit Explorer with a simple
"hello world" running on Python 3.8. I ran
Datastore IT tests and they passed, so I
think this solves the problem. If something
goes wrong, let me know!
Sorry for inconvenience and thanks Tyson for
the rescue.

On Tue, Sep 22, 2020 at 11:25 PM Udi Meiri
mailto:eh...@google.com>>
wrote:

Thanks, Tyson!

On Tue, Sep 22, 2020 at 11:11 AM Tyson
Hamilton mailto:tyso...@google.com>> wrote:

I re-enabled the AppEngine app. Today
that app has both the required
datastore app and the perfkit app
baked into the container image. What
should happen, is that the perfkit
app is removed from that image, but
the datastore related stuff remains
functional.

On Tue, Sep 22, 2020 at 10:37 AM Udi
Meiri mailto:eh...@google.com>> wrote:

Is it possible to create a simple
"hello world" application instead?

On Tue, Sep 22, 2020 at 10:35 AM
Udi Meiri mailto:eh...@google.com>> wrote:

Disabling this broke our
   

Re: Shutting down Perfkit Explorer

[Kamil Wasilewski]

They have been migrated: http://metrics.beam.apache.org

The website doesn't support HTTPS, so if you can't access it, you may need
to add an exception to your browser extension.

On Fri, Sep 25, 2020 at 3:49 PM Etienne Chauchot 
wrote:

> Hi all,
>
> I'm coming a bit after the battle, but how should we access the dashboards
> now (load tests, nexmark etc...)?
>
> Are the dashboards lost or have they migrated to another environment ?
>
> Thanks
>
> Etienne
> On 24/09/2020 13:51, Robert Burke wrote:
>
> LGTM
> Good clear message.
>
> On Thu, Sep 24, 2020, 4:47 AM Kamil Wasilewski <
> kamil.wasilew...@polidea.com> wrote:
>
>> The message has been updated: https://apache-beam-testing.appspot.com/
>>
>> On Thu, Sep 24, 2020 at 12:07 PM Kamil Wasilewski <
>> kamil.wasilew...@polidea.com> wrote:
>>
>>> I'm not sure if such a jira issue exists. I'm also not convinced that we
>>> need a new one. New jira means there is an action to be taken, now or in
>>> the future. Our goal is only to make sure no one would disable the app
>>> engine accidently.
>>>
>>> Putting some information instead of "hello world" is a good idea. I will
>>> update it today. Thank you Tobiasz for the link, it will be very helpful.
>>>
>>>
>>> On Thu, Sep 24, 2020 at 11:32 AM Tobiasz Kędzierski <
>>> tobiasz.kedzier...@polidea.com> wrote:
>>>
 I agree with Robert, putting some information instead of "hello world"
 could prevent disabling GAE and connected problems in the future.
 If a relevant Jira issue exists it would be a great idea to put a link
 to it.
 What do you think about adding a link to GCP documentation mentioning
 this dependency [1] ?

 BR Tobiasz

 [1]
 https://cloud.google.com/datastore/docs/reference/libraries#dependency_on_application



 On Wed, Sep 23, 2020 at 5:33 PM Robert Burke 
 wrote:

> Perhaps instead of "hello world" the message could refer to a jira
> about the Datastore IT tests?
> I suspect if we don't we'll just have a repeat of "we shut down app
> engine since it was just running a hello world, and the Datastore tests
> died".
>
> On Wed, Sep 23, 2020, 8:14 AM Kamil Wasilewski <
> kamil.wasilew...@polidea.com> wrote:
>
>> An error message that Udi sent is pretty self explanatory. Disabling
>> Google App Engine caused Datastore to be not accessible too. What's
>> interesting, it doesn't make any difference if the application on GAE is
>> actually using Datastore or not. GAE must be simply turned on and that's
>> the only requirement.
>>
>> I replaced Perfkit Explorer with a simple "hello world" running on
>> Python 3.8. I ran Datastore IT tests and they passed, so I think this
>> solves the problem. If something goes wrong, let me know!
>> Sorry for inconvenience and thanks Tyson for the rescue.
>>
>> On Tue, Sep 22, 2020 at 11:25 PM Udi Meiri  wrote:
>>
>>> Thanks, Tyson!
>>>
>>> On Tue, Sep 22, 2020 at 11:11 AM Tyson Hamilton 
>>> wrote:
>>>
 I re-enabled the AppEngine app. Today that app has both the
 required datastore app and the perfkit app baked into the container 
 image.
 What should happen, is that the perfkit app is removed from that 
 image, but
 the datastore related stuff remains functional.

 On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri 
 wrote:

> Is it possible to create a simple "hello world" application
> instead?
>
> On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri 
> wrote:
>
>> Disabling this broke our Datastore ITs. Apparently you must have
>> an application for Datastore to work. From the Datastore dashboard:
>> The project apache-beam-testing does not exist or it does not
>> contain an active Cloud Datastore or Cloud Firestore database. 
>> Please visit
>> http://console.cloud.google.com to create a project or
>> https://console.cloud.google.com/datastore/setup?project=apache-beam-testing
>> to add a Cloud Datastore or Cloud Firestore database. Note that Cloud
>> Datastore or Cloud Firestore always have an associated App Engine 
>> app and
>> this app must not be disabled.
>> New failure:
>> https://ci-beam.apache.org/job/beam_PostCommit_Python36/2959/
>>
>>
>>
>> On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski <
>> kamil.wasilew...@polidea.com> wrote:
>>
>>> Thanks. The application has been disabled.
>>>
>>> On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay 
>>> wrote:
>>>
 +1. Thank you for the cleanup.

 On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton <
 tyso...@google.com> wrote:

> +1 to removing, thank you Kamil.
>
>

Re: Shutting down Perfkit Explorer

[Etienne Chauchot]

Hi all,

I'm coming a bit after the battle, but how should we access the 
dashboards now (load tests, nexmark etc...)?


Are the dashboards lost or have they migrated to another environment ?

Thanks

Etienne

On 24/09/2020 13:51, Robert Burke wrote:

LGTM
Good clear message.

On Thu, Sep 24, 2020, 4:47 AM Kamil Wasilewski 
mailto:kamil.wasilew...@polidea.com>> 
wrote:


The message has been updated: https://apache-beam-testing.appspot.com/

On Thu, Sep 24, 2020 at 12:07 PM Kamil Wasilewski
mailto:kamil.wasilew...@polidea.com>> wrote:

I'm not sure if such a jira issue exists. I'm also not
convinced that we need a new one. New jira means there is an
action to be taken, now or in the future. Our goal is only to
make sure no one would disable the app engine accidently.

Putting some information instead of "hello world" is a good
idea. I will update it today. Thank you Tobiasz for the link,
it will be very helpful.


On Thu, Sep 24, 2020 at 11:32 AM Tobiasz Kędzierski
mailto:tobiasz.kedzier...@polidea.com>> wrote:

I agree with Robert, putting some information instead of
"hello world" could prevent disabling GAE and connected
problems in the future.
If a relevant Jira issue exists it would be a great idea
to put a link to it.
What do you think about adding a link to GCP documentation
mentioning this dependency [1] ?

BR Tobiasz

[1]

https://cloud.google.com/datastore/docs/reference/libraries#dependency_on_application



On Wed, Sep 23, 2020 at 5:33 PM Robert Burke
mailto:rob...@frantil.com>> wrote:

Perhaps instead of "hello world" the message could
refer to a jira about the Datastore IT tests?
I suspect if we don't we'll just have a repeat of "we
shut down app engine since it was just running a hello
world, and the Datastore tests died".

On Wed, Sep 23, 2020, 8:14 AM Kamil Wasilewski
mailto:kamil.wasilew...@polidea.com>> wrote:

An error message that Udi sent is pretty self
explanatory. Disabling Google App Engine caused
Datastore to be not accessible too. What's
interesting, it doesn't make any difference if the
application on GAE is actually using Datastore or
not. GAE must be simply turned on and that's the
only requirement.

I replaced Perfkit Explorer with a simple "hello
world" running on Python 3.8. I ran Datastore IT
tests and they passed, so I think this solves the
problem. If something goes wrong, let me know!
Sorry for inconvenience and thanks Tyson for the
rescue.

On Tue, Sep 22, 2020 at 11:25 PM Udi Meiri
mailto:eh...@google.com>> wrote:

Thanks, Tyson!

On Tue, Sep 22, 2020 at 11:11 AM Tyson
Hamilton mailto:tyso...@google.com>> wrote:

I re-enabled the AppEngine app. Today that
app has both the required datastore app
and the perfkit app baked into the
container image. What should happen, is
that the perfkit app is removed from that
image, but the datastore related stuff
remains functional.

On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri
mailto:eh...@google.com>> wrote:

Is it possible to create a simple
"hello world" application instead?

On Tue, Sep 22, 2020 at 10:35 AM Udi
Meiri mailto:eh...@google.com>> wrote:

Disabling this broke our Datastore
ITs. Apparently you must have an
application for Datastore to work.
From the Datastore dashboard:
The project apache-beam-testing
does not exist or it does not
contain an active Cloud Datastore
or Cloud Firestore database.
Please visit
http://console.cloud.google.com to
create a project or

https://console.cloud.google.com/datastore/setup?project=apache-be

Re: Shutting down Perfkit Explorer

[Robert Burke]

LGTM
Good clear message.

On Thu, Sep 24, 2020, 4:47 AM Kamil Wasilewski 
wrote:

> The message has been updated: https://apache-beam-testing.appspot.com/
>
> On Thu, Sep 24, 2020 at 12:07 PM Kamil Wasilewski <
> kamil.wasilew...@polidea.com> wrote:
>
>> I'm not sure if such a jira issue exists. I'm also not convinced that we
>> need a new one. New jira means there is an action to be taken, now or in
>> the future. Our goal is only to make sure no one would disable the app
>> engine accidently.
>>
>> Putting some information instead of "hello world" is a good idea. I will
>> update it today. Thank you Tobiasz for the link, it will be very helpful.
>>
>>
>> On Thu, Sep 24, 2020 at 11:32 AM Tobiasz Kędzierski <
>> tobiasz.kedzier...@polidea.com> wrote:
>>
>>> I agree with Robert, putting some information instead of "hello world"
>>> could prevent disabling GAE and connected problems in the future.
>>> If a relevant Jira issue exists it would be a great idea to put a link
>>> to it.
>>> What do you think about adding a link to GCP documentation mentioning
>>> this dependency [1] ?
>>>
>>> BR Tobiasz
>>>
>>> [1]
>>> https://cloud.google.com/datastore/docs/reference/libraries#dependency_on_application
>>>
>>>
>>>
>>> On Wed, Sep 23, 2020 at 5:33 PM Robert Burke  wrote:
>>>
 Perhaps instead of "hello world" the message could refer to a jira
 about the Datastore IT tests?
 I suspect if we don't we'll just have a repeat of "we shut down app
 engine since it was just running a hello world, and the Datastore tests
 died".

 On Wed, Sep 23, 2020, 8:14 AM Kamil Wasilewski <
 kamil.wasilew...@polidea.com> wrote:

> An error message that Udi sent is pretty self explanatory. Disabling
> Google App Engine caused Datastore to be not accessible too. What's
> interesting, it doesn't make any difference if the application on GAE is
> actually using Datastore or not. GAE must be simply turned on and that's
> the only requirement.
>
> I replaced Perfkit Explorer with a simple "hello world" running on
> Python 3.8. I ran Datastore IT tests and they passed, so I think this
> solves the problem. If something goes wrong, let me know!
> Sorry for inconvenience and thanks Tyson for the rescue.
>
> On Tue, Sep 22, 2020 at 11:25 PM Udi Meiri  wrote:
>
>> Thanks, Tyson!
>>
>> On Tue, Sep 22, 2020 at 11:11 AM Tyson Hamilton 
>> wrote:
>>
>>> I re-enabled the AppEngine app. Today that app has both the required
>>> datastore app and the perfkit app baked into the container image. What
>>> should happen, is that the perfkit app is removed from that image, but 
>>> the
>>> datastore related stuff remains functional.
>>>
>>> On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri  wrote:
>>>
 Is it possible to create a simple "hello world" application instead?

 On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri 
 wrote:

> Disabling this broke our Datastore ITs. Apparently you must have
> an application for Datastore to work. From the Datastore dashboard:
> The project apache-beam-testing does not exist or it does not
> contain an active Cloud Datastore or Cloud Firestore database. Please 
> visit
> http://console.cloud.google.com to create a project or
> https://console.cloud.google.com/datastore/setup?project=apache-beam-testing
> to add a Cloud Datastore or Cloud Firestore database. Note that Cloud
> Datastore or Cloud Firestore always have an associated App Engine app 
> and
> this app must not be disabled.
> New failure:
> https://ci-beam.apache.org/job/beam_PostCommit_Python36/2959/
>
>
>
> On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski <
> kamil.wasilew...@polidea.com> wrote:
>
>> Thanks. The application has been disabled.
>>
>> On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay 
>> wrote:
>>
>>> +1. Thank you for the cleanup.
>>>
>>> On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton <
>>> tyso...@google.com> wrote:
>>>
 +1 to removing, thank you Kamil.

 On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
 kamil.wasilew...@polidea.com> wrote:

> Hello everyone,
>
> Beam support for Python 2 is coming to an end. Consequently,
> we should make sure no Python 2 applications are running as a 
> part of
> Beam's infrastructure. As you may know, Beam is still hosting a 
> Python 2
> application on Google App Engine. This application is Perfkit 
> Explorer [1].
>
> Perfkit Explorer has been used as a dashboarding tool for a
> long time. Few months ago, it was deprec

Re: Shutting down Perfkit Explorer

[Kamil Wasilewski]

The message has been updated: https://apache-beam-testing.appspot.com/

On Thu, Sep 24, 2020 at 12:07 PM Kamil Wasilewski <
kamil.wasilew...@polidea.com> wrote:

> I'm not sure if such a jira issue exists. I'm also not convinced that we
> need a new one. New jira means there is an action to be taken, now or in
> the future. Our goal is only to make sure no one would disable the app
> engine accidently.
>
> Putting some information instead of "hello world" is a good idea. I will
> update it today. Thank you Tobiasz for the link, it will be very helpful.
>
>
> On Thu, Sep 24, 2020 at 11:32 AM Tobiasz Kędzierski <
> tobiasz.kedzier...@polidea.com> wrote:
>
>> I agree with Robert, putting some information instead of "hello world"
>> could prevent disabling GAE and connected problems in the future.
>> If a relevant Jira issue exists it would be a great idea to put a link to
>> it.
>> What do you think about adding a link to GCP documentation mentioning
>> this dependency [1] ?
>>
>> BR Tobiasz
>>
>> [1]
>> https://cloud.google.com/datastore/docs/reference/libraries#dependency_on_application
>>
>>
>>
>> On Wed, Sep 23, 2020 at 5:33 PM Robert Burke  wrote:
>>
>>> Perhaps instead of "hello world" the message could refer to a jira about
>>> the Datastore IT tests?
>>> I suspect if we don't we'll just have a repeat of "we shut down app
>>> engine since it was just running a hello world, and the Datastore tests
>>> died".
>>>
>>> On Wed, Sep 23, 2020, 8:14 AM Kamil Wasilewski <
>>> kamil.wasilew...@polidea.com> wrote:
>>>
 An error message that Udi sent is pretty self explanatory. Disabling
 Google App Engine caused Datastore to be not accessible too. What's
 interesting, it doesn't make any difference if the application on GAE is
 actually using Datastore or not. GAE must be simply turned on and that's
 the only requirement.

 I replaced Perfkit Explorer with a simple "hello world" running on
 Python 3.8. I ran Datastore IT tests and they passed, so I think this
 solves the problem. If something goes wrong, let me know!
 Sorry for inconvenience and thanks Tyson for the rescue.

 On Tue, Sep 22, 2020 at 11:25 PM Udi Meiri  wrote:

> Thanks, Tyson!
>
> On Tue, Sep 22, 2020 at 11:11 AM Tyson Hamilton 
> wrote:
>
>> I re-enabled the AppEngine app. Today that app has both the required
>> datastore app and the perfkit app baked into the container image. What
>> should happen, is that the perfkit app is removed from that image, but 
>> the
>> datastore related stuff remains functional.
>>
>> On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri  wrote:
>>
>>> Is it possible to create a simple "hello world" application instead?
>>>
>>> On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri  wrote:
>>>
 Disabling this broke our Datastore ITs. Apparently you must have an
 application for Datastore to work. From the Datastore dashboard:
 The project apache-beam-testing does not exist or it does not
 contain an active Cloud Datastore or Cloud Firestore database. Please 
 visit
 http://console.cloud.google.com to create a project or
 https://console.cloud.google.com/datastore/setup?project=apache-beam-testing
 to add a Cloud Datastore or Cloud Firestore database. Note that Cloud
 Datastore or Cloud Firestore always have an associated App Engine app 
 and
 this app must not be disabled.
 New failure:
 https://ci-beam.apache.org/job/beam_PostCommit_Python36/2959/



 On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski <
 kamil.wasilew...@polidea.com> wrote:

> Thanks. The application has been disabled.
>
> On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay 
> wrote:
>
>> +1. Thank you for the cleanup.
>>
>> On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton <
>> tyso...@google.com> wrote:
>>
>>> +1 to removing, thank you Kamil.
>>>
>>> On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
>>> kamil.wasilew...@polidea.com> wrote:
>>>
 Hello everyone,

 Beam support for Python 2 is coming to an end. Consequently, we
 should make sure no Python 2 applications are running as a part of 
 Beam's
 infrastructure. As you may know, Beam is still hosting a Python 2
 application on Google App Engine. This application is Perfkit 
 Explorer [1].

 Perfkit Explorer has been used as a dashboarding tool for a
 long time. Few months ago, it was deprecated in favour of new 
 Grafana
 dashboards [2].

 Perfkit Explorer doesn't support Python 3, so a viable solution
 to the problem is to shut down Perfkit Explorer co

Re: Shutting down Perfkit Explorer

[Kamil Wasilewski]

I'm not sure if such a jira issue exists. I'm also not convinced that we
need a new one. New jira means there is an action to be taken, now or in
the future. Our goal is only to make sure no one would disable the app
engine accidently.

Putting some information instead of "hello world" is a good idea. I will
update it today. Thank you Tobiasz for the link, it will be very helpful.


On Thu, Sep 24, 2020 at 11:32 AM Tobiasz Kędzierski <
tobiasz.kedzier...@polidea.com> wrote:

> I agree with Robert, putting some information instead of "hello world"
> could prevent disabling GAE and connected problems in the future.
> If a relevant Jira issue exists it would be a great idea to put a link to
> it.
> What do you think about adding a link to GCP documentation mentioning this
> dependency [1] ?
>
> BR Tobiasz
>
> [1]
> https://cloud.google.com/datastore/docs/reference/libraries#dependency_on_application
>
>
>
> On Wed, Sep 23, 2020 at 5:33 PM Robert Burke  wrote:
>
>> Perhaps instead of "hello world" the message could refer to a jira about
>> the Datastore IT tests?
>> I suspect if we don't we'll just have a repeat of "we shut down app
>> engine since it was just running a hello world, and the Datastore tests
>> died".
>>
>> On Wed, Sep 23, 2020, 8:14 AM Kamil Wasilewski <
>> kamil.wasilew...@polidea.com> wrote:
>>
>>> An error message that Udi sent is pretty self explanatory. Disabling
>>> Google App Engine caused Datastore to be not accessible too. What's
>>> interesting, it doesn't make any difference if the application on GAE is
>>> actually using Datastore or not. GAE must be simply turned on and that's
>>> the only requirement.
>>>
>>> I replaced Perfkit Explorer with a simple "hello world" running on
>>> Python 3.8. I ran Datastore IT tests and they passed, so I think this
>>> solves the problem. If something goes wrong, let me know!
>>> Sorry for inconvenience and thanks Tyson for the rescue.
>>>
>>> On Tue, Sep 22, 2020 at 11:25 PM Udi Meiri  wrote:
>>>
 Thanks, Tyson!

 On Tue, Sep 22, 2020 at 11:11 AM Tyson Hamilton 
 wrote:

> I re-enabled the AppEngine app. Today that app has both the required
> datastore app and the perfkit app baked into the container image. What
> should happen, is that the perfkit app is removed from that image, but the
> datastore related stuff remains functional.
>
> On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri  wrote:
>
>> Is it possible to create a simple "hello world" application instead?
>>
>> On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri  wrote:
>>
>>> Disabling this broke our Datastore ITs. Apparently you must have an
>>> application for Datastore to work. From the Datastore dashboard:
>>> The project apache-beam-testing does not exist or it does not
>>> contain an active Cloud Datastore or Cloud Firestore database. Please 
>>> visit
>>> http://console.cloud.google.com to create a project or
>>> https://console.cloud.google.com/datastore/setup?project=apache-beam-testing
>>> to add a Cloud Datastore or Cloud Firestore database. Note that Cloud
>>> Datastore or Cloud Firestore always have an associated App Engine app 
>>> and
>>> this app must not be disabled.
>>> New failure:
>>> https://ci-beam.apache.org/job/beam_PostCommit_Python36/2959/
>>>
>>>
>>>
>>> On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski <
>>> kamil.wasilew...@polidea.com> wrote:
>>>
 Thanks. The application has been disabled.

 On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay 
 wrote:

> +1. Thank you for the cleanup.
>
> On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton 
> wrote:
>
>> +1 to removing, thank you Kamil.
>>
>> On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
>> kamil.wasilew...@polidea.com> wrote:
>>
>>> Hello everyone,
>>>
>>> Beam support for Python 2 is coming to an end. Consequently, we
>>> should make sure no Python 2 applications are running as a part of 
>>> Beam's
>>> infrastructure. As you may know, Beam is still hosting a Python 2
>>> application on Google App Engine. This application is Perfkit 
>>> Explorer [1].
>>>
>>> Perfkit Explorer has been used as a dashboarding tool for a long
>>> time. Few months ago, it was deprecated in favour of new Grafana 
>>> dashboards
>>> [2].
>>>
>>> Perfkit Explorer doesn't support Python 3, so a viable solution
>>> to the problem is to shut down Perfkit Explorer completely. That 
>>> would also
>>> reduce costs, because at the moment we have two similar 
>>> applications doing
>>> the same thing.
>>>
>>> What do you think?
>>> If nobody has any objections, I'd like to shut down Perfkit
>>> Explorer next week.
>

Re: Shutting down Perfkit Explorer

[Tobiasz Kędzierski]

I agree with Robert, putting some information instead of "hello world"
could prevent disabling GAE and connected problems in the future.
If a relevant Jira issue exists it would be a great idea to put a link to
it.
What do you think about adding a link to GCP documentation mentioning this
dependency [1] ?

BR Tobiasz

[1]
https://cloud.google.com/datastore/docs/reference/libraries#dependency_on_application



On Wed, Sep 23, 2020 at 5:33 PM Robert Burke  wrote:

> Perhaps instead of "hello world" the message could refer to a jira about
> the Datastore IT tests?
> I suspect if we don't we'll just have a repeat of "we shut down app engine
> since it was just running a hello world, and the Datastore tests died".
>
> On Wed, Sep 23, 2020, 8:14 AM Kamil Wasilewski <
> kamil.wasilew...@polidea.com> wrote:
>
>> An error message that Udi sent is pretty self explanatory. Disabling
>> Google App Engine caused Datastore to be not accessible too. What's
>> interesting, it doesn't make any difference if the application on GAE is
>> actually using Datastore or not. GAE must be simply turned on and that's
>> the only requirement.
>>
>> I replaced Perfkit Explorer with a simple "hello world" running on Python
>> 3.8. I ran Datastore IT tests and they passed, so I think this solves the
>> problem. If something goes wrong, let me know!
>> Sorry for inconvenience and thanks Tyson for the rescue.
>>
>> On Tue, Sep 22, 2020 at 11:25 PM Udi Meiri  wrote:
>>
>>> Thanks, Tyson!
>>>
>>> On Tue, Sep 22, 2020 at 11:11 AM Tyson Hamilton 
>>> wrote:
>>>
 I re-enabled the AppEngine app. Today that app has both the required
 datastore app and the perfkit app baked into the container image. What
 should happen, is that the perfkit app is removed from that image, but the
 datastore related stuff remains functional.

 On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri  wrote:

> Is it possible to create a simple "hello world" application instead?
>
> On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri  wrote:
>
>> Disabling this broke our Datastore ITs. Apparently you must have an
>> application for Datastore to work. From the Datastore dashboard:
>> The project apache-beam-testing does not exist or it does not contain
>> an active Cloud Datastore or Cloud Firestore database. Please visit
>> http://console.cloud.google.com to create a project or
>> https://console.cloud.google.com/datastore/setup?project=apache-beam-testing
>> to add a Cloud Datastore or Cloud Firestore database. Note that Cloud
>> Datastore or Cloud Firestore always have an associated App Engine app and
>> this app must not be disabled.
>> New failure:
>> https://ci-beam.apache.org/job/beam_PostCommit_Python36/2959/
>>
>>
>>
>> On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski <
>> kamil.wasilew...@polidea.com> wrote:
>>
>>> Thanks. The application has been disabled.
>>>
>>> On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay 
>>> wrote:
>>>
 +1. Thank you for the cleanup.

 On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton 
 wrote:

> +1 to removing, thank you Kamil.
>
> On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
> kamil.wasilew...@polidea.com> wrote:
>
>> Hello everyone,
>>
>> Beam support for Python 2 is coming to an end. Consequently, we
>> should make sure no Python 2 applications are running as a part of 
>> Beam's
>> infrastructure. As you may know, Beam is still hosting a Python 2
>> application on Google App Engine. This application is Perfkit 
>> Explorer [1].
>>
>> Perfkit Explorer has been used as a dashboarding tool for a long
>> time. Few months ago, it was deprecated in favour of new Grafana 
>> dashboards
>> [2].
>>
>> Perfkit Explorer doesn't support Python 3, so a viable solution
>> to the problem is to shut down Perfkit Explorer completely. That 
>> would also
>> reduce costs, because at the moment we have two similar applications 
>> doing
>> the same thing.
>>
>> What do you think?
>> If nobody has any objections, I'd like to shut down Perfkit
>> Explorer next week.
>>
>> Thanks,
>> Kamil
>>
>> [1] https://apache-beam-testing.appspot.com
>> [2] http://metrics.beam.apache.org/
>>
>

Re: Shutting down Perfkit Explorer

[Robert Burke]

Perhaps instead of "hello world" the message could refer to a jira about
the Datastore IT tests?
I suspect if we don't we'll just have a repeat of "we shut down app engine
since it was just running a hello world, and the Datastore tests died".

On Wed, Sep 23, 2020, 8:14 AM Kamil Wasilewski 
wrote:

> An error message that Udi sent is pretty self explanatory. Disabling
> Google App Engine caused Datastore to be not accessible too. What's
> interesting, it doesn't make any difference if the application on GAE is
> actually using Datastore or not. GAE must be simply turned on and that's
> the only requirement.
>
> I replaced Perfkit Explorer with a simple "hello world" running on Python
> 3.8. I ran Datastore IT tests and they passed, so I think this solves the
> problem. If something goes wrong, let me know!
> Sorry for inconvenience and thanks Tyson for the rescue.
>
> On Tue, Sep 22, 2020 at 11:25 PM Udi Meiri  wrote:
>
>> Thanks, Tyson!
>>
>> On Tue, Sep 22, 2020 at 11:11 AM Tyson Hamilton 
>> wrote:
>>
>>> I re-enabled the AppEngine app. Today that app has both the required
>>> datastore app and the perfkit app baked into the container image. What
>>> should happen, is that the perfkit app is removed from that image, but the
>>> datastore related stuff remains functional.
>>>
>>> On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri  wrote:
>>>
 Is it possible to create a simple "hello world" application instead?

 On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri  wrote:

> Disabling this broke our Datastore ITs. Apparently you must have an
> application for Datastore to work. From the Datastore dashboard:
> The project apache-beam-testing does not exist or it does not contain
> an active Cloud Datastore or Cloud Firestore database. Please visit
> http://console.cloud.google.com to create a project or
> https://console.cloud.google.com/datastore/setup?project=apache-beam-testing
> to add a Cloud Datastore or Cloud Firestore database. Note that Cloud
> Datastore or Cloud Firestore always have an associated App Engine app and
> this app must not be disabled.
> New failure:
> https://ci-beam.apache.org/job/beam_PostCommit_Python36/2959/
>
>
>
> On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski <
> kamil.wasilew...@polidea.com> wrote:
>
>> Thanks. The application has been disabled.
>>
>> On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay  wrote:
>>
>>> +1. Thank you for the cleanup.
>>>
>>> On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton 
>>> wrote:
>>>
 +1 to removing, thank you Kamil.

 On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
 kamil.wasilew...@polidea.com> wrote:

> Hello everyone,
>
> Beam support for Python 2 is coming to an end. Consequently, we
> should make sure no Python 2 applications are running as a part of 
> Beam's
> infrastructure. As you may know, Beam is still hosting a Python 2
> application on Google App Engine. This application is Perfkit 
> Explorer [1].
>
> Perfkit Explorer has been used as a dashboarding tool for a long
> time. Few months ago, it was deprecated in favour of new Grafana 
> dashboards
> [2].
>
> Perfkit Explorer doesn't support Python 3, so a viable solution to
> the problem is to shut down Perfkit Explorer completely. That would 
> also
> reduce costs, because at the moment we have two similar applications 
> doing
> the same thing.
>
> What do you think?
> If nobody has any objections, I'd like to shut down Perfkit
> Explorer next week.
>
> Thanks,
> Kamil
>
> [1] https://apache-beam-testing.appspot.com
> [2] http://metrics.beam.apache.org/
>

Re: Shutting down Perfkit Explorer

[Kamil Wasilewski]

An error message that Udi sent is pretty self explanatory. Disabling Google
App Engine caused Datastore to be not accessible too. What's interesting,
it doesn't make any difference if the application on GAE is actually using
Datastore or not. GAE must be simply turned on and that's the only
requirement.

I replaced Perfkit Explorer with a simple "hello world" running on Python
3.8. I ran Datastore IT tests and they passed, so I think this solves the
problem. If something goes wrong, let me know!
Sorry for inconvenience and thanks Tyson for the rescue.

On Tue, Sep 22, 2020 at 11:25 PM Udi Meiri  wrote:

> Thanks, Tyson!
>
> On Tue, Sep 22, 2020 at 11:11 AM Tyson Hamilton 
> wrote:
>
>> I re-enabled the AppEngine app. Today that app has both the required
>> datastore app and the perfkit app baked into the container image. What
>> should happen, is that the perfkit app is removed from that image, but the
>> datastore related stuff remains functional.
>>
>> On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri  wrote:
>>
>>> Is it possible to create a simple "hello world" application instead?
>>>
>>> On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri  wrote:
>>>
 Disabling this broke our Datastore ITs. Apparently you must have an
 application for Datastore to work. From the Datastore dashboard:
 The project apache-beam-testing does not exist or it does not contain
 an active Cloud Datastore or Cloud Firestore database. Please visit
 http://console.cloud.google.com to create a project or
 https://console.cloud.google.com/datastore/setup?project=apache-beam-testing
 to add a Cloud Datastore or Cloud Firestore database. Note that Cloud
 Datastore or Cloud Firestore always have an associated App Engine app and
 this app must not be disabled.
 New failure:
 https://ci-beam.apache.org/job/beam_PostCommit_Python36/2959/



 On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski <
 kamil.wasilew...@polidea.com> wrote:

> Thanks. The application has been disabled.
>
> On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay  wrote:
>
>> +1. Thank you for the cleanup.
>>
>> On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton 
>> wrote:
>>
>>> +1 to removing, thank you Kamil.
>>>
>>> On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
>>> kamil.wasilew...@polidea.com> wrote:
>>>
 Hello everyone,

 Beam support for Python 2 is coming to an end. Consequently, we
 should make sure no Python 2 applications are running as a part of 
 Beam's
 infrastructure. As you may know, Beam is still hosting a Python 2
 application on Google App Engine. This application is Perfkit Explorer 
 [1].

 Perfkit Explorer has been used as a dashboarding tool for a long
 time. Few months ago, it was deprecated in favour of new Grafana 
 dashboards
 [2].

 Perfkit Explorer doesn't support Python 3, so a viable solution to
 the problem is to shut down Perfkit Explorer completely. That would 
 also
 reduce costs, because at the moment we have two similar applications 
 doing
 the same thing.

 What do you think?
 If nobody has any objections, I'd like to shut down Perfkit
 Explorer next week.

 Thanks,
 Kamil

 [1] https://apache-beam-testing.appspot.com
 [2] http://metrics.beam.apache.org/

>>>

Re: Shutting down Perfkit Explorer

[Udi Meiri]

Thanks, Tyson!

On Tue, Sep 22, 2020 at 11:11 AM Tyson Hamilton  wrote:

> I re-enabled the AppEngine app. Today that app has both the required
> datastore app and the perfkit app baked into the container image. What
> should happen, is that the perfkit app is removed from that image, but the
> datastore related stuff remains functional.
>
> On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri  wrote:
>
>> Is it possible to create a simple "hello world" application instead?
>>
>> On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri  wrote:
>>
>>> Disabling this broke our Datastore ITs. Apparently you must have an
>>> application for Datastore to work. From the Datastore dashboard:
>>> The project apache-beam-testing does not exist or it does not contain an
>>> active Cloud Datastore or Cloud Firestore database. Please visit
>>> http://console.cloud.google.com to create a project or
>>> https://console.cloud.google.com/datastore/setup?project=apache-beam-testing
>>> to add a Cloud Datastore or Cloud Firestore database. Note that Cloud
>>> Datastore or Cloud Firestore always have an associated App Engine app and
>>> this app must not be disabled.
>>> New failure:
>>> https://ci-beam.apache.org/job/beam_PostCommit_Python36/2959/
>>>
>>>
>>>
>>> On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski <
>>> kamil.wasilew...@polidea.com> wrote:
>>>
 Thanks. The application has been disabled.

 On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay  wrote:

> +1. Thank you for the cleanup.
>
> On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton 
> wrote:
>
>> +1 to removing, thank you Kamil.
>>
>> On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
>> kamil.wasilew...@polidea.com> wrote:
>>
>>> Hello everyone,
>>>
>>> Beam support for Python 2 is coming to an end. Consequently, we
>>> should make sure no Python 2 applications are running as a part of 
>>> Beam's
>>> infrastructure. As you may know, Beam is still hosting a Python 2
>>> application on Google App Engine. This application is Perfkit Explorer 
>>> [1].
>>>
>>> Perfkit Explorer has been used as a dashboarding tool for a long
>>> time. Few months ago, it was deprecated in favour of new Grafana 
>>> dashboards
>>> [2].
>>>
>>> Perfkit Explorer doesn't support Python 3, so a viable solution to
>>> the problem is to shut down Perfkit Explorer completely. That would also
>>> reduce costs, because at the moment we have two similar applications 
>>> doing
>>> the same thing.
>>>
>>> What do you think?
>>> If nobody has any objections, I'd like to shut down Perfkit Explorer
>>> next week.
>>>
>>> Thanks,
>>> Kamil
>>>
>>> [1] https://apache-beam-testing.appspot.com
>>> [2] http://metrics.beam.apache.org/
>>>
>>


smime.p7s
Description: S/MIME Cryptographic Signature

Re: Shutting down Perfkit Explorer

[Tyson Hamilton]

I re-enabled the AppEngine app. Today that app has both the required
datastore app and the perfkit app baked into the container image. What
should happen, is that the perfkit app is removed from that image, but the
datastore related stuff remains functional.

On Tue, Sep 22, 2020 at 10:37 AM Udi Meiri  wrote:

> Is it possible to create a simple "hello world" application instead?
>
> On Tue, Sep 22, 2020 at 10:35 AM Udi Meiri  wrote:
>
>> Disabling this broke our Datastore ITs. Apparently you must have an
>> application for Datastore to work. From the Datastore dashboard:
>> The project apache-beam-testing does not exist or it does not contain an
>> active Cloud Datastore or Cloud Firestore database. Please visit
>> http://console.cloud.google.com to create a project or
>> https://console.cloud.google.com/datastore/setup?project=apache-beam-testing
>> to add a Cloud Datastore or Cloud Firestore database. Note that Cloud
>> Datastore or Cloud Firestore always have an associated App Engine app and
>> this app must not be disabled.
>> New failure:
>> https://ci-beam.apache.org/job/beam_PostCommit_Python36/2959/
>>
>>
>>
>> On Tue, Sep 22, 2020 at 2:16 AM Kamil Wasilewski <
>> kamil.wasilew...@polidea.com> wrote:
>>
>>> Thanks. The application has been disabled.
>>>
>>> On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay  wrote:
>>>
 +1. Thank you for the cleanup.

 On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton 
 wrote:

> +1 to removing, thank you Kamil.
>
> On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
> kamil.wasilew...@polidea.com> wrote:
>
>> Hello everyone,
>>
>> Beam support for Python 2 is coming to an end. Consequently, we
>> should make sure no Python 2 applications are running as a part of Beam's
>> infrastructure. As you may know, Beam is still hosting a Python 2
>> application on Google App Engine. This application is Perfkit Explorer 
>> [1].
>>
>> Perfkit Explorer has been used as a dashboarding tool for a long
>> time. Few months ago, it was deprecated in favour of new Grafana 
>> dashboards
>> [2].
>>
>> Perfkit Explorer doesn't support Python 3, so a viable solution to
>> the problem is to shut down Perfkit Explorer completely. That would also
>> reduce costs, because at the moment we have two similar applications 
>> doing
>> the same thing.
>>
>> What do you think?
>> If nobody has any objections, I'd like to shut down Perfkit Explorer
>> next week.
>>
>> Thanks,
>> Kamil
>>
>> [1] https://apache-beam-testing.appspot.com
>> [2] http://metrics.beam.apache.org/
>>
>

Re: Shutting down Perfkit Explorer

[Kamil Wasilewski]

Thanks. The application has been disabled.

On Fri, Sep 18, 2020 at 8:46 PM Ahmet Altay  wrote:

> +1. Thank you for the cleanup.
>
> On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton  wrote:
>
>> +1 to removing, thank you Kamil.
>>
>> On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
>> kamil.wasilew...@polidea.com> wrote:
>>
>>> Hello everyone,
>>>
>>> Beam support for Python 2 is coming to an end. Consequently, we should
>>> make sure no Python 2 applications are running as a part of Beam's
>>> infrastructure. As you may know, Beam is still hosting a Python 2
>>> application on Google App Engine. This application is Perfkit Explorer [1].
>>>
>>> Perfkit Explorer has been used as a dashboarding tool for a long time.
>>> Few months ago, it was deprecated in favour of new Grafana dashboards [2].
>>>
>>> Perfkit Explorer doesn't support Python 3, so a viable solution to the
>>> problem is to shut down Perfkit Explorer completely. That would also reduce
>>> costs, because at the moment we have two similar applications doing the
>>> same thing.
>>>
>>> What do you think?
>>> If nobody has any objections, I'd like to shut down Perfkit Explorer
>>> next week.
>>>
>>> Thanks,
>>> Kamil
>>>
>>> [1] https://apache-beam-testing.appspot.com
>>> [2] http://metrics.beam.apache.org/
>>>
>>

Re: Shutting down Perfkit Explorer

[Ahmet Altay]

+1. Thank you for the cleanup.

On Fri, Sep 18, 2020 at 8:24 AM Tyson Hamilton  wrote:

> +1 to removing, thank you Kamil.
>
> On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
> kamil.wasilew...@polidea.com> wrote:
>
>> Hello everyone,
>>
>> Beam support for Python 2 is coming to an end. Consequently, we should
>> make sure no Python 2 applications are running as a part of Beam's
>> infrastructure. As you may know, Beam is still hosting a Python 2
>> application on Google App Engine. This application is Perfkit Explorer [1].
>>
>> Perfkit Explorer has been used as a dashboarding tool for a long time.
>> Few months ago, it was deprecated in favour of new Grafana dashboards [2].
>>
>> Perfkit Explorer doesn't support Python 3, so a viable solution to the
>> problem is to shut down Perfkit Explorer completely. That would also reduce
>> costs, because at the moment we have two similar applications doing the
>> same thing.
>>
>> What do you think?
>> If nobody has any objections, I'd like to shut down Perfkit Explorer next
>> week.
>>
>> Thanks,
>> Kamil
>>
>> [1] https://apache-beam-testing.appspot.com
>> [2] http://metrics.beam.apache.org/
>>
>

Re: Shutting down Perfkit Explorer

[Tyson Hamilton]

+1 to removing, thank you Kamil.

On Fri, Sep 18, 2020 at 6:05 AM Kamil Wasilewski <
kamil.wasilew...@polidea.com> wrote:

> Hello everyone,
>
> Beam support for Python 2 is coming to an end. Consequently, we should
> make sure no Python 2 applications are running as a part of Beam's
> infrastructure. As you may know, Beam is still hosting a Python 2
> application on Google App Engine. This application is Perfkit Explorer [1].
>
> Perfkit Explorer has been used as a dashboarding tool for a long time. Few
> months ago, it was deprecated in favour of new Grafana dashboards [2].
>
> Perfkit Explorer doesn't support Python 3, so a viable solution to the
> problem is to shut down Perfkit Explorer completely. That would also reduce
> costs, because at the moment we have two similar applications doing the
> same thing.
>
> What do you think?
> If nobody has any objections, I'd like to shut down Perfkit Explorer next
> week.
>
> Thanks,
> Kamil
>
> [1] https://apache-beam-testing.appspot.com
> [2] http://metrics.beam.apache.org/
>

Shutting down Perfkit Explorer

[Kamil Wasilewski]

Hello everyone,

Beam support for Python 2 is coming to an end. Consequently, we should make
sure no Python 2 applications are running as a part of Beam's
infrastructure. As you may know, Beam is still hosting a Python 2
application on Google App Engine. This application is Perfkit Explorer [1].

Perfkit Explorer has been used as a dashboarding tool for a long time. Few
months ago, it was deprecated in favour of new Grafana dashboards [2].

Perfkit Explorer doesn't support Python 3, so a viable solution to the
problem is to shut down Perfkit Explorer completely. That would also reduce
costs, because at the moment we have two similar applications doing the
same thing.

What do you think?
If nobody has any objections, I'd like to shut down Perfkit Explorer next
week.

Thanks,
Kamil

[1] https://apache-beam-testing.appspot.com
[2] http://metrics.beam.apache.org/