Re: TEST_START_CLUSTER_ARGS are not applied in custom cluster tests
Thanks MJ and Philip for the replies. I added a comment: https://gerrit.cloudera.org/c/9291/11/tests/common/custom_cluster_test_suite.py#128 Cheers, Lars On Tue, Feb 13, 2018 at 3:06 PM, Matthew Jacobs wrote: > I think that it was an oversight. That said, given that custom cluster > tests may also set the same flags (e.g. mem_limit as Phil pointed out) > I'm not sure the desired behavior would be immediately obvious. It may > just be best to clarify the name and/or comment on > TEST_START_CLUSTER_ARGS. > > On Tue, Feb 13, 2018 at 11:02 AM, Philip Zeyliger > wrote: > > It's definitely handy. I've used it for specifying things like > --mem_limit > > when running inside a container. I think Lars' question is more about > what > > the custom cluster tests should do with it. > > > > -- Philip > > > > On Tue, Feb 13, 2018 at 10:48 AM, Matthew Jacobs > > wrote: > > > >> I had introduced this while I was working at Cloudera for jenkins jobs > >> to specify RM-related parameters to the test environment. At that > >> time, I had one or more jenkins jobs that were setting this > >> environment variable. I do not know if Cloudera or any other Impala > >> developers are trying to use this. If nobody seems to be using this > >> anymore, it would be good to remove. > >> > >> Best, > >> mj > >> > >> commit adf4b4863d7ea38fda21ec76f5699b4fd31e10c0 > >> Author: Matthew Jacobs > >> Date: Wed May 13 17:14:11 2015 -0700 > >> > >> Allow specifying cluster start args in run-all-tests.sh > >> > >> This will enable jenkins jobs to specify custom arguments when > >> starting the mini cluster (via start-impala-cluster.py). This > >> will be used to create a jenkins job that runs tests with RM > >> enabled. > >> > >> Change-Id: I96a2e8d90db448581bbf448f3df514381f79fb27 > >> Reviewed-on: http://gerrit.cloudera.org:8080/380 > >> Reviewed-by: Matthew Jacobs > >> Tested-by: Internal Jenkins > >> > >> On Tue, Feb 13, 2018 at 10:18 AM, Lars Volker wrote: > >> > Hi All, > >> > > >> > Mike noticed in a review that TEST_START_CLUSTER_ARGS are not applied > in > >> > custom cluster tests and we wondered whether that is on purpose or a > bug. > >> > Can someone with historic knowledge shed some light on it? The > relevant > >> > code is here: > >> > > >> > https://github.com/apache/impala/blob/5f7599687748b1e1ce0a5a34d38cc4 > >> 9a4c4cd9f5/tests/common/custom_cluster_test_suite.py#L124 > >> > > >> > Thanks, Lars > >> >
Re: TEST_START_CLUSTER_ARGS are not applied in custom cluster tests
I think that it was an oversight. That said, given that custom cluster tests may also set the same flags (e.g. mem_limit as Phil pointed out) I'm not sure the desired behavior would be immediately obvious. It may just be best to clarify the name and/or comment on TEST_START_CLUSTER_ARGS. On Tue, Feb 13, 2018 at 11:02 AM, Philip Zeyliger wrote: > It's definitely handy. I've used it for specifying things like --mem_limit > when running inside a container. I think Lars' question is more about what > the custom cluster tests should do with it. > > -- Philip > > On Tue, Feb 13, 2018 at 10:48 AM, Matthew Jacobs > wrote: > >> I had introduced this while I was working at Cloudera for jenkins jobs >> to specify RM-related parameters to the test environment. At that >> time, I had one or more jenkins jobs that were setting this >> environment variable. I do not know if Cloudera or any other Impala >> developers are trying to use this. If nobody seems to be using this >> anymore, it would be good to remove. >> >> Best, >> mj >> >> commit adf4b4863d7ea38fda21ec76f5699b4fd31e10c0 >> Author: Matthew Jacobs >> Date: Wed May 13 17:14:11 2015 -0700 >> >> Allow specifying cluster start args in run-all-tests.sh >> >> This will enable jenkins jobs to specify custom arguments when >> starting the mini cluster (via start-impala-cluster.py). This >> will be used to create a jenkins job that runs tests with RM >> enabled. >> >> Change-Id: I96a2e8d90db448581bbf448f3df514381f79fb27 >> Reviewed-on: http://gerrit.cloudera.org:8080/380 >> Reviewed-by: Matthew Jacobs >> Tested-by: Internal Jenkins >> >> On Tue, Feb 13, 2018 at 10:18 AM, Lars Volker wrote: >> > Hi All, >> > >> > Mike noticed in a review that TEST_START_CLUSTER_ARGS are not applied in >> > custom cluster tests and we wondered whether that is on purpose or a bug. >> > Can someone with historic knowledge shed some light on it? The relevant >> > code is here: >> > >> > https://github.com/apache/impala/blob/5f7599687748b1e1ce0a5a34d38cc4 >> 9a4c4cd9f5/tests/common/custom_cluster_test_suite.py#L124 >> > >> > Thanks, Lars >>
Re: TEST_START_CLUSTER_ARGS are not applied in custom cluster tests
It's definitely handy. I've used it for specifying things like --mem_limit when running inside a container. I think Lars' question is more about what the custom cluster tests should do with it. -- Philip On Tue, Feb 13, 2018 at 10:48 AM, Matthew Jacobs wrote: > I had introduced this while I was working at Cloudera for jenkins jobs > to specify RM-related parameters to the test environment. At that > time, I had one or more jenkins jobs that were setting this > environment variable. I do not know if Cloudera or any other Impala > developers are trying to use this. If nobody seems to be using this > anymore, it would be good to remove. > > Best, > mj > > commit adf4b4863d7ea38fda21ec76f5699b4fd31e10c0 > Author: Matthew Jacobs > Date: Wed May 13 17:14:11 2015 -0700 > > Allow specifying cluster start args in run-all-tests.sh > > This will enable jenkins jobs to specify custom arguments when > starting the mini cluster (via start-impala-cluster.py). This > will be used to create a jenkins job that runs tests with RM > enabled. > > Change-Id: I96a2e8d90db448581bbf448f3df514381f79fb27 > Reviewed-on: http://gerrit.cloudera.org:8080/380 > Reviewed-by: Matthew Jacobs > Tested-by: Internal Jenkins > > On Tue, Feb 13, 2018 at 10:18 AM, Lars Volker wrote: > > Hi All, > > > > Mike noticed in a review that TEST_START_CLUSTER_ARGS are not applied in > > custom cluster tests and we wondered whether that is on purpose or a bug. > > Can someone with historic knowledge shed some light on it? The relevant > > code is here: > > > > https://github.com/apache/impala/blob/5f7599687748b1e1ce0a5a34d38cc4 > 9a4c4cd9f5/tests/common/custom_cluster_test_suite.py#L124 > > > > Thanks, Lars >
Re: TEST_START_CLUSTER_ARGS are not applied in custom cluster tests
I had introduced this while I was working at Cloudera for jenkins jobs to specify RM-related parameters to the test environment. At that time, I had one or more jenkins jobs that were setting this environment variable. I do not know if Cloudera or any other Impala developers are trying to use this. If nobody seems to be using this anymore, it would be good to remove. Best, mj commit adf4b4863d7ea38fda21ec76f5699b4fd31e10c0 Author: Matthew Jacobs Date: Wed May 13 17:14:11 2015 -0700 Allow specifying cluster start args in run-all-tests.sh This will enable jenkins jobs to specify custom arguments when starting the mini cluster (via start-impala-cluster.py). This will be used to create a jenkins job that runs tests with RM enabled. Change-Id: I96a2e8d90db448581bbf448f3df514381f79fb27 Reviewed-on: http://gerrit.cloudera.org:8080/380 Reviewed-by: Matthew Jacobs Tested-by: Internal Jenkins On Tue, Feb 13, 2018 at 10:18 AM, Lars Volker wrote: > Hi All, > > Mike noticed in a review that TEST_START_CLUSTER_ARGS are not applied in > custom cluster tests and we wondered whether that is on purpose or a bug. > Can someone with historic knowledge shed some light on it? The relevant > code is here: > > https://github.com/apache/impala/blob/5f7599687748b1e1ce0a5a34d38cc49a4c4cd9f5/tests/common/custom_cluster_test_suite.py#L124 > > Thanks, Lars
TEST_START_CLUSTER_ARGS are not applied in custom cluster tests
Hi All, Mike noticed in a review that TEST_START_CLUSTER_ARGS are not applied in custom cluster tests and we wondered whether that is on purpose or a bug. Can someone with historic knowledge shed some light on it? The relevant code is here: https://github.com/apache/impala/blob/5f7599687748b1e1ce0a5a34d38cc49a4c4cd9f5/tests/common/custom_cluster_test_suite.py#L124 Thanks, Lars
