Re: Mark Miller's beasting script what do we do instead if we remove Ant from trunk.
I was trying to make this work but ran into a few walls of my own gradle
knowledge. I was doing this with a trivial project, not the whole of
lucene, so I'm sure there will be additional integration pains.
Regardless, sharing my progress thus far:
configure(rootProject) {
Integer.getInteger("beast.iters", 3).times {
tasks.create(name: "beastRun$it", type:Test)
}
task beast() {
dependsOn rootProject.collect {
tasks.matching { task -> task.name.startsWith("beastRun") }
}
}
}
I was able to get this to execute a test multiple times, and save the
output to separate test directories (build/test-results/beastRunN). It
respects -Dbeast.iters so you can control the number of executions
Things still to address
- this pollutes the task graph and especially if you are running 'gradle
tasks' it gets messy. Probably need to use register instead of create, but
I couldn't make that work with the API.
- I couldn't get them to run in parallel, they were running serially for me
- I didn't get this to accept the standard test selection flags.
Maybe I'll have more time during the week, or maybe somebody else will run
with this.
Mike
On Sat, Aug 15, 2020 at 4:11 PM Dawid Weiss wrote:
> I suspect the "beasting" script would be best simulated by creating X
> test tasks (with different names but identical configuration). Then
> gradle can take care of parallelism serving those tasks to its worker
> VMs. This would accomplish many things at once: everything would run
> from within a single gradle parent process (daemon or not), tests
> would be run in parallel, the first failure could abort execution of
> subsequent tasks, etc.
>
> There are some open questions - whether beasting should run with the
> same temporary folders (probably not) whether gradle will be smart
> enough to collect report for multiple test with the same name
> (probably not)... you know - the devil is in the details. But I think
> it'd be an ideal way to do it: no need for external trickery, scripts,
> multi-platform compatibility, etc.
>
> Anyone up to the challenge? :)
>
> Dawid
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
Re: Mark Miller's beasting script what do we do instead if we remove Ant from trunk.
I suspect the "beasting" script would be best simulated by creating X test tasks (with different names but identical configuration). Then gradle can take care of parallelism serving those tasks to its worker VMs. This would accomplish many things at once: everything would run from within a single gradle parent process (daemon or not), tests would be run in parallel, the first failure could abort execution of subsequent tasks, etc. There are some open questions - whether beasting should run with the same temporary folders (probably not) whether gradle will be smart enough to collect report for multiple test with the same name (probably not)... you know - the devil is in the details. But I think it'd be an ideal way to do it: no need for external trickery, scripts, multi-platform compatibility, etc. Anyone up to the challenge? :) Dawid - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Namgyu Kim to the PMC
Welcome, Namgyu! On Thu, Aug 6, 2020 at 9:40 PM Yonik Seeley wrote: > > Congrats Namgyu! > > -Yonik > > > On Sun, Aug 2, 2020 at 7:19 PM Ishan Chattopadhyaya > wrote: >> >> I am pleased to announce that Namgyu Kim has accepted the PMC's invitation >> to join. >> >> Congratulations and welcome, Namgyu! - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Munendra SN to the PMC
Welcome, Munendra! On Thu, Aug 6, 2020 at 9:38 PM Yonik Seeley wrote: > Congrats Munendra! > > -Yonik > > > On Sun, Aug 2, 2020 at 7:20 PM Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote: > >> I am pleased to announce that Munendra SN has accepted the PMC's >> invitation to join. >> >> Congratulations and welcome, Munendra! >> >
Re: Welcome Gus Heck to the PMC
Welcome, Gus! On Thu, Aug 6, 2020 at 9:39 PM Yonik Seeley wrote: > > Congrats Gus! > > -Yonik > > On Sun, Aug 2, 2020 at 7:21 PM Ishan Chattopadhyaya > wrote: >> >> I am pleased to announce that Gus Heck has accepted the PMC's invitation to >> join. >> >> Congratulations and welcome, Gus! - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Mike Drob to the PMC
Late add from me too, Mike: Welcome! On Fri, Jul 31, 2020 at 8:16 AM Noble Paul wrote: > > Welcome Mike! > > On Thu, Jul 30, 2020 at 12:33 AM Erik Hatcher wrote: > > > > Oh yeah! Welcome, Mike! > > > > > On Jul 24, 2020, at 3:55 PM, Anshum Gupta wrote: > > > > > > I am pleased to announce that Mike Drob has accepted the PMC's invitation > > > to join. > > > > > > Congratulations and welcome, Mike! > > > > > > -- > > > Anshum Gupta > > > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > -- > - > Noble Paul > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Michael Sokolov to the PMC
Thanks, fellow necro-posters ! On Thu, Jul 30, 2020 at 10:20 AM Noble Paul wrote: > > Congratulations and welcome, Mike! > > On Sat, Jul 25, 2020 at 3:07 AM Christine Poerschke (BLOOMBERG/ > LONDON) wrote: > > > > Welcome Mike! > > > > From: dev@lucene.apache.org At: 07/03/20 13:15:05 > > To: dev@lucene.apache.org > > Subject: Re: Welcome Michael Sokolov to the PMC > > > > Thanks Adrien, and to the whole PMC > > > > Mike > > > > On Fri, Jul 3, 2020, 7:57 AM Adrien Grand wrote: > >> > >> I am pleased to announce that Michael Sokolov has accepted the PMC's > >> invitation to join. > >> > >> Welcome Michael! > >> > >> -- > >> Adrien > > > > > > > -- > - > Noble Paul > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Mark Miller's beasting script what do we do instead if we remove Ant from trunk.
And to answer the original question, I've used it from time to time.
Also I think "gradle magic" might be best if it can ensure that gradle
doesn't go through the process of checking for the need to recompile etc on
every cycle.
I'm against containers being required for something like this because my
desktop BIOS fails to support virtualization correctly so some things
related to containerization don't want to run it or run in limited ways. If
it's really the best option for everyone else, fine, but I'm not
enthusiastic about it.
@Robert Muir mark's script does use parallel.
Also I suspect I can guess, but could we define "pure unit test" to ensure
I haven't got a different concept of that than what you mean?
On Sat, Aug 15, 2020 at 11:25 AM Gus Heck wrote:
> If you're beasting a single test that won't take advantage of multiple
> processors...
>
> On Sat, Aug 15, 2020 at 10:31 AM Robert Muir wrote:
>
>> Is a beasting script needed with gradle? Can't you do it efficiently with
>> a simple shell script loop? If you have the gradle daemon running, overhead
>> will be minimal (unlike ant) ? That's kinda the point of the daemon right?
>>
>> Here's the kind of loop i typically use for stuff like this, right from
>> bash console: log the output for each run to individual files and just
>> print the exit status so i can see progress as it goes.
>>
>> for i in {1..100}; do
>> gradlew test ... > /tmp/$i.log 2>&1
>> echo -n $?
>> done
>>
>> On Sat, Aug 15, 2020 at 9:37 AM Erick Erickson
>> wrote:
>>
>>> I often use “The best Lucene / Solr beasting script in the world. TM.” (
>>> https://gist.github.com/markrmiller/dbdb792216dc98b018ad).
>>>
>>> The it struck me out of the blue (well actually since I’ve been beasting
>>> for SOLR-14750 a lot), “Gosh, when we remove Ant from trunk I won’t be able
>>> to use the beast script”.
>>>
>>> So I wanted to ask how many people use it and what people’s thoughts are
>>> about what to do after we remove Ant from trunk. Off the top of my head,
>>> options are:
>>>
>>> 1> find some Gradle magic that will do this.
>>>
>>> 2> modify the script to work with Gradle instead (if possible).
>>>
>>> 3> just do without it or an equivalent (bo!)
>>>
>>> 4> In the brave new world of containers, is there a simple way to fire
>>> up N simultaneous but independent instances of the _same_ test and keep
>>> them running until the limit is reached? We’d need to be able to change
>>> code and fire up the script without too much work, and having the results
>>> all written to beast/res/1/(1, 2, 3, 4, 5)/stdout is necessary, as well as
>>> being able to have an “all succeeded” message at the end.
>>>
>>> 5> ???
>>>
>>> I’ll ping mark and see if he’s got some magic that he’s been using with
>>> the reference impl.
>>>
>>> I don’t think tests.iters is a good replacement for two reasons:
>>>
>>> a> part of the usefulness of the beasting script is that it fires up a
>>> bunch of parallel tests, stressing the system.
>>>
>>> b> last I knew, it needs to be run for pure unit tests, and Solr has
>>> lots of tests that are not.
>>> This is from Robert Muir from some time ago specific to Ant, would
>>> it still be valid?
>>>
>>> ---
>>> Long answer: It has been discussed on the list several times.
>>> tests.iters does not do what a lot of people seem to think.
>>> From what i remember: it creates your test *class* a single time and
>>> runs stuff repeatedly... I think only (new *instance*, setup,
>>> teardown, etc). I don't even think it runs e.g. beforeclass/afterclass
>>> stuff between iterations either, but that still wouldnt make it
>>> completely safe anyway unless the whole class was recreated (a test
>>> could leave a static variable in a different state or something).
>>>
>>> So if your test isn't a pure unit test, its probably not ready for
>>> tests.iters. Just use ant beast, it works :)
>>> ---
>>>
>>>
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: [VOTE] Solr to become a top-level Apache project (TLP)
Just catching up after a couple of weeks away here; phew there is lots going on! Re splitting the projects, there has been some project reorganization (we created a new TLP in Apache), but no real change yet. I think the idea is to get the software separated, and then people will follow their hearts/interests. For tracking the software re-org, we have this issue: https://issues.apache.org/jira/browse/SOLR-14497 On Sat, Jul 25, 2020 at 1:41 AM Marcus Eagan wrote: > > Hi All, > > When do these projects separate into their own TLP's? What is the timeline? > Or is there documentation on the process of splitting an existing project > into two separate projects? > > Best, > > Marcus > > On Tue, May 19, 2020 at 1:51 AM Dawid Weiss wrote: >> >> It is now Tuesday, May 19, 2020, 8:46 AM GMT. >> >> A re-spin of the vote closing/ summary. >> >> 48 votes have been posted: 37 from PMC members, 8 from committers and >> 3 from user community. >> >> From this total 8 people voted -1 and 40 people voted +1. Vote >> distribution among PMC members (binding votes): >> >> [+1]: 33 >> [-1]: 4 >> >> The vote has passed. >> >> Dawid >> >> All collected votes in a spreadsheet: >> >> https://docs.google.com/spreadsheets/d/1ZmR3C2EgA57QIeJJ3ejKCTUkHdG1lqPOU29heqybkKs/edit?usp=sharing >> >> And in plain text: >> >> +1 >> >> PMC Ishan Chattopadhyaya (ichattopadhy...@gmail.com) >> PMC Doron Cohen (cdor...@gmail.com) >> PMC Shai Erera (ser...@gmail.com) >> PMC Ryan Ernst (r...@iernst.net) >> COMMITTER Jim Ferenczi (jim.feren...@gmail.com) >> PMC Otis Gospodnetic (otis.gospodne...@gmail.com) >> PMC Dennis Gove (dpg...@gmail.com) >> PMC Adrien Grand (jpou...@gmail.com) >> PMC Martijn v Groningen (martijn.v.gronin...@gmail.com) >> PMC Mark Harwood (mharw...@apache.org) >> PMC Erik Hatcher (erik.hatc...@gmail.com) >> PMC Shawn Heisey (apa...@elyograg.org) >> PMC Jan Høydahl (jan@cominvent.com) >> COMMITTER Namgyu Kim (kng0...@gmail.com) >> PMC Nicholas Knize (nkn...@gmail.com) >> PMC Shalin Shekhar Mangar (shalinman...@gmail.com) >> PMC Michael McCandless (luc...@mikemccandless.com) >> PMC Christian Moen (c...@atilika.com) >> -- Gora Mohanty (g...@mimirtech.com) >> PMC Robert Muir (rcm...@gmail.com) >> PMC Nhat Nguyen (nhat.ngu...@elastic.co.invalid) >> PMC Kevin Risden (kris...@apache.org) >> PMC Steven A Rowe (sar...@gmail.com) >> PMC Uwe Schindler (u...@thetaphi.de) >> PMC Koji Sekiguchi (koji.sekigu...@rondhuit.com) >> COMMITTER Atri Sharma (a...@apache.org) >> -- Lucky Sharma (goku0...@gmail.com) >> PMC David Smiley (david.w.smi...@gmail.com) >> COMMITTER Michael Sokolov (msoko...@gmail.com) >> PMC Tommaso Teofili (tommaso.teof...@gmail.com) >> PMC Varun Thacker (va...@vthacker.in) >> COMMITTER Tomoko Uchida (tomoko.uchida.1...@gmail.com) >> PMC Andi Vajda (o...@ovaltofu.org) >> PMC Ignacio Vera (iver...@gmail.com) >> PMC Dawid Weiss (dawid.we...@gmail.com) >> PMC Simon Willnauer (simon.willna...@gmail.com) >> PMC Alan Woodward (romseyg...@gmail.com) >> PMC Karl Wright (daddy...@gmail.com) >> COMMITTER Alexandre Rafalovitch (arafa...@gmail.com) >> PMC Andrzej Białecki (a...@getopt.org) >> >> -1 >> >> PMC Joel Bernstein (joels...@gmail.com) >> COMMITTER Mike Drob (md...@apache.org) >> PMC Jason Gerlowski (gerlowsk...@gmail.com) >> PMC Anshum Gupta (ans...@anshumgupta.net) >> COMMITTER Gus Heck (gus.h...@gmail.com) >> COMMITTER Chris Hostetter (hossman_luc...@fucit.org) >> PMC Tomás Fernández Löbbe (tomasflo...@gmail.com) >> -- Kevin Watters (kwatt...@kmwllc.com) >> >> On Tue, May 12, 2020 at 9:36 AM Dawid Weiss wrote: >> > >> > Dear Lucene and Solr developers! >> > >> > According to an earlier [DISCUSS] thread on the dev list [2], I am >> > calling for a vote on the proposal to make Solr a top-level Apache >> > project (TLP) and separate Lucene and Solr development into two >> > independent entities. >> > >> > To quickly recap the reasons and consequences of such a move: it seems >> > like the reasons for the initial merge of Lucene and Solr, around 10 >> > years ago, have been achieved. Both projects are in good shape and >> > exhibit signs of independence already (mailing lists, committers, >> > patch flow). There are many technical considerations that would make >> > development much easier if we move Solr out into its own TLP. >> > >> > We discussed this issue [2] and both PMC members and committers had a >> > chance to review all the pros and cons and express their views. The >> > discussion showed that there are clearly different opinions on the >> > matter - some people are in favor, some are neutral, others are >> > against or not seeing the point of additional labor. Realistically, I >> > don't think reaching 100% level consensus is going to be possible -- >> > we are a diverse bunch with different opinions and personalities. I >> > firmly believe this is the right direction hence the decision to put >> > it under the voting process. Should something take a wrong turn in the >> > future (as some folks worry it may), a
Re: Mark Miller's beasting script what do we do instead if we remove Ant from trunk.
sure but you can probably install a tool like 'parallel' if you want that.
On Sat, Aug 15, 2020 at 11:25 AM Gus Heck wrote:
> If you're beasting a single test that won't take advantage of multiple
> processors...
>
> On Sat, Aug 15, 2020 at 10:31 AM Robert Muir wrote:
>
>> Is a beasting script needed with gradle? Can't you do it efficiently with
>> a simple shell script loop? If you have the gradle daemon running, overhead
>> will be minimal (unlike ant) ? That's kinda the point of the daemon right?
>>
>> Here's the kind of loop i typically use for stuff like this, right from
>> bash console: log the output for each run to individual files and just
>> print the exit status so i can see progress as it goes.
>>
>> for i in {1..100}; do
>> gradlew test ... > /tmp/$i.log 2>&1
>> echo -n $?
>> done
>>
>> On Sat, Aug 15, 2020 at 9:37 AM Erick Erickson
>> wrote:
>>
>>> I often use “The best Lucene / Solr beasting script in the world. TM.” (
>>> https://gist.github.com/markrmiller/dbdb792216dc98b018ad).
>>>
>>> The it struck me out of the blue (well actually since I’ve been beasting
>>> for SOLR-14750 a lot), “Gosh, when we remove Ant from trunk I won’t be able
>>> to use the beast script”.
>>>
>>> So I wanted to ask how many people use it and what people’s thoughts are
>>> about what to do after we remove Ant from trunk. Off the top of my head,
>>> options are:
>>>
>>> 1> find some Gradle magic that will do this.
>>>
>>> 2> modify the script to work with Gradle instead (if possible).
>>>
>>> 3> just do without it or an equivalent (bo!)
>>>
>>> 4> In the brave new world of containers, is there a simple way to fire
>>> up N simultaneous but independent instances of the _same_ test and keep
>>> them running until the limit is reached? We’d need to be able to change
>>> code and fire up the script without too much work, and having the results
>>> all written to beast/res/1/(1, 2, 3, 4, 5)/stdout is necessary, as well as
>>> being able to have an “all succeeded” message at the end.
>>>
>>> 5> ???
>>>
>>> I’ll ping mark and see if he’s got some magic that he’s been using with
>>> the reference impl.
>>>
>>> I don’t think tests.iters is a good replacement for two reasons:
>>>
>>> a> part of the usefulness of the beasting script is that it fires up a
>>> bunch of parallel tests, stressing the system.
>>>
>>> b> last I knew, it needs to be run for pure unit tests, and Solr has
>>> lots of tests that are not.
>>> This is from Robert Muir from some time ago specific to Ant, would
>>> it still be valid?
>>>
>>> ---
>>> Long answer: It has been discussed on the list several times.
>>> tests.iters does not do what a lot of people seem to think.
>>> From what i remember: it creates your test *class* a single time and
>>> runs stuff repeatedly... I think only (new *instance*, setup,
>>> teardown, etc). I don't even think it runs e.g. beforeclass/afterclass
>>> stuff between iterations either, but that still wouldnt make it
>>> completely safe anyway unless the whole class was recreated (a test
>>> could leave a static variable in a different state or something).
>>>
>>> So if your test isn't a pure unit test, its probably not ready for
>>> tests.iters. Just use ant beast, it works :)
>>> ---
>>>
>>>
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>
Re: Mark Miller's beasting script what do we do instead if we remove Ant from trunk.
If you're beasting a single test that won't take advantage of multiple
processors...
On Sat, Aug 15, 2020 at 10:31 AM Robert Muir wrote:
> Is a beasting script needed with gradle? Can't you do it efficiently with
> a simple shell script loop? If you have the gradle daemon running, overhead
> will be minimal (unlike ant) ? That's kinda the point of the daemon right?
>
> Here's the kind of loop i typically use for stuff like this, right from
> bash console: log the output for each run to individual files and just
> print the exit status so i can see progress as it goes.
>
> for i in {1..100}; do
> gradlew test ... > /tmp/$i.log 2>&1
> echo -n $?
> done
>
> On Sat, Aug 15, 2020 at 9:37 AM Erick Erickson
> wrote:
>
>> I often use “The best Lucene / Solr beasting script in the world. TM.” (
>> https://gist.github.com/markrmiller/dbdb792216dc98b018ad).
>>
>> The it struck me out of the blue (well actually since I’ve been beasting
>> for SOLR-14750 a lot), “Gosh, when we remove Ant from trunk I won’t be able
>> to use the beast script”.
>>
>> So I wanted to ask how many people use it and what people’s thoughts are
>> about what to do after we remove Ant from trunk. Off the top of my head,
>> options are:
>>
>> 1> find some Gradle magic that will do this.
>>
>> 2> modify the script to work with Gradle instead (if possible).
>>
>> 3> just do without it or an equivalent (bo!)
>>
>> 4> In the brave new world of containers, is there a simple way to fire up
>> N simultaneous but independent instances of the _same_ test and keep them
>> running until the limit is reached? We’d need to be able to change code and
>> fire up the script without too much work, and having the results all
>> written to beast/res/1/(1, 2, 3, 4, 5)/stdout is necessary, as well as
>> being able to have an “all succeeded” message at the end.
>>
>> 5> ???
>>
>> I’ll ping mark and see if he’s got some magic that he’s been using with
>> the reference impl.
>>
>> I don’t think tests.iters is a good replacement for two reasons:
>>
>> a> part of the usefulness of the beasting script is that it fires up a
>> bunch of parallel tests, stressing the system.
>>
>> b> last I knew, it needs to be run for pure unit tests, and Solr has lots
>> of tests that are not.
>> This is from Robert Muir from some time ago specific to Ant, would it
>> still be valid?
>>
>> ---
>> Long answer: It has been discussed on the list several times.
>> tests.iters does not do what a lot of people seem to think.
>> From what i remember: it creates your test *class* a single time and
>> runs stuff repeatedly... I think only (new *instance*, setup,
>> teardown, etc). I don't even think it runs e.g. beforeclass/afterclass
>> stuff between iterations either, but that still wouldnt make it
>> completely safe anyway unless the whole class was recreated (a test
>> could leave a static variable in a different state or something).
>>
>> So if your test isn't a pure unit test, its probably not ready for
>> tests.iters. Just use ant beast, it works :)
>> ---
>>
>>
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
--
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)
Re: Mark Miller's beasting script what do we do instead if we remove Ant from trunk.
Is a beasting script needed with gradle? Can't you do it efficiently with a
simple shell script loop? If you have the gradle daemon running, overhead
will be minimal (unlike ant) ? That's kinda the point of the daemon right?
Here's the kind of loop i typically use for stuff like this, right from
bash console: log the output for each run to individual files and just
print the exit status so i can see progress as it goes.
for i in {1..100}; do
gradlew test ... > /tmp/$i.log 2>&1
echo -n $?
done
On Sat, Aug 15, 2020 at 9:37 AM Erick Erickson
wrote:
> I often use “The best Lucene / Solr beasting script in the world. TM.” (
> https://gist.github.com/markrmiller/dbdb792216dc98b018ad).
>
> The it struck me out of the blue (well actually since I’ve been beasting
> for SOLR-14750 a lot), “Gosh, when we remove Ant from trunk I won’t be able
> to use the beast script”.
>
> So I wanted to ask how many people use it and what people’s thoughts are
> about what to do after we remove Ant from trunk. Off the top of my head,
> options are:
>
> 1> find some Gradle magic that will do this.
>
> 2> modify the script to work with Gradle instead (if possible).
>
> 3> just do without it or an equivalent (bo!)
>
> 4> In the brave new world of containers, is there a simple way to fire up
> N simultaneous but independent instances of the _same_ test and keep them
> running until the limit is reached? We’d need to be able to change code and
> fire up the script without too much work, and having the results all
> written to beast/res/1/(1, 2, 3, 4, 5)/stdout is necessary, as well as
> being able to have an “all succeeded” message at the end.
>
> 5> ???
>
> I’ll ping mark and see if he’s got some magic that he’s been using with
> the reference impl.
>
> I don’t think tests.iters is a good replacement for two reasons:
>
> a> part of the usefulness of the beasting script is that it fires up a
> bunch of parallel tests, stressing the system.
>
> b> last I knew, it needs to be run for pure unit tests, and Solr has lots
> of tests that are not.
> This is from Robert Muir from some time ago specific to Ant, would it
> still be valid?
>
> ---
> Long answer: It has been discussed on the list several times.
> tests.iters does not do what a lot of people seem to think.
> From what i remember: it creates your test *class* a single time and
> runs stuff repeatedly... I think only (new *instance*, setup,
> teardown, etc). I don't even think it runs e.g. beforeclass/afterclass
> stuff between iterations either, but that still wouldnt make it
> completely safe anyway unless the whole class was recreated (a test
> could leave a static variable in a different state or something).
>
> So if your test isn't a pure unit test, its probably not ready for
> tests.iters. Just use ant beast, it works :)
> ---
>
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
Mark Miller's beasting script what do we do instead if we remove Ant from trunk.
I often use “The best Lucene / Solr beasting script in the world. TM.” (https://gist.github.com/markrmiller/dbdb792216dc98b018ad). The it struck me out of the blue (well actually since I’ve been beasting for SOLR-14750 a lot), “Gosh, when we remove Ant from trunk I won’t be able to use the beast script”. So I wanted to ask how many people use it and what people’s thoughts are about what to do after we remove Ant from trunk. Off the top of my head, options are: 1> find some Gradle magic that will do this. 2> modify the script to work with Gradle instead (if possible). 3> just do without it or an equivalent (bo!) 4> In the brave new world of containers, is there a simple way to fire up N simultaneous but independent instances of the _same_ test and keep them running until the limit is reached? We’d need to be able to change code and fire up the script without too much work, and having the results all written to beast/res/1/(1, 2, 3, 4, 5)/stdout is necessary, as well as being able to have an “all succeeded” message at the end. 5> ??? I’ll ping mark and see if he’s got some magic that he’s been using with the reference impl. I don’t think tests.iters is a good replacement for two reasons: a> part of the usefulness of the beasting script is that it fires up a bunch of parallel tests, stressing the system. b> last I knew, it needs to be run for pure unit tests, and Solr has lots of tests that are not. This is from Robert Muir from some time ago specific to Ant, would it still be valid? --- Long answer: It has been discussed on the list several times. tests.iters does not do what a lot of people seem to think. From what i remember: it creates your test *class* a single time and runs stuff repeatedly... I think only (new *instance*, setup, teardown, etc). I don't even think it runs e.g. beforeclass/afterclass stuff between iterations either, but that still wouldnt make it completely safe anyway unless the whole class was recreated (a test could leave a static variable in a different state or something). So if your test isn't a pure unit test, its probably not ready for tests.iters. Just use ant beast, it works :) --- - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Namgyu Kim to the PMC
Welcome Namgyu! On Sat, Aug 15, 2020 at 12:34 AM Christine Poerschke (BLOOMBERG/ LONDON) wrote: > > Welcome Namgyu! > > From: dev@lucene.apache.org At: 08/07/20 02:39:56 > To: dev@lucene.apache.org > Subject: Re: Welcome Namgyu Kim to the PMC > > Congrats Namgyu! > > -Yonik > > > On Sun, Aug 2, 2020 at 7:19 PM Ishan Chattopadhyaya > wrote: >> >> I am pleased to announce that Namgyu Kim has accepted the PMC's invitation >> to join. >> >> Congratulations and welcome, Namgyu! > > -- - Noble Paul - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Survey on ManagedResources feature
>I use MangedResource#StorageIO and its implementations as a convenient way to >abstract away the underlying config storage when creating plugins that need to >support both, SolrCloud and Solr Standalone. Can you give us some more details on how you use it? On Sat, Aug 15, 2020 at 7:32 PM Noble Paul wrote: > > >As authentication is plugged into the SolrDispatchFilter I would assume that > >you would need to be authenticated to read/write Managed Resources > > I'm talking about the authorization plugins > > On Fri, Aug 14, 2020 at 10:20 PM Matthias Krueger wrote: > > > > > > As authentication is plugged into the SolrDispatchFilter I would assume > > that you would need to be authenticated to read/write Managed Resources but > > no authorization is checked (i.e. any authenticated user can read/write > > them), correct? > > > > Anyway, I came across Managed Resources in at least two scenarios: > > > > The LTR plugin is using them for updating model/features. > > I use MangedResource#StorageIO and its implementations as a convenient way > > to abstract away the underlying config storage when creating plugins that > > need to support both, SolrCloud and Solr Standalone. > > > > IMO an abstraction that allows distributing configuration (ML models, > > configuration snippets, external file fields...) that exceeds the typical > > ZK size limits to SolrCloud while also supporting Solr Standalone would be > > nice to have. > > > > Matt > > > > > > On 12.08.20 02:08, Noble Paul wrote: > > > > The end point is served by restlet. So, your rules are not going to be > > honored. The rules work only if it is served by a Solr request handler > > > > On Wed, Aug 12, 2020, 12:46 AM Jason Gerlowski > > wrote: > >> > >> Hey Noble, > >> > >> Can you explain what you mean when you say it's not secured? Just for > >> those of us who haven't been following the discussion so far? On the > >> surface of things users taking advantage of our RuleBasedAuth plugin > >> can secure this API like they can any other HTTP API. Or are you > >> talking about some other security aspect here? > >> > >> Jason > >> > >> On Tue, Aug 11, 2020 at 9:55 AM Noble Paul wrote: > >> > > >> > Hi all, > >> > The end-point for Managed resources is not secured. So it needs to be > >> > fixed/eliminated. > >> > > >> > I would like to know what is the level of adoption for that feature > >> > and if it is a critical feature for users. > >> > > >> > Another possibility is to offer a replacement for the feature using a > >> > different API > >> > > >> > Your feedback will help us decide on what a potential solution should be > >> > > >> > -- > >> > - > >> > Noble Paul > > > > -- > - > Noble Paul -- - Noble Paul - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Gus Heck to the PMC
Welcome Gus! On Sat, Aug 15, 2020 at 12:36 AM Christine Poerschke (BLOOMBERG/ LONDON) wrote: > > Welcome Gus! > > From: dev@lucene.apache.org At: 08/07/20 02:38:58 > To: dev@lucene.apache.org > Subject: Re: Welcome Gus Heck to the PMC > > Congrats Gus! > > -Yonik > > On Sun, Aug 2, 2020 at 7:21 PM Ishan Chattopadhyaya > wrote: >> >> I am pleased to announce that Gus Heck has accepted the PMC's invitation to >> join. >> >> Congratulations and welcome, Gus! > > -- - Noble Paul - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Welcome Munendra SN to the PMC
Welcome Munendra On Sat, Aug 15, 2020 at 12:37 AM Christine Poerschke (BLOOMBERG/ LONDON) < cpoersc...@bloomberg.net> wrote: > Welcome Munendra! > > From: dev@lucene.apache.org At: 08/07/20 02:38:27 > To: dev@lucene.apache.org > Subject: Re: Welcome Munendra SN to the PMC > > Congrats Munendra! > > -Yonik > > > On Sun, Aug 2, 2020 at 7:20 PM Ishan Chattopadhyaya < > ichattopadhy...@gmail.com> wrote: > >> I am pleased to announce that Munendra SN has accepted the PMC's >> invitation to join. >> >> Congratulations and welcome, Munendra! >> > > -- - Noble Paul
Re: Survey on ManagedResources feature
>As authentication is plugged into the SolrDispatchFilter I would assume that >you would need to be authenticated to read/write Managed Resources I'm talking about the authorization plugins On Fri, Aug 14, 2020 at 10:20 PM Matthias Krueger wrote: > > > As authentication is plugged into the SolrDispatchFilter I would assume that > you would need to be authenticated to read/write Managed Resources but no > authorization is checked (i.e. any authenticated user can read/write them), > correct? > > Anyway, I came across Managed Resources in at least two scenarios: > > The LTR plugin is using them for updating model/features. > I use MangedResource#StorageIO and its implementations as a convenient way to > abstract away the underlying config storage when creating plugins that need > to support both, SolrCloud and Solr Standalone. > > IMO an abstraction that allows distributing configuration (ML models, > configuration snippets, external file fields...) that exceeds the typical ZK > size limits to SolrCloud while also supporting Solr Standalone would be nice > to have. > > Matt > > > On 12.08.20 02:08, Noble Paul wrote: > > The end point is served by restlet. So, your rules are not going to be > honored. The rules work only if it is served by a Solr request handler > > On Wed, Aug 12, 2020, 12:46 AM Jason Gerlowski wrote: >> >> Hey Noble, >> >> Can you explain what you mean when you say it's not secured? Just for >> those of us who haven't been following the discussion so far? On the >> surface of things users taking advantage of our RuleBasedAuth plugin >> can secure this API like they can any other HTTP API. Or are you >> talking about some other security aspect here? >> >> Jason >> >> On Tue, Aug 11, 2020 at 9:55 AM Noble Paul wrote: >> > >> > Hi all, >> > The end-point for Managed resources is not secured. So it needs to be >> > fixed/eliminated. >> > >> > I would like to know what is the level of adoption for that feature >> > and if it is a critical feature for users. >> > >> > Another possibility is to offer a replacement for the feature using a >> > different API >> > >> > Your feedback will help us decide on what a potential solution should be >> > >> > -- >> > - >> > Noble Paul -- - Noble Paul - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
