Re: Deprecate Schemaless Mode?

[David Smiley]

Thanks for starting this thread Marcus!  For a historical note, the current
_default configSet being "data driven" (aka "schemaless", a worse name) is
largely because of SOLR-10272
  Maybe I should have
fought harder against it then.  I threatened to veto but I was placated by
it being easily disabled.  And it's true; you can disable it, and there are
some loud warnings on the CLI so... yeah.

I think my views most align with Gus.  The name "default" is suggestive of
good settings you ought to change if you know what you are doing.  Perhaps
there simply can be no reasonable "default" for a search platform.  There
might be "basic minimal blah blah" etc. that _is_ the default choice if you
don't specify it but naming the configSet itself as "default" gives too
much blessing to it.  I've seen too many configs with tons of stuff that
were there because it was inherited, and then it's hard to guess what's
_actually_ being used.  Alexandre Rafalov had done some great work in
figuring out how to minimize configs.  There's more to do there.

I'd be happy to see basically any change though; even a simple change from
opt-out to opt-in to "data driven" URPs.  I don't like the status quo.

BTW I've also seen people try to take "bin/solr -e cloud" to production :-(
  "Hey look, this is how a tutorial told me to run SolrCloud" (so the logic
goes).

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Tue, Aug 4, 2020 at 2:24 PM Jan Høydahl  wrote:

> Learning mode won’t work if you have 10 existing collections and want to
> create #11. We could rather have a SchemaLearningUpdateHandler so people
> could explicitly post documents to say  /schema-guess to modify the schema.
> We could even have this implicit. Then the _default config would have just
> _root_, is and a few more, and if you want guessing you first send a number
> of docs to /schema-guess endpoint and then inspect in schema browser what
> you got. That handler could support a Parma =true which would wipe
> the schema to start guessing from scratch.
>
> Jan Høydahl
>
> 4. aug. 2020 kl. 15:30 skrev Gus Heck :
>
> 
> Interesting read. Might have changed now that we have authentication
> capabilities... but let's not thread jack :)
>
> On Tue, Aug 4, 2020 at 8:28 AM Erick Erickson 
> wrote:
>
>> Having the admin UI allow uploads may not be secure. When I had a similar
>> idea a long time ago it got shot down, see the discussion at:
>> https://issues.apache.org/jira/browse/SOLR-5287.
>>
>> I _think_ this is a different issue if the configs have to be residing on
>> the system, not coming in from outside, just FYI...
>>
>> > On Aug 3, 2020, at 7:03 PM, Gus Heck  wrote:
>> >
>> >
>> >
>> > On Mon, Aug 3, 2020 at 5:03 PM Erick Erickson 
>> wrote:
>> > Gus’s point about implementing something before removing it is well
>> taken, but we can deprecate it immediately without removing it. Gus’s point
>> about dynamic fields not being found until later in the cycle is well
>> taken, but not enough to persuade me.
>> >
>> > Fair enough :)
>> >
>> > I’m not enthusiastic about multiple getting started schemas. The whole
>> motivation behind schemaless is that the user doesn’t need to know about
>> schemas to get started. By providing multiple “getting started” schemas we
>> require them to become aware of schemas again.
>> >
>> > Here's my theory (which may or may not be persuasive :) )
>> >
>> > My thinking in that suggestion is that the majority of the problem is
>> due to the fact that people new to a technology will tend to latch onto the
>> defaults that come with something as being something that should be held
>> onto until you have a good reason to change it. This is reasonable because
>> changing things you don't understand willy nilly is often a road to pain.
>> And people DO want a safe starting point and we should give it to them
>> because it makes their life easier once they get a little further down the
>> road, but this is not compatible with the easy-start schemaless mode.
>> Looking at https://lucene.apache.org/solr/guide/8_5/solr-tutorial.html I
>> see that the initial tutorial experience is fully scripted, and the user
>> won't likely notice if they are told to ignore _default or guessing-proto
>> in favor of the tech products config set... BUT when they do get to the
>> point of looking at the config name they'll see the more descriptive name.
>> So rather than seeing "_default" and thinking "Ah ha! Here's something I
>> can take as gospel and not change until I have a reason!" they'll see
>> "guessing-proto" or "dynamic-proto" and say "Hunh, I wonder what that
>> means?" which is a good question for them to ask I think.
>> >
>> > The concept of a default lays in a strong bias of not touching it
>> (IMHO) which will be wrong most of the time no matter what we give them as
>> a default. If something must be a default I'd favor a non-managed,
>> non-dynamic, 

Re: Welcome Gus Heck to the PMC

[David Smiley]

Welcome Gus!

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Sun, Aug 2, 2020 at 7:21 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I am pleased to announce that Gus Heck has accepted the PMC's invitation
> to join.
>
> Congratulations and welcome, Gus!
>

Re: Welcome Munendra SN to the PMC

[David Smiley]

Welcome Munendra!

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Sun, Aug 2, 2020 at 7:20 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I am pleased to announce that Munendra SN has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Munendra!
>

Re: Welcome Namgyu Kim to the PMC

[David Smiley]

Welcome Namgyu!

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Sun, Aug 2, 2020 at 7:20 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I am pleased to announce that Namgyu Kim has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Namgyu!
>

Re: Welcome Namgyu Kim to the PMC

[Anshum Gupta]

Congratulations and welcome, Namgyu!


On Sun, Aug 2, 2020 at 4:19 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I am pleased to announce that Namgyu Kim has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Namgyu!
>


-- 
Anshum Gupta

Re: Welcome Gus Heck to the PMC

[Anshum Gupta]

Congratulations and welcome, Gus!

On Sun, Aug 2, 2020 at 4:20 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I am pleased to announce that Gus Heck has accepted the PMC's invitation
> to join.
>
> Congratulations and welcome, Gus!
>


-- 
Anshum Gupta

Re: Welcome Munendra SN to the PMC

[Anshum Gupta]

Congratulations and welcome, Munendra!


On Sun, Aug 2, 2020 at 4:19 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I am pleased to announce that Munendra SN has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Munendra!
>


-- 
Anshum Gupta

Re: Welcome Namgyu Kim to the PMC

[Dawid Weiss]

Congratulations and welcome, Namgyu!
Dawid

On Mon, Aug 3, 2020 at 1:20 AM Ishan Chattopadhyaya
 wrote:
>
> I am pleased to announce that Namgyu Kim has accepted the PMC's invitation to 
> join.
>
> Congratulations and welcome, Namgyu!

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Gus Heck to the PMC

[Dawid Weiss]

Welcome and congratulations, Gus!
Dawid

On Mon, Aug 3, 2020 at 1:21 AM Ishan Chattopadhyaya
 wrote:
>
> I am pleased to announce that Gus Heck has accepted the PMC's invitation to 
> join.
>
> Congratulations and welcome, Gus!

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Munendra SN to the PMC

[Dawid Weiss]

Congratulations and welcome, Munendra!
Dawid

On Mon, Aug 3, 2020 at 1:20 AM Ishan Chattopadhyaya
 wrote:
>
> I am pleased to announce that Munendra SN has accepted the PMC's invitation 
> to join.
>
> Congratulations and welcome, Munendra!

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Deprecate Schemaless Mode?

[Jan Høydahl]

Learning mode won’t work if you have 10 existing collections and want to create 
#11. We could rather have a SchemaLearningUpdateHandler so people could 
explicitly post documents to say  /schema-guess to modify the schema. We could 
even have this implicit. Then the _default config would have just _root_, is 
and a few more, and if you want guessing you first send a number of docs to 
/schema-guess endpoint and then inspect in schema browser what you got. That 
handler could support a Parma =true which would wipe the schema to start 
guessing from scratch.

Jan Høydahl

> 4. aug. 2020 kl. 15:30 skrev Gus Heck :
> 
> 
> Interesting read. Might have changed now that we have authentication 
> capabilities... but let's not thread jack :)
> 
>> On Tue, Aug 4, 2020 at 8:28 AM Erick Erickson  
>> wrote:
>> Having the admin UI allow uploads may not be secure. When I had a similar 
>> idea a long time ago it got shot down, see the discussion at: 
>> https://issues.apache.org/jira/browse/SOLR-5287.
>> 
>> I _think_ this is a different issue if the configs have to be residing on 
>> the system, not coming in from outside, just FYI...
>> 
>> > On Aug 3, 2020, at 7:03 PM, Gus Heck  wrote:
>> > 
>> > 
>> > 
>> > On Mon, Aug 3, 2020 at 5:03 PM Erick Erickson  
>> > wrote:
>> > Gus’s point about implementing something before removing it is well taken, 
>> > but we can deprecate it immediately without removing it. Gus’s point about 
>> > dynamic fields not being found until later in the cycle is well taken, but 
>> > not enough to persuade me. 
>> > 
>> > Fair enough :) 
>> >  
>> > I’m not enthusiastic about multiple getting started schemas. The whole 
>> > motivation behind schemaless is that the user doesn’t need to know about 
>> > schemas to get started. By providing multiple “getting started” schemas we 
>> > require them to become aware of schemas again.
>> > 
>> > Here's my theory (which may or may not be persuasive :) ) 
>> > 
>> > My thinking in that suggestion is that the majority of the problem is due 
>> > to the fact that people new to a technology will tend to latch onto the 
>> > defaults that come with something as being something that should be held 
>> > onto until you have a good reason to change it. This is reasonable because 
>> > changing things you don't understand willy nilly is often a road to pain. 
>> > And people DO want a safe starting point and we should give it to them 
>> > because it makes their life easier once they get a little further down the 
>> > road, but this is not compatible with the easy-start schemaless mode. 
>> > Looking at https://lucene.apache.org/solr/guide/8_5/solr-tutorial.html I 
>> > see that the initial tutorial experience is fully scripted, and the user 
>> > won't likely notice if they are told to ignore _default or guessing-proto 
>> > in favor of the tech products config set... BUT when they do get to the 
>> > point of looking at the config name they'll see the more descriptive name. 
>> > So rather than seeing "_default" and thinking "Ah ha! Here's something I 
>> > can take as gospel and not change until I have a reason!" they'll see 
>> > "guessing-proto" or "dynamic-proto" and say "Hunh, I wonder what that 
>> > means?" which is a good question for them to ask I think. 
>> > 
>> > The concept of a default lays in a strong bias of not touching it (IMHO) 
>> > which will be wrong most of the time no matter what we give them as  a 
>> > default. If something must be a default I'd favor a non-managed, 
>> > non-dynamic, non-guessing minimal schema with the required fields, and an 
>> > id field, maybe a _text_ field, and a comment pointing to the section of 
>> > the ref guide where they can copy and paste in all the stuff that's 
>> > currently in our base schema as example (things like the text_ga type), IF 
>> > they want it. I get really tired of seeing mile long schemas that have a 
>> > ton of unused stuff that is retained because people didn't know if they 
>> > needed it or not... 
>> > 
>> > Note that not having some default would break back compat, on bin/solr but 
>> > changing the default is also a break of sorts. 
>> >  
>> > 
>> > All that said, maybe we could rethink the approach. My two objections are:
>> > 1> schemaless, by updating the schema based on a very small sample set is 
>> > very susceptible to failing early and often
>> > 2> Constantly updating the config in ZK and reloading the collections 
>> > seems very hard to get right.
>> >  
>> > I have for some time thought the inability to upload and download a config 
>> > (or files within a config) via the web UI was a gap. But I found it easier 
>> > to write https://plugins.gradle.org/plugin/com.needhamsoftware.solr-gradle 
>> > than add that feature to the UI :)
>> >  
>> > So I can imagine a “getting started” mode that indexed to the glob field 
>> > while creating a schema. Ideally, it would be necessary to enable it 
>> > specifically rather than have it be the default. I’d imagine 

Re: 8.6.1 Release

[Houston Putman]

Thanks for volunteering David, I missed that.

I have a patch for the backwards incompatibility that caused the 8.6.1 RC1
to fail, if anyone wants to give input.
https://github.com/apache/lucene-solr/pull/1716

- Houston

On Mon, Aug 3, 2020 at 12:38 AM David Smiley  wrote:

> FYI I volunteered to do the docker 8.6.1 release already:
> https://github.com/docker-solr/docker-solr/issues/328#issuecomment-666494362
>
> I recommend other committers interested in docker "Watch" that repo, just
> as you probably do for lucene-solr.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Sat, Aug 1, 2020 at 11:49 PM Houston Putman 
> wrote:
>
>> Its a manual process, but there are scripts available in the docker-solr
>> repo to do it.
>>
>> The releases also have to be verified by the docker folks, but i think
>> that goes pretty quickly.
>>
>> Ill make sure to get that process kicked off whenever the release gets
>> cemented.
>>
>> - Houston
>>
>> On Sat, Aug 1, 2020 at 6:13 PM Marcus Eagan 
>> wrote:
>>
>>>
>>>
>>> I agree with Ishan on the need for the Docker image check. It’s an
>>> additional burden but most people are deploying Solr as a container these
>>> days.
>>>
>>> Marcus
>>>
>>> On Sat, Aug 1, 2020 at 13:18 Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com> wrote:
>>>
 For some reason, 8.6.0 docker images are not generated and pushed.
 After the 8.6.1 release, can we ensure that the docker images are pushed?
 Any idea how does that happen, generally?
 Can we add this as a step in the release process please?

 On Fri, Jul 31, 2020 at 2:47 AM Houston Putman 
 wrote:

> The first RC has been pushed and the vote has been started.
>
> Release note drafts can be found here:
>
> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseNote861
> https://cwiki.apache.org/confluence/display/SOLR/ReleaseNote861
>
> Any changes can be made there directly.
>
> - Houston
>
> On Thu, Jul 30, 2020 at 11:51 AM Houston Putman <
> houstonput...@gmail.com> wrote:
>
>> Ok, I'm finalizing the 8.6.1 release, so please do not push any
>> commits unless there is a large bug we have not yet identified.
>>
>> I'll be sending out the RC1 later today.
>>
>> - Houston
>>
>> On Thu, Jul 30, 2020 at 11:41 AM Atri Sharma  wrote:
>>
>>> This seems like a patch release for targeting a specific blocking
>>> issue in 8.6. 8.7 would seem over arching?
>>>
>>> On Thu, 30 Jul 2020 at 20:33, Houston Putman <
>>> houstonput...@gmail.com> wrote:
>>>
 I started the process last night. Is there a reason why you want
 this to make it into 8.6.1 and not 8.7?

 - Houston

 On Thu, Jul 30, 2020 at 8:13 AM Noble Paul 
 wrote:

> Can I cherry-pick SOLR-14634: Limit the HTTP security headers to
> "/solr" end point ?
>
> if the build is not already made?
>
> On Thu, Jul 30, 2020 at 4:14 AM David Smiley 
> wrote:
> >
> > I have a PR up: https://github.com/apache/lucene-solr/pull/1706
> reviews welcome; I won't merge to a release branch without one.
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Wed, Jul 29, 2020 at 10:43 AM Houston Putman <
> houstonput...@gmail.com> wrote:
> >>
> >> Thanks Jan and David.
> >>
> >> I think that's the last issue. So after you commit it I'll
> begin the release process.
> >>
> >> - Houston
> >>
> >> On Wed, Jul 29, 2020 at 10:29 AM David Smiley <
> dsmi...@apache.org> wrote:
> >>>
> >>> https://issues.apache.org/jira/browse/LUCENE-9443
> >>> This regression on the highlighter is trivial; just revert a
> commit to a file and maybe suppress a warning.  I'll get this into 
> 8.6.1
> today.
> >>>
> >>> ~ David Smiley
> >>> Apache Lucene/Solr Search Developer
> >>> http://www.linkedin.com/in/davidwsmiley
> >>>
> >>>
> >>> On Wed, Jul 29, 2020 at 6:04 AM Jan Høydahl <
> jan@cominvent.com> wrote:
> 
>  Merged!
> 
>  28. jul. 2020 kl. 23:22 skrev Houston Putman <
> houstonput...@gmail.com>:
> 
>  +1 to the change.
> 
>  I think the last issue remaining is Jan's Zookeeper client
> port fix. After that is merged I will start with the release.
> 
>  - Houston
> 
>  On Tue, Jul 28, 2020 at 5:12 PM Gus Heck 
> wrote:
> >
> > Ishan suggested I also supply the slight clarification to

RE: Migration of Lucene Jobs and Nodes

[Uwe Schindler]

Sorry still busy…

 

Can you create the folder for Lucene and maybe export the jobs’ XML. They are 
about 20 jobs all with Regex /^(Lucene|Solr)-.*/

 

You can attach it to this mail thread or place them in a Pastebin/Gist.

 

Uwe

 

-

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Gavin McDonald  
Sent: Tuesday, July 28, 2020 10:01 PM
To: Uwe Schindler 
Cc: dev@lucene.apache.org; builds 
Subject: Re: Migration of Lucene Jobs and Nodes

 

Hi Uwe

 

On Tue, Jul 28, 2020 at 9:17 PM Uwe Schindler mailto:uschind...@apache.org> > wrote:

Hi Gavin,

I was one of the first ones who responded,

 

Yup I remember :)

 

but I had not time to take action yet. I also wanted to wait for some early 
adopters to see the possible problems.
I will come back to you on Slack. Would it be possible to get a zip file of the 
current Job configs for inspection and transformation?

 

It would yes, not sure much easier that would be as compared to just 
re-creating from new, I guess that also depends on the number 

of jobs.

 

 


Uwe

Am July 28, 2020 6:39:25 PM UTC schrieb Gavin McDonald mailto:gmcdon...@apache.org> >:

Hi,

As detailed in previous emails and the shiny yellow banner on
builds.apache.org  , your jobs need migrating to 
ci-builds.apache.org  ; and ,
in your case, your two lucene nodes.

All this before August 15th

Please let me know when you want me to move your nodes over to the new
server.

Thanks




 

-- 

  

 

Gavin McDonald

Systems Administrator

ASF Infrastructure Team

Re: Welcome Munendra SN to the PMC

[Tomoko Uchida]

Welcome, Munendra!



2020年8月4日(火) 15:23 Shalin Shekhar Mangar :

> Congratulations and welcome, Munendra!
>
> On Mon, Aug 3, 2020 at 4:50 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> I am pleased to announce that Munendra SN has accepted the PMC's
>> invitation to join.
>>
>> Congratulations and welcome, Munendra!
>>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: Welcome Gus Heck to the PMC

[Tomoko Uchida]

Welcome, Gus!



2020年8月4日(火) 22:06 Gus Heck :

> Thanks all, a pleasure and an honor to join and I look forward to
> continuing to improve lucene & solr  :)
>
> On Tue, Aug 4, 2020 at 2:23 AM Shalin Shekhar Mangar <
> shalinman...@gmail.com> wrote:
>
>> Congratulations and welcome, Gus!
>>
>> On Mon, Aug 3, 2020 at 4:51 AM Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>>
>>> I am pleased to announce that Gus Heck has accepted the PMC's
>>> invitation to join.
>>>
>>> Congratulations and welcome, Gus!
>>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)
>

Re: Welcome Namgyu Kim to the PMC

[Tomoko Uchida]

Welcome, Namgyu!



2020年8月4日(火) 15:23 Shalin Shekhar Mangar :

> Congratulations and welcome, Namgyu!
>
> On Mon, Aug 3, 2020 at 4:49 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> I am pleased to announce that Namgyu Kim has accepted the PMC's
>> invitation to join.
>>
>> Congratulations and welcome, Namgyu!
>>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: Deprecate Schemaless Mode?

[Gus Heck]

Interesting read. Might have changed now that we have authentication
capabilities... but let's not thread jack :)

On Tue, Aug 4, 2020 at 8:28 AM Erick Erickson 
wrote:

> Having the admin UI allow uploads may not be secure. When I had a similar
> idea a long time ago it got shot down, see the discussion at:
> https://issues.apache.org/jira/browse/SOLR-5287.
>
> I _think_ this is a different issue if the configs have to be residing on
> the system, not coming in from outside, just FYI...
>
> > On Aug 3, 2020, at 7:03 PM, Gus Heck  wrote:
> >
> >
> >
> > On Mon, Aug 3, 2020 at 5:03 PM Erick Erickson 
> wrote:
> > Gus’s point about implementing something before removing it is well
> taken, but we can deprecate it immediately without removing it. Gus’s point
> about dynamic fields not being found until later in the cycle is well
> taken, but not enough to persuade me.
> >
> > Fair enough :)
> >
> > I’m not enthusiastic about multiple getting started schemas. The whole
> motivation behind schemaless is that the user doesn’t need to know about
> schemas to get started. By providing multiple “getting started” schemas we
> require them to become aware of schemas again.
> >
> > Here's my theory (which may or may not be persuasive :) )
> >
> > My thinking in that suggestion is that the majority of the problem is
> due to the fact that people new to a technology will tend to latch onto the
> defaults that come with something as being something that should be held
> onto until you have a good reason to change it. This is reasonable because
> changing things you don't understand willy nilly is often a road to pain.
> And people DO want a safe starting point and we should give it to them
> because it makes their life easier once they get a little further down the
> road, but this is not compatible with the easy-start schemaless mode.
> Looking at https://lucene.apache.org/solr/guide/8_5/solr-tutorial.html I
> see that the initial tutorial experience is fully scripted, and the user
> won't likely notice if they are told to ignore _default or guessing-proto
> in favor of the tech products config set... BUT when they do get to the
> point of looking at the config name they'll see the more descriptive name.
> So rather than seeing "_default" and thinking "Ah ha! Here's something I
> can take as gospel and not change until I have a reason!" they'll see
> "guessing-proto" or "dynamic-proto" and say "Hunh, I wonder what that
> means?" which is a good question for them to ask I think.
> >
> > The concept of a default lays in a strong bias of not touching it (IMHO)
> which will be wrong most of the time no matter what we give them as  a
> default. If something must be a default I'd favor a non-managed,
> non-dynamic, non-guessing minimal schema with the required fields, and an
> id field, maybe a _text_ field, and a comment pointing to the section of
> the ref guide where they can copy and paste in all the stuff that's
> currently in our base schema as example (things like the text_ga type), IF
> they want it. I get really tired of seeing mile long schemas that have a
> ton of unused stuff that is retained because people didn't know if they
> needed it or not...
> >
> > Note that not having some default would break back compat, on bin/solr
> but changing the default is also a break of sorts.
> >
> >
> > All that said, maybe we could rethink the approach. My two objections
> are:
> > 1> schemaless, by updating the schema based on a very small sample set
> is very susceptible to failing early and often
> > 2> Constantly updating the config in ZK and reloading the collections
> seems very hard to get right.
> >
> > I have for some time thought the inability to upload and download a
> config (or files within a config) via the web UI was a gap. But I found it
> easier to write
> https://plugins.gradle.org/plugin/com.needhamsoftware.solr-gradle than
> add that feature to the UI :)
> >
> > So I can imagine a “getting started” mode that indexed to the glob field
> while creating a schema. Ideally, it would be necessary to enable it
> specifically rather than have it be the default. I’d imagine this being
> coupled with some kind of “export schema” button. So the process would be
> > > start Solr with -Dsolr.learningmode.confg=some_config_name.
> > > index a bunch of documents, perhaps prototyping the search app on the
> dynamic glob field.
> > > The admin UI should have a big, intrusive banner saying “RUNNING IN
> LEARNING MODE” with instructions on what to do next.
> > > In that mode there’d need to be a “save schema” button or something.
> What I’d like that to do would be examine the index and write a new schema
> somewhere. If ths was the mode, then you’d be able to run it any time.
> >
> > +1 for anything that makes a round-trip of working with the schema
> easier, but not really a fan of learning mode.
> >
> >
> >
>
>
> -
> To unsubscribe, e-mail: 

Re: Welcome Gus Heck to the PMC

[Gus Heck]

Thanks all, a pleasure and an honor to join and I look forward to
continuing to improve lucene & solr  :)

On Tue, Aug 4, 2020 at 2:23 AM Shalin Shekhar Mangar 
wrote:

> Congratulations and welcome, Gus!
>
> On Mon, Aug 3, 2020 at 4:51 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> I am pleased to announce that Gus Heck has accepted the PMC's invitation
>> to join.
>>
>> Congratulations and welcome, Gus!
>>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>


-- 
http://www.needhamsoftware.com (work)
http://www.the111shift.com (play)

Re: Deprecate Schemaless Mode?

[Erick Erickson]

Having the admin UI allow uploads may not be secure. When I had a similar idea 
a long time ago it got shot down, see the discussion at: 
https://issues.apache.org/jira/browse/SOLR-5287.

I _think_ this is a different issue if the configs have to be residing on the 
system, not coming in from outside, just FYI...

> On Aug 3, 2020, at 7:03 PM, Gus Heck  wrote:
> 
> 
> 
> On Mon, Aug 3, 2020 at 5:03 PM Erick Erickson  wrote:
> Gus’s point about implementing something before removing it is well taken, 
> but we can deprecate it immediately without removing it. Gus’s point about 
> dynamic fields not being found until later in the cycle is well taken, but 
> not enough to persuade me. 
> 
> Fair enough :) 
>  
> I’m not enthusiastic about multiple getting started schemas. The whole 
> motivation behind schemaless is that the user doesn’t need to know about 
> schemas to get started. By providing multiple “getting started” schemas we 
> require them to become aware of schemas again.
> 
> Here's my theory (which may or may not be persuasive :) ) 
> 
> My thinking in that suggestion is that the majority of the problem is due to 
> the fact that people new to a technology will tend to latch onto the defaults 
> that come with something as being something that should be held onto until 
> you have a good reason to change it. This is reasonable because changing 
> things you don't understand willy nilly is often a road to pain. And people 
> DO want a safe starting point and we should give it to them because it makes 
> their life easier once they get a little further down the road, but this is 
> not compatible with the easy-start schemaless mode. Looking at 
> https://lucene.apache.org/solr/guide/8_5/solr-tutorial.html I see that the 
> initial tutorial experience is fully scripted, and the user won't likely 
> notice if they are told to ignore _default or guessing-proto in favor of the 
> tech products config set... BUT when they do get to the point of looking at 
> the config name they'll see the more descriptive name. So rather than seeing 
> "_default" and thinking "Ah ha! Here's something I can take as gospel and not 
> change until I have a reason!" they'll see "guessing-proto" or 
> "dynamic-proto" and say "Hunh, I wonder what that means?" which is a good 
> question for them to ask I think. 
> 
> The concept of a default lays in a strong bias of not touching it (IMHO) 
> which will be wrong most of the time no matter what we give them as  a 
> default. If something must be a default I'd favor a non-managed, non-dynamic, 
> non-guessing minimal schema with the required fields, and an id field, maybe 
> a _text_ field, and a comment pointing to the section of the ref guide where 
> they can copy and paste in all the stuff that's currently in our base schema 
> as example (things like the text_ga type), IF they want it. I get really 
> tired of seeing mile long schemas that have a ton of unused stuff that is 
> retained because people didn't know if they needed it or not... 
> 
> Note that not having some default would break back compat, on bin/solr but 
> changing the default is also a break of sorts. 
>  
> 
> All that said, maybe we could rethink the approach. My two objections are:
> 1> schemaless, by updating the schema based on a very small sample set is 
> very susceptible to failing early and often
> 2> Constantly updating the config in ZK and reloading the collections seems 
> very hard to get right.
>  
> I have for some time thought the inability to upload and download a config 
> (or files within a config) via the web UI was a gap. But I found it easier to 
> write https://plugins.gradle.org/plugin/com.needhamsoftware.solr-gradle than 
> add that feature to the UI :)
>  
> So I can imagine a “getting started” mode that indexed to the glob field 
> while creating a schema. Ideally, it would be necessary to enable it 
> specifically rather than have it be the default. I’d imagine this being 
> coupled with some kind of “export schema” button. So the process would be
> > start Solr with -Dsolr.learningmode.confg=some_config_name.
> > index a bunch of documents, perhaps prototyping the search app on the 
> > dynamic glob field.
> > The admin UI should have a big, intrusive banner saying “RUNNING IN 
> > LEARNING MODE” with instructions on what to do next.
> > In that mode there’d need to be a “save schema” button or something. What 
> > I’d like that to do would be examine the index and write a new schema 
> > somewhere. If ths was the mode, then you’d be able to run it any time.
> 
> +1 for anything that makes a round-trip of working with the schema easier, 
> but not really a fan of learning mode.  
>  
> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Namgyu Kim to the PMC

[Shalin Shekhar Mangar]

Congratulations and welcome, Namgyu!

On Mon, Aug 3, 2020 at 4:49 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I am pleased to announce that Namgyu Kim has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Namgyu!
>


-- 
Regards,
Shalin Shekhar Mangar.

Re: Welcome Gus Heck to the PMC

[Shalin Shekhar Mangar]

Congratulations and welcome, Gus!

On Mon, Aug 3, 2020 at 4:51 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I am pleased to announce that Gus Heck has accepted the PMC's invitation
> to join.
>
> Congratulations and welcome, Gus!
>


-- 
Regards,
Shalin Shekhar Mangar.

Re: Welcome Munendra SN to the PMC

[Shalin Shekhar Mangar]

Congratulations and welcome, Munendra!

On Mon, Aug 3, 2020 at 4:50 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I am pleased to announce that Munendra SN has accepted the PMC's
> invitation to join.
>
> Congratulations and welcome, Munendra!
>


-- 
Regards,
Shalin Shekhar Mangar.

Re: Welcome Munendra SN to the PMC

[Tomás Fernández Löbbe]

Congrats and Welcome!

On Mon, Aug 3, 2020 at 9:25 AM Steve Rowe  wrote:

> Congrats and welcome, Munendra!
>
> --
> Steve
>
> > On Aug 2, 2020, at 7:19 PM, Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >
> > I am pleased to announce that Munendra SN has accepted the PMC's
> invitation to join.
> >
> > Congratulations and welcome, Munendra!
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Welcome Gus Heck to the PMC

[Tomás Fernández Löbbe]

Welcome, Gus!!

On Mon, Aug 3, 2020 at 9:27 AM Steve Rowe  wrote:

> Congrats and welcome, Gus!
>
> --
> Steve
>
> > On Aug 2, 2020, at 7:20 PM, Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >
> > I am pleased to announce that Gus Heck has accepted the PMC's invitation
> to join.
> >
> > Congratulations and welcome, Gus!
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Welcome Namgyu Kim to the PMC

[Tomás Fernández Löbbe]

Welcome!

On Mon, Aug 3, 2020 at 9:50 PM Christian Moen  wrote:

> Congrats, Namgyu!
>
> On Mon, Aug 3, 2020 at 8:19 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> I am pleased to announce that Namgyu Kim has accepted the PMC's
>> invitation to join.
>>
>> Congratulations and welcome, Namgyu!
>>
>