[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918114#comment-16918114
]
Mikhail Khludnev edited comment on SOLR-13035 at 8/29/19 12:20 PM:
---
Hello [~sarkaramr...@gmail.com] , I resolved conflicts a little, and launched
windows script. Here's what I have
At first I added -v -V and examine output. This one make sense
GC_LOG_OPTS = "-Xlog:gc*:file=\"data\logs\solr_gc.log\":time,up
This one looks odd
SOLR_DATA_HOME = data\data\data
Here's what I have in Solr Admin. Shouldn't tmp folder moved to writable home
-Djava.io.tmpdir=lucene-solr\solr\server\tmp
That's odd
-Dsolr.data.home=data\data\data
These are ok
-Dsolr.log.dir=data\logs
-Dsolr.solr.home=\lucene-solr\solr\server\solr
-Xlog:gc*:file="data\logs\solr_gc.log":
!image-2019-08-28-23-57-39-826.png!
I'm not sure if this is expected behavior. Also, notice creating pid file at
solr/bin/solr-8983.port
Also usage prompt seems vague to me
-w dir Solr will create all writable directories and files relative to this.
solr.data.home if relative, will be set as SOLR_VAR_ROOT\{this}/solr.data.home
solr.log.dir if relative, will be set as SOLR_VAR_ROOT\{this}/solr.log.dir
These references SOLR_VAR_ROOT\{this} isn't obvious.
was (Author: mkhludnev):
Hello [~sarkaramr...@gmail.com] , I resolved conflicts a little, and launched
windows script. Here's what I have
At first I added -v -V and examine output. This one make sense
GC_LOG_OPTS = "-Xlog:gc*:file=\"data\logs\solr_gc.log\":time,up
This one looks odd
SOLR_DATA_HOME = data\data\data
Here's what I have in Solr Admin. Shouldn't tmp folder moved to writable home
-Djava.io.tmpdir=lucene-solr\solr\server\tmp
That's odd
-Dsolr.data.home=data\data\data
This is ok
-Dsolr.log.dir=data\logs
-Dsolr.solr.home=\lucene-solr\solr\server\solr
-Xlog:gc*:file="data\logs\solr_gc.log":
!image-2019-08-28-23-57-39-826.png!
I'm not sure if this is expected behavior.
Also usage prompt seems vague to me
-w dir Solr will create all writable directories and files relative to this.
solr.data.home if relative, will be set as SOLR_VAR_ROOT\{this}/solr.data.home
solr.log.dir if relative, will be set as SOLR_VAR_ROOT\{this}/solr.log.dir
These references SOLR_VAR_ROOT\{this} isn't obvious.
> Utilize solr.data.home / solrDataHome in solr.xml to set all writable files
> in single directory
> ---
>
> Key: SOLR-13035
> URL: https://issues.apache.org/jira/browse/SOLR-13035
> Project: Solr
> Issue Type: Improvement
>Reporter: Amrit Sarkar
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Attachments: SOLR-13035.patch, SOLR-13035.patch, SOLR-13035.patch,
> SOLR-13035.patch, SOLR-13035.patch, image-2019-08-28-23-57-39-826.png
>
>
> {{solr.data.home}} system property or {{solrDataHome}} in _solr.xml_ is
> already available as per SOLR-6671.
> The writable content in Solr are index files, core properties, and ZK data if
> embedded zookeeper is started in SolrCloud mode. It would be great if all
> writable content can come under the same directory to have separate READ-ONLY
> and WRITE-ONLY directories.
> It can then also solve official docker Solr image issues:
> https://github.com/docker-solr/docker-solr/issues/74
> https://github.com/docker-solr/docker-solr/issues/133
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742776#comment-16742776
]
Amrit Sarkar edited comment on SOLR-13035 at 1/15/19 6:49 AM:
--
Fresh patch uploaded, the design:
1. {{SOLR_VAR_ROOT}} introduced, defaults to {{SOLR_TIP}} (installation dir)
2. {{SOLR_DATA_HOME}} will be resolved to {{}}/data if not
passed explicitly
3. {{SOLR_LOGS_DIR}} will be resolved to {{}}/logs if not passed
explicitly
4. {{SOLR_PID_DIR}} will be resolved to {{}}/, if not passed
then as before, {{}}/bin
a. {{SOLR_DATA_HOME}} now will be resolved to both instancePath and dataDir for
cores, unlike just dataDir before.
b. {{SOLR_DATA_HOME}} if not present in the server, an attempt will be made to
create the specified directory.
I have only added tests for {{SOLR_DATA_HOME}}, not sure how to test startup
script changes except manually. If everyone agrees with the above, I will add
relevant documentation and finish this up.
was (Author: sarkaramr...@gmail.com):
Fresh patch uploaded, the design:
1. {{SOLR_VAR_ROOT}} introduced, defaults to {{SOLR_TIP}} (installation dir)
2. {{SOLR_DATA_HOME}} will be resolved to {{}}/data if not
passed explicitly
3. {{SOLR_LOGS_DIR}} will be resolved to {{}}/logs if not passed
explicitly
4. {{SOLR_PID_DIR}} will be resolved to {{}}/, if not passed
then as before, {{}}/bin
a. {{SOLR_DATA_HOME}} now will be resolved to both instancePath and dataDir for
cores, unlike just dataDir before.
b. {{SOLR_DATA_HOME}} is not present in the server, an attempt will be made to
create the specified directory.
I have only added tests for {{SOLR_DATA_HOME}}, not sure how to test startup
script changes except manually. If everyone agrees with the above, I will add
relevant documentation and finish this up.
> Utilize solr.data.home / solrDataHome in solr.xml to set all writable files
> in single directory
> ---
>
> Key: SOLR-13035
> URL: https://issues.apache.org/jira/browse/SOLR-13035
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Amrit Sarkar
>Priority: Major
> Attachments: SOLR-13035.patch, SOLR-13035.patch, SOLR-13035.patch,
> SOLR-13035.patch, SOLR-13035.patch
>
>
> {{solr.data.home}} system property or {{solrDataHome}} in _solr.xml_ is
> already available as per SOLR-6671.
> The writable content in Solr are index files, core properties, and ZK data if
> embedded zookeeper is started in SolrCloud mode. It would be great if all
> writable content can come under the same directory to have separate READ-ONLY
> and WRITE-ONLY directories.
> It can then also solve official docker Solr image issues:
> https://github.com/docker-solr/docker-solr/issues/74
> https://github.com/docker-solr/docker-solr/issues/133
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742776#comment-16742776
]
Amrit Sarkar edited comment on SOLR-13035 at 1/15/19 6:06 AM:
--
Fresh patch uploaded, the design:
1. {{SOLR_VAR_ROOT}} introduced, defaults to {{SOLR_TIP}} (installation dir)
2. {{SOLR_DATA_HOME}} will be resolved to {{}}/data if not
passed explicitly
3. {{SOLR_LOGS_DIR}} will be resolved to {{}}/logs if not passed
explicitly
4. {{SOLR_PID_DIR}} will be resolved to {{}}/, if not passed
then as before, {{}}/bin
a. {{SOLR_DATA_HOME}} now will be resolved to both instancePath and dataDir for
cores, unlike just dataDir before.
b. {{SOLR_DATA_HOME}} is not present in the server, an attempt will be made to
create the specified directory.
I have only added tests for {{SOLR_DATA_HOME}}, not sure how to test startup
script changes except manually. If everyone agrees with the above, I will add
relevant documentation and finish this up.
was (Author: sarkaramr...@gmail.com):
Fresh patch uploaded, the design:
1. {{SOLR_VAR_ROOT}} introduced, defaults to {{SOLR_TIP}} (installation dir)
2. {{SOLR_DATA_HOME}} will be resolved to {{}}/data if not
passed explicitly
3. {{SOLR_LOGS_DIR}} will be resolved to {{}}/logs if not passed
explicitly
4. {{SOLR_PID_DIR}} will be resolved to {{}}/, if not passed
then as before, {{}}/bin
a. {{SOLR_DATA_HOME}} now will be resolved to both instancePath and dataDir for
cores, unlike just dataDir before.
b. {{SOLR_DATA_HOME}} is not present in the server, an attempt will be made to
create the specified directory.
I have only added tests for {{SOLR_DATA_HOME}}, not sure how to test startup
script changes except manually. If everyone agrees with all good above, I will
add relevant documentation and finish this up.
> Utilize solr.data.home / solrDataHome in solr.xml to set all writable files
> in single directory
> ---
>
> Key: SOLR-13035
> URL: https://issues.apache.org/jira/browse/SOLR-13035
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Amrit Sarkar
>Priority: Major
> Attachments: SOLR-13035.patch, SOLR-13035.patch, SOLR-13035.patch,
> SOLR-13035.patch, SOLR-13035.patch
>
>
> {{solr.data.home}} system property or {{solrDataHome}} in _solr.xml_ is
> already available as per SOLR-6671.
> The writable content in Solr are index files, core properties, and ZK data if
> embedded zookeeper is started in SolrCloud mode. It would be great if all
> writable content can come under the same directory to have separate READ-ONLY
> and WRITE-ONLY directories.
> It can then also solve official docker Solr image issues:
> https://github.com/docker-solr/docker-solr/issues/74
> https://github.com/docker-solr/docker-solr/issues/133
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742776#comment-16742776
]
Amrit Sarkar edited comment on SOLR-13035 at 1/15/19 6:05 AM:
--
Fresh patch uploaded, the design:
1. {{SOLR_VAR_ROOT}} introduced, defaults to {{SOLR_TIP}} (installation dir)
2. {{SOLR_DATA_HOME}} will be resolved to {{}}/data if not
passed explicitly
3. {{SOLR_LOGS_DIR}} will be resolved to {{}}/logs if not passed
explicitly
4. {{SOLR_PID_DIR}} will be resolved to {{}}/, if not passed
then as before, {{}}/bin
a. {{SOLR_DATA_HOME}} now will be resolved to both instancePath and dataDir for
cores, unlike just dataDir before.
b. {{SOLR_DATA_HOME}} is not present in the server, an attempt will be made to
create the specified directory.
I have only added tests for {{SOLR_DATA_HOME}}, not sure how to test startup
script changes except manually. If everyone agrees with all good above, I will
add relevant documentation and finish this up.
was (Author: sarkaramr...@gmail.com):
Fresh patch uploaded, the design:
1. {{SOLR_VAR_ROOT}} introduced, defaults to {{SOLR_TIP}} (installation dir)
2. {{SOLR_DATA_HOME}} will be resolved to {{}}/data if not
passed explicitly
3. {{SOLR_LOGS_DIR}} will be resolved to {{}}/logs if not passed
explicitly
4. {{SOLR_PID_DIR}} will be resolved to {{}}/, if not passed
then as before, {{}}/bin
a. SOLR_DATA_HOME now will be resolved to both instancePath and dataDir for
cores, unlike just dataDir before.
b. SOLR_DATA_HOME is not present in the server, an attempt will be made to
create the specified directory.
I have only added tests for SOLR_DATA_HOME, not sure how to test startup script
changes except manually. If everyone agrees with all good above, I will add
relevant documentation and finish this up.
> Utilize solr.data.home / solrDataHome in solr.xml to set all writable files
> in single directory
> ---
>
> Key: SOLR-13035
> URL: https://issues.apache.org/jira/browse/SOLR-13035
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Amrit Sarkar
>Priority: Major
> Attachments: SOLR-13035.patch, SOLR-13035.patch, SOLR-13035.patch,
> SOLR-13035.patch
>
>
> {{solr.data.home}} system property or {{solrDataHome}} in _solr.xml_ is
> already available as per SOLR-6671.
> The writable content in Solr are index files, core properties, and ZK data if
> embedded zookeeper is started in SolrCloud mode. It would be great if all
> writable content can come under the same directory to have separate READ-ONLY
> and WRITE-ONLY directories.
> It can then also solve official docker Solr image issues:
> https://github.com/docker-solr/docker-solr/issues/74
> https://github.com/docker-solr/docker-solr/issues/133
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733214#comment-16733214
]
Amrit Sarkar edited comment on SOLR-13035 at 1/3/19 4:48 PM:
-
Thank you Jan and Shalin for the fruitful discussion above. So we have
consensus on SOLR_VAR_ROOT with it default pointing to SOLR_TIP.
Working on it.
was (Author: sarkaramr...@gmail.com):
Thank you Jan and Shalin for the fruitful discussion above and we do have
consensus on SOLR_VAR_ROOT with it default pointing to SOLR_TIP.
Working on it.
> Utilize solr.data.home / solrDataHome in solr.xml to set all writable files
> in single directory
> ---
>
> Key: SOLR-13035
> URL: https://issues.apache.org/jira/browse/SOLR-13035
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Amrit Sarkar
>Priority: Major
> Attachments: SOLR-13035.patch, SOLR-13035.patch
>
>
> {{solr.data.home}} system property or {{solrDataHome}} in _solr.xml_ is
> already available as per SOLR-6671.
> The writable content in Solr are index files, core properties, and ZK data if
> embedded zookeeper is started in SolrCloud mode. It would be great if all
> writable content can come under the same directory to have separate READ-ONLY
> and WRITE-ONLY directories.
> It can then also solve official docker Solr image issues:
> https://github.com/docker-solr/docker-solr/issues/74
> https://github.com/docker-solr/docker-solr/issues/133
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711533#comment-16711533
]
Amrit Sarkar edited comment on SOLR-13035 at 12/14/18 4:10 PM:
---
Thanks [~janhoy] for the feedback;
bq. Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
bq. In general, I'd prefer of much of the logic regarding folder resolution etc
was done in the common SolrCLI.java so as little logic as possible needs to be
duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
bq. While I believe solr.xml could be created if not exist, I'm not so sure we
should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
bq. I agree on the goal of making Solr more container friendly and clearly
separate what paths can be made R/O and what paths typically need a separate
volume mounted. Can you perhaps create a diagram that clearly shows the various
directory paths we have today with their defaults and how they will be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> server --> solr --> [data_dir] Index files
SOLR_TIP -> server --> solr --> [instance_dir] Core properties
SOLR_TIP -> server --> solr --> [zoo_data] Embedded ZK data
SOLR_TIP -> server --> [logs]
READ specific contents in the same directory [server/solr]
SOLR_TIP -> server --> solr --> solr.xml [changes requires NODE
restart]
SOLR_TIP -> server --> solr --> [configsets] Default config sets
{code}
For the below-stated startup command with the current patch; R/W directories
will look like following;
{code}
cd $SOLR_TIP
bin/solr start -c -p 8983 -t data -l logs
{code}
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> data -> [data_dir] Index files
[instance_dir] Core properties
[zoo_data] Embedded ZK data
SOLR_TIP -> logs
All other respective dirs would be READ specific, and changes
in them requires NODE restart.
{code}
Adding support for adding solr.xml if not exist will be helpful. That way, as
mentioned above pointing SOLR_HOME to an empty directory would be enough to
achieve defined R/W directories. Though intention of writing the patch was to
seperate directories which are created / modified after node restart. Default
{{SOLR_HOME}} can still point to [SOLR_TIP]/server/solr and picks up default
solr.xml.
Looking forward to feedback.
was (Author: sarkaramr...@gmail.com):
host: ftp.lucidworks.com
port: 22
@Override
public Collection getApis() {
return singletonList(new ApiBag.ReqHandlerToApi(this,
getSpec("node.Health")));
}
@Override
public Boolean registerV1() {
return Boolean.FALSE;
}
@Override
public Boolean registerV2() {
return Boolean.TRUE;
}
{
"description": "Provides information about system health for a node.",
"methods": ["GET"],
"url": {
"paths": [
"/node/health"
]
}
}
Thanks [~janhoy] for the feedback;
bq. Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
bq. In general, I'd prefer of much of the logic regarding folder resolution etc
was done in the common SolrCLI.java so as little logic as possible needs to be
duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
bq. While I believe solr.xml could be created if not exist, I'm not so sure we
should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
bq. I agree on the goal of making Solr more container friendly and clearly
separate
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711533#comment-16711533
]
Amrit Sarkar edited comment on SOLR-13035 at 12/14/18 4:05 PM:
---
host: ftp.lucidworks.com
port: 22
@Override
public Collection getApis() {
return singletonList(new ApiBag.ReqHandlerToApi(this,
getSpec("node.Health")));
}
@Override
public Boolean registerV1() {
return Boolean.FALSE;
}
@Override
public Boolean registerV2() {
return Boolean.TRUE;
}
{
"description": "Provides information about system health for a node.",
"methods": ["GET"],
"url": {
"paths": [
"/node/health"
]
}
}
Thanks [~janhoy] for the feedback;
bq. Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
bq. In general, I'd prefer of much of the logic regarding folder resolution etc
was done in the common SolrCLI.java so as little logic as possible needs to be
duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
bq. While I believe solr.xml could be created if not exist, I'm not so sure we
should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
bq. I agree on the goal of making Solr more container friendly and clearly
separate what paths can be made R/O and what paths typically need a separate
volume mounted. Can you perhaps create a diagram that clearly shows the various
directory paths we have today with their defaults and how they will be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> server --> solr --> [data_dir] Index files
SOLR_TIP -> server --> solr --> [instance_dir] Core properties
SOLR_TIP -> server --> solr --> [zoo_data] Embedded ZK data
SOLR_TIP -> server --> [logs]
READ specific contents in the same directory [server/solr]
SOLR_TIP -> server --> solr --> solr.xml [changes requires NODE
restart]
SOLR_TIP -> server --> solr --> [configsets] Default config sets
{code}
For the below-stated startup command with the current patch; R/W directories
will look like following;
{code}
cd $SOLR_TIP
bin/solr start -c -p 8983 -t data -l logs
{code}
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> data -> [data_dir] Index files
[instance_dir] Core properties
[zoo_data] Embedded ZK data
SOLR_TIP -> logs
All other respective dirs would be READ specific, and changes
in them requires NODE restart.
{code}
Adding support for adding solr.xml if not exist will be helpful. That way, as
mentioned above pointing SOLR_HOME to an empty directory would be enough to
achieve defined R/W directories. Though intention of writing the patch was to
seperate directories which are created / modified after node restart. Default
{{SOLR_HOME}} can still point to [SOLR_TIP]/server/solr and picks up default
solr.xml.
Looking forward to feedback.
was (Author: sarkaramr...@gmail.com):
Thanks [~janhoy] for the feedback;
bq. Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
bq. In general, I'd prefer of much of the logic regarding folder resolution etc
was done in the common SolrCLI.java so as little logic as possible needs to be
duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
bq. While I believe solr.xml could be created if not exist, I'm not so sure we
should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
bq. I agree on the goal of making Solr more container friendly and clearly
separate
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711533#comment-16711533
]
Amrit Sarkar edited comment on SOLR-13035 at 12/14/18 4:03 PM:
---
Thanks [~janhoy] for the feedback;
bq. Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
bq. In general, I'd prefer of much of the logic regarding folder resolution etc
was done in the common SolrCLI.java so as little logic as possible needs to be
duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
bq. While I believe solr.xml could be created if not exist, I'm not so sure we
should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
bq. I agree on the goal of making Solr more container friendly and clearly
separate what paths can be made R/O and what paths typically need a separate
volume mounted. Can you perhaps create a diagram that clearly shows the various
directory paths we have today with their defaults and how they will be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> server --> solr --> [data_dir] Index files
SOLR_TIP -> server --> solr --> [instance_dir] Core properties
SOLR_TIP -> server --> solr --> [zoo_data] Embedded ZK data
SOLR_TIP -> server --> [logs]
READ specific contents in the same directory [server/solr]
SOLR_TIP -> server --> solr --> solr.xml [changes requires NODE
restart]
SOLR_TIP -> server --> solr --> [configsets] Default config sets
{code}
For the below-stated startup command with the current patch; R/W directories
will look like following;
{code}
cd $SOLR_TIP
bin/solr start -c -p 8983 -t data -l logs
{code}
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> data -> [data_dir] Index files
[instance_dir] Core properties
[zoo_data] Embedded ZK data
SOLR_TIP -> logs
All other respective dirs would be READ specific, and changes
in them requires NODE restart.
{code}
Adding support for adding solr.xml if not exist will be helpful. That way, as
mentioned above pointing SOLR_HOME to an empty directory would be enough to
achieve defined R/W directories. Though intention of writing the patch was to
seperate directories which are created / modified after node restart. Default
{{SOLR_HOME}} can still point to [SOLR_TIP]/server/solr and picks up default
solr.xml.
Looking forward to feedback.
was (Author: sarkaramr...@gmail.com):
Thanks [~janhoy] for the feedback;
bq. Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
bq. In general, I'd prefer of much of the logic regarding folder resolution etc
was done in the common SolrCLI.java so as little logic as possible needs to be
duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
bq. While I believe solr.xml could be created if not exist, I'm not so sure we
should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
bq. I agree on the goal of making Solr more container friendly and clearly
separate what paths can be made R/O and what paths typically need a separate
volume mounted. Can you perhaps create a diagram that clearly shows the various
directory paths we have today with their defaults and how they will be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
{code}
WRITE
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711533#comment-16711533
]
Amrit Sarkar edited comment on SOLR-13035 at 12/14/18 4:03 PM:
---
Thanks [~janhoy] for the feedback;
bq. Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
bq. In general, I'd prefer of much of the logic regarding folder resolution etc
was done in the common SolrCLI.java so as little logic as possible needs to be
duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
bq. While I believe solr.xml could be created if not exist, I'm not so sure we
should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
bq. I agree on the goal of making Solr more container friendly and clearly
separate what paths can be made R/O and what paths typically need a separate
volume mounted. Can you perhaps create a diagram that clearly shows the various
directory paths we have today with their defaults and how they will be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> server --> solr --> [data_dir] Index files
SOLR_TIP -> server --> solr --> [instance_dir] Core properties
SOLR_TIP -> server --> solr --> [zoo_data] Embedded ZK data
SOLR_TIP -> server --> [logs]
READ specific contents in the same directory [server/solr]
SOLR_TIP -> server --> solr --> solr.xml [changes requires NODE
restart]
SOLR_TIP -> server --> solr --> [configsets] Default config sets
{code}
For the below-stated startup command with the current patch; R/W directories
will look like following;
{code}
cd $SOLR_TIP
bin/solr start -c -p 8983 -t data -l logs
{code}
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> data -> [data_dir] Index files
[instance_dir] Core properties
[zoo_data] Embedded ZK data
SOLR_TIP -> logs
All other respective dirs would be READ specific, and changes
in them requires NODE restart.
{code}
Adding support for adding solr.xml if not exist will be helpful. That way, as
mentioned above pointing SOLR_HOME to an empty directory would be enough to
achieve defined R/W directories. Though intention of writing the patch was to
seperate directories which are created / modified after node restart. Default
{{SOLR_HOME}} can still point to [SOLR_TIP]/server/solr and picks up default
solr.xml.
Looking forward to feedback.
was (Author: sarkaramr...@gmail.com):
Thanks [~janhoy] for the feedback;
bq. Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
bq. In general, I'd prefer of much of the logic regarding folder resolution etc
was done in the common SolrCLI.java so as little logic as possible needs to be
duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
bq. While I believe solr.xml could be created if not exist, I'm not so sure we
should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
bq. I agree on the goal of making Solr more container friendly and clearly
separate what paths can be made R/O and what paths typically need a separate
volume mounted. Can you perhaps create a diagram that clearly shows the various
directory paths we have today with their defaults and how they will be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
{code}
WRITE specific dirs from
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711533#comment-16711533
]
Amrit Sarkar edited comment on SOLR-13035 at 12/6/18 2:49 PM:
--
Thanks [~janhoy] for the feedback;
bq. Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
bq. In general, I'd prefer of much of the logic regarding folder resolution etc
was done in the common SolrCLI.java so as little logic as possible needs to be
duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
bq. While I believe solr.xml could be created if not exist, I'm not so sure we
should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
bq. I agree on the goal of making Solr more container friendly and clearly
separate what paths can be made R/O and what paths typically need a separate
volume mounted. Can you perhaps create a diagram that clearly shows the various
directory paths we have today with their defaults and how they will be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> server --> solr --> [data_dir] Index files
SOLR_TIP -> server --> solr --> [instance_dir] Core properties
SOLR_TIP -> server --> solr --> [zoo_data] Embedded ZK data
SOLR_TIP -> server --> [logs]
READ specific contents in the same directory [server/solr]
SOLR_TIP -> server --> solr --> solr.xml [changes requires NODE
restart]
SOLR_TIP -> server --> solr --> [configsets] Default config sets
{code}
For the below-stated startup command with the current patch; R/W directories
will look like following;
{code}
cd $SOLR_TIP
bin/solr start -c -p 8983 -t data -l logs
{code}
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> [data_dir] Index files
SOLR_TIP -> [instance_dir] Core properties
SOLR_TIP -> [zoo_data] Embedded ZK data
SOLR_TIP -> [logs]
All other respective dirs would be READ specific; and changes
in them requires NODE restart.
{code}
Adding support for adding solr.xml if not exist will be helpful. That way, as
mentioned above pointing SOLR_HOME to an empty directory would be enough to
achieve defined R/W directories. Though intention of writing the patch was to
seperate directories which are created / modified after node restart. Default
{{SOLR_HOME}} can still point to [SOLR_TIP]/server/solr and picks up default
solr.xml.
Looking forward to feedback.
was (Author: sarkaramr...@gmail.com):
Thanks [~janhoy] for the feedback;
> Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
> to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
>In general, I'd prefer of much of the logic regarding folder resolution etc
>was done in the common SolrCLI.java so as little logic as possible needs to be
>duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
>While I believe solr.xml could be created if not exist, I'm not so sure we
>should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
> I agree on the goal of making Solr more container friendly and clearly
> separate what paths can be made R/O and what paths typically need a separate
> volume mounted. Can you perhaps create a diagram that clearly shows the
> various directory paths we have today with their defaults and how they will
> be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> server --> solr -->
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711533#comment-16711533
]
Amrit Sarkar edited comment on SOLR-13035 at 12/6/18 2:48 PM:
--
Thanks [~janhoy] for the feedback;
> Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
> to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
>In general, I'd prefer of much of the logic regarding folder resolution etc
>was done in the common SolrCLI.java so as little logic as possible needs to be
>duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
>While I believe solr.xml could be created if not exist, I'm not so sure we
>should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
> I agree on the goal of making Solr more container friendly and clearly
> separate what paths can be made R/O and what paths typically need a separate
> volume mounted. Can you perhaps create a diagram that clearly shows the
> various directory paths we have today with their defaults and how they will
> be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> server --> solr --> [data_dir] Index files
SOLR_TIP -> server --> solr --> [instance_dir] Core properties
SOLR_TIP -> server --> solr --> [zoo_data] Embedded ZK data
SOLR_TIP -> server --> [logs]
READ specific contents in the same directory [server/solr]
SOLR_TIP -> server --> solr --> solr.xml [changes requires NODE
restart]
SOLR_TIP -> server --> solr --> [configsets] Default config sets
For the below-stated startup command with the current patch; R/W directories
will look like following;
{code}
cd $SOLR_TIP
bin/solr start -c -p 8983 -t data -l logs
{code}
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> [data_dir] Index files
SOLR_TIP -> [instance_dir] Core properties
SOLR_TIP -> [zoo_data] Embedded ZK data
SOLR_TIP -> [logs]
All other respective dirs would be READ specific; and changes
in them requires NODE restart.
Adding support for adding solr.xml if not exist will be helpful. That way, as
mentioned above pointing SOLR_HOME to an empty directory would be enough to
achieve defined R/W directories. Though intention of writing the patch was to
seperate directories which are created / modified after node restart. Default
{{SOLR_HOME}} can still point to [SOLR_TIP]/server/solr and picks up default
solr.xml.
Looking forward to feedback.
was (Author: sarkaramr...@gmail.com):
Thanks [~janhoy] for the feedback;
> Looks scary to do SOLR_LOGS_DIR="$(pwd)/$SOLR_LOGS_DIR". It would be better
> to require the user to configure absolute path here.
Sure. I felt supporting relative path makes it a bit easy for deploying
multiple Solr nodes on the machine/server. If {{$(pwd)/$SOLR_LOGS_DIR}} is not
the right way to do; probably {{SOLR_TIP/$SOLR_LOGS_DIR}}. The absolute path is
supported as the original design.
>In general, I'd prefer of much of the logic regarding folder resolution etc
>was done in the common SolrCLI.java so as little logic as possible needs to be
>duplicated in bin/solr and bin/solr.cmd
Yeah makes sense. I followed the same resolution criteria implemented for
SOLR_HOME, will move that logic to SolrCLI.java.
>While I believe solr.xml could be created if not exist, I'm not so sure we
>should go create SOLR_DATA_HOME if it does not exist?
Sure. Creating SOLR_DATA_HOME if not exist was again minor improvement for
end-user, just like $SOLR_LOGS_DIR, though not necessary.
> I agree on the goal of making Solr more container friendly and clearly
> separate what paths can be made R/O and what paths typically need a separate
> volume mounted. Can you perhaps create a diagram that clearly shows the
> various directory paths we have today with their defaults and how they will
> be changed?
We intend not to change the default respective directory paths but instead
makes it easy to run in a container environment.
SOLR_TIP -> solr installation directory
Default state:
WRITE specific dirs from Solr/Zk node.
SOLR_TIP -> server --> solr --> [data_dir] Index files
[jira] [Comment Edited] (SOLR-13035) Utilize solr.data.home / solrDataHome in solr.xml to set all writable files in single directory
[
https://issues.apache.org/jira/browse/SOLR-13035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16710768#comment-16710768
]
Amrit Sarkar edited comment on SOLR-13035 at 12/6/18 2:03 PM:
--
Thanks [~elyograg] and [~janhoy],
I see in SOLR-6671 the motivation behind {{solr.data.home}} and understand why
we don't want wish to have core properties under the same directory. I strongly
agree with [~shalinmangar] on making Solr easy to use with dockers/containers
and don't have to configure out of ordinary and have workarounds.
With current patch, entire {{server/solr}} can be READ-ONLY, and directories
specified at Solr startup:
{code}
cd $SOLR_TIP
bin/solr start -c -p 8983 -t data -l logs
{code}
would be WRITE-ONLY.
I can work on a design we have consensus on. Looking forward to feedback, I can
start right on, and as mentioned we can get in version 8.0.
was (Author: sarkaramr...@gmail.com):
Thanks [~elyograg] and [~janhoy],
I see in SOLR-6671 the motivation behind {{solr.data.home}} and understand why
we don't want wish to have core properties under the same directory. I strongly
agree with [~shalinmangar] on making Solr easy to use with dockers/containers
and don't have to configure out of ordinary and have workarounds.
With current patch, entire {{server/solr}} can be READ-ONLY, and directories
specified at Solr startup:
{code}
bin/solr start -c -p 8983 -t data -l logs
{code}
would be WRITE-ONLY.
I can work on a design we have consensus on. Looking forward to feedback, I can
start right on, and as mentioned we can get in version 8.0.
> Utilize solr.data.home / solrDataHome in solr.xml to set all writable files
> in single directory
> ---
>
> Key: SOLR-13035
> URL: https://issues.apache.org/jira/browse/SOLR-13035
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Amrit Sarkar
>Priority: Major
> Attachments: SOLR-13035.patch, SOLR-13035.patch
>
>
> {{solr.data.home}} system property or {{solrDataHome}} in _solr.xml_ is
> already available as per SOLR-6671.
> The writable content in Solr are index files, core properties, and ZK data if
> embedded zookeeper is started in SolrCloud mode. It would be great if all
> writable content can come under the same directory to have separate READ-ONLY
> and WRITE-ONLY directories.
> It can then also solve official docker Solr image issues:
> https://github.com/docker-solr/docker-solr/issues/74
> https://github.com/docker-solr/docker-solr/issues/133
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
