This is an automated email from the ASF dual-hosted git repository.
houston pushed a commit to branch branch_9x
in repository https://gitbox.apache.org/repos/asf/solr.git
The following commit(s) were added to refs/heads/branch_9x by this push:
new 19271fdf318 SOLR-16742: Make a slim and full Solr release (#1556)
19271fdf318 is described below
commit 19271fdf3182f4e9a9af29d4a274ec3baa3350dd
Author: Houston Putman <[email protected]>
AuthorDate: Wed Apr 12 11:55:21 2023 -0400
SOLR-16742: Make a slim and full Solr release (#1556)
- Binary generation
- Docker image & official Dockerfile generation/testing
- Documentation
(cherry picked from commit a34eba3a76061900860bc9f9e78c8bc788ef15d6)
---
dev-docs/running-in-docker.adoc | 5 +
dev-docs/solr-source-code.adoc | 3 +
dev-tools/scripts/releaseWizard.yaml | 24 +-
dev-tools/scripts/smokeTestRelease.py | 13 +-
solr/CHANGES.txt | 2 +
solr/distribution/build.gradle | 19 +-
solr/docker/README.md | 2 +
solr/docker/build.gradle | 321 +++++++++++++--------
solr/docker/gradle-help.txt | 28 +-
solr/docker/templates/Dockerfile.body.template | 4 +-
.../templates/Dockerfile.local.header.template | 6 +-
.../templates/Dockerfile.official.header.template | 28 +-
solr/packaging/README.txt | 2 +
solr/packaging/build.gradle | 112 +++++--
.../configuration-guide/pages/solr-modules.adoc | 3 +
.../deployment-guide/pages/installing-solr.adoc | 14 +-
.../monitoring-with-prometheus-and-grafana.adoc | 5 +-
.../deployment-guide/pages/solr-in-docker.adoc | 12 +
.../getting-started/pages/solr-tutorial.adoc | 2 +-
.../pages/major-changes-in-solr-9.adoc | 5 +
20 files changed, 419 insertions(+), 191 deletions(-)
diff --git a/dev-docs/running-in-docker.adoc b/dev-docs/running-in-docker.adoc
index e2471bc10c8..0e455ce4ceb 100644
--- a/dev-docs/running-in-docker.adoc
+++ b/dev-docs/running-in-docker.adoc
@@ -20,6 +20,11 @@ The task will output the image name to use at the end of the
build.
`./gradlew docker`
+Or if you want to test with the "slim" Solr distribution:
+
+`./gradlew docker -Psolr.docker.dist=slim`
+
+
For more info on building an image, run:
`./gradlew helpDocker`
diff --git a/dev-docs/solr-source-code.adoc b/dev-docs/solr-source-code.adoc
index 9daf86cd2d4..66640c0100a 100644
--- a/dev-docs/solr-source-code.adoc
+++ b/dev-docs/solr-source-code.adoc
@@ -18,6 +18,9 @@ command to see the available options for building, testing,
and packaging Solr.
`./gradlew dev` will create a Solr executable suitable for development.
cd to `./solr/packaging/build/dev` and run the `bin/solr` script
to start Solr.
+It will also create a "slim" Solr executable based on the "slim" Solr
distribution.
+You can find this environment at `./solr/packaging/build/dev-slim`.
+Use either `./gradlew devSlim` or `./gradlew devFull` to create just one/
NOTE: `gradlew` is the "Gradle Wrapper" and will automatically download and
start using the correct version of Gradle for Solr.
diff --git a/dev-tools/scripts/releaseWizard.yaml
b/dev-tools/scripts/releaseWizard.yaml
index 03fbf075c29..c5c70e5c4a8 100644
--- a/dev-tools/scripts/releaseWizard.yaml
+++ b/dev-tools/scripts/releaseWizard.yaml
@@ -861,13 +861,16 @@ groups:
python3 -u dev-tools/scripts/smokeTestRelease.py \
{{ release_candidate_location }}
- You can build a release-candidate of the official docker image using the
following command:
+ You can build a release-candidate of the official docker images (full &
slim) using the following command:
DIST_BASE={{ dist_url_base }} && \
RC_FOLDER={{ release_candidate_rc_folder }} && \
- docker build $DIST_BASE/$RC_FOLDER/solr/docker/Dockerfile.official \
+ docker build
$DIST_BASE/$RC_FOLDER/solr/docker/Dockerfile.official-full \
--build-arg SOLR_DOWNLOAD_URL=$DIST_BASE/$RC_FOLDER/solr/solr-{{
release_version }}.tgz \
- -t solr-rc:{{ release_version }}-{{ rc_number }}
+ -t solr-rc:{{ release_version }}-{{ rc_number }}} && \
+ docker build
$DIST_BASE/$RC_FOLDER/solr/docker/Dockerfile.official-slim \
+ --build-arg SOLR_DOWNLOAD_URL=$DIST_BASE/$RC_FOLDER/solr/solr-{{
release_version }}-slim.tgz \
+ -t solr-rc:{{ release_version }}-{{ rc_number }}-slim
The vote will be open for at least 72 hours i.e. until {{ vote_close }}.
@@ -1055,19 +1058,24 @@ groups:
comment: Checkout the most recent main version
logfile: solr_docker_git_checkout.log
- !Command
- cmd: "mkdir -p {{ docker_version }} && cp {{ [dist_file_path,
dist_folder, 'solr', 'docker', 'Dockerfile.official'] | path_join }} {{
docker_version }}/Dockerfile"
+ cmd: "mkdir -p {{ docker_version }} && cp {{ [dist_file_path,
dist_folder, 'solr', 'docker', 'Dockerfile.official-full'] | path_join }} {{
docker_version }}/Dockerfile"
cwd: solr-docker
- comment: Add the new Dockerfile to the repo
- logfile: solr_docker_add_dockerfile.log
+ comment: Add the new full distribution Dockerfile to the repo
+ logfile: solr_docker_add_dockerfile_full.log
- !Command
- cmd: 'git add {{ docker_version }}/Dockerfile && git commit -m
"Apache Solr release {{ release_version }}"'
+ cmd: "mkdir -p {{ docker_version }} && cp {{ [dist_file_path,
dist_folder, 'solr', 'docker', 'Dockerfile.official-slim'] | path_join }} {{
docker_version }}-slim/Dockerfile"
+ cwd: solr-docker
+ comment: Add the new slim distribution Dockerfile to the repo
+ logfile: solr_docker_add_dockerfile_slim.log
+ - !Command
+ cmd: 'git add {{ docker_version }}/Dockerfile {{ docker_version
}}-slim/Dockerfile && git commit -m "Apache Solr release {{ release_version }}"'
cwd: solr-docker
comment: Commit the new Dockerfile
logfile: solr_docker_commit.log
- !Command
cmd: git show HEAD
cwd: solr-docker
- comment: Check that the commit look correct. Only one Dockerfile
should be added or updated.
+ comment: Check that the commit looks correct. Only two Dockerfiles
(full and slim) should be added or updated.
logfile: solr_docker_check_commit.log
tee: true
- !Command
diff --git a/dev-tools/scripts/smokeTestRelease.py
b/dev-tools/scripts/smokeTestRelease.py
index 8b1ca48e865..ccd38ba0b51 100755
--- a/dev-tools/scripts/smokeTestRelease.py
+++ b/dev-tools/scripts/smokeTestRelease.py
@@ -266,9 +266,12 @@ def checkSigs(urlString, version, tmpDir, isSigned,
keysFile):
raise RuntimeError('solr: artifact %s has wrong sigs: expected %s but
got %s' % (artifact, expectedSigs, sigs))
expected = ['solr-%s-src.tgz' % version,
- 'solr-%s.tgz' % version]
+ 'solr-%s.tgz' % version,
+ 'solr-%s-slim.tgz' % version]
actual = [x[0] for x in artifacts]
+ expected.sort()
+ actual.sort()
if expected != actual:
raise RuntimeError('solr: wrong artifacts: expected %s but got %s' %
(expected, actual))
@@ -535,12 +538,14 @@ def unpackAndVerify(java, tmpDir, artifact, gitRevision,
version, testArgs):
os.chdir(destDir)
print(' unpack %s...' % artifact)
unpackLogFile = '%s/solr-unpack-%s.log' % (tmpDir, artifact)
+ expected = artifact
if artifact.endswith('.tar.gz') or artifact.endswith('.tgz'):
+ expected = artifact[:artifact.find('.t')]
+ expected = expected.replace("-src", "") # Remove when the src tgz
subdirectory has "-src" appended
run('tar xzf %s/%s' % (tmpDir, artifact), unpackLogFile)
# make sure it unpacks to proper subdir
l = os.listdir(destDir)
- expected = 'solr-%s' % version
if l != [expected]:
raise RuntimeError('unpack produced entries %s; expected only %s' % (l,
expected))
@@ -569,6 +574,7 @@ def verifyUnpacked(java, artifact, unpackPath, gitRevision,
version, testArgs):
os.chdir(unpackPath)
isSrc = artifact.find('-src') != -1
+ isSlim = artifact.find('-slim') != -1
# Check text files in release
print(" %s" % artifact)
@@ -607,6 +613,8 @@ def verifyUnpacked(java, artifact, unpackPath, gitRevision,
version, testArgs):
is_in_list(in_solr_folder, expected_src_solr_files)
if len(in_solr_folder) > 0:
raise RuntimeError('solr: unexpected files/dirs in artifact %s solr/
folder: %s' % (artifact, in_solr_folder))
+ elif isSlim:
+ is_in_list(in_root_folder, ['bin', 'docker', 'docs', 'example',
'licenses', 'server', 'lib'])
else:
is_in_list(in_root_folder, ['bin', 'modules', 'docker',
'prometheus-exporter', 'docs', 'example', 'licenses', 'server', 'lib'])
@@ -1147,6 +1155,7 @@ def smokeTest(java, baseURL, gitRevision, version,
tmpDir, isSigned, local_keys,
checkSigs(solrPath, version, tmpDir, isSigned, keysFile)
if not downloadOnly:
unpackAndVerify(java, tmpDir, 'solr-%s.tgz' % version, gitRevision,
version, testArgs)
+ unpackAndVerify(java, tmpDir, 'solr-%s-slim.tgz' % version, gitRevision,
version, testArgs)
unpackAndVerify(java, tmpDir, 'solr-%s-src.tgz' % version, gitRevision,
version, testArgs)
print()
print('Test Maven artifacts...')
diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt
index 17ba721a36d..ff95b37a87d 100644
--- a/solr/CHANGES.txt
+++ b/solr/CHANGES.txt
@@ -13,6 +13,8 @@ New Features
them into SolrCloud shards. The API is available at `POST
/api/collections/collName/shards/shardName/install`
(v2), or at `GET /solr/admin/collections?action=INSTALLSHARD` (v1). (Jason
Gerlowski)
+* SOLR-16742: Produce a slim Solr binary and docker image, without the
Prometheus Exporter and Solr Modules. (Houston Putman)
+
Improvements
---------------------
diff --git a/solr/distribution/build.gradle b/solr/distribution/build.gradle
index 4f89ed5f3ad..f30e5603b08 100644
--- a/solr/distribution/build.gradle
+++ b/solr/distribution/build.gradle
@@ -62,7 +62,8 @@ dependencies {
docker project(path: ':solr:docker', configuration:
project.ext.withSignedArtifacts ? 'packagingOfficial' : 'packagingLocal')
}
-def distTarTask = rootProject.getTasksByName("distTar", true)[0]
+def fullDistTarTask = rootProject.getTasksByName("fullDistTar", true)[0]
+def slimDistTarTask = rootProject.getTasksByName("slimDistTar", true)[0]
// Compute checksums for release archives.
task computeChecksums(type: Checksum) {
@@ -71,7 +72,8 @@ task computeChecksums(type: Checksum) {
files = objects.fileCollection()
[
tasks.assembleSourceTgz,
- distTarTask,
+ fullDistTarTask,
+ slimDistTarTask,
].each { dep ->
dependsOn dep
files += dep.outputs.files
@@ -80,8 +82,11 @@ task computeChecksums(type: Checksum) {
outputDir = file("${buildDir}/checksums")
}
-task signBinaryTgz(type: Sign) {
- sign distTarTask
+task signFullBinaryTgz(type: Sign) {
+ sign fullDistTarTask
+}
+task signSlimBinaryTgz(type: Sign) {
+ sign slimDistTarTask
}
task signSourceTgz(type: Sign) {
// The source tgz is not an archive task so be explicit about the outputs to
sign.
@@ -90,7 +95,8 @@ task signSourceTgz(type: Sign) {
}
task signReleaseArchives(type: Sync) {
- from tasks.signBinaryTgz
+ from tasks.signFullBinaryTgz
+ from tasks.signSlimBinaryTgz
from tasks.signSourceTgz
into "${buildDir}/signatures"
@@ -130,7 +136,8 @@ task assembleRelease(type: Sync) {
from tasks.prepareGitRev
from tasks.assembleSourceTgz
- from distTarTask
+ from fullDistTarTask
+ from slimDistTarTask
from tasks.computeChecksums
diff --git a/solr/docker/README.md b/solr/docker/README.md
index c8df31b72d5..c9c6a9b78da 100644
--- a/solr/docker/README.md
+++ b/solr/docker/README.md
@@ -30,6 +30,7 @@ Building from the Solr Binary Distribution
Officially-compliant Docker images can be built directly from the Solr binary
distribution (i.e. `solr-<version>.tgz`).
A Dockerfile is included in the binary distribution, under
`solr-<version>/docker/Dockerfile`, and is the same one used when building a
docker image via Gradle.
+The slim binary distribution can also be used by using `solr-<version>-slim`
instead of `solr-<version>`.
To build the Docker image, pass the Solr TGZ as the Docker context and provide
the path of the Dockerfile.
Note, that Docker will accept either a URL or a local TGZ file, but each
require slightly different syntax.
@@ -38,6 +39,7 @@ Therefore custom Solr releases or official releases can be
used to create custom
```bash
docker build -f solr-X.Y.Z/docker/Dockerfile - < solr-X.Y.Z.tgz
docker build -f solr-X.Y.Z/docker/Dockerfile
https://www.apache.org/dyn/closer.lua/solr/X.Y.Z/solr-X.Y.Z.tgz
+docker build -f solr-X.Y.Z-slim/docker/Dockerfile
https://www.apache.org/dyn/closer.lua/solr/X.Y.Z/solr-X.Y.Z-slim.tgz
```
When building the image, Solr accepts arguments for customization. Currently
only one argument is accepted:
diff --git a/solr/docker/build.gradle b/solr/docker/build.gradle
index 5a0ce4c9316..a84e4ec245a 100644
--- a/solr/docker/build.gradle
+++ b/solr/docker/build.gradle
@@ -24,11 +24,16 @@ description = 'Solr Docker image'
apply plugin: 'base'
// Solr Docker inputs
+def dockerImageSolrDist = "${ -> propertyOrEnvOrDefault("solr.docker.dist",
"SOLR_DOCKER_DIST", 'full') }"
+def isDistSlim = {String dist -> dist.toLowerCase(Locale.ROOT) == "slim"}
+def isImageSlim = { -> isDistSlim(dockerImageSolrDist) }
+def distToSuffix = {String dist -> isDistSlim(dist) ? "-slim" : ""}
+def dockerImageDistSuffix = "${ -> distToSuffix(dockerImageSolrDist)}"
def dockerImageRepo = "${ -> propertyOrEnvOrDefault("solr.docker.imageRepo",
"SOLR_DOCKER_IMAGE_REPO", "apache/solr") }"
-def dockerImageTag = "${ -> propertyOrEnvOrDefault("solr.docker.imageTag",
"SOLR_DOCKER_IMAGE_TAG", project.version) }"
+def dockerImageTag = "${ -> propertyOrEnvOrDefault("solr.docker.imageTag",
"SOLR_DOCKER_IMAGE_TAG", project.version + dockerImageDistSuffix) }"
def dockerImageName = "${ -> propertyOrEnvOrDefault("solr.docker.imageName",
"SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}") }"
def baseDockerImage = "${ -> propertyOrEnvOrDefault("solr.docker.baseImage",
"SOLR_DOCKER_BASE_IMAGE", 'eclipse-temurin:17-jre-jammy') }"
-def officialDockerImageName = "${ ->
propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME",
"${dockerImageRepo}-official:${dockerImageTag}") }"
+def officialDockerImageName = {String dist -> "${ ->
propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME",
"${dockerImageRepo}-official:${dockerImageTag}${distToSuffix(dist)}") }" }
def releaseGpgFingerprint = "${ ->
propertyOrDefault('signing.gnupg.keyName',propertyOrDefault('signing.keyId',''))
}"
@@ -36,7 +41,7 @@ def releaseGpgFingerprint = "${ ->
propertyOrDefault('signing.gnupg.keyName',pro
def imageIdFile = "$buildDir/image-id"
def smokeTestOfficial = "$buildDir/smoke-check-official"
-def imageIdFileOfficial = "$smokeTestOfficial/image-id"
+def imageIdFileOfficial = { String dist -> "$smokeTestOfficial/$dist/image-id"
}
configurations {
packaging {
@@ -48,18 +53,30 @@ configurations {
packagingOfficial {
canBeResolved = true
}
- solrTgz {
+ solrFullTgz {
canBeConsumed = false
canBeResolved = true
}
- solrTgzSignature {
+ solrSlimTgz {
+ canBeConsumed = false
+ canBeResolved = true
+ }
+ solrFullTgzSignature {
+ canBeConsumed = false
+ canBeResolved = true
+ }
+ solrSlimTgzSignature {
canBeConsumed = false
canBeResolved = true
}
dockerImage {
canBeResolved = true
}
- dockerOfficialSmokeCheckImage {
+ dockerOfficialFullSmokeCheckImage {
+ canBeConsumed = false
+ canBeResolved = true
+ }
+ dockerOfficialSlimSmokeCheckImage {
canBeConsumed = false
canBeResolved = true
}
@@ -79,18 +96,27 @@ dependencies {
builtBy 'createDockerfileLocal'
}
- packagingOfficial files("${dockerfilesDirPath}/Dockerfile.official") {
- builtBy 'createDockerfileOfficial'
+ packagingOfficial files("${dockerfilesDirPath}/Dockerfile.official-full") {
+ builtBy 'createDockerfileOfficialFull'
+ }
+
+ packagingOfficial files("${dockerfilesDirPath}/Dockerfile.official-slim") {
+ builtBy 'createDockerfileOfficialSlim'
}
- solrTgz project(path: ":solr:packaging", configuration: "solrTgz")
- solrTgzSignature project(path: ":solr:packaging", configuration:
'solrTgzSignature')
+ solrFullTgz project(path: ":solr:packaging", configuration: "solrFullTgz")
+ solrSlimTgz project(path: ":solr:packaging", configuration: "solrSlimTgz")
+ solrFullTgzSignature project(path: ":solr:packaging", configuration:
'solrFullTgzSignature')
+ solrSlimTgzSignature project(path: ":solr:packaging", configuration:
'solrSlimTgzSignature')
dockerImage files(imageIdFile) {
builtBy 'dockerBuild'
}
- dockerOfficialSmokeCheckImage files(imageIdFileOfficial) {
- builtBy 'testBuildDockerfileOfficial'
+ dockerOfficialFullSmokeCheckImage files(imageIdFileOfficial("Full")) {
+ builtBy 'testBuildDockerfileOfficialFull'
+ }
+ dockerOfficialSlimSmokeCheckImage files(imageIdFileOfficial("Slim")) {
+ builtBy 'testBuildDockerfileOfficialSlim'
}
}
@@ -110,12 +136,13 @@ def checksum = { file ->
task assemblePackaging(type: Sync) {
description = 'Assemble docker scripts and Dockerfile for Solr Packaging'
+ dependsOn configurations.packagingLocal
from(projectDir, {
include "scripts/**"
include "README.md"
})
- from(dockerfilesDirPath, {
+ from(configurations.packagingLocal, {
include 'Dockerfile.local'
rename {
'Dockerfile'
@@ -124,7 +151,7 @@ task assemblePackaging(type: Sync) {
into packagingDir
}
-task dockerBuild(dependsOn: configurations.solrTgz) {
+task dockerBuild() {
group = 'Docker'
description = 'Build Solr docker image'
@@ -132,13 +159,16 @@ task dockerBuild(dependsOn: configurations.solrTgz) {
inputs.properties([
baseDockerImage: baseDockerImage
])
- inputs.files(configurations.solrTgz)
+ var solrTgzConfiguration = isImageSlim() ? configurations.solrSlimTgz :
configurations.solrFullTgz
+ inputs.files(solrTgzConfiguration)
+ inputs.property("isSlimImage", isImageSlim())
+ dependsOn(solrTgzConfiguration)
doLast {
exec {
- standardInput = configurations.solrTgz.singleFile.newDataInputStream()
+ standardInput = solrTgzConfiguration.singleFile.newDataInputStream()
commandLine "docker", "build",
- "-f", "solr-${ -> project.version }/docker/Dockerfile",
+ "-f", "solr-${ -> project.version
}${dockerImageDistSuffix}/docker/Dockerfile",
"--iidfile", imageIdFile,
"--build-arg", "BASE_IMAGE=${ ->
inputs.properties.baseDockerImage}",
"-"
@@ -152,6 +182,7 @@ task dockerBuild(dependsOn: configurations.solrTgz) {
project.logger.lifecycle("\tID: \t${ -> dockerImageId }")
project.logger.lifecycle("\tBase Image: \t${ -> baseDockerImage }")
project.logger.lifecycle("\tSolr Version: \t${ -> project.version }")
+ project.logger.lifecycle("\tSolr Distribution:
\t${dockerImageDistSuffix.isEmpty() ? "Full" : "Slim"}")
}
outputs.files(imageIdFile)
@@ -191,7 +222,12 @@ task testDocker(type: TestDockerImageTask, dependsOn:
tasks.dockerBuild) {
// include/exclude options are designed for people who know their
customizations will break some tests
testCasesInclude.value(Arrays.asList(propertyOrEnvOrDefault("solr.docker.tests.include",
"SOLR_DOCKER_TESTS_INCLUDE", ",").split(",")))
-
testCasesExclude.value(Arrays.asList(propertyOrEnvOrDefault("solr.docker.tests.exclude",
"SOLR_DOCKER_TESTS_EXCLUDE", ",").split(",")))
+ var excludeCases = new
HashSet(Arrays.asList(propertyOrEnvOrDefault("solr.docker.tests.exclude",
"SOLR_DOCKER_TESTS_EXCLUDE", ",").split(",")))
+ if (isImageSlim()) {
+ // The slim image does not contain the prometheus-exporter.
+ excludeCases.add("prometheus-exporter")
+ }
+ testCasesExclude.value(excludeCases)
}
task dockerPush(dependsOn: tasks.dockerTag) {
@@ -236,6 +272,7 @@ ext {
task createBodySnippetDockerfile(type: Copy) {
from 'templates/Dockerfile.body.template'
into "$buildDir/snippets/"
+ outputs.file("$buildDir/snippets/Dockerfile.body.snippet")
rename { name -> name.replace("template","snippet") }
filteringCharset 'UTF-8'
@@ -253,6 +290,7 @@ ext {
// where 'FOO' is the key used in 'props' (NOTE the leading and trailing
literal '_' characters)
dfLocalDetails = [
name: 'Local',
+ template: 'local',
desc: 'Dockerfile used to create local Solr docker images directly from
Solr release tgz file',
// NOTE: There should be no reason for Dockerfile.local to include unique
values
@@ -261,9 +299,10 @@ ext {
// be defined in the task creation.
props: [:]
]
- dfOfficialDetails = [
- name: 'Official',
- desc: 'Dockerfile used to create official Solr docker images published to
hub.docker.io',
+ dfOfficialFullDetails = [
+ name: 'OfficialFull',
+ template: 'official',
+ desc: 'Dockerfile used to create official Full Solr docker images
published to hub.docker.io',
props: [
// NOTE: Only include values here that are distinct and unique to the
Official Dockerfiles
//
@@ -271,14 +310,32 @@ ext {
// be defined in the task creation
// NOTE: SHA is lazy computed...
- 'SOLR_TGZ_SHA': "${ -> checksum(configurations.solrTgz.singleFile) }",
- 'RELEASE_MANAGER_GPG_FINGERPRINT': "${ -> releaseGpgFingerprint}"
+ 'SOLR_TGZ_SHA': "${ -> checksum(configurations.solrFullTgz.singleFile)
}",
+ 'RELEASE_MANAGER_GPG_FINGERPRINT': "${ -> releaseGpgFingerprint}",
+ 'SOLR_DIST': ""
]
]
+ dfOfficialSlimDetails = [
+ name: 'OfficialSlim',
+ template: 'official',
+ desc: 'Dockerfile used to create official Slim Solr docker images
published to hub.docker.io',
+ props: [
+ // NOTE: Only include values here that are distinct and unique to
the Official Dockerfiles
+ //
+ // Values identical in both Dockerfiles should use consistent names
in both templates and
+ // be defined in the task creation
+
+ // NOTE: SHA is lazy computed...
+ 'SOLR_TGZ_SHA': "${ ->
checksum(configurations.solrSlimTgz.singleFile) }",
+ 'RELEASE_MANAGER_GPG_FINGERPRINT': "${ -> releaseGpgFingerprint}",
+ 'SOLR_DIST': "-slim"
+ ]
+ ]
}
/*
This section creates the following gradle tasks:
-- createDockerfileOfficial
+- createDockerfileOfficialFull
+- createDockerfileOfficialSlim
- createDockerfileLocal
Both will create a self-standing Dockerfile that can be used to build a Solr
image.
@@ -288,10 +345,10 @@ These templates can be found in the templates/ directory.
The snippets of each section (header and body) are saved to build/snippets
after they are templated and before they are combined.
The final Dockerfiles are merely the snippet headers combined with the snippet
body.
*/
-[ dfLocalDetails, dfOfficialDetails ].each{ details ->
- def fileName = "Dockerfile.${ -> details.name.toLowerCase(Locale.ROOT) }"
+[ dfLocalDetails, dfOfficialFullDetails, dfOfficialSlimDetails ].each{ details
->
+ def fileName = "Dockerfile.${ -> details.name.replaceAll("(.)([A-Z])",
"\$1-\$2").toLowerCase(Locale.ROOT) }"
def outFile = file("$dockerfilesDirPath/${fileName}")
-
+
tasks.create("createDockerfile${details.name}", Copy) {
description "Creates ${details.desc}"
@@ -307,10 +364,10 @@ The final Dockerfiles are merely the snippet headers
combined with the snippet b
inputs.properties(props)
inputs.file("$buildDir/snippets/Dockerfile.body.snippet")
outputs.file(outFile)
-
- from "templates/${fileName}.header.template"
+
+ from "templates/Dockerfile.${details.template}.header.template"
into "$buildDir/snippets/"
- rename { name -> name.replace("template","snippet") }
+ rename { fileName + ".header.snippet" }
filteringCharset 'UTF-8'
filter( commentFilter )
filter( { line ->
@@ -343,123 +400,155 @@ The final Dockerfiles are merely the snippet headers
combined with the snippet b
}
}
assemblePackaging.dependsOn tasks.createDockerfileLocal
-tasks.createDockerfileOfficial.dependsOn configurations.solrTgz // to lazy
compute SHA
+tasks.createDockerfileOfficialFull.dependsOn configurations.solrFullTgz // to
lazy compute SHA
+tasks.createDockerfileOfficialSlim.dependsOn configurations.solrSlimTgz // to
lazy compute SHA
+tasks.createDockerfileOfficialSlim.mustRunAfter
tasks.createDockerfileOfficialFull // Eliminate a weird warning
+
+task createDockerfileOfficial {
+ dependsOn tasks.createDockerfileOfficialFull
+ dependsOn tasks.createDockerfileOfficialSlim
+}
// sanity check...
if (''.equals(releaseGpgFingerprint)) {
gradle.taskGraph.whenReady { graph ->
- if ( graph.hasTask(createDockerfileOfficial) ) {
+ if ( graph.hasTask(tasks.createDockerfileOfficialFull) ||
graph.hasTask(tasks.createDockerfileOfficialSlim) ) {
throw new GradleException("No GPG keyName found, please see
help/publishing.txt (GPG key is neccessary to create Dockerfile.official)")
}
}
}
-task testBuildDockerfileOfficial(type: Copy) {
- description = 'Test "docker build" works with our generated
Dockerfile.official using Mocked URLs'
+[ 'Full', 'Slim' ].each { variant ->
+ def lowerVariant = variant.toLowerCase(Locale.ROOT)
- dependsOn createDockerfileOfficial
- dependsOn configurations.solrTgz
- dependsOn configurations.solrTgzSignature
+ tasks.create("testBuildDockerfileOfficial${variant}", Copy) {
+ description = "Test 'docker build' works with our generated
Dockerfile.official-${lowerVariant} using Mocked URLs"
- def mockHttpdHome = file("$smokeTestOfficial/mock-httpd-home");
-
- inputs.file("$dockerfilesDirPath/Dockerfile.official")
- outputs.dir(mockHttpdHome)
- outputs.file(imageIdFileOfficial)
+ dependsOn tasks.named("createDockerfileOfficial${variant}")
+ def mockHttpdHome =
file("$smokeTestOfficial/$lowerVariant/mock-httpd-home");
- from configurations.solrTgzSignature
- from configurations.solrTgz
- into mockHttpdHome
-
- doLast {
- // A file to record the container ID of our mock httpd
- def mockServerIdFile =
file("${buildDir}/dockerfile-mock-artifact-server-cid.txt")
-
- // if we encounter any problems running our test, we'll fill this in and
use it to suppress any
- // other exceptions we encounter on cleanup...
- def mainException = null;
-
- // TODO: setup a little 'suppressOrThrow(Exception)' closure for reuse
below....
-
- try {
- // run an httpd server to host our artifacts
- logger.lifecycle('Running mock HTTPD server our testing...');
- exec {
- commandLine 'docker', 'run',
- '--cidfile', mockServerIdFile,
- '--rm',
- '-d',
- '-v', "${mockHttpdHome.absoluteFile}:/data",
- '-w', '/data',
- 'python:3-alpine', 'python', '-m', 'http.server', '9876'
- }
- try {
- def mockServerId = mockServerIdFile.text
- def mockServerIpStdOut = new ByteArrayOutputStream()
- exec{
- commandLine 'docker', 'inspect', "--format={{range
.NetworkSettings.Networks}}{{.IPAddress}}{{end}}", mockServerId
- standardOutput = mockServerIpStdOut
- }
- def mockServerIp = mockServerIpStdOut.toString().trim()
+ inputs.file("$dockerfilesDirPath/Dockerfile.official-$lowerVariant")
+ outputs.dir(mockHttpdHome)
+ outputs.file(imageIdFileOfficial(variant))
- // *NOW* we can actually run our docker build command...
- logger.lifecycle('Running docker build on Dockerfile.official...');
- exec {
- standardInput =
file("${dockerfilesDirPath}/Dockerfile.official").newDataInputStream()
- commandLine 'docker', 'build',
- '--add-host', "mock-solr-dl-server:${mockServerIp}",
- '--no-cache', // force fresh downloads from our current network
- "--iidfile", imageIdFileOfficial,
- '--build-arg',
"SOLR_CLOSER_URL=http://mock-solr-dl-server:9876/solr-${ ->
project.version}.tgz",
- '--build-arg',
"SOLR_ARCHIVE_URL=http://mock-solr-dl-server:9876/solr-${ ->
project.version}.tgz",
- '--tag', officialDockerImageName,
- '-'
- }
+ dependsOn configurations.named("solr${variant}Tgz")
+ dependsOn configurations.named("solr${variant}TgzSignature")
+ from configurations.named("solr${variant}Tgz")
+ from configurations.named("solr${variant}TgzSignature")
+ into mockHttpdHome
- def officialDockerImageId = file(imageIdFileOfficial).text
+ doLast {
+ // A file to record the container ID of our mock httpd
+ def mockServerIdFile =
file("$smokeTestOfficial/$lowerVariant/dockerfile-mock-artifact-server-cid.txt")
- // Print information on the image after it has been created
- project.logger.lifecycle("\"Official\" Solr Docker Image Tagged")
- project.logger.lifecycle("\tID: \t$officialDockerImageId")
- project.logger.lifecycle("\tTag: \t$officialDockerImageName")
- } finally {
- // Try to shut down our mock httpd server....
- if (mockServerIdFile.exists()) {
+ // if we encounter any problems running our test, we'll fill this in and
use it to suppress any
+ // other exceptions we encounter on cleanup...
+ def mainException = null;
+
+ // TODO: setup a little 'suppressOrThrow(Exception)' closure for reuse
below....
+
+ try {
+ // run an httpd server to host our artifacts
+ logger.lifecycle('Running mock HTTPD server our testing...');
+ exec {
+ commandLine 'docker', 'run',
+ '--cidfile', mockServerIdFile,
+ '--rm',
+ '-d',
+ '-v', "${mockHttpdHome.absoluteFile}:/data",
+ '-w', '/data',
+ 'python:3-alpine', 'python', '-m', 'http.server', '9876'
+ }
+ try {
def mockServerId = mockServerIdFile.text
- try {
- exec { commandLine 'docker', 'stop', mockServerId }
- } catch (Exception e) {
- logger.error("Unable to stop docker container ${mockServerId}", e)
- if (null != mainException) {
- mainException.addSuppressed(e);
- } else {
- mainException = e;
- throw e;
+ def mockServerIpStdOut = new ByteArrayOutputStream()
+ exec {
+ commandLine 'docker', 'inspect', "--format={{range
.NetworkSettings.Networks}}{{.IPAddress}}{{end}}", mockServerId
+ standardOutput = mockServerIpStdOut
+ }
+ def mockServerIp = mockServerIpStdOut.toString().trim()
+
+ // *NOW* we can actually run our docker build command...
+ logger.lifecycle('Running docker build on Dockerfile.official...');
+ exec {
+ standardInput =
file("${dockerfilesDirPath}/Dockerfile.official-${lowerVariant}").newDataInputStream()
+ commandLine 'docker', 'build',
+ '--add-host', "mock-solr-dl-server:${mockServerIp}",
+ '--no-cache', // force fresh downloads from our current network
+ "--iidfile", imageIdFileOfficial(variant),
+ '--build-arg',
"SOLR_CLOSER_URL=http://mock-solr-dl-server:9876/solr-${->
project.version}${distToSuffix(variant)}.tgz",
+ '--build-arg',
"SOLR_ARCHIVE_URL=http://mock-solr-dl-server:9876/solr-${->
project.version}${distToSuffix(variant)}.tgz",
+ '--tag', officialDockerImageName(variant),
+ '-'
+ }
+
+ def officialDockerImageId = file(imageIdFileOfficial(variant)).text
+
+ // Print information on the image after it has been created
+ project.logger.lifecycle("\"Official $variant\" Solr Docker Image
Tagged")
+ project.logger.lifecycle("\tID: \t$officialDockerImageId")
+ project.logger.lifecycle("\tTag:
\t${officialDockerImageName(variant)}")
+ } finally {
+ // Try to shut down our mock httpd server....
+ if (mockServerIdFile.exists()) {
+ def mockServerId = mockServerIdFile.text
+ try {
+ exec { commandLine 'docker', 'stop', mockServerId }
+ } catch (Exception e) {
+ logger.error("Unable to stop docker container ${mockServerId}",
e)
+ if (null != mainException) {
+ mainException.addSuppressed(e);
+ } else {
+ mainException = e;
+ throw e;
+ }
+ } finally {
+ project.delete(mockServerIdFile)
}
- } finally {
- project.delete(mockServerIdFile)
}
}
+ } catch (Exception e) {
+ mainException = e
+ throw e;
}
- } catch (Exception e) {
- mainException = e
- throw e;
}
}
}
-task testDockerfileOfficial(type: TestDockerImageTask, dependsOn:
configurations.dockerOfficialSmokeCheckImage) {
- description = 'Smoke Test Solr docker image built from the official
Dockerfile'
+task testBuildDockerfileOfficial {
+ dependsOn tasks.testBuildDockerfileOfficialFull
+ dependsOn tasks.testBuildDockerfileOfficialSlim
+}
+
+task testDockerfileOfficialFull(type: TestDockerImageTask, dependsOn:
configurations.dockerOfficialFullSmokeCheckImage) {
+ description = 'Smoke Test Full Solr docker image built from the official
Dockerfile'
+
+ idFile = file(imageIdFileOfficial("Full"))
+ outputDir = file("$smokeTestOfficial/full/test-results")
- idFile = file(imageIdFileOfficial)
- outputDir = file("$smokeTestOfficial/test-results")
-
// This test does not respect the include/exclude properties that
`testDocker` does.
// All docker tests will be run, no matter the properties specified.
testCasesInclude.empty()
testCasesExclude.empty()
}
+task testDockerfileOfficialSlim(type: TestDockerImageTask, dependsOn:
configurations.dockerOfficialSlimSmokeCheckImage) {
+ description = 'Smoke Test Slim Solr docker image built from the official
Slim Dockerfile'
+
+ idFile = file(imageIdFileOfficial("Slim"))
+ outputDir = file("$smokeTestOfficial/slim/test-results")
+
+ // This test does not respect the include/exclude properties that
`testDocker` does.
+ // All docker tests will be run, no matter the properties specified.
+ testCasesInclude.empty()
+ testCasesExclude = ['prometheus-exporter']
+}
+
+task testDockerfileOfficial {
+ dependsOn tasks.testDockerfileOfficialFull
+ dependsOn tasks.testDockerfileOfficialSlim
+}
+
// Re-usable class for running tests...
abstract class TestDockerImageTask extends DefaultTask {
diff --git a/solr/docker/gradle-help.txt b/solr/docker/gradle-help.txt
index 61faa3400c6..2f4cd114ac5 100644
--- a/solr/docker/gradle-help.txt
+++ b/solr/docker/gradle-help.txt
@@ -21,6 +21,11 @@ Base Docker Image: (The docker image used for the "FROM" in
the Solr Dockerfile)
EnvVar: SOLR_DOCKER_BASE_IMAGE
Gradle Property: -Psolr.docker.baseImage
+Solr Distribution: (Either Full or Slim, the solr binary distribution to build
the docker image with)
+ Default: "full"
+ EnvVar: SOLR_DOCKER_DIST
+ Gradle Property: -Psolr.docker.dist
+
Tagging and Pushing
-------
@@ -83,28 +88,33 @@ Run tests in parallel:
Gradle Property: -Psolr.docker.tests.concurrent=true
-------
-The Official Solr Image
+The Official Solr Images
-------
-The Official Solr Docker Image is also generated within this module.
-This section should only be used by developers testing that their changes to
the Solr project are compatible with the Official image.
+The Official Solr Docker Images are also generated within this module.
+One Dockerfile/Docker image is created for each Solr binary package, "full"
and "slim".
+This section should only be used by developers testing that their changes to
the Solr project are compatible with the Official images.
All users should build custom images using the instructions above.
-NOTE: All gradle commands for the Official Dockerfile below require the Solr
artifacts to be signed with a GPG Key.
+NOTE: All gradle commands for the Official Dockerfiles below require the Solr
artifacts to be signed with a GPG Key.
For necessary inputs and properties, please refer to:
gradlew helpPublishing
-You can use the following command to build an official Solr Dockerfile.
-The Dockerfile will be created at: solr/docker/build/Dockerfile.official
+You can use the following command to build both official Solr Dockerfiles.
+The Dockerfiles will be created at:
+- solr/docker/build/Dockerfile.official-full
+- solr/docker/build/Dockerfile.official-slim
gradlew createDockerfileOfficial
-You can also test the official docker image using the following command.
-This will build the official Dockerfile, create a local server to host the
local Solr artifacts, and build the Official Solr image using this local server.
+You can also test building the official docker images using the following
command.
+This will build the official Dockerfiles, create a local server to host the
local Solr artifacts, and build the Official Solr images using this local
server.
gradlew testBuildDockerfileOfficial
-You can also run the official Docker image built by the command above through
all Solr Docker tests with the following:
+You can also run the official Docker images built by the command above through
all Solr Docker tests with the following:
gradlew testDockerfileOfficial
+
+In order to build/test just one of the "full" or "slim" images/Dockerfiles,
just append "Full" or "Slim" to any of the *DockerfileOfficial tasks above.
diff --git a/solr/docker/templates/Dockerfile.body.template
b/solr/docker/templates/Dockerfile.body.template
index 63ac7334718..ea627e2a6c5 100644
--- a/solr/docker/templates/Dockerfile.body.template
+++ b/solr/docker/templates/Dockerfile.body.template
@@ -65,8 +65,8 @@ RUN set -ex; \
chmod 0664 /etc/default/solr.in.sh; \
mkdir -p -m0770 /var/solr; \
chown -R "$SOLR_USER:0" /var/solr; \
- ln -s /opt/solr/modules /opt/solr/contrib; \
- ln -s /opt/solr/prometheus-exporter /opt/solr/modules/prometheus-exporter;
+ test ! -e /opt/solr/modules || ln -s /opt/solr/modules /opt/solr/contrib; \
+ test ! -e /opt/solr/prometheus-exporter || ln -s
/opt/solr/prometheus-exporter /opt/solr/modules/prometheus-exporter;
RUN set -ex; \
apt-get update; \
diff --git a/solr/docker/templates/Dockerfile.local.header.template
b/solr/docker/templates/Dockerfile.local.header.template
index 6007ffd3389..810d84f7e5c 100644
--- a/solr/docker/templates/Dockerfile.local.header.template
+++ b/solr/docker/templates/Dockerfile.local.header.template
@@ -30,10 +30,12 @@
# available at https://hub.docker.com/_/solr -- however this file can be used
to build docker images from
# a Solr release artifact -- either from a remote TGZ file, or from an TGZ
artifact you have downloaded
# locally.
-# Example:
+# Examples:
# docker build -f solr-X.Y.Z/docker/Dockerfile
https://www.apache.org/dyn/closer.lua/solr/X.Y.Z/solr-X.Y.Z.tgz
-# Example:
+# docker build -f solr-X.Y.Z-slim/docker/Dockerfile
https://www.apache.org/dyn/closer.lua/solr/X.Y.Z/solr-X.Y.Z-slim.tgz
+# Examples:
# docker build -f solr-X.Y.Z/docker/Dockerfile - < solr-X.Y.Z.tgz
+# docker build -f solr-X.Y.Z-slim/docker/Dockerfile - <
solr-X.Y.Z-slim.tgz
ARG BASE_IMAGE=_REPLACE_BASE_IMAGE_
diff --git a/solr/docker/templates/Dockerfile.official.header.template
b/solr/docker/templates/Dockerfile.official.header.template
index c1e99133c94..96bf2e51edd 100644
--- a/solr/docker/templates/Dockerfile.official.header.template
+++ b/solr/docker/templates/Dockerfile.official.header.template
@@ -24,6 +24,8 @@ FROM _REPLACE_BASE_IMAGE_
# TODO: remove things that exist solely for downstream specialization since
Dockerfile.local now exists for that
ARG SOLR_VERSION="_REPLACE_SOLR_VERSION_"
+# empty for the full distribution, "-slim" for the slim distribution
+ARG SOLR_DIST="_REPLACE_SOLR_DIST_"
ARG SOLR_SHA512="_REPLACE_SOLR_TGZ_SHA_"
ARG SOLR_KEYS="_REPLACE_RELEASE_MANAGER_GPG_FINGERPRINT_"
@@ -37,9 +39,9 @@ ARG SOLR_DOWNLOAD_URL
ARG SOLR_DOWNLOAD_SERVER
# These should never be overridden except for the purposes of testing the
Dockerfile before release
-ARG
SOLR_CLOSER_URL="http://www.apache.org/dyn/closer.lua?action=download&filename=/solr/solr/$SOLR_VERSION/solr-$SOLR_VERSION.tgz";
-ARG
SOLR_DIST_URL="https://www.apache.org/dist/solr/solr/$SOLR_VERSION/solr-$SOLR_VERSION.tgz";
-ARG
SOLR_ARCHIVE_URL="https://archive.apache.org/dist/solr/solr/$SOLR_VERSION/solr-$SOLR_VERSION.tgz";
+ARG
SOLR_CLOSER_URL="http://www.apache.org/dyn/closer.lua?action=download&filename=/solr/solr/$SOLR_VERSION/solr-$SOLR_VERSION$SOLR_DIST.tgz";
+ARG
SOLR_DIST_URL="https://www.apache.org/dist/solr/solr/$SOLR_VERSION/solr-$SOLR_VERSION$SOLR_DIST.tgz";
+ARG
SOLR_ARCHIVE_URL="https://archive.apache.org/dist/solr/solr/$SOLR_VERSION/solr-$SOLR_VERSION$SOLR_DIST.tgz";
RUN set -ex; \
apt-get update; \
@@ -65,26 +67,26 @@ RUN set -ex; \
MAX_REDIRECTS=4; \
SKIP_GPG_CHECK=true; \
elif [ -n "$SOLR_DOWNLOAD_SERVER" ]; then \
-
SOLR_DOWNLOAD_URL="$SOLR_DOWNLOAD_SERVER/$SOLR_VERSION/solr-$SOLR_VERSION.tgz";
\
+
SOLR_DOWNLOAD_URL="$SOLR_DOWNLOAD_SERVER/$SOLR_VERSION/solr-$SOLR_VERSION$SOLR_DIST.tgz";
\
fi; \
for url in $SOLR_DOWNLOAD_URL $SOLR_CLOSER_URL $SOLR_DIST_URL
$SOLR_ARCHIVE_URL; do \
- if [ -f "/opt/solr-$SOLR_VERSION.tgz" ]; then break; fi; \
+ if [ -f "/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz" ]; then break; fi; \
echo "downloading $url"; \
- if wget -t 10 --max-redirect $MAX_REDIRECTS --retry-connrefused -nv "$url"
-O "/opt/solr-$SOLR_VERSION.tgz"; then break; else rm -f
"/opt/solr-$SOLR_VERSION.tgz"; fi; \
+ if wget -t 10 --max-redirect $MAX_REDIRECTS --retry-connrefused -nv "$url"
-O "/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz"; then break; else rm -f
"/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz"; fi; \
done; \
- if [ ! -f "/opt/solr-$SOLR_VERSION.tgz" ]; then echo "failed all download
attempts for solr-$SOLR_VERSION.tgz"; exit 1; fi; \
+ if [ ! -f "/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz" ]; then echo "failed all
download attempts for solr-$SOLR_VERSION$SOLR_DIST.tgz"; exit 1; fi; \
if [ -z "$SKIP_GPG_CHECK" ]; then \
echo "downloading $SOLR_ARCHIVE_URL.asc"; \
- wget -nv "$SOLR_ARCHIVE_URL.asc" -O "/opt/solr-$SOLR_VERSION.tgz.asc"; \
- echo "$SOLR_SHA512 */opt/solr-$SOLR_VERSION.tgz" | sha512sum -c -; \
- (>&2 ls -l "/opt/solr-$SOLR_VERSION.tgz"
"/opt/solr-$SOLR_VERSION.tgz.asc"); \
- gpg --batch --verify "/opt/solr-$SOLR_VERSION.tgz.asc"
"/opt/solr-$SOLR_VERSION.tgz"; \
+ wget -nv "$SOLR_ARCHIVE_URL.asc" -O
"/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz.asc"; \
+ echo "$SOLR_SHA512 */opt/solr-$SOLR_VERSION$SOLR_DIST.tgz" | sha512sum -c
-; \
+ (>&2 ls -l "/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz"
"/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz.asc"); \
+ gpg --batch --verify "/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz.asc"
"/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz"; \
else \
echo "Skipping GPG validation due to non-Apache build"; \
fi; \
{ command -v gpgconf; gpgconf --kill all || :; }; \
rm -r "$GNUPGHOME"; \
- tar -C /opt --extract --preserve-permissions --file
"/opt/solr-$SOLR_VERSION.tgz"; \
- rm "/opt/solr-$SOLR_VERSION.tgz"*; \
+ tar -C /opt --extract --preserve-permissions --file
"/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz"; \
+ rm "/opt/solr-$SOLR_VERSION$SOLR_DIST.tgz"*; \
apt-get -y remove gpg dirmngr && apt-get -y autoremove;
diff --git a/solr/packaging/README.txt b/solr/packaging/README.txt
index d92997c1a25..3fab2773bb0 100644
--- a/solr/packaging/README.txt
+++ b/solr/packaging/README.txt
@@ -67,10 +67,12 @@ example/
directory containing various examples.
modules/
+ (Not included in the "slim" release)
Contains modules to extend the functionality of Solr.
Libraries for these modules can be found under modules/*/lib
prometheus-exporter/
+ (Not included in the "slim" release)
Contains a separate application to monitor Solr instances and export
Prometheus metrics
docker/
diff --git a/solr/packaging/build.gradle b/solr/packaging/build.gradle
index 5ce4507e09e..f06d9e48d70 100644
--- a/solr/packaging/build.gradle
+++ b/solr/packaging/build.gradle
@@ -29,7 +29,9 @@ description = 'Solr distribution packaging'
ext {
distDir = file("$buildDir/solr-${version}")
+ slimDistDir = file("$buildDir/solr-${version}-slim")
devDir = file("$buildDir/dev")
+ slimDevDir = file("$buildDir/dev-slim")
}
configurations {
@@ -39,12 +41,10 @@ configurations {
server
docs
docker
- solrTgz
- solrTgzSignature
-}
-
-artifacts {
- solrTgz(distTar)
+ solrFullTgz
+ solrSlimTgz
+ solrFullTgzSignature
+ solrSlimTgzSignature
}
dependencies {
@@ -62,14 +62,19 @@ dependencies {
docker project(path: ':solr:docker', configuration: 'packaging')
- solrTgzSignature files("$buildDir/distributions/solr-${version}.tgz.asc") {
- builtBy ":solr:distribution:signBinaryTgz"
+ solrFullTgzSignature
files("$buildDir/distributions/solr-${version}.tgz.asc") {
+ builtBy ":solr:distribution:signFullBinaryTgz"
+ }
+
+ solrSlimTgzSignature
files("$buildDir/distributions/solr-${version}-slim.tgz.asc") {
+ builtBy ":solr:distribution:signSlimBinaryTgz"
}
}
distributions {
- main {
+ slim {
distributionBaseName = 'solr'
+ distributionClassifier = "slim"
contents {
from(rootDir, {
@@ -92,18 +97,10 @@ distributions {
into 'lib'
})
- from(configurations.modules, {
- into "modules"
- })
-
from(configurations.example, {
into "example"
})
- from(configurations.prometheusExporter, {
- into "prometheus-exporter"
- })
-
from(configurations.server, {
into "server"
})
@@ -128,34 +125,97 @@ distributions {
"**/bin/postlogs",
"**/bin/solr",
"**/bin/init.d/solr",
- "**/bin/solr-exporter",
]) { copy ->
copy.setMode(0755)
}
}
}
+ full {
+ distributionBaseName = 'solr'
+ contents {
+ // Build on-top of the slim distribution
+ with(distributions.slim.getContents())
+
+ from(configurations.modules, {
+ into "modules"
+ })
+
+ from(configurations.prometheusExporter, {
+ into "prometheus-exporter"
+ filesMatching([
+ "bin/*",
+ ]) { copy ->
+ copy.setMode(0755)
+ }
+ })
+ }
+ }
}
-installDist {
+installFullDist {
into distDir
}
-task dev(type: Copy) {
- description "Assemble Solr distribution into 'development' folder at
${devDir}"
+installSlimDist {
+ into slimDistDir
+}
+
+assembleDist {
+ dependsOn tasks.assembleFullDist
+ dependsOn tasks.assembleSlimDist
+}
+
+installDist {
+ dependsOn tasks.installFullDist
+ dependsOn tasks.installSlimDist
+}
+
+task devFull(type: Copy) {
+ description "Assemble Full Solr distribution into 'development' folder at
${devDir}"
group "build"
- from installDist.outputs
+ from installFullDist.outputs
into devDir
}
-distTar {
+task devSlim(type: Copy) {
+ description "Assemble Slim Solr distribution into 'development' folder at
${slimDevDir}"
+ group "build"
+
+ from installSlimDist.outputs
+ into slimDevDir
+}
+
+task dev {
+ description "Assemble Solr distributions into 'development' folders at
${devDir} and ${slimDevDir}"
+ group "build"
+
+ dependsOn tasks.devFull
+ dependsOn tasks.devSlim
+}
+
+fullDistTar {
compression = Compression.GZIP
}
-distZip.enabled = false
+slimDistTar {
+ compression = Compression.GZIP
+}
+
+distTar.dependsOn tasks.fullDistTar
+distTar.dependsOn tasks.slimDistTar
+
+fullDistZip.enabled = false
+slimDistZip.enabled = false
-assemble.dependsOn installDist
+assemble.dependsOn tasks.installDist
+assemble.dependsOn tasks.installSlimDist
+
+artifacts {
+ solrFullTgz(fullDistTar)
+ solrSlimTgz(slimDistTar)
+}
task downloadBats(type: NpmTask) {
group = 'Build Dependency Download'
@@ -172,7 +232,7 @@ task downloadBats(type: NpmTask) {
}
task integrationTests(type: BatsTask) {
- dependsOn installDist
+ dependsOn installFullDist
dependsOn downloadBats
def integrationTestOutput = "$buildDir/test-output"
diff --git
a/solr/solr-ref-guide/modules/configuration-guide/pages/solr-modules.adoc
b/solr/solr-ref-guide/modules/configuration-guide/pages/solr-modules.adoc
index 8e203abd0e2..93a5f0b4ff6 100644
--- a/solr/solr-ref-guide/modules/configuration-guide/pages/solr-modules.adoc
+++ b/solr/solr-ref-guide/modules/configuration-guide/pages/solr-modules.adoc
@@ -26,6 +26,9 @@ Each module produces a separate `.jar` file in the build,
packaged in the module
All additional dependencies required by the module, and not provided by Solr
core, are also packaged there.
This helps keep the main core of Solr small and lean.
+Solr modules are not included in the `slim` variant of the Solr binary release.
+Please download/install the full Solr binary release in order to use Solr
modules.
+
== Installing a module
The easiest way to enable a module is to list the modules you intend to use
either in the
diff --git
a/solr/solr-ref-guide/modules/deployment-guide/pages/installing-solr.adoc
b/solr/solr-ref-guide/modules/deployment-guide/pages/installing-solr.adoc
index 115d48092f0..b9addf2cdb3 100644
--- a/solr/solr-ref-guide/modules/deployment-guide/pages/installing-solr.adoc
+++ b/solr/solr-ref-guide/modules/deployment-guide/pages/installing-solr.adoc
@@ -26,12 +26,16 @@ Please be sure to review the
xref:system-requirements.adoc[] before starting Sol
Solr is available from the Solr website.
Download the latest release https://solr.apache.org/downloads.html.
-There are two separate packages:
+There are three separate packages:
-* `solr-{solr-full-version}.tgz` the binary package, for all operating systems.
+* `solr-{solr-full-version}.tgz` the full binary package, for all operating
systems. This package includes all first-party Solr modules and accessories
(e.g. the `prometheus-exporter`).
+* `solr-{solr-full-version}-slim.tgz` the slim binary package, for all
operating systems. This package only includes what is necessary to run Solr.
Modules and accessories (e.g. the `prometheus-exporter`) are not included.
* `solr-{solr-full-version}-src.tgz` the package Solr source code.
This is useful if you want to develop on Solr without using the official Git
repository.
+Two docker images are provided that utilize the full and slim binaries.
+Please refer to the xref:solr-in-docker.adoc[] page for information on how to
use them.
+
== Preparing for Installation
When getting started with Solr, all you need to do is extract the Solr
distribution archive to a directory of your choosing.
@@ -94,10 +98,12 @@ It is described in more detail in the section
xref:taking-solr-to-production.ado
modules/::
Solr's `modules` directory includes 1st-party add-ons for specialized features
that enhance Solr.
See the section xref:configuration-guide:solr-modules.adoc[] for more
information.
+*This is not included in the `slim` distribution.*
prometheus-exporter/::
A standalone application, included under `bin/`, that montiors Solr instances
and produces Prometheus metrics.
See the section
xref:deployment-guide:monitoring-with-prometheus-and-grafana.adoc[] for more
information.
+*This is not included in the `slim` distribution.*
docker/::
This contains a Dockerfile to build a Docker image from the binary
distribution, that is compatible with the official image.
@@ -119,7 +125,7 @@ The `example` directory includes several types of examples
that demonstrate vari
See the section <<Solr Examples>> below for more details on what is in this
directory.
licenses/::
-The `licenses` directory includes all of the licenses for 3rd party libraries
used by Solr.
+The `licenses` directory includes all of the licenses for 3rd party libraries
used by that distribution of Solr.
server/::
This directory is where the heart of the Solr application resides.
@@ -132,7 +138,7 @@ See the section xref:configuring-logging.adoc[] for more
details on how to custo
== Solr Examples
-Solr includes a number of example documents and configurations to use when
getting started.
+The full Solr distribution includes a number of example documents and
configurations to use when getting started.
If you ran through the xref:getting-started:solr-tutorial.adoc[], you have
already interacted with some of these files.
Here are the examples included with Solr:
diff --git
a/solr/solr-ref-guide/modules/deployment-guide/pages/monitoring-with-prometheus-and-grafana.adoc
b/solr/solr-ref-guide/modules/deployment-guide/pages/monitoring-with-prometheus-and-grafana.adoc
index dbd4744f7bb..b05631f89a6 100644
---
a/solr/solr-ref-guide/modules/deployment-guide/pages/monitoring-with-prometheus-and-grafana.adoc
+++
b/solr/solr-ref-guide/modules/deployment-guide/pages/monitoring-with-prometheus-and-grafana.adoc
@@ -18,6 +18,9 @@
If you use https://prometheus.io[Prometheus] and https://grafana.com[Grafana]
for metrics storage and data visualization, Solr includes a Prometheus exporter
to collect metrics and other data.
+The Prometheus exporter is included with the full Solr distribution, and is
located under `prometheus-exporter/`.
+It is not included in the `slim` Solr distribution.
+
A Prometheus exporter (`solr-exporter`) allows users to monitor not only Solr
metrics which come from the xref:metrics-reporting.adoc#metrics-api[Metrics
API], but also facet counts which come from xref:query-guide:faceting.adoc[]
and responses to xref:configuration-guide:collections-api.adoc[] commands and
xref:ping.adoc[] requests.
This graphic provides a more detailed view:
@@ -25,8 +28,6 @@ This graphic provides a more detailed view:
.solr-exporter Diagram
image::monitoring-with-prometheus-and-grafana/solr-exporter-diagram.png[image,width=600]
-The Prometheus exporter is included with the Solr distribution, and is located
under `prometheus-exporter/`.
-
There are three aspects to running `solr-exporter`:
* Modify the `solr-exporter-config.xml` to define the data to collect.
diff --git
a/solr/solr-ref-guide/modules/deployment-guide/pages/solr-in-docker.adoc
b/solr/solr-ref-guide/modules/deployment-guide/pages/solr-in-docker.adoc
index 556af22f72c..491b2ba248c 100644
--- a/solr/solr-ref-guide/modules/deployment-guide/pages/solr-in-docker.adoc
+++ b/solr/solr-ref-guide/modules/deployment-guide/pages/solr-in-docker.adoc
@@ -24,6 +24,18 @@ Instructions below apply to `solr:8.0.0` and above.
See the https://hub.docker.com/_/solr?tab=tags[Docker Hub page] for a full
list of tags and architectures available.
+=== Available images
+
+Two docker images are provided for each release, full and slim.
+These correspond to the two binary distributions that are produced for each
Solr release.
+The docker images for these distributions can be found at:
+
+[horizontal]
+Full distribution:: `solr:<version>`
+Slim distribution:: `solr:<version>-slim`
+
+Please refer to the
xref:installing-solr.adoc#available-solr-packages[Available Solr Packages]
section for more information on each distribution.
+
=== Running Solr with host-mounted directories
Typically users first want to run a single standalone Solr server in a
container, with a single core for data, while storing data in a local directory.
diff --git
a/solr/solr-ref-guide/modules/getting-started/pages/solr-tutorial.adoc
b/solr/solr-ref-guide/modules/getting-started/pages/solr-tutorial.adoc
index df73ff2917b..13f6418d581 100644
--- a/solr/solr-ref-guide/modules/getting-started/pages/solr-tutorial.adoc
+++ b/solr/solr-ref-guide/modules/getting-started/pages/solr-tutorial.adoc
@@ -42,7 +42,7 @@ For best results, please run the browser showing this
tutorial and the Solr serv
== Unpack Solr
-Begin by unzipping the Solr release and changing your working directory to the
subdirectory where Solr was installed.
+Begin by unzipping the full Solr release and changing your working directory
to the subdirectory where Solr was installed.
For example, with a shell in UNIX, Cygwin, or MacOS:
[,console]
diff --git
a/solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-9.adoc
b/solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-9.adoc
index 1c07cd637b4..04e8338c80b 100644
---
a/solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-9.adoc
+++
b/solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-9.adoc
@@ -65,6 +65,11 @@ In Solr 8, it was possible to add docValues to a schema
without re-indexing via
Due to changes in Lucene 9, that isn't possible any more.
== Solr 9.3
+=== Binary Releases
+* Solr now comes with two
xref:deployment-guide:installing-solr.adoc#available-solr-packages[binary
releases] and xref:deployment-guide:solr-in-docker.adoc#available-images[docker
images], **full** and **slim**.
+The
xref:deployment-guide:installing-solr.adoc#available-solr-packages[Installing
Solr] page provides information on what is included in each.
+Please refer to the https://solr.apache.org/downloads.html[Solr Downloads]
site for information on how to download these offerings.
+
=== Shard Management
* Solr now provides an
xref:deployment-guide:shard-management.adoc#installshard[INSTALLSHARD] API to
allow users who have built (per-shard) indices offline to import them into
SolrCloud shards.
