This is an automated email from the ASF dual-hosted git repository.
baodi pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new 6e206e625119 Fix retention and backlog quota diagram (#973)
6e206e625119 is described below
commit 6e206e625119786e186af071fd9136cdba51035b
Author: Baodi Shi <[email protected]>
AuthorDate: Tue Nov 12 15:50:35 2024 +0800
Fix retention and backlog quota diagram (#973)
* Fix retention and backlog quota diagram
* Address code review
* Add source diagrams file
---
diagrams/backlog-quota-new.graffle | Bin 0 -> 794790 bytes
diagrams/retention-new.graffle | Bin 0 -> 763658 bytes
docs/cookbooks-retention-expiry.md | 10 +-
static/assets/backlog-quota-new.svg | 262 +++++++++++++++++++++
static/assets/retention-new.svg | 233 ++++++++++++++++++
.../version-3.0.x/cookbooks-retention-expiry.md | 11 +-
.../version-3.3.x/cookbooks-retention-expiry.md | 10 +-
.../version-4.0.x/cookbooks-retention-expiry.md | 10 +-
8 files changed, 515 insertions(+), 21 deletions(-)
diff --git a/diagrams/backlog-quota-new.graffle
b/diagrams/backlog-quota-new.graffle
new file mode 100644
index 000000000000..ca7c967399be
Binary files /dev/null and b/diagrams/backlog-quota-new.graffle differ
diff --git a/diagrams/retention-new.graffle b/diagrams/retention-new.graffle
new file mode 100644
index 000000000000..2193c0ba99e6
Binary files /dev/null and b/diagrams/retention-new.graffle differ
diff --git a/docs/cookbooks-retention-expiry.md
b/docs/cookbooks-retention-expiry.md
index 67062b72582b..8d7a3ee96477 100644
--- a/docs/cookbooks-retention-expiry.md
+++ b/docs/cookbooks-retention-expiry.md
@@ -37,7 +37,7 @@ Pulsar's [admin interface](admin-api-overview.md) enables you
to manage both ret
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
The diagram below illustrates the concept of message retention.
-
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgments, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
@@ -189,7 +189,7 @@ admin.namespaces().getRetention(namespace);
You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
The diagram below illustrates the concept of backlog quota.
-
+
Setting a backlog quota involves setting:
@@ -204,9 +204,9 @@ The following retention policies are available:
| `producer_exception` | The producer throws an exception when trying
to send a message |
| `consumer_backlog_eviction` | The broker will begin discarding backlog
messages |
-> #### Beware the distinction between retention policy types
-> As you may have noticed, there are two definitions of the term "retention
policy" in Pulsar, one that applies to persistent storage of messages not in
backlogs, and one that applies to messages within backlogs.
-
+:::note
+The scope of retention includes the backlog, so retention must be greater than
the backlog quota. Otherwise, Pulsar will throw an error when both are set.
+:::
Backlog quotas are handled at the namespace level. They can be managed via:
diff --git a/static/assets/backlog-quota-new.svg
b/static/assets/backlog-quota-new.svg
new file mode 100644
index 000000000000..aac00dc87b62
--- /dev/null
+++ b/static/assets/backlog-quota-new.svg
@@ -0,0 +1,262 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd";>
+<svg xmlns:dc="http://purl.org/dc/elements/1.1/";
xmlns="http://www.w3.org/2000/svg"; version="1.1"
xmlns:xl="http://www.w3.org/1999/xlink"; viewBox="-258 670 1150 580"
width="1150" height="580">
+ <defs>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="DimensionArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-1 -5 8 10" markerWidth="8" markerHeight="10" color="black">
+ <g>
+ <path d="M 0 0 L 5.866667 0 M 5.866667 3.6666667 L 5.866667 -3.6666667
M 0 1.4666667 L 5.1333333 0 L 0 -1.4666667" fill="none" stroke="currentColor"
stroke-width="1"/>
+ </g>
+ </marker>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="DimensionArrow_Marker_2" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-7 -5 8 10" markerWidth="8" markerHeight="10" color="black">
+ <g>
+ <path d="M 0 0 L -5.866667 0 M -5.866667 -3.6666667 L -5.866667
3.6666667 M 0 -1.4666667 L -5.1333333 0 L 0 1.4666667" fill="none"
stroke="currentColor" stroke-width="1"/>
+ </g>
+ </marker>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-1 -4 8 8" markerWidth="8" markerHeight="8" color="black">
+ <g>
+ <path d="M 5.866667 0 L 0 -2.2 L 0 2.2 Z" fill="currentColor"
stroke="currentColor" stroke-width="1"/>
+ </g>
+ </marker>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="DimensionArrow_Marker_3" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-1 -5 8 10" markerWidth="8" markerHeight="10" color="#b34400">
+ <g>
+ <path d="M 0 0 L 5.866667 0 M 5.866667 3.6666667 L 5.866667 -3.6666667
M 0 1.4666667 L 5.1333333 0 L 0 -1.4666667" fill="none" stroke="currentColor"
stroke-width="1"/>
+ </g>
+ </marker>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="DimensionArrow_Marker_4" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-7 -5 8 10" markerWidth="8" markerHeight="10" color="#b34400">
+ <g>
+ <path d="M 0 0 L -5.866667 0 M -5.866667 -3.6666667 L -5.866667
3.6666667 M 0 -1.4666667 L -5.1333333 0 L 0 1.4666667" fill="none"
stroke="currentColor" stroke-width="1"/>
+ </g>
+ </marker>
+ </defs>
+ <g id="Canvas_9" fill="none" stroke-opacity="1" fill-opacity="1"
stroke="none" stroke-dasharray="none">
+ <title>Canvas 9</title>
+ <rect fill="white" x="-258" y="670" width="1150" height="580"/>
+ <g id="Canvas_9_Layer_1">
+ <title>Layer 1</title>
+ <g id="Graphic_39">
+ <rect x="803.1791" y="962.2902" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="803.1791" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(808.1791 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">0</tspan>
+ </text>
+ </g>
+ <g id="Graphic_38">
+ <rect x="758.1791" y="962.2902" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="758.1791" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(763.1791 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">1</tspan>
+ </text>
+ </g>
+ <g id="Graphic_37">
+ <rect x="713.1791" y="962.2902" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="713.1791" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(718.1791 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">2</tspan>
+ </text>
+ </g>
+ <g id="Graphic_36">
+ <rect x="668.17914" y="962.2902" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="668.17914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(673.17914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">3</tspan>
+ </text>
+ </g>
+ <g id="Graphic_35">
+ <rect x="623.17914" y="962.2902" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="623.17914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(628.17914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">4</tspan>
+ </text>
+ </g>
+ <g id="Graphic_34">
+ <rect x="578.17914" y="962.2902" width="45" height="57.552"
fill="#1790ff"/>
+ <rect x="578.17914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(583.17914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">5</tspan>
+ </text>
+ </g>
+ <g id="Graphic_33">
+ <rect x="532.92914" y="962.2902" width="45" height="57.552"
fill="#1790ff"/>
+ <rect x="532.92914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(537.92914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">6</tspan>
+ </text>
+ </g>
+ <g id="Graphic_32">
+ <rect x="487.92914" y="962.2902" width="45" height="57.552"
fill="#1790ff"/>
+ <rect x="487.92914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(492.92914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">7</tspan>
+ </text>
+ </g>
+ <g id="Graphic_31">
+ <rect x="442.92914" y="962.2902" width="45" height="57.552"
fill="#1790ff"/>
+ <rect x="442.92914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(447.92914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">8</tspan>
+ </text>
+ </g>
+ <g id="Graphic_30">
+ <rect x="397.92914" y="962.2902" width="45" height="57.552"
fill="white"/>
+ <rect x="397.92914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(402.92914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">9</tspan>
+ </text>
+ </g>
+ <g id="Graphic_29">
+ <rect x="352.92914" y="962.2902" width="45" height="57.552"
fill="white"/>
+ <rect x="352.92914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(357.92914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">10</tspan>
+ </text>
+ </g>
+ <g id="Graphic_28">
+ <rect x="307.92914" y="962.2902" width="45" height="57.552"
fill="white"/>
+ <rect x="307.92914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(312.92914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">11</tspan>
+ </text>
+ </g>
+ <g id="Graphic_27">
+ <rect x="262.92914" y="962.2902" width="45" height="57.552"
fill="white"/>
+ <rect x="262.92914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(267.92914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">12</tspan>
+ </text>
+ </g>
+ <g id="Graphic_26">
+ <rect x="217.67914" y="962.2902" width="45" height="57.552"
fill="white"/>
+ <rect x="217.67914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(222.67914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">13</tspan>
+ </text>
+ </g>
+ <g id="Graphic_25">
+ <rect x="172.67914" y="962.2902" width="45" height="57.552"
fill="white"/>
+ <rect x="172.67914" y="962.2902" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(177.67914 981.8422)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">14</tspan>
+ </text>
+ </g>
+ <g id="Line_24">
+ <line x1="181.47904" y1="938.2466" x2="433.8749" y2="939.4692"
marker-end="url(#DimensionArrow_Marker)"
marker-start="url(#DimensionArrow_Marker_2)" stroke="black"
stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_23">
+ <rect x="220.96978" y="917.3867" width="140.69" height="42.783997"
fill="white"/>
+ <text transform="translate(225.96978 922.3867)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="14" fill="black"
x="39.809" y="13">Backlog</tspan>
+ <tspan font-family="Helvetica Neue" font-size="14" fill="black"
x="0" y="29.392">(Unacked messages)</tspan>
+ </text>
+ </g>
+ <g id="Line_22">
+ <line x1="451.47476" y1="939.5239" x2="839.7872" y2="940.0569"
marker-end="url(#DimensionArrow_Marker)"
marker-start="url(#DimensionArrow_Marker_2)" stroke="black"
stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_21">
+ <rect x="591.19746" y="926.5901" width="102.582" height="26.391998"
fill="white"/>
+ <text transform="translate(596.19746 931.5901)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="14" fill="black"
x="0" y="13">Ackd message</tspan>
+ </text>
+ </g>
+ <g id="Line_20">
+ <line x1="441.7328" y1="866.396" x2="442.7823" y2="947.8569"
marker-end="url(#FilledArrow_Marker)" stroke="black" stroke-linecap="round"
stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_19">
+ <path d="M 391.20414 820.256 L 491.65414 820.256 C 503.53594 820.256
513.17914 830.3674 513.17914 842.826 C 513.17914 855.2846 503.53594 865.396
491.65414 865.396 L 391.20414 865.396 C 379.32234 865.396 369.67914 855.2846
369.67914 842.826 C 369.67914 830.3674 379.32234 820.256 391.20414 820.256 Z"
fill="white"/>
+ <path d="M 391.20414 820.256 L 491.65414 820.256 C 503.53594 820.256
513.17914 830.3674 513.17914 842.826 C 513.17914 855.2846 503.53594 865.396
491.65414 865.396 L 391.20414 865.396 C 379.32234 865.396 369.67914 855.2846
369.67914 842.826 C 369.67914 830.3674 379.32234 820.256 391.20414 820.256 Z"
stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"/>
+ <text transform="translate(389.02914 836.686)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="10" fill="black"
x="24.245002" y="10">Subsctiption</tspan>
+ </text>
+ </g>
+ <g id="Line_18">
+ <line x1="181.47914" y1="1039.2671" x2="614.78715" y2="1039.8163"
marker-end="url(#DimensionArrow_Marker_3)"
marker-start="url(#DimensionArrow_Marker_4)" stroke="#b34400"
stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_17">
+ <rect x="341.239" y="1026.3412" width="106.712" height="26.391998"
fill="white"/>
+ <text transform="translate(346.239 1031.3412)" fill="#b34400">
+ <tspan font-family="Helvetica Neue" font-size="14" fill="#b34400"
x="0" y="13">Retained policy</tspan>
+ </text>
+ </g>
+ <g id="Line_16">
+ <line x1="632.3867" y1="1039.738" x2="839.7876" y2="1037.6311"
marker-end="url(#DimensionArrow_Marker)"
marker-start="url(#DimensionArrow_Marker_2)" stroke="black"
stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_15">
+ <rect x="652.8256" y="1025.5065" width="162.992" height="26.391998"
fill="white"/>
+ <text transform="translate(657.8256 1030.5065)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="14" fill="black"
x="0" y="13">Will be deleted message</tspan>
+ </text>
+ </g>
+ <g id="Graphic_14">
+ <path d="M -172.3954 965.91 L -71.94539 965.91 C -60.06359 965.91
-50.420393 976.0214 -50.420393 988.48 C -50.420393 1000.9386 -60.06359 1011.05
-71.94539 1011.05 L -172.3954 1011.05 C -184.2772 1011.05 -193.9204 1000.9386
-193.9204 988.48 C -193.9204 976.0214 -184.2772 965.91 -172.3954 965.91 Z"
fill="white"/>
+ <path d="M -172.3954 965.91 L -71.94539 965.91 C -60.06359 965.91
-50.420393 976.0214 -50.420393 988.48 C -50.420393 1000.9386 -60.06359 1011.05
-71.94539 1011.05 L -172.3954 1011.05 C -184.2772 1011.05 -193.9204 1000.9386
-193.9204 988.48 C -193.9204 976.0214 -184.2772 965.91 -172.3954 965.91 Z"
stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"/>
+ <text transform="translate(-174.5704 979.256)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="19.656002" y="15">Producer</tspan>
+ </text>
+ </g>
+ <g id="Graphic_13">
+ <rect x="417.59364" y="890.8036" width="49.234" height="25.363998"
fill="white"/>
+ <text transform="translate(422.59364 895.8036)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="13" fill="black"
x="18829382e-20" y="12">Cursor</tspan>
+ </text>
+ </g>
+ <g id="Graphic_12">
+ <rect x="262.92914" y="1129.312" width="59.5" height="22.307999"
fill="#1790ff"/>
+ <rect x="262.92914" y="1129.312" width="59.5" height="22.307999"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_11">
+ <rect x="262.92914" y="1167.896" width="59.5" height="22"
fill="#d3e3fd"/>
+ <rect x="262.92914" y="1167.896" width="59.5" height="22"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_10">
+ <text transform="translate(336.1409 1129.714)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="2.224" y="15">Acked messages within retention limit (not ready for deletion)
</tspan>
+ </text>
+ </g>
+ <g id="Graphic_9">
+ <text transform="translate(340.42914 1169.672)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="0" y="15">Acked messages over retention limit (ready for deletion)</tspan>
+ </text>
+ </g>
+ <g id="Graphic_8">
+ <rect x="49.37514" y="959.704" width="99.804" height="57.552"
fill="white"/>
+ <rect x="49.37514" y="959.704" width="99.804" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(54.37514 972.088)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="14" fill="black"
x="25.463" y="13">A new </tspan>
+ <tspan font-family="Helvetica Neue" font-size="14" fill="black"
x="16.636" y="29.392">message</tspan>
+ </text>
+ </g>
+ <g id="Graphic_7">
+ <path d="M -27.616858 995.31 L -27.616858 981.65 L 10.704667 981.65 L
10.704667 974.256 L 31.883142 988.48 L 10.704667 1002.704 L 10.704667 995.31 Z"
fill="#666"/>
+ <path d="M -27.616858 995.31 L -27.616858 981.65 L 10.704667 981.65 L
10.704667 974.256 L 31.883142 988.48 L 10.704667 1002.704 L 10.704667 995.31 Z"
stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ </g>
+ <g id="Graphic_6">
+ <rect x="-194.5" y="778.5" width="292.328" height="57.552"
fill="white"/>
+ <path d="M -194.5 778.5 L 97.828 778.5 L 97.828 836.052 L -194.5
836.052 Z" stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ <text transform="translate(-189.5 788.8409)" fill="#3c4053">
+ <tspan font-family="Helvetica Neue" font-weight="bold"
font-size="15" fill="#3c4053" x="0" y="15">Assume a message is 1MB</tspan>
+ <tspan font-family="Helvetica Neue" font-weight="bold"
font-size="15" fill="#3c4053" x="0" y="33.435104">Retention Policy: {size: 10MB
Time: 0}</tspan>
+ </text>
+ </g>
+ <g id="Graphic_5">
+ <path d="M 84.20031 1107.9678 L 100.12536 1030.3121 L 27.037475
1107.9678 L -179.5 1107.9678 C -187.78427 1107.9678 -194.5 1114.6836 -194.5
1122.9678 L -194.5 1175.1665 C -194.5 1183.4508 -187.78427 1190.1665 -179.5
1190.1665 L 169.30354 1190.1665 C 177.5878 1190.1665 184.30354 1183.4508
184.30354 1175.1665 L 184.30354 1122.9678 C 184.30354 1114.6836 177.5878
1107.9678 169.30354 1107.9678 Z" fill="white"/>
+ <path d="M 84.20031 1107.9678 L 100.12536 1030.3121 L 27.037475
1107.9678 L -179.5 1107.9678 C -187.78427 1107.9678 -194.5 1114.6836 -194.5
1122.9678 L -194.5 1175.1665 C -194.5 1183.4508 -187.78427 1190.1665 -179.5
1190.1665 L 169.30354 1190.1665 C 177.5878 1190.1665 184.30354 1183.4508
184.30354 1175.1665 L 184.30354 1122.9678 C 184.30354 1114.6836 177.5878
1107.9678 169.30354 1107.9678 Z" stroke="#b1001c" stroke-linecap="round"
stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_4">
+ <text transform="translate(-184.11686 1130.594)" fill="#b1001c">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="#b1001c"
x="0" y="15">The new message is over the backlog quota limit. </tspan>
+ <tspan font-family="Helvetica Neue" font-size="16" fill="#b1001c"
x="0" y="33.448">Pulsar applies backlog retention policy here.</tspan>
+ </text>
+ </g>
+ <g id="Graphic_3">
+ <path d="M 266.11135 799.5 L 270.211 911.1893 L 306.7213 799.5 L
382.1146 799.5 C 390.39886 799.5 397.1146 792.7843 397.1146 784.5 L 397.1146
732 C 397.1146 723.7157 390.39886 717 382.1146 717 L 175.88314 717 C 167.59887
717 160.88314 723.7157 160.88314 732 L 160.88314 784.5 C 160.88314 792.7843
167.59887 799.5 175.88314 799.5 Z" fill="white"/>
+ <path d="M 266.11135 799.5 L 270.211 911.1893 L 306.7213 799.5 L
382.1146 799.5 C 390.39886 799.5 397.1146 792.7843 397.1146 784.5 L 397.1146
732 C 397.1146 723.7157 390.39886 717 382.1146 717 L 175.88314 717 C 167.59887
717 160.88314 723.7157 160.88314 732 L 160.88314 784.5 C 160.88314 792.7843
167.59887 799.5 175.88314 799.5 Z" stroke="black" stroke-linecap="round"
stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_2">
+ <text transform="translate(169.67078 731.5)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="10.170364" y="15">Unacked messages within </tspan>
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.970364" y="33.448">the backlog quota limit are </tspan>
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="35.642364" y="51.895996">kept in the backlog</tspan>
+ </text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/static/assets/retention-new.svg b/static/assets/retention-new.svg
new file mode 100644
index 000000000000..d0fc70f96961
--- /dev/null
+++ b/static/assets/retention-new.svg
@@ -0,0 +1,233 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd";>
+<svg xmlns:dc="http://purl.org/dc/elements/1.1/";
xmlns="http://www.w3.org/2000/svg"; version="1.1"
xmlns:xl="http://www.w3.org/1999/xlink"; viewBox="-377 761 1e3 450" width="1000"
height="450">
+ <defs>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="DimensionArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-1 -5 8 10" markerWidth="8" markerHeight="10" color="black">
+ <g>
+ <path d="M 0 0 L 5.866667 0 M 5.866667 3.6666667 L 5.866667 -3.6666667
M 0 1.4666667 L 5.1333333 0 L 0 -1.4666667" fill="none" stroke="currentColor"
stroke-width="1"/>
+ </g>
+ </marker>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="DimensionArrow_Marker_2" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-7 -5 8 10" markerWidth="8" markerHeight="10" color="black">
+ <g>
+ <path d="M 0 0 L -5.866667 0 M -5.866667 -3.6666667 L -5.866667
3.6666667 M 0 -1.4666667 L -5.1333333 0 L 0 1.4666667" fill="none"
stroke="currentColor" stroke-width="1"/>
+ </g>
+ </marker>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="FilledArrow_Marker" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-1 -4 8 8" markerWidth="8" markerHeight="8" color="black">
+ <g>
+ <path d="M 5.866667 0 L 0 -2.2 L 0 2.2 Z" fill="currentColor"
stroke="currentColor" stroke-width="1"/>
+ </g>
+ </marker>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="DimensionArrow_Marker_3" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-1 -5 8 10" markerWidth="8" markerHeight="10" color="#b34400">
+ <g>
+ <path d="M 0 0 L 5.866667 0 M 5.866667 3.6666667 L 5.866667 -3.6666667
M 0 1.4666667 L 5.1333333 0 L 0 -1.4666667" fill="none" stroke="currentColor"
stroke-width="1"/>
+ </g>
+ </marker>
+ <marker orient="auto" overflow="visible" markerUnits="strokeWidth"
id="DimensionArrow_Marker_4" stroke-linejoin="miter" stroke-miterlimit="10"
viewBox="-7 -5 8 10" markerWidth="8" markerHeight="10" color="#b34400">
+ <g>
+ <path d="M 0 0 L -5.866667 0 M -5.866667 -3.6666667 L -5.866667
3.6666667 M 0 -1.4666667 L -5.1333333 0 L 0 1.4666667" fill="none"
stroke="currentColor" stroke-width="1"/>
+ </g>
+ </marker>
+ </defs>
+ <g id="Canvas_8" fill="none" stroke-opacity="1" fill-opacity="1"
stroke="none" stroke-dasharray="none">
+ <title>Canvas 8</title>
+ <rect fill="white" x="-377" y="761" width="1e3" height="450"/>
+ <g id="Canvas_8_Layer_1">
+ <title>Layer 1</title>
+ <g id="Graphic_34">
+ <rect x="521.546" y="965.9462" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="521.546" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(526.546 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">0</tspan>
+ </text>
+ </g>
+ <g id="Graphic_33">
+ <rect x="476.546" y="965.9462" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="476.546" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(481.546 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">1</tspan>
+ </text>
+ </g>
+ <g id="Graphic_32">
+ <rect x="431.546" y="965.9462" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="431.546" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(436.546 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">2</tspan>
+ </text>
+ </g>
+ <g id="Graphic_31">
+ <rect x="386.546" y="965.9462" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="386.546" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(391.546 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">3</tspan>
+ </text>
+ </g>
+ <g id="Graphic_30">
+ <rect x="341.546" y="965.9462" width="45" height="57.552"
fill="#d3e3fd"/>
+ <rect x="341.546" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(346.546 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">4</tspan>
+ </text>
+ </g>
+ <g id="Graphic_29">
+ <rect x="296.546" y="965.9462" width="45" height="57.552"
fill="#1790ff"/>
+ <rect x="296.546" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="1"/>
+ <text transform="translate(301.546 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">5</tspan>
+ </text>
+ </g>
+ <g id="Graphic_28">
+ <rect x="251.296" y="965.9462" width="45" height="57.552"
fill="#1790ff"/>
+ <rect x="251.296" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(256.296 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">6</tspan>
+ </text>
+ </g>
+ <g id="Graphic_27">
+ <rect x="206.296" y="965.9462" width="45" height="57.552"
fill="#1790ff"/>
+ <rect x="206.296" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(211.296 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">7</tspan>
+ </text>
+ </g>
+ <g id="Graphic_26">
+ <rect x="161.296" y="965.9462" width="45" height="57.552"
fill="#1790ff"/>
+ <rect x="161.296" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(166.296 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">8</tspan>
+ </text>
+ </g>
+ <g id="Graphic_25">
+ <rect x="116.296" y="965.9462" width="45" height="57.552"
fill="white"/>
+ <rect x="116.296" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(121.296 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="13.052" y="15">9</tspan>
+ </text>
+ </g>
+ <g id="Graphic_24">
+ <rect x="71.296" y="965.9462" width="45" height="57.552" fill="white"/>
+ <rect x="71.296" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(76.296 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">10</tspan>
+ </text>
+ </g>
+ <g id="Graphic_23">
+ <rect x="26.296" y="965.9462" width="45" height="57.552" fill="white"/>
+ <rect x="26.296" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(31.296 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">11</tspan>
+ </text>
+ </g>
+ <g id="Graphic_22">
+ <rect x="-18.704" y="965.9462" width="45" height="57.552"
fill="white"/>
+ <rect x="-18.704" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(-13.704 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">12</tspan>
+ </text>
+ </g>
+ <g id="Graphic_21">
+ <rect x="-63.954" y="965.9462" width="45" height="57.552"
fill="white"/>
+ <rect x="-63.954" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(-58.954 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">13</tspan>
+ </text>
+ </g>
+ <g id="Graphic_20">
+ <rect x="-108.954" y="965.9462" width="45" height="57.552"
fill="white"/>
+ <rect x="-108.954" y="965.9462" width="45" height="57.552"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ <text transform="translate(-103.954 985.4982)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="8.604" y="15">14</tspan>
+ </text>
+ </g>
+ <g id="Line_19">
+ <line x1="-100.1541" y1="941.9026" x2="152.24173" y2="943.1252"
marker-end="url(#DimensionArrow_Marker)"
marker-start="url(#DimensionArrow_Marker_2)" stroke="black"
stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_18">
+ <rect x="-51.087277" y="920.0837" width="150.025" height="44.839996"
fill="white"/>
+ <text transform="translate(-46.08728 925.0837)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="15" fill="black"
x="42.6525" y="14">Backlog</tspan>
+ <tspan font-family="Helvetica Neue" font-size="15" fill="black"
x="73896445e-20" y="31.42">(Unacked messages)</tspan>
+ </text>
+ </g>
+ <g id="Line_17">
+ <line x1="169.84161" y1="943.1499" x2="557.746" y2="942.3585"
marker-end="url(#DimensionArrow_Marker)"
marker-start="url(#DimensionArrow_Marker_2)" stroke="black"
stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_16">
+ <rect x="306.05698" y="929.0506" width="109.195" height="27.419998"
fill="white"/>
+ <text transform="translate(311.05698 934.0506)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="15" fill="black"
x="0" y="14">Ackd message</tspan>
+ </text>
+ </g>
+ <g id="Line_15">
+ <line x1="160.09966" y1="870.052" x2="161.14914" y2="951.513"
marker-end="url(#FilledArrow_Marker)" stroke="black" stroke-linecap="round"
stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_14">
+ <path d="M 109.571 823.912 L 210.021 823.912 C 221.9028 823.912
231.546 834.0234 231.546 846.482 C 231.546 858.9406 221.9028 869.052 210.021
869.052 L 109.571 869.052 C 97.6892 869.052 88.046 858.9406 88.046 846.482 C
88.046 834.0234 97.6892 823.912 109.571 823.912 Z" fill="white"/>
+ <path d="M 109.571 823.912 L 210.021 823.912 C 221.9028 823.912
231.546 834.0234 231.546 846.482 C 231.546 858.9406 221.9028 869.052 210.021
869.052 L 109.571 869.052 C 97.6892 869.052 88.046 858.9406 88.046 846.482 C
88.046 834.0234 97.6892 823.912 109.571 823.912 Z" stroke="black"
stroke-linecap="round" stroke-linejoin="round" stroke-width="2"/>
+ <text transform="translate(107.396 840.342)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="10" fill="black"
x="24.245002" y="10">Subsctiption</tspan>
+ </text>
+ </g>
+ <g id="Line_13">
+ <line x1="-100.15401" y1="1042.9232" x2="333.154" y2="1043.4723"
marker-end="url(#DimensionArrow_Marker_3)"
marker-start="url(#DimensionArrow_Marker_4)" stroke="#b34400"
stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_12">
+ <rect x="56.151844" y="1029.4832" width="113.62" height="27.419998"
fill="white"/>
+ <text transform="translate(61.151845 1034.4832)" fill="#b34400">
+ <tspan font-family="Helvetica Neue" font-size="15" fill="#b34400"
x="0" y="14">Retained policy</tspan>
+ </text>
+ </g>
+ <g id="Line_11">
+ <line x1="350.75355" y1="1043.394" x2="558.15445" y2="1041.2871"
marker-end="url(#DimensionArrow_Marker)"
marker-start="url(#DimensionArrow_Marker_2)" stroke="black"
stroke-linecap="round" stroke-linejoin="round" stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_10">
+ <rect x="365.7285" y="1028.6485" width="173.92" height="27.419998"
fill="white"/>
+ <text transform="translate(370.7285 1033.6485)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="15" fill="black"
x="0" y="14">Will be deleted message</tspan>
+ </text>
+ </g>
+ <g id="Graphic_9">
+ <path d="M -302.879 969.566 L -218.529 969.566 C -208.5516 969.566
-200.454 979.6774 -200.454 992.136 C -200.454 1004.5946 -208.5516 1014.706
-218.529 1014.706 L -302.879 1014.706 C -312.8564 1014.706 -320.954 1004.5946
-320.954 992.136 C -320.954 979.6774 -312.8564 969.566 -302.879 969.566 Z"
fill="white"/>
+ <path d="M -302.879 969.566 L -218.529 969.566 C -208.5516 969.566
-200.454 979.6774 -200.454 992.136 C -200.454 1004.5946 -208.5516 1014.706
-218.529 1014.706 L -302.879 1014.706 C -312.8564 1014.706 -320.954 1004.5946
-320.954 992.136 C -320.954 979.6774 -312.8564 969.566 -302.879 969.566 Z"
stroke="black" stroke-linecap="round" stroke-linejoin="round" stroke-width="2"/>
+ <text transform="translate(-303.904 982.912)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="10.456001" y="15">Producer</tspan>
+ </text>
+ </g>
+ <g id="Graphic_8">
+ <rect x="135.9605" y="894.4596" width="49.234" height="25.363998"
fill="white"/>
+ <text transform="translate(140.9605 899.4596)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="13" fill="black"
x="18829382e-20" y="12">Cursor</tspan>
+ </text>
+ </g>
+ <g id="Graphic_7">
+ <rect x="-171.454" y="1109.552" width="59.5" height="22.307999"
fill="#1790ff"/>
+ <rect x="-171.454" y="1109.552" width="59.5" height="22.307999"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_6">
+ <rect x="-171.454" y="1148.136" width="59.5" height="22"
fill="#d3e3fd"/>
+ <rect x="-171.454" y="1148.136" width="59.5" height="22"
stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1.5"/>
+ </g>
+ <g id="Graphic_5">
+ <text transform="translate(-98.24224 1109.954)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="2.224" y="15">Acked messages within retention limit (not ready for deletion)
</tspan>
+ </text>
+ </g>
+ <g id="Graphic_4">
+ <text transform="translate(-93.954 1149.912)" fill="black">
+ <tspan font-family="Helvetica Neue" font-size="16" fill="black"
x="0" y="15">Acked messages over retention limit (ready for deletion)</tspan>
+ </text>
+ </g>
+ <g id="Graphic_3">
+ <path d="M -177.454 998.966 L -177.454 985.306 L -153.63248 985.306 L
-153.63248 977.912 L -132.454 992.136 L -153.63248 1006.36 L -153.63248 998.966
Z" fill="#666"/>
+ <path d="M -177.454 998.966 L -177.454 985.306 L -153.63248 985.306 L
-153.63248 977.912 L -132.454 992.136 L -153.63248 1006.36 L -153.63248 998.966
Z" stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-width="1"/>
+ </g>
+ <g id="Graphic_2">
+ <rect x="-316.954" y="811.5" width="292.328" height="57.552"
fill="white"/>
+ <path d="M -316.954 811.5 L -24.626 811.5 L -24.626 869.052 L -316.954
869.052 Z" stroke="black" stroke-linecap="round" stroke-linejoin="round"
stroke-dasharray="4.0,4.0" stroke-width="1"/>
+ <text transform="translate(-311.954 821.8409)" fill="#3c4053">
+ <tspan font-family="Helvetica Neue" font-weight="bold"
font-size="15" fill="#3c4053" x="0" y="15">Assume a message is 1MB</tspan>
+ <tspan font-family="Helvetica Neue" font-weight="bold"
font-size="15" fill="#3c4053" x="0" y="33.435104">Retention Policy: {size: 10MB
Time: 0}</tspan>
+ </text>
+ </g>
+ </g>
+ </g>
+</svg>
diff --git a/versioned_docs/version-3.0.x/cookbooks-retention-expiry.md
b/versioned_docs/version-3.0.x/cookbooks-retention-expiry.md
index 7c8a0216e3d7..48cc110af6d2 100644
--- a/versioned_docs/version-3.0.x/cookbooks-retention-expiry.md
+++ b/versioned_docs/version-3.0.x/cookbooks-retention-expiry.md
@@ -36,7 +36,7 @@ Pulsar's [admin interface](admin-api-overview.md) enables you
to manage both ret
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
The diagram below illustrates the concept of message retention.
-
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgments, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
@@ -188,7 +188,7 @@ admin.namespaces().getRetention(namespace);
You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
The diagram below illustrates the concept of backlog quota.
-
+
Setting a backlog quota involves setting:
@@ -203,10 +203,9 @@ Policy | Action
`producer_exception` | The producer throws an exception when trying to send a
message
`consumer_backlog_eviction` | The broker will begin discarding backlog messages
-
-> #### Beware the distinction between retention policy types
-> As you may have noticed, there are two definitions of the term "retention
policy" in Pulsar, one that applies to persistent storage of messages not in
backlogs, and one that applies to messages within backlogs.
-
+:::note
+The scope of retention includes the backlog, so retention must be greater than
the backlog quota. Otherwise, Pulsar will throw an error when both are set.
+:::
Backlog quotas are handled at the namespace level. They can be managed via:
diff --git a/versioned_docs/version-3.3.x/cookbooks-retention-expiry.md
b/versioned_docs/version-3.3.x/cookbooks-retention-expiry.md
index df46ab3e05fd..b0ad2535aec5 100644
--- a/versioned_docs/version-3.3.x/cookbooks-retention-expiry.md
+++ b/versioned_docs/version-3.3.x/cookbooks-retention-expiry.md
@@ -37,7 +37,7 @@ Pulsar's [admin interface](admin-api-overview.md) enables you
to manage both ret
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
The diagram below illustrates the concept of message retention.
-
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgments, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
@@ -189,7 +189,7 @@ admin.namespaces().getRetention(namespace);
You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
The diagram below illustrates the concept of backlog quota.
-
+
Setting a backlog quota involves setting:
@@ -204,9 +204,9 @@ The following retention policies are available:
| `producer_exception` | The producer throws an exception when trying
to send a message |
| `consumer_backlog_eviction` | The broker will begin discarding backlog
messages |
-> #### Beware the distinction between retention policy types
-> As you may have noticed, there are two definitions of the term "retention
policy" in Pulsar, one that applies to persistent storage of messages not in
backlogs, and one that applies to messages within backlogs.
-
+:::note
+The scope of retention includes the backlog, so retention must be greater than
the backlog quota. Otherwise, Pulsar will throw an error when both are set.
+:::
Backlog quotas are handled at the namespace level. They can be managed via:
diff --git a/versioned_docs/version-4.0.x/cookbooks-retention-expiry.md
b/versioned_docs/version-4.0.x/cookbooks-retention-expiry.md
index 67062b72582b..8d7a3ee96477 100644
--- a/versioned_docs/version-4.0.x/cookbooks-retention-expiry.md
+++ b/versioned_docs/version-4.0.x/cookbooks-retention-expiry.md
@@ -37,7 +37,7 @@ Pulsar's [admin interface](admin-api-overview.md) enables you
to manage both ret
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
The diagram below illustrates the concept of message retention.
-
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgments, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
@@ -189,7 +189,7 @@ admin.namespaces().getRetention(namespace);
You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
The diagram below illustrates the concept of backlog quota.
-
+
Setting a backlog quota involves setting:
@@ -204,9 +204,9 @@ The following retention policies are available:
| `producer_exception` | The producer throws an exception when trying
to send a message |
| `consumer_backlog_eviction` | The broker will begin discarding backlog
messages |
-> #### Beware the distinction between retention policy types
-> As you may have noticed, there are two definitions of the term "retention
policy" in Pulsar, one that applies to persistent storage of messages not in
backlogs, and one that applies to messages within backlogs.
-
+:::note
+The scope of retention includes the backlog, so retention must be greater than
the backlog quota. Otherwise, Pulsar will throw an error when both are set.
+:::
Backlog quotas are handled at the namespace level. They can be managed via:
