[GitHub] [kafka] Hangleton commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…
Hangleton commented on code in PR #13584:
URL: https://github.com/apache/kafka/pull/13584#discussion_r1212777323
##
core/src/test/scala/unit/kafka/log/LogSegmentTest.scala:
##
@@ -65,6 +68,32 @@ class LogSegmentTest {
Utils.delete(logDir)
}
+ /**
+ * LogSegmentOffsetOverflowException should be thrown while appending the
logs if:
+ * 1. largestOffset < 0
+ * 2. largestOffset - baseOffset < 0
+ * 3. largestOffset - baseOffset > Integer.MAX_VALUE
+ */
+ @ParameterizedTest
+ @CsvSource(Array(
+"0, -2147483648",
+"0, 2147483648",
+"1, 0",
+"100, 10",
+"2147483648, 0",
+"-2147483648, 0",
+"2147483648,4294967296"
+ ))
+ def testAppendForLogSegmentOffsetOverflowException(baseOffset: Long,
largestOffset: Long): Unit = {
Review Comment:
Got it, thanks for confirming!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] Hangleton commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…
Hangleton commented on code in PR #13584:
URL: https://github.com/apache/kafka/pull/13584#discussion_r1191598672
##
core/src/test/scala/unit/kafka/log/LogSegmentTest.scala:
##
@@ -65,6 +68,32 @@ class LogSegmentTest {
Utils.delete(logDir)
}
+ /**
+ * LogSegmentOffsetOverflowException should be thrown while appending the
logs if:
+ * 1. largestOffset < 0
+ * 2. largestOffset - baseOffset < 0
+ * 3. largestOffset - baseOffset > Integer.MAX_VALUE
+ */
+ @ParameterizedTest
+ @CsvSource(Array(
+"0, -2147483648",
+"0, 2147483648",
+"1, 0",
+"100, 10",
+"2147483648, 0",
+"-2147483648, 0",
+"2147483648,4294967296"
+ ))
+ def testAppendForLogSegmentOffsetOverflowException(baseOffset: Long,
largestOffset: Long): Unit = {
Review Comment:
I see, thanks for checking. Could we maybe confirm if the overflow exception
comes from this line in `TimeIndex#maybeAppend`:
```
mmap.putInt(relativeOffset(offset))
```
And if that is the case, why is the relative offset < 0 or > max_int?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] Hangleton commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…
Hangleton commented on code in PR #13584:
URL: https://github.com/apache/kafka/pull/13584#discussion_r1191598230
##
core/src/test/scala/unit/kafka/log/LogSegmentTest.scala:
##
@@ -65,6 +68,32 @@ class LogSegmentTest {
Utils.delete(logDir)
}
+ /**
+ * LogSegmentOffsetOverflowException should be thrown while appending the
logs if:
+ * 1. largestOffset < 0
+ * 2. largestOffset - baseOffset < 0
+ * 3. largestOffset - baseOffset > Integer.MAX_VALUE
+ */
+ @ParameterizedTest
+ @CsvSource(Array(
+"0, -2147483648",
+"0, 2147483648",
+"1, 0",
+"100, 10",
+"2147483648, 0",
+"-2147483648, 0",
+"2147483648,4294967296"
+ ))
+ def testAppendForLogSegmentOffsetOverflowException(baseOffset: Long,
largestOffset: Long): Unit = {
Review Comment:
I see, thanks for checking. Could we maybe confirm if the overflow exception
comes from this line in `TimeIndex#maybeAppend`:
```
mmap.putInt(relativeOffset(offset))
```
And if that is the case, why is the relative offset < 0 or > max_int?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] Hangleton commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…
Hangleton commented on code in PR #13584:
URL: https://github.com/apache/kafka/pull/13584#discussion_r1188635591
##
core/src/test/scala/unit/kafka/log/LogSegmentTest.scala:
##
@@ -65,6 +68,32 @@ class LogSegmentTest {
Utils.delete(logDir)
}
+ /**
+ * LogSegmentOffsetOverflowException should be thrown while appending the
logs if:
+ * 1. largestOffset < 0
Review Comment:
Nit: we can remove (1) since (1) => (2).
##
core/src/test/scala/unit/kafka/log/LogSegmentTest.scala:
##
@@ -65,6 +68,32 @@ class LogSegmentTest {
Utils.delete(logDir)
}
+ /**
+ * LogSegmentOffsetOverflowException should be thrown while appending the
logs if:
+ * 1. largestOffset < 0
+ * 2. largestOffset - baseOffset < 0
+ * 3. largestOffset - baseOffset > Integer.MAX_VALUE
+ */
+ @ParameterizedTest
+ @CsvSource(Array(
+"0, -2147483648",
+"0, 2147483648",
+"1, 0",
+"100, 10",
+"2147483648, 0",
+"-2147483648, 0",
+"2147483648,4294967296"
Review Comment:
nit: Note that the max int value on the Java platform is `2147483647` so we
are exercising max int + 1.
##
core/src/test/scala/unit/kafka/log/LogSegmentTest.scala:
##
@@ -65,6 +68,32 @@ class LogSegmentTest {
Utils.delete(logDir)
}
+ /**
+ * LogSegmentOffsetOverflowException should be thrown while appending the
logs if:
+ * 1. largestOffset < 0
+ * 2. largestOffset - baseOffset < 0
+ * 3. largestOffset - baseOffset > Integer.MAX_VALUE
+ */
+ @ParameterizedTest
+ @CsvSource(Array(
+"0, -2147483648",
+"0, 2147483648",
+"1, 0",
+"100, 10",
+"2147483648, 0",
+"-2147483648, 0",
+"2147483648,4294967296"
+ ))
+ def testAppendForLogSegmentOffsetOverflowException(baseOffset: Long,
largestOffset: Long): Unit = {
Review Comment:
What is the expected behaviour for `baseOffset == largestOffset`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [kafka] Hangleton commented on a diff in pull request #13584: MINOR: Add log segment unit tests, If the maximum offset beyond index, appen…
Hangleton commented on code in PR #13584:
URL: https://github.com/apache/kafka/pull/13584#discussion_r1168535982
##
core/src/test/scala/unit/kafka/log/LogSegmentTest.scala:
##
@@ -65,6 +66,21 @@ class LogSegmentTest {
Utils.delete(logDir)
}
+ /**
+ * If the maximum offset beyond index, appended to the log section, it
throws LogSegmentOffsetOverflowException
+ */
+ @Test
+ def testAppendForLogSegmentOffsetOverflowException(): Unit = {
+val seg = createSegment(0)
+val largestOffset: Long = Integer.MAX_VALUE + 1
+val largestTimestamp: Long = Time.SYSTEM.milliseconds()
+val shallowOffsetOfMaxTimestamp: Long = 0
+val memoryRecords: MemoryRecords = records(0, "hello")
+assertThrows(classOf[LogSegmentOffsetOverflowException], () => {
+ seg.append(largestOffset, largestTimestamp, shallowOffsetOfMaxTimestamp,
memoryRecords)
Review Comment:
nit: `largestTimestamp` could be misleading as it could make believe the
timestamp itself is set to the maximum possible value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
