dawidwys commented on a change in pull request #34:
URL: https://github.com/apache/flink-benchmarks/pull/34#discussion_r724223665
##########
File path:
src/main/java/org/apache/flink/benchmark/CheckpointingTimeBenchmark.java
##########
@@ -65,37 +61,20 @@
import java.util.concurrent.Executors;
import java.util.function.Function;
-import static java.util.concurrent.TimeUnit.MILLISECONDS;
import static java.util.concurrent.TimeUnit.SECONDS;
import static
org.apache.flink.api.common.eventtime.WatermarkStrategy.noWatermarks;
/**
* The test verifies that the debloating kicks in and properly downsizes
buffers. In the end the
* checkpoint should take ~2(number of rebalance) * DEBLOATING_TARGET.
- *
- * <p>Some info about the chosen numbers:
- *
- * <ul>
- * <li>The minimal memory segment size is decreased (256b) so that the
scaling possibility is
- * higher. Memory segments start with 4kb
- * <li>A memory segment of the minimal size fits ~3 records (of size 64b),
each record takes ~1ms
- * to be processed by the sink
- * <li>We have 2 (exclusive buffers) * 4 (parallelism) + 8 floating = 64
buffers per gate, with
- * 300 ms debloating target and ~1ms/record processing speed, we can
buffer 300/64 = ~4.5
- * records in a buffer after debloating which means the size of a buffer
is slightly above the
- * minimal memory segment size.
- * <li>The buffer debloating target of 300ms means a checkpoint should take
~2(number of
- * exchanges)*300ms=~600ms
- * </ul>
*/
@OutputTimeUnit(SECONDS)
-@Warmup(iterations = 4)
public class CheckpointingTimeBenchmark extends BenchmarkBase {
public static final int JOB_PARALLELISM = 4;
- public static final MemorySize START_MEMORY_SEGMENT_SIZE =
MemorySize.parse("4 kb");
Review comment:
I increased the memory segment size to increase the range in which
buffer debloating works. After debloating, fully backpressured pipeline has
buffers of `~1000-2000b`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
