ClownfishYang commented on a change in pull request #394: URL: https://github.com/apache/flink-web/pull/394#discussion_r525725073
########## File path: contributing/code-style-and-quality-common.zh.md ########## @@ -137,186 +137,182 @@ the main path ``` -## 4. Design and Structure +## 4. 设计和结构 -While it is hard to exactly specify what constitutes a good design, there are some properties that can serve as a _litmus test_ for a good design. If these properties are given, the chances are good that the design is going into a good direction. If these properties cannot be achieved, there is a high probability that the design is flawed. +虽然很难确切地指定一个好的设计是由什么构成的，但是有一些属性可以作为好的设计的试金石。如果设计上拥有这些属性，那么就有可能得到好的发展。否则，设计就很有可能存在缺陷。 -### Immutability and Eager Initialization +### 不变性（Immutability）和急切初始化（Eager Initialization） -1. Try to use immutable types where possible, especially for APIs, messages, identifiers, properties, configuration, etc. -2. A good general approach is to try and make as many fields of a class `final` as possible. -3. Classes that are used as keys in maps should be strictly immutable and only have `final` fields (except maybe auxiliary fields, like lazy cached hash codes). -4. Eagerly initialize classes. There should be no `init()` or `setup()` methods. Once the constructor completes, the object should be usable. +1. 尽可能尝试使用不可变类型，尤其是 API、消息、标识符、属性、配置等等。 +2. 一个好的通用方法是尽可能地将类中的字段设置为 `final`。 +3. 在 map 中作为 key 的类应该是严格不可变的，并且只有 final 字段（可能除了辅助的字段，如延迟缓存的 hash code）。 +4. 急切初始化类。不应该有 `init()` 或 `setup()` 方法。构造函数完成后，对象应该就可用。 -### Nullability of the Mutable Parts +### 可变部件（Mutable Parts）的可空性（Nullability） -For nullability, the Flink codebase aims to follow these conventions: +Flink 代码库对于可空性旨在遵循以下约定: -* Fields, parameters, and return types are always non-null, unless indicated otherwise -* All fields, parameters and method types that can be null should be annotated with `@javax.annotation.Nullable`. -That way, you get warnings from IntelliJ about all sections where you have to reason about potential null values. -* For all mutable (non-final) fields that are not annotated, the assumption is that while the field value changes, there always is a value. - * This should be double check whether these can in fact not be null throughout the lifetime of the object. +* 除非有特殊说明，否则字段、参数和返回类型始终都是非 null。 +* 所有可以为 null 的字段、参数和方法类型都要使用 `@javax.annotation.Nullable` 注解（Annotated）。 +这样 IntelliJ 就能够对可能出现的 null 值进行警告. +* 对于那些没有添加注解的可变（not-final）字段，就无法确定字段值是否为 null。 + * 此时应该仔细的检查这些值在对象的整个生命周期中是否可以不为 null。 -_Note: This means that `@Nonnull` annotations are usually not necessary, but can be used in certain cases to override a previous annotation, or to point non-nullability out in a context where one would expect a nullable value._ +_注意: 大部分情况下是不需要 `@Nonnull` 注解的，但有些时候可以用来覆盖之前的注解，或者在不可以为 null 的上下文（Context）中，还想要得到 null 值。_ -`Optional` is a good solution as a return type for method that may or may not have a result, so nullable return types are good candidates to be replaced with `Optional`. -See also [usage of Java Optional](code-style-and-quality-java.md#java-optional). +对于不确定是否有结果返回的方法，`Optional` 作为方法的返回类型是个很好的解决方案，可以用 `Optional` 来代替所有可以为 null 的返回类型。 +参考 [Java Optional 的用法](code-style-and-quality-java.zh.md#java-optional). -### Avoid Code Duplication +### 避免重复的代码 -1. Whenever you are about to copy/paste some code, or reproduce a similar type of functionality in a different place, think about the ways how to refactor/reuse/abstract the changes to avoid the duplication. -2. Common behavior between different specializations should be shared in a common component (or a shared superclass). -3. Always use “private static final” constants instead of duplicating strings or other special values at different locations. Constants should be declared in the top member area of a class. +1. 当你准备复制/粘贴一些代码，或者在不同的地方实现类似的功能时，就要考虑怎么去重构、复用、抽象来避免重复的代码。 +2. 不同模块之间的相同特性应该抽象到公共组件（或父类）中。 +3. 常量应该声明在类顶部的成员区域中，并且是使用“private static final”修饰，而不是在不同的地方复制。 -### Design for Testability +### 可测性设计（Design for Testability） -Code that is easily testable typically has good separation of concerns and is structured to be reusable outside the original context (by being easily reusable in tests). +容易进行测试的代码通常能够很好的使关注点分离，并且可以在其他地方重复使用（测试的时候很容易重复使用）。 -A good summary or problems / symptoms and recommended refactoring is in the PDF linked below. -Please note that while the examples in the PDF often use a dependency injection framework (Guice), it works in the same way without such a framework.[^1] +下面的 PDF 链接中有对问题的总结和重构的建议。需要注意的是，虽然 PDF 中的示例使用 Guice 作为依赖注入框架，但是如果没有使用这个框架，它也能达到相同的效果。[^1] [http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf](http://misko.hevery.com/attachments/Guide-Writing%20Testable%20Code.pdf) -Here is a compact summary of the most important aspects. +下面是重点方面的简要总结 -**Inject dependencies** +**依赖关系的注入** -Reusability becomes easier if constructors don’t create their dependencies (the objects assigned to the fields), but accept them as parameters. +如果构造对象时不创建依赖项（分配给字段的对象），而是通过接受参数的方式，那么可重用性（Reusability）的实现就会变得更容易。 -* Effectively, constructors should have no `new` keyword. -* Exceptions are creating a new empty collection (`new ArrayList<>()`) or similar auxiliary fields (objects that have only primitive dependencies). +* 实际上，构造函数应该没有 `new` 关键字。 +* 不建议创建一个新的空集合（`new ArrayList<>()`）或类似的辅助字段（对象有且仅有原始的依赖）。 -To make instantiation easy / readable, add factory methods or additional convenience constructors to construct whole object with dependencies. +为了让对象更容易的实例化和易读，可以添加工厂方法或者其他简便的构造函数来对其依赖关系进行构造。 -In no case should it ever be required to use a reflection or a “Whitebox” util to change the fields of an object in a test, or to use PowerMock to intercept a “new” call and supply a mock. +在任何情况下，都不应该使用反射或者“白盒”工具来修改测试过程中对象的字段，或者使用 PowerMock 拦截“新”的调用并进行模拟。 -**Avoid “too many collaborators”** +**避免 “过多的协作者”** -If you have to take a big set of other components into account during testing (“too many collaborators”), consider refactoring. +如果在测试过程中需要大量的其他组件（“过多的协作者”），那么就要考虑重构了。 -The component/class you want to test probably depends on another broad component (and its implementation), rather than on the minimal interface (abstraction) required for its work. +有时候需要测试的组件/类可能依赖于另外的公共组件（及其实现），而不是其工作所需的最小接口（抽象）。 -In that case, segregate the interfaces (factor out the minimal required interface) and supply a test stub in that case. +在这种情况下，隔离接口（将所需的最小接口抽取出来）并在这种情况下提供测试存根。 -* For example, if testing a S3RecoverableMultiPartUploader requires actual S3 access -then the S3 access should be factored out into an interface and test should replace it by a test stub -* This naturally requires to be able to inject dependencies (see above) +* 例如，假设测试的 S3RecoverableMultiPartUploader 需要实际的 S3 进行访问，那么应该将 S3 的访问抽取成一个接口，测试时使用测试存根来代替它。 +* 这当然需要能够注入依赖项（参见上文）。 -⇒ Please note that these steps often require more effort in implementing tests (factoring out interfaces, creating dedicated test stubs), but make the tests more resilient to changes in other components, i.e., you do not need to touch the tests when making unrelated changes. +⇒ 需要注意的是，实现测试的这些步骤（抽取接口、创建专用测试存根）通常需要更多工作，但是这对于其他组件的修改更具有弹性，也就是说没有修改的组件不需要进行测试。 -**Write targeted tests** +**编写针对性测试** -* <span style="text-decoration:underline;">Test contracts not implementations</span>: Test that after a sequence of actions, the components are in a certain state, rather than testing that the components followed a sequence of internal state modifications. - * For example, a typical antipattern is to check whether one specific method was called as part of the test -* A way to enforce this is to try to follow the _Arrange_, _Act_, _Assert_ test structure when writing a unit test ([https://xp123.com/articles/3a-arrange-act-assert/](https://xp123.com/articles/3a-arrange-act-assert/)) +* <span style="text-decoration:underline;">契约测试而不是实现</span>： 经过一系列的测试操作之后，组件处于某种状态，而不是测试组件遵循一系列内部状态的修改。 + * 例如，经典的反面模式是检查一个特定的方法是否作为测试的一部分被调用。 +* 实现这一点的方法是在编写单元测试时尽量的遵循 Arrange、Act、Assert 测试结构([https://xp123.com/articles/3a-arrange-act-assert/](https://xp123.com/articles/3a-arrange-act-assert/)) - This helps to communicate the intention of the test (what is the scenario under test) rather than the mechanics of the tests. The technical bits go to a static methods at the bottom of the test class. - - Example of tests in Flink that follow this pattern are: + 这有助于传递测试的意图（测试的场景是什么），而不是测试的机制。技术部分进入到测试类底部的静态方法。 + + Flink 中遵循此模式的测试示例如下: * [https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java](https://github.com/apache/flink/blob/master/flink-core/src/test/java/org/apache/flink/util/LinkedOptionalMapTest.java) * [https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java](https://github.com/apache/flink/blob/master/flink-filesystems/flink-s3-fs-base/src/test/java/org/apache/flink/fs/s3/common/writer/RecoverableMultiPartUploadImplTest.java) -**Avoid Mockito - Use reusable test implementations** - -* Mockito-based tests tend to be costly to maintain in the long run by encouraging duplication of functionality and testing for implementation rather than effect - * More details: [https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8](https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8) -* Instead, create reusable test implementations and utilities - * That way, when some class changes, we only have to update a few test utils or mocks +**避免 Mockito - 使用可重用的测试实现** +* 从长远来看，基于 Mockito 的测试维护成本往往很高，因为它更倾向于功能的重复测试和实现测试，而不是效果测试。 + * 更多的细节： [https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8](https://docs.google.com/presentation/d/1fZlTjOJscwmzYadPGl23aui6zopl94Mn5smG-rB0qT8) +* 相反的，应该创建可重用的测试实现和实用程序。 + * 这样，当某些类修改时，只需要更新一些测试工具或 mock。 -### Performance Awareness -We can conceptually distinguish between code that “coordinates” and code that “processes data”. Code that coordinates should always favor simplicity and cleanness. Data processing code is highly performance critical and should optimize for performance. +### 性能意识 -That means still applying the general idea of the sections above, but possibly forgoing some aspects in some place, in order to achieve higher performance. +我们可以从概念上区分“协调”代码和“数据处理”代码。协调代码应该始终保持简单和干净，而数据处理代码的性能至关重要，应该对性能进行优化。 +这说明应用上面各部分的基本思想，但是为了获得更高的性能，可能会在某些地方放弃掉某一些方面。 -**Which code paths are Data Processing paths?** -* <span style="text-decoration:underline;">Per-record code paths:</span> Methods and code paths that are called for each record. Found for example in Connectors, Serializers, State Backends, Formats, Tasks, Operators, Metrics, runtime data structures, etc. -* <span style="text-decoration:underline;">I/O methods:</span> Transferring messages or chunks of data in buffers. Examples are in the RPC system, Network Stack, FileSystems, Encoders / Decoders, etc. +**哪些代码是数据处理代码？** +* <span style="text-decoration:underline;">每行代码的记录：</span> 记录每行代码的调用的方法和路径。例如在连接器、序列化器、状态后端、格式、任务、操作符、度量、运行时的数据结构等等。 +* <span style="text-decoration:underline;">I/O 方法：</span> 在缓冲区中传输消息或数据块。例如在 RPC 系统、网络堆栈、文件系统、编码器/解码器等等。 -**Things that performance critical code may do that we would otherwise avoid** -* Using (and reusing) mutable objects to take pressure off the GC (and sometimes help with cache locality), thus forgoing the strive for immutability. -* Using primitive types, arrays of primitive types, or MemorySegment/ByteBuffer and encoding meaning into the primitive types and byte sequences, rather than encapsulating the behavior in dedicated classes and using objects. -* Structuring the code to amortize expensive work (allocations, lookups, virtual method calls, …) across multiple records, for example by doing the work once per buffer/bundle/batch. -* Code layout optimized for the JIT rather than for readability. Examples are inlining fields from other classes (in cases where it is doubtful whether the JIT would do that optimization at runtime), or structuring code to help the JIT compiler with inlining, loop unrolling, vectorization, etc. +**有些代码的性能问题是可以避免的** +* 使用（和重用）可变对象来减轻 GC 的压力（有时候还能帮助缓存局部性），从而放弃不可变性。 +* 使用基本类型、基本类型数组或 MemorySegment/ByteBuffer 或将含义编码为基本类型和字节序列，而不是将行为封装在专用类中并使用对象。 +* 通过跨多个记录来分摊构造代码的昂贵工作（分配、查找、虚拟方法的调用...），例如，通过对缓冲区/绑定/批处理执行一次工作。 +* 针对 JIT 优化而不是可读性的代码布局。例如，来自其他类的内联字段（不确定 JIT 是否会在运行时进行优化的情况下），或者通过构造代码来帮助 JIT 编译器进行内联、循环展开、向量化等等。 -## 5. Concurrency and Threading -**Most code paths should not require any concurrency.** The right internal abstractions should obviate the need for concurrency in almost all cases. +## 5. 线程和并发性 -* The Flink core and runtime use concurrency to provide these building blocks. -Examples are in the RPC system, Network Stack, in the Task’s mailbox model, or some predefined Source / Sink utilities. -* We are not fully there, but any new addition that introduces implements its own concurrency should be under scrutiny, unless it falls into the above category of core system building blocks. -* Contributors should reach out to committers if they feel they need to implement concurrent code to see if there is an existing abstraction/building-block, or if one should be added. +**大多数的代码不需要任何的并发** 正确的内部抽象应该是所有情况下都不需要考虑并发性的问题。 +* Flink 的core 和runtime 构建块都是通过并发来实现的。例如 RPC 系统、网络堆栈、任务的邮箱模型或者一些预定于的 Source/Sink 实用程序。 +* 虽然现在还没有完全做到这一点，但是所有自身实现了并发性的新添加模块都应该仔细的检查，除非它属于上述的系统核心构建块。 +* 如果贡献者（Contributors）查看现有的抽象/构建块不能实现并发代码，需要添加并发代码或者抽象/构建块，那么贡献者应该与提交者（Committers）联系。 -**When developing a component think about threading model and synchronization points ahead.** +**在开发组件时，请提前考虑线程模型和同步点** -* For example: single threaded, blocking, non-blocking, synchronous, asynchronous, multi threaded, thread pool, message queues, volatile, synchronized block/methods, mutexes, atomics, callbacks, … -* Getting those things right and thinking about them ahead is even more important than designing classes interfaces/responsibilities, since it’s much harder to change later on. +* 例如：单线程、阻塞、非阻塞、同步、异步、多线程、线程池、消息队列、可见性（Volatile）、同步代码块/方法、互斥、原子、回调等等。 +* 提前把这些事情都考虑好甚至比设计类接口/职责更重要，因为后面想要修改的话就难了。 -**Try to avoid using threads all together if possible in any way.** +**尽可能的避免以任何方式共享线程** -* If you feel you have a case for spawning a thread, point this out in the pull request as something to be explicitly reviewed. +* 如果有产生线程的情况，需要在 pull 请求中指出这一点，作为要显式检查的内容。 -**Be aware that using threads is in fact much harder than it initially looks** +**要知道使用线程实际上比初始化要难的多** -* Clean shutdown of threads is very tricky. -* Handling interruptions in a rock solid fashion (avoid both slow shutdown and live locks) requires almost a Java Wizard -* Ensuring clean error propagation out of threads in all cases needs thorough design. -* Complexity of multi-threaded application/component/class grows exponentially, with each additional synchronisation point/block/critical section. Your code initially might be easy enough to understand, but can quickly grow beyond that point. -* Proper testing of multithreaded code is basically impossible, while alternative approaches (like asynchronous code, non-blocking code, actor model with message queues) are quite easy to test. -* Usually multi-threaded code is often even less efficient compared to alternative approaches on modern hardware. +* 清除关闭线程非常棘手。 +* 以绝对可靠的方式处理中断（避免缓慢关闭和活动锁）差不多需要一个 Java Wizard。 +* 在所有情况下，确保线程清除的错误进行传播都需要深入的设计。 +* 多线程应用程序/组件/类的复杂度随着每个额外的同步点/块/临界部分呈指数级增长。可能你的代码最开始很容易理解，但是很快就不是了。 +* 想要正确的测试多线程代码基本上是不可能的，而替代的方法（如异步代码、非阻塞代码、带有消息队列的actor 模型）却能很容易的测试。 +* 通常情况下多线程代码的效率比现代硬件上的替代方法更低。 -**Be aware of the java.util.concurrent.CompletableFuture** +**注意 java.util.concurrent.CompletableFuture** -* Like with other concurrent code, there should rarely be the need to use a CompletableFuture -* Completing a future would also complete on the calling thread any chained futures that are waiting for the result to be completed, unless a completion executor specified explicitly. -* This can be intentional, if the entire execution should be synchronous / single-threaded, as for example in parts of the Scheduler / ExecutionGraph. - * Flink even makes use of a “main-thread executor” to allow calling chained handlers in the same thread as a single-threaded RPC endpoint runs -* This can be unexpected, if the thread that completes the future is a sensitive thread. - * It may be better to use `CompletableFuture.supplyAsync(value, executor)` in that case, instead of `future.complete(value)` when an executor is available -* When blocking on a future awaiting completion, always supply a timeout for a result instead of waiting indefinitely, and handle timeouts explicitly. -* Use `CompletableFuture.allOf()`/`anyOf()`, `ExecutorCompletionService`, or `org.apache.flink.runtime.concurrent.FutureUtils#waitForAll` if you need to wait for: all the results/any of the results/all the results but handled by (approximate) completion order. +* 和其他并发代码一样，很少需要用到 CompletableFuture。 +* 除非有明确地指定完成的执行程序，否则在调用线程上完成的 future 将等待所有的链式 futures 完成。 +* 如果整个执行都是同步/单线程的，那么很有可能是故意的，例如在 Scheduler/ExecutionGraph 的某些部分中。 + * Flink 甚至允许使用“main-thread-executor”来运行单线程的 RPC 端点中的相同的线程中调用链式处理程序。 +* 如果完成 future 的线程是敏感线程（Thread-Sensitive），那么这可能是没有向导的。 + * 这种情况下，最好使用 `CompletableFuture.supplyAsync(value, executor)` 而不是在 executor 可用时使用 `future.complete(value)`。 +* 当阻塞等待 future 完成时，始终提供超时的结果，而不是无限期地等待，并且显式地处理超时。 +* 如果需要等待：所有的结果/任意的结果/由（差不多）完成顺序处理的所有结果，请使用 `CompletableFuture.allOf()`/`anyOf()`， `ExecutorCompletionService`，或 `org.apache.flink.runtime.concurrent.FutureUtils#waitForAll`。 -## 6. Dependencies and Modules +## 6. 模块和依赖 Review comment: ```suggestion <a name="dependencies-and-modules"></a> ## 6. 模块和依赖 ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
