Hello,
I totally agree that we should start to think about the module organization.
I suggest making the new confluence page where we define new rules on how to
develop modules.
In my opinion, at least, the following topics should be covered there(it makes
sense to discuss every topic separately, not here):
* In which cases new modules are required
* The naming of modules, packages
* Class dependency management ( inversion of control, no more context)
* Test organization ( module contains only unit, module tests. All integration
tests are in the extra module)
* Feature/Module lifecycle - experimental, stable, unstable, deprecated
Let's get back to the original topic. I agree with the package naming rule:
if the module name is configuration the package name should be
org.apache.ignite.configuration and only after that any other subpackages.
Also, I don't sure that we need ignite- prefix in the module name because it
doesn't have any extra information:
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-configuration</artifactId>
we don't lose anything if convert it to
<groupId>org.apache.ignite</groupId>
<artifactId>configuration</artifactId>
I also hope that jigsaw can help us somehow with class visibility.
But if not we can take agreement that for example 'internal' package -
shouldn't be touched outside of the module -
of course, using the proper class access level is also the solution(where it is
possible)
org.apache.ignite.configuration.Configurator // it has access to the internal
package but it shouldn't return any class from the internal package only from
the public one.
org.apache.ignite.configuration.DynamicProperty //interface
org.apache.ignite.configuration.internal.properties.IntDynamicProperty
User API makes sense only for one end module - ignite(or ignite-core) which
depends on all other modules
and doing some integration(adapters) and provide final API for the user.
So I agree that separated module Ignite-API with zero dependencies will be a
good solution.
configuration module:
Configurator.baseline().enabled() -> DynamicProperties
ignite-api module:
BaselineConfiguration.enabled() -> boolean //interface
ignite module:
BaselineConfigurationImpl implements BaselineConfiguration{
Configurator configurator;
public boolean enabled(){
return configurator.baseline().enabled().value();
}
}
So maybe my example is not so good. But I want to show that end-user API will
be defined only in ignite-api
and you need to adapt it in ignite module which leads to some
overhead(especially in my example)
but it makes development pretty manageable/predictable -
you can easily implement a new module without any worries that user starts to
use it.
It will be available only after making changes in ignite-api.
The major advantage here is the small size of ignite-api which allows to
carefully review every change
which allows keeping ignite API in better quality(I hope at least)
Nikolay, maybe is it better to discuss your question on a separate topic?
Because looks like it is a pretty discussable topic.
--
Best regards,
Anton Kalashnikov
09.12.2020, 10:31, "Nikolay Izhikov" <nizhi...@apache.org>:
> Hello, Zhenya, Ivan.
>
>> Hello Nikolay, if i find out introduced features structure in some project,
>> i would prefer to choose different one )
>
> Many, of the real world users disagree with you.
> Please, take a look at some examples from widely used projects:
>
> Kafka -
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/annotation/InterfaceStability.java#L28
> - Stable, Evolving, Unstable
>
> Spark -
> https://github.com/apache/spark/tree/master/common/tags/src/main/java/org/apache/spark/annotation
> - AlphaComponent, DeveloperApi, Evolving, Experimental, Private,
> Stable, Unstable
>
>> Having officially "unstable" features doesn't sound good for product
>> reputation.
>
> Can’t agree with you.
>
> Forcing ourselves to make perfect API from the first try we just put too much
> pressure on every decision.
> Every developer making mistakes.
> The product is evolving and the API too - it’s totally OK.
>
> For every new feature time required to be adopted and used in real-world
> production.
> I believe, slight API changes is totally fine for early adopters.
> Moreover, I think, that we should warn our users that some feature is very
> fresh and can have issues.
>
> So, Why Kafka and Spark is good enough to have unstable API and Ignite not? :)
>
>> 9 дек. 2020 г., в 10:08, Ivan Bessonov <bessonov...@gmail.com> написал(а):
>>
>> Conversation shifted into an unintended direction, but I agree.
>>
>> I think that if API can (or will) be changed then it should be deprecated.
>> For that
>> we can introduce @IgniteDeprecated that will contain Ignite version when
>> API is planned to be removed. Otherwise it's either stable or experimental.
>> Having officially "unstable" features doesn't sound good for product
>> reputation.
>>
>> As for the modularization - I'm all for this idea. If we don't force
>> ourselves to
>> organize code properly then we'll end up with the same problems as we have
>> in the current code base. And this way there's a hope of having good tests
>> that can be completed in minutes, not hours. At least new ones.
>>
>> BTW, did we have any discussions about dependency injection and all this
>> stuff?
>> Seems like a related topic to me.
>>
>> ср, 9 дек. 2020 г. в 09:47, Zhenya Stanilovsky <arzamas...@mail.ru.invalid>:
>>
>>> Hello Nikolay, if i find out introduced features structure in some
>>> project, i would prefer to choose different one )
>>>
>>>>>> Hello, Alexey.
>>>>>>
>>>>>> Think we can extend our @IgniteExperimental annotation.
>>>>>>
>>>>>> `@IgniteExperimental` - mark features that are truly experimental and
>>> can be completely removed in future releases.
>>>>>> `@NotRecommended` - mark features that widely adopted by the users but
>>> implemented wrong or have known issues that can’t be fixed.
>>>>>> `@NotStable` - mark features supported by community but API not stable
>>> and can be reworked in the next release.
>>>>>> `@Stable` - mark features that are completely OK and here to stay.
>>>>>>
>>>>>> We should output notes about these annotations in the JavaDoc, also.
>>>>>> What do you think?
>>>>>>
>>>>>>> 8 дек. 2020 г., в 12:49, Alexey Goncharuk <
>>> alexey.goncha...@gmail.com > написал(а):
>>>>>>> Igniters,
>>>>>>>
>>>>>>> I want to tackle the topic of modules structure in Ignite 3. So far,
>>> the
>>>>>>> modules in Ignite are mostly defined intuitively which leads to some
>>>>>>> complications:
>>>>>>>
>>>>>>> - Ignite public API is separated from the rest of the code only by
>>>>>>> package name. This leads to private classes leaking to public API
>>> which is
>>>>>>> very hard to catch even during the review process (we missed a bunch
>>> of
>>>>>>> such leaks for new metrics API [1] and I remember this happening for
>>> almost
>>>>>>> every SPI)
>>>>>>> - Classes from 'internal' packages are considered to be 'free for
>>> grabs'
>>>>>>> in every place of the code. This leads to tight coupling and
>>> abstraction
>>>>>>> leakage in the code. An example of such a case - an often cast of
>>>>>>> WALPointer to FileWALPointer, so that the community decided to get
>>> rid of
>>>>>>> the WALPointer interface altogether [2]
>>>>>>> - Overall code complexity. Because of the lack of inter-module
>>>>>>> interaction rules, we are free to add new methods and callbacks to any
>>>>>>> class, which leads to duplicating entities and verbose interfaces. A
>>> good
>>>>>>> example of this is the clear duplication of methods in
>>>>>>> IgniteCacheOffheapManager and IgniteCacheOffheapManager.DataStore [3]
>>>>>>>
>>>>>>> I think we need to work out some rules that will help us define and
>>> control
>>>>>>> both Ignite public API and module internal API which still defines a
>>> clear
>>>>>>> contract for other modules. Some ideas:
>>>>>>>
>>>>>>> - Perhaps we can move all user public classed and interfaces to an
>>>>>>> Ignite-API module which will have no dependencies on implementation
>>>>>>> modules. This will prevent private classes from leaking to the API
>>> module.
>>>>>>> - We need somehow define which classes from a module are exposed to
>>>>>>> other modules, and which classes are left for module-private usage.
>>> Maybe
>>>>>>> Java's jigsaw will help us here, but maybe we will be ok with just
>>> more
>>>>>>> strict java access modifiers usage :) The idea here is that a module
>>> should
>>>>>>> never touch a dependent module's private classes, ever. The exported
>>>>>>> classes and interfaces are still free to be modified between
>>> releases, as
>>>>>>> long as it is not a user public API.
>>>>>>> - A module should be logically complete, thus it may be beneficial if
>>>>>>> module name matches with the code package it provides (e.g.
>>> configuration
>>>>>>> -> org.apache.ignite.configuration, replication ->
>>>>>>> org.apache.ignite.replication, raft->org.apache.ignite.raft, etc)
>>>>>>>
>>>>>>> Any other principles/rules we can apply to make the code structure
>>> more
>>>>>>> concise? Thoughts?
>>>>>>>
>>>>>>> --AG
>>>>>>>
>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12552
>>>>>>> [2] https://issues.apache.org/jira/browse/IGNITE-13513
>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13220
>>
>> --
>> Sincerely yours,
>> Ivan Bessonov
