Is there any significant gain by doing this?
Thanks
On Sat, Aug 19, 2017 at 10:36 AM, Vlad Rozov <vro...@apache.org> wrote:
The parent level pom artifact Id is not referenced by applications, it
exists simply to group modules together under current malhar project.
The
name can be changed to library-parent(root) or
apex-library-parent(root) or
something similar. Note that the proposal includes rename of the project
from apex-malhar to apex-library along with the version change.
Thank you,
Vlad
On 8/19/17 10:10, Pramod Immaneni wrote:
Would library module then become apex-library-library? Why change the
malhar artifact id as the project is called apex-malhar.
Thanks
On Sat, Aug 19, 2017 at 10:03 AM, Vlad Rozov <vro...@apache.org>
wrote:
Overall I support the proposal and this is what I was in favor of
during
the last discussion. I'd also propose changing the name of the
artifact
id
and git repo from malhar to apex-library as part of the major version
change. It will make it more consistent with the entire project and
will
also allow using 1.0 (1.0-SNAPSHOT) version that more closely reflects
maturity of the library.
Thank you,
Vlad
On 8/18/17 17:04, Thomas Weise wrote:
The proposed change does NOT require changes to existing apps. It
follows
what you see for example in the JDK where older classes are retired
without
making breaking major release changes.
Even though not required technically, it looks like the preference
is to
make this change with a 4.0 release. I will therefore propose to
change
the
master branch to 4.0-SNAPSHOT and make this and other changes deemed
worthwhile.
To me it is more important that to newcomers the codebase does not
look
like an outdated legacy situation and therefore if I have to make a
choice
than pretty much any version number will do for that. So far I have
seen
very little interest and initiative in addressing maintenance work
like
the
packages, checkstyle, CI stability, documentation etc. So it would
be
great
if people interested would step up and contribute.
Here is the next proposal how to proceed:
- create release-3.8 branch for last 3.8 release right away
- update master to 4.0-SNAPSHOT as part of the open PR #664
- identify further cleanup work for master
- release 4.0
Thanks,
Thomas
On Thu, Aug 17, 2017 at 4:48 PM, Amol Kekre <a...@datatorrent.com>
wrote:
This following pull request should be taken up in 4.0.0. See my
comments
in
https://github.com/apache/apex-malhar/pull/664
<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%
2Fapache%2Fapex-malhar%2Fpull%2F664&sa=D&ust=1503020269444000&usg=
AFQjCNHDPZIg2e6I33Jb1XjB5Ir1FkdURQ>
https://github.com/apache/apex-malhar/pull/662
<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%
2Fapache%2Fapex-malhar%2Fpull%2F662&sa=D&ust=1503020269444000&usg=
AFQjCNF7Xde7X55M3qmc8z-D5y3ZwNg7Fg>
This merge should not be done without a consensus. This will
require
code
changes to existing apps.
Thks,
Amol
E:a...@datatorrent.com | M: 510-449-2606 | Twitter: @*amolhkekre*
www.datatorrent.com
On Mon, Aug 14, 2017 at 7:48 PM, Thomas Weise <t...@apache.org>
wrote:
Hi,
I opened following PRs for the package change:
https://github.com/apache/apex-malhar/pull/662
Moves all classes with history retained (hence 2 commits). Also
contains
checkstyle and other mechanical changes.
https://github.com/apache/apex-malhar/pull/664
Adds backward compatibility jar.
Once above PRs are merged the new artifact can be deployed and
introduced
as dependency in malhar-library.
Please review.
Thanks,
Thomas
On Sun, Jul 16, 2017 at 7:04 AM, Thomas Weise <t...@apache.org>
wrote:
My original list of work items contained the b/w compatibility
aspect,
I
don't think there should be any confusion of whether it will be
covered
here or not.
The proposed shading will provide the old classes and they will
be
frozen
as of release 3.7. That's the same as making a copy of the code
and
never
again making changes to the original classes. This cannot be
accomplished
by using the older 3.7 release in your project because you cannot
use
2
different versions of Malhar in parallel unless you apply shading.
The shaded artifact will only expose the com.datatorrent classes,
and
they
will be self-contained as the rest of the classes that they may
depend
on
are shaded. The shaded artifact does not evolve, there are not
more
changes
to com.datatorrent classes after they are relocated in master.
Thanks,
Thomas
On Sun, Jul 16, 2017 at 2:00 AM, Pramod Immaneni <
pra...@datatorrent.com
wrote:
I don't think we can limit the topic strictly to relocation
without
having
a good b/w compatibility story or at least one that goes far
enough.
The shading idea sounds interesting. Why not let the shaded
version
move
forward with each release till we hit a major release. If it is
going
to
remain pegged at 3.7.0, why shade in the first place as the
regular
3.7.0
release would do the same job and it would be same as the loss of
b/w
compatibility with newer releases.
Thanks
On Sat, Jul 15, 2017 at 7:57 AM, Thomas Weise <t...@apache.org>
wrote:
Discussing what in the future might become stable needs to be a
separate
thread, it will be a much bigger discussion.
The topic here is to relocate the packages. With a few exceptions
relocation won't affect the semantic versioning. Semantic
versioning
is
essentially not effective for Malhar because almost everything
is
@Evolving
(and there are reasons for that.. -> separate topic)
I don't really like the idea of creating bw compatibility stubs
for
the
follow up PR. It creates even more clutter in the source tree
than
there
already is and so here is an alternative suggestion:
https://github.com/tweise/apex-malhar/blob/malhar37-
compat/shaded-malhar37/pom.xml
Create a shaded artifact that provides the old
com.datatorrent.*
classes as
of release 3.7. Users can include that artifact if they don't
want
to
change import statements. At the same time they have an
incentive
to
switch
to the relocated classes to take advantage of bug fixes and new
functionality.
I will work on the first PR that does the relocate. In the
meantime,
we
can
finalize what backward compatibility support we want to provide
and
how.
Thanks,
Thomas
On Fri, Jul 14, 2017 at 11:33 AM, Pramod Immaneni <
pra...@datatorrent.com>
wrote:
How about coming up with a list of @Evolving operators that we
would
like
to see in the eventual stable list and move those along with the
not
@Evolving ones in org.apache.apex with b/w stubs and leave the
rest
as
they
are. Then have a follow up JIRA for the rest to be moved over to
contrib
and be deprecated.
Thanks
On Fri, Jul 14, 2017 at 10:37 AM, Thomas Weise <
thomas.we...@gmail.com>
wrote:
We need to keep the discussion here on topic, if other things
are
piled
on
top then nothing gets done.
Most operators today are not stable, they are @Evolving. So
perhaps
it
is
necessary to have a separate discussion about that, outside of
this
thread.
My guess is that there are only a few operators that we could
declare
stable. Specifically, under contrib the closest one might have
been
Kafka,
but that is already superseded by the newer versions.
Thomas
On Fri, Jul 14, 2017 at 10:21 AM, Pramod Immaneni <
pra...@datatorrent.com>
wrote:
We had a discussion a while back, agreed to relegate
non-stable
and
experimental operators to contrib and also added this to
contribution
guidelines. We aexecuted on this and cleaned up the repo by
moving
operators deemed non-suitable for production use at that time
to
contrib.
So I wouldn't say the operators that are at the top level
today
or
the
ones
in library are 0.x.x quality. Granted, we may need to do one
more
pass
as
some of the operators may no longer be considered the right
implementations
with the advent of the windowed operator.
Since contrib used to be the place that housed operators
that
required
third party libraries, there are some production quality
operators in
there
that need to be pulled out into top level modules.
I think we are in agreement that for operators that we
consider
stable,
we
should provide a b/w stub. I would add that we strongly
consider
creating
the org.apache.apex counterparts of any stable operators that
are
in
contrib out in top level modules and have the com.datatorrent
stubs
in
contrib.
For the operators not considered stable, I would prefer we
either
leave
the
current package path as is or if we change the package path
then
create
the
b/w stub as I am not sure how widely they are in use (so, in
essence,
preserve semantic versioning). It would be good if there is a
followup
JIRA
that takes a look at what other operators can be moved to
contrib
with
the
advent of the newer frameworks and understanding.
Thanks
On Fri, Jul 14, 2017 at 9:44 AM, Thomas Weise <
t...@apache.org
wrote:
Most of the operators evolve, as is quite clearly indicated
by
@Evolving
annotations. So while perhaps 0.x.x would be a more
appropriate
version
number, I don't think you can change that.
Thomas
On Fri, Jul 14, 2017 at 9:35 AM, Vlad Rozov <
v.ro...@datatorrent.com
wrote:
If entire library is not stable, its version should be
0.x.x
and
there
should not be any semantic versioning enabled or implied.
It
evolves.
If
some operators are stable as 3.8.x version suggests, the
library
should
follow semantic versioning and it is not OK to make a
major
change
that
breaks semantic versioning across the entire library. It
is
not a
finding
an excuse not to make a change. For me semantic versioning
and
backward
compatibility is more important than a proper package
name.
Thank you,
Vlad
On 7/14/17 09:11, Thomas Weise wrote:
Semantic versioning makes sense when you have a stable
baseline.
As
long
as
frequent fixes need to be made to address basic issues,
it
makes
no
sense
to declare operators stable, which is why they are marked
evolving.
Instead of finding reasons to avoid changes that should
have
been
made a
long time ago, why not discuss what a "stable operator"
is
and
which
ones
deserve to be in that category. Those are the ones that
will
get
the
backward compatibility stub.
Thanks,
Thomas
On Fri, Jul 14, 2017 at 8:34 AM, Vlad Rozov <
v.ro...@datatorrent.com>
wrote:
My preference is to agree that the next Malhar release is
4.0.0
and
make
the proposed changes when 3.8.0 goes out. Otherwise why
to
keep
semantic
versioning checks on Malhar in case there is no version
compatibility.
Thank you,
Vlad
On 7/14/17 08:11, Thomas Weise wrote:
next release is 3.8.0
On Fri, Jul 14, 2017 at 7:55 AM, Vlad Rozov <
v.ro...@datatorrent.com>
wrote:
next release - 3.9.0 or 4.0.0?
Thank you,
Vlad
On 7/13/17 22:25, Thomas Weise wrote:
It is time to resurrect this thread and get going with
the
work.
For the next release, I will sign up to do the
package
move
in
Malhar:
https://issues.apache.org/
jira/browse/APEXMALHAR-2517
In general this will be straightforward; most classes
in
Malhar
are
marked
evolving and it is trivial for users to change import
statements.
However,
I would suggest to discuss if there are selected
operators
that
are
worth
keeping a backward compatibility stub in the old
location.
Here is my plan:
1. Relocate all classes in *lib* and *contrib* within
one
PR -
this
is
all
IDE automated work
2. Add backward compatibility classes, if, any in
separate
PR
3. Create PR for Megh library to reflect moved
classes
Thanks,
Thomas
On Wed, Mar 1, 2017 at 2:24 PM, Pramod Immaneni <
pra...@datatorrent.com
wrote:
Inline
On Mon, Feb 27, 2017 at 11:03 PM, Thomas Weise <
t...@apache.org>
wrote:
-->
On Mon, Feb 27, 2017 at 1:27 PM, Pramod Immaneni <
pra...@datatorrent.com
wrote:
For malhar, for existing operators, I prefer we do
this as
part
of
the
planned refactoring for breaking the monolith
modules
into
baby
packages
and would also prefer deprecating the existing
operators
in
place.
Refactor into smaller modules was discussed for
malhar-contrib
and
given
the overall state of that module I think it is OK
to
defer
package
renaming
there. I do however prefer to see the package rename
addressed
for
other
modules, especially for the main library module.
Should we consider breaking the library into
smaller
modules
as
well,
the
file/block operators for example probably can be in
their
own
module
from
just an organizational perspective.
This
will help us achieve two things. First, the user
will
see
all
the
new
changes at once as opposed to dealing with it
twice
(with
package
rename
and dependency changes) and second it will allow
for
a
smoother
transition
as the existing code will still work in a
deprecated
state.
It
will
also
give a more consistent structure to malhar. For new
operators,
we
can
go
with the new package path but we need to ensure
they
will
get
moved
into
the baby packages as well.
I think existing operators should be renamed so
that
git
history
remains. A
possible solution for backward compatibility could
be
to
subsequently
add
empty subclasses in the previous location (for
existing
Thank you,
Vlad
