Re: Reform of Incubator

2015-08-05 Thread Bertrand Delacretaz
On Tue, Aug 4, 2015 at 8:50 PM, Joe Brockmeier j...@zonker.net wrote:
 ...I may misunderstand or have lost track of how that's handled in all the
 discussion...

you're not alone - IMO the only way such proposals can work is based
on a concise wiki page that explains the proposal and gives everybody
a single reference about what's being discussed. Long email threads
only work for those who can constantly pay attention to them.

-Bertrand

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Podlings and the ASF maturity model (was: Reform of Incubator...)

2015-08-05 Thread Bertrand Delacretaz
Hi,

On Wed, Aug 5, 2015 at 12:24 AM, Dennis E. Hamilton
dennis.hamil...@acm.org wrote:
 ...I understand the maturity model to be something to aspire to and that 
 Apache Projects
 will always be working toward it.  I mean TLPs, not podlings, although 
 podlings should be
 aware of it and also aspire to it...

I don't see why podlings should be different here, once they are about
to graduate.

Why can't we define our incubation process as a way for podlings to
learn to operate according to that maturity model [1]?

This would allow us to use the maturity model [1] as a checklist for
graduating podlings - do you see anything in there that shouldn't be
required from a podling that's about to graduate?

-Bertrand

[1] http://community.apache.org/apache-way/apache-project-maturity-model.html

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: Wiki access

2015-08-05 Thread Ralph Goers
The userid is RalphGoers.

Ralph

 On Aug 4, 2015, at 5:41 PM, Marvin Humphrey mar...@rectangular.com wrote:
 
 On Tue, Aug 4, 2015 at 5:30 PM, Ralph Goers ralph.go...@dslextreme.com 
 wrote:
 For some reason I am not able to edit any pages on the incubator wiki.  I 
 could swear I used to be able to do that.  Does someone have karma to fix 
 this?
 
 Like all Apache wikis, the Incubator wiki had to implement
 whitelisting to counter spam.  I don't see anything resembling your
 name on http://wiki.apache.org/incubator/ContributorsGroup.  Let me
 know your Incubator wiki login and I'll add it.
 
 Marvin Humphrey
 
 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org
 
 


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: Wiki access

2015-08-05 Thread Upayavira
Done.

On Wed, Aug 5, 2015, at 09:59 AM, Ralph Goers wrote:
 The userid is RalphGoers.
 
 Ralph
 
  On Aug 4, 2015, at 5:41 PM, Marvin Humphrey mar...@rectangular.com wrote:
  
  On Tue, Aug 4, 2015 at 5:30 PM, Ralph Goers ralph.go...@dslextreme.com 
  wrote:
  For some reason I am not able to edit any pages on the incubator wiki.  I 
  could swear I used to be able to do that.  Does someone have karma to fix 
  this?
  
  Like all Apache wikis, the Incubator wiki had to implement
  whitelisting to counter spam.  I don't see anything resembling your
  name on http://wiki.apache.org/incubator/ContributorsGroup.  Let me
  know your Incubator wiki login and I'll add it.
  
  Marvin Humphrey
  
  -
  To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
  For additional commands, e-mail: general-h...@incubator.apache.org
  
  
 
 
 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org
 

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



[VOTE] Release Apache Taverna Parent 1-incubating-RC4 Apache Taverna Language 0.15.0-incubating RC4

2015-08-05 Thread Ian Dunlop
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Hello,

The Apache Taverna Incubator PPMC has voted +5 to release

Apache Taverna Parent 1-incubating (RC4)
Apache Taverna Language 0.15.0-incubating (RC4)

Incubator PMC members please review and vote on this incubator release.

Apache Taverna Language is a set of APIs for workflow definitions
(SCUFL2) and workflow inputs/outputs/run (DataBundle), as consumed and
produced by the Apache Taverna workflow system. The API includes
support for working with Research Object Bundles, and loading/saving
Taverna workflows in different formats.

Please see original [VOTE] thread
http://mail-archives.apache.org/mod_mbox/incubator-taverna-dev/201507.mb
ox/%3C55B8DBC7.2040806%40manchester.ac.uk%3E

and [DISCUSS] thread
http://mail-archives.apache.org/mod_mbox/incubator-taverna-dev/201507.mb
ox/%3C55B8DC3F.2070602%40manchester.ac.uk%3E

Please note that the git commit ids were incorrect in the original
[VOTE] email but were corrected during the vote process.
Also, during the release checks some NOTICE files were found to
contain information that was already in the top level LICENSE and a
bug was raised against it, see
https://issues.apache.org/jira/browse/TAVERNA-864

The release candidates to be voted over are available at:

https://dist.apache.org/repos/dist/dev/incubator/taverna/source/taverna-
parent-1-incubating-RC4/
https://dist.apache.org/repos/dist/dev/incubator/taverna/source/taverna-
language-0.15.0-incubating-RC4/

SHA-1 checksums:
3eb09278b914b96fb5c6059523b1bb3e69a09334
taverna-parent-1-incubating-source-release.zip
4eeb37d141fea284cc8de17635d97a47b279ea91
taverna-language-0.15.0-incubating-source-release.zip

MD5 checksums:
bccfe1116189f5ccc640fcbed4a4f96f
taverna-parent-1-incubating-source-release.zip
685b3aed3833a0b921d129995944dfc3
taverna-language-0.15.0-incubating-source-release.zip

Build the release candidates in the above order, using:

Java 7: mvn clean install -Dmaven.javadoc.skip=true  (see
https://issues.apache.org/jira/browse/TAVERNA-867 for the reasons the
maven command line params are required)
Java 8: mvn clean install

The release candidates correspond to the following git commits:

https://git-wip-us.apache.org/repos/asf?p=incubator-taverna-maven-parent
.git;a=commit;h=b12a1cc0ed2540507ab43107124e8ee911da4ddf

https://git-wip-us.apache.org/repos/asf?p=incubator-taverna-language.git
;a=commit;h=e2a9954e796d886d137eed3091b21637e9c8d1fd

Release candidates are signed with a GPG key available at:

https://dist.apache.org/repos/dist/release/incubator/taverna/KEYS

A staged Maven repository is available for review at:

https://repository.apache.org/content/repositories/orgapachetaverna-1006
/

The changelog for this release is available from JIRA:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=1231832
2version=12332247
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=1231832
2version=12332246

Please vote on releasing these packages as:

Apache Taverna Maven Parent 1-incubating
Apache Taverna Language 0.15.0-incubating

The vote will be open for a minimum of 72 hours.

[ ] +1 Release this package
[ ] 0 I don't feel strongly about it, but
don't object
[ ] -1 Do not release this package because...

Cheers,

Ian
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVwe91AAoJEPK45GBX+Cy5lMsH/jiDvvduoZ43y9SVMQcbIvdn
GZDXOviSco5NAWnekqVzR5juXXHGVCZD56NFDfDkzlS1EinQP11HWEZjx1Tgz5TL
0ghvwRtymc5s6Vu2SMlWeiZszja/lD2430zc28jwtBhesd3xckUIa2VHg9viDiaT
w5iCQQB95TiCnShJaUDbFg0TiLB2BH/xZrb07967MsyH1YeSOI72aOhVkxbFthlE
ELJhRmKBtmfl/RJoPydzrs4IOvvml57hcJQT6DjONGEuaIzSvOJVcV8l+oZgmM1a
QqK6bjS0I+rmcK6udAFrgZmtNKsI/n+qufnaTxg1j1ghZRhDWo/ZAawpzGBo/+I=
=K8dX
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



RE: Podlings and the ASF maturity model (was: Reform of Incubator...)

2015-08-05 Thread Dennis E. Hamilton
OK, thanks Bertrand.  My recollection of the maturity-model discussion was that 
it is about an ideal state and not some gate, and that projects would always be 
correcting their state toward that defined in the model.  I confirm that the 
current document simply characterizes what the state is for a mature project.

I see no statement anywhere that a TLP, or a graduating incubator project, must 
pass through a maturity assessment.  It would certainly be useful for a podling 
to conduct a self-assessment of its maturity before requesting graduation.  

It would also be useful for an operating TLP to assess itself with respect to 
the model, especially if there are concerns in that regard.

I am neutral on how this fits into graduation criteria for podlings.  However, 
if it is used for gating purposes, I think that needs to be made very clear by 
the IPMC lest it just lead to more randomizing of the podling seasoning 
process.  In particular, mentors need to be on board with respect to their 
responsibilities.  I can also imagine a graduation going forward (just as 
releases can) with certain items remaining to be addressed post-graduation.  I 
note that, at the moment, there is no direct reference to the Project Maturity 
Model at the https://www.apache.org/dev/project-requirements draft nor at the 
Incubation Policy, 
https://incubator.apache.org/incubation/Incubation_Policy.html.

I can also imagine a TLP using (or being asked to use) the project maturity 
model as a checklist in assessing their current state in a report to the Board. 
 

Are these the kinds of applications that folks have in mind?

 -- Dennis

-Original Message-
From: Bertrand Delacretaz [mailto:bdelacre...@apache.org] 
Sent: Wednesday, August 5, 2015 00:44
To: Incubator General general@incubator.apache.org; dennis.hamil...@acm.org
Subject: Podlings and the ASF maturity model (was: Reform of Incubator...)

Hi,

On Wed, Aug 5, 2015 at 12:24 AM, Dennis E. Hamilton
dennis.hamil...@acm.org wrote:
 ...I understand the maturity model to be something to aspire to and that 
 Apache Projects
 will always be working toward it.  I mean TLPs, not podlings, although 
 podlings should be
 aware of it and also aspire to it...

I don't see why podlings should be different here, once they are about
to graduate.

Why can't we define our incubation process as a way for podlings to
learn to operate according to that maturity model [1]?

This would allow us to use the maturity model [1] as a checklist for
graduating podlings - do you see anything in there that shouldn't be
required from a podling that's about to graduate?

-Bertrand

[1] http://community.apache.org/apache-way/apache-project-maturity-model.html

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



RE: third party tooling.

2015-08-05 Thread Dennis E. Hamilton
I don't have an answer about tooling with respect to the need for 
widely-available tools.  If the concern is for ability to use free tools on 
Windows, but there are alternatives to requiring an expensive commercial IDE 
for users while still taking advantage of the Microsoft development tool chain.

The Visual Studio Community Edition 2013 is free to use for open-source 
projects.  I would recommend it simply because it is available for development 
on and for Windows and should work for Corinthia (although I haven't tried your 
builds there).  This works if you are creating/publishing Visual Studio 
projects.

It might also be desirable to use Visual Studio Community Edition 2015, which 
is now released and available. 

-Original Message-
From: jan i [mailto:j...@apache.org] 
Sent: Wednesday, August 5, 2015 12:04
To: general@incubator.apache.org
Subject: third party tooling.

Hi.

We have recently (again) on different list discussed third party libraries.
Some strong
opinions have been aired.

So we have rules/policies for libraries but how about tooling, I have not
been able to
find any do not do this page.

I am about to prepare a release for Corinthia, which is a C99 project. I
would like to
write in the release note, that on ms-Windows we test with Visual Studio
2013, simply
because that is a fact.

But Visual Studio 2013 is not a free version so which rules/policies will I
break by not
testing with e.g. GCC on windows ?
(I hope none, but better ask than have to cut a new release candidate).

rgds
jan i.


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: third party tooling.

2015-08-05 Thread jan i
On 5 August 2015 at 22:46, Dennis E. Hamilton dennis.hamil...@acm.org
wrote:

 I don't have an answer about tooling with respect to the need for
 widely-available tools.  If the concern is for ability to use free tools on
 Windows, but there are alternatives to requiring an expensive commercial
 IDE for users while still taking advantage of the Microsoft development
 tool chain.

 The Visual Studio Community Edition 2013 is free to use for open-source
 projects.  I would recommend it simply because it is available for
 development on and for Windows and should work for Corinthia (although I
 haven't tried your builds there).  This works if you are
 creating/publishing Visual Studio projects.

 It might also be desirable to use Visual Studio Community Edition 2015,
 which is now released and available.

I was not asking so much about which tools would be desirable to use.

I want to make sure I have not overlooked a policy or rule.

rgds
jan i.



 -Original Message-
 From: jan i [mailto:j...@apache.org]
 Sent: Wednesday, August 5, 2015 12:04
 To: general@incubator.apache.org
 Subject: third party tooling.

 Hi.

 We have recently (again) on different list discussed third party libraries.
 Some strong
 opinions have been aired.

 So we have rules/policies for libraries but how about tooling, I have not
 been able to
 find any do not do this page.

 I am about to prepare a release for Corinthia, which is a C99 project. I
 would like to
 write in the release note, that on ms-Windows we test with Visual Studio
 2013, simply
 because that is a fact.

 But Visual Studio 2013 is not a free version so which rules/policies will I
 break by not
 testing with e.g. GCC on windows ?
 (I hope none, but better ask than have to cut a new release candidate).

 rgds
 jan i.


 -
 To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
 For additional commands, e-mail: general-h...@incubator.apache.org




Re: third party tooling.

2015-08-05 Thread Roman Shaposhnik
On Wed, Aug 5, 2015 at 1:50 PM, jan i j...@apache.org wrote:
 On 5 August 2015 at 22:46, Dennis E. Hamilton dennis.hamil...@acm.org
 wrote:

 I don't have an answer about tooling with respect to the need for
 widely-available tools.  If the concern is for ability to use free tools on
 Windows, but there are alternatives to requiring an expensive commercial
 IDE for users while still taking advantage of the Microsoft development
 tool chain.

 The Visual Studio Community Edition 2013 is free to use for open-source
 projects.  I would recommend it simply because it is available for
 development on and for Windows and should work for Corinthia (although I
 haven't tried your builds there).  This works if you are
 creating/publishing Visual Studio projects.

 It might also be desirable to use Visual Studio Community Edition 2015,
 which is now released and available.

 I was not asking so much about which tools would be desirable to use.

 I want to make sure I have not overlooked a policy or rule.

I'm not aware of any policy like that. That said, I'd say the rule in my book
is very close to Linux packaging guidelines. Open source software *must*
be bootsrappable from source using *only* open source software binaries
as the input.

It is absolutely fine to use the closed source tools to facilitate the release
process, etc. But the above rule has to hold if you want to call yourself
open source.

Thanks,
Roman.

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: apache binary distributions

2015-08-05 Thread Ralph Goers

 On Aug 5, 2015, at 5:44 PM, Roman Shaposhnik ro...@shaposhnik.org wrote:
 
 On Tue, Aug 4, 2015 at 1:08 PM, Ted Dunning ted.dunn...@gmail.com wrote:
 On Tue, Aug 4, 2015 at 2:22 AM, Jochen Theodorou blackd...@gmx.org wrote:
 
 It was also mentioned here, that for example publishing snapshot builds to
 maven central is not allowed. I guess in the release document they are
 basically to be handled as nightly builds and as such not for the general
 public, thus only for the dev-list. It was said, that having the SNAPSHOT
 appendix in the jar name as well as not being able to automatically get
 them via maven without having to add that tag is not enough for the
 end-user to know for, that this is no official release. And that if such
 things are going into the distribution repository, they have to be handled
 as release, including voting and such. For that I guess it does not matter
 if it is the apache repository or something else.
 
 What would happen if a third party would do this? Is the project/apache
 required to do something about this? I mean if you read this:
 http://mail-archives.apache.org/mod_mbox/incubator-general/201506.mbox/%3CD1B01671.4EE90%25rvesse%40dotnetrdf.org%3E
 some even see nightly builds, not communicated beyond the dev-list on
 non-apache servers already as a problem.
 
 Let us put that last part a step up... Let us assume someone takes one of
 the released sources of one of the java projects out there, makes maven
 artifacts out of it and publishes them at maven central. Is that ok? I mean
 that is very near the distributor case, so it should be ok, or not?
 
 
 That is fine.  Just make sure that the published org is NOT org.apache.foo
 
 What do you mean by publishing org in the context of the Maven Central?
 
 Thanks,
 Roman.

Maven Central has rules about what they will and won’t accept.  
1. My understanding is they will only accept artifacts that have a groupId of 
org.apache if they come from the Apache repository manager, except for what 
would have to be unusual circumstances (they might, for example, accept a new 
version of a project that has moved to the attic, but I would expect that they 
would try to confirm that wherever the artifact came from has taken over that 
project).
2. You cannot publish SNAPSHOTs to Maven Central.

See https://maven.apache.org/guides/mini/guide-central-repository-upload.html 
https://maven.apache.org/guides/mini/guide-central-repository-upload.html

Ralph

Re: third party tooling.

2015-08-05 Thread Marvin Humphrey
jan i wrote:

 I want to make sure I have not overlooked a policy or rule.

Corinthia is not required to be compatible with any particular platform.
Consider why this is desirable: one might provide open source code which only
works with a proprietary compiler, and then someone else might take your open
source and adapt it for use with a different compiler.

So long as the licensing of the open source product distributed by Apache is
not impacted by the platform choice in such a way that it would cause it to
violate law or Apache policy, we're in the clear.

See this Legal FAQ:

  http://www.apache.org/legal/resolved#platform

Roman Shaposhnik wrote in reply to jan i:

 I'm not aware of any policy like that. That said, I'd say the rule in my book
 is very close to Linux packaging guidelines. Open source software *must*
 be bootsrappable from source using *only* open source software binaries
 as the input.

 It is absolutely fine to use the closed source tools to facilitate the release
 process, etc. But the above rule has to hold if you want to call yourself
 open source.

That would add conditions which go beyond OSI's widely accepted Open Source
Definition:

  http://opensource.org/osd

Also, Linux packaging guidelines?  What does that refer to?  The Debian Free
Software Guidelines?

Marvin Humphrey

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: third party tooling.

2015-08-05 Thread Roman Shaposhnik
On Wed, Aug 5, 2015 at 4:15 PM, Marvin Humphrey mar...@rectangular.com wrote:
 Roman Shaposhnik wrote in reply to jan i:

 I'm not aware of any policy like that. That said, I'd say the rule in my book
 is very close to Linux packaging guidelines. Open source software *must*
 be bootsrappable from source using *only* open source software binaries
 as the input.

 It is absolutely fine to use the closed source tools to facilitate the 
 release
 process, etc. But the above rule has to hold if you want to call yourself
 open source.

 That would add conditions which go beyond OSI's widely accepted Open Source
 Definition:

Correct. And that's why I prefixed my statement with a disclaimer of
in my book.
OSI guidelines are really date in quite a few areas at this point
(API, code base
complexity and cloud are the key areas).

 Also, Linux packaging guidelines?  What does that refer to?  The Debian Free
 Software Guidelines?

The page that I remember off the top of my head is this one:

https://fedoraproject.org/wiki/Packaging:Guidelines#No_inclusion_of_pre-built_binaries_or_libraries

IOW, you can call yourself open source software all you want,
but unless you get an exception from Fedora Packaging Committee
you are not open enough for the distribution to consider your work.

This is, btw, why Java-based software packages with their pervasive
use of Maven have such a hard time integrating properly into Linux
distros.

Thanks,
Roman.

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: Podlings and the ASF maturity model (was: Reform of Incubator...)

2015-08-05 Thread Roman Shaposhnik
On Wed, Aug 5, 2015 at 12:43 AM, Bertrand Delacretaz
bdelacre...@apache.org wrote:
 Hi,

 On Wed, Aug 5, 2015 at 12:24 AM, Dennis E. Hamilton
 dennis.hamil...@acm.org wrote:
 ...I understand the maturity model to be something to aspire to and that 
 Apache Projects
 will always be working toward it.  I mean TLPs, not podlings, although 
 podlings should be
 aware of it and also aspire to it...

 I don't see why podlings should be different here, once they are about
 to graduate.

 Why can't we define our incubation process as a way for podlings to
 learn to operate according to that maturity model [1]?

 This would allow us to use the maturity model [1] as a checklist for
 graduating podlings - do you see anything in there that shouldn't be
 required from a podling that's about to graduate?

I see it as a useful checklist that may uncover interesting issues within
the graduating podling. I don't see anything in there that would qualify
as an unambiguous gating criteria.

Thanks,
Roman.

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



[DISCUSS] Apex Incubation Proposal

2015-08-05 Thread Amol Kekre
I would like to start a discussion on DataTorrent's core engine and its
operators joining the ASF as an incubating project under the name Apex.

The proposal is available on the wiki here:
https://wiki.apache.org/incubator/ApexProposal

The text of the proposal is also available at the end of this email

Apex is an enterprise grade native YARN big data-in-motion platform that
unifies batch and stream processing. Apex is a highly distributed,
performant, fault tolerant, stateful and easily operable platform.

Thanks in advance for your time and help.

Thks,
Amol



== Abstract ==
Apex is an enterprise grade native YARN big data-in-motion platform that
unifies stream processing as well as batch processing. Apex processes big
data in-motion in a highly scalable, highly performant, fault tolerant,
stateful, secure, distributed, and an easily operable way. It provides a
simple API that enables users to write or re-use generic Java code, thereby
lowering the expertise needed to write big data applications.

Functional and operational specifications are separated. Apex is designed
in a way to enable users to write their own code (aka user defined
functions) as is and leave all operability to the platform. The API is very
simple and is designed to allow users to drop in their code as is. The
platform mainly deals with operability and treats functional code as a
black box. Operability includes fault tolerance, scalability, security,
ease of use, metrics api, webservices, etc. In other words there is no
separation of UDF (user defined functions), as all functional code is UDF.
This frees users to focus on functional development, and lets platform
provide operability support. The same code runs as is with different
operability attributes. The data-in-motion architecture of Apex unifies
stream as well as batch processing in a single platform. Since Apex is a
native YARN application, it leverages all the components of YARN without
duplication. Apex was developed with YARN in mind and has no overlapping
components/functionality with YARN.

The Apex platform is supplemented by project Malhar, which is a library of
operators that implement common business logic functions needed by
customers who want to quickly develop applications. These operators provide
access to HDFS, S3, NFS, FTP, and other file systems;  Kafka, ActiveMQ,
RabbitMQ, JMS, and other message systems; MySql, Cassandra, MongoDB, Redis,
HBase, CouchDB and other databases along with JDBC connectors. The Malhar
library also includes a host of other common business logic patterns that
help users to significantly reduce the time it takes to go into production.
Ease of integration with all other big data technologies is one of the
primary missions of Malhar.

== Proposal ==
The goal of this proposal is to establish the core engine of DataTorrent
RTS product as an Apache Software Foundation (ASF) project in order to
build a vibrant, diverse, and self-governed open source community around
the technology. DataTorrent will continue to sell management tools,
application building tools, easy to use big data applications, and custom
high end business logic operators. This proposal covers the Apex source
code (written in Java), Apex documentation and other materials currently
available on https://github.com/DataTorrent/Apex. This proposal also covers
the Malhar source code (written in Java), Malhar documentation, and other
materials currently available on https://github.com/DataTorrent/Malhar. We
have done a trademark check on the name Apex, and have concluded that the
Apex name is likely to be a suitable project name.

== Background ==
DataTorrent RTS is a mature and robust product developed as a native YARN
application. RTS 1.0 was launched in summer of 2014; RTS 2.0 was launched
in Jan 2015. Both were well received by customers. RTS 3.0 was launched at
end of July 2015. RTS is among the first enterprise grade platform that was
developed from the ground up as native YARN application. DataTorrent RTS is
currently maintained by engineers as a closed source project. Even though
the engineers behind RTS are experienced software engineers and are
knowledge leaders in data-in-motion platforms, they have had little
exposure to the open source governance process. Customers are currently
running applications based on DataTorrent RTS in production.

== Rationale ==
Big data applications written for non-Hadoop platforms typically require
major rewrites  to get them to work with Hadoop. This rewriting creates a
significant bottleneck in terms of resources (expertise) which in turn
jeopardizes the viability of such an endeavour. It is hard enough to
acquire big data expertise, demanding additional expertise to do a major
code conversion makes it a very hard problem for projects to successfully
migrate to Hadoop. Also, due to the batch processing nature of Hadoop’s
MapReduce paradigm, users 

Re: apache binary distributions

2015-08-05 Thread Roman Shaposhnik
On Tue, Aug 4, 2015 at 1:08 PM, Ted Dunning ted.dunn...@gmail.com wrote:
 On Tue, Aug 4, 2015 at 2:22 AM, Jochen Theodorou blackd...@gmx.org wrote:

 It was also mentioned here, that for example publishing snapshot builds to
 maven central is not allowed. I guess in the release document they are
 basically to be handled as nightly builds and as such not for the general
 public, thus only for the dev-list. It was said, that having the SNAPSHOT
 appendix in the jar name as well as not being able to automatically get
 them via maven without having to add that tag is not enough for the
 end-user to know for, that this is no official release. And that if such
 things are going into the distribution repository, they have to be handled
 as release, including voting and such. For that I guess it does not matter
 if it is the apache repository or something else.

 What would happen if a third party would do this? Is the project/apache
 required to do something about this? I mean if you read this:
 http://mail-archives.apache.org/mod_mbox/incubator-general/201506.mbox/%3CD1B01671.4EE90%25rvesse%40dotnetrdf.org%3E
 some even see nightly builds, not communicated beyond the dev-list on
 non-apache servers already as a problem.

 Let us put that last part a step up... Let us assume someone takes one of
 the released sources of one of the java projects out there, makes maven
 artifacts out of it and publishes them at maven central. Is that ok? I mean
 that is very near the distributor case, so it should be ok, or not?


 That is fine.  Just make sure that the published org is NOT org.apache.foo

What do you mean by publishing org in the context of the Maven Central?

Thanks,
Roman.

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



Re: apache binary distributions

2015-08-05 Thread Roman Shaposhnik
On Tue, Aug 4, 2015 at 2:22 AM, Jochen Theodorou blackd...@gmx.org wrote:
 Am 03.08.2015 21:46, schrieb David Nalley:

 On Mon, Aug 3, 2015 at 5:55 AM, Jochen Theodorou blackd...@gmx.org
 wrote:

 Hi all,

 some of the general discussion recently made me wonder about one point
 with
 regards to binary distributions. It was pointed out, that a binary
 distribution of a source code release has to be handled like a release
 itself, and that there should be no download source of it outside of
 apache.
 This seems to be one motivation for the asf having its own maven
 repository.

 I seem to misunderstand something here, or why can there be apache maven
 artifacts in maven central and package in linux distributions for for
 example httpd, if this policy is followed? I mean it was even suggested
 to
 use the trademark to forbid the distribution through third parties. I am
 quite irritated about this.

 bye blackdrag


 I am not aware of any policy that dictates that (but would love to see
 links.)


 yeah, next time I will do that better. Getting the stuff out of here, will
 require me reading thousands of mails through that stupid web interface and
 google doesn't help either.

 I am aware that releases MUST at least be distributed via
 dist.apache.org [1], but that isn't exclusive, meaning the PMC is
 welcome to distribute _released software_ via other means (PyPy, NPM,
 Maven, Docker Registry, CPAN, Bintray, carrier pigeon, etc).

 --David
 [1] http://www.apache.org/dev/release.html#where-do-releases-go


 The problem already starts with that what a release is on
 http://www.apache.org/dev/release.html

 I read that as anything that goes beyond the dev-list is to be handled as
 release. It does not say by whom. And there is no mentioning of the
 releasing of released software, only the distribution of releases.

As you probably remember we've discussed this issue not that long time
ago: http://thread.gmane.org/gmane.comp.apache.incubator.general/49852

The consensus there is that as long as you're communicating intent
clearly you can let downstream developers test/develop against your
development artifacts. IOW, the definition of developers starts including
downstream developers integrating with your project.

 But anyway... le tme phrase some scenarios and question:

 Let us assume httpd makes the release 2.4.10, a linux distributor takes the
 source, adapts them (for example security patches), compiles packages out of
 it and releases it as http://packages.ubuntu.com/vivid-updates/apache2-bin
 in source and binary form. Then it means they took a release and made their
 own release out of it, while using the apache name.

Correct. At that point it constitutes a derived work. The Apache License is
very permissive that way, but it is considered a good practice to distinguish
the derived work by at leas a version ID.

That is also, how all of the Hadoop ecosystem vendors are creating dervived
works when they distribute Apache Hadoop as part of their products. E.g.
the version string of Cloudera's Hadoop is: ASF_VERSION-CDH_VERSION.
This is in line with most of the Linux packaging guidelines as well.

 The point being here, for the end-user this will be
 the official release, not what is found on the apache servers. Why is this
 ok?

Because technically it is an artifact that is a derived work.

 It was also mentioned here, that for example publishing snapshot builds to
 maven central is not allowed.

This is where it gets tricky with our current policy. Personally I see
absolutely
no reason to NOT publish Maven artifacts as widely as possible provide
that the version ID or name communicates the intent. It seems, however,
that I'm in a minority here (although truth be told nobody has been able
to communicate a convincing enough argument for why my approach may
be dangerous to the foundation and/or Projects).

 What would happen if a third party would do this? Is the project/apache
 required to do something about this? I mean if you read this:
 http://mail-archives.apache.org/mod_mbox/incubator-general/201506.mbox/%3CD1B01671.4EE90%25rvesse%40dotnetrdf.org%3E
 some even see nightly builds, not communicated beyond the dev-list on
 non-apache servers already as a problem.

Third party is at complete liberty of doing so. Provide the artifact is marked
in such a way that is can NOT be confused with an official ASF artifact
(IOW it can be called a derived work).

Again, this happens all the time with Hadoop vendors. Their Maven repos
host -SNAPSHOTS of essentially re-built ASF projects.

 Let us put that last part a step up... Let us assume someone takes one of
 the released sources of one of the java projects out there, makes maven
 artifacts out of it and publishes them at maven central. Is that ok? I mean
 that is very near the distributor case, so it should be ok, or not?

I honestly see no problem with that, again provided that the artifact can NOT
be confused with the one coming from Apache project.

Thanks,