Re: [1/2] ant git commit: Deprecate CollectionUtils and Enumerations; reduce explicit use of Enumeration

[Johan Corveleyn]

On Fri, May 18, 2018 at 10:07 AM, Gintautas Grigelionis
 wrote:
> I accepted the original criticism that going Enumeration -> List -> Stream
> has an overhead and I tried to address that by using a decorator.
> I believe that Streams API can at least implement the logic run by an
> original Enumeration in a more concise way, or provide more powerful idioms.
> That IMO makes it worth the while to investigate the Streams alternatives.
>
> In the process I also found out that other iterators could be used in a
> more efficient way, or that there is a redundant object construction going
> on.
> If you wish, I may spend some time to describe the changes I introduced,
> and maybe then we could discuss the their merits.
>
> Gintas

As a lurker on this list, but also Java developer in $dayjob [1], I'd
like to add my 2 cents, FWIW.

I see two things going on that are causing a lot of irritation / frustration:

1) Large codebase-wide changes for adapting old (well working) code to
new API's, syntax, idioms, ... usually mostly cosmetic. From my
personal perspective (I know, I carry little weight here, I'm just an
innocent bystander) this is not a good idea. Except in specific cases
for example for security reasons, or where there is a clear benefit
also for users (for example it improves performance significantly in
certain scenarios -- Note that I say "significantly", because shaving
off 1 byte off an operation that's only run once by normal users is
not worth the code churn / review effort / risk (yes that means you
need to measure and show the significant improvement if you want to
use this as an argument)).

I remember years ago we had a similar discussion at $dayjob, when
someone proposed to standardize our rules for grouping import
statements. "Should we now bulk-change our entire codebase?" After
discussion the consensus was "no" (needless code churn, review burden,
possible future backport conflicts, ...), except when you're going to
edit the class anyway. So the simple rule was: "use the new style in
new code, and in code that you're going to change for 'external
reasons' anyway".


Which brings me to the more important problem:

2) There is clearly no consensus here on the merits / desirability of
these bulk changes. Gintas, I understand your desire to move things
along, and you clearly have an opinion about it, but IMHO you really
need to stop doing any more bulk changes as long as there is no
consensus.

Several people have voiced objections on this lists ([2], [3], [4],
...), so why are you continuing to do this? When people start raising
flags you really should pause for a moment, and start discussing
before making more commits. By making further disputable commits you
are making the discussion more difficult (it's much more difficult to
freely speak your mind if the code has already been changed ... the
work has already been done and it's even more work to revert it, ...).


[1] Full disclosure: I work at the same company as Maarten Coene. I'm
also an ASF member and PMC member of the Subversion project, but that
doesn't really matter here.

[2] 
https://mail-archives.apache.org/mod_mbox/ant-dev/201804.mbox/%3C5682399f-c52e-49bd-a48b-740feaf60653%40apache.org%3E
and its entire thread
[3] 
https://mail-archives.apache.org/mod_mbox/ant-dev/201804.mbox/%3C001301d3e070%246555ff10%243001fd30%24%40de%3E
and previous answers by other committers in that same thread.
[4] 
https://mail-archives.apache.org/mod_mbox/ant-dev/201805.mbox/%3C87efitnm5f.fsf_-_%40v45346.1blu.de%3E

-- 
Johan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: Ivy - any future or is it also going to be retired?

[Johan Corveleyn]

On Mon, Dec 5, 2016 at 11:33 AM, Jaikiran Pai  wrote:
>
>
> On Monday 05 December 2016 03:26 PM, Gintautas Grigelionis wrote:
>>
>> If you want to push Ivy, you need integrations with IDE.
>
>
> +1
>
> I haven't voted in that mail about retiring IvyDE because I don't know what
> a -1 (which is what I would have voted) would do in reality. As noted in
> that other voting thread, the last release was in 2013 and last real code
> was in 2014. So on one hand I want that project to stay (and hope to be
> actively developed) and on the other I don't think that's going to be happen
> even if it isn't retired.
>
>
> Overall, IMO, if Ivy project itself needs to be used in serious projects,
> there needs to be an active community and usable ecosystem around it, headed
> by developers who are able to invest time and interest in it.

An innocent bystander's opinion (I'm not an ivy / ant developer, but
I'm active in the Apache Subversion community): First of all, this is
a volunteer-driven open source project. You're entirely free to use it
or to use something else ... no hard feelings. Second, I'm guessing
the ivy (and ant) project would very much welcome extra hands. So if
this project is important to you, and you want a release to happen ...
why don't you join the effort and help drive it. IMHO, if serious
projects need it, they should also put in serious time, and start
scratching some of their own itches (and, you know, become a
substantial part of the active community you're asking for here).

If right now the problem is "there is noone able to create / manage
new releases" (but some people are still around to watch over the code
and fix bugs), maybe someone can step up specifically for the job of
release manager ...


-- 
Johan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: ant wrapper script testing

[Johan Corveleyn]

On Mon, Oct 31, 2016 at 9:20 PM, Jeff Adamson  wrote:
>
> We will give this more thought. Ant only needs one replacement engine to work 
> on all platforms, but I don't know of a specific reason that nawk should not 
> be just as capable as sed.
>
> To summarize,
>
> Using 4-backslash replacement nawk on ubuntu:
>
> arg : -Dfoo=dollar$_backtick`_single'_double"_trailingbackslash\
> quoted_arg: "-Dfoo=dollar\$_backtick\`_single'_double\"_trailingbackslash\\"
> 
> [echo] foo=dollar$_backtick`_single'_double"_trailingbackslash\_
>
> Using 8-backslash replacement nawk on ubuntu:
>
> arg : -Dfoo=dollar$_backtick`_single'_double"_trailingbackslash\
> quoted_arg: "-Dfoo=dollar\$_backtick\`_single'_double\"_trailingbackslash"
> 
> [echo] foo=dollar$_backtick`_single'_double"_trailingbackslash\\_
>
> Using 4-backslash replacement nawk(?) on your machine:
>
> arg   : -Dfoo=trail\ingbackslash\
> quoted_arg: "-Dfoo=trail\ingbackslash\"
> 
> error output (clearly failed to double the backslash during quoting)
>
> Using 8-backslash replacement nawk on your machine:
>
> arg   : -Dfoo=trail\ingbackslash\
> quoted_arg: "-Dfoo=trail\\ingbackslash\\"
> ...
> arg : -Dfoo=trail\ingbackslash\
> quoted_arg: "-Dfoo=trail\\ingbackslash\\"
>
>
> The goal of the line is to replace one backslash literal with two. The other 
> awk patterns are correctly replacing with two backslashes in the replacement 
> pattern; introducing a single new backslash literal in the output. Not sure 
> why this backslash should be different. I think 4-backslash is the correct 
> number unless something is doing an extra pass of interpretation of the 
> argument.
>
>
> Maybe the slash-string pattern is being interpreted differently. Can we try 
> it as just plain double-quote strings?
>
> 'awk')
> esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub("", ""); 
> print }' )"
> esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub("\\$", "\\$"); print 
> }' )"
> esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub("\"", "\\\""); print 
> }' )"
> esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub("`", "\\`"); print }' 
> )"

I'm afraid that doesn't work either:

[[
bash-4.1> diff ant.new ant.new.edited
63,66c63,66
< esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{
gsub(/\\/, ""); print }' )"
< esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{
gsub(/\$/, "\\$");  print }' )"
< esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{
gsub(/\"/, "\\\""); print }' )"
< esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub(/`/,
 "\\`");  print }' )"
---
> esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub("", 
> ""); print }' )"
> esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub("\\$", 
> "\\$");  print }' )"
> esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub("\"", 
> "\\\""); print }' )"
> esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub("`",  "\\`"); 
>  print }' )"
bash-4.1> ./ant.new.edited --execdebug "-Dfoo=trail\\ingbackslash\\" -f test.xml
arg   : -Dfoo=trail\ingbackslash\
quoted_arg: "-Dfoo=trail\ingbackslash\"
arg   : -f
quoted_arg: "-f"
arg   : test.xml
quoted_arg: "test.xml"
exec "$JAVACMD" -Xmx512m -Djava.awt.headless=true -XX:MaxPermSize=140m
-classpath "$LOCALCLASSPATH" -Dant.home="$ANT_HOME"
-Dant.library.dir="$ANT_LIB"  org.apache.tools.ant.launch.Launcher
-cp "$CLASSPATH"  "-Dfoo=trail\ingbackslash\" "-f" "test.xml"
Buildfile: build.xml does not exist!
Build failed
]]


I think it would indeed be better to focus on 'sed' only now, because
we can get it to work well cross-platform. No need to make it also
work with (n)awk.

-- 
Johan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: ant wrapper script testing

[Johan Corveleyn]

On Fri, Oct 28, 2016 at 10:41 PM, Johan Corveleyn <jcor...@gmail.com> wrote:
...
> Yes, that works perfectly! On Solaris 11, with your latest version of
> the ant wrapper. When I run your test.sh, with the test.xml, I get
> exactly the correct output.
>
> However, with esc_tool=awk it fails. Don't know why yet, but the first
> property makes it fail already:
>
> [[
> bash-4.1> ./ant.new
> "-Dfoo=dollar\$_backtick\`_single'_double\"_trailingbackslash\\" -f
> test.xml
> Buildfile: build.xml does not exist!
> Build failed
> ]]
>
> With -version there is no problem, and running with '-f test.xml'
> without properties also works (and fwiw, "-Ddoublespace=  " also
> works).

Okay, so the problem with the awk variant on Solaris is only with the
trailing backslash. If I remove that from the test input, all other
tests work.

This change (doubling the backslashes even more) seems to fix it, but
I have no idea if it breaks the awk variant on other platforms:

[[
bash-4.1> diff ant.new ant.new.edited
63c63
< esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{
gsub(/\\/, ""); print }' )"
---
> esc_arg="$(printf '%s' "$esc_arg" | "$awk_exec" '{ gsub(/\\/, 
> ""); print }' )"
]]


Here is output of the awk variant with your version:

[[
bash-4.1> ./ant.new --execdebug "-Dfoo=trail\\ingbackslash\\" -f test.xml
arg   : -Dfoo=trail\ingbackslash\
quoted_arg: "-Dfoo=trail\ingbackslash\"
arg   : -f
quoted_arg: "-f"
arg   : test.xml
quoted_arg: "test.xml"
exec "$JAVACMD" -Xmx512m -Djava.awt.headless=true -XX:MaxPermSize=140m
-classpath "$LOCALCLASSPATH" -Dant.home="$ANT_HOME"
-Dant.library.dir="$ANT_LIB"  org.apache.tools.ant.launch.Launcher
-cp "$CLASSPATH"  "-Dfoo=trail\ingbackslash\" "-f" "test.xml"
Buildfile: build.xml does not exist!
Build failed
]]


And with ant.new.edited:

[[
bash-4.1> ./ant.new.edited --execdebug "-Dfoo=trail\\ingbackslash\\" -f test.xml
arg   : -Dfoo=trail\ingbackslash\
quoted_arg: "-Dfoo=trail\\ingbackslash\\"
arg   : -f
quoted_arg: "-f"
arg   : test.xml
quoted_arg: "test.xml"
exec "$JAVACMD" -Xmx512m -Djava.awt.headless=true -XX:MaxPermSize=140m
-classpath "$LOCALCLASSPATH" -Dant.home="$ANT_HOME"
-Dant.library.dir="$ANT_LIB"  org.apache.tools.ant.launch.Launcher
-cp "$CLASSPATH"  "-Dfoo=trail\ingbackslash\" "-f" "test.xml"
Buildfile: /data/tools/ant/apache-ant-1.9.7/bin/test.xml

run:
 [echo] hello world
 [echo] foo=trail\ingbackslash\_
 [echo] bar=${bar}_
 [echo] nl=${nl}_
 [echo] nlnl=${nlnl}_
 [echo] nlnlnl=${nlnlnl}_
 [echo] doublespace=${doublespace}_
 [echo] end

BUILD SUCCESSFUL
Total time: 0 seconds
]]

-- 
Johan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: ant wrapper script testing

[Johan Corveleyn]

On Wed, Oct 26, 2016 at 4:16 PM, Jeff Adamson  wrote:
>
> I am feeling pretty good as the introduction of the newline to the printf
> does not fundamentally change much about the updated processing. My main
> concern was if that additional newline was going to make it into the final
> arguments as extra text. This does not seem to be the case on mac or
> ubuntu, so I guess sed must always eat the last newline when piped in this
> manner.
> I think the most important takaway from the 1.9.7 release are the quoting
> of the final eval, the padding the argument before escaping to support
> empty and whitespace, and the newline for the sed input on Solaris.
>
> Not sure if attachments would go through mailing list so I'll just copy and
> paste inline as they are short.
>
> My ant file is test.xml. Used to echo's a bunch of input properties. The
> trailing _ is to delimit the value to make it readable and prevent ant echo
> from trimming the output:
>
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>
> My test.sh passes a variety of characters as the properties to be echoed.
> Includes literals that use an unballanced single-quote, double-quote,
> backtick, dollar, backslash (trailing on value). Also properties with 1, 2,
> and 3 consecutive trailing newline literals to see if they come through
> accurately. The execdebug flag shows each original argument and the
> quoted/escaped form of the argument used within eval.
>
> =
> #!/bin/sh
>
> ant >/dev/null || exit 1
>
> # literal newline char
> NL="
> "
>
> ./dist/bin/ant --execdebug \
>   "-Dfoo=dollar\$_backtick\`_single'_double\"_trailingbackslash\\" \
>   "-Dbar=trailingnewline$NL" \
>   "-Dnl=$NL" \
>   "-Dnlnl=$NL$NL" \
>   "-Dnlnlnl=$NL$NL$NL" \
>   "-Ddoublespace=  " \
>   "-Dx=y" \
>   "-f" "test.xml"
> =
>
> Expected output is:
>
> Buildfile: /home/jwa/Developer/projects/apache/ant/test.xml
>
> run:
>  [echo] hello world
>  [echo] foo=dollar$_backtick`_single'_double"_trailingbackslash\_
>  [echo] bar=trailingnewline
>  [echo] _
>  [echo] nl=
>  [echo] _
>  [echo] nlnl=
>  [echo]
>  [echo] _
>  [echo] nlnlnl=
>  [echo]
>  [echo]
>  [echo] _
>  [echo] doublespace=  _
>  [echo] end
>
>
>
> Regards,
> Jeff Adamson

Yes, that works perfectly! On Solaris 11, with your latest version of
the ant wrapper. When I run your test.sh, with the test.xml, I get
exactly the correct output.

However, with esc_tool=awk it fails. Don't know why yet, but the first
property makes it fail already:

[[
bash-4.1> ./ant.new
"-Dfoo=dollar\$_backtick\`_single'_double\"_trailingbackslash\\" -f
test.xml
Buildfile: build.xml does not exist!
Build failed
]]

With -version there is no problem, and running with '-f test.xml'
without properties also works (and fwiw, "-Ddoublespace=  " also
works).

-- 
Johan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: ant wrapper script testing

[Johan Corveleyn]

On Tue, Oct 25, 2016 at 9:29 PM, Jeff Adamson  wrote:
>
> Oops, had the nawk check inverted.
>
> should have been 
> https://github.com/jwadamson/ant-1/commit/c1ac08abcaad0030064d62933dda278b5326f92b
>

Great, it works on Solaris 11, both with 'sed' and 'awk' as esc_tool. Nice job!

Now, I only tested 'ant -version'. I've actually not been following
the original discussion that lead to all the changes in the wrapper
script (I only noticed that 1.9.7 was broken on Solaris for simple
inputs). Are there some specific inputs you would like me to test on
Solaris? Or is there perhaps some automatic test I can run for
exercising the various inputs to the script?

-- 
Johan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: ant wrapper script testing

[Johan Corveleyn]

On Tue, Oct 25, 2016 at 3:50 PM, Johan Corveleyn <jcor...@gmail.com> wrote:
> On Sat, Oct 22, 2016 at 4:11 PM, Stefan Bodewig <bode...@apache.org> wrote:
>> On 2016-10-18, Jeff Adamson wrote:
>>
>>>   I made a new pull request based on the ongoing work.
>>
>>> https://github.com/apache/ant/pull/25
>>
>> Yes, thanks. I've merged it.
>>
>>> What do you think and can we get some users to test behaviors on a variety
>>> of platforms?
>>
>> I really hope we can.
>
> I've tested 1.9.7 with the above change to the ant wrapper script
> (I've copy pasted the "raw" version of the merge to 1.9.x of the above
> pull request) on Solaris 11, but it doesn't work. Not with
> esc_tool=sed and neither with esc_tool=awk. It's easy to test by
> simply executing "ant -version".
>
> See the output below:
>
> [[[
> bash-4.1> uname -a
> SunOS hostname-redacted 5.11 11.3 i86pc i386 i86pc
> bash-4.1> ./ant.new -version   ### with esc_tool=sed
> Buildfile: build.xml does not exist!
> Build failed
> bash-4.1> vi ant.new   ### change line 25 to esc_tool=awk
> bash-4.1> ./ant.new -version
> awk: syntax error near line 1
> awk: illegal statement near line 1
> awk: syntax error near line 1
> awk: illegal statement near line 1
> awk: syntax error near line 1
> awk: illegal statement near line 1
> awk: syntax error near line 1
> awk: illegal statement near line 1
> Buildfile: build.xml does not exist!
> Build failed
> ]]]
>
> I don't understand the sed command (I'm not much of a sed expert), but
> I understand why the awk command fails: on Solaris "awk" really is the
> "old awk", without the "new awk" extensions added in 1984 or so. This
> means "gsub" is not available in "awk" on Solaris. To use gsub, you
> have to invoke "nawk" on Solaris.
>
> So if you want the awk variant to work on Solaris, you either have to
> do it without gsub, or use nawk as executable (but I guess nawk won't
> work on some other unix platforms).
>
> It would probably be easier to make the sed statement work, but I have
> to experiment a bit with that.

OK, so the problem is that sed on Solaris does not process lines that
don't end with a newline. So if I add a '\n' to the printf statement
that's piped to sed, on line 49, then it works on Solaris. I don't
know if that breaks other platforms of course. See patch below:

[[[
--- ant.new Tue Oct 25 16:01:51 2016
+++ ant.new.edited  Tue Oct 25 16:02:19 2016
@@ -46,7 +46,7 @@
 case "$esc_tool" in
   'sed')
 #  mac sed does not support group-0, so pattern uses group-1
-esc_arg="$(printf '%s' "$esc_arg" | sed -e 's@\([$"\\`]\)@\\\1@g')"
+esc_arg="$(printf '%s\n' "$esc_arg" | sed -e 's@\([$"\\`]\)@\\\1@g')"
 ;;
   'awk')
 esc_arg="$(printf '%s' "$esc_arg" | awk '{ gsub(/\\/,
""); print }' )"

]]]

Then at least 'ant -version' works on Solaris.

-- 
Johan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: ant wrapper script testing

[Johan Corveleyn]

On Sat, Oct 22, 2016 at 4:11 PM, Stefan Bodewig  wrote:
> On 2016-10-18, Jeff Adamson wrote:
>
>>   I made a new pull request based on the ongoing work.
>
>> https://github.com/apache/ant/pull/25
>
> Yes, thanks. I've merged it.
>
>> What do you think and can we get some users to test behaviors on a variety
>> of platforms?
>
> I really hope we can.

I've tested 1.9.7 with the above change to the ant wrapper script
(I've copy pasted the "raw" version of the merge to 1.9.x of the above
pull request) on Solaris 11, but it doesn't work. Not with
esc_tool=sed and neither with esc_tool=awk. It's easy to test by
simply executing "ant -version".

See the output below:

[[[
bash-4.1> uname -a
SunOS hostname-redacted 5.11 11.3 i86pc i386 i86pc
bash-4.1> ./ant.new -version   ### with esc_tool=sed
Buildfile: build.xml does not exist!
Build failed
bash-4.1> vi ant.new   ### change line 25 to esc_tool=awk
bash-4.1> ./ant.new -version
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 1
awk: illegal statement near line 1
Buildfile: build.xml does not exist!
Build failed
]]]

I don't understand the sed command (I'm not much of a sed expert), but
I understand why the awk command fails: on Solaris "awk" really is the
"old awk", without the "new awk" extensions added in 1984 or so. This
means "gsub" is not available in "awk" on Solaris. To use gsub, you
have to invoke "nawk" on Solaris.

So if you want the awk variant to work on Solaris, you either have to
do it without gsub, or use nawk as executable (but I guess nawk won't
work on some other unix platforms).

It would probably be easier to make the sed statement work, but I have
to experiment a bit with that.

-- 
Johan

-
To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org
For additional commands, e-mail: dev-h...@ant.apache.org

Re: migration to git and web site publication

[Johan Corveleyn]

On Tue, Apr 29, 2014 at 9:32 AM, Dominique Devienne ddevie...@gmail.com
wrote:
 On Tue, Apr 29, 2014 at 5:00 AM, Antoine Levy Lambert anto...@gmx.de
wrote:
 On Apr 28, 2014, at 1:29 AM, Jan Matèrne (jhm) apa...@materne.de wrote:
 The web sites will remain in svn in any event because svnpubsub is the
 only supported mechanism to maintain web sites AFAIK.
 [...] git/gitpubsub is working?

  Apache Infra does not support gitpubsub for [...] web site.

 git is not the tool of choice to deal with large files such as the ones
on dist.apache.org.
 That might be part of the reason why infra supports only svn for the web
site.

 Just for info, what makes you say that? Why would git be worse than
 SVN for large files?

 Is that because all clones must copy all versions (compressed deltas
 I guess) of those large files? --DD

Various people indicate that git doesn't work well with large binary files:
https://www.google.com/#q=git+large+binary+files

-- 
Johan