[freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
It doesn't look like a big deal but if all of Fred was using lazy evaluation for logging it would make one common use case very annoying: Say I'm working on a specific class and need to run that class with debug logging enabled. This would have the side effect of changing the compiled LOG.debug(

[freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
As to why it still creates garbage even if you do not touch the arguments in the testWithout method, I would guess that different inlining rules apply to large methods, small methods and methods that are already inlined within other methods. To give definitive answer I'll have to look through the

[freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
Your test has nothing to do with stack allocation because you never use the objects that you pass to the log_info method so JIT removes them. Apply the following patch and see yourself create garbage in both testWith and testWithout. --- Test2.java 2012-03-23 23:01:05.540475497 -0400 +++ Test.ja

[freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
Wrong. Extracting the log statement into its own method and calling that method inside the while loop still creates garbage. Stack frames are different. Read again exactly how Escape Analysis works. The objects it prevents from creating do not appear in heap histograms. Take a "jmap -histo " a

[freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
The only bug-vector I see in the current predicate approach is that when used incorrectly it cause the size of methods to explode. There are some hard-coded thresholds for the method size in bytecodes that turn off some or all JIT optimizations; there are also a whole host of compilation parameter

[freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
Not really. Here, I'll change my call to allocate everything on one level: while (true) { log(" list size is {1} ", new Integer(listToLog.size())); } I don't see how using this you can log primitive values without garbage. On Fri, Mar 23, 2012 at 9:33 P

[freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
You're right, the example you gave will put the arguments on the stack. I wanted to clarify Escape Analysis in its current form works only if a new object doesn't exit the stack frame where it's created. Because of that, the moment you do something slightly more complicated you will create garbage

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
It doesn't look like a big deal but if all of Fred was using lazy evaluation for logging it would make one common use case very annoying: Say I'm working on a specific class and need to run that class with debug logging enabled. This would have the side effect of changing the compiled LOG.debug(

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Ximin Luo
I see. Actually we didn't even need the side effects thing - I did notice that your example stopped doing GC when setting "shouldLog = false" always, so I tried to introduce your "if (shouldLog) { shouldLog = false; }" into my example, but was still unable to reproduce GC. Then changing - System.

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
As to why it still creates garbage even if you do not touch the arguments in the testWithout method, I would guess that different inlining rules apply to large methods, small methods and methods that are already inlined within other methods. To give definitive answer I'll have to look through the

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
Your test has nothing to do with stack allocation because you never use the objects that you pass to the log_info method so JIT removes them. Apply the following patch and see yourself create garbage in both testWith and testWithout. --- Test2.java 2012-03-23 23:01:05.540475497 -0400 +++ Test.ja

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Ximin Luo
On 24/03/12 02:44, Zlatin Balevsky wrote: > Wrong. Extracting the log statement into its own method and calling > that method inside the while loop still creates garbage. Stack frames > are different. > How do you explain the results obtained from running my test code, then? Turn on -verbose:gc

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
Wrong. Extracting the log statement into its own method and calling that method inside the while loop still creates garbage. Stack frames are different. Read again exactly how Escape Analysis works. The objects it prevents from creating do not appear in heap histograms. Take a "jmap -histo " a

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Ximin Luo
On 24/03/12 01:56, Zlatin Balevsky wrote: > Not really. Here, I'll change my call to allocate everything on one level: > > while (true) { > log(" list size is {1} ", > new Integer(listToLog.size())); > } > > I don't see how using this you can log primiti

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
The only bug-vector I see in the current predicate approach is that when used incorrectly it cause the size of methods to explode. There are some hard-coded thresholds for the method size in bytecodes that turn off some or all JIT optimizations; there are also a whole host of compilation parameter

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
Not really. Here, I'll change my call to allocate everything on one level: while (true) { log(" list size is {1} ", new Integer(listToLog.size())); } I don't see how using this you can log primitive values without garbage. On Fri, Mar 23, 2012 at 9:33 P

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Ximin Luo
Sorry, bad typing. "inside its own" not "inside its only". On 24/03/12 01:33, Ximin Luo wrote: > That is because in your example you're allocating all those new Object()s in > the same stack frame, so the allocator runs out of space on the stack. > > If you put the call to log(x, new Object()) in

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Ximin Luo
That is because in your example you're allocating all those new Object()s in the same stack frame, so the allocator runs out of space on the stack. If you put the call to log(x, new Object()) inside its only function call, each Object will be destroyed once that call exits, and no GC occurs. Exam

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Zlatin Balevsky
You're right, the example you gave will put the arguments on the stack. I wanted to clarify Escape Analysis in its current form works only if a new object doesn't exit the stack frame where it's created. Because of that, the moment you do something slightly more complicated you will create garbage

[freenet-dev] Logging in Fred

2012-03-23 Thread Matthew Toseland
> > >>>>> > - Either lightweight or with optional features. > >>>>> Else, it would only > >>>>> > transfer bloat to freenet-ext.jar. For example: > >>>>> log2socket, config > >>>>> > management and multiple logging instances; > >>>>> > - Implementable in a few LoC. Specially, it > >>>>> shouldn't need specialized > >>>>> > Formatter and Writer. > >>>>> > > >>>>> > Plus, it should be fast. > >>>>> > > >>>>> > From the quick research I made (yep, too many > >>>>> lists): > >>>>> > > >>>>> > - SLF4J already fails on point one: it is simply a > >>>>> wrapper; > >>>>> > - The Java logging API fails on point two: > >>>>> specialized classes would > >>>>> > have to be written to deal with log rotation, > >>>>> per-class filtering and > >>>>> > formatting, plus a wrapper for > >>>>> Logger.{info,warning,...}() methods. > >>>>> > Exactly the same as a custom logger, with one more > >>>>> dependency and using > >>>>> > more LoC; > >>>>> > > >>>>> > No dependancies, it's part of the JDK, isn't it? > >>>>> > > >>>>> > More classes need to be loaded at startup. It's just me > >>>>> thinking too much. > >>>>> > > >>>>> > > >>>>> > However, if it's not a clearer/simpler API, it > >>>>> probably > >>>>> doesn't make > >>>>> > much sense. > >>>>> > > >>>>> > - Log4J seems to fail on point one - it only > >>>>> lacks a > >>>>> button that brings > >>>>> > back the dead. It seems interesting, and I haven't > >>>>> dropped this yet. > >>>>> > > >>>>> > In either case (custom or external), log* would be > >>>>> banished. Forever. > >>>>> > > >>>>> > I don't follow. You object to using a separate logs > >>>>> folder? > >>>>> > > >>>>> > log* == log{MINOR,DEBUG}, not the logs folder. > >> > > -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/a2034fe0/attachment.pgp>

[freenet-dev] Logging in Fred

2012-03-23 Thread Matthew Toseland
be fast. > >>> > > >>> >From the quick research I made (yep, too many > >>> lists): > >>> > > >>> > - SLF4J already fails on point one: it is simply a > >>> wrapper; > >>> > - The Java logging API fails on point two: > >>> specialized classes would > >>> > have to be written to deal with log rotation, > >>> per-class filtering and > >>> > formatting, plus a wrapper for > >>> Logger.{info,warning,...}() methods. > >>> > Exactly the same as a custom logger, with one more > >>> dependency and using > >>> > more LoC; > >>> > > >>> > No dependancies, it's part of the JDK, isn't it? > >>> > > >>> > More classes need to be loaded at startup. It's just me > >>> thinking too much. > >>> > > >>> > > >>> > However, if it's not a clearer/simpler API, it probably > >>> doesn't make > >>> > much sense. > >>> > > >>> > - Log4J seems to fail on point one - it only lacks a > >>> button that brings > >>> > back the dead. It seems interesting, and I haven't > >>> dropped this yet. > >>> > > >>> > In either case (custom or external), log* would be > >>> banished. Forever. > >>> > > >>> > I don't follow. You object to using a separate logs > >>> folder? > >>> > > >>> > log* == log{MINOR,DEBUG}, not the logs folder. > > -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/c4ee8c15/attachment.pgp>

[freenet-dev] Refactoring Freenet and Library was Re: Gun.IO and Freenet

2012-03-23 Thread Matthew Toseland
lass-Path in the jar. > > > > You are planning to have a packaged version which is able to > > auto-update-over-Freenet? > > > > Currently class-path hardcoded freenet-ext.jar for convenience more than > > much else. 99% of installs use the wrapper. > > Any software package should support being installed into any arbitrary layout. > It's just good practise. You have to tell it where everything is somehow. At a minimum that involves a config file. If we are using the wrapper we can probably find the jars from that. If we are not I guess they could be configurable but it's messy IMHO. See, different definitions of elegance - don't impose yours on me if you don't want me to impose mine on you! Alternatively we could come to a civil compromise. -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/f4d5de49/attachment.pgp>

[freenet-dev] Refactoring Freenet and Library was Re: Gun.IO and Freenet

2012-03-23 Thread Matthew Toseland
- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/50876992/attachment.pgp>

[freenet-dev] Logging in Fred

2012-03-23 Thread Matthew Toseland
us a wrapper for > > Logger.{info,warning,...}() methods. > > > Exactly the same as a custom logger, with one more > > dependency and using > > > more LoC; > > > > > > No dependancies, it's part of the JDK, isn't it? > > > > > > More classes need to be loaded at startup. It's just me > > thinking too much. > > > > > > > > > However, if it's not a clearer/simpler API, it probably > > doesn't make > > > much sense. > > > > > > - Log4J seems to fail on point one - it only lacks a > > button that brings > > > back the dead. It seems interesting, and I haven't > > dropped this yet. > > > > > > In either case (custom or external), log* would be > > banished. Forever. > > > > > > I don't follow. You object to using a separate logs folder? > > > > > > log* == log{MINOR,DEBUG}, not the logs folder. -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/1a822204/attachment.pgp>

[freenet-dev] Logging in Fred

2012-03-23 Thread Matthew Toseland
s a digitally signed message part. URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/18a90512/attachment.pgp>

[freenet-dev] Logging in Fred

2012-03-23 Thread Marco Schulze
Sure. Since the class name lookups will only be used for debugging, they can be as generic as necessary, as long it doesn't add any requirements to users. Match-on-load and match-on-demand are both fine. On 23-03-2012 13:19, Matthew Toseland wrote: > On Friday 23 Mar 2012 16:00:15 Marco Schulze

[freenet-dev] Logging in Fred

2012-03-23 Thread Marco Schulze
Yes, and yes. On 23-03-2012 12:43, Matthew Toseland wrote: > On Friday 23 Mar 2012 15:29:44 you wrote: >> Right now, the map is only used to list class thresholds which are >> different from the global threshold, which means it is empty 99% of the >> time. This is the simplest solution, but it als

[freenet-dev] Logging in Fred

2012-03-23 Thread Marco Schulze
Right now, the map is only used to list class thresholds which are different from the global threshold, which means it is empty 99% of the time. This is the simplest solution, but it also means that the possibility of lock contention is way higher. However, unless this proves to be very bad in

[freenet-dev] Logging in Fred

2012-03-23 Thread Ximin Luo
>>>>>https://emu.freenetproject.__org/cgi-bin/mailman/listinfo/__devl >>>>><https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl> >>>>> >>>>> >>>>> >>>>> ___ >>>>> Devl mailing list >>>>> Devl at freenetproject.org <mailto:Devl at freenetproject.org> >>><mailto:Devl at freenetproject.org <mailto:Devl at freenetproject.org>> >>>>> https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >>>> >>>> >>>>-- >>>>GPG: 4096R/5FBBDBCE >>>>https://github.com/infinity0 >>>>https://bitbucket.org/infinity0 >>>>https://launchpad.net/~infinity0 <https://launchpad.net/%7Einfinity0> >>>> >>>> >>>>___ >>>>Devl mailing list >>>>Devl at freenetproject.org <mailto:Devl at freenetproject.org> >>><mailto:Devl at freenetproject.org <mailto:Devl at freenetproject.org>> >>>>https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >>>> >>>> >>>> >>>> ___ >>>> Devl mailing list >>>> Devl at freenetproject.org <mailto:Devl at freenetproject.org> >>>> https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >>> >>> >>>-- >>>GPG: 4096R/5FBBDBCE >>>https://github.com/infinity0 >>>https://bitbucket.org/infinity0 >>>https://launchpad.net/~infinity0 <https://launchpad.net/%7Einfinity0> >>> >>> >>>___ >>>Devl mailing list >>>Devl at freenetproject.org <mailto:Devl at freenetproject.org> >>>https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >>> >>> >>> >>> >>> ___ >>> Devl mailing list >>> Devl at freenetproject.org >>> https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl >> >> >> -- >> GPG: 4096R/5FBBDBCE >> https://github.com/infinity0 >> https://bitbucket.org/infinity0 >> https://launchpad.net/~infinity0 >> >> ___ >> Devl mailing list >> Devl at freenetproject.org >> https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > ___ > Devl mailing list > Devl at freenetproject.org > https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -- GPG: 4096R/5FBBDBCE https://github.com/infinity0 https://bitbucket.org/infinity0 https://launchpad.net/~infinity0 -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/01dd04a7/attachment.pgp>

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Marco Schulze
Sure. Since the class name lookups will only be used for debugging, they can be as generic as necessary, as long it doesn't add any requirements to users. Match-on-load and match-on-demand are both fine. On 23-03-2012 13:19, Matthew Toseland wrote: On Friday 23 Mar 2012 16:00:15 Marco Schulze

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Matthew Toseland
On Friday 23 Mar 2012 16:00:15 Marco Schulze wrote: > Yes, and yes. > > On 23-03-2012 12:43, Matthew Toseland wrote: > > On Friday 23 Mar 2012 15:29:44 you wrote: > >> Right now, the map is only used to list class thresholds which are > >> different from the global threshold, which means it is emp

[freenet-dev] Logging in Fred

2012-03-23 Thread Nicolas Hernandez
Nicolas -- next part -- An HTML attachment was scrubbed... URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/05a86057/attachment.html>

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Marco Schulze
Yes, and yes. On 23-03-2012 12:43, Matthew Toseland wrote: On Friday 23 Mar 2012 15:29:44 you wrote: Right now, the map is only used to list class thresholds which are different from the global threshold, which means it is empty 99% of the time. This is the simplest solution, but it also means

[freenet-dev] Logging in Fred

2012-03-23 Thread David ‘Bombe’ Roden
Yes, kids, both your penisses are incredibly long. Now shut up and let the grown-ups do some refactoring. Greetings, David On 23.03.2012, at 04:47, Ximin Luo wrote: > LOL, are you kidding me? > > Please point to the exact lines of code that results in "double-digit > millisecond paus

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Matthew Toseland
On Friday 23 Mar 2012 15:29:44 you wrote: > Right now, the map is only used to list class thresholds which are > different from the global threshold, which means it is empty 99% of the > time. This is the simplest solution, but it also means that the > possibility of lock contention is way highe

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Marco Schulze
Right now, the map is only used to list class thresholds which are different from the global threshold, which means it is empty 99% of the time. This is the simplest solution, but it also means that the possibility of lock contention is way higher. However, unless this proves to be very bad in

[freenet-dev] Refactoring Freenet and Library was Re: Gun.IO and Freenet

2012-03-23 Thread Ian Clarke
. Ian. -- Ian Clarke Personal blog: http://blog.locut.us/ -- next part -- An HTML attachment was scrubbed... URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/968304fb/attachment.html>

[freenet-dev] Logging in Fred

2012-03-23 Thread Marco Schulze
No one is benchmarking JRE's JIT. If you look closer, there are figures with the average runtime without JIT, exactly to avoid overspecialization. With JIT, there is a reduction of 0.02% in runtime. Without JIT, this becomes 0.06%. Negligible to all but the few people who do everything in assem

Re: [freenet-dev] Refactoring Freenet and Library was Re: Gun.IO and Freenet

2012-03-23 Thread Matthew Toseland
On Thursday 22 Mar 2012 20:34:53 Ximin Luo wrote: > On 22/03/12 12:00, Matthew Toseland wrote: > > On Wednesday 21 Mar 2012 00:30:42 Ximin Luo wrote: > >> "Implicit dependency" means that changing some code in one area results in > >> a > >> logical error in the other, that is not immediately appa

Re: [freenet-dev] Refactoring Freenet and Library was Re: Gun.IO and Freenet

2012-03-23 Thread Matthew Toseland
On Friday 23 Mar 2012 13:23:50 Ian Clarke wrote: > On Thu, Mar 22, 2012 at 7:00 AM, Matthew Toseland > wrote: > > > Making stuff static is tempting but means we can't do > > multi-nodes-in-one-VM tests > > Have we ever actually done that? It seems like we're always making > decisions to accomod

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Matthew Toseland
On Friday 23 Mar 2012 00:18:02 Marco Schulze wrote: > I already have all but log rotation and async ready, and haven't yet > found a single benchmark supporting the use of a branch as the > performance holy grail. For example (outputting to /dev/null): > > public static void main (String[] args)

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Matthew Toseland
On Friday 23 Mar 2012 11:35:44 Ximin Luo wrote: > lol, OK. > > I am just annoyed with everyone that keeps arguing "we need to implement our > own generic X framework because of minor issue Y whose importance I am > over-inflating, even though countless other projects with similar requirements > ge

Re: [freenet-dev] Refactoring Freenet and Library was Re: Gun.IO and Freenet

2012-03-23 Thread Ian Clarke
On Thu, Mar 22, 2012 at 7:00 AM, Matthew Toseland wrote: > Making stuff static is tempting but means we can't do > multi-nodes-in-one-VM tests Have we ever actually done that? It seems like we're always making decisions to accomodate rare (and sometimes fictional) edge-cases. Ian. -- Ian Cl

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Ximin Luo
lol, OK. I am just annoyed with everyone that keeps arguing "we need to implement our own generic X framework because of minor issue Y whose importance I am over-inflating, even though countless other projects with similar requirements get around these adequately". To anyone doing refactoring wor

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Marco Schulze
No one is benchmarking JRE's JIT. If you look closer, there are figures with the average runtime without JIT, exactly to avoid overspecialization. With JIT, there is a reduction of 0.02% in runtime. Without JIT, this becomes 0.06%. Negligible to all but the few people who do everything in asse

[freenet-dev] Logging in Fred

2012-03-23 Thread Ximin Luo
nfo/devl > > > > > > -- > > GPG: 4096R/5FBBDBCE > > https://github.com/infinity0 > > https://bitbucket.org/infinity0 > > https://launchpad.net/~infinity0 > <https://launchpad.net/%7Einfinity0> &g

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread Nicolas Hernandez
On Fri, Mar 23, 2012 at 8:58 AM, David ‘Bombe’ Roden < bo...@pterodactylus.net> wrote: > Yes, kids, both your penisses are incredibly long. Now shut up and let the > grown-ups do some refactoring. > > > Greetings, > >David > > Thanks daddy ! Nicolas ___

Re: [freenet-dev] Logging in Fred

2012-03-23 Thread David ‘Bombe’ Roden
Yes, kids, both your penisses are incredibly long. Now shut up and let the grown-ups do some refactoring. Greetings, David On 23.03.2012, at 04:47, Ximin Luo wrote: > LOL, are you kidding me? > > Please point to the exact lines of code that results in "double-digit > millisecond paus

[freenet-dev] Logging in Fred

2012-03-23 Thread Ximin Luo
-- > GPG: 4096R/5FBBDBCE > https://github.com/infinity0 > https://bitbucket.org/infinity0 > https://launchpad.net/~infinity0 > > > ___ > Devl mailing list > Devl at freenetproject.org <mailto:Devl at freenetproject.org> > https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl > > > > ___ > Devl mailing list > Devl at freenetproject.org > https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl -- GPG: 4096R/5FBBDBCE https://github.com/infinity0 https://bitbucket.org/infinity0 https://launchpad.net/~infinity0 -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20120323/4fe16ae1/attachment.pgp>