Re: [HACKYSTAT-DEV-L] Weak/Soft/Phantom References

Tue, 19 Sep 2006 16:31:27 -0700

This is amusing! What started this whole conversational thread was a thread-related NPE in the ThreeKeyCache system which was produced by an enhancement I made this summer to _add_ hard references to the most recently added N elements in the cache. :-) This was a design pattern I'd seen in a couple of places to make the GC mechanism work more smoothly. 

Cheers,
Philip
--On Tuesday, September 19, 2006 11:48 AM -1000 "(Cedric) Qin ZHANG" <[EMAIL PROTECTED]> wrote: 


Philip mentioned that too. But I think it's a bad idea. Using strong reference 
requires
you to know when to release references. If you already know when you need to 
release
references, what's the point of using soft references.

Using strong reference to mess with GC is a recipe for disaster: (1) When you 
fine tune
your system to match GC behavior of JVM, you are stuck with that JVM forever, 
since gc
internal implementation is always changing. (2) It's also bad from code design 
point of
view.

-Cedric


Aaron Akihisa Kagawa wrote:

A co-worker had the idea that maybe we need a more sophisticated cache that 
uses a
combination of Strong and Soft References.  The idea would be that "newer" 
things in
the cache would have strong and soft references. To be sure that this newer 
data does
not get GC'd.  Then at some point when some threshold of time and size are 
exceeded
the oldest strong references would be released.

How does that sound?

thanks, Aaron

----- Original Message -----
From: "(Cedric) Qin ZHANG" <[EMAIL PROTECTED]>
Date: Tuesday, September 19, 2006 9:54 am
Subject: Re: [HACKYSTAT-DEV-L] Weak/Soft/Phantom References
To: [email protected]



My bad. It's already SoftReference. I don't why I had the pre-
conception
that WeakHashMap was used in ThreeKeyCache implementation.

I will probably do some research on the internal implementation of
Sun
GC on my 18-hour trip from here to NY, and report back if I have
any
useful finding.

By the way, GC in client and server JVM behaves differently. On
client
JVM, it's supposed to do small memory sweeps frequently in order to
hide
the latency from a user.

Cheers,

Cedric



Philip Johnson wrote:


Hmm... did I inadvertantly type 'weak' reference somewhere?


That's a mistake. The only non-strong reference (in the
ThreeKeyCache class, anyway) is a soft reference. And even soft
references were definitely reclaimed too fast, at least in my
trials under JDK 1.4 this summer when I was developing the
FileMetric DPD redesign.


What I've been reading recently in various articles is:

(a) there aren't supposed to be major differences between 1.4 and


1.5 with respect to this issue; and


(b) The JDK 1.5 GC does a better job of 'waiting' to reclaim


SoftReferences than prior GCs.


When I eventually get some spare cycles, I will profile Hackystat


and figure out what's currently the case.


Cheers,
Philip

----- Original Message -----
From: "(Cedric) Qin ZHANG" <[EMAIL PROTECTED]>
Date: Monday, September 18, 2006 10:57 pm
Subject: [HACKYSTAT-DEV-L] Weak/Soft/Phantom References
To: [email protected]




Philip mentioned that Java GC reclaims weakly referenced objects
too
fast, often times before the cached object could be reused.
According to
the IBM article listed below, it seems that it's just a matter


of


replacing WeakReference with SoftReference in Hackystat code. I


am


quite
interested in knowing why there need to be more than one type of
non-strong reference in Java. Do we have any Java GC expert on


this


list
that can probably shed light on this topic?

Thanks.

Cedric



Philip Johnson (JIRA) wrote:



    [ http://hackydev.ics.hawaii.edu:8080/browse/HACK-



796?page=comments#action_11837 ]



Philip Johnson commented on HACK-796:
-------------------------------------

I found a couple more links that might of use when doing


further





research on this issue:



Shows a soft reference based hashmap:




http://www.javaspecialists.co.za/archive/newsletter.do?issue=098&locale=en_US>>>


An article about soft references:
http://www-128.ibm.com/developerworks/java/library/j-jtp01246.html

A bug report indicating that maybe soft references don't get


GC'd





so fast in JDK 1.5:



http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6349930






NullPointerException in DailyProjectFileMetrics
-----------------------------------------------

         Key: HACK-796
         URL: http://hackydev.ics.hawaii.edu:8080/browse/HACK-796
     Project: Hackystat
        Type: Bug








    Reporter: Qin Zhang
    Assignee: Philip Johnson
    Priority: Critical
     Fix For: 7.5








To reproduce: use telemetry wall to hit the server hard.  It



seems that DailyProjectFileMetric is not written in a thread-


safe way.





09/17 13:13:18 DailyProjectFileMetric failed to initialize!
Hackystat-7 09-Sep-2006
java.lang.NullPointerException
        at java.util.LinkedList.remove(LinkedList.java:647)
        at java.util.LinkedList.removeLast(LinkedList.java:144)
        at



org.hackystat.core.kernel.cache.ThreeKeyCache.put(ThreeKeyCache.java:252)>>
at
org.hackystat.core.kernel.sensordata.SensorDataCache.getSensorData(SensorDataCache.ja
va:48)





        at



org.hackystat.sdt.filemetric.dailyproject.FileMetricRawDataMap.<init>(FileMetricRawDa
taMap.java:91)>>         at
org.hackystat.sdt.filemetric.dailyproject.DailyProjectFileMetric.initialize(DailyProj
ectFileMetric.java:117)





        at



org.hackystat.sdt.filemetric.dailyproject.DailyProjectFileMetric.getAllFileTypes(Dail
yProjectFileMetric.java:152)>>         at
org.hackystat.sdt.filemetric.reducer.LanguageFileMetricReducer.getAllFileTypes(Langua
geFileMetricReducer.java:101)





        at



org.hackystat.sdt.filemetric.reducer.LanguageFileMetricReducer.compute(LanguageFileMe
tricReducer.java:72)>>         at
org.hackystat.core.telemetry.reducer.TelemetryReducerManager.compute(TelemetryReducer
Manager.java:141)





        at



org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.resolveReducerCall(Telemetr
yEvaluator.java:380)>>         at
org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.resolveFunctionCall(Telemet
ryEvaluator.java:322)





        at



org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.resolveExpression(Telemetry
Evaluator.java:285)>>         at
org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.evaluate(TelemetryEvaluator
.java:56)





        at



org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.evaluate(TelemetryEvaluator
.java:140)>>         at
org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.evaluate(TelemetryEvaluator
.java:259)





        at



org.hackystat.core.telemetry.analysis.TelemetryReportAnalysis.prepareReportForJspRend
ering(TelemetryReportAnalysis.java:133)>>         at
org.hackystat.core.telemetry.analysis.TelemetryExpertAnalysis.process(TelemetryExpert
Analysis.java:146)





        at



org.hackystat.core.kernel.mvc.Controller.doPost(Controller.java:271)>>         
at
org.hackystat.core.kernel.mvc.Controller.doGet(Controller.java:300)





        at



javax.servlet.http.HttpServlet.service(HttpServlet.java:689)>>





  at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)



        at



org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterCha
in.java:252)>>         at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:
173)





        at



org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)>>
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)





        at



org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)>>
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)





        at



org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)>>
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)





        at



org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)>>
at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection
(Http11BaseProtocol.java:664)





        at



org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)>>
at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThrea
d.java:80)





        at



org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)>>
at java.lang.Thread.run(Thread.java:595)

Reply via email to