Re: [HACKYSTAT-DEV-L] Weak/Soft/Phantom References

Tue, 19 Sep 2006 14:48:55 -0700

Philip mentioned that too. But I think it's a bad idea. Using strong reference requires you to know when to release references. If you already know when you need to release references, what's the point of using soft references.
Using strong reference to mess with GC is a recipe for disaster: (1) When you fine tune your system to match GC behavior of JVM, you are stuck with that JVM forever, since gc internal implementation is always changing. (2) It's also bad from code design point of view. 

-Cedric


Aaron Akihisa Kagawa wrote:

A co-worker had the idea that maybe we need a more sophisticated cache that uses a combination of Strong and Soft References.  The idea would be that "newer" things in the cache would have strong and soft references. To be sure that this newer data does not get GC'd.  Then at some point when some threshold of time and size are exceeded the oldest strong references would be released. 

How does that sound? 


thanks, Aaron

----- Original Message -----
From: "(Cedric) Qin ZHANG" <[EMAIL PROTECTED]>
Date: Tuesday, September 19, 2006 9:54 am
Subject: Re: [HACKYSTAT-DEV-L] Weak/Soft/Phantom References
To: [email protected]


  

My bad. It's already SoftReference. I don't why I had the pre-

conception 
that WeakHashMap was used in ThreeKeyCache implementation.



I will probably do some research on the internal implementation of 
Sun 
GC on my 18-hour trip from here to NY, and report back if I have 
any 
useful finding.



By the way, GC in client and server JVM behaves differently. On 
client 
JVM, it's supposed to do small memory sweeps frequently in order to 
hide 
the latency from a user.


Cheers,

Cedric



Philip Johnson wrote:

    

Hmm... did I inadvertantly type 'weak' reference somewhere?  
      

That's a mistake. The only non-strong reference (in the 
ThreeKeyCache class, anyway) is a soft reference. And even soft 
references were definitely reclaimed too fast, at least in my 
trials under JDK 1.4 this summer when I was developing the 
FileMetric DPD redesign.
    

What I've been reading recently in various articles is:


(a) there aren't supposed to be major differences between 1.4 and 
      

1.5 with respect to this issue; and

    

(b) The JDK 1.5 GC does a better job of 'waiting' to reclaim 
      

SoftReferences than prior GCs.

    

When I eventually get some spare cycles, I will profile Hackystat 
      

and figure out what's currently the case. 
    

Cheers,
Philip

----- Original Message -----
From: "(Cedric) Qin ZHANG" <[EMAIL PROTECTED]>
Date: Monday, September 18, 2006 10:57 pm
Subject: [HACKYSTAT-DEV-L] Weak/Soft/Phantom References
To: [email protected]


  
      

Philip mentioned that Java GC reclaims weakly referenced objects 
too 
fast, often times before the cached object could be reused. 
According to 
the IBM article listed below, it seems that it's just a matter 
        

of 
    

replacing WeakReference with SoftReference in Hackystat code. I 
        

am 
    

quite 
interested in knowing why there need to be more than one type of 
non-strong reference in Java. Do we have any Java GC expert on 
        

this 
    

list 
that can probably shed light on this topic?


Thanks.

Cedric



Philip Johnson (JIRA) wrote:

    
        

    [ http://hackydev.ics.hawaii.edu:8080/browse/HACK-

      
          

796?page=comments#action_11837 ] 
    
        

Philip Johnson commented on HACK-796:
-------------------------------------


I found a couple more links that might of use when doing 
          

further 
    

      
          

research on this issue:

    
        

Shows a soft reference based hashmap:


      
          

http://www.javaspecialists.co.za/archive/newsletter.do?issue=098&locale=en_US>>>     
    

An article about soft references:
http://www-128.ibm.com/developerworks/java/library/j-jtp01246.html


A bug report indicating that maybe soft references don't get 
          

GC'd 
    

      
          

so fast in JDK 1.5:

    
        

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6349930



  
      
          

NullPointerException in DailyProjectFileMetrics
-----------------------------------------------

         Key: HACK-796
         URL: http://hackydev.ics.hawaii.edu:8080/browse/HACK-796
     Project: Hackystat
        Type: Bug

    
        
            

  
      
          

    Reporter: Qin Zhang
    Assignee: Philip Johnson
    Priority: Critical
     Fix For: 7.5

    
        
            

  
      
          

To reproduce: use telemetry wall to hit the server hard.  It 
        
            

seems that DailyProjectFileMetric is not written in a thread-

        

safe way.

    

    
        

09/17 13:13:18 DailyProjectFileMetric failed to initialize!
Hackystat-7 09-Sep-2006
java.lang.NullPointerException
        at java.util.LinkedList.remove(LinkedList.java:647)
        at java.util.LinkedList.removeLast(LinkedList.java:144)

        at 
        
            

org.hackystat.core.kernel.cache.ThreeKeyCache.put(ThreeKeyCache.java:252)>>     
    at 
org.hackystat.core.kernel.sensordata.SensorDataCache.getSensorData(SensorDataCache.java:48)

    

    
        

        at 
        
            

org.hackystat.sdt.filemetric.dailyproject.FileMetricRawDataMap.<init>(FileMetricRawDataMap.java:91)>>
         at 
org.hackystat.sdt.filemetric.dailyproject.DailyProjectFileMetric.initialize(DailyProjectFileMetric.java:117)

    

    
        

        at 
        
            

org.hackystat.sdt.filemetric.dailyproject.DailyProjectFileMetric.getAllFileTypes(DailyProjectFileMetric.java:152)>>
         at 
org.hackystat.sdt.filemetric.reducer.LanguageFileMetricReducer.getAllFileTypes(LanguageFileMetricReducer.java:101)

    

    
        

        at 
        
            

org.hackystat.sdt.filemetric.reducer.LanguageFileMetricReducer.compute(LanguageFileMetricReducer.java:72)>>
         at 
org.hackystat.core.telemetry.reducer.TelemetryReducerManager.compute(TelemetryReducerManager.java:141)

    

    
        

        at 
        
            

org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.resolveReducerCall(TelemetryEvaluator.java:380)>>
         at 
org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.resolveFunctionCall(TelemetryEvaluator.java:322)

    

    
        

        at 
        
            

org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.resolveExpression(TelemetryEvaluator.java:285)>>
         at 
org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.evaluate(TelemetryEvaluator.java:56)

    

    
        

        at 
        
            

org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.evaluate(TelemetryEvaluator.java:140)>>
         at 
org.hackystat.core.telemetry.evaluator.TelemetryEvaluator.evaluate(TelemetryEvaluator.java:259)

    

    
        

        at 
        
            

org.hackystat.core.telemetry.analysis.TelemetryReportAnalysis.prepareReportForJspRendering(TelemetryReportAnalysis.java:133)>>
         at 
org.hackystat.core.telemetry.analysis.TelemetryExpertAnalysis.process(TelemetryExpertAnalysis.java:146)

    

    
        

        at 
        
            

org.hackystat.core.kernel.mvc.Controller.doPost(Controller.java:271)>>         
at org.hackystat.core.kernel.mvc.Controller.doGet(Controller.java:300)

    

    
        

        at 
        
            

javax.servlet.http.HttpServlet.service(HttpServlet.java:689)>>   
        

  
    

  at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)

    
        

        at 
        
            

org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)>>
         at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)

    

    
        

        at 
        
            

org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)>>
         at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)

    

    
        

        at 
        
            

org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)>> 
        at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)

    

    
        

        at 
        
            

org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)>>
         at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)

    

    
        

        at 
        
            

org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)>>    
     at 
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)

    

    
        

        at 
        
            

org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)>>
         at 
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)

    

    
        

        at 
        
            

org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)>>
         at java.lang.Thread.run(Thread.java:595)

    

    
        

    
        
            

  
      
          






















					Reply via email to