[
https://issues.apache.org/jira/browse/HBASE-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610482#action_12610482
]
Billy Pearson commented on HBASE-674:
-------------------------------------
I changed my flush size to 16MB from my default 128MB and run a large job here
is some lines from the logs
also I added back StringUtils.humanReadableInt(this.memcacheSize.get()) on the
end so I could see if the size was growing after every flush
{code}
2008-07-04 03:09:46,684 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished memcache flush for region webdata,,1214968800601 in 5595ms, sequence
id=237700684, 12.9m, 351.2k
2008-07-04 03:15:02,169 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished memcache flush for region webdata,,1214968800601 in 5741ms, sequence
id=239128833, 13.4m, 188.1k
2008-07-04 03:15:04,145 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished memcache flush for region webdata,,1214968800601 in 1975ms, sequence
id=239155758, 167.6k, 222.8k
{code}
so the above looks good now the last number is the memcacheSize.get() and its
moveing down and up so thats good to see I thank this patch solved my problem
of the flushes.
I run the job for quite a while and flushes seams to happen normal in place of
less time between flushes.
The only other thing I see on this issue is the size before flush of the
memcache is still off like the logs stack posted above
{code}
2008-07-04 03:09:41,089 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Started memcache flush for region webdata,,1214968800601. Current region
memcache size 16.0m
2008-07-04 03:09:42,587 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
Added /hbase/webdata/1748955538/anchor/mapfiles/5595070426400799233 with 29142
entries, sequence id 237700684, data size 3.7m, file size 507.2k
2008-07-04 03:09:43,719 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
Added /hbase/webdata/1748955538/stime/mapfiles/4943615301545296809 with 2872
entries, sequence id 237700684, data size 154.9k, file size 24.3k
2008-07-04 03:09:45,197 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
Added /hbase/webdata/1748955538/in_rank/mapfiles/367761225010760821 with 36409
entries, sequence id 237700684, data size 4.5m, file size 415.8k
2008-07-04 03:09:45,470 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
Added /hbase/webdata/1748955538/size/mapfiles/6451240725630689572 with 2872
entries, sequence id 237700684, data size 135.9k, file size 28.4k
2008-07-04 03:09:46,683 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
Added /hbase/webdata/1748955538/last_seen/mapfiles/3472123077784461203 with
36410 entries, sequence id 237700684, data size 4.4m, file size 409.3k
2008-07-04 03:09:46,684 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Finished memcache flush for region webdata,,1214968800601 in 5595ms, sequence
id=237700684, 12.9m, 351.2k
{code}
16m is whats reported as the size before the flush and the total data flushed
was 12.9m
> memcache size unreliable
> ------------------------
>
> Key: HBASE-674
> URL: https://issues.apache.org/jira/browse/HBASE-674
> Project: Hadoop HBase
> Issue Type: Bug
> Affects Versions: 0.1.2
> Reporter: stack
> Assignee: Jim Kellerman
> Fix For: 0.2.0
>
> Attachments: 674-v2.patch, 674.patch, patch.txt
>
>
> Multiple updates against same row/column/ts will be seen as increments to
> cache size on insert but when we then play the memcache at flush time, we'll
> only see the most recent entry and decrement the memcache size by whatever
> its size; memcache will be off.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
