Re: Catalog management performance issue

Ruth Hoffman Tue, 17 Aug 2010 08:34:03 -0700

Hi Micha:

Could I ask how big are your data sets? The reason I ask is that severalyears ago, prior to 9.04 I had an out-of-memory problem that was neverresolved. The database tables that I was querying had between 4 - 12million records. My database (and OFBiz usage of the same) went awaybefore I couldn't do any further investigation into the problem.


So, I'm still curious about OFBiz support for really large data sets.

Regards,
Ruth

Michał Cukierman wrote:

Thanks for the quick answer.

I have allocated 2GB of memory, so it's not a case. The problem is that
I work on huge data sets. Version is 10.4 but 9.04 is has the same code.
I have ommited those informations 'couse as you see - it's not relevant
to the topic.

I don't understand your last assumption. Maybe it will someting in my
understanding?

For now I think the best would be to improve the code I have proposed as
it fixes:
1) My issues
2) Coding way of selecting everything to get a couple of categories.

So, thats why I asked about:

1) How to implement Right/Left joins on DynamicViewEntity

(I was able to find out how to creat inner join only)

2) How to implement my whereString

(Haven't found in a code such a use case)


Regards,
MichaÅ‚


Dnia 2010-08-17, wto o godzinie 07:54 -0700, BJ Freeman pisze:

as far as performance how much memory have you allocated to ofbiz?
what version of ofbiz are you using?
There is a top Category that should be checked for first.
have not been in that part of the code for a while.

MichaÅ‚ Cukierman sent the following on 8/17/2010 7:41 AM:

Hi,

I have a problem with performence (and Out of Memmory Issue) with
catalog management module. There is a section invoked after
Choose Top Category

Collection<GenericValue>  allCategories =
delegator.findList("ProductCategory", null, null, null, null, false);
for (GenericValue curCat: allCategories) {
        Collection<GenericValue>  parentCats =
curCat.getRelatedCache("CurrentProductCategoryRollup");

  if (parentCats.isEmpty())
        results.add(curCat);
}

As we see. To get categories without parents we need to fetch for all
categories and for each of them, find a parent.

I do not know Ofbiz api very well for now, so I have implemented:
EntityWhereString where =
EntityCondition.makeConditionWhere("PRODUCT_CATEGORY.PRODUCT_CATEGORY_ID
not in" +
" (select PRODUCT_CATEGORY_ROLLUP.PRODUCT_CATEGORY_ID from
PRODUCT_CATEGORY_ROLLUP)");
Set<String>  fieldsToSelect = FastSet.newInstance();
                        fieldsToSelect.add("productCategoryId");
                        results.addAll(delegator.findList("ProductCategory", 
where,
fieldsToSelect, null, null, false));


I simply do invoke a SQL query in the database:

select PRODUCT_CATEGORY.PRODUCT_CATEGORY_ID from PRODUCT_CATEGORY where
PRODUCT_CATEGORY.PRODUCT_CATEGORY_ID not in
(select PRODUCT_CATEGORY_ROLLUP.PRODUCT_CATEGORY_ID from
PRODUCT_CATEGORY_ROLLUP);

Is there a way ( for sure is ) to implement it in a better manner? For
now it works on my system, but after improving the query it could be
passed to a production.

So I got two questions in fact:

1) How to implement Right/Left joins on DynamicViewEntity
2) How to implement my whereString


Thank you in advance,

Re: Catalog management performance issue

Reply via email to