Loïc Bellier - Numérigraphe has proposed merging lp:~numerigraphe-team/ocb-addons/7.0-fill-inventory-OOM into lp:ocb-addons.
Requested reviews: Lionel Sausin - Numérigraphe (lionel-sausin) OpenERP Community Backports Team (ocb) Related bugs: Bug #1312045 in OpenERP Community Backports (Addons): "Importing inventory pushes the OS to out-of-memory condition when many Stock Moves exist" https://bugs.launchpad.net/ocb-addons/+bug/1312045 For more details, see: https://code.launchpad.net/~numerigraphe-team/ocb-addons/7.0-fill-inventory-OOM/+merge/217049 We're having problems importing an inventory for a location that contains >630000 Stock Moves. The problem seems to lie in the python loop based on the browse() of the Stock Moves: it preloads too much data and pushes the OS to an out-of-memory condition (which Linux responds to by killing the openerp server). This patch solve this bug by splitting stock moves lines ids on separated list of 10 000 lines Run GREEN on runbot. MP on standard 7.0 https://code.launchpad.net/~numerigraphe-team/openobject-addons/7.0-fill-inventory-OOM/+merge/217031 -- https://code.launchpad.net/~numerigraphe-team/ocb-addons/7.0-fill-inventory-OOM/+merge/217049 Your team OpenERP Community Backports Team is requested to review the proposed merge of lp:~numerigraphe-team/ocb-addons/7.0-fill-inventory-OOM into lp:ocb-addons.
=== modified file 'stock/wizard/stock_fill_inventory.py' --- stock/wizard/stock_fill_inventory.py 2014-03-24 09:14:18 +0000 +++ stock/wizard/stock_fill_inventory.py 2014-04-24 13:47:38 +0000 @@ -22,6 +22,7 @@ from openerp.osv import fields, osv, orm from openerp.tools.translate import _ from openerp.tools import mute_logger +import math class stock_fill_inventory(osv.osv_memory): _name = "stock.fill.inventory" @@ -103,28 +104,31 @@ for location in location_ids: datas = {} res[location] = {} - move_ids = move_obj.search(cr, uid, ['|',('location_dest_id','=',location),('location_id','=',location),('state','=','done')], context=context) + all_move_ids = move_obj.search(cr, uid, ['|',('location_dest_id','=',location),('location_id','=',location),('state','=','done')], context=context) local_context = dict(context) local_context['raise-exception'] = False - for move in move_obj.browse(cr, uid, move_ids, context=context): - lot_id = move.prodlot_id.id - prod_id = move.product_id.id - if move.location_dest_id.id != move.location_id.id: - if move.location_dest_id.id == location: - qty = uom_obj._compute_qty_obj(cr, uid, move.product_uom,move.product_qty, move.product_id.uom_id, context=local_context) - else: - qty = -uom_obj._compute_qty_obj(cr, uid, move.product_uom,move.product_qty, move.product_id.uom_id, context=local_context) - - - if datas.get((prod_id, lot_id)): - qty += datas[(prod_id, lot_id)]['product_qty'] - - # Floating point sum could introduce tiny rounding errors : - # Use the UoM API for the rounding (same UoM in & out). - qty = uom_obj._compute_qty_obj(cr, uid, - move.product_id.uom_id, qty, - move.product_id.uom_id) - datas[(prod_id, lot_id)] = {'product_id': prod_id, 'location_id': location, 'product_qty': qty, 'product_uom': move.product_id.uom_id.id, 'prod_lot_id': lot_id} + # due to memory high usage, i made smaller lists of 10 000 elements + for i in range(int(math.ceil(len(all_move_ids) / 10000.0))): + move_ids = all_move_ids[i * 10000: (i + 1) * 10000] + for move in move_obj.browse(cr, uid, move_ids, context=context): + lot_id = move.prodlot_id.id + prod_id = move.product_id.id + if move.location_dest_id.id != move.location_id.id: + if move.location_dest_id.id == location: + qty = uom_obj._compute_qty_obj(cr, uid, move.product_uom,move.product_qty, move.product_id.uom_id, context=local_context) + else: + qty = -uom_obj._compute_qty_obj(cr, uid, move.product_uom,move.product_qty, move.product_id.uom_id, context=local_context) + + + if datas.get((prod_id, lot_id)): + qty += datas[(prod_id, lot_id)]['product_qty'] + + # Floating point sum could introduce tiny rounding errors : + # Use the UoM API for the rounding (same UoM in & out). + qty = uom_obj._compute_qty_obj(cr, uid, + move.product_id.uom_id, qty, + move.product_id.uom_id) + datas[(prod_id, lot_id)] = {'product_id': prod_id, 'location_id': location, 'product_qty': qty, 'product_uom': move.product_id.uom_id.id, 'prod_lot_id': lot_id} if datas: flag = True
-- Mailing list: https://launchpad.net/~openerp-community-reviewer Post to : openerp-community-reviewer@lists.launchpad.net Unsubscribe : https://launchpad.net/~openerp-community-reviewer More help : https://help.launchpad.net/ListHelp