Re: [gentoo-portage-dev] PATCH: gentoolkit: Make portage.config object a global object
On Saturday 17 September 2005 02:32, Alec Warner wrote: Jason Stubbs wrote: On Saturday 17 September 2005 01:59, Paul Varner wrote: http://bugs.gentoo.org/show_bug.cgi?id=90680 Author: Paul Varner The current implementation of gentoolkit creates a portage.config object for every package object that it creates. While this is the correct thing to do from an object-oriented programming point of view, this implementation consumes an excessive amount of memory and CPU. The proposed patch changes the portage.config object for each package object to point to a single global object. If no one sees any serious issues with the patch, I will be placing it into gentoolkit. I tried doing this once before locally, but found some issue with it. Unfortunately, I can't remember what that issue was. If you are calling setcpv() for every call to the package object that utilizes the config object and no utilizing packages (in gentoolkit or otherwise) are utilizing threading, it should theoretically be okay. Actually, I think it was the threading issue that delayed the fix. I can't remember the model for this, but there is some logic along the lines of intercepting config object writes with setattr and then cloning the config object. That way if the config is read-only only 1 is instantiated, but if you attempt to modify it, the config would clone itself, then proceed with the modification and return the cloned copy. Not sure how easy that would be to implement, perhaps some sort of wrapper class? To share such an object the right way from an OO perspective would require to pass the object along at package object instanciation. I doubt though that the config object should be modified. But if it must in some cases a lazy copy scheme is probably most efficient. You'd probably have the editing code do something like: this-editableConfig()-changeAttribute(...) Where the editableConfig function checks whether a copy has been made, if so, it will just use that copy, else it will make a copy and return it. Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgpDlxwohvwF2.pgp Description: PGP signature
Re: [gentoo-portage-dev] PATCH: gentoolkit: Make portage.config object a global object
On Monday 19 September 2005 10:26, Jason Stubbs wrote: On Monday 19 September 2005 17:18, Paul de Vrieze wrote: I doubt though that the config object should be modified. The Package object needs to call setcpv() on the config object to get at the per-package USE flags after they have been stacked. But if it must in some cases a lazy copy scheme is probably most efficient. Unfortunately not in this case. There would either be one config instance (for processes that don't deal with USE flags) or as many config instances as there are packages (for processes that do). Should probably read the source a bit but you're completely right of course. The only thing that could be done for this is splitting up the object, or having it do internal sharing of shared things. Or to make it clear, it is not some cases, but almost all cases. Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgplfCFHSRRMu.pgp Description: PGP signature
[gentoo-portage-dev] PATCH glep31 checking
Hola.
http://glep.gentoo.org/glep-0031.html-- the details
http://bugs.gentoo.org/106544-- the bug
http://bugs.gentoo.org/attachment.cgi?=68828 -- the patch
Attached the patch also; one additional tweak is that file.size is now
a fatal check, since the tree seem's to finally be clean.
~harring
Index: repoman
===
--- repoman (revision 1992)
+++ repoman (working copy)
@@ -13,6 +13,13 @@
sys.path = [/usr/lib/portage/pym]+sys.path
version=1.2
+allowed_filename_chars=a-zA-Z0-9._-+:
+allowed_filename_chars_set = {}
+map(allowed_filename_chars_set.setdefault, map(chr, range(ord('a'),
ord('z')+1)))
+map(allowed_filename_chars_set.setdefault, map(chr, range(ord('A'),
ord('Z')+1)))
+map(allowed_filename_chars_set.setdefault, map(chr, range(ord('0'),
ord('9')+1)))
+map(allowed_filename_chars_set.setdefault, map(chr, map(ord, [., -, _,
+, :])))
+
import string,signal,re,pickle,tempfile
import portage
@@ -21,6 +28,8 @@
import portage_dep
import cvstree
import time
+import codecs
+
from output import *
#bold, darkgreen, darkred, green, red, turquoise, yellow
@@ -85,6 +94,8 @@
filedir.missing:Package lacks a files directory,
file.executable:Ebuilds, digests, metadata.xml, Manifest, and
ChangeLog do note need the executable bit,
file.size:Files in the files directory must be under 20k,
+ file.name:File/dir name must be composed of only the following
chars: %s % allowed_filename_chars,
+ file.UTF8:File is not UTF8 compliant,
KEYWORDS.missing:Ebuilds that have a missing KEYWORDS variable,
LICENSE.missing:Ebuilds that have a missing LICENSE variable,
DESCRIPTION.missing:Ebuilds that have a missing DESCRIPTION
variable,
@@ -146,7 +157,6 @@
IUSE.invalid,
ebuild.minorsyn,
ebuild.badheader,
-file.size,
metadata.missing,
metadata.bad,
virtual.versioned
@@ -663,6 +673,29 @@
stats[file.executable] += 1
fails[file.executable].append(checkdir+/+y)
digestlist=[]
+
+ for y in checkdirlist:
+ for c in y.strip(os.path.sep):
+ if c not in allowed_filename_chars_set:
+ stats[file.name] += 1
+ fails[file.name].append(%s/%s: char '%s' %
(checkdir, y, c))
+ break
+
+ if not (y in (ChangeLog, metadata.xml) or
y.endswith(.ebuild)):
+ continue
+ try:
+ line = 1
+ for l in codecs.open(y, r, utf8):
+ line +=1
+ except UnicodeDecodeError, ue:
+ stats[file.UTF8] += 1
+ s = ue.object[:ue.start]
+ l2 = s.count(\n)
+ line += l2
+ if l2 != 0:
+ s = s[s.rfind(\n) + 1:]
+ fails[file.UTF8].append(%s/%s: line %i, just after:
'%s' % (checkdir, y, line, s))
+
if isCvs:
try:
mystat=os.stat(checkdir+/files)[0]
@@ -799,6 +832,13 @@
stats[file.size] += 1
fails[file.size].append((+
str(mystat.st_size/1024) + K) +x+/files/+y)
+ for c in y.strip(os.path.sep):
+ if c not in allowed_filename_chars_set:
+ stats[file.name] += 1
+ fails[file.name].append(%s/%s: char
'%s' % (checkdir, y, c))
+ break
+
+
if ChangeLog not in checkdirlist:
stats[changelog.missing]+=1
fails[changelog.missing].append(x+/ChangeLog)
pgpoxv44vqNjL.pgp
Description: PGP signature
Re: [gentoo-portage-dev] PATCH glep31 checking
On Mon, Sep 19, 2005 at 04:12:08PM -0500, Brian Harring wrote: Attached the patch also; one additional tweak is that file.size is now a fatal check, since the tree seem's to finally be clean. Dropped the file.size becoming fatal change on the bug, and intend to for the final version. Either tweak the patch yourself, or gank it from the bug... it's a one line reversion. :) Pls test, kthx. ~harring pgpSn9miVa1jT.pgp Description: PGP signature
