Re: Re: [RFC] lang/pypy

2013-03-10 Thread David Naylor
On Sunday, 10 March 2013 00:37:15 poyop...@puripuri.plala.or.jp wrote:
 Hi, good work, David.

Hi Kuro

 It compiles, packages and works flawlessly here for replacement of 1.9
 so far for a week.

Great, thanks.  Good to know :-)
 
 On my compile box, amd64/Athlon64 5050e 2.6GHz 2 core/8GB,
 memory requirement for translation processes is far less than
 warning shown. It prevents build with
 
 | warn: this system has insufficient memory, expected at least 9227MiB RAM
 
 however my translation processes under -DPYPY_IGNORE_MEMORY take 2GB
 for normal binary and 2.5GB for sandboxed one so they aggregate 4.5GB
 to run parallel. 

This is good news.  Could you please detail how you measured peak memory?  I 
might need to retest the port.  

 This is far less than expected, no page thrashing,
 no hang, no stuttering. It does not matter being with pypy1.9 or pypy2.0
 (yes, I built twice for 2.0 self hosting. Not tried with cPython.)
 
 So I think this warning is a bit excessive that makes everyone just put
 PYPY_IGNORE_MEMORY=1 in their make.conf. Just in my case.

I'll disable the test for now and revise my estimation.  Thanks for reporting 
back.  

Regards

signature.asc
Description: This is a digitally signed message part.


Re: [RFC] lang/pypy

2013-03-10 Thread poyopoyo
Hi David,

At Sun, 10 Mar 2013 21:18:12 +0300,
David Naylor wrote:
  however my translation processes under -DPYPY_IGNORE_MEMORY take 2GB
  for normal binary and 2.5GB for sandboxed one so they aggregate 4.5GB
  to run parallel. 
 
 This is good news.  Could you please detail how you measured peak memory?  I 
 might need to retest the port.  

I heuristically measured them with top(1) RESources, amount of ZFS
ARC and swap increased. Oh and free memory after one of translation
process finised. I'm afraid I couldn't provide reproducible method for
environments for others.

 I'll disable the test for now and revise my estimation.  Thanks for reporting 
 back.  

I wonder where so much difference of memory usage between yours
and mine comes from. 5.5GB vs 2.5GB on the same platform (64bit/pypy)
is not so negligible.

-- 
Kuro poyop...@puripuri.plala.or.jp

PS. like this, please. thanks for the offer.
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org


Re: [RFC] lang/pypy

2013-03-09 Thread poyopoyo
Hi, good work, David.

It compiles, packages and works flawlessly here for replacement of 1.9
so far for a week.

On my compile box, amd64/Athlon64 5050e 2.6GHz 2 core/8GB,
memory requirement for translation processes is far less than
warning shown. It prevents build with
| warn: this system has insufficient memory, expected at least 9227MiB RAM
however my translation processes under -DPYPY_IGNORE_MEMORY take 2GB
for normal binary and 2.5GB for sandboxed one so they aggregate 4.5GB
to run parallel. This is far less than expected, no page thrashing,
no hang, no stuttering. It does not matter being with pypy1.9 or pypy2.0
(yes, I built twice for 2.0 self hosting. Not tried with cPython.)

So I think this warning is a bit excessive that makes everyone just put
PYPY_IGNORE_MEMORY=1 in their make.conf. Just in my case.

-- 
kuro
___
freebsd-ports@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to freebsd-ports-unsubscr...@freebsd.org


[RFC] lang/pypy

2013-03-02 Thread David Naylor
Hi All.

After many months of (sporadic) work I would like to introduce pypy-2.0.b1.  

Could you please have a look at, and test, my proposed changes (attached) and 
the wiki page at http://wiki.FreeBSD.org/PyPy.  

I would like to commit these changes (after incorporating feedback) sometime 
next week.  Feel free to update the wiki yourselves ;-).  

Regards

David

P.S. Please keep my mentors CCed in any discussions :-)
Index: pypy/Makefile
===
--- pypy/Makefile	(revision 312473)
+++ pypy/Makefile	(working copy)
@@ -2,8 +2,7 @@
 # $FreeBSD$
 
 PORTNAME=	pypy
-DISTVERSION=	1.9
-PORTREVISION=	2
+DISTVERSION=	2.0-beta1
 CATEGORIES=	lang python java
 MASTER_SITES=	https://bitbucket.org/pypy/pypy/get/
 DISTNAME=	release-${DISTVERSION}
@@ -18,18 +17,30 @@
 LIB_DEPENDS=	expat:${PORTSDIR}/textproc/expat2 \
 		ffi:${PORTSDIR}/devel/libffi
 
-OPTIONS_DEFINE=	SANDBOX
+CLI_DESC=	(BROKEN) Translate a CLI (.NET) based pypy
+JVM_DESC=	(BROKEN) Translate a JVM (Java) based pypy
+PYPY_DESC=	Use pypy to translate (faster but uses more memory)
 SANDBOX_DESC=	Translate a sandboxed pypy
+.if !defined(PYPY_INST)
+OPTIONS_DEFINE+=	CLI JVM SANDBOX
+.endif
+LOCALBASE?=	/usr/local
+.if exists(${LOCALBASE}/bin/pypy)
+OPTIONS_DEFINE+=	PYPY
+.endif
 
+ALL_TARGET=	${PYPY_NAMES}
 BUILD_WRKSRC=	${WRKDIR}
 USE_BZIP2=	yes
 USE_ICONV=	yes
 USE_GETTEXT=	yes
+MAKE_JOBS_SAFE=	yes
+MAKEFILE=	${FILESDIR}/Makefile
 PKGINSTALL=	${WRKDIR}/pkg-install
 PKGDEINSTALL=	${WRKDIR}/pkg-deinstall
-WRKSRC=		${WRKDIR}/pypy-pypy-341e1e3821ff
+WRKSRC=		${WRKDIR}/pypy-pypy-fcb6b056f00e
 
-PYPY_VER=	${DISTVERSION}
+PYPY_VER=	${DISTVERSION:C|([0-9])\.([0-9]).*|\1.\2|}
 PYTHON_IMPL_VER=	2.7
 PYPY_LIBDIR=	lib/pypy${PYPY_VER}
 PYPY_INCLUDEDIR=	include/pypy${PYPY_VER}
@@ -38,20 +49,26 @@
 PLIST_SUB+=	PYPY_LIBDIR=${PYPY_LIBDIR} \
 		PYPY_INCLUDEDIR=${PYPY_INCLUDEDIR}
 
-MAKE_ENV+=	PYPY_LOCALBASE=${LOCALBASE}
-.if exists(/usr/bin/clang)
-MAKE_ARGS+=	CC=clang
-MAKE_JOBS_SAFE=	yes
-.endif
+MAKE_ENV+=	DISTVERSION=${DISTVERSION} PYTHON_CMD=${PYTHON_CMD} \
+		WRKSRC=${WRKSRC} PYPY_LOCALBASE=${LOCALBASE}
 
-# XXX !.include bsd.port.pre.mk as USE_* need to be set prior
 .include bsd.port.options.mk
-.include ${.CURDIR}/files/bsd.pypy.inst.mk
+.include ${MASTERDIR}/files/bsd.pypy.inst.mk
 
-.if defined(PACKAGE_BUILDING)
-MANUAL_PACKAGE_BUILD=	fails to finish compilation on pointyhat, reason unknown
+.if ${OSVERSION}  124 || ( ${ARCH} != i386  ${ARCH} != amd64 )
+.if ${CC:T} == cc  ( exists(/usr/bin/clang) || exists(${LOCALBASE}/clang) )
+CC=		clang
+.else
+USE_GCC=	yes
 .endif
+.endif
 
+.if ${PORT_OPTIONS:MPYPY} || defined(PYTHON_CMD)
+PYTHON_CMD?=	${LOCALBASE}/bin/pypy
+.else
+USE_PYTHON_BUILD=	2.5+
+.endif
+
 # List of PyPy instances
 .if !defined(PYPY_INST)
 PYPY_INST=	DEFAULT
@@ -60,13 +77,26 @@
 PYPY_INST+=	SANDBOX
 .endif
 
+.if ${PORT_OPTIONS:MCLI}
+PYPY_INST+=	CLI
+.endif
+
+.if ${PORT_OPTIONS:MJVM}
+PYPY_INST+=	JVM
+.endif
+
 .endif # !defined(PYPY_INST)
 
-PYPY_NAMES=
+MAKE_ENV+=	PYPY_INST=${PYPY_INST}
+
 .for inst in ${PYPY_INST}
 
 PYPY_NAMES+=	${PYPY_${inst}_NAME}
 PYPY_PRIMARY?=	${PYPY_${inst}_NAME}
+MAKE_ENV+=	PYPY_${inst}_NAME=${PYPY_${inst}_NAME} \
+		PYPY_${inst}_OBJSPACE_ARGS=${PYPY_${inst}_OBJSPACE_ARGS} \
+		PYPY_${inst}_OPT=${PYPY_${inst}_OPT} \
+		PYPY_${inst}_TRANSLATE_ARGS=${PYPY_${inst}_TRANSLATE_ARGS}
 
 # Check if the boehm GC will be used
 .if ${PYPY_${inst}_OPT} == 0 || ${PYPY_${inst}_OPT} == 1 || ${PYPY_${inst}_OPT} == size
@@ -85,24 +115,6 @@
 
 .endfor # inst in ${PYPY_INST}
 
-# Use pypy if it is installed, else use python (to translate)
-.if !defined(PY)
-.if !defined(PYPY)
-.if ${PYPY_PRIMARY} == pypy
-PYPY!=		${WHICH} ${PYPY_PRIMARY} 2 /dev/null || true
-.else
-PYPY!=		${WHICH} ${PYPY_PRIMARY} 2 /dev/null || ${WHICH} pypy 2 /dev/null || true
-.endif
-.endif # !defined(PYPY)
-
-.if exists(${PYPY})
-PY=		${PYPY}
-.else
-USE_PYTHON_BUILD=	2.5+
-PY=		${PYTHON_CMD}
-.endif
-.endif # !defined(PY)
-
 .if defined(WITH_BOEHM_GC)
 LIB_DEPENDS+=	gc.1:${PORTSDIR}/devel/boehm-gc
 .endif
@@ -117,7 +129,7 @@
 
 .if defined(WITH_JVM)
 USE_JAVA=	yes
-JAVA_VERSION=	1.6+
+JAVA_VERSION=	1.5+
 ONLY_FOR_ARCHS=	i386 powerpc
 ONLY_FOR_ARCHS_REASON=	only translates on 32bit systems
 BROKEN=		JVM backend broken, partially supported upstream
@@ -149,19 +161,59 @@
 .endfor # inst in ${PYPY_INST}
 .endif # !defined(PYPY_JITTABLE)
 
-pre-fetch:
-	@${ECHO} PyPy requires a large amount of free RAM and time to translate and compile.
-	@${ECHO}
-	@${ECHO} To translate, PyPy requires on 32bit 3G (min 2G) free RAM and on 64bit
-	@${ECHO} 6G (min 4G) free RAM.  Also, to compile, PyPy on amd64 gcc requires an
-	@${ECHO} extra 4G however clang only requires 400M (CC=clang) but clang is slower
-	@${ECHO} in compiling PyPy.
-	@${ECHO}
-	@${ECHO} If memory is in short supply consider using a lower optimisation level
-	@${ECHO} (e.g. PYPY_DEFAULT_OPT=2) however that makes PyPy much slower.  Also,
-	@${ECHO}