[Wikitech-l] HipHop VM support: ArrayObject and filter_var()
Several people from the HipHop team at Facebook just met with several people from WMF. Also, in the last couple of days, I've been doing some research into what it would take to make MediaWiki support HipHop VM. The answer is: not very much. There's two features that we use, mostly in extensions, that the Facebook people are not keen to implement due to their complexity: ArrayObject and filter_var(). It seems that it would be much easier for us to stop using them than for those features to be implemented in HipHop. So I'd like to suggest that we refuse new code submissions in Gerrit that use these features, if they are targeted for WMF production, and that we start work on migrating away from the existing uses of those features. There's a few other SPL features that we don't use at the moment and we should avoid introducing if possible due to lack of support in HipHop: * CachingIterator * EmptyIterator * GlobIterator * InfiniteIterator * LimitIterator * MultipleIterator * NoRewindIterator * ParentIterator * RecursiveArrayIterator * RecursiveCachingIterator * RecursiveFilterIterator * RecursiveRegexIterator * RecursiveTreeIterator * RegexIterator * SplDoublyLinkedList * SplFixedArray * SplHeap * SplMaxHeap * SplMinHeap * SplPriorityQueue * SplQueue * SplStack * SplTempFileObject We are not yet promising that we are indeed going to start using HipHop in WMF production, and we don't have any timetables. But HipHop has evolved to the point where supporting it is almost trivial, at least for test installations, so I think it makes sense to establish policies which will avoid making migration to HipHop more difficult. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On Thu, 09 May 2013 18:10:53 -0700, Tim Starling wrote: * SplDoublyLinkedList * SplFixedArray * SplHeap * SplMaxHeap * SplMinHeap * SplPriorityQueue * SplQueue * SplStack It would be nice if HHVM would support these. I'm not sure what is so complex about these that HHVM can implement array() but not these simple classes. While working on my skin rewrite ideas, trying to implement the template syntax parsing was much MUCH more readable and less prone to mistakes (like those that would occur if php cloned the array() in a place it shouldn't cause I typed something wrong) when working with SplDoublyLinkedList, SplQueue, and SplStack. -- Tim Starling -- ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/] ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On 10/05/13 11:28, Daniel Friesen wrote: > On Thu, 09 May 2013 18:10:53 -0700, Tim Starling > wrote: > >> * SplDoublyLinkedList >> * SplFixedArray >> * SplHeap >> * SplMaxHeap >> * SplMinHeap >> * SplPriorityQueue >> * SplQueue >> * SplStack > > It would be nice if HHVM would support these. I'm not sure what is so > complex about these that HHVM can implement array() but not these > simple classes. > > While working on my skin rewrite ideas, trying to implement the > template syntax parsing was much MUCH more readable and less prone to > mistakes (like those that would occur if php cloned the array() in a > place it shouldn't cause I typed something wrong) when working with > SplDoublyLinkedList, SplQueue, and SplStack. You can always implement the parts you need in pure PHP. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On Thu, May 9, 2013 at 9:46 PM, Tim Starling wrote: > You can always implement the parts you need in pure PHP. True. It might be worthwhile to make some sort of Spl compatibility library that loads in PHP versions of those classes if they do not exists. That way we can use them without dropping HHVM support. *-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On 10/05/13 11:48, Tyler Romeo wrote: > On Thu, May 9, 2013 at 9:46 PM, Tim Starling wrote: > >> You can always implement the parts you need in pure PHP. > > > True. It might be worthwhile to make some sort of Spl compatibility library > that loads in PHP versions of those classes if they do not exists. That way > we can use them without dropping HHVM support. If you can write a complete and accurate compatibility class in pure PHP, then it can be included in HipHop in the system/classes directory. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On Thu, May 9, 2013 at 9:57 PM, Tim Starling wrote: > If you can write a complete and accurate compatibility class in pure > PHP, then it can be included in HipHop in the system/classes directory. > Interesting. I'll try and work on this. *-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
Hey, I can see why SPL might require extra work in HipHop to support. At the same time I find it somewhat unfortunate this means one cannot use the Standard PHP Library. Question: Is it possible to use HipHop only for part of a codebase. For instance, if there is some MW extension that depends on a third party library we have no control over that makes uses of SPL, would this cause problems? Suggestion: If such a policy is implemented, I'd be great to have a job that checks for violations of it on our CI. Cheers -- Jeroen De Dauw http://www.bn2vs.com Don't panic. Don't be evil. -- ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On 10.05.2013, 5:10 Tim wrote: > Several people from the HipHop team at Facebook just met with several > people from WMF. Also, in the last couple of days, I've been doing > some research into what it would take to make MediaWiki support HipHop > VM. The answer is: not very much. > There's two features that we use, mostly in extensions, that the > Facebook people are not keen to implement due to their complexity: > ArrayObject and filter_var(). It seems that it would be much easier > for us to stop using them than for those features to be implemented in > HipHop. > So I'd like to suggest that we refuse new code submissions in Gerrit > that use these features, if they are targeted for WMF production, and > that we start work on migrating away from the existing uses of those > features. > There's a few other SPL features that we don't use at the moment and > we should avoid introducing if possible due to lack of support in HipHop: What about namespaces? Last time I heard HH didn't support them - is it safe to assume that support for them is coming? -- Best regards, Max Semenik ([[User:MaxSem]]) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On Fri, May 10, 2013 at 9:49 AM, Max Semenik wrote: > On 10.05.2013, 5:10 Tim wrote: > >> Several people from the HipHop team at Facebook just met with several >> people from WMF. Also, in the last couple of days, I've been doing >> some research into what it would take to make MediaWiki support HipHop >> VM. The answer is: not very much. > >> There's two features that we use, mostly in extensions, that the >> Facebook people are not keen to implement due to their complexity: >> ArrayObject and filter_var(). It seems that it would be much easier >> for us to stop using them than for those features to be implemented in >> HipHop. > >> So I'd like to suggest that we refuse new code submissions in Gerrit >> that use these features, if they are targeted for WMF production, and >> that we start work on migrating away from the existing uses of those >> features. > >> There's a few other SPL features that we don't use at the moment and >> we should avoid introducing if possible due to lack of support in HipHop: > > What about namespaces? Last time I heard HH didn't support them - is > it safe to assume that support for them is coming? > Yes. As I understood it yesterday, this is just about to land in master. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On Fri, May 10, 2013 at 8:05 AM, Jeroen De Dauw wrote: > Hey, > > I can see why SPL might require extra work in HipHop to support. At the > same time I find it somewhat unfortunate this means one cannot use the > Standard PHP Library. > Yeah, but I think it's a workable issue. And the HH team seems very amenable to feature requests (and patches!), so implementing parts of the SPL are certainly possible over the long term. As Tim points out, for ArrayObject and filter_var() it's non trivial to implement (even Zend's implementation of the former is 2000+ LOC). > Question: Is it possible to use HipHop only for part of a codebase. For > instance, if there is some MW extension that depends on a third party > library we have no control over that makes uses of SPL, would this cause > problems? > I don't think so, as HipHop is a standalone implementation of PHP and doesn't even really know about the php binary/libraries. > Suggestion: If such a policy is implemented, I'd be great to have a job > that checks for violations of it on our CI. > This is a very good idea, and something I'll take a stab at handling. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
Le 10/05/13 03:10, Tim Starling a écrit : > So I'd like to suggest that we refuse new code submissions in Gerrit > that use these features, if they are targeted for WMF production, This can be done using CodeSniffer. We can write a specific style standard such as "WikimediaProduction" that will spurt error whenever some function / class is encountered. -- Antoine "hashar" Musso ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
Le 10/05/13 03:10, Tim Starling a écrit : > There's a few other SPL features that we don't use at the moment and > we should avoid introducing if possible due to lack of support in HipHop: I wish we actually used Spl :-] They are nice classes providing all kind of useful features: http://php.net/manual/en/book.spl.php Although I fully agree we should not add more to be able to use HipHop, I think we should also look at having them implemented in HipHop. -- Antoine "hashar" Musso ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On 10.05.2013 17:58, Chad wrote: On Fri, May 10, 2013 at 8:05 AM, Jeroen De Dauw wrote: Hey, I can see why SPL might require extra work in HipHop to support. At the same time I find it somewhat unfortunate this means one cannot use the Standard PHP Library. Yeah, but I think it's a workable issue. And the HH team seems very amenable to feature requests (and patches!), so implementing parts of the SPL are certainly possible over the long term. As Tim points out, for ArrayObject and filter_var() it's non trivial to implement (even Zend's implementation of the former is 2000+ LOC). System and development software, such as OS, compilers, language libraries, different kinds of VM's and so on are really huge and 2000+ lines of code actually is a *little* amount. It's not a framework or wiki. It's development software written in low-level language. Dmitriy ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On 10.05.2013 23:51, Antoine Musso wrote: Le 10/05/13 03:10, Tim Starling a écrit : There's a few other SPL features that we don't use at the moment and we should avoid introducing if possible due to lack of support in HipHop: I wish we actually used Spl :-] They are nice classes providing all kind of useful features: http://php.net/manual/en/book.spl.php Yes it is strange that Facebook does not need SPL. Especially because since PHP 5.3 SPL is not optional anymore. ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On Sat, May 11, 2013 at 3:31 AM, Dmitriy Sintsov wrote: > On 10.05.2013 17:58, Chad wrote: >> >> On Fri, May 10, 2013 at 8:05 AM, Jeroen De Dauw >> wrote: >>> >>> Hey, >>> >>> I can see why SPL might require extra work in HipHop to support. At the >>> same time I find it somewhat unfortunate this means one cannot use the >>> Standard PHP Library. >>> >> Yeah, but I think it's a workable issue. And the HH team seems very >> amenable to feature requests (and patches!), so implementing parts of >> the SPL are certainly possible over the long term. >> >> As Tim points out, for ArrayObject and filter_var() it's non trivial to >> implement (even Zend's implementation of the former is 2000+ LOC). >> > System and development software, such as OS, compilers, language libraries, > different kinds of VM's and so on are really huge and 2000+ lines of code > actually is a *little* amount. > It's not a framework or wiki. It's development software written in low-level > language. Indeed, 2000 isn't really a lot, I was just trying to give an order of magnitude so people wouldn't think it was like <50 LOC or something. -Chad ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
FWIW, here is what I have so far: http://pastebin.com/hUQ92DfB I haven't tested it yet because my PHP environment is not behaving, and the only class I haven't implemented fully is SplHeap. *-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com On Sat, May 11, 2013 at 8:57 AM, Chad wrote: > On Sat, May 11, 2013 at 3:31 AM, Dmitriy Sintsov > wrote: > > On 10.05.2013 17:58, Chad wrote: > >> > >> On Fri, May 10, 2013 at 8:05 AM, Jeroen De Dauw > > >> wrote: > >>> > >>> Hey, > >>> > >>> I can see why SPL might require extra work in HipHop to support. At the > >>> same time I find it somewhat unfortunate this means one cannot use the > >>> Standard PHP Library. > >>> > >> Yeah, but I think it's a workable issue. And the HH team seems very > >> amenable to feature requests (and patches!), so implementing parts of > >> the SPL are certainly possible over the long term. > >> > >> As Tim points out, for ArrayObject and filter_var() it's non trivial to > >> implement (even Zend's implementation of the former is 2000+ LOC). > >> > > System and development software, such as OS, compilers, language > libraries, > > different kinds of VM's and so on are really huge and 2000+ lines of code > > actually is a *little* amount. > > It's not a framework or wiki. It's development software written in > low-level > > language. > > Indeed, 2000 isn't really a lot, I was just trying to give an order of > magnitude > so people wouldn't think it was like <50 LOC or something. > > -Chad > > ___ > Wikitech-l mailing list > Wikitech-l@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On 12.05.2013 1:18, Tyler Romeo wrote: FWIW, here is what I have so far: http://pastebin.com/hUQ92DfB I haven't tested it yet because my PHP environment is not behaving, and the only class I haven't implemented fully is SplHeap. Perhaps you should send the link to HipHop developers (or to their list, if there's any). ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On Sun, May 12, 2013 at 1:56 PM, Dmitriy Sintsov wrote: > Perhaps you should send the link to HipHop developers (or to their list, > if there's any). Once I finish it and it's working I plan on doing just that. *-- * *Tyler Romeo* Stevens Institute of Technology, Class of 2015 Major in Computer Science www.whizkidztech.com | tylerro...@gmail.com ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] HipHop VM support: ArrayObject and filter_var()
On 10/05/13 22:05, Jeroen De Dauw wrote: > Question: Is it possible to use HipHop only for part of a codebase. For > instance, if there is some MW extension that depends on a third party > library we have no control over that makes uses of SPL, would this cause > problems? It would be possible to have a separate cluster running regular PHP, and to access it via internal API requests or by frontend routing, similar to how we do image scaling. But the justification would need to be pretty strong to justify that complexity. -- Tim Starling ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
