Hi Carlos, thanks for your feedback. In terms of PAYG, I just have a sense that at the language level, the 'default' should be that it *works* as per design. If I take the example of Vector... then numeric Vector types will not run faster in javascript. Using all the method calls will have a substantial impact compared to using a normal Array. But code should 'work' the same across targets. I do have some ideas for application framework classes outside 'language'/Vector that would provide target-agnostic fast regular class implementations for numeric typed Arrays, and I hope to work on that soon as well, which takes more advantage of the optimized native options on each target (Vector in swf, TypedArray in javascript). But to achieve that it will likely be 'always fixed length' in this case, I will see if it seems worth trying to be able to toggle that, or maybe it can be a bead or opt-in support to get that functionality. But that should provide a good option for developers seeking to get the best performance for numeric collections. I expect to work on this during June. Or developers can of course always choose to implement their own custom performance code in COMPILE::JS / COMPILE::SWF variations, which is always an option.
Hi Piotr, sorry about that email formatting issue. Thanks for your follow up on this. Here is a paste of that part, which includes the code. https://paste.apache.org/dPWy Also, if you want to see the tests for Vector, you can see them here: https://github.com/apache/royale-asjs/blob/improvements/Language/manualtests/UnitTests/src/main/royale/flexUnitTests/language/LanguageTesterTestVector.as You should see large parts of that series of tests fail outside the branch, and in some cases there were compiler errors too iirc (or maybe I am confusing some compiler fixes I did for other things, can't be sure now) On Sun, May 26, 2019 at 4:58 AM Carlos Rovira <carlosrov...@apache.org> wrote: > Hi! > > Read it! :), was long but I think this emails are needed since it brings > many thoughts that only the developer knows and otherwise would end never > known by the rest of the community. So Great! > > Greg, first, many thanks for working on this. I think we get other point > solved, and I think not only Vector, but many other fixes and things coming > in your branch. > > For my part, as you said, the branch is tested against my real project, > that I think is a very good way to test against lots of working code. So > for my is ok to merge, not only cause it does not brake anything, but cause > it's a big step forward. > > Since it brings things we didn't have before like Vector, is good to bring > this. Then we can continue iterating to add missed things like AMF support > or to evolve to other state or even change internal things if other check > and see ways of enhancements. > > About the Vector implementation. My opinion is that I think as you that > Vector is an intrinsic part of AS3, so better to have an implementation to > complete AS3 in Royale. About having multiple implementations, I think that > could be good in the future if things can be done that way, but better > start from this starting point and see what happens in next months-years. > IOW, better have a real implementation now that a potentially one that > still is not here. We need to see the goal and approach to it progressively > or we end getting nothing since the goal is a huge one not easy to reach. > > For me the 3kb is totally normal, and as well a great achievement of your > implementation. > I must say that my way of thinking in Royale is to be totally aligned with > concepts like PAYG, but sometimes we need to break rules when the goal > demand it, or we can end trying to apply the same principle to all things, > when sometimes is not the best. So this is importante, rules are really > good, but we need to always put behind a layer of human thinking about what > we are trying to accomplish > > Thanks Greg for reaching another important milestone! Amazing work! :) > > Carlos > > PD: Piotr, about gmail cutting the email, I'm using gmail and get to read > it completely, maybe could be a problem of gmail in mobile? I'm reading on > gmail desktop. The only problem I had was reading posted source code, but > since it was tiny, I could finally read it... > > > > > > El sáb., 25 may. 2019 a las 15:37, Piotr Zarzycki (< > piotrzarzyck...@gmail.com>) escribió: > > > Hi Greg, > > > > Could you please post the code from this email in the paste.apache.org - > > right now is unreadable in my Gmail account. > > > > Thanks, > > Piotr > > > > On Sat, May 25, 2019, 4:51 AM Greg Dove <greg.d...@gmail.com> wrote: > > > > > OK, At the risk of making what follows even longer... a warning: *this > > is a > > > long read*. It might veer off topic in one or two places, but it is > > mostly > > > about why, and how Vector is implemented the way it is in the branch. > > > Basically I would like to merge this branch into develop as soon as > > > possible, i actually think it is ready as is, but I am keen to get > other > > > input before I do. > > > Read it if you're interested. If you're not interested in reading it > then > > > please try the branch on any 'real' projects that you have in play. > > Carlos > > > has tested against his, and there are no issues, but it would be good > to > > > get wider input. > > > > > > now to the textual onslaught: > > > > > > > > > *Vector* > > > > > > One of the things I did before I worked on this Vector implementation > was > > > to detach myself from ‘how I use Vector when I code’, to take a step > back > > > and challenge myself to rethink how I thought about it in general. > > > > > > Vector is *special*. Vector is not *a class* in a typical sense. It is > > the > > > closest thing that as3 has to generics which, when used at a class > > level, I > > > like to think of as the ability to generate multiple class variations > > from > > > a single class definition (that is just the way I think of them). So, > at > > > some level Vector could be considered as an infinite set of distinct > > > classes, based on an infinite set of possible specified element types > > > (whatever you see between the angle brackets in code). It is not > actually > > > implemented natively in AVM this way iiuc, but that is the outward > > > appearance, and that is also the behaviour it has when using type > > checking > > > in as3. > > > > > > Aside from the runtime typesafe behaviour, Vector’s strong typing means > > > there is additional compile time support and, related, the possibility > of > > > extra IDE support, both of which help the developer. > > > > > > Vector ‘types’ are also final ‘classes’ enforced by the compiler, so > > cannot > > > be extended. > > > > > > Additionally, on looking closer, there are some other quirky rules for > > > runtime typing behaviour. These are related to the implementation in > AVM. > > > The 3 Vector types for Vector.<int>, Vector.<uint>, and Vector.<Number> > > are > > > individual base types, and then there is a 4th base type, Vector.<*> > > which > > > serves as the ‘base class’ of all other Vector ‘subclasses’. The 3 > > numeric > > > types are faster than Array, and I expect this was the reason for the > > > separation, because these are used in a lot of other flash apis, > > > particularly for swapping with byte level data and things like > BitmapData > > > etc. The others are more simply for the strong typing aspects, because > > > (perhaps less so with Boolean and String at a guess) they are somewhat > > > slower than Array in AVM, because of the overhead of runtime > typechecking > > > and coercions etc. So the main selling points for choosing a Vector in > > > flash were a) compile time and runtime type safety and b) faster > > > performance for the 3 numeric Vector types. > > > > > > One implication of these ‘rules’ means that to check if *something* is > a > > > Vector of any type in swf, I need to do (mything is Vector.<int> || > > myThing > > > is Vector.<uint> || myThing is Vector.<Number> || myThing is > Vector.<*>) > > or > > > use reflection (which is slower). > > > > > > The above implies that, for language conformance (considered as > > conformance > > > with as3 documentation/spec and the behaviour of the reference > > > implementation that we have for any ‘unknowns’) in an emulation, a > > regular > > > single as3 class implementation would not create a result that is > > > consistent with the spec in terms of its representation of typing, and > > its > > > behaviour when using type checking. > > > > > > I tried to take all of these things into consideration. > > > > > > > > > > > > *How is it implemented in the branch?* > > > > > > Vector is implemented using the lightweight ‘synthType’ approach I > added > > > previously to support for int and uint types as ‘classes’. This > provides > > a > > > light integration for ‘is’ and ‘as’ checks, but in javascript itself a > > > Vector instance is simply a tagged native javascript Array instance > (i.e. > > > an Array with something that identifies it as being ‘not a regular > > Array’). > > > The Array methods are enhanced at the instance level to support the > type > > > coercions that take place in an actionscript Vector. Using Array makes > > > sense because Vector instances are essentially typed Arrays. But it is > > not > > > possible to extend Array in javascript and have things work properly > (it > > is > > > possible to use a different approach with es6 Proxy, but that is not > > > available in IE11). > > > > > > This implementation also means that a ‘Vector’ instance should have > > > functionality that works mostly as expected if it is passed directly to > > a 3 > > > rd party native javascript library that is expecting an Array instance. > > It > > > will not support errors when changing the length if the Vector instance > > is > > > ‘fixed’ length, for example but otherwise it should function much the > > same > > > as a regular Array in the 3rd party code. > > > > > > The ‘synthType’ approach can be used because Vector types are final, so > > it > > > does not need to conform to a regular Royale class definition that can > be > > > extended and can be much ‘lighter’. As with int and uint ‘classes’, no > > > Vector constructor exists until the first one is requested. There is > one > > > internal private class in Language that provides most of the > > functionality > > > for Vector, and element coercion functions are generated specifically > for > > > each Vector ‘subtype’. Overall, it behaves more like a subclass > ‘factory’ > > > with each individual type’s constructor being created on demand and > > cached > > > for future use. > > > > > > Reflection and serialization support have not been added yet, but I > > > certainly had these in mind also with this approach. I just wanted to > > reach > > > a sensible intermediate point for now, and then add those in later. > > > > > > In summary, the Vector implementation in the branch provides the > > following: > > > > > > -distinct types (conforming in terms of type checking) for each Vector > > > subtype > > > > > > -lightweight integration with Language ‘is’ and ‘as’ support > > > > > > -each Vector subtype has a unique constructor generated the first time > it > > > is requested, the constructor does not exist prior to that. > > > > > > -expected to be compatible with 3rd party native javascript that > expects > > a > > > regular javascript Array > > > > > > -serialization (amf) and reflection are not yet supported but were > > > considered, and that is planned as next steps. > > > > > > As at the time of writing, the current implementation is supported by > 300 > > > Vector-specific assertions that run side-by-side between javascript and > > swf > > > in the manualtest UnitTests project. Apart from the differences created > > by > > > explicitly switching off some type safety checks with > > > @suppressvectorindexchecking, there is only one verifiable difference > in > > > these tests between swf and javsacript. Probably there are things > outside > > > the test coverage that might need addressing still, but I think it’s a > > > pretty good start. > > > > > > These tests should be easily ported to framework level unit tests > > > (RoyaleUnit) as the same approach has been used to add FlexUnit tests > in > > > the past. Testing/Development is primarily in Chrome on windows, but > the > > > same tests were subsequently verified on IE11, Edge, Opera and Firefox > on > > > windows. > > > > > > *Implementation Info* > > > > > > *More quirks* > > > > > > A ‘fixed == true’ Vector cannot have its length changed. This means > that > > > changes via the length setter and the mutation methods (push, pop, > shift, > > > unshift etc) all throw errors if used when a Vector is ‘fixed == true’. > > > But… splice method also can change length. And in AVM this works even > > with > > > a fixed length Vector, which might possibly be a bug but is one that is > > > matched in the implementation to conform with flash. > > > > > > *Size impact.* > > > > > > Alex was concerned with size impact for HelloWorld because the beads > were > > > using a Vector.<IBead> type in javascript in one of the application > > > framework base classes. But the requirement was for that to simply to > be > > an > > > Array in javascript, the compiler was only achieving that by default > > > because there was no real Vector implementation. So I did a couple of > > > things here. Firstly I changed the application framework code to > reflect > > > the different requirement for javascript. Secondly, I added some doc > > > comment directives to suppress exporting on public members. Then I > added > > > this to the Language methods that are for compiler-generated Vector > (and > > > int and uint) support so that they are eligible for dead code > elimination > > > if not used. This effectively means that it is fully PAYG – it is only > > > there if you need it, but if you need it there is also no effort to add > > it > > > in as an optional dependency. If, because of this, there is a potential > > for > > > issues with modules or something I have not thought of, that will be > > > another thing to solve, but if so, I will find a way to address it. > > > > > > If the main concern with ‘size’ is data transfer, then looking at the > > > minified output size in HelloWorld is not really meaningful (or perhaps > > it > > > is better to say that it is only as meaningful as looking at the size > of > > an > > > uncompressed swf compared to normal deployment of a compressed swf). In > > any > > > real deployment the javascript should be served as gzipped. So I > compared > > > gzipped (‘normal’ compression) versions of HelloWorld.js between those > > with > > > the unused Language support methods (for int/uint and Vector) and their > > > dependencies vs. builds without. The support for all that functionality > > > comes in at under 3 Kb difference in the gzipped output. I personally > > don’t > > > think that would register as important in most cases. However, it is > > pretty > > > close to zero impact now if it is not used. In fact the HelloWorld > > > application is now about 55 bytes smaller than it was before, so I > > consider > > > it ‘carbon-netural’. And these are at the gzipped levels (it is also > > > slightly smaller when not gzipped, although it can sometimes be the > case > > > that the larger of 2 files compresses better than a smaller one, > > depending > > > on the content). > > > > > > As a general comment about size… in my experience over the last 10 > years > > or > > > so, mostly in relation to Flex apps, size was most often not among the > > > highest priority issues. Sure, start-up time often was. But that is not > > > size per se. Often ‘size’ issues were first and foremost ‘asset size’ > > > issues (embedded or separate). While I know overall that it remains > > > important, I personally think the growth in bandwidth and device memory > > in > > > general has outpaced the growth in the size of web apps even as the > world > > > has turned to mobile. In other words, I think we probably passed a > hurdle > > > point in the mobile side of things. Admittedly that is just > ‘perception’ > > > and maybe there is data that shows I am quite wrong or your experience > on > > > specific client projects could be quite different. The point of me > > > explaining that was, however, to illustrate why I think ‘size’ sits > > where I > > > think it does in general in terms of importance: I consider it to be > > behind > > > stability/reliability first which is followed by performance in > > performance > > > critical code. Overriding that general flow (which again, is just my > > > opinion), are whatever my client’s specific requirements are for any > > > particular project. In other words I am not sure how much of > HelloWorld’s > > > size is going to be an important deciding factor for someone to choose > to > > > use Royale. I think the ‘I got it to work!’ is the real kicker here. > This > > > paragraph was not to cause any conflict, and I already addressed the > > ‘size’ > > > impact in the branch, so I’m not saying it wasn’t necessary to do that > > > (although I do suspect the same attention was not given to all other > > things > > > in the past that affected HelloWorld). It’s more to promote discussion > > and > > > thought in general. Trying to be all one aspect (small size) can > > compromise > > > other aspects (reliability or stability or compatibility, for example). > > And > > > trying to accommodate all aspects (reliability, small size, exceptional > > > performance) somehow very likely increases complexity because there is > a > > > need to accommodate the explicit decisions for trade-offs between > > competing > > > aspects. It would be good to have a guide of priorities based on the > > needs > > > of the user base (while I might have a strong opinion about what I > think > > is > > > important, I don’t assume that my views necessarily represent wider > user > > > needs, because I have not done research to support that – and it’s not > > > really a twitter poll kind of thing!). Anyhow I got off track there a > > bit… > > > back to Vector… > > > > > > > > > > > > *Performance.* > > > > > > No-one should expect a non-native Vector implementation to perform > faster > > > than native Array in javascript. Its main benefit is type safety, and > > that > > > is the reason it should be selected for use over Array. (In swf, native > > > numeric Vector types *are* selected for performance, but the other > Vector > > > types are also at least slightly slower than Array, so selecting them > is > > > based on the need for type safety) > > > > > > There are, however some ways to get 100% Array performance with this > > > implementation, and in a loop, for example it would be similar to > > avoiding > > > method calls (like push/pop) and only using index assignments that is > > > typical of performance optimization in swf. > > > > > > So these two methods provide almost Identical performance in js: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > *private function testArrayInt2(iterations:uint):Number{ var > > start:Date > > > = new Date(); var inst:Array = new Array(iterations); for > > (var > > > i:uint=0;i<iterations; i++) { inst[i] = i; } return new > > > Date().valueOf() - start.valueOf(); } /** * * > > > @royalesuppressvectorindexcheck true */ private function > > > testVectorInt2(iterations:uint):Number{ var start:Date = new > Date(); > > > var inst:Vector.<uint> = new Vector.<uint>(iterations); for > > > (var i:uint=0;i<iterations; i++) { inst[i] = i; } > return > > > new Date().valueOf() - start.valueOf(); }* > > > > > > The doc comment on the 2nd method prevented the compiler from > generating > > an > > > index check wrapper around the ‘i’ in inst[i] > > > > > > The above 2 methods take around 5-7 ms for 500,000 iterations on my > > machine > > > (chrome/windows). This is quite a bit faster than the same methods in > the > > > swf (non-debug) build on my machine. The Vector constructor is also > doing > > > extra work in this case by setting all the 500,000 slots to numeric 0. > > But > > > on modern browsers it is using a fast native Array.fill method to do > > that, > > > so it’s probably more noticeable on IE11 which uses a for loop to > > > pre-populate the contents with numeric 0 values. > > > > > > For reference, the generated code for the second method above, looks > like > > > this: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > */** * * @royalesuppressvectorindexcheck true * @private * @param > > > {number} iterations * @return {number} */ > > > > > > > > > flexUnitTests.language.LanguageTesterTestVectorPerf.prototype.flexUnitTests_language_LanguageTesterTestVectorPerf_testVectorInt2 > > > = function(iterations) { var /** @type {Date} */ start = new Date(); > > > var /** @type {Array.<number>} */ inst = new > > > (org.apache.royale.utils.Language.synthVector('uint'))(iterations); > for > > > (var /** @type {number} */ i = 0; i < iterations; i++) { inst[i] = > i; > > > } return new Date().valueOf() - start.valueOf(); };* > > > > > > The methods with type checking in them, like push, unshift etc will be > > > quite a lot slower though. This should not be a surprise. In swf, when > > you > > > look at something like Vector.<IBead> it can be up to 30% slower than > > Array > > > for methods like push, iirc at native level. In javascript it will be a > > lot > > > more. But in both cases it has type safety. And for small collection > > sizes > > > in non-performance sensitive code it is probably not going to make a > > > meaningul difference. > > > > > > Known limitations > > > > > > 1. myVectorInstance[‘constructor’] can be different between js > and > > > swf > > > > > > 2. There is some reliance on not obscuring the type from the > > compiler > > > for things to work correctly. Example: > > > > > > var myObscuredType:* = new Vector.<String>(20,true) // creates a fixed > > > Vector of length 20 > > > > > > myObscuredType.length = 21; //should throw an error, but does not in > > > javascript. > > > > > > This will work as long as the compiler knows that the type with the > > length > > > setter is some type of Vector, otherwise not. > > > > > > > > > > > > *Performance tuning* > > > > > > *@royalesuppressvectorindexcheck* > > > > > > The above doc comment directive avoids checking for out-of-valid-range > > > index values on assignments. This is extremely desirable inside loops > > which > > > are usually constrained to the valid range in any case. There is a top > > > level compiler configuration setting for this, but local settings win > > > (turning off or on generally with true/false, or suppressing > specifically > > > for individual local variable names) > > > > > > > > > > > > Other options. > > > > > > The above one was obvious to me as an important tuning option. I > started > > > with adding another but pulled it out in favor of adding them based on > > > actual need. > > > > > > There are many options: > > > > > > Instance based: reducing type checking with a compiler-only generated > > > additional constructor parameter, triggered by some doc comment > > directive. > > > This could be in the form of adding in compiler generated alternatve > > calls > > > to things like ‘unsafePush’ and ‘unsafePop’ on instances, or perhaps > > > literal calls to Array.prototype.push.call(inst, pushItem), triggered > by > > > some doc comment directive > > > > > > Global changes:Adding some compiler define value that (along with: > > > ||goog.DEBUG) suppresses many runtime checks in the release Vector > > > implementation. This would allow a blanket approach to dialing runtime > > > safety down (or keeping them) in the release build. It would apply > across > > > all libraries and local project code. > > > > > > Some combinations of the above that allow sweeping optimizations with > > local > > > exceptions. > > > > > > Basically, this branch currently represents what I have reason to > believe > > > is a high conformance Vector implementation which should be the default > > as > > > it is an as3 core type. I think there are many options to scale things > > > back, but rather than assume what they should be, I’d prefer to hear > from > > > users and address *real* needs, so that we only end up with > configuration > > > options that matter. > > > > > > > > > > > > *Advantages of a single Vector implementation – the case for avoiding a > > > ‘vector emulation class’* > > > > > > ‘Language emulation’ is a foundational layer that is not in the same > > > category as ‘application framework’ where we can define things how we > > want. > > > Because the royale sdk includes both aspects inside ‘framework’ (and > that > > > was obviously not the case for flex sdk in the past) it may be easy to > > > overlook that difference. > > > > > > Vector typing is a core part of AS3 language. Therefore it should > conform > > > and be reliable. Having multiple implementations for a core language > > > feature seems wrong and could easily lead to libraries of code that are > > > incompatible. Additionally, as pointed out earlier, Vector is more > like a > > > subclass factory (‘runtime’ generics) than a single class, so won’t be > > well > > > represented by a regular class implementation. > > > > > > Alex, I know you added support for a Vector emulation class approach to > > > meet the needs of someone asking for improved Vector support, in part > > > because we did not have someone who had volunteered to write our own. I > > did > > > that last part now. I am also suggesting that the emulation class > > approach > > > is not a good long term solution because of the above reasons. And the > > full > > > set of functionality is less than 3Kb in the gzipped output, so I’d > > suggest > > > that as long as I can add needs-based performance tuning options for > > those > > > who want them, the benefits of having a single conforming > implementation > > > which is performance scalable are better than the risks associated with > > > adding partially non-conforming ones. Therefore I’d like to ask if we > can > > > remove that functionality now that we have our own 'Vector' (I can do > the > > > work if you prefer as I know your focus is understandably on other > > things). > > > > > > *Future:* > > > > > > It will be easily possibly to alter the current implementation to use > es6 > > > proxy which would likely provide the most reliable way to get full > Vector > > > conformance indistinguishable from flash. Es6 proxy is transparent for > > > Array: > > > > > > var p = new Proxy(new Array(), {}); > > > > > > Array.isArray(p) // true > > > > > > This should allow trapping things like the length setter before it gets > > to > > > the underlying Array target, and doing checking for fixed == true etc. > > The > > > same is true for numeric index-based assignment of values etc. > > > > > > es6 Proxy is kinda cool. At the moment IE11 is the only meaningful > target > > > that does not support this. > > > > > > I am only mentioning this here because I did think about this, and I > > > believe that the current implementation could be upgraded quite easily > to > > > use this approach if it makes sense (when IE11 is a distant memory – > > > perhaps sometime next year!). > > > > > > > > > > > > Phew, you made it! Well done. > > > > > > Feel free to share your thoughts or ask any questions you might have. > If > > > you have neither of those at this time, please still try to find time > to > > > test any large projects you have in the branch, and provide feedback or > > > share any concerns you might have after doing that. > > > thanks, > > > Greg > > > > > > > > > On Thu, May 23, 2019 at 7:55 PM Greg Dove <greg.d...@gmail.com> wrote: > > > > > > > Re XMLTest manualtest... > > > > > > > > Yep those were the ones I ported, Harbs. I think I got them all but > may > > > > have missed some. I added a bunch more as well. Hopefully these can > be > > > > easily migrated to what Josh has been working on. > > > > > > > > > > > > On Thu, 23 May 2019, 19:43 Harbs, <harbs.li...@gmail.com> wrote: > > > > > > > >> As far as XML unit tests go, the starting point should be XMLTest in > > > >> manual tests. (Almost) every time I ran into an issue, I added it to > > > that > > > >> project. > > > >> > > > >> I think I might have been lax on the last few issues I fixed, so we > > > >> should probably go through the later XML commits and make sure we > have > > > >> tests for that. > > > >> > > > >> As far as Node goes, I think we probably need conditional Node > > > >> compilation to handle Node-specific (and browser specific) code in a > > > PAYG > > > >> way. > > > >> > > > >> To technically handle the parsing, something like > > > >> https://github.com/isaacs/sax-js <https://github.com/isaacs/sax-js> > > is > > > a > > > >> good starting point and something like this > > > >> https://github.com/nfarina/xmldoc < > https://github.com/nfarina/xmldoc> > > > >> might be useful to either use or modify. > > > >> > > > >> Harbs > > > >> > > > >> > On May 23, 2019, at 10:18 AM, Greg Dove <greg.d...@gmail.com> > > wrote: > > > >> > > > > >> > All, I started porting some adhoc XML tests to UnitTests and > > > eventually > > > >> > ended up spending quite a bit of time on addressing issues that > > arose > > > >> for > > > >> > XML before getting back to Vector stuff. > > > >> > I think XML probably needs many more unit tests before we get to > 1.0 > > > >> > because it has an extensive api. I have not used royale with Node > > yet, > > > >> but > > > >> > XML also needs some thought about how to get it working on Node, I > > > >> assume. > > > >> > Because XML uses the browser's parser and Node does not have one > by > > > >> > default, then using the same code will need something to take the > > > place > > > >> of > > > >> > the browser's native parser for Node. There is a lib in npm that > > might > > > >> be > > > >> > useful for that, but I don't know how that might work with licence > > > etc. > > > >> > Anyhow, that is just an observation, I will focus on Vector in > this > > > >> > thread... I will post here late tomorrow local time with more > info, > > > and > > > >> > discussion points. I am keen to see this merged in, but also keen > to > > > get > > > >> > buy-in first. > > > >> > > > > >> > > > > >> > On Fri, May 10, 2019 at 11:15 PM Carlos Rovira < > > > carlosrov...@apache.org > > > >> > > > > >> > wrote: > > > >> > > > > >> >> Hi Greg, > > > >> >> > > > >> >> thanks for reporting. I can share here that I was able to test > your > > > >> branch > > > >> >> with our real project and seems all goes well. > > > >> >> Could make a intense test, but almost app is working and we found > > > just > > > >> a > > > >> >> few type error coercions that your code was able to catch (so > > great! > > > >> :)) > > > >> >> and must be solved as you merge the branch in. > > > >> >> > > > >> >> I think that if Vector is something new and others don't have > > > >> problems, the > > > >> >> branch can be merged and Vector discussions can be done after > that, > > > >> since > > > >> >> it will not break anything since there's no uses of that code > since > > > is > > > >> new, > > > >> >> but the other changes can be very beneficial > > > >> >> > > > >> >> thanks in advance for your progress in all this stuff :) > > > >> >> > > > >> >> Carlos > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> El vie., 10 may. 2019 a las 8:44, Greg Dove (< > greg.d...@gmail.com > > >) > > > >> >> escribió: > > > >> >> > > > >> >>> All, I am really sorry, I keep thinking I will be able to get > back > > > to > > > >> >> this, > > > >> >>> but I have some other personal things taking my spare time at > the > > > >> moment. > > > >> >>> These will be done in 2 days, and I then will update the branch > > with > > > >> some > > > >> >>> extra stuff, and continue this discussion with a focus on Vector > > > >> >> (bringing > > > >> >>> some other relevant discussion on the same topic from Alex as > > well) > > > at > > > >> >> that > > > >> >>> time. Sorry to set the wrong expectations earlier. > > > >> >>> > > > >> >>> > > > >> >>> On Tue, May 7, 2019 at 9:01 AM Greg Dove <greg.d...@gmail.com> > > > wrote: > > > >> >>> > > > >> >>>> Thanks for the feedback, Josh, Carlos, Alex. > > > >> >>>> > > > >> >>>> js-complex-implicit-coercions > > > >> >>>> js-resolve-uncertain > > > >> >>>> js-vector-index-checks > > > >> >>>> > > > >> >>>> I will make those changes for compiler settings at some point > in > > > the > > > >> >>>> branch later today, invert the config default values to match, > > and > > > >> swap > > > >> >>> all > > > >> >>>> 'off' settings in the framework builds (ant and maven) from > true > > to > > > >> >>> false. > > > >> >>>> I will also add compiler tests for these settings (either today > > or > > > >> >>>> tomorrow). At the moment I only tested the new settings in the > > code > > > >> >>> result > > > >> >>>> tests in javascript. > > > >> >>>> > > > >> >>>> In another day or two I will post a call to discuss the Vector > > > >> >>>> implementation in more detail. For Vectors, the > > > >> js-vector-index-checks > > > >> >>> was > > > >> >>>> the obvious first candidate for dialing back on the impact of > > > runtime > > > >> >>>> type-checking, but there are a number of options for 'dialing' > > > other > > > >> >>>> aspects back (or even forward) and choosing the scope of their > > > effect > > > >> >>>> (local code, local project, or entire codebase code including > > > >> external > > > >> >>>> swcs). I already had stub code for the start of something else > to > > > >> >> remove > > > >> >>>> typechecking in mutation methods ('push', 'shift', 'pop' etc) > but > > > >> >> removed > > > >> >>>> it in favour of discussing and reviewing it first. Coming up > > with > > > a > > > >> >>>> 'usable' set of options will really benefit from your > collective > > > >> input, > > > >> >>> so > > > >> >>>> I hope you can participate. > > > >> >>>> > > > >> >>>> > > > >> >>>> > > > >> >>>> > > > >> >>>> > > > >> >>>> On Tue, May 7, 2019 at 4:19 AM Alex Harui > > <aha...@adobe.com.invalid > > > > > > > >> >>>> wrote: > > > >> >>>> > > > >> >>>>> +1 to renaming the options to the positive. > > > >> >>>>> > > > >> >>>>> On 5/6/19, 8:12 AM, "Josh Tynjala" <joshtynj...@apache.org> > > > wrote: > > > >> >>>>> > > > >> >>>>> Hey Greg, > > > >> >>>>> > > > >> >>>>> I haven't had a chance to look through all of the changes, > > but > > > >> one > > > >> >>>>> thing caught my eye. I find it confusing when a boolean value > is > > > >> named > > > >> >>> with > > > >> >>>>> a "negative" phrase. For instance, your new compiler options > > have > > > >> "no" > > > >> >>> in > > > >> >>>>> the name: > > > >> >>>>> > > > >> >>>>> js-no-complex-implicit-coercions > > > >> >>>>> js-no-resolve-uncertain > > > >> >>>>> js-no-vector-index-checks > > > >> >>>>> > > > >> >>>>> As they are named, true means no, and so false means yes. > > With > > > >> >> this > > > >> >>>>> kind of naming, I find that I always need to take a moment to > > > >> remember > > > >> >>>>> which means which. I think it would be better if true means > yes > > > and > > > >> >>> false > > > >> >>>>> means no. > > > >> >>>>> > > > >> >>>>> - Josh > > > >> >>>>> > > > >> >>>>> On 2019/05/05 08:00:20, Greg Dove <greg.d...@gmail.com> > > wrote: > > > >> >>>>>> So... just an overview of recent work I have been doing. > > > >> >> Summery > > > >> >>>>> up front, > > > >> >>>>>> some extra detail further down... please try things with the > > > >> >>> branch > > > >> >>>>> if you > > > >> >>>>>> have time. > > > >> >>>>>> > > > >> >>>>>> In the *improvements/Language* branch there are many updates > > > >> >>> inside > > > >> >>>>>> Language and related updates inside the compiler to address > > > >> >> these > > > >> >>>>> main > > > >> >>>>>> areas: > > > >> >>>>>> -Fixes/better support for int and uint types at runtime > > > >> >>>>>> -Fixes for strict equality comparisons when instantiated > types > > > >> >> are > > > >> >>>>>> uncertain, or known to be problematic in these cases for > > > >> >> specific > > > >> >>>>> types > > > >> >>>>>> that are known. > > > >> >>>>>> -Complex implicit coercions (throws errors if assigned type > is > > > >> >>>>> incorrect) > > > >> >>>>>> -Vectors - test-driven development of a conforming > > > >> >> implementation. > > > >> >>>>>> > > > >> >>>>>> The new features are supported by almost 350 new assertion > > tests > > > >> >>>>> (in the > > > >> >>>>>> UnitTests manualtests project). This was not a trivial amount > > of > > > >> >>>>> work :) > > > >> >>>>>> > > > >> >>>>>> I still have a few things to work on in the branch, including > > > >> >> some > > > >> >>>>> tuning > > > >> >>>>>> for the new configuration settings and adding tests to the > > > >> >>> compiler > > > >> >>>>> for > > > >> >>>>>> those, but I would be keen for others to test the branch and > > try > > > >> >>> it > > > >> >>>>> with > > > >> >>>>>> real projects, and provide feedback. So this is > > > >> >>>>> 'improvements/Language' for > > > >> >>>>>> both royale-asjs and royale-compiler. > > > >> >>>>>> In particular, please take Vector for a spin and see if you > can > > > >> >>>>> break > > > >> >>>>>> anything and let me know! > > > >> >>>>>> Note the new configuration settings a bit further down (and > see > > > >> >>>>> examples > > > >> >>>>>> here for how to switch them off globally: > > > >> >>>>>> mvn: > > > >> >>>>>> > > > >> >>>>> > > > >> >>> > > > >> >> > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Froyale-asjs%2Fblob%2Fimprovements%2FLanguage%2Fframeworks%2Fprojects%2Fpom.xml%23L88&data=02%7C01%7Caharui%40adobe.com%7Ce44a5f9a81b141a8414908d6d2352ee6%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636927523209585267&sdata=ZDbgmmsdx4m6D9Bnel839Lxi4sVh8kwNLKK4HS%2F%2ByW8%3D&reserved=0 > > > >> >>>>>> ant: > > > >> >>>>>> > > > >> >>>>> > > > >> >>> > > > >> >> > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Froyale-asjs%2Fblob%2Fimprovements%2FLanguage%2Fframeworks%2Fjs%2Fprojects%2FBasicJS%2Fsrc%2Fmain%2Fconfig%2Fcompile-js-config.xml%23L106&data=02%7C01%7Caharui%40adobe.com%7Ce44a5f9a81b141a8414908d6d2352ee6%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636927523209585267&sdata=kvr13pxvGktxjV7QLu0mWdKJZfpr3zk6AHCj%2FM74Ym4%3D&reserved=0 > > > >> >>>>>> ) > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>>> A couple of examples: > > > >> >>>>>> I tried compiling Tour de Jewel with the new features > switched > > > >> >> on, > > > >> >>>>> it it > > > >> >>>>>> immediately highlighted a runtime error where a 'bead' was > > being > > > >> >>>>> added > > > >> >>>>>> which was not actually an IBead. This was detected in a > Vector > > > >> >>> push > > > >> >>>>>> operation. Although it was not causing problems, it is a good > > > >> >>>>> example of > > > >> >>>>>> something that would have failed at runtime in the flash > > player, > > > >> >>>>> making it > > > >> >>>>>> much easier to identify and fix. > > > >> >>>>>> > > > >> >>>>>> I have switched the extra outputs off for all the framework > > code > > > >> >>> in > > > >> >>>>> the > > > >> >>>>>> branch. But I did try a couple of projects with them on. As > an > > > >> >>>>> example, > > > >> >>>>>> after building XML with them on it throws a runtime error > when > > > >> >>>>> calling one > > > >> >>>>>> of the methods in XML. > > > >> >>>>>> The method has the wrong argument type (Element type when it > > > >> >>> should > > > >> >>>>> -iirc- > > > >> >>>>>> be Node). So these can catch errors in your code that are > > silent > > > >> >>>>> because > > > >> >>>>>> there is no strong typechecking at runtime. > > > >> >>>>>> The above is the implicit complex coercion in action. it is > > like > > > >> >>> if > > > >> >>>>> you did > > > >> >>>>>> in flash player : > > > >> >>>>>> var myArray:Array = [new ByteArray()]; > > > >> >>>>>> var sprite:Sprite = myArray[0]; //runtime error here > > > >> >>>>>> This does not happen currently in Royale javascript, but is > now > > > >> >>>>> supported > > > >> >>>>>> in the branch (and you can switch it off). This is an > expansion > > > >> >> of > > > >> >>>>> some of > > > >> >>>>>> Josh's great work in the past with implicit primitive > coercions > > > >> >>>>> (which > > > >> >>>>>> don't throw errors but coerce to the correct type). > > > >> >>>>>> > > > >> >>>>>> *New configuration settings* > > > >> >>>>>> js-no-complex-implicit-coercions > > > >> >>>>>> default: false (i.e. ensures runtime safety when assigning an > > > >> >>>>> unknown type > > > >> >>>>>> to a known type ) > > > >> >>>>>> local doc comment directive > > > >> >>>>>> switching: @royalesuppresscompleximplicitcoercion > > > >> >>>>>> > > > >> >>>>>> js-no-resolve-uncertain > > > >> >>>>>> default: false (i.e. ensures instances that are safe in > certain > > > >> >>>>>> comparisons ) > > > >> >>>>>> local doc comment directive switching: > > > >> >>>>> @royalesuppressresolveuncertain > > > >> >>>>>> > > > >> >>>>>> js-no-vector-index-checks > > > >> >>>>>> default: false (i.e. vector index checking is on) > > > >> >>>>>> local doc comment directive switching: > > > >> >>>>> @royalesuppressvectorindexcheck > > > >> >>>>>> > > > >> >>>>>> *-Fixes problems/provides more general support for int and > uint > > > >> >>>>> types at > > > >> >>>>>> runtime* > > > >> >>>>>> Josh's recent assignment implicit coercions made a big > > > >> >> difference > > > >> >>>>> for these > > > >> >>>>>> (and other primitive types), but runtime support either > caused > > > >> >>>>> errors or > > > >> >>>>>> bad results. > > > >> >>>>>> Things like > > > >> >>>>>> var myClass = int; > > > >> >>>>>> > > > >> >>>>>> var x:* = new myClass(22.5); > > > >> >>>>>> trace( x === 22 ) //true > > > >> >>>>>> > > > >> >>>>>> The above works now in the branch. iirc I think there is more > > > >> >> than > > > >> >>>>> one > > > >> >>>>>> issue with that in develop. > > > >> >>>>>> I started with this based on issue #273 which also now is > fixed > > > >> >> in > > > >> >>>>> the > > > >> >>>>>> branch. > > > >> >>>>>> > > > >> >>>>>> int and uint are implemented are not needed like this in most > > > >> >>>>> cases, so the > > > >> >>>>>> are not real 'classes' but very simple instances of > 'synthetic > > > >> >>>>> Types' that > > > >> >>>>>> are only 'created' if/when they are requested for the first > > > >> >> time. > > > >> >>>>> Vectors > > > >> >>>>>> (because they are kind of like factory-generated classes) use > > > >> >> the > > > >> >>>>> same > > > >> >>>>>> underlying mechanism, but are more complicated than int and > > uint > > > >> >>> in > > > >> >>>>> terms > > > >> >>>>>> of their supporting implementation. uint and int are almost > > > >> >>> defined > > > >> >>>>> in a > > > >> >>>>>> single line of code, not so for Vectors. Another candidate > for > > a > > > >> >>>>> synthetic > > > >> >>>>>> type might be 'Class', but I will see about that. > > > >> >>>>>> > > > >> >>>>>> *-Fixes for strict equality comparisons in when instantiated > > > >> >> types > > > >> >>>>> are > > > >> >>>>>> uncertain, or known to be problematic for types that are > > known.* > > > >> >>>>>> Certain explicit instantiations of primitive types are > swapped > > > >> >> to > > > >> >>>>> coercions. > > > >> >>>>>> Things like 'new String('test')' are now output simply as > > > >> >>>>> String('test'). > > > >> >>>>>> Resolution of uncertain instantiations > > > >> >>>>>> Where a class is not known, the instantiation of that class > is > > > >> >>>>> wrapped in a > > > >> >>>>>> 'resolveUncertain' method call. This calls the low level > native > > > >> >>>>> 'valueOf()' > > > >> >>>>>> method on the instance, which resolves it to primitive types > if > > > >> >>>>> possible. > > > >> >>>>>> > > > >> >>>>>> The above changes provide consistency with AVM when values , > > > >> >> even > > > >> >>>>> those > > > >> >>>>>> with typing obscured, are used in strict equality > comparisons. > > > >> >>>>> These cases > > > >> >>>>>> may not bet mainstream, but that is exactly the type of thing > > > >> >> the > > > >> >>>>> causes a > > > >> >>>>>> lot of headscratching when things don't work. Note that > > > >> >>>>> Array.indexOf also > > > >> >>>>>> uses strict equality comparisons, so this is not just fixing > > > >> >>>>> results of === > > > >> >>>>>> or !== across these edge cases. > > > >> >>>>>> > > > >> >>>>>> *-Complex implicit coercions* > > > >> >>>>>> I expanded on Josh's implicit primitive type coercions to > > > >> >> support > > > >> >>>>> more > > > >> >>>>>> complex coercions > > > >> >>>>>> (this is on by default, but explicitly off in the framework) > > > >> >>>>>> So this works now like flash player: > > > >> >>>>>> var myClass:MyClass = someArray[i]; //if the assigned value > > from > > > >> >>>>>> someArray[i] is not a MyClass type, error is thrown > > > >> >>>>>> This can be switched off at compiler level, or tuned within > > > >> >>> methods > > > >> >>>>> (on or > > > >> >>>>>> off in contrast to compiler level setting) with a specific > doc > > > >> >>>>> comment > > > >> >>>>>> directive. (i.e. like royaleignorecoercion) > > > >> >>>>>> Output in debug mode shows these implicit coercions prefixed > > > >> >> with > > > >> >>>>> /* > > > >> >>>>>> implicit cast */ so you can easily review the number of > > > >> >> locations > > > >> >>>>> this is > > > >> >>>>>> affecting by doing 'find in files' and looking at the > locations > > > >> >>> and > > > >> >>>>> count. > > > >> >>>>>> While it will probably be a good thing to switch off in a > final > > > >> >>>>> release > > > >> >>>>>> build, it can help find problems during development, > > > >> >> particularly > > > >> >>>>> as more > > > >> >>>>>> and more code is not being parallel tested in the flash > player > > > >> >>>>> where error > > > >> >>>>>> trapping like this is automatic. > > > >> >>>>>> I switched this off in framework, but it could help find code > > > >> >>>>> errors in the > > > >> >>>>>> framework when it is switched on > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>>> *-Vectors* > > > >> >>>>>> Vectors are 'smoke and mirrors' currently in develop - it is > > > >> >>>>> basically the > > > >> >>>>>> compiler pretending that they are Vectors (they are Arrays). > > > >> >> This > > > >> >>>>> gives a > > > >> >>>>>> small amount of compile time safety, but still leaves large > > gaps > > > >> >>>>> when > > > >> >>>>>> compared with the real thing and many things that you could > > > >> >> assume > > > >> >>>>> would be > > > >> >>>>>> safe will not be. Assuming it worked properly could be even > > > >> >>>>> considered a > > > >> >>>>>> little 'dangerous'. > > > >> >>>>>> > > > >> >>>>>> There are 260 new assertion tests for Vectors, including some > > > >> >> that > > > >> >>>>> relate > > > >> >>>>>> to a new doc comment directive @suppressvectorindexchecking > > > >> >> which > > > >> >>>>> avoids > > > >> >>>>>> (intensive) checking for range errrors (and will be desirable > > to > > > >> >>>>> switch off > > > >> >>>>>> in a lot of cases, such as in length constrained loops etc). > > > >> >>>>>> You can see the Vector tests here: > > > >> >>>>>> > > > >> >>>>> > > > >> >>> > > > >> >> > > > >> > > > > > > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Froyale-asjs%2Fblob%2Fimprovements%2FLanguage%2Fmanualtests%2FUnitTests%2Fsrc%2Fmain%2Froyale%2FflexUnitTests%2Flanguage%2FLanguageTesterTestVector.as%23L65&data=02%7C01%7Caharui%40adobe.com%7Ce44a5f9a81b141a8414908d6d2352ee6%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636927523209585267&sdata=NNJ6cfAOqGHPya5oyADDhwBwkWpNkng%2Fk0%2BvrzZm7aM%3D&reserved=0 > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>>> > > > >> >>>>>> *Miscellaneous* > > > >> >>>>>> -When addressing some sourcemap related stuff for Vectors, I > > > >> >> fixed > > > >> >>>>> an > > > >> >>>>>> unrelated sourcemap issue that was caused by methods which > had > > > >> >>>>> metadata > > > >> >>>>>> attached. The mapping now correctly aligns with the original > > > >> >>>>> function > > > >> >>>>>> keyword in these cases. > > > >> >>>>>> > > > >> >>>>> > > > >> >>>>> > > > >> >>>>> > > > >> >>> > > > >> >> > > > >> >> > > > >> >> -- > > > >> >> Carlos Rovira > > > >> >> http://about.me/carlosrovira > > > >> >> > > > >> > > > >> > > > > > > > > -- > Carlos Rovira > http://about.me/carlosrovira >
