> So, when I say the DAL implements features that might otherwise be found >> in a typical ORM, I am not saying the DAL implements an ORM design pattern, >> just that it replicates functionality for which you might otherwise use an >> ORM. >> > > No, it does not do that. It implements very different functionality, that > may have a similar API and the same terminology used, which honestly I find > quite confusing - border-line misleading. >
If by "functionality" you mean doing the exact same thing in the exact same way, then there aren't even two ORM's that can be said to have the same functionality. So what's your point -- that no two distinct software libraries can be said to have similar functionality? I don't know how many ways I can try to make the same point, so I guess I'll try one more time. By "functionality", I'm thinking of things like "querying a database to retrieve some records and converting them to a format that can be used in Python", and "inserting records in a database", and "updating records in a database". These are things you can do with a DAL and things you can do with an ORM. There are many other such things. They are not implemented in the same fashion, nor are they executed using the same abstractions within the application code, but they achieve similar goals. So, you can use the web2py DAL to do things that you might otherwise do with an ORM. > For example, in an ORM, you can define a method in a class that returns a >> value calculated from the fields of a database record. In the web2py DAL, >> this same functionality can be achieved using a virtual field or lazy field. >> > > There are no lazy-fields in web2py, and I find the terminology misleading > - as I said - *lazyness *in the context of *database-access*, is a * > deferred-query* - NOT a *deferred-calculation* of the *results *of a > query. > In the context of Row fields, the term "lazy" means that the value is "filled in" sometime after creation (typically at access time). Whether that lazy "filling in" involves database access or not depends on the nature of the field. If the field is simply a "virtual" field calculated based on the values of other fields in the Row, then there is no sense in which you would be deferring database access, as the value is not stored in the database. In that case, you are simply deferring calculation. On the other hand, reference fields do allow one to access the referenced record, and the database access in that case is in fact deferred. Likewise, Row objects can include LazySet attributes that defer database access of referencing records. Finally, a web2py virtual field or method field can do whatever you want it to do, including deferred database queries of any sort. So yes, there *are* lazy fields in web2py, both of the deferred-calculation type and the deferred-database-access type. In any case, they're not officially called "lazy" fields in the API -- that's just a term that is commonly used. > Deferred calculations of field-results are generally useless - > web-applications are generally I/O-Bound much more than CPU-Bound - so the > benefits of deferring is mute in post-query calculations compared to > benefits in deferred-queries that are used within the context of > transaction-operation-optimizations > Deferring calculations is certainly not useless, and you may even care more about being CPU-bound than I/O-bound if you're using an async web server, but even if we stipulate the above, that still doesn't change the definition of the word "lazy". > which is the context most people would thing of whenever they here the > term *Lazy* thrown about a database-context > I don't know -- sounds like an empirical question. At least within the web2py community, though, I think the term is understood. > I don't know if the SQLA CORE has virtual fields, but if it doesn't, I >> would suppose it leaves this kind of functionality to the ORM. >> > > That's irellevant to the comparison of SQLA-Core vs. web2py-DAL, since I > am not suggesting using the SQLA-Core and dumping it's ORM > I completely agree, which is why that point had nothing to do with a comparison of the DAL to SQLA Core. If you will recall, the point was that the DAL includes some functionality for which you might otherwise use an ORM. If the SQLA Core doesn't have virtual fields, then you need to jump to the ORM for that functionality. Hence, some of the 20,000+ lines of SQLA ORM code are for generating functionality already available via other means in the DAL -- hence, the sheer size of the SQLA ORM does not necessarily imply a high degree of usefulness over and above what you can do with the DAL. - quite the opposite - and since virtual-fields are actually much more > beneficial when used within an ORM layer, as opposed to a DAL one. > I have no idea how you can justify that claim. > The only relevance for this point to this discussion, is the comparison > of the sized of the code-bases. I get that this was what you meant. > Then why were you complaining about its irrelevance to a point you know I wasn't making? > No, let's not. My point is not that any of those items properly belong to >> either a DAL or an ORM, or that they can only be implemented with either a >> DAL or an ORM design pattern. Rather, you had claimed that the SQLA CORE is >> equivalent to the web2py DAL and that all 20,000+ lines of SQLA ORM code >> must therefore be providing unique functionality not available in the DAL >> (thus implying that the ORM must be useful). I was just suggesting that the >> DAL might be doing more than the SQLA CORE (at least in some areas), and >> that the DAL might possibly be offering some features for which you would >> otherwise need the SQLA ORM. >> > >> You are saying that a lot of web2py's extra-features that extend on-top > of the DAL, might not be included in SQLA's Core, but rather may represent > a big portion of the 20K lines of code of the ORM, which would then suggest > that the features I was excited about may actually represent a much minor > portion of the 20K code-base, which would then suggest that they may be > small, and therefore legitimate for being considered "useless". You could > have said so more clearly (like I just did) and prevent the confusion. > Well, it was a conversion, so it unfolded in parts, but here's what I said. First, in response to you claiming the DAL had no features beyond the SQLA Core: The web2py DAL has a lot of features that might otherwise be found in an ORM. I'm not very familiar with SQLA, but I suspect the DAL has some features not present in CORE but similar to functionality included in the ORM. Then in response to your skepticism at the above, I listed a bunch of DAL features that I thought might not be in Core, and finally ended with this: I didn't say there were ORM features in the DAL, just that it includes features that you might otherwise expect to find in an ORM (e.g., something like virtual fields). In other words, some of what you get with that 20,000+ lines of ORM code might be functionality that is in fact available in the web2py DAL. Sounds a lot like your summary above, right? > Now, if you would have seen the lecture I gave Massimo the link to watch, > you would have seen how complex these features might be, so I doubt they > are implemented within a small code-base. But if they do, this would > degrade your argument that this is such a "substantial-investment" as you > called it... > You argue: 20,000+ lines of code ==> therefore not useless. I argue: Not all 20,000+ lines of code provide unique functionality ==> therefore possibly useless (useless == not worth the development/maintenance effort for the incremental benefits over what we already have). To do an ORM like SQLA on top of the DAL, you probably would need a substantial code base because you would have to re-implement much of what the DAL already does but with the ORM. On the other hand, as I have suggested, you may be able to get some of the features you like in the SQLA ORM within the DAL itself (i.e., without building an ORM on top of it). That would probably also be a substantial effort, but obviously less than building a full ORM. > What I meant, was that the "essence" of an ORM, is NOT these features. Any >> kind of Access-Layer-API may contain some of these features, yes - I agree >> - but that was not the point - the point was that the MAIN role an ORM has, >> is not found in these features - it is in the mapping of relations to >> domain-model objects - which is not what they are all about (again, >> check-out my comment to Derek below). > > I agree with you. And yet you now well know that was not my point, so not sure why you keep beating this very dead horse. > ...I found it misleading to allude that by the mear inclusions of such >> features in web2py, that it somehow get's it closer to providing ORM >> functionality - this is not the case - as ORM-functionality - at it's core >> - is the design-pattern of providing domain-model tools - not the existence >> of convinience-auxiliary features. > > It appears you have found a statement I did not make misleading. There's not much I can do about that. Nevertheless, I believe the DAL does provide "domain-model tools." > I have read this whole DAL section in the book today - I know you can >> super-impose virtual-fields onto row-objects that result from a JOIN >> operation - but that's such an edge-case with such marginal-utility, that >> it can hardly even be considered a "feature"... > > I was comparing that to the fact that ORM classes may contain >> representations of fields from multiple-tables from the get-go - so it can >> be used for "querying" multiple-tables - NOT for result-manipulations of >> JOINs (Which has questionable-utility at best) > > It would help if you could show an example of what you are talking about. Show something in SQLA that you find much more difficult in web2py. > In the DAL, Virtual/Computed-fields can NOT generate implicit calls to >>> foreign-table-fields. >>> >> >> Yes, they can with recursive selects. >> > > I touched on Recursive-selects further along, they are useless for 2 > reasons: > > 1. They are Active-Record-like implementations - using it on a field > within a for-loop is a big no-no for DAL usage, as there it would generate > tons of queries. In an ORM, because it is statefull, you could do an > eager-load for the foreign-table in order to cache it - THEN it would be > usefull, as the foreign-table-access would not generate tons of queries. > So, you do a query to load the entire foreign table just in case you might need it? In web2py, you would do a join, and only when you need it, which sounds a lot more efficient. > 2. It is only applicable for single-relational fields - which are few and > far between in real-life code - for more complex relationship, the > backward-relational-inference brakes down. SQLA solves this with the > "relationship" object - basically defining relationships on both ends, so > that the ORM object-graph contains a bi-directional reference (the creator > had to talk to Guido to find-our how to do that, so it would not create a > cyclical-object-reference that might become a memory-leak, but eventually > they got it working). > web2py recursive selects go both ways. > I'm not quite sure what you mean here. Even in an ORM, in order to >>> calculate the value of a virtual field, you first have to retrieve the >>> input field values from the database; and when creating a computed field, >>> you still ultimately have to write the computed value to the database. >>> >> > No. ORMs are statefull - It you have already data loaded from a oreviouse > query in those source-fields, than a virtual-field would not have to > round-trip to the database at all - it would just use the cached-values in > the source-fields. > Same in web2py. You query the db, get a Rows object, and you can keep referencing that Rows object. Yes, if you make changes to some records and then need to query a subset (that you cannot easily extract from the Rows object), then you need to update and re-query. How often do we have this situation, though, and how large is the efficiency gain? Do we build a whole SQLA-like ORM just for that? And do we even need an ORM for that, or can something similar be done with DAL objects? > As for computed-fields, again, the benefit is for storing a > computed-value, so it will not have to be re-calculated. So yes, > "eventually" the computed-value would be saved to the database, but it > doesn't have to occur in the calculation-operation - it can just leave the > calculated-value in memory, for further use in subsequent queries within > the same transaction. > See above. > Additionally, similarly-to-virtual fields, here again the source-fields > for the calculation, may already exist in memory from previouse queries - > so the calculation would not requier a round-trip to the database. > You can update records in a Rows object. > No - they are not limited to ORMs - but they ARE limited to > statefull-frameworks, which most web-frameworks are and web2py is not. > Transitional-scope-persistence may be implemented in a DAL in a statefull > web-framework (which again, web2py isn't) - but the utility would still > be limited when compared to having that within an ORM domain-class > object-graph. > > That said, you are right about one thing - if the DAL would get > statefulness, many of these features would become immensely more beneficial > - almost automatically. It is a matter of statefullness - not or > design-pattern of domain-classes. > > But still, a statefull-domain-class implementation, would still benefit > more out of such features than a web2py's DAL, ans so would still > be superior. > Maybe. It remains to be seen. > I think in this very comment, and the one before, as well as my response > to Derek about ORMs, as well as in many lectures I've posted here, there > are more than sufficient reasons to see benefits of using SQLA's ORM as > opposed to web2py's DAL. It is an architectural-issue, not a use-case one. > Given all I've alrady written, insisting on requesting for actual > use-cases, when the architectural-issues are obviously already so glaring - > it would seem like a picky - borderline straw-men - argument. > I can't say I follow this line of reasoning. You implement architectures to solve problems -- if you can't articulate the problem you're solving with your architecture, I think you have bigger problems. Are you really saying you can't come up with a single use case where this would be beneficial? Anthony -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
