Is there somewhere I can look at your work in progress code?
On 21 June 2014 19:57, Daniel Pyrathon <[email protected]> wrote: > Hi All, > > This week I have done the following > > *- Openend PR and merged Options (_meta) unittests* > https://github.com/django/django/pull/2823 > This is a working test suite of model/options.py file. Is is currently > testing the current Options implementation. > > *- Re-implemented test-suite in the new API* > My new proposed API has been implemented on top of the Options unittests. > This means new and old API give the same results! and therefore the new API > is a working re-implementation. > > *- Performance optimizations* > Currently, the biggest issue I have had (in the new API) regards > performance. I am doing small optimisations and benchmarking with cProfile. > If anyone has some good hints on how to benchmark single functions in > Django please let me know! > > Regards, > Dan > > > On Friday, June 13, 2014 10:11:41 PM UTC+2, Aymeric Augustin wrote: > >> I like this simplification a lot. I hope you can make it work. >> >> Just check that you don’t "overload" fields, by storing in field objects >> information that doesn’t belong there. >> >> -- >> Aymeric. >> >> >> >> On 13 juin 2014, at 19:53, Daniel Pyrathon <[email protected]> wrote: >> >> Hi All, >> >> This week's work was a follow-up of last week's work, with some new >> discoveries with regards to the new API: >> >> *1) Improved the current _meta implementation of unittests* >> I refactored the current meta unittest branch into a more compact >> version. I also added test cases for Generic Foreign Keys, RelatedField and >> more improvements on virtual fields. This section will actually be our >> first merge to master: a unit test suite for the current implementation. >> >> This branch is found here: https://github.com/ >> PirosB3/django/tree/soc2014_meta_unittests >> >> *2) Refactored the new _meta API spec* >> By implementing the new API, I found new redundancies in the current >> implementation that we can avoid in the new API spec: >> >> *1) Only return field instances* >> the current implementation has a common return pattern: (field_object, >> model, direct, m2m). After a few tests I realized that model, direct, m2m >> and field_name are all redundant and can be derived from only field_object. >> Since there are only 3-4 places that actually use direct and m2m it makes >> sense to remove this function from the new API spec. >> Here I show how all the information can be derived form field_instance ( >> https://gist.github.com/PirosB3/6cb4badbb1b8c2e41a96/revisions), I ran >> the entire test suite and things look good. The only issue I can see now is >> from a performance point of view. >> >> *2) Provide only 2 API endpoints* >> >> - get_fields(types, opts) -> (field_instance, field_instance, ...) >> - get_field(field_name) -> field_instance >> >> The rest is all (I might be very wrong! don't trust me) redundant. By >> continuing with this iterative approach, I will soon find out if I am >> correct or not ;) >> >> >> This branch is found here: https://github.com/ >> PirosB3/django/tree/soc2014_meta_refactor >> >> For any questions, as usual, let me know >> Dan >> >> >> On Friday, June 6, 2014 7:03:36 PM UTC+2, Daniel Pyrathon wrote: >>> >>> Hi All, >>> >>> Based on the work done last week, this week I have worked on the >>> following: >>> >>> *1) Covered the current _meta implementation of unittests* >>> The current Options is not covered by unit tests at all, I have created >>> the model_options test module containing one or more unit tests for each >>> endpoint and usage. Each endpoint is tested with many different models and >>> fields (data, m2m, related objects, related m2m, virtual, ..). Each unit >>> test asserts equality in field naming and field type. Endpoints that return >>> the model are also tested for equality. >>> >>> This branch is found here: https://github.com/ >>> PirosB3/django/tree/soc2014_meta_unittests >>> >>> *2) Pulled in tests from soc2014_meta_unittests and tested the new >>> implementation* >>> The previous branch that I developed on contains the new API >>> implementation, I have pulled in the tests from soc2014_meta_unittests and >>> I have switched the old API calls to the new API calls (with a few >>> adjustments). I have successfully made all tests pass even though I have >>> found some design issues that need to be addressed in my next update call. >>> >>> This branch is found here: https://github.com/ >>> PirosB3/django/tree/soc2014_meta_refactor >>> >>> *3) Created a new branch that maps the old implementation with the new* >>> Today I started putting my new API in "production". This is obviously >>> nowhere near a finalised version but it helps me spot some edge cases that >>> are not in the unit-tests. Each issue found has been converted into a >>> standalone unit-test and has been proved to fail. >>> Unfortunately, this made me realise of other design issues related to >>> the new return type of get_fields -> (field_name, field), A decision will >>> be taken on monday. >>> >>> This branch is found here: https://github.com/ >>> PirosB3/django/tree/soc2014_meta_refactor_implementation as is for >>> personal use only >>> >>> For any questions, let me know! >>> Dan >>> >>> >>> On Sunday, June 1, 2014 6:10:53 PM UTC+2, Daniel Pyrathon wrote: >>>> >>>> Hi All, >>>> >>>> An update on my side, some interesting work has happened this week: me >>>> and Russell have decided to start on the implementation early in order to >>>> understand better the internals of Options. Currently I am working on the >>>> following: >>>> >>>> *Providing one single function: get_fields(types, opts, **kwargs)* >>>> Previously we had identified a number of functions that contained >>>> similar data but had a different return type. We are going to provide one >>>> function that takes a set of field types + options and returns the same >>>> output everywhere: ((field_name, field), ...). This has the benefit of >>>> simplicity and is more maintainable than the previous approach. >>>> >>>> TYPES = DATA, M2M, FK, RELATED_OBJECT, RELATED_M2M >>>> OPTS = NONE, LOCAL_ONLY, CONCRETE, INCLUDE_HIDDEN, INCLUDE_PROXY, >>>> VIRTUAL >>>> >>>> *Providing two functions for retrieving details of a specific field* >>>> As specified in my previous document, in many parts of the code we just >>>> want to retrieve a field object by name, other times we have a field but >>>> need other metadata such as: owner (model_class), direct (bool), m2m >>>> (bool). We will provide two functions: >>>> >>>> get_field(field_name) -> field_instance >>>> >>>> get_field_details(field_instance) -> direct, m2m, (still to be defined) >>>> >>>> While we still have not entirely defined what *get_field_details *will >>>> return, this will be done soon. >>>> >>>> >>>> *Building a test suite for the existing API* >>>> The new API development will be driven by a test suite that will >>>> compare the current (legacy) API with the new implementation. While return >>>> types will be different, we are asserting that all the correct fields and >>>> metadata are returned. Building a test suite means we can start >>>> implementing before a final API spec is finalised. It also means we can >>>> iterate faster and, from my perspective, I also understand a lot more of >>>> the current implementation. We are testing each combination of fields and >>>> options together. >>>> >>>> My current implementation is visible here: https://github.com/ >>>> PirosB3/django/compare/soc2014_meta_refactor >>>> >>>> For any questions or suggestions, let me know. >>>> >>>> Daniel Pyrathon >>>> >>>> >>>> On Monday, May 26, 2014 1:28:31 AM UTC+2, Daniel Pyrathon wrote: >>>>> >>>>> Hi All, >>>>> >>>>> Just to make you know, I have put up the current _meta API >>>>> documentation here: >>>>> http://162.219.6.191:8000/ref/models/meta.html?highlight=_meta >>>>> As always, feel free to ask questions. >>>>> >>>>> Daniel >>>>> >>>>> On Monday, May 26, 2014 1:26:27 AM UTC+2, Daniel Pyrathon wrote: >>>>>> >>>>>> Hi Josh, >>>>>> >>>>>> The meta API specified in the docs (https://github.com/PirosB3/ >>>>>> django/blob/meta_documentation/docs/ref/models/meta.txt) is the >>>>>> current API. I have documented this in order to understand more of the >>>>>> current implementation and it will be good to show a comparison when a >>>>>> new >>>>>> meta API will be approved. >>>>>> >>>>>> My current proposal (https://gist.github.com/ >>>>>> PirosB3/371704ed40ed093d5a82) and it will be discussed tomorrow with >>>>>> Russell. I will post as soon as I have updates. >>>>>> >>>>>> Daniel Pyrathon >>>>>> >>>>>> On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote: >>>>>>> >>>>>>> Hi Daniel, >>>>>>> >>>>>>> Nice work putting that document together. Is the meta document you >>>>>>> put together the current API or is it the API you are proposing? If the >>>>>>> latter, a few suggestions (and if others disagree, please point that >>>>>>> out): >>>>>>> >>>>>>> - Remove all mention of caching. That should be an implementation >>>>>>> detail only, and not a requirement for other implementations. >>>>>>> - the *_with_model methods really rub me up the wrong way. I would >>>>>>> prefer always returning the _with_model variant, and letting the caller >>>>>>> discard the model if they don't need it. >>>>>>> - I'm not a fan of virtual and concrete fields, though I have to >>>>>>> admit I'm not sure how they're different, especially in the context of >>>>>>> different implementations. >>>>>>> - Not sure that m2m should be differentiated from related. >>>>>>> - init_name_map should be an implementation detail. >>>>>>> - normalize_together should be an implementation detail. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Josh >>>>>>> >>>>>>> On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote: >>>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> In the last days I have built a documentation of the current state >>>>>>>> of Options. Based on feedback and prototyping I have thought of a >>>>>>>> potential >>>>>>>> interface for _meta that can solve the issues currently present, such >>>>>>>> as >>>>>>>> redundancy (in code and in caching systems). The interface has also >>>>>>>> been >>>>>>>> thought to be maintainable and is a base that can be used to create >>>>>>>> custom >>>>>>>> meta stores. >>>>>>>> Obviously this is far from perfect, It will need many iterations >>>>>>>> and maybe it is too complex. I would really love to gain as much >>>>>>>> feedback >>>>>>>> as possible so it can be discussed with Russell and the community on >>>>>>>> Monday. >>>>>>>> >>>>>>>> The documentation of _meta can be found here: https://github.com/ >>>>>>>> PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt >>>>>>>> I will be refining the document in the next days, I will also be >>>>>>>> publishing the docs on a webserver and will be linking a URL soon. >>>>>>>> >>>>>>>> My proposal has been published here: >>>>>>>> https://gist.github.com/PirosB3/371704ed40ed093d5a82 >>>>>>>> In the next days I will be iterating over the feedback gained, and >>>>>>>> based on one very interesting suggestion on IRC, I will try to see how >>>>>>>> my >>>>>>>> API syntax looks in modelforms.py. >>>>>>>> >>>>>>>> As said previously, and feedback is greatly appreciated. >>>>>>>> >>>>>>>> Hi from Pycon IT! >>>>>>>> >>>>>>>> Daniel Pyrathon >>>>>>>> >>>>>>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: >>>>>>>>> >>>>>>>>> Best of luck! >>>>>>>>> >>>>>>>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: >>>>>>>>>> >>>>>>>>>> Hi All, >>>>>>>>>> >>>>>>>>>> Today I will be starting my weekly updates on my SoC project: >>>>>>>>>> refactoring Meta to a stable API. For anyone who missed out, you >>>>>>>>>> will be >>>>>>>>>> able to view it here: https://docs.google.com/document/d/1yp2_ >>>>>>>>>> skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit >>>>>>>>>> >>>>>>>>>> This week is the first official week of SoC. Me and my mentor >>>>>>>>>> (Russell) are initially approaching the work in the following way: >>>>>>>>>> >>>>>>>>>> - *Document the existing Meta API* >>>>>>>>>> For each endpoint, document the following: >>>>>>>>>> - Input parameters and return type >>>>>>>>>> - Caching pattern used >>>>>>>>>> - Where it's called from (internally and externally to Meta) >>>>>>>>>> - Why is it being called >>>>>>>>>> - When is it being called >>>>>>>>>> >>>>>>>>>> - *Propose an initial refactor plan* >>>>>>>>>> Once the documentation has been done, I should have a better >>>>>>>>>> idea of the current implementation. This will allow me to mock a >>>>>>>>>> proposed >>>>>>>>>> implementation that will be reviewed at my next update call, on >>>>>>>>>> Monday. >>>>>>>>>> >>>>>>>>>> My next update will be posted on Friday, just to make sure the >>>>>>>>>> community is informed of my progress. For any major updates that >>>>>>>>>> require >>>>>>>>>> community approval, I will be creating separate threads. >>>>>>>>>> My name on the internet is pirosb3, so if you want to have a chat >>>>>>>>>> about my progress feel free to contact me! The branch I am currently >>>>>>>>>> working on is https://github.com/PirosB3/ >>>>>>>>>> django/tree/meta_documentation >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Daniel Pyrathon >>>>>>>>>> >>>>>>>>> >>>>>>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: >>>>>>>>> >>>>>>>>> Best of luck! >>>>>>>>> >>>>>>>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: >>>>>>>>>> >>>>>>>>>> Hi All, >>>>>>>>>> >>>>>>>>>> Today I will be starting my weekly updates on my SoC project: >>>>>>>>>> refactoring Meta to a stable API. For anyone who missed out, you >>>>>>>>>> will be >>>>>>>>>> able to view it here: https://docs.google.com/document/d/1yp2_ >>>>>>>>>> skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit >>>>>>>>>> >>>>>>>>>> This week is the first official week of SoC. Me and my mentor >>>>>>>>>> (Russell) are initially approaching the work in the following way: >>>>>>>>>> >>>>>>>>>> - *Document the existing Meta API* >>>>>>>>>> For each endpoint, document the following: >>>>>>>>>> - Input parameters and return type >>>>>>>>>> - Caching pattern used >>>>>>>>>> - Where it's called from (internally and externally to Meta) >>>>>>>>>> - Why is it being called >>>>>>>>>> - When is it being called >>>>>>>>>> >>>>>>>>>> - *Propose an initial refactor plan* >>>>>>>>>> Once the documentation has been done, I should have a better >>>>>>>>>> idea of the current implementation. This will allow me to mock a >>>>>>>>>> proposed >>>>>>>>>> implementation that will be reviewed at my next update call, on >>>>>>>>>> Monday. >>>>>>>>>> >>>>>>>>>> My next update will be posted on Friday, just to make sure the >>>>>>>>>> community is informed of my progress. For any major updates that >>>>>>>>>> require >>>>>>>>>> community approval, I will be creating separate threads. >>>>>>>>>> My name on the internet is pirosb3, so if you want to have a chat >>>>>>>>>> about my progress feel free to contact me! The branch I am currently >>>>>>>>>> working on is https://github.com/PirosB3/ >>>>>>>>>> django/tree/meta_documentation >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Daniel Pyrathon >>>>>>>>>> >>>>>>>>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> >> Visit this group at http://groups.google.com/group/django-developers. >> To view this discussion on the web visit https://groups.google. >> com/d/msgid/django-developers/78f45c61-9626-4aa9-a854- >> 64b11ba44572%40googlegroups.com >> <https://groups.google.com/d/msgid/django-developers/78f45c61-9626-4aa9-a854-64b11ba44572%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/django-developers. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/445fea1d-406c-4ae9-b869-c02f189a2b86%40googlegroups.com > <https://groups.google.com/d/msgid/django-developers/445fea1d-406c-4ae9-b869-c02f189a2b86%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAG_XiSCpV2nrftzUKwTaaFyuZHaxZhMBLPBC1sCZyqtbTUNNXw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
