Sure! https://github.com/PirosB3/django/compare/soc2014_meta_refactor_performance?expand=1
This contains the following: - Recently merged unit test suite for model/options - New API implementation: get_new_fields, get_new_field. - Implementation of recently merged unit test suite with new API - Small optimizations (BIG work in progress) If you have any suggestions please let me know! Dan On Sunday, June 22, 2014 2:14:56 AM UTC+2, Curtis Maloney wrote: > > Is there somewhere I can look at your work in progress code? > > > On 21 June 2014 19:57, Daniel Pyrathon <pir...@gmail.com <javascript:>> > wrote: > >> Hi All, >> >> This week I have done the following >> >> *- Openend PR and merged Options (_meta) unittests* >> https://github.com/django/django/pull/2823 >> This is a working test suite of model/options.py file. Is is currently >> testing the current Options implementation. >> >> *- Re-implemented test-suite in the new API* >> My new proposed API has been implemented on top of the Options unittests. >> This means new and old API give the same results! and therefore the new API >> is a working re-implementation. >> >> *- Performance optimizations* >> Currently, the biggest issue I have had (in the new API) regards >> performance. I am doing small optimisations and benchmarking with cProfile. >> If anyone has some good hints on how to benchmark single functions in >> Django please let me know! >> >> Regards, >> Dan >> >> >> On Friday, June 13, 2014 10:11:41 PM UTC+2, Aymeric Augustin wrote: >> >>> I like this simplification a lot. I hope you can make it work. >>> >>> Just check that you don’t "overload" fields, by storing in field objects >>> information that doesn’t belong there. >>> >>> -- >>> Aymeric. >>> >>> >>> >>> On 13 juin 2014, at 19:53, Daniel Pyrathon <pir...@gmail.com> wrote: >>> >>> Hi All, >>> >>> This week's work was a follow-up of last week's work, with some new >>> discoveries with regards to the new API: >>> >>> *1) Improved the current _meta implementation of unittests* >>> I refactored the current meta unittest branch into a more compact >>> version. I also added test cases for Generic Foreign Keys, RelatedField and >>> more improvements on virtual fields. This section will actually be our >>> first merge to master: a unit test suite for the current implementation. >>> >>> This branch is found here: https://github.com/ >>> PirosB3/django/tree/soc2014_meta_unittests >>> >>> *2) Refactored the new _meta API spec* >>> By implementing the new API, I found new redundancies in the current >>> implementation that we can avoid in the new API spec: >>> >>> *1) Only return field instances* >>> the current implementation has a common return pattern: (field_object, >>> model, direct, m2m). After a few tests I realized that model, direct, m2m >>> and field_name are all redundant and can be derived from only field_object. >>> Since there are only 3-4 places that actually use direct and m2m it makes >>> sense to remove this function from the new API spec. >>> Here I show how all the information can be derived form field_instance ( >>> https://gist.github.com/PirosB3/6cb4badbb1b8c2e41a96/revisions), I ran >>> the entire test suite and things look good. The only issue I can see now is >>> from a performance point of view. >>> >>> *2) Provide only 2 API endpoints* >>> >>> - get_fields(types, opts) -> (field_instance, field_instance, ...) >>> - get_field(field_name) -> field_instance >>> >>> The rest is all (I might be very wrong! don't trust me) redundant. By >>> continuing with this iterative approach, I will soon find out if I am >>> correct or not ;) >>> >>> >>> This branch is found here: https://github.com/ >>> PirosB3/django/tree/soc2014_meta_refactor >>> >>> For any questions, as usual, let me know >>> Dan >>> >>> >>> On Friday, June 6, 2014 7:03:36 PM UTC+2, Daniel Pyrathon wrote: >>>> >>>> Hi All, >>>> >>>> Based on the work done last week, this week I have worked on the >>>> following: >>>> >>>> *1) Covered the current _meta implementation of unittests* >>>> The current Options is not covered by unit tests at all, I have created >>>> the model_options test module containing one or more unit tests for each >>>> endpoint and usage. Each endpoint is tested with many different models and >>>> fields (data, m2m, related objects, related m2m, virtual, ..). Each unit >>>> test asserts equality in field naming and field type. Endpoints that >>>> return >>>> the model are also tested for equality. >>>> >>>> This branch is found here: https://github.com/ >>>> PirosB3/django/tree/soc2014_meta_unittests >>>> >>>> *2) Pulled in tests from soc2014_meta_unittests and tested the new >>>> implementation* >>>> The previous branch that I developed on contains the new API >>>> implementation, I have pulled in the tests from soc2014_meta_unittests and >>>> I have switched the old API calls to the new API calls (with a few >>>> adjustments). I have successfully made all tests pass even though I have >>>> found some design issues that need to be addressed in my next update call. >>>> >>>> This branch is found here: https://github.com/ >>>> PirosB3/django/tree/soc2014_meta_refactor >>>> >>>> *3) Created a new branch that maps the old implementation with the new* >>>> Today I started putting my new API in "production". This is obviously >>>> nowhere near a finalised version but it helps me spot some edge cases that >>>> are not in the unit-tests. Each issue found has been converted into a >>>> standalone unit-test and has been proved to fail. >>>> Unfortunately, this made me realise of other design issues related to >>>> the new return type of get_fields -> (field_name, field), A decision will >>>> be taken on monday. >>>> >>>> This branch is found here: https://github.com/ >>>> PirosB3/django/tree/soc2014_meta_refactor_implementation as is for >>>> personal use only >>>> >>>> For any questions, let me know! >>>> Dan >>>> >>>> >>>> On Sunday, June 1, 2014 6:10:53 PM UTC+2, Daniel Pyrathon wrote: >>>>> >>>>> Hi All, >>>>> >>>>> An update on my side, some interesting work has happened this week: me >>>>> and Russell have decided to start on the implementation early in order to >>>>> understand better the internals of Options. Currently I am working on the >>>>> following: >>>>> >>>>> *Providing one single function: get_fields(types, opts, **kwargs)* >>>>> Previously we had identified a number of functions that contained >>>>> similar data but had a different return type. We are going to provide one >>>>> function that takes a set of field types + options and returns the same >>>>> output everywhere: ((field_name, field), ...). This has the benefit of >>>>> simplicity and is more maintainable than the previous approach. >>>>> >>>>> TYPES = DATA, M2M, FK, RELATED_OBJECT, RELATED_M2M >>>>> OPTS = NONE, LOCAL_ONLY, CONCRETE, INCLUDE_HIDDEN, INCLUDE_PROXY, >>>>> VIRTUAL >>>>> >>>>> *Providing two functions for retrieving details of a specific field* >>>>> As specified in my previous document, in many parts of the code we >>>>> just want to retrieve a field object by name, other times we have a field >>>>> but need other metadata such as: owner (model_class), direct (bool), m2m >>>>> (bool). We will provide two functions: >>>>> >>>>> get_field(field_name) -> field_instance >>>>> >>>>> get_field_details(field_instance) -> direct, m2m, (still to be >>>>> defined) >>>>> >>>>> While we still have not entirely defined what *get_field_details *will >>>>> return, this will be done soon. >>>>> >>>>> >>>>> *Building a test suite for the existing API* >>>>> The new API development will be driven by a test suite that will >>>>> compare the current (legacy) API with the new implementation. While >>>>> return >>>>> types will be different, we are asserting that all the correct fields and >>>>> metadata are returned. Building a test suite means we can start >>>>> implementing before a final API spec is finalised. It also means we can >>>>> iterate faster and, from my perspective, I also understand a lot more of >>>>> the current implementation. We are testing each combination of fields and >>>>> options together. >>>>> >>>>> My current implementation is visible here: https://github.com/ >>>>> PirosB3/django/compare/soc2014_meta_refactor >>>>> >>>>> For any questions or suggestions, let me know. >>>>> >>>>> Daniel Pyrathon >>>>> >>>>> >>>>> On Monday, May 26, 2014 1:28:31 AM UTC+2, Daniel Pyrathon wrote: >>>>>> >>>>>> Hi All, >>>>>> >>>>>> Just to make you know, I have put up the current _meta API >>>>>> documentation here: >>>>>> http://162.219.6.191:8000/ref/models/meta.html?highlight=_meta >>>>>> As always, feel free to ask questions. >>>>>> >>>>>> Daniel >>>>>> >>>>>> On Monday, May 26, 2014 1:26:27 AM UTC+2, Daniel Pyrathon wrote: >>>>>>> >>>>>>> Hi Josh, >>>>>>> >>>>>>> The meta API specified in the docs (https://github.com/PirosB3/ >>>>>>> django/blob/meta_documentation/docs/ref/models/meta.txt) is the >>>>>>> current API. I have documented this in order to understand more of the >>>>>>> current implementation and it will be good to show a comparison when a >>>>>>> new >>>>>>> meta API will be approved. >>>>>>> >>>>>>> My current proposal (https://gist.github.com/ >>>>>>> PirosB3/371704ed40ed093d5a82) and it will be discussed tomorrow >>>>>>> with Russell. I will post as soon as I have updates. >>>>>>> >>>>>>> Daniel Pyrathon >>>>>>> >>>>>>> On Saturday, May 24, 2014 10:37:49 AM UTC+2, Josh Smeaton wrote: >>>>>>>> >>>>>>>> Hi Daniel, >>>>>>>> >>>>>>>> Nice work putting that document together. Is the meta document you >>>>>>>> put together the current API or is it the API you are proposing? If >>>>>>>> the >>>>>>>> latter, a few suggestions (and if others disagree, please point that >>>>>>>> out): >>>>>>>> >>>>>>>> - Remove all mention of caching. That should be an implementation >>>>>>>> detail only, and not a requirement for other implementations. >>>>>>>> - the *_with_model methods really rub me up the wrong way. I would >>>>>>>> prefer always returning the _with_model variant, and letting the >>>>>>>> caller >>>>>>>> discard the model if they don't need it. >>>>>>>> - I'm not a fan of virtual and concrete fields, though I have to >>>>>>>> admit I'm not sure how they're different, especially in the context of >>>>>>>> different implementations. >>>>>>>> - Not sure that m2m should be differentiated from related. >>>>>>>> - init_name_map should be an implementation detail. >>>>>>>> - normalize_together should be an implementation detail. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Josh >>>>>>>> >>>>>>>> On Saturday, 24 May 2014 05:05:02 UTC+10, Daniel Pyrathon wrote: >>>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> In the last days I have built a documentation of the current state >>>>>>>>> of Options. Based on feedback and prototyping I have thought of a >>>>>>>>> potential >>>>>>>>> interface for _meta that can solve the issues currently present, such >>>>>>>>> as >>>>>>>>> redundancy (in code and in caching systems). The interface has also >>>>>>>>> been >>>>>>>>> thought to be maintainable and is a base that can be used to create >>>>>>>>> custom >>>>>>>>> meta stores. >>>>>>>>> Obviously this is far from perfect, It will need many iterations >>>>>>>>> and maybe it is too complex. I would really love to gain as much >>>>>>>>> feedback >>>>>>>>> as possible so it can be discussed with Russell and the community on >>>>>>>>> Monday. >>>>>>>>> >>>>>>>>> The documentation of _meta can be found here: https://github.com/ >>>>>>>>> PirosB3/django/blob/meta_documentation/docs/ref/models/meta.txt >>>>>>>>> I will be refining the document in the next days, I will also be >>>>>>>>> publishing the docs on a webserver and will be linking a URL soon. >>>>>>>>> >>>>>>>>> My proposal has been published here: >>>>>>>>> https://gist.github.com/PirosB3/371704ed40ed093d5a82 >>>>>>>>> In the next days I will be iterating over the feedback gained, and >>>>>>>>> based on one very interesting suggestion on IRC, I will try to see >>>>>>>>> how my >>>>>>>>> API syntax looks in modelforms.py. >>>>>>>>> >>>>>>>>> As said previously, and feedback is greatly appreciated. >>>>>>>>> >>>>>>>>> Hi from Pycon IT! >>>>>>>>> >>>>>>>>> Daniel Pyrathon >>>>>>>>> >>>>>>>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: >>>>>>>>>> >>>>>>>>>> Best of luck! >>>>>>>>>> >>>>>>>>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: >>>>>>>>>>> >>>>>>>>>>> Hi All, >>>>>>>>>>> >>>>>>>>>>> Today I will be starting my weekly updates on my SoC project: >>>>>>>>>>> refactoring Meta to a stable API. For anyone who missed out, you >>>>>>>>>>> will be >>>>>>>>>>> able to view it here: https://docs.google.com/document/d/1yp2_ >>>>>>>>>>> skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit >>>>>>>>>>> >>>>>>>>>>> This week is the first official week of SoC. Me and my mentor >>>>>>>>>>> (Russell) are initially approaching the work in the following way: >>>>>>>>>>> >>>>>>>>>>> - *Document the existing Meta API* >>>>>>>>>>> For each endpoint, document the following: >>>>>>>>>>> - Input parameters and return type >>>>>>>>>>> - Caching pattern used >>>>>>>>>>> - Where it's called from (internally and externally to >>>>>>>>>>> Meta) >>>>>>>>>>> - Why is it being called >>>>>>>>>>> - When is it being called >>>>>>>>>>> >>>>>>>>>>> - *Propose an initial refactor plan* >>>>>>>>>>> Once the documentation has been done, I should have a better >>>>>>>>>>> idea of the current implementation. This will allow me to mock a >>>>>>>>>>> proposed >>>>>>>>>>> implementation that will be reviewed at my next update call, on >>>>>>>>>>> Monday. >>>>>>>>>>> >>>>>>>>>>> My next update will be posted on Friday, just to make sure the >>>>>>>>>>> community is informed of my progress. For any major updates that >>>>>>>>>>> require >>>>>>>>>>> community approval, I will be creating separate threads. >>>>>>>>>>> My name on the internet is pirosb3, so if you want to have a >>>>>>>>>>> chat about my progress feel free to contact me! The branch I am >>>>>>>>>>> currently >>>>>>>>>>> working on is https://github.com/PirosB3/ >>>>>>>>>>> django/tree/meta_documentation >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Daniel Pyrathon >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> On Tuesday, May 20, 2014 3:25:45 PM UTC+2, Josh Smeaton wrote: >>>>>>>>>> >>>>>>>>>> Best of luck! >>>>>>>>>> >>>>>>>>>> On Tuesday, 20 May 2014 03:56:06 UTC+10, Daniel Pyrathon wrote: >>>>>>>>>>> >>>>>>>>>>> Hi All, >>>>>>>>>>> >>>>>>>>>>> Today I will be starting my weekly updates on my SoC project: >>>>>>>>>>> refactoring Meta to a stable API. For anyone who missed out, you >>>>>>>>>>> will be >>>>>>>>>>> able to view it here: https://docs.google.com/document/d/1yp2_ >>>>>>>>>>> skqkxyrc0egdRv6ofnRGCI9nmvxDFBkCXgy0Jwo/edit >>>>>>>>>>> >>>>>>>>>>> This week is the first official week of SoC. Me and my mentor >>>>>>>>>>> (Russell) are initially approaching the work in the following way: >>>>>>>>>>> >>>>>>>>>>> - *Document the existing Meta API* >>>>>>>>>>> For each endpoint, document the following: >>>>>>>>>>> - Input parameters and return type >>>>>>>>>>> - Caching pattern used >>>>>>>>>>> - Where it's called from (internally and externally to >>>>>>>>>>> Meta) >>>>>>>>>>> - Why is it being called >>>>>>>>>>> - When is it being called >>>>>>>>>>> >>>>>>>>>>> - *Propose an initial refactor plan* >>>>>>>>>>> Once the documentation has been done, I should have a better >>>>>>>>>>> idea of the current implementation. This will allow me to mock a >>>>>>>>>>> proposed >>>>>>>>>>> implementation that will be reviewed at my next update call, on >>>>>>>>>>> Monday. >>>>>>>>>>> >>>>>>>>>>> My next update will be posted on Friday, just to make sure the >>>>>>>>>>> community is informed of my progress. For any major updates that >>>>>>>>>>> require >>>>>>>>>>> community approval, I will be creating separate threads. >>>>>>>>>>> My name on the internet is pirosb3, so if you want to have a >>>>>>>>>>> chat about my progress feel free to contact me! The branch I am >>>>>>>>>>> currently >>>>>>>>>>> working on is https://github.com/PirosB3/ >>>>>>>>>>> django/tree/meta_documentation >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Daniel Pyrathon >>>>>>>>>>> >>>>>>>>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Django developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to django-developers+unsubscr...@googlegroups.com. >>> To post to this group, send email to django-d...@googlegroups.com. >>> >>> Visit this group at http://groups.google.com/group/django-developers. >>> To view this discussion on the web visit https://groups.google. >>> com/d/msgid/django-developers/78f45c61-9626-4aa9-a854- >>> 64b11ba44572%40googlegroups.com >>> <https://groups.google.com/d/msgid/django-developers/78f45c61-9626-4aa9-a854-64b11ba44572%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-develop...@googlegroups.com <javascript:>. >> To post to this group, send email to django-d...@googlegroups.com >> <javascript:>. >> Visit this group at http://groups.google.com/group/django-developers. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-developers/445fea1d-406c-4ae9-b869-c02f189a2b86%40googlegroups.com >> >> <https://groups.google.com/d/msgid/django-developers/445fea1d-406c-4ae9-b869-c02f189a2b86%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/d7334506-3b2f-4e46-aae0-ca66d190c05a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
