> In particular, with the idea of including volatile fields such as stock count, price or similar in the future.
As discussed on previous gitlab MRs, volatile data should not be mixed up with part fields. Fields are specifically non-volatile. If you want to make a proposal for some system to get this volatile data somehow, that is fine, but it needs to be kept separate from fields. -Jon On Sat, Apr 27, 2024 at 12:36 PM Rosy <[email protected]> wrote: > Hi Seth > > Thanks for the information. > > I think it would be possible to work with the "If-Modified-Since" method. > However, this would mean that if a change occurred, the entire component > master would be reloaded and not just the changes since the last call. The > goal with the parameter is that the server only sends the changes, allowing > quick work in the symbol chooser. > > Transfer encoding: chunked is certainly an exciting method but only > concerns the transfer between the client and the server. My idea is that > when the first page has arrived, the next page is loaded asynchronously and > during this time the JSON file is parsed and the parts are cached. > > I have attached the latest version of my document. > The primary change is that instead of the key "chooser": bool in the > fields, I inserted a key "type": int, which makes this a little more > modular and oh an example with the idea that volatile fields may be > supported in the future. > > Generally when I talk about the cache, I mean the parts objects within > Kicad. > Possibly an inherited cash register from LIB_SYMBOL supplemented with ID > and timestamp. > > For me it is generally important that working in Kicad is quick. > And since it is a real-time database that regularly contains new/different > data, I would like to develop a good caching concept. In particular, with > the idea of including volatile fields such as stock count, price or similar > in the future. > > Rosy > > On Thursday, April 25, 2024 at 8:10:30 PM UTC+2 [email protected] wrote: > >> Hi Rosy- >> >> These are standard HTTP headers that are supported by all major web >> server and caching software. >> >> If-Unmodified-Since: >> https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Unmodified-Since >> Transfer Encoding: >> https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding >> >> >> Seth >> >> [image: KiCad Services Corporation Logo] >> Seth Hillbrand >> *Lead Developer* >> +1-530-302-5483 <(530)%20302-5483> >> Long Beach, CA >> www.kipro-pcb.com [email protected] >> >> >> On Thu, Apr 25, 2024 at 1:55 AM Rosy <[email protected]> wrote: >> >>> For me it would be important that the data that is changed on the server >>> is recognized and adopted by KiCad. >>> Unfortunately, English is not my native language and I may not >>> understand exactly what you mean. Could you give me a small example of how >>> I should understand this or a source where I can read more about it? >>> >>> On Thursday, April 25, 2024 at 2:39:55 AM UTC+2 [email protected] >>> wrote: >>> >>>> Hi Rosy- >>>> >>>> Thanks for the interesting ideas on a version 2. >>>> >>>> One thing that I would like to focus on is utilizing standard >>>> mechanisms where possible. So, for example, in the section on caching, I >>>> would strongly prefer to utilize HTTP-based header calls instead of sending >>>> the caching data request to the web application. Here, we could utilize >>>> If-Unmodified-Since in a GET request and properly parse the response in >>>> KiCad (HTTP412) to see whether we could utilize locally cached data. >>>> Similarly, using transfer encoding instead of chunking the data in the >>>> backend will make the implementation more universal and speed for people. >>>> >>>> The reason for this preference is to allow the use of standard tools to >>>> optimize content delivery without our needing to code additional logic to >>>> support it. >>>> >>>> Seth >>>> >>>> [image: KiCad Services Corporation Logo] >>>> Seth Hillbrand >>>> *Lead Developer* >>>> +1-530-302-5483 <(530)%20302-5483> >>>> Long Beach, CA >>>> www.kipro-pcb.com [email protected] >>>> >>>> >>>> On Wed, Apr 24, 2024 at 2:57 PM Rosy <[email protected]> wrote: >>>> >>>>> Hello everyone >>>>> >>>>> I found a solution for issue >>>>> https://gitlab.com/kicad/code/kicad/-/issues/17584 that neither >>>>> changes the behavior of the library nor affects the performance. To do >>>>> this, I recorded the two merge requests: >>>>> master: https://gitlab.com/kicad/code/kicad/-/merge_requests/1914 >>>>> 8.0: https://gitlab.com/kicad/code/kicad/-/merge_requests/1925 >>>>> >>>>> I have also developed a solution for issue >>>>> https://gitlab.com/kicad/code/kicad/-/issues/17569 , which >>>>> occasionally checks before a symbol is loaded whether the cache has >>>>> already >>>>> been created and, if not, created first. >>>>> To do this, I recorded the two merge requests: >>>>> master: https://gitlab.com/kicad/code/kicad/-/merge_requests/1927 >>>>> 8.0: https://gitlab.com/kicad/code/kicad/-/merge_requests/1926 >>>>> >>>>> >>>>> I think it's important that dynamic libraries like the http library >>>>> can also add descriptions to the sublibraries. To do this, I recorded the >>>>> following merge requests: >>>>> master: https://gitlab.com/kicad/code/kicad/-/merge_requests/1921 >>>>> 8.0: https://gitlab.com/kicad/code/kicad/-/merge_requests/1919 >>>>> >>>>> I've had various thoughts about the issue >>>>> https://gitlab.com/kicad/code/kicad/-/issues/17570 and my other ideas. >>>>> One of the problems with the existing solution is that certain >>>>> requests return an array, making it almost impossible to expand. >>>>> I could imagine shouting a v2 of the http library. Attached I have a >>>>> document that describes how I imagine this. >>>>> >>>>> I would be happy to receive your feedback. >>>>> >>>>> On Monday, April 22, 2024 at 5:09:39 PM UTC+2 Rosy wrote: >>>>> >>>>>> My goal is that the library remains compatible with existing APIs but >>>>>> that the various issues can be solved. I think it is important that the >>>>>> same functions as the standard functions are available for the http >>>>>> libraries. The issues mentioned show where this is currently not >>>>>> possible. >>>>>> I also find it an important function that a description can be given >>>>>> for the categories, as is the case with the standard libraries. I haven't >>>>>> entered an issu for this but have already added an MR. >>>>>> >>>>>> Something that I noticed and that I would certainly like to change is >>>>>> that at the moment the parts are stored in different caches with >>>>>> different >>>>>> loading states. >>>>>> >>>>>> There should only be one part cache. >>>>>> >>>>>> I currently have the idea that another key called “fields” can be >>>>>> returned by the API when the root request is made. Only if this is sent >>>>>> by >>>>>> the API will the new features be activated, then the field list can be >>>>>> preloaded via a new endpoint. This enables short field keys even for >>>>>> fields >>>>>> with long names. >>>>>> I would also like to introduce modify fields that allow only those >>>>>> parts that have undergone updates to be reloaded. >>>>>> >>>>>> The communication could look something like this: >>>>>> >>>>>> # root >>>>>> { >>>>>> "categories" : "", >>>>>> "parts" : "", >>>>>> "fields" : "" //activate new fields features >>>>>> } >>>>>> >>>>>> # fields.json >>>>>> { >>>>>> "reference":{ >>>>>> "show" : true, >>>>>> "default_show_name" : false >>>>>> }, >>>>>> "value":{ >>>>>> "default_show" : true, >>>>>> "default_show_name" : false >>>>>> }, >>>>>> "footprint":{ >>>>>> "default_show" : false, >>>>>> "default_show_name" : false >>>>>> }, >>>>>> "datasheet":{ >>>>>> "default_show" : false, >>>>>> "default_show_name" : false >>>>>> }, >>>>>> "description":{ >>>>>> "default_show" : false, >>>>>> "default_show_name" : false >>>>>> }, >>>>>> "keywords" : {}, >>>>>> "mfn" :{ //must be unique >>>>>> "name" : "Manufacturer Number", //must be unique >>>>>> "show_on_chooser" : true, //only needed if not false >>>>>> "default_show" : false, //only needed if not false >>>>>> "default_show_name" : false //only needed if not false >>>>>> }, >>>>>> "mfurl" : { //must be unique >>>>>> "name" : "Manufacturer URL", //must be unique >>>>>> "show_on_chooser" : false, //only needed if not false >>>>>> "default_show" : false, //only needed if not false >>>>>> "default_show_name" : false //only needed if not false >>>>>> } >>>>>> } >>>>>> >>>>>> # categories.json >>>>>> [ >>>>>> { >>>>>> "id" : "1", //must be unique >>>>>> "name" : "Resistor", //must be unique >>>>>> "description" : "Thick Film SMD Resistors", >>>>>> "modify" : 1713798563 //datetime seconds since 1.1.1970 >>>>>> }, >>>>>> { >>>>>> "id" : "2", //must be unique >>>>>> "name" : "Capacitor", //must be unique >>>>>> "description" : "SMD MLCC Capacitors", >>>>>> "modify" : 1713798563 //datetime seconds since 1.1.1970 >>>>>> } >>>>>> ] >>>>>> >>>>>> # /parts/category/{category_id}.json >>>>>> { >>>>>> "parts":[ >>>>>> { >>>>>> "id" : "1", //must be unique >>>>>> "name" : "MF-ab123", //must be unique >>>>>> "modify" : 1713798563 //datetime seconds since 1.1.1970 >>>>>> "value" : "12k", >>>>>> "description" : "Tick Film SMD Resistor 12kΩ, 1%", >>>>>> "keywords" : "12kOhm", >>>>>> "mfn": "ab123" >>>>>> }, >>>>>> { >>>>>> "id" : "2", //must be unique >>>>>> "name" : "MF-ab102", //must be unique >>>>>> "modify" : 1713798563 //datetime seconds since 1.1.1970 >>>>>> "value" : "1k", >>>>>> "description" : "Tick Film SMD Resistor 1kΩ, 1%", >>>>>> "keywords" : "1kOhm", >>>>>> "mfn": "ab102" >>>>>> } >>>>>> ] >>>>>> } >>>>>> >>>>>> parts/{part_id}.json >>>>>> { >>>>>> "id" : "2", >>>>>> "symbolIdStr": "passive:R", >>>>>> "exclude_from_bom": "False", >>>>>> "exclude_from_board": "False", >>>>>> "exclude_from_sim": "True", >>>>>> "fields":{ >>>>>> "reference":{ >>>>>> "value" : "R", //can not be empty or a number >>>>>> "show" : true, //only needed if not default >>>>>> "show_name" : false //only needed if not default >>>>>> }, >>>>>> "value":{ //only needed if not default for show or show_name >>>>>> //value ignored if is preloaded >>>>>> "show" : true, //only needed if not default >>>>>> "show_name" : false //only needed if not default >>>>>> }, >>>>>> "footprint":{ //only needed if exists or not default for show or >>>>>> show_name >>>>>> "value" : "resistor:R", //only needed if exists >>>>>> "show" : false, //only needed if not default >>>>>> "show_name" : false //only needed if not default >>>>>> }, >>>>>> "datasheet":{ //only needed if exists or not default for show or >>>>>> show_name >>>>>> "value" : "$(DATA)/part.pdf", //only needed if exists >>>>>> "show" : false, //only needed if not default >>>>>> "show_name" : false //only needed if not default >>>>>> }, >>>>>> "description":{ //only needed if not default for show or show_name >>>>>> //value ignored if is preloaded >>>>>> "show" : false, >>>>>> "show_name" : false >>>>>> }, "mfn" :{ //only needed if not default for show or show_name >>>>>> //value ignored if is preloaded >>>>>> "show" : false, //only needed if not default >>>>>> "show_name" : false //only needed if not default >>>>>> }, >>>>>> "mfurl" :{ >>>>>> "value" : "http:mfn.com", >>>>>> "show" : false, //only needed if not default >>>>>> "show_name" : false //only needed if not default >>>>>> } >>>>>> }, >>>>>> "order" : { >>>>>> "mfn", "mfurl" //define the order in witch the filds are added to >>>>>> the symbol >>>>>> } >>>>>> } >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Monday, April 22, 2024 at 4:23:59 PM UTC+2 [email protected] >>>>>> wrote: >>>>>> >>>>>>> Hi Rosy, >>>>>>> >>>>>>> > I would be happy to invest time in revising the http library so >>>>>>> that the various problems can be solved, but I would like to know who I >>>>>>> should talk to so that the work is not in vain but is then accepted? >>>>>>> >>>>>>> This list is a good place, please say more about what you propose to >>>>>>> do before you start work. >>>>>>> >>>>>>> I have been away from KiCad this past week and haven't had the >>>>>>> chance >>>>>>> to look at the MRs yet. Maybe other developers will also chime in >>>>>>> here. >>>>>>> >>>>>>> -Jon >>>>>>> >>>>>>> On Mon, Apr 22, 2024 at 2:06 AM Rosy <[email protected]> wrote: >>>>>>> > >>>>>>> > In the meantime I have familiarized myself with the existing >>>>>>> solution and the code. It is now clear to me what the problem with 17569 >>>>>>> is. >>>>>>> > >>>>>>> > I would be happy to invest time in revising the http library so >>>>>>> that the various problems can be solved, but I would like to know who I >>>>>>> should talk to so that the work is not in vain but is then accepted? >>>>>>> > >>>>>>> > On Friday, April 19, 2024 at 9:36:39 AM UTC+2 Rosy wrote: >>>>>>> >> >>>>>>> >> Hello everyone >>>>>>> >> >>>>>>> >> I currently use KiCad privately and would like to use it in our >>>>>>> company in the future. >>>>>>> >> >>>>>>> >> It is important that there is a good solution to manage the >>>>>>> libraries centrally. The http library is ideal because it enables a >>>>>>> clean >>>>>>> separation of frontend and backend. >>>>>>> >> >>>>>>> >> Unfortunately, there are still a few things that I think need to >>>>>>> be improved a bit. >>>>>>> >> >>>>>>> >> I have currently created a merge request with which it is now >>>>>>> possible to provide the http sublibraries with a description, which I >>>>>>> consider to be an essential function and hope that it will be accepted >>>>>>> promptly. >>>>>>> >> >>>>>>> >> >>>>>>> >> I have also recorded the following issues regarding the http >>>>>>> library. >>>>>>> >> >>>>>>> >> https://gitlab.com/kicad/code/kicad/-/issues/17584 >>>>>>> >> >>>>>>> >> https://gitlab.com/kicad/code/kicad/-/issues/17570 >>>>>>> >> >>>>>>> >> https://gitlab.com/kicad/code/kicad/-/issues/17569 >>>>>>> >> >>>>>>> >> For issues 17584/17570 Andre Iwers and I have submitted the merge >>>>>>> request >>>>>>> >> 1910/1912/1914 >>>>>>> >> A possible solution has been developed, which is currently being >>>>>>> discussed in MR 1910, as opinions differ on fields that are only >>>>>>> displayed >>>>>>> in the Choose Symbol dialog. >>>>>>> >> What is your opinion? >>>>>>> >> Otherwise, I would be happy to work out another solution. >>>>>>> >> >>>>>>> >> Does anyone have some tips for where and how I could tackle >>>>>>> problem 17569? >>>>>>> >> >>>>>>> >> Thank you for your feedback. >>>>>>> > >>>>>>> > -- >>>>>>> > You received this message because you are subscribed to the Google >>>>>>> Groups "KiCad Developers" group. >>>>>>> > To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> > To view this discussion on the web visit >>>>>>> https://groups.google.com/a/kicad.org/d/msgid/devlist/a801fafd-930f-4b75-b718-2e706a06a0efn%40kicad.org. >>>>>>> >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "KiCad Developers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/a/kicad.org/d/msgid/devlist/519f23dc-97bf-4ce5-8c5b-be4197c3b938n%40kicad.org >>>>> <https://groups.google.com/a/kicad.org/d/msgid/devlist/519f23dc-97bf-4ce5-8c5b-be4197c3b938n%40kicad.org?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- > You received this message because you are subscribed to the Google Groups > "KiCad Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/a/kicad.org/d/msgid/devlist/c60cfe3b-fbff-4806-af87-749ac984acc9n%40kicad.org > <https://groups.google.com/a/kicad.org/d/msgid/devlist/c60cfe3b-fbff-4806-af87-749ac984acc9n%40kicad.org?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "KiCad Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/kicad.org/d/msgid/devlist/CA%2BqGbCCME8b2F9DOS-gL8feAoimQ34wA4mEw9JE_ARvid06kjA%40mail.gmail.com.
