You're right, I would definitely like to do this separately. I just wanted to keep the interface so dynamic that a future step in this direction would not be obstructed. Which is why I made a type for fields and not just a flag.
What is your view on caching? Thank you for all the artistic contributions. On Saturday, April 27, 2024 at 7:48:16 PM UTC+2 [email protected] wrote: > > My idea, which has not yet been thought of, is that the LIB_SYMBOL has a > separate list of volatile fields that are only for the symbol chooser and > may also be displayed differently, for example with italics or something > like that. > > If volatile data is implemented, it should be supported in more places > than just the symbol chooser. People will want to query updated volatile > data for parts already in the design, not just look up parts in the symbol > chooser. > > In general I think it would be best to completely separate these two > ideas. If you have some improvements in mind for the HTTP libraries, this > should be done independently from a proposal for a volatile data API. > Doing it this way would mean that the volatile data API could be more > broadly useful: for example, people may want to use it with parts that are > not placed from a HTTP library. > > -Jon > > On Sat, Apr 27, 2024 at 1:30 PM Rosy <[email protected]> wrote: > >> Hi Jon >> >> Thank you for your feedback, I am aware of the discussion and that is >> exactly why this is my suggestion, that this is an absolutely separate type >> and has nothing to do with the common fields. Even the server query is >> handled separately as this is definitely not used by every API. >> >> My idea, which has not yet been thought of, is that the LIB_SYMBOL has a >> separate list of volatile fields that are only for the symbol chooser and >> may also be displayed differently, for example with italics or something >> like that. >> >> I definitely wouldn't implement the whole part with the volatile fields >> in the first step; it's just important to me that it's already taken into >> account in the solution. >> >> By the way, the current status of my documentation is below >> https://gitlab.com/RosyDev/kicad-dev-docs/-/blob/http-lib-v2/content/apis-and-binding/http-libraries/http-lib-v2-00.adoc?ref_type= >> >> heads visible. >> >> greetings >> Rosy >> On Saturday, April 27, 2024 at 7:11:32 PM UTC+2 [email protected] wrote: >> >>> > In particular, with the idea of including volatile fields such as >>> stock count, price or similar in the future. >>> >>> As discussed on previous gitlab MRs, volatile data should not be mixed >>> up with part fields. >>> >>> Fields are specifically non-volatile. >>> >>> If you want to make a proposal for some system to get this volatile data >>> somehow, that is fine, but it needs to be kept separate from fields. >>> >>> -Jon >>> >>> On Sat, Apr 27, 2024 at 12:36 PM Rosy <[email protected]> wrote: >>> >>>> Hi Seth >>>> >>>> Thanks for the information. >>>> >>>> I think it would be possible to work with the "If-Modified-Since" >>>> method. However, this would mean that if a change occurred, the entire >>>> component master would be reloaded and not just the changes since the last >>>> call. The goal with the parameter is that the server only sends the >>>> changes, allowing quick work in the symbol chooser. >>>> >>>> Transfer encoding: chunked is certainly an exciting method but only >>>> concerns the transfer between the client and the server. My idea is that >>>> when the first page has arrived, the next page is loaded asynchronously >>>> and >>>> during this time the JSON file is parsed and the parts are cached. >>>> >>>> I have attached the latest version of my document. >>>> The primary change is that instead of the key "chooser": bool in the >>>> fields, I inserted a key "type": int, which makes this a little more >>>> modular and oh an example with the idea that volatile fields may be >>>> supported in the future. >>>> >>>> Generally when I talk about the cache, I mean the parts objects within >>>> Kicad. >>>> Possibly an inherited cash register from LIB_SYMBOL supplemented with >>>> ID and timestamp. >>>> >>>> For me it is generally important that working in Kicad is quick. >>>> And since it is a real-time database that regularly contains >>>> new/different data, I would like to develop a good caching concept. In >>>> particular, with the idea of including volatile fields such as stock >>>> count, >>>> price or similar in the future. >>>> >>>> Rosy >>>> >>>> On Thursday, April 25, 2024 at 8:10:30 PM UTC+2 [email protected] >>>> wrote: >>>> >>>>> Hi Rosy- >>>>> >>>>> These are standard HTTP headers that are supported by all major web >>>>> server and caching software. >>>>> >>>>> If-Unmodified-Since: >>>>> https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Unmodified-Since >>>>> Transfer Encoding: >>>>> https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding >>>>> >>>>> >>>>> Seth >>>>> >>>>> [image: KiCad Services Corporation Logo] >>>>> Seth Hillbrand >>>>> *Lead Developer* >>>>> +1-530-302-5483 <(530)%20302-5483> >>>>> Long Beach, CA >>>>> www.kipro-pcb.com [email protected] >>>>> >>>>> >>>>> On Thu, Apr 25, 2024 at 1:55 AM Rosy <[email protected]> wrote: >>>>> >>>>>> For me it would be important that the data that is changed on the >>>>>> server is recognized and adopted by KiCad. >>>>>> Unfortunately, English is not my native language and I may not >>>>>> understand exactly what you mean. Could you give me a small example of >>>>>> how >>>>>> I should understand this or a source where I can read more about it? >>>>>> >>>>>> On Thursday, April 25, 2024 at 2:39:55 AM UTC+2 [email protected] >>>>>> wrote: >>>>>> >>>>>>> Hi Rosy- >>>>>>> >>>>>>> Thanks for the interesting ideas on a version 2. >>>>>>> >>>>>>> One thing that I would like to focus on is utilizing standard >>>>>>> mechanisms where possible. So, for example, in the section on caching, >>>>>>> I >>>>>>> would strongly prefer to utilize HTTP-based header calls instead of >>>>>>> sending >>>>>>> the caching data request to the web application. Here, we could >>>>>>> utilize >>>>>>> If-Unmodified-Since in a GET request and properly parse the response in >>>>>>> KiCad (HTTP412) to see whether we could utilize locally cached data. >>>>>>> Similarly, using transfer encoding instead of chunking the data in the >>>>>>> backend will make the implementation more universal and speed for >>>>>>> people. >>>>>>> >>>>>>> The reason for this preference is to allow the use of standard tools >>>>>>> to optimize content delivery without our needing to code additional >>>>>>> logic >>>>>>> to support it. >>>>>>> >>>>>>> Seth >>>>>>> >>>>>>> [image: KiCad Services Corporation Logo] >>>>>>> Seth Hillbrand >>>>>>> *Lead Developer* >>>>>>> +1-530-302-5483 <(530)%20302-5483> >>>>>>> Long Beach, CA >>>>>>> www.kipro-pcb.com [email protected] >>>>>>> >>>>>>> >>>>>>> On Wed, Apr 24, 2024 at 2:57 PM Rosy <[email protected]> wrote: >>>>>>> >>>>>>>> Hello everyone >>>>>>>> >>>>>>>> I found a solution for issue >>>>>>>> https://gitlab.com/kicad/code/kicad/-/issues/17584 that neither >>>>>>>> changes the behavior of the library nor affects the performance. To do >>>>>>>> this, I recorded the two merge requests: >>>>>>>> master: https://gitlab.com/kicad/code/kicad/-/merge_requests/1914 >>>>>>>> 8.0: https://gitlab.com/kicad/code/kicad/-/merge_requests/1925 >>>>>>>> >>>>>>>> I have also developed a solution for issue >>>>>>>> https://gitlab.com/kicad/code/kicad/-/issues/17569 , which >>>>>>>> occasionally checks before a symbol is loaded whether the cache has >>>>>>>> already >>>>>>>> been created and, if not, created first. >>>>>>>> To do this, I recorded the two merge requests: >>>>>>>> master: https://gitlab.com/kicad/code/kicad/-/merge_requests/1927 >>>>>>>> 8.0: https://gitlab.com/kicad/code/kicad/-/merge_requests/1926 >>>>>>>> >>>>>>>> >>>>>>>> I think it's important that dynamic libraries like the http library >>>>>>>> can also add descriptions to the sublibraries. To do this, I recorded >>>>>>>> the >>>>>>>> following merge requests: >>>>>>>> master: https://gitlab.com/kicad/code/kicad/-/merge_requests/1921 >>>>>>>> 8.0: https://gitlab.com/kicad/code/kicad/-/merge_requests/1919 >>>>>>>> >>>>>>>> I've had various thoughts about the issue >>>>>>>> https://gitlab.com/kicad/code/kicad/-/issues/17570 and my other >>>>>>>> ideas. >>>>>>>> One of the problems with the existing solution is that certain >>>>>>>> requests return an array, making it almost impossible to expand. >>>>>>>> I could imagine shouting a v2 of the http library. Attached I have >>>>>>>> a document that describes how I imagine this. >>>>>>>> >>>>>>>> I would be happy to receive your feedback. >>>>>>>> >>>>>>>> On Monday, April 22, 2024 at 5:09:39 PM UTC+2 Rosy wrote: >>>>>>>> >>>>>>>>> My goal is that the library remains compatible with existing APIs >>>>>>>>> but that the various issues can be solved. I think it is important >>>>>>>>> that the >>>>>>>>> same functions as the standard functions are available for the http >>>>>>>>> libraries. The issues mentioned show where this is currently not >>>>>>>>> possible. >>>>>>>>> I also find it an important function that a description can be >>>>>>>>> given for the categories, as is the case with the standard libraries. >>>>>>>>> I >>>>>>>>> haven't entered an issu for this but have already added an MR. >>>>>>>>> >>>>>>>>> Something that I noticed and that I would certainly like to change >>>>>>>>> is that at the moment the parts are stored in different caches with >>>>>>>>> different loading states. >>>>>>>>> >>>>>>>>> There should only be one part cache. >>>>>>>>> >>>>>>>>> I currently have the idea that another key called “fields” can be >>>>>>>>> returned by the API when the root request is made. Only if this is >>>>>>>>> sent by >>>>>>>>> the API will the new features be activated, then the field list can >>>>>>>>> be >>>>>>>>> preloaded via a new endpoint. This enables short field keys even for >>>>>>>>> fields >>>>>>>>> with long names. >>>>>>>>> I would also like to introduce modify fields that allow only those >>>>>>>>> parts that have undergone updates to be reloaded. >>>>>>>>> >>>>>>>>> The communication could look something like this: >>>>>>>>> >>>>>>>>> # root >>>>>>>>> { >>>>>>>>> "categories" : "", >>>>>>>>> "parts" : "", >>>>>>>>> "fields" : "" //activate new fields features >>>>>>>>> } >>>>>>>>> >>>>>>>>> # fields.json >>>>>>>>> { >>>>>>>>> "reference":{ >>>>>>>>> "show" : true, >>>>>>>>> "default_show_name" : false >>>>>>>>> }, >>>>>>>>> "value":{ >>>>>>>>> "default_show" : true, >>>>>>>>> "default_show_name" : false >>>>>>>>> }, >>>>>>>>> "footprint":{ >>>>>>>>> "default_show" : false, >>>>>>>>> "default_show_name" : false >>>>>>>>> }, >>>>>>>>> "datasheet":{ >>>>>>>>> "default_show" : false, >>>>>>>>> "default_show_name" : false >>>>>>>>> }, >>>>>>>>> "description":{ >>>>>>>>> "default_show" : false, >>>>>>>>> "default_show_name" : false >>>>>>>>> }, >>>>>>>>> "keywords" : {}, >>>>>>>>> "mfn" :{ //must be unique >>>>>>>>> "name" : "Manufacturer Number", //must be unique >>>>>>>>> "show_on_chooser" : true, //only needed if not false >>>>>>>>> "default_show" : false, //only needed if not false >>>>>>>>> "default_show_name" : false //only needed if not false >>>>>>>>> }, >>>>>>>>> "mfurl" : { //must be unique >>>>>>>>> "name" : "Manufacturer URL", //must be unique >>>>>>>>> "show_on_chooser" : false, //only needed if not false >>>>>>>>> "default_show" : false, //only needed if not false >>>>>>>>> "default_show_name" : false //only needed if not false >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> # categories.json >>>>>>>>> [ >>>>>>>>> { >>>>>>>>> "id" : "1", //must be unique >>>>>>>>> "name" : "Resistor", //must be unique >>>>>>>>> "description" : "Thick Film SMD Resistors", >>>>>>>>> "modify" : 1713798563 //datetime seconds since 1.1.1970 >>>>>>>>> }, >>>>>>>>> { >>>>>>>>> "id" : "2", //must be unique >>>>>>>>> "name" : "Capacitor", //must be unique >>>>>>>>> "description" : "SMD MLCC Capacitors", >>>>>>>>> "modify" : 1713798563 //datetime seconds since 1.1.1970 >>>>>>>>> } >>>>>>>>> ] >>>>>>>>> >>>>>>>>> # /parts/category/{category_id}.json >>>>>>>>> { >>>>>>>>> "parts":[ >>>>>>>>> { >>>>>>>>> "id" : "1", //must be unique >>>>>>>>> "name" : "MF-ab123", //must be unique >>>>>>>>> "modify" : 1713798563 //datetime seconds since 1.1.1970 >>>>>>>>> "value" : "12k", >>>>>>>>> "description" : "Tick Film SMD Resistor 12kΩ, 1%", >>>>>>>>> "keywords" : "12kOhm", >>>>>>>>> "mfn": "ab123" >>>>>>>>> }, >>>>>>>>> { >>>>>>>>> "id" : "2", //must be unique >>>>>>>>> "name" : "MF-ab102", //must be unique >>>>>>>>> "modify" : 1713798563 //datetime seconds since 1.1.1970 >>>>>>>>> "value" : "1k", >>>>>>>>> "description" : "Tick Film SMD Resistor 1kΩ, 1%", >>>>>>>>> "keywords" : "1kOhm", >>>>>>>>> "mfn": "ab102" >>>>>>>>> } >>>>>>>>> ] >>>>>>>>> } >>>>>>>>> >>>>>>>>> parts/{part_id}.json >>>>>>>>> { >>>>>>>>> "id" : "2", >>>>>>>>> "symbolIdStr": "passive:R", >>>>>>>>> "exclude_from_bom": "False", >>>>>>>>> "exclude_from_board": "False", >>>>>>>>> "exclude_from_sim": "True", >>>>>>>>> "fields":{ >>>>>>>>> "reference":{ >>>>>>>>> "value" : "R", //can not be empty or a number >>>>>>>>> "show" : true, //only needed if not default >>>>>>>>> "show_name" : false //only needed if not default >>>>>>>>> }, >>>>>>>>> "value":{ //only needed if not default for show or show_name >>>>>>>>> //value ignored if is preloaded >>>>>>>>> "show" : true, //only needed if not default >>>>>>>>> "show_name" : false //only needed if not default >>>>>>>>> }, >>>>>>>>> "footprint":{ //only needed if exists or not default for show or >>>>>>>>> show_name >>>>>>>>> "value" : "resistor:R", //only needed if exists >>>>>>>>> "show" : false, //only needed if not default >>>>>>>>> "show_name" : false //only needed if not default >>>>>>>>> }, >>>>>>>>> "datasheet":{ //only needed if exists or not default for show or >>>>>>>>> show_name >>>>>>>>> "value" : "$(DATA)/part.pdf", //only needed if exists >>>>>>>>> "show" : false, //only needed if not default >>>>>>>>> "show_name" : false //only needed if not default >>>>>>>>> }, >>>>>>>>> "description":{ //only needed if not default for show or show_name >>>>>>>>> //value ignored if is preloaded >>>>>>>>> "show" : false, >>>>>>>>> "show_name" : false >>>>>>>>> }, "mfn" :{ //only needed if not default for show or show_name >>>>>>>>> //value ignored if is preloaded >>>>>>>>> "show" : false, //only needed if not default >>>>>>>>> "show_name" : false //only needed if not default >>>>>>>>> }, >>>>>>>>> "mfurl" :{ >>>>>>>>> "value" : "http:mfn.com", >>>>>>>>> "show" : false, //only needed if not default >>>>>>>>> "show_name" : false //only needed if not default >>>>>>>>> } >>>>>>>>> }, >>>>>>>>> "order" : { >>>>>>>>> "mfn", "mfurl" //define the order in witch the filds are added >>>>>>>>> to the symbol >>>>>>>>> } >>>>>>>>> } >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Monday, April 22, 2024 at 4:23:59 PM UTC+2 [email protected] >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Rosy, >>>>>>>>>> >>>>>>>>>> > I would be happy to invest time in revising the http library so >>>>>>>>>> that the various problems can be solved, but I would like to know >>>>>>>>>> who I >>>>>>>>>> should talk to so that the work is not in vain but is then accepted? >>>>>>>>>> >>>>>>>>>> This list is a good place, please say more about what you propose >>>>>>>>>> to >>>>>>>>>> do before you start work. >>>>>>>>>> >>>>>>>>>> I have been away from KiCad this past week and haven't had the >>>>>>>>>> chance >>>>>>>>>> to look at the MRs yet. Maybe other developers will also chime in >>>>>>>>>> here. >>>>>>>>>> >>>>>>>>>> -Jon >>>>>>>>>> >>>>>>>>>> On Mon, Apr 22, 2024 at 2:06 AM Rosy <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> > >>>>>>>>>> > In the meantime I have familiarized myself with the existing >>>>>>>>>> solution and the code. It is now clear to me what the problem with >>>>>>>>>> 17569 >>>>>>>>>> is. >>>>>>>>>> > >>>>>>>>>> > I would be happy to invest time in revising the http library so >>>>>>>>>> that the various problems can be solved, but I would like to know >>>>>>>>>> who I >>>>>>>>>> should talk to so that the work is not in vain but is then accepted? >>>>>>>>>> > >>>>>>>>>> > On Friday, April 19, 2024 at 9:36:39 AM UTC+2 Rosy wrote: >>>>>>>>>> >> >>>>>>>>>> >> Hello everyone >>>>>>>>>> >> >>>>>>>>>> >> I currently use KiCad privately and would like to use it in >>>>>>>>>> our company in the future. >>>>>>>>>> >> >>>>>>>>>> >> It is important that there is a good solution to manage the >>>>>>>>>> libraries centrally. The http library is ideal because it enables a >>>>>>>>>> clean >>>>>>>>>> separation of frontend and backend. >>>>>>>>>> >> >>>>>>>>>> >> Unfortunately, there are still a few things that I think need >>>>>>>>>> to be improved a bit. >>>>>>>>>> >> >>>>>>>>>> >> I have currently created a merge request with which it is now >>>>>>>>>> possible to provide the http sublibraries with a description, which >>>>>>>>>> I >>>>>>>>>> consider to be an essential function and hope that it will be >>>>>>>>>> accepted >>>>>>>>>> promptly. >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> I have also recorded the following issues regarding the http >>>>>>>>>> library. >>>>>>>>>> >> >>>>>>>>>> >> https://gitlab.com/kicad/code/kicad/-/issues/17584 >>>>>>>>>> >> >>>>>>>>>> >> https://gitlab.com/kicad/code/kicad/-/issues/17570 >>>>>>>>>> >> >>>>>>>>>> >> https://gitlab.com/kicad/code/kicad/-/issues/17569 >>>>>>>>>> >> >>>>>>>>>> >> For issues 17584/17570 Andre Iwers and I have submitted the >>>>>>>>>> merge request >>>>>>>>>> >> 1910/1912/1914 >>>>>>>>>> >> A possible solution has been developed, which is currently >>>>>>>>>> being discussed in MR 1910, as opinions differ on fields that are >>>>>>>>>> only >>>>>>>>>> displayed in the Choose Symbol dialog. >>>>>>>>>> >> What is your opinion? >>>>>>>>>> >> Otherwise, I would be happy to work out another solution. >>>>>>>>>> >> >>>>>>>>>> >> Does anyone have some tips for where and how I could tackle >>>>>>>>>> problem 17569? >>>>>>>>>> >> >>>>>>>>>> >> Thank you for your feedback. >>>>>>>>>> > >>>>>>>>>> > -- >>>>>>>>>> > You received this message because you are subscribed to the >>>>>>>>>> Google Groups "KiCad Developers" group. >>>>>>>>>> > To unsubscribe from this group and stop receiving emails from >>>>>>>>>> it, send an email to [email protected]. >>>>>>>>>> > To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/a/kicad.org/d/msgid/devlist/a801fafd-930f-4b75-b718-2e706a06a0efn%40kicad.org. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "KiCad Developers" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> >>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/a/kicad.org/d/msgid/devlist/519f23dc-97bf-4ce5-8c5b-be4197c3b938n%40kicad.org >>>>>>>> >>>>>>>> <https://groups.google.com/a/kicad.org/d/msgid/devlist/519f23dc-97bf-4ce5-8c5b-be4197c3b938n%40kicad.org?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "KiCad Developers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/a/kicad.org/d/msgid/devlist/c60cfe3b-fbff-4806-af87-749ac984acc9n%40kicad.org >>>> >>>> <https://groups.google.com/a/kicad.org/d/msgid/devlist/c60cfe3b-fbff-4806-af87-749ac984acc9n%40kicad.org?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- You received this message because you are subscribed to the Google Groups "KiCad Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/kicad.org/d/msgid/devlist/3ab48152-fc47-4edd-992a-3fcda30eddc6n%40kicad.org.
