BINGO! That didn't take long. I knew the non-sql database question would rear it's ugly head. It just happened quicker than I thought it would ;)
On 8/29/20 9:58 AM, Tim Hawkins wrote: > https://calcite.apache.org/docs/adapter.html > > Apache calcite is an SQL query engine for non-sql data sources. > > > > On Sat, Aug 29, 2020, 21:21 Jeff Young <j...@rokeby.ie > <mailto:j...@rokeby.ie>> wrote: > > Surely there must be an open source impl of an ODBC interface on a > CSV file? > > Although I’m not sure of the desire to avoid MySQL. It’s remarkably > easy to set up an instance (or auto-deploy one with an app). > > Apologies if we’ve already talked about that; I’ll confess to not > having followed this thread 100%…. > >> On 29 Aug 2020, at 14:11, Wayne Stambaugh <stambau...@gmail.com >> <mailto:stambau...@gmail.com>> wrote: >> >> I would most likely reject any solution that was tied to a particular >> database. All this would do is open up a Pandora's box of >> complaints as >> to why we didn't use database A over database B. ODBC is the most >> flexible solution that I am aware of and allows users to choose their >> preferred database. >> >> Cheers, >> >> Wayne >> >> On 8/29/20 8:29 AM, Jon Evans wrote: >>> Putting aside the fact that I think this feature isn't really >>> aimed at >>> hobbyists, I would not be opposed to people wanting to extend it >>> beyond >>> ODBC but that comes with extra complexity that must be handled. >>> >>> With ODBC, KiCad just needs to know about the interface to >>> retrieve the >>> data. >>> >>> With a CSV file, KiCad actually needs to read that file in and >>> keep it >>> in memory. Watch for modifications on disk, or else lock it >>> exclusively. Things like that. >>> >>> I'm not sure I really see the advantage of a CSV-backed >>> "database" over >>> the existing KiCad library system, if we're talking about a >>> single user. >>> >>> -Jon >>> >>> On Sat, Aug 29, 2020 at 8:19 AM Mark Roszko >>> <mark.ros...@gmail.com <mailto:mark.ros...@gmail.com> >>> <mailto:mark.ros...@gmail.com>> wrote: >>> >>>> Sqlite was a quick&dirty way to test if my ideas would work. >>> There are ODBC wrappers for SQLite........ >>> >>>> I mean libreoffice would do for the management. >>> Yes, and you know what you would use? Not CSV files. >>> LibreOffice Base / Microsoft Access. >>> This is the office suite database that's basically SQLite and >>> there >>> are ODBC wrappers as well :D >>> >>>> Also, why would a hobbyist fire up a sql database when a CSV file >>> would be sufficient? I mean libreoffice would do for the >>> management. >>> >>> KiCad's uses aren't limited to hobbyists... >>> And, you assume there aren't hobbyists like me who will gladly >>> take >>> that ODBC link and spin up an frontend in a few hours to the whole >>> system :D >>> >>> >>> On Sat, Aug 29, 2020 at 4:22 AM Johann Wilhelm >>> <johann.wilhelm@wilhelm.consulting >>> <mailto:johann.wilhelm@wilhelm.consulting>> wrote: >>> >>> Hi there! >>> >>> Well, then my comment was not completely wrong. >>> >>> Sqlite was a quick&dirty way to test if my ideas would >>> work. For >>> my future productive system I really want to use a mix of >>> couchdb and maybe postgres. i.e. a JSON document storage >>> for the >>> component information and sql for inventory management. >>> >>> So ODBC would not work well for me. >>> >>> Also, why would a hobbyist fire up a sql database when a CSV >>> file would be sufficient? I mean libreoffice would do for the >>> management. >>> >>> Additionally, I'd suggest looking at the BOM creation. There, >>> external programs are called. >>> >>> So why not define a dataformat (xml, json, csv,...) and just >>> call an external app or read from a file/uri? >>> >>> Anyways, I would volunteer for implementing some alternatives >>> (read from file/uri and output of executable) to the ODBC >>> interface if someone guides me through the KiCad procedures. >>> >>> Regards, >>> Johann >>> >>> >>> Jon Evans <j...@craftyjon.com >>> <mailto:j...@craftyjon.com> <mailto:j...@craftyjon.com>> schrieb >>> am Fr., 28. Aug. 2020, 19:54: >>> >>> My idea is to make it possible for KiCad to talk to an >>> external database. >>> The database itself (and its schema) will not be >>> defined as >>> part of this spec, and will not be part of KiCad. >>> >>> The only requirement is that you have some columns in your >>> schema that KiCad understands (for example, to point >>> to the >>> right symbols and footprints in the KiCad libraries). >>> The planned interface to connect to the database is ODBC. >>> This would in theory allow using sqlite to create a >>> database >>> as a file on disk, although I anticipate that most users >>> will be using something like MariaDB or Postgres. >>> >>> There has been some discussion of supporting talking >>> to web >>> interfaces via some REST API, or even talking to arbitrary >>> interfaces via Python scripting, but that discussion >>> should >>> stay separate. >>> The original thread was about getting component >>> information >>> out of a database and I just wanted to let people know >>> that >>> I am working on this. >>> >>> People are welcome to also discuss the ideas of getting >>> component information from a web API or from some >>> Python script. >>> But, I am not working on that right now, and there hasn't >>> even been a conversation started on what that spec would >>> look like. >>> >>> Best, >>> Jon >>> >>> On Fri, Aug 28, 2020 at 1:42 PM Johann Wilhelm >>> <johann.wilhelm@wilhelm.consulting >>> <mailto:johann.wilhelm@wilhelm.consulting>> wrote: >>> >>> Hi Jon, >>> >>> Well, you're idea was to define a database or at >>> least a >>> database schema if I understood this correctly: >>> >>> What I plan on implementing is not a "full" >>> database management system, but >>> >>> rather an interface to grab info out of a >>> database, just like you say. >>> >>> The only difference is, there are no plans to >>> store actual symbols or >>> >>> footprints in the database. >>> >>> The symbols and footprints will still be >>> stored in "normal" KiCad library >>> >>> files; the database will just contain >>> "pointers" telling KiCad which symbol >>> >>> to use (and what library it can be found in). >>> >>> >>> >>> I was pointing toward a specification of a data >>> format.So either read the data from a file or >>> webinterface (that's why I used URI). >>> My script-set currently gives me that information via >>> http://localhost:8000/API/Component/getMatchWisdom >>> >>> Others would maybe like to save a file in ~/.KiCad (so >>> the URI would be file://~/.KiCad) >>> >>> If "database"/"database interface" would include >>> something which could read from files and/or web >>> resources, well, never mind my comment :) >>> >>> With "global parameter" I mean something which >>> could be >>> part of a .pro file. >>> >>> Regards, >>> Johann >>> >>> >>> Am Fr., 28. Aug. 2020 um 19:04 Uhr schrieb Jon Evans >>> <j...@craftyjon.com >>> <mailto:j...@craftyjon.com> <mailto:j...@craftyjon.com>>: >>> >>> Hi Johann, >>> >>> I am not sure exactly what you are arguing >>> against. >>> I don't see any difference between what you are >>> looking for and what is in my spec. >>> I am not sure what you mean by "global URI >>> parameter" but the part picker will be able to >>> filter using any of the external data present. >>> >>> Best, >>> Jon >>> >>> On Fri, Aug 28, 2020 at 12:56 PM Johann Wilhelm >>> <johann.wilhelm@wilhelm.consulting >>> <mailto:johann.wilhelm@wilhelm.consulting>> wrote: >>> >>> Hi everyone! >>> >>> I'm new here in the mailing list but I'm >>> currently building my own electronics business >>> around KiCad which I use and love for >>> years now. >>> >>> I spent the last weeks, trying to tie together >>> procurement, PCB design, and fabrication in a >>> single script-set. >>> Just to have it mentioned - before getting >>> self-employed, I worked for 10 years in the >>> electronics industry in a huge enterprise. >>> So I had some strict requirements for my >>> tooling >>> and I have some very strong opinions on how my >>> perfect workflow should look like :) >>> >>> I'm very sorry but I need to say: PLEASE, >>> don't >>> just throw a part database at the community! >>> Why? Because everyone has different >>> approaches how to do procurement and inventory >>> management! >>> It's ok (and actually good!) if you try to >>> come >>> up with something but PLEASE, go for a defined >>> filter interface! >>> >>> What I would suggest implementing - >>> actually, I >>> have plans to implement it myself in >>> mid-term - >>> is a simple checkbox in the component dialog >>> named "Apply Parts Filter" and a global URI >>> parameter that provides the filter (or better: >>> the source of "wisdom" used by the filter). >>> Maybe another parameter defining the >>> default for >>> the checkbox would be nice as well... >>> >>> I think even a simple CSV with columns for >>> Symbol, Footprint, and Value could provide >>> sufficient information for effective >>> filtering/adaption of the symbol-tree. >>> >>> A CSV like this: >>> >>> >>> "Audio:TLV320AIC23BPW", "TLV320AIC23BPW", >>> "Package_SO:TSSOP-28_4.4x9.7mm_P0.65mm" >>> "Device:R_Small", "1k", >>> "Resistor_SMD:R_0805_2012Metric" >>> "Device:R*", "12k4", >>> "Resistor_SMD:R_0805_2012Metric" >>> "Device:R*", "12k4", >>> "Resistor_SMD:R_1206_3216Metric" >>> "Device:R*", "10R", >>> "Resistor_SMD:R_0805_2012Metric" >>> >>> >>> should result in a component tree like this: >>> >>> Audio >>> >>> TLV320AIC23BPW <-- this symbol >>> has a >>> matching default footprint and value >>> >>> Device >>> >>> R <-- this symbol is included in the >>> CSV data >>> >>> 12k4 <-- a value of 12k4 would >>> match Device:R and a corresponding >>> symbol is added by the filter >>> >>> Resistor_SMD:R_0805_2012Metric >>> <--- both footprints would >>> match Device:R 12k4 so the >>> filter adds both symbols with >>> the footprint and values >>> fields >>> complemented >>> Resistor_SMD:R_0805_2012Metric >>> <--- .... >>> >>> 10R >>> >>> Resistor_SMD:R_0805_2012Metric >>> >>> R_Small >>> >>> 1k <-- 1k only matches >>> Device:R_Small >>> >>> Resistor_SMD:R_0805_2012Metric >>> >>> 12k4 >>> >>> Resistor_SMD:R_0805_2012Metric >>> Resistor_SMD:R_0805_2012Metric >>> >>> 10R >>> >>> Resistor_SMD:R_0805_2012Metric >>> >>> >>> So the filter should add all symbols >>> referenced >>> in the CSV. >>> If a symbol has no matching footprint and/or >>> value => "create" them on fly >>> >>> With this, you can use the whole symbol >>> library >>> or the existing parts in your inventory. >>> The best of both worlds! >>> >>> Regards, >>> Johann >>> >>> >>> Am Fr., 28. Aug. 2020 um 16:38 Uhr schrieb >>> Brian >>> <lotha...@gmail.com >>> <mailto:lotha...@gmail.com> <mailto:lotha...@gmail.com>>: >>> >>> Just want to add my +1 for the interface >>> approach. I am glad to hear that is the >>> intent (I’ve not had time to read the >>> proposal). With such an interface, as has >>> been pointed out already, >>> data-source-specific implementations >>> should >>> be relatively straightforward, and then I >>> don’t have to potentially throw away my >>> years of privately curated data or figure >>> out how to cram it into some alternate >>> schema. All I would need to do is >>> write the >>> code to answer the questions presented by >>> the interface. >>> >>> Hopefully in the next few days I’ll be >>> able >>> to read Jon’s draft spec and comment >>> from a >>> better-informed position. >>> >>> Cheers, >>> -Brian >>> >>>> On Aug 28, 2020, at 9:43 AM, Jon Evans >>>> <j...@craftyjon.com >>>> <mailto:j...@craftyjon.com> >>>> <mailto:j...@craftyjon.com>> wrote: >>>> >>>> >>>> Hi Clemens, >>>> >>>> On Fri, Aug 28, 2020 at 9:34 AM Clemens >>>> Koller <c...@embeon.de >>>> <mailto:c...@embeon.de> >>>> <mailto:c...@embeon.de>> wrote: >>>> >>>> Hello! >>>> >>>> This is related to the previous >>>> thread: "Automatic assignment of >>>> footprint with a database" >>>> >>>> I would generally prefer assemble >>>> real >>>> components on a real PCB right from >>>> the beginning instead of first >>>> placing >>>> generic components and then assign >>>> footprints + manufacturers + >>>> types + x >>>> manually. This seems extra work for >>>> each component which could >>>> possibly be >>>> avoided. >>>> >>>> >>>> Me too! >>>> >>>> >>>> So, regarding on the Kicad >>>> codebase, I >>>> would very likely not recomment to >>>> embed a full component management / >>>> database system, since this might >>>> vary >>>> from site to site and even from >>>> project/assembly house to >>>> project/house. But it would be great >>>> to be able to have an interface to >>>> grab a component out of a >>>> database and >>>> Kicad grabs all desired / a selection >>>> of individual columns out of the >>>> database as needed. >>>> This might include the actual >>>> footprints stored in the database >>>> as well. >>>> >>>> >>>> What I plan on implementing is not a >>>> "full" database management system, but >>>> rather an interface to grab info out of a >>>> database, just like you say. >>>> The only difference is, there are no >>>> plans >>>> to store actual symbols or footprints in >>>> the database. >>>> The symbols and footprints will still be >>>> stored in "normal" KiCad library files; >>>> the database will just contain "pointers" >>>> telling KiCad which symbol to use (and >>>> what library it can be found in). >>>> >>>> BTW, the example schema in your email >>>> looks very familiar to me. This is the >>>> kind of data source that can be used with >>>> the feature I am talking about: just add >>>> columns to the schema for tracking which >>>> KiCad symbol and footprint(s) should be >>>> used for a part. >>>> >>>> Regarding the advanced features you >>>> mention, some of them sound like they >>>> would be handled by a PLM tool. >>>> Some of the PLM tools I have worked with >>>> can interface to external databases for >>>> managing this kind of component library >>>> for an EDA or mechanical CAD tool. >>>> >>>> Best, >>>> Jon >>>> >>>> _______________________________________________ >>>> Mailing list: >>>> https://launchpad.net/~kicad-developers >>>> Post to : >>>> kicad-developers@lists.launchpad.net >>>> <mailto:kicad-developers@lists.launchpad.net> >>>> >>>> <mailto:kicad-developers@lists.launchpad.net> >>>> Unsubscribe : >>>> https://launchpad.net/~kicad-developers >>>> More help : >>>> https://help.launchpad.net/ListHelp >>> >>> _______________________________________________ >>> Mailing list: >>> https://launchpad.net/~kicad-developers >>> Post to : >>> kicad-developers@lists.launchpad.net >>> <mailto:kicad-developers@lists.launchpad.net> >>> <mailto:kicad-developers@lists.launchpad.net> >>> Unsubscribe : >>> https://launchpad.net/~kicad-developers >>> More help : >>> https://help.launchpad.net/ListHelp >>> >>> _______________________________________________ >>> Mailing list: >>> https://launchpad.net/~kicad-developers >>> Post to : >>> kicad-developers@lists.launchpad.net >>> <mailto:kicad-developers@lists.launchpad.net> >>> <mailto:kicad-developers@lists.launchpad.net> >>> Unsubscribe : >>> https://launchpad.net/~kicad-developers >>> More help >>> : https://help.launchpad.net/ListHelp >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~kicad-developers >>> Post to : kicad-developers@lists.launchpad.net >>> <mailto:kicad-developers@lists.launchpad.net> >>> <mailto:kicad-developers@lists.launchpad.net> >>> Unsubscribe : https://launchpad.net/~kicad-developers >>> More help : https://help.launchpad.net/ListHelp >>> >>> >>> >>> -- >>> Mark >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~kicad-developers >>> Post to : kicad-developers@lists.launchpad.net >>> <mailto:kicad-developers@lists.launchpad.net> >>> Unsubscribe : https://launchpad.net/~kicad-developers >>> More help : https://help.launchpad.net/ListHelp >>> >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~kicad-developers >> Post to : kicad-developers@lists.launchpad.net >> <mailto:kicad-developers@lists.launchpad.net> >> Unsubscribe : https://launchpad.net/~kicad-developers >> More help : https://help.launchpad.net/ListHelp > > _______________________________________________ > Mailing list: https://launchpad.net/~kicad-developers > Post to : kicad-developers@lists.launchpad.net > <mailto:kicad-developers@lists.launchpad.net> > Unsubscribe : https://launchpad.net/~kicad-developers > More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~kicad-developers Post to : kicad-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~kicad-developers More help : https://help.launchpad.net/ListHelp