Shawn and Garot I like the parsing capability of the lucene and its ability to stem incoming queries.. If you are able to start populating your data then we *should* be able to start identifying which root terms we can pull for building lucene-indexes I have an upcoming stuck-in-JFK-airport-on-labor-day-waiting-for-the-next-am-flight-of-USscare for an interrupted timeframe(i never learned how to 'sleep' in airport terminal) as long as I can find a wifi connection (and an AC connection within 6 hours) and use a working MySQL JDBC connection string..I should be able to bang out a quick prototype..
Let me know how i can help out, Martin Gainty ______________________________________________ ..place long-winded disclaimer here.. > From: ag4ve...@gmail.com > Date: Thu, 23 Aug 2012 10:42:55 -0400 > Subject: Re: Machine Learning > To: garotconk...@yahoo.com > CC: webmas...@lisol.co.uk; mgai...@hotmail.com; mysql@lists.mysql.com > > just a few thoughts (things to look into) > > if you want to populate a db with command parameters, i'd mine the man > pages. it's a consistent format and you should be able to find a > parser for whatever language you prefer. > > if you want to see what has been entered and statistics about that > process, look at the audit framework. it won't tell you success or > failure, but you might be able to derive that based on the time, > memory, and process usage that you can get from audit. auditd also > maintains a db - you might look into the engine and schema they use. > > you might also find some interesting topics looking into language > processing (after all, commands are a type of language). people > generally use other database engines to do this type of thing - > elastic search or some other things people have done with lucene > maybe. > > do let us know where you go with this. it sounds quite interesting > (and might help me with something i've been looking into)... and > getting zsh to autocomplete options might be a bonus if you parse man > pages and shove it into a db (i might do that part for you if i get a > lazy afternoon). > > On Wed, Aug 22, 2012 at 7:51 PM, Garot Conklin <garotconk...@yahoo.com> wrote: > > I am going to work up a visual diagram for this to better... visualize > > (LOL) the lay of the land. I will forward it when i have it completed then > > I hope we can continue the architectural discussion here! Thanks > > > > -Garot > > > > > > garotconk...@yahoo.com > > > > > > ________________________________ > > From: Garot Conklin <garotconk...@yahoo.com> > > To: Garot Conklin <garotconk...@yahoo.com>; "webmas...@lisol.co.uk" > > <webmas...@lisol.co.uk>; 'Martin Gainty' <mgai...@hotmail.com>; > > "mysql@lists.mysql.com" <mysql@lists.mysql.com> > > Sent: Tuesday, August 21, 2012 4:40 PM > > Subject: Re: Machine Learning > > > > to elaborate: > > > > I would almost rather qualify from the OS perspective what will work in > > advance, but take into consideration any failing condition. I may be > > getting a bit too specific at this juncture however as I am already apply > > this logic to issues I see in MY environment rather than total ambiguity, > > which is a best position here. > > > > I am ultimately expecting the learning algorithm to create new and > > excitingly effective ways of stringing commands together to solve issues, > > this being a notable aspect in and of itself, then creating some > > centralization based on what the machine determines as a best solution. > > > > Seems a relative "snapshot of the system" pre and post execution would be > > favorable as well so long as resources are not taxed as a result. This > > discussion is leading me to believe that this will be a multifaceted db > > solution without a doubt comprised of multiple layers of abstraction > > ultimately rolling up into a single master cluster for all the higher level > > analytics and internal testing cycles to then roll out the new primary > > protocols, which may themselves take place in a sub-instance of each > > possible scenario. > > > > I have also though of populating a db or multiple db's with actual text in > > the form of .pdf's of actual technical manuals... not certain the benefit > > of the written language abstraction there but the thought process > > extraction might be an interesting end indeed. > > > > > > > > garotconk...@yahoo.com > > > > > > ________________________________ > > From: Garot Conklin <garotconk...@yahoo.com> > > To: "webmas...@lisol.co.uk" <webmas...@lisol.co.uk>; 'Martin Gainty' > > <mgai...@hotmail.com>; "mysql@lists.mysql.com" <mysql@lists.mysql.com> > > Sent: Tuesday, August 21, 2012 4:09 PM > > Subject: Re: Machine Learning > > > > YES! > > > > > > garotconk...@yahoo.com > > > > > > ________________________________ > > From: "webmas...@lisol.co.uk" <webmas...@lisol.co.uk> > > To: 'Garot Conklin' <garotconk...@yahoo.com>; 'Martin Gainty' > > <mgai...@hotmail.com>; mysql@lists.mysql.com > > Sent: Tuesday, August 21, 2012 3:37 PM > > Subject: RE: Machine Learning > > > > > > Ah, > > > > Getting clearer and clearer. > > > > So these ‘nodes’ could ‘learn’ and ‘teach’ at the same time – right ? For > > instance, N1 runs a command in ‘domain’ D20 which it successful – it send > > information to node N20 that is the authority on domain D20 and N20 records > > it as success; N5 runs a command in domain D20 which goes wrong, and sends > > info to N20 and N20 records failure and sends a correct call to N5. Is this > > what you have in mind ? > > > > There are a lot of variables to be considered, for instance: > > 1) Command A version 1 can run very happily on operating system > > version 2, but fails in OS version 1 > > 2) Command A version 1 can run happily on OS version 2 on a 2G RAM, > > but fail on OS version 2 on 1G RAM > > 3) etc > > > > I think the DB design issues will become straightforward once the model is > > quite clear. > > > > Justin > > > > From:Garot Conklin [mailto:garotconk...@yahoo.com] > > Sent: 21 August 2012 17:14 > > To: webmas...@lisol.co.uk; 'Martin Gainty'; mysql@lists.mysql.com > > Subject: Re: Machine Learning > > > > 1)Refer to it for ‘knowledge’ (for instance, of what the latest > > version of a command is) > > It would most likely end up being "central" in this sense: > > A distributed collection of systems; i.e. (possible defined in > > roles) > > DB's > > FE's > > REPL's > > Cache's > > Each DB would have its own collection of remediation's that would > > then be indexed to populate a central db for trending/correlation etc... > > The "Collective" itself would function as a single conceptual > > implementation. A VIP for example might be associated with a specific role, > > say Web FE's and remediate only/all of them, but only have some relative > > access to the core db of say the Network from a Primary index perspective > > to make/draw associations/conclusions to issues at hand. > > > > > > 2)Send their ‘knowledge’ (for instance, of the latest command > > versions) to it for storage and distribution to others > > Rather than "latest command versions" I envision this to be more > > encapsulated as "latest successful invocation of the command string" and it > > inverse as well (to trend/metric-ize the failures thus lending to perpetual > > optimization). > > > > I like how this is fleshing out... This is helping me to define what I am > > really trying to accomplish. Thanks very much for everyone responding here, > > this is wonderful, please keep this going... > > > > garotconk...@yahoo.com > > > > ________________________________ > > > > From:"webmas...@lisol.co.uk" <webmas...@lisol.co.uk> > > To: 'Martin Gainty' <mgai...@hotmail.com>; garotconk...@yahoo.com; > > mysql@lists.mysql.com > > Sent: Tuesday, August 21, 2012 11:19 AM > > Subject: RE: Machine Learning > > > > Hi Garot, > > > > > > > > Ok, the concept is getting clearer, but let’s bring this down to earth a > > little bit more. I love DB design and problem-solving and am quite curious > > about this. > > > > > > > > Is the idea that you have a central computer (not HAL J) somewhere so that > > other computers can: > > > > 1) Refer to it for ‘knowledge’ (for instance, of what the latest > > version of a command is) > > > > 2) Send their ‘knowledge’ (for instance, of the latest command > > versions) to it for storage and distribution to others > > > > > > > > If this is the model, then the knowledge base can build up organically over > > time – I think. Or is this too simplistic ? > > > > > > > > Thanks, > > > > Justin > > > > > > > > From: Martin Gainty [mailto:mgai...@hotmail.com] > > Sent: 21 August 2012 00:25 > > To: garotconk...@yahoo.com; webmas...@lisol.co.uk; mysql@lists.mysql.com > > Subject: RE: Machine Learning > > > > > > > > When I hear 'AI' I always imagine theres a HAL 9001 behind the scenes that > > is running the show constantly admonishing its creator to "take another > > stress pill" > > > > Sounds like a fun project > > > > Keep us apprised, > > Martin Gainty > > ______________________________________________ > > Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité > > > > > > Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene > > Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte > > Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht > > dient lediglich dem Austausch von Informationen und entfaltet keine > > rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von > > E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen. > > > > Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le > > destinataire prévu, nous te demandons avec bonté que pour satisfaire > > informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie > > de ceci est interdite. Ce message sert à l'information seulement et n'aura > > pas n'importe quel effet légalement obligatoire. Étant donné que les email > > peuvent facilement être sujets à la manipulation, nous ne pouvons accepter > > aucune responsabilité pour le contenu fourni. > > > > > > > > > > > > _____ > > > > Date: Mon, 20 Aug 2012 13:50:04 -0700 > > From: garotconk...@yahoo.com > > Subject: Re: Machine Learning > > To: webmas...@lisol.co.uk; mgai...@hotmail.com; mysql@lists.mysql.com > > > > Ya the idea is not anything new, but must be apparently quit difficult or > > not a priority as I have yet to find it already implemented anywhere... Far > > be it from me to not make some attempt here anyway... > > > > > > > > I am creating a fully automated framework from which a distributed > > infrastructure can be maintained. I have been writing automation > > scripts/code for some time now and the logical progression is to embark on a > > full concept of systems health auto remediation. I have numerous > > "monitoring" solutions under my control however none that properly (in my > > opinion) implements any real learning algorithms from which to draw even a > > minimalists view of automation. I like mySQL therefor began thinking about > > creating the aspects (lobes) of the "brain" as a relational database(s). So > > this is only one facet of what I am trying to do, however leveraging a full > > command set of shell utilities/commands/programs seemed to be a good > > starting point before I get into the "hard" stuff ! > > > > > > > > -Garot > > > > > > > > garotconk...@yahoo.com > > > > _____ > > > > From: "webmas...@lisol.co.uk" <webmas...@lisol.co.uk> > > To: 'Martin Gainty' <mgai...@hotmail.com>; garotconk...@yahoo.com; > > mysql@lists.mysql.com > > Sent: Monday, August 20, 2012 3:55 PM > > Subject: RE: Machine Learning > > > > > > Hi Garot, > > > > > > > > You'll have to elaborate some more ... I understand you may want to protect > > the idea as well, so if you can narrow it down to some technical specifics > > then it'll help. > > > > > > > > What is the objective of this system, for instance ? > > > > > > > > Thanks, > > > > Justin > > > > > > > > From: Martin Gainty [mailto:mgai...@hotmail.com] > > Sent: 20 August 2012 19:23 > > To: garotconk...@yahoo.com; webmas...@lisol.co.uk; mysql@lists.mysql.com > > Subject: RE: Machine Learning > > > > > > > > > > > > From: garotconk...@yahoo.com > > Subject: Re: Machine Learning > > To: webmas...@lisol.co.uk; mgai...@hotmail.com; mysql@lists.mysql.com > > > > > > My initial thought was to propagate the db with everything and allow the > > algorithm to then begin to determin trends/patterns > > MG>which trends or patterns will you be modelling? > > > > and begin either an indexing methodology > > MG>which indexes are you considering: Unique index, primary index or foreign > > index? > > > > additional table/db creation process or both to further optimize the calls > > being made > > MG>optimize based on execution time or diskspace allocated, EliminatingFTS > > or some other criteria? > > MG>https://dev.mysql.com/doc/refman/5.5/en/optimization.html > > > > and build in some internal levels of redundancy. > > MG>what about replication > > MG>http://dev.mysql.com/doc/refman/5.5/en/replication.html > > > > I am actually approaching this with some degree of biological conception in > > the multipathing within our own brains however until I have something up and > > running under some substantial load however I may not get a complete > > picture. > > > > Thanks, > > > > Garot > > > > > > Interesting > > Martin > > > > _____ > > > > From: webmas...@lisol.co.uk <webmas...@lisol.co.uk>; > > To: 'Garot Conklin' <garotconk...@yahoo.com>; 'Martin Gainty' > > <mgai...@hotmail.com>; <mysql@lists.mysql.com>; > > Subject: RE: Machine Learning > > Sent: Mon, Aug 20, 2012 7:13:25 AM > > > > > > Hi Garot, > > > > This sounds an interesting idea. > > > > Are you looking to store all known commands and their options or are you > > looking for a 'formula' for calling any unix command ? > > > > The reason for my question is that, at the end of the day, a unix command is > > just a program that is run in the operating system. Each program comes with > > its own options and acceptable inputs. I don't know if there is a rule or > > convention for structuring these commands. > > > > Are you then looking to build a system that 'knows' all commands and 'how > > to' call them ? > > > > Thanks, > > Justin > > > > -----Original Message----- > > From: Garot Conklin [mailto:garotconk...@yahoo.com] > > Sent: 20 August 2012 03:39 > > To: Martin Gainty; mysql@lists.mysql.com > > Subject: Re: Machine Learning > > > > The initial goal is to provide a working framework from which to call all > > UNIX shell command combinations as the underlying storage mechanism for a > > machine learning algorithm. I would like to build a completely self aware > > instantiation that will maintain itself on all levels... I postulate that > > the first place to start would be in determining a method for maintaining > > all possible remediation combinations including the unknown to eventually be > > learned from and populate new knowledge into the database. Thank you for > > the reply, > > > > Garot > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe: http://lists.mysql.com/mysql >
