Re: [wsjt-devel] ALL.txt idea
Hi Sandro, Thanks for the suggestion about using the ISO 8601 format for Date/Time labels in ALL.TXT. I have made this change in code revision 7312. -- Joe, K1JT On 11/8/2016 4:12 PM, Alessandro Gorobey wrote: > Hi Claude and All, > > ALL.TXT is borne to dump all the lines in receive window. > > I think that insert in a database is very expansive as space and > processor and lost the simplicity of search with preferred programs. > > The format of file is changed in years and actually use some more > characters after introduction of new decoders. > > Sincerely I not like the months names in the markers and a ISO 8601 > format will be preferable, using machines with different languages. > > 2013-lug-06 16:24 14.078 MHz JT9 > 2013-jul-06 16:24 14.078 MHz JT9 > 2013-07-06 16:24 14.078 MHz JT9 <== ISO > > Sincerely I think that ALL.TXT without the nationalization strings is > good for the work. > -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
The date is thereonly when the time rolls over or you restart WSJT-X. Another reason CSV won't work too well.2359 -3 0.1 1209 # KF6LYF W7KR -15 2016-Jan-26 00:00 14.076 MHz JT9+JT65 -25 0.2 2630 @ CQ WB7CTI DN06 I use the date and keep it around for displaying with the message info. de Mike W9MDB From: Dan Malcolm <dmalcol...@mchsi.com> To: 'WSJT software development' <wsjt-devel@lists.sourceforge.net> Sent: Tuesday, November 8, 2016 5:21 PM Subject: Re: [wsjt-devel] ALL.txt idea Am I missing something? I don't get any date indicators in ALL.TXT. I get time in 24 hour format, and of course whatever else appears in the "Band Activity" window, but no date. IMHO the date (in ISO format) would be a useful piece of information, but so far I have gotten along without it. _ Dan Malcolm CFI/II K4SHQ -Original Message- From: Alessandro Gorobey [mailto:algi...@libero.it] Sent: Tuesday, November 08, 2016 3:12 PM To: wsjt-devel@lists.sourceforge.net Subject: Re: [wsjt-devel] ALL.txt idea Hi Claude and All, ALL.TXT is borne to dump all the lines in receive window. I think that insert in a database is very expansive as space and processor and lost the simplicity of search with preferred programs. The format of file is changed in years and actually use some more characters after introduction of new decoders. Sincerely I not like the months names in the markers and a ISO 8601 format will be preferable, using machines with different languages. 2013-lug-06 16:24 14.078 MHz JT9 2013-jul-06 16:24 14.078 MHz JT9 2013-07-06 16:24 14.078 MHz JT9 <== ISO Sincerely I think that ALL.TXT without the nationalization strings is good for the work. -- 73 Sandro IW3RAB Il 08/11/2016 16:49, Claude Frantz ha scritto: > On 11/07/2016 06:11 PM, Roger Rehr W3SZ wrote: > > Hi Roger, > > Writing such simple structures in a SQL database is overkill, in my > humble opinion. Concerning the ADIF log, there is no general > definition of a SQL Database mapping outside of specific logging > programs. The new alternative XML definition of ADIF 3.* is not of > great help because XML based databases are a very special matter. > > Previously, I have pointed to the problem of the ADIF TIME_ON field > which contains the data which should be in the TIME_OFF and to the > problems around this matter. Up to now, I have found no response. > > Best 88 de Claude (DJ0OT) > > -- > Developer Access Program for Intel Xeon Phi Processors Access > to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > -- 73 Sandro IW3RAB -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
Am I missing something? I don't get any date indicators in ALL.TXT. I get time in 24 hour format, and of course whatever else appears in the "Band Activity" window, but no date. IMHO the date (in ISO format) would be a useful piece of information, but so far I have gotten along without it. _ Dan Malcolm CFI/II K4SHQ -Original Message- From: Alessandro Gorobey [mailto:algi...@libero.it] Sent: Tuesday, November 08, 2016 3:12 PM To: wsjt-devel@lists.sourceforge.net Subject: Re: [wsjt-devel] ALL.txt idea Hi Claude and All, ALL.TXT is borne to dump all the lines in receive window. I think that insert in a database is very expansive as space and processor and lost the simplicity of search with preferred programs. The format of file is changed in years and actually use some more characters after introduction of new decoders. Sincerely I not like the months names in the markers and a ISO 8601 format will be preferable, using machines with different languages. 2013-lug-06 16:24 14.078 MHz JT9 2013-jul-06 16:24 14.078 MHz JT9 2013-07-06 16:24 14.078 MHz JT9 <== ISO Sincerely I think that ALL.TXT without the nationalization strings is good for the work. -- 73 Sandro IW3RAB Il 08/11/2016 16:49, Claude Frantz ha scritto: > On 11/07/2016 06:11 PM, Roger Rehr W3SZ wrote: > > Hi Roger, > > Writing such simple structures in a SQL database is overkill, in my > humble opinion. Concerning the ADIF log, there is no general > definition of a SQL Database mapping outside of specific logging > programs. The new alternative XML definition of ADIF 3.* is not of > great help because XML based databases are a very special matter. > > Previously, I have pointed to the problem of the ADIF TIME_ON field > which contains the data which should be in the TIME_OFF and to the > problems around this matter. Up to now, I have found no response. > > Best 88 de Claude (DJ0OT) > > -- > Developer Access Program for Intel Xeon Phi Processors Access > to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > -- 73 Sandro IW3RAB -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
On 11/07/2016 06:11 PM, Roger Rehr W3SZ wrote: Hi Roger, Writing such simple structures in a SQL database is overkill, in my humble opinion. Concerning the ADIF log, there is no general definition of a SQL Database mapping outside of specific logging programs. The new alternative XML definition of ADIF 3.* is not of great help because XML based databases are a very special matter. Previously, I have pointed to the problem of the ADIF TIME_ON field which contains the data which should be in the TIME_OFF and to the problems around this matter. Up to now, I have found no response. Best 88 de Claude (DJ0OT) -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
_I_ personally do not use ALL.txt or any other artifact of WSJT-X, but I do use WSJT-X often and am a software engineer that would rather err on the side of KISS than a monolithic app with more and more library dependencies. Emitting CSV isn't too much extra effort to add to WSJT-X, but adding a vendor-specific database library is, IMHO. If the rows are not 'consistently formatted' enough for CSV, then they're likely unsuitable for a structured relational DB as well. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com<mailto:dtil...@captechconsulting.com> On Nov 7, 2016, at 4:12 PM, Black Michael <mdblac...@yahoo.com<mailto:mdblac...@yahoo.com>> wrote: Would be more true if the records were consistently formatted...but they aren't. What would you use a database for? de Mike W9MDB From: David Tiller <dtil...@captechconsulting.com<mailto:dtil...@captechconsulting.com>> To: WSJT software development <wsjt-devel@lists.sourceforge.net<mailto:wsjt-devel@lists.sourceforge.net>> Sent: Monday, November 7, 2016 3:08 PM Subject: Re: [wsjt-devel] ALL.txt idea I think Dan's idea of using comma- or tab-separated fields is a nice compromise between locking users into a particular database format (regardless of how ubiquitous) and unstructured rows. CSV can be imported/parsed/chunked/scattered/gathered 100 ways to Sunday. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com<mailto:dtil...@captechconsulting.com> On Nov 7, 2016, at 4:01 PM, Dan Malcolm <dmalcol...@mchsi.com<mailto:dmalcol...@mchsi.com>> wrote: > I too use ALL.TXT. But I believe a move towards some DB functionality could > be made use comma or tab delimited fields (CSV file) instead of a space. > The resulting text file could easily be imported into any DB management > system that I am familiar with. I don't believe it would add much to the > code. > > Just my $.02 worth. > > _ > Dan Malcolm CFI/II > K4SHQ > > > -Original Message- > From: Greg Beam [mailto:ki7m...@gmail.com<mailto:ki7m...@gmail.com>] > Sent: Monday, November 07, 2016 10:00 AM > To: Black Michael <mdblac...@yahoo.com<mailto:mdblac...@yahoo.com>>; WSJT > software development > <wsjt-devel@lists.sourceforge.net<mailto:wsjt-devel@lists.sourceforge.net>> > Subject: Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > I am not sure if the ALL.txt discussion has come up before (probably has at > some point), but certainly CALL3.txt has made the rounds several times. I > was able to get CALL3 Database functionality working with SQLite for WSJT, > and have a stand alone application for WSPR to process millions of spots the > archives produce monthly, however, it does add a level of complexity to the > application that may be more than is needed for minimal operation. > > It's hard to dismiss the simplicity of flat text files. On the other hand, > having data in tables lends itself to many positive benefits. For example, > the queries in this thread would be a fairly simple SQL query rather than > requiring a Black Belt in command-line foo to extract it. > Not to mention, the SQL language is cross-platform, and generally, no > additional tools are required, but many exit at the users option. > > I am not suggesting that WSJT-X integrate post processing tools, rather, I > am merely suggesting that the data could be stored in a different format ( > DB tables) that allows easy access by third party developers or primary > users, in a more structured manner. > > > 73's > Greg, KI7MT > > On 11/7/2016 7:18 AM, Black Michael wrote: >> I wouldn't be inclined to add database support when the most common >> operations are achievable with the existing ALL.TXT. Haven't we >> discussed this before? Somebody said something about putting the >> messages in an sqlite database as I recall and had implemented something. >> Though I'm sure we could make it portable it's adding a fair bit of >> new stuff and overhead for doing this simple task. Right now my logic >> is about 100 lines of code and can pick up non-standard end-of-qso >> messages to some degree right now. >> >> Is there some other function desirable from a database that would >> justify it? I can see where adding a field for "Rx Frequency" window >> message presence would help in processing QSOs. Non-standard QSO >> messages would be an SQL challenge. >> >> >> de Mike W9MDB >> >> >> >> ---------- >> -- >> *From:* Greg Beam <k
Re: [wsjt-devel] ALL.txt idea
Probably could add a 2nd file rather than changing ALL.txt -- backwards compatible ya' know.Or make an option. Then use the UDP packet elements to generate the CSV file since that should be fully fleshed plus auto-updated when protocols are added and such. de Mike W9MDB From: David Tiller <dtil...@captechconsulting.com> To: WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 3:08 PM Subject: Re: [wsjt-devel] ALL.txt idea I think Dan's idea of using comma- or tab-separated fields is a nice compromise between locking users into a particular database format (regardless of how ubiquitous) and unstructured rows. CSV can be imported/parsed/chunked/scattered/gathered 100 ways to Sunday. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com On Nov 7, 2016, at 4:01 PM, Dan Malcolm <dmalcol...@mchsi.com> wrote: > I too use ALL.TXT. But I believe a move towards some DB functionality could > be made use comma or tab delimited fields (CSV file) instead of a space. > The resulting text file could easily be imported into any DB management > system that I am familiar with. I don't believe it would add much to the > code. > > Just my $.02 worth. > > _ > Dan Malcolm CFI/II > K4SHQ > > > -Original Message- > From: Greg Beam [mailto:ki7m...@gmail.com] > Sent: Monday, November 07, 2016 10:00 AM > To: Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > Subject: Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > I am not sure if the ALL.txt discussion has come up before (probably has at > some point), but certainly CALL3.txt has made the rounds several times. I > was able to get CALL3 Database functionality working with SQLite for WSJT, > and have a stand alone application for WSPR to process millions of spots the > archives produce monthly, however, it does add a level of complexity to the > application that may be more than is needed for minimal operation. > > It's hard to dismiss the simplicity of flat text files. On the other hand, > having data in tables lends itself to many positive benefits. For example, > the queries in this thread would be a fairly simple SQL query rather than > requiring a Black Belt in command-line foo to extract it. > Not to mention, the SQL language is cross-platform, and generally, no > additional tools are required, but many exit at the users option. > > I am not suggesting that WSJT-X integrate post processing tools, rather, I > am merely suggesting that the data could be stored in a different format ( > DB tables) that allows easy access by third party developers or primary > users, in a more structured manner. > > > 73's > Greg, KI7MT > > On 11/7/2016 7:18 AM, Black Michael wrote: >> I wouldn't be inclined to add database support when the most common >> operations are achievable with the existing ALL.TXT. Haven't we >> discussed this before? Somebody said something about putting the >> messages in an sqlite database as I recall and had implemented something. >> Though I'm sure we could make it portable it's adding a fair bit of >> new stuff and overhead for doing this simple task. Right now my logic >> is about 100 lines of code and can pick up non-standard end-of-qso >> messages to some degree right now. >> >> Is there some other function desirable from a database that would >> justify it? I can see where adding a field for "Rx Frequency" window >> message presence would help in processing QSOs. Non-standard QSO >> messages would be an SQL challenge. >> >> >> de Mike W9MDB >> >> >> >> ---------- >> -- >> *From:* Greg Beam <ki7m...@gmail.com> >> *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development >> <wsjt-devel@lists.sourceforge.net> >> *Sent:* Monday, November 7, 2016 12:37 AM >> *Subject:* Re: [wsjt-devel] ALL.txt idea >> >> Hi Mike, >> >> This is another email I just pulled out of the spam box. No wonder I >> was missing the other half of the conversation. >> >> Ive been thinking about these text files for some time now. There has >> to be a better way than exercising command-line foo to get the data >> folks need. Not that I mind the command line, I spend most of my time >> there, but that's not very helpful to the masses. >> >> There seems to be allot of different uses for CALL3.txt and the >> ALL.txt files. I keep circling back to database tables
Re: [wsjt-devel] ALL.txt idea
Would be more true if the records were consistently formatted...but they aren't. What would you use a database for? de Mike W9MDB From: David Tiller <dtil...@captechconsulting.com> To: WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 3:08 PM Subject: Re: [wsjt-devel] ALL.txt idea I think Dan's idea of using comma- or tab-separated fields is a nice compromise between locking users into a particular database format (regardless of how ubiquitous) and unstructured rows. CSV can be imported/parsed/chunked/scattered/gathered 100 ways to Sunday. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com On Nov 7, 2016, at 4:01 PM, Dan Malcolm <dmalcol...@mchsi.com> wrote: > I too use ALL.TXT. But I believe a move towards some DB functionality could > be made use comma or tab delimited fields (CSV file) instead of a space. > The resulting text file could easily be imported into any DB management > system that I am familiar with. I don't believe it would add much to the > code. > > Just my $.02 worth. > > _ > Dan Malcolm CFI/II > K4SHQ > > > -Original Message- > From: Greg Beam [mailto:ki7m...@gmail.com] > Sent: Monday, November 07, 2016 10:00 AM > To: Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > Subject: Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > I am not sure if the ALL.txt discussion has come up before (probably has at > some point), but certainly CALL3.txt has made the rounds several times. I > was able to get CALL3 Database functionality working with SQLite for WSJT, > and have a stand alone application for WSPR to process millions of spots the > archives produce monthly, however, it does add a level of complexity to the > application that may be more than is needed for minimal operation. > > It's hard to dismiss the simplicity of flat text files. On the other hand, > having data in tables lends itself to many positive benefits. For example, > the queries in this thread would be a fairly simple SQL query rather than > requiring a Black Belt in command-line foo to extract it. > Not to mention, the SQL language is cross-platform, and generally, no > additional tools are required, but many exit at the users option. > > I am not suggesting that WSJT-X integrate post processing tools, rather, I > am merely suggesting that the data could be stored in a different format ( > DB tables) that allows easy access by third party developers or primary > users, in a more structured manner. > > > 73's > Greg, KI7MT > > On 11/7/2016 7:18 AM, Black Michael wrote: >> I wouldn't be inclined to add database support when the most common >> operations are achievable with the existing ALL.TXT. Haven't we >> discussed this before? Somebody said something about putting the >> messages in an sqlite database as I recall and had implemented something. >> Though I'm sure we could make it portable it's adding a fair bit of >> new stuff and overhead for doing this simple task. Right now my logic >> is about 100 lines of code and can pick up non-standard end-of-qso >> messages to some degree right now. >> >> Is there some other function desirable from a database that would >> justify it? I can see where adding a field for "Rx Frequency" window >> message presence would help in processing QSOs. Non-standard QSO >> messages would be an SQL challenge. >> >> >> de Mike W9MDB >> >> >> >> ---------- >> -- >> *From:* Greg Beam <ki7m...@gmail.com> >> *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development >> <wsjt-devel@lists.sourceforge.net> >> *Sent:* Monday, November 7, 2016 12:37 AM >> *Subject:* Re: [wsjt-devel] ALL.txt idea >> >> Hi Mike, >> >> This is another email I just pulled out of the spam box. No wonder I >> was missing the other half of the conversation. >> >> Ive been thinking about these text files for some time now. There has >> to be a better way than exercising command-line foo to get the data >> folks need. Not that I mind the command line, I spend most of my time >> there, but that's not very helpful to the masses. >> >> There seems to be allot of different uses for CALL3.txt and the >> ALL.txt files. I keep circling back to database tables and I've played >> with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with >> WSJT-X I've not gotten through yet. >>
Re: [wsjt-devel] ALL.txt idea
I think Dan's idea of using comma- or tab-separated fields is a nice compromise between locking users into a particular database format (regardless of how ubiquitous) and unstructured rows. CSV can be imported/parsed/chunked/scattered/gathered 100 ways to Sunday. -- David Tiller Sr. Architect/Lead Consultant | CapTech (804) 304-0638 | dtil...@captechconsulting.com On Nov 7, 2016, at 4:01 PM, Dan Malcolm <dmalcol...@mchsi.com> wrote: > I too use ALL.TXT. But I believe a move towards some DB functionality could > be made use comma or tab delimited fields (CSV file) instead of a space. > The resulting text file could easily be imported into any DB management > system that I am familiar with. I don't believe it would add much to the > code. > > Just my $.02 worth. > > _ > Dan Malcolm CFI/II > K4SHQ > > > -Original Message- > From: Greg Beam [mailto:ki7m...@gmail.com] > Sent: Monday, November 07, 2016 10:00 AM > To: Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > Subject: Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > I am not sure if the ALL.txt discussion has come up before (probably has at > some point), but certainly CALL3.txt has made the rounds several times. I > was able to get CALL3 Database functionality working with SQLite for WSJT, > and have a stand alone application for WSPR to process millions of spots the > archives produce monthly, however, it does add a level of complexity to the > application that may be more than is needed for minimal operation. > > It's hard to dismiss the simplicity of flat text files. On the other hand, > having data in tables lends itself to many positive benefits. For example, > the queries in this thread would be a fairly simple SQL query rather than > requiring a Black Belt in command-line foo to extract it. > Not to mention, the SQL language is cross-platform, and generally, no > additional tools are required, but many exit at the users option. > > I am not suggesting that WSJT-X integrate post processing tools, rather, I > am merely suggesting that the data could be stored in a different format ( > DB tables) that allows easy access by third party developers or primary > users, in a more structured manner. > > > 73's > Greg, KI7MT > > On 11/7/2016 7:18 AM, Black Michael wrote: >> I wouldn't be inclined to add database support when the most common >> operations are achievable with the existing ALL.TXT. Haven't we >> discussed this before? Somebody said something about putting the >> messages in an sqlite database as I recall and had implemented something. >> Though I'm sure we could make it portable it's adding a fair bit of >> new stuff and overhead for doing this simple task. Right now my logic >> is about 100 lines of code and can pick up non-standard end-of-qso >> messages to some degree right now. >> >> Is there some other function desirable from a database that would >> justify it? I can see where adding a field for "Rx Frequency" window >> message presence would help in processing QSOs. Non-standard QSO >> messages would be an SQL challenge. >> >> >> de Mike W9MDB >> >> >> >> -------------- >> -- >> *From:* Greg Beam <ki7m...@gmail.com> >> *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development >> <wsjt-devel@lists.sourceforge.net> >> *Sent:* Monday, November 7, 2016 12:37 AM >> *Subject:* Re: [wsjt-devel] ALL.txt idea >> >> Hi Mike, >> >> This is another email I just pulled out of the spam box. No wonder I >> was missing the other half of the conversation. >> >> Ive been thinking about these text files for some time now. There has >> to be a better way than exercising command-line foo to get the data >> folks need. Not that I mind the command line, I spend most of my time >> there, but that's not very helpful to the masses. >> >> There seems to be allot of different uses for CALL3.txt and the >> ALL.txt files. I keep circling back to database tables and I've played >> with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with >> WSJT-X I've not gotten through yet. >> >> Maybe if we sit down and try to capture most or at least some of the >> requirements, we could start kicking around some DB integration design >> concepts. >> >> 73's >> Greg, KI7MT >> >> >> On 11/6/2016 11:03 AM, Black Michael wrote: >>> Pretty simplewhen, for examp
Re: [wsjt-devel] ALL.txt idea
I too use ALL.TXT. But I believe a move towards some DB functionality could be made use comma or tab delimited fields (CSV file) instead of a space. The resulting text file could easily be imported into any DB management system that I am familiar with. I don't believe it would add much to the code. Just my $.02 worth. _ Dan Malcolm CFI/II K4SHQ -Original Message- From: Greg Beam [mailto:ki7m...@gmail.com] Sent: Monday, November 07, 2016 10:00 AM To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, I am not sure if the ALL.txt discussion has come up before (probably has at some point), but certainly CALL3.txt has made the rounds several times. I was able to get CALL3 Database functionality working with SQLite for WSJT, and have a stand alone application for WSPR to process millions of spots the archives produce monthly, however, it does add a level of complexity to the application that may be more than is needed for minimal operation. It's hard to dismiss the simplicity of flat text files. On the other hand, having data in tables lends itself to many positive benefits. For example, the queries in this thread would be a fairly simple SQL query rather than requiring a Black Belt in command-line foo to extract it. Not to mention, the SQL language is cross-platform, and generally, no additional tools are required, but many exit at the users option. I am not suggesting that WSJT-X integrate post processing tools, rather, I am merely suggesting that the data could be stored in a different format ( DB tables) that allows easy access by third party developers or primary users, in a more structured manner. 73's Greg, KI7MT On 11/7/2016 7:18 AM, Black Michael wrote: > I wouldn't be inclined to add database support when the most common > operations are achievable with the existing ALL.TXT. Haven't we > discussed this before? Somebody said something about putting the > messages in an sqlite database as I recall and had implemented something. > Though I'm sure we could make it portable it's adding a fair bit of > new stuff and overhead for doing this simple task. Right now my logic > is about 100 lines of code and can pick up non-standard end-of-qso > messages to some degree right now. > > Is there some other function desirable from a database that would > justify it? I can see where adding a field for "Rx Frequency" window > message presence would help in processing QSOs. Non-standard QSO > messages would be an SQL challenge. > > > de Mike W9MDB > > > > -- > -- > *From:* Greg Beam <ki7m...@gmail.com> > *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > *Sent:* Monday, November 7, 2016 12:37 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > This is another email I just pulled out of the spam box. No wonder I > was missing the other half of the conversation. > > Ive been thinking about these text files for some time now. There has > to be a better way than exercising command-line foo to get the data > folks need. Not that I mind the command line, I spend most of my time > there, but that's not very helpful to the masses. > > There seems to be allot of different uses for CALL3.txt and the > ALL.txt files. I keep circling back to database tables and I've played > with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with > WSJT-X I've not gotten through yet. > > Maybe if we sit down and try to capture most or at least some of the > requirements, we could start kicking around some DB integration design > concepts. > > 73's > Greg, KI7MT > > > On 11/6/2016 11:03 AM, Black Michael wrote: >> Pretty simplewhen, for example, on eQSL, you get a mismatched QSL >> you can look at your messages to see if they got it wrong or you did. >> And you can submit the evidence to them so they can correct or you >> correct your own. >> >> I've also used it when noticing in JTAlert that a band isn't >> confirmed...I can look them up and see the evidence again and either >> correct my log or ask them to correct theirs. >> >> With LOTW you don't have that luxury.since there's no visibility of >> mismatched QSOs >> >> And yes...this is a WSJT Dev list item as I'm proposing submitting a >> patch to make this easy for everyone instead of just those with grep, >> EMACS, or whatever they may be currently using. >> >> de Mike W9MDB >> >> >> -------------
Re: [wsjt-devel] ALL.txt idea
I use the All.txt now for research by parsing it and feeding it into an SQLite database. I think that having WSJTX write to an SQLite or equivalent database would be a good solution, and as David says, separate purpose-built utilities accessing that database could serve a wide range of end users with varying needs without needlessly increasing the complexity of WSJTX. 73, Roger Rehr W3SZ On 11/7/2016 10:22 AM, David Tiller wrote: > > Michael, > > > My suggestion would be to follow the Unix design philosophy - do one > thing, and do it well. I vote for any of the ALL.TXT > searching/manipulation/modification be handled in a separate utility > that's decoupled from the WSJT-X codebase. > > > -- > > *David Tiller | *Senior Manager > dtil...@captechconsulting.com > c 804.304.0638 / o 804.355.0511 > > <http://www.captechconsulting.com/> > > <http://www.facebook.com/CapTechCareers> > <http://www.twitter.com/CapTechListens> > <http://www.linkedin.com/company/captech-ventures> > <http://www.stackoverflow.com/jobs/companies/captech-consulting> > <http://www.youtube.com/user/CapTechConsulting> > <https://www.instagram.com/lifeatcaptech/> > > *Best Firms 2016 #2* in Information Technology > > > > > *From:* Black Michael <mdblac...@yahoo.com> > *Sent:* Monday, November 7, 2016 9:18 AM > *To:* Greg Beam; WSJT software development > *Subject:* Re: [wsjt-devel] ALL.txt idea > > I wouldn't be inclined to add database support when the most common > operations are achievable with the existing ALL.TXT. Haven't we > discussed this before? Somebody said something about putting the > messages in an sqlite database as I recall and had implemented something. > Though I'm sure we could make it portable it's adding a fair bit of > new stuff and overhead for doing this simple task. Right now my logic > is about 100 lines of code and can pick up non-standard end-of-qso > messages to some degree right now. > > Is there some other function desirable from a database that would > justify it? I can see where adding a field for "Rx Frequency" window > message presence would help in processing QSOs. Non-standard QSO > messages would be an SQL challenge. > > > de Mike W9MDB > > > > ---- > *From:* Greg Beam <ki7m...@gmail.com> > *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > *Sent:* Monday, November 7, 2016 12:37 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > This is another email I just pulled out of the spam box. No wonder I was > missing the other half of the conversation. > > Ive been thinking about these text files for some time now. There has to > be a better way than exercising command-line foo to get the data folks > need. Not that I mind the command line, I spend most of my time there, > but that's not very helpful to the masses. > > There seems to be allot of different uses for CALL3.txt and the ALL.txt > files. I keep circling back to database tables and I've played with the > CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've > not gotten through yet. > > Maybe if we sit down and try to capture most or at least some of the > requirements, we could start kicking around some DB integration design > concepts. > > 73's > Greg, KI7MT > > > On 11/6/2016 11:03 AM, Black Michael wrote: > > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > > you can look at your messages to see if they got it wrong or you did. > > And you can submit the evidence to them so they can correct or you > > correct your own. > > > > I've also used it when noticing in JTAlert that a band isn't > > confirmed...I can look them up and see the evidence again and either > > correct my log or ask them to correct theirs. > > > > With LOTW you don't have that luxury.since there's no visibility of > > mismatched QSOs > > > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > > patch to make this easy for everyone instead of just those with grep, > > EMACS, or whatever they may be currently using. > > > > de Mike W9MDB > > > > > > > > *From:* Greg Beam <ki7m...@gmail.com <mailto:ki7m...@gmail.com>> > > *To:* WSJT software development <wsjt-devel@lists.sourceforge.net > <mailto:wsjt-devel@lists.sourceforge.net>> > > *Se
Re: [wsjt-devel] ALL.txt idea
Why would you want to do that? Far too complex, requires internet, and what advantage do you gain? de Mike W9MDB From: Stephen Treubig (K4WBF) <k4...@marauderlabs.co> To: WSJT software development <wsjt-devel@lists.sourceforge.net>; Black Michael <mdblac...@yahoo.com> Sent: Monday, November 7, 2016 10:54 AM Subject: RE: [wsjt-devel] ALL.txt idea What about putting the database in the cloud and doing a API call from WJST to it? C/T K4WBF -Original Message- From: Greg Beam [mailto:ki7m...@gmail.com] Sent: Monday, November 7, 2016 10:00 AM To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, I am not sure if the ALL.txt discussion has come up before (probably has at some point), but certainly CALL3.txt has made the rounds several times. I was able to get CALL3 Database functionality working with SQLite for WSJT, and have a stand alone application for WSPR to process millions of spots the archives produce monthly, however, it does add a level of complexity to the application that may be more than is needed for minimal operation. It's hard to dismiss the simplicity of flat text files. On the other hand, having data in tables lends itself to many positive benefits. For example, the queries in this thread would be a fairly simple SQL query rather than requiring a Black Belt in command-line foo to extract it. Not to mention, the SQL language is cross-platform, and generally, no additional tools are required, but many exit at the users option. I am not suggesting that WSJT-X integrate post processing tools, rather, I am merely suggesting that the data could be stored in a different format ( DB tables) that allows easy access by third party developers or primary users, in a more structured manner. 73's Greg, KI7MT On 11/7/2016 7:18 AM, Black Michael wrote: > I wouldn't be inclined to add database support when the most common > operations are achievable with the existing ALL.TXT. Haven't we > discussed this before? Somebody said something about putting the > messages in an sqlite database as I recall and had implemented something. > Though I'm sure we could make it portable it's adding a fair bit of > new stuff and overhead for doing this simple task. Right now my logic > is about 100 lines of code and can pick up non-standard end-of-qso > messages to some degree right now. > > Is there some other function desirable from a database that would > justify it? I can see where adding a field for "Rx Frequency" window > message presence would help in processing QSOs. Non-standard QSO > messages would be an SQL challenge. > > > de Mike W9MDB > > > > -- > -- > *From:* Greg Beam <ki7m...@gmail.com> > *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > *Sent:* Monday, November 7, 2016 12:37 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > This is another email I just pulled out of the spam box. No wonder I > was missing the other half of the conversation. > > Ive been thinking about these text files for some time now. There has > to be a better way than exercising command-line foo to get the data > folks need. Not that I mind the command line, I spend most of my time > there, but that's not very helpful to the masses. > > There seems to be allot of different uses for CALL3.txt and the > ALL.txt files. I keep circling back to database tables and I've played > with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with > WSJT-X I've not gotten through yet. > > Maybe if we sit down and try to capture most or at least some of the > requirements, we could start kicking around some DB integration design > concepts. > > 73's > Greg, KI7MT > > > On 11/6/2016 11:03 AM, Black Michael wrote: >> Pretty simplewhen, for example, on eQSL, you get a mismatched QSL >> you can look at your messages to see if they got it wrong or you did. >> And you can submit the evidence to them so they can correct or you >> correct your own. >> >> I've also used it when noticing in JTAlert that a band isn't >> confirmed...I can look them up and see the evidence again and either >> correct my log or ask them to correct theirs. >> >> With LOTW you don't have that luxury.since there's no visibility of >> mismatched QSOs >> >> And yes...this is a WSJT Dev list item as I'm proposing submitting a >> patch to make this easy for everyone instead of just those with grep, >> EMACS, or whatever they may be currently using. >> >> de Mike W9
Re: [wsjt-devel] ALL.txt idea
Michael, My suggestion would be to follow the Unix design philosophy - do one thing, and do it well. I vote for any of the ALL.TXT searching/manipulation/modification be handled in a separate utility that's decoupled from the WSJT-X codebase. -- David Tiller | Senior Manager dtil...@captechconsulting.com c 804.304.0638 / o 804.355.0511 [http://www.captechconsulting.com/siggen/logo.png]<http://www.captechconsulting.com/> [http://www.captechconsulting.com/siggen/Dkblue_Facebook.png]<http://www.facebook.com/CapTechCareers> [http://www.captechconsulting.com/siggen/Dkblue_Twitter.png] <http://www.twitter.com/CapTechListens> [http://www.captechconsulting.com/siggen/Dkblue_LinkedIn.png] <http://www.linkedin.com/company/captech-ventures> [http://www.captechconsulting.com/siggen/Dkblue_StackOverflow.png] <http://www.stackoverflow.com/jobs/companies/captech-consulting> [http://www.captechconsulting.com/siggen/Dkblue_YouTube.png] <http://www.youtube.com/user/CapTechConsulting> [http://www.captechconsulting.com/siggen/Dkblue_Instagram.png] <https://www.instagram.com/lifeatcaptech/> Best Firms 2016 #2 in Information Technology From: Black Michael <mdblac...@yahoo.com> Sent: Monday, November 7, 2016 9:18 AM To: Greg Beam; WSJT software development Subject: Re: [wsjt-devel] ALL.txt idea I wouldn't be inclined to add database support when the most common operations are achievable with the existing ALL.TXT. Haven't we discussed this before? Somebody said something about putting the messages in an sqlite database as I recall and had implemented something. Though I'm sure we could make it portable it's adding a fair bit of new stuff and overhead for doing this simple task. Right now my logic is about 100 lines of code and can pick up non-standard end-of-qso messages to some degree right now. Is there some other function desirable from a database that would justify it? I can see where adding a field for "Rx Frequency" window message presence would help in processing QSOs. Non-standard QSO messages would be an SQL challenge. de Mike W9MDB From: Greg Beam <ki7m...@gmail.com> To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 12:37 AM Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, This is another email I just pulled out of the spam box. No wonder I was missing the other half of the conversation. Ive been thinking about these text files for some time now. There has to be a better way than exercising command-line foo to get the data folks need. Not that I mind the command line, I spend most of my time there, but that's not very helpful to the masses. There seems to be allot of different uses for CALL3.txt and the ALL.txt files. I keep circling back to database tables and I've played with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've not gotten through yet. Maybe if we sit down and try to capture most or at least some of the requirements, we could start kicking around some DB integration design concepts. 73's Greg, KI7MT On 11/6/2016 11:03 AM, Black Michael wrote: > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > you can look at your messages to see if they got it wrong or you did. > And you can submit the evidence to them so they can correct or you > correct your own. > > I've also used it when noticing in JTAlert that a band isn't > confirmed...I can look them up and see the evidence again and either > correct my log or ask them to correct theirs. > > With LOTW you don't have that luxury.since there's no visibility of > mismatched QSOs > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > patch to make this easy for everyone instead of just those with grep, > EMACS, or whatever they may be currently using. > > de Mike W9MDB > > > > *From:* Greg Beam <ki7m...@gmail.com<mailto:ki7m...@gmail.com>> > *To:* WSJT software development > <wsjt-devel@lists.sourceforge.net<mailto:wsjt-devel@lists.sourceforge.net>> > *Sent:* Sunday, November 6, 2016 11:40 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hello All, > > I'm not sure this is a WSJT Dev list item, and, I arrived late to the > party on this is seems, but: > > What is the end goal here? What are you trying accomplish with the > ALL.txt file? > > 73's > Greg, KI7MT > > > On 11/6/2016 10:05 AM, Claude Frantz wrote: >> On 11/05/2016 03:42 PM, Black Michael wrote: >> >> Hi Michael, >> >>> And how are you doing that? I wrote a program so all I have to do is >>> "qsos dj0ot&q
Re: [wsjt-devel] ALL.txt idea
Hi Mike, I am not sure if the ALL.txt discussion has come up before (probably has at some point), but certainly CALL3.txt has made the rounds several times. I was able to get CALL3 Database functionality working with SQLite for WSJT, and have a stand alone application for WSPR to process millions of spots the archives produce monthly, however, it does add a level of complexity to the application that may be more than is needed for minimal operation. It's hard to dismiss the simplicity of flat text files. On the other hand, having data in tables lends itself to many positive benefits. For example, the queries in this thread would be a fairly simple SQL query rather than requiring a Black Belt in command-line foo to extract it. Not to mention, the SQL language is cross-platform, and generally, no additional tools are required, but many exit at the users option. I am not suggesting that WSJT-X integrate post processing tools, rather, I am merely suggesting that the data could be stored in a different format ( DB tables) that allows easy access by third party developers or primary users, in a more structured manner. 73's Greg, KI7MT On 11/7/2016 7:18 AM, Black Michael wrote: > I wouldn't be inclined to add database support when the most common > operations are achievable with the existing ALL.TXT. Haven't we > discussed this before? Somebody said something about putting the > messages in an sqlite database as I recall and had implemented something. > Though I'm sure we could make it portable it's adding a fair bit of new > stuff and overhead for doing this simple task. Right now my logic is > about 100 lines of code and can pick up non-standard end-of-qso messages > to some degree right now. > > Is there some other function desirable from a database that would > justify it? I can see where adding a field for "Rx Frequency" window > message presence would help in processing QSOs. Non-standard QSO > messages would be an SQL challenge. > > > de Mike W9MDB > > > > > *From:* Greg Beam <ki7m...@gmail.com> > *To:* Black Michael <mdblac...@yahoo.com>; WSJT software development > <wsjt-devel@lists.sourceforge.net> > *Sent:* Monday, November 7, 2016 12:37 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hi Mike, > > This is another email I just pulled out of the spam box. No wonder I was > missing the other half of the conversation. > > Ive been thinking about these text files for some time now. There has to > be a better way than exercising command-line foo to get the data folks > need. Not that I mind the command line, I spend most of my time there, > but that's not very helpful to the masses. > > There seems to be allot of different uses for CALL3.txt and the ALL.txt > files. I keep circling back to database tables and I've played with the > CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've > not gotten through yet. > > Maybe if we sit down and try to capture most or at least some of the > requirements, we could start kicking around some DB integration design > concepts. > > 73's > Greg, KI7MT > > > On 11/6/2016 11:03 AM, Black Michael wrote: >> Pretty simplewhen, for example, on eQSL, you get a mismatched QSL >> you can look at your messages to see if they got it wrong or you did. >> And you can submit the evidence to them so they can correct or you >> correct your own. >> >> I've also used it when noticing in JTAlert that a band isn't >> confirmed...I can look them up and see the evidence again and either >> correct my log or ask them to correct theirs. >> >> With LOTW you don't have that luxury.since there's no visibility of >> mismatched QSOs >> >> And yes...this is a WSJT Dev list item as I'm proposing submitting a >> patch to make this easy for everyone instead of just those with grep, >> EMACS, or whatever they may be currently using. >> >> de Mike W9MDB >> >> >> ------------ >> *From:* Greg Beam <ki7m...@gmail.com <mailto:ki7m...@gmail.com>> >> *To:* WSJT software development <wsjt-devel@lists.sourceforge.net > <mailto:wsjt-devel@lists.sourceforge.net>> >> *Sent:* Sunday, November 6, 2016 11:40 AM >> *Subject:* Re: [wsjt-devel] ALL.txt idea >> >> Hello All, >> >> I'm not sure this is a WSJT Dev list item, and, I arrived late to the >> party on this is seems, but: >> >> What is the end goal here? What are you trying accomplish with the >> ALL.txt file? >> >> 73's >> Greg, KI7MT >> >> >> On 11
Re: [wsjt-devel] ALL.txt idea
Here's what I have now. It's able to pick out non-standard messages at the end of a QSO. Should pick out other non-standards in the middle too. So this example QSO pops out when searching my files. 2014-Nov-21 2136 Transmitting 28.0758 MHz JT65: WB6BNE W9MDB EM492014-Nov-21 2137 -1 -0.2 1334 # W9MDB WB6BNE -102014-Nov-21 2138 Transmitting 28.0758 MHz JT65: WB6BNE W9MDB R-012014-Nov-21 2139 -1 -0.2 1334 # W9MDB RRR 732014-Nov-21 2140 Transmitting 28.0758 MHz JT65: TU 5 BANDS 73 I use a one line batch file called qsos.bat to run it. I backup my ALL.TXT regularly plus keep archived prior years in Dropbox so I have 3 files to search right now. C:\Users\%USERNAME%\Dropbox\Ham\WSJT-X\alltxt w9mdb %1 c:\Users\%USERNAME%\Dropbox\Ham\WSJT-X So running "qsos wb6bne" will find every "ALL*.[txt|TXT]" file in the path and search for the requested QSO partner. https://www.dropbox.com/s/c8ysex4z7wbb2wl/alltxt.exe?dl=1 https://www.dropbox.com/s/vdugalwbsz36xkm/alltxt.c?dl=1 This is a standalone program right now so Windows needs the dirent function. In WSJT-X would use a Qt function to do that me thinkst. https://www.dropbox.com/s/84vv2cyufehssz1/dirent.c?dl=1 https://www.dropbox.com/s/hszext4ap13vph2/dirent.h?dl=1 de Mike W9MDB From: Greg Beam <ki7m...@gmail.com> To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 12:37 AM Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, This is another email I just pulled out of the spam box. No wonder I was missing the other half of the conversation. Ive been thinking about these text files for some time now. There has to be a better way than exercising command-line foo to get the data folks need. Not that I mind the command line, I spend most of my time there, but that's not very helpful to the masses. There seems to be allot of different uses for CALL3.txt and the ALL.txt files. I keep circling back to database tables and I've played with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've not gotten through yet. Maybe if we sit down and try to capture most or at least some of the requirements, we could start kicking around some DB integration design concepts. 73's Greg, KI7MT On 11/6/2016 11:03 AM, Black Michael wrote: > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > you can look at your messages to see if they got it wrong or you did. > And you can submit the evidence to them so they can correct or you > correct your own. > > I've also used it when noticing in JTAlert that a band isn't > confirmed...I can look them up and see the evidence again and either > correct my log or ask them to correct theirs. > > With LOTW you don't have that luxury.since there's no visibility of > mismatched QSOs > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > patch to make this easy for everyone instead of just those with grep, > EMACS, or whatever they may be currently using. > > de Mike W9MDB > > > > *From:* Greg Beam <ki7m...@gmail.com> > *To:* WSJT software development <wsjt-devel@lists.sourceforge.net> > *Sent:* Sunday, November 6, 2016 11:40 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hello All, > > I'm not sure this is a WSJT Dev list item, and, I arrived late to the > party on this is seems, but: > > What is the end goal here? What are you trying accomplish with the > ALL.txt file? > > 73's > Greg, KI7MT > > > On 11/6/2016 10:05 AM, Claude Frantz wrote: >> On 11/05/2016 03:42 PM, Black Michael wrote: >> >> Hi Michael, >> >>> And how are you doing that? I wrote a program so all I have to do is >>> "qsos dj0ot" >> >> I'm simply using the emacs editor, which has fine search functions. >> >>> Don't have any with you but here's what the output for me looks like for >>> DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" >>> wildcard match) >> >> OK ! The same result is possible using the grep utility, which is >> available in nearly all Unix and Linux distributions. It's really a >> standard tool. >> >> Best 88 de Claude (DJ0OT) >> > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > w
Re: [wsjt-devel] ALL.txt idea
I wouldn't be inclined to add database support when the most common operations are achievable with the existing ALL.TXT. Haven't we discussed this before? Somebody said something about putting the messages in an sqlite database as I recall and had implemented something.Though I'm sure we could make it portable it's adding a fair bit of new stuff and overhead for doing this simple task. Right now my logic is about 100 lines of code and can pick up non-standard end-of-qso messages to some degree right now. Is there some other function desirable from a database that would justify it? I can see where adding a field for "Rx Frequency" window message presence would help in processing QSOs. Non-standard QSO messages would be an SQL challenge. de Mike W9MDB From: Greg Beam <ki7m...@gmail.com> To: Black Michael <mdblac...@yahoo.com>; WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Monday, November 7, 2016 12:37 AM Subject: Re: [wsjt-devel] ALL.txt idea Hi Mike, This is another email I just pulled out of the spam box. No wonder I was missing the other half of the conversation. Ive been thinking about these text files for some time now. There has to be a better way than exercising command-line foo to get the data folks need. Not that I mind the command line, I spend most of my time there, but that's not very helpful to the masses. There seems to be allot of different uses for CALL3.txt and the ALL.txt files. I keep circling back to database tables and I've played with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've not gotten through yet. Maybe if we sit down and try to capture most or at least some of the requirements, we could start kicking around some DB integration design concepts. 73's Greg, KI7MT On 11/6/2016 11:03 AM, Black Michael wrote: > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > you can look at your messages to see if they got it wrong or you did. > And you can submit the evidence to them so they can correct or you > correct your own. > > I've also used it when noticing in JTAlert that a band isn't > confirmed...I can look them up and see the evidence again and either > correct my log or ask them to correct theirs. > > With LOTW you don't have that luxury.since there's no visibility of > mismatched QSOs > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > patch to make this easy for everyone instead of just those with grep, > EMACS, or whatever they may be currently using. > > de Mike W9MDB > > > > *From:* Greg Beam <ki7m...@gmail.com> > *To:* WSJT software development <wsjt-devel@lists.sourceforge.net> > *Sent:* Sunday, November 6, 2016 11:40 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hello All, > > I'm not sure this is a WSJT Dev list item, and, I arrived late to the > party on this is seems, but: > > What is the end goal here? What are you trying accomplish with the > ALL.txt file? > > 73's > Greg, KI7MT > > > On 11/6/2016 10:05 AM, Claude Frantz wrote: >> On 11/05/2016 03:42 PM, Black Michael wrote: >> >> Hi Michael, >> >>> And how are you doing that? I wrote a program so all I have to do is >>> "qsos dj0ot" >> >> I'm simply using the emacs editor, which has fine search functions. >> >>> Don't have any with you but here's what the output for me looks like for >>> DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" >>> wildcard match) >> >> OK ! The same result is possible using the grep utility, which is >> available in nearly all Unix and Linux distributions. It's really a >> standard tool. >> >> Best 88 de Claude (DJ0OT) >> > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net <mailto:wsjt-devel@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > > > > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xe
Re: [wsjt-devel] ALL.txt idea
Hi Mike, This is another email I just pulled out of the spam box. No wonder I was missing the other half of the conversation. Ive been thinking about these text files for some time now. There has to be a better way than exercising command-line foo to get the data folks need. Not that I mind the command line, I spend most of my time there, but that's not very helpful to the masses. There seems to be allot of different uses for CALL3.txt and the ALL.txt files. I keep circling back to database tables and I've played with the CALL.txt file a bit with WSJT, but the Qt / C++ deal with WSJT-X I've not gotten through yet. Maybe if we sit down and try to capture most or at least some of the requirements, we could start kicking around some DB integration design concepts. 73's Greg, KI7MT On 11/6/2016 11:03 AM, Black Michael wrote: > Pretty simplewhen, for example, on eQSL, you get a mismatched QSL > you can look at your messages to see if they got it wrong or you did. > And you can submit the evidence to them so they can correct or you > correct your own. > > I've also used it when noticing in JTAlert that a band isn't > confirmed...I can look them up and see the evidence again and either > correct my log or ask them to correct theirs. > > With LOTW you don't have that luxury.since there's no visibility of > mismatched QSOs > > And yes...this is a WSJT Dev list item as I'm proposing submitting a > patch to make this easy for everyone instead of just those with grep, > EMACS, or whatever they may be currently using. > > de Mike W9MDB > > > > *From:* Greg Beam <ki7m...@gmail.com> > *To:* WSJT software development <wsjt-devel@lists.sourceforge.net> > *Sent:* Sunday, November 6, 2016 11:40 AM > *Subject:* Re: [wsjt-devel] ALL.txt idea > > Hello All, > > I'm not sure this is a WSJT Dev list item, and, I arrived late to the > party on this is seems, but: > > What is the end goal here? What are you trying accomplish with the > ALL.txt file? > > 73's > Greg, KI7MT > > > On 11/6/2016 10:05 AM, Claude Frantz wrote: >> On 11/05/2016 03:42 PM, Black Michael wrote: >> >> Hi Michael, >> >>> And how are you doing that? I wrote a program so all I have to do is >>> "qsos dj0ot" >> >> I'm simply using the emacs editor, which has fine search functions. >> >>> Don't have any with you but here's what the output for me looks like for >>> DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" >>> wildcard match) >> >> OK ! The same result is possible using the grep utility, which is >> available in nearly all Unix and Linux distributions. It's really a >> standard tool. >> >> Best 88 de Claude (DJ0OT) >> > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net <mailto:wsjt-devel@lists.sourceforge.net> > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > > > > > -- > Developer Access Program for Intel Xeon Phi Processors > Access to Intel Xeon Phi processor-based developer platforms. > With one year of Intel Parallel Studio XE. > Training and support from Colfax. > Order your platform today. http://sdm.link/xeonphi > > > > ___ > wsjt-devel mailing list > wsjt-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/wsjt-devel > -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
Installing Git for windows (Git Bash) will give all sorts of powerful *Nix tools; Grep, Awk, Sed, Cat, Cut, TR and so on. Additionally, If you have JTSDK-Win installed, you have all those tools and many more available through any of the JTSDK-ENV windows. 73's Greg, KI7MT > Hi Michael, > >> You're making my point that most can't do this...most don't have either >> of those tools. > > They are available for a MS-Windows, but not per default. > -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
With my utility:Searching
c:\Users\Black\Dropbox\Ham\WSJT-X\ALL-ICOM.txtSearching
c:\Users\Black\Dropbox\Ham\WSJT-X\ALL.TXT2016-Jul-26 2201 Transmitting 14.076
MHz JT9 : NO7T W9MDB EM492016-Jul-26 2202 -12 -0.1 2846
@ W9MDB NO7T -192016-Jul-26 2203 Transmitting 14.076 MHz JT9 : NO7T W9MDB
EM492016-Jul-26 2203 Transmitting 14.076 MHz JT9 : NO7T W9MDB
R-142016-Jul-26 2205 Transmitting 14.076 MHz JT9 : NO7T W9MDB
732016-Aug-27 1344 Transmitting 7.076 MHz JT65 : NO7T W9MDB
EM492016-Oct-11 1325 Transmitting 7.076 MHz JT9 : NO7T W9MDB
EM492016-Oct-11 1327 Transmitting 7.076 MHz JT9 : NO7T W9MDB
EM49Searching c:\Users\Black\Dropbox\Ham\WSJT-X\ALL2015.txt
With your method i get 1,559 records.
If you stack the greps you get closer...but no date.
C:\Users\Black\Dropbox\Ham\WSJT-X>grep -i NO7T all*.txt| grep W9MDBALL.TXT:2201
Transmitting 14.076 MHz JT9: NO7T W9MDB EM49ALL.TXT:2202 -12 -0.1 2846 @
W9MDB NO7T -19ALL.TXT:2203 Transmitting 14.076 MHz JT9: NO7T W9MDB
EM49ALL.TXT:2203 Transmitting 14.076 MHz JT9: NO7T W9MDB R-14ALL.TXT:2205
Transmitting 14.076 MHz JT9: NO7T W9MDB 73ALL.TXT:1344 Transmitting 7.076
MHz JT65: NO7T W9MDB EM49ALL.TXT:1325 Transmitting 7.076 MHz JT9: NO7T
W9MDB EM49ALL.TXT:1327 Transmitting 7.076 MHz JT9: NO7T W9MDB EM49
Of over 4,000 users how many do you think even know how to do this?For those of
us on the developers list more likely to know how.
But we shouldn't be writing software for "experts" alone.
de Mike W9MDB
From: Claude Frantz <claude.fra...@bayern-mail.de>
To: wsjt-devel@lists.sourceforge.net
Sent: Sunday, November 6, 2016 1:10 PM
Subject: Re: [wsjt-devel] ALL.txt idea
On 11/06/2016 07:06 PM, Black Michael wrote:
Hi Michael,
> You're making my point that most can't do this...most don't have either
> of those tools.
They are available for a MS-Windows, but not per default.
> I did use grep for a while and found it inadequate.
You get exactly the same result using "grep -i DJ0OT ALL*.TXT", e.g.
> My util, for example, reformats the line to include the date on every line.
> You can then pump that through grep again to look for a year or month of
> interest if you have too many results.
grep is a very flexible tool. Entering "grep -P
'(^2[0-9]{3}-[A-Z][a-z]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}|DJ0OT)' ALL.TXT"
and you get the date too.
In fact, you can add numerous other addition features, if you really
want, but do you think that this is really necessary to solve the
mentioned problem ?
Best 88 de Claude
--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
On 11/06/2016 07:06 PM, Black Michael wrote:
Hi Michael,
> You're making my point that most can't do this...most don't have either
> of those tools.
They are available for a MS-Windows, but not per default.
> I did use grep for a while and found it inadequate.
You get exactly the same result using "grep -i DJ0OT ALL*.TXT", e.g.
> My util, for example, reformats the line to include the date on every line.
> You can then pump that through grep again to look for a year or month of
> interest if you have too many results.
grep is a very flexible tool. Entering "grep -P
'(^2[0-9]{3}-[A-Z][a-z]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}|DJ0OT)' ALL.TXT"
and you get the date too.
In fact, you can add numerous other addition features, if you really
want, but do you think that this is really necessary to solve the
mentioned problem ?
Best 88 de Claude
--
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today. http://sdm.link/xeonphi
___
wsjt-devel mailing list
wsjt-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
You're making my point that most can't do this...most don't have either of those tools.I did use grep for a while and found it inadequate.My util, for example, reformats the line to include the date on every line.You can then pump that through grep again to look for a year or month of interest if you have too many results.If I added it to WSJT-X I'd add that capability. de Mike W9MDB From: Claude Frantz <claude.fra...@bayern-mail.de> To: wsjt-devel@lists.sourceforge.net Sent: Sunday, November 6, 2016 11:05 AM Subject: Re: [wsjt-devel] ALL.txt idea On 11/05/2016 03:42 PM, Black Michael wrote: Hi Michael, > And how are you doing that? I wrote a program so all I have to do is > "qsos dj0ot" I'm simply using the emacs editor, which has fine search functions. > Don't have any with you but here's what the output for me looks like for > DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" > wildcard match) OK ! The same result is possible using the grep utility, which is available in nearly all Unix and Linux distributions. It's really a standard tool. Best 88 de Claude (DJ0OT) -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
Pretty simplewhen, for example, on eQSL, you get a mismatched QSL you can look at your messages to see if they got it wrong or you did.And you can submit the evidence to them so they can correct or you correct your own. I've also used it when noticing in JTAlert that a band isn't confirmed...I can look them up and see the evidence again and either correct my log or ask them to correct theirs. With LOTW you don't have that luxury.since there's no visibility of mismatched QSOs And yes...this is a WSJT Dev list item as I'm proposing submitting a patch to make this easy for everyone instead of just those with grep, EMACS, or whatever they may be currently using. de Mike W9MDB From: Greg Beam <ki7m...@gmail.com> To: WSJT software development <wsjt-devel@lists.sourceforge.net> Sent: Sunday, November 6, 2016 11:40 AM Subject: Re: [wsjt-devel] ALL.txt idea Hello All, I'm not sure this is a WSJT Dev list item, and, I arrived late to the party on this is seems, but: What is the end goal here? What are you trying accomplish with the ALL.txt file? 73's Greg, KI7MT On 11/6/2016 10:05 AM, Claude Frantz wrote: > On 11/05/2016 03:42 PM, Black Michael wrote: > > Hi Michael, > >> And how are you doing that? I wrote a program so all I have to do is >> "qsos dj0ot" > > I'm simply using the emacs editor, which has fine search functions. > >> Don't have any with you but here's what the output for me looks like for >> DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" >> wildcard match) > > OK ! The same result is possible using the grep utility, which is > available in nearly all Unix and Linux distributions. It's really a > standard tool. > > Best 88 de Claude (DJ0OT) > -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
And how are you doing that? I wrote a program so all I have to do is "qsos dj0ot" Don't have any with you but here's what the output for me looks like for DJ0TP -- it looks at two files (actually looks at every "ALL*.txt" wildcard match) I doubt many ops have the ability to do this easily. Searching C:\Users\mike/Dropbox/Ham/WSJT-X/ALL.TXT2014-Oct-15 1624 -1 0.9 1512 # W9MDB DJ0TP JO432014-Oct-15 1625 Transmitting 21.0755 MHz JT65: DJ0TP W9MDB -012014-Oct-15 1626 -2 0.9 1512 # W9MDB DJ0TP R-052014-Oct-15 1627 Transmitting 21.0755 MHz JT65: DJ0TP W9MDB RRR2014-Oct-15 1628 -4 0.9 1512 # W9MDB DJ0TP 732014-Oct-15 1629 Transmitting 21.0755 MHz JT65: DJ0TP W9MDB 73Searching C:\Users\mike/Dropbox/Ham/WSJT-X/ALL-ICOM.TXT de Mike W9MDB From: Claude Frantz <claude.fra...@bayern-mail.de> To: wsjt-devel@lists.sourceforge.net Sent: Saturday, November 5, 2016 4:47 AM Subject: Re: [wsjt-devel] ALL.txt idea On 11/05/2016 07:14 AM, Black Michael wrote: Hi Michael, > I use the ALL.txt to answer questions about QSOs when needed (mostly > from eQSL). > So when I see a reject I can check the ALL.txt to see what happened. > So I can search for a callsign and pull up the history of that callsign > with me. That is exactly what I'm doing too. > I can write an addition to WSJT-X that I think should appear under the > File menu and replace the "Erase ALL.txt" with a dialog with several > options. > > #1 Erase > #2 Rename > #3 Search > > Would such an addition be OK? In my opinion, it's not necessary. Best 88 de Claude (DJ0OT) -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
Re: [wsjt-devel] ALL.txt idea
On 11/05/2016 07:14 AM, Black Michael wrote: Hi Michael, > I use the ALL.txt to answer questions about QSOs when needed (mostly > from eQSL). > So when I see a reject I can check the ALL.txt to see what happened. > So I can search for a callsign and pull up the history of that callsign > with me. That is exactly what I'm doing too. > I can write an addition to WSJT-X that I think should appear under the > File menu and replace the "Erase ALL.txt" with a dialog with several > options. > > #1 Erase > #2 Rename > #3 Search > > Would such an addition be OK? In my opinion, it's not necessary. Best 88 de Claude (DJ0OT) -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi ___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
[wsjt-devel] ALL.txt idea
I use the ALL.txt to answer questions about QSOs when needed (mostly from eQSL). So when I see a reject I can check the ALL.txt to see what happened.So I can search for a callsign and pull up the history of that callsign with me. In addition, I keep ALL.txt files around and archive them off by year so I can search back more than one year. I can write an addition to WSJT-X that I think should appear under the File menu and replace the "Erase ALL.txt" with a dialog with several options. #1 Erase#2 Rename#3 Search Would such an addition be OK? de Mike W9MDB -- Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi___ wsjt-devel mailing list wsjt-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/wsjt-devel
