[gentoo-user] Inconsistent behavior in my Gentoo OS instance
Hi! I have tried hard to make this message to the point. But that wasn't easy, as what I figured out about gzip --only after hours of work-- I still do need confirmation about, even though all appears correct. gzip apparently inconsistent behavior occupies the most part of the report on inconsistencies here (esp. the script make_gzip_archives_consistent.sh). Another part is actually on Wireshark mailing list. Pls. see: Filtering on (negated) frame.time_relative filters out wrong frame.number https://www.wireshark.org/lists/wireshark-users/201704/msg00037.html as well as my study at: https://www.croatiafidelis.hr/foss/cap/cap-170313-git-devuan-mail/git-devuan-mail-4.php (and the previous ones there, but I gave the last as it is simplest/fastest to check) There is information that any advanced reader can easily provide by retracing some of my steps there, and which would clear some uncertainties here. The third part is about Bash apparent misbehavior. And the fourth part is about eix complaining it has a bug, it's near the bottom part of this email. That is my starting point, and at this stage, after having spent quite a few hours on this issue below (but I have other issues not-directly-related to my system that haven't allowed me to finish and send this yet)... This issue below I thought at first could be related to my Wireshark woes, and now I don't... and at this stage (the proofreading time) I'm actually asking for any confirmation/denial on my findings about it... miro@g0n ~ $ ls -lh wireshark-users_miroR_q_bug_q.txt -rw-r--r-- 1 miro miro 802 2017-04-30 17:16 wireshark-users_miroR_q_bug_q.txt miro@g0n ~ $ ls -l --block-size=K wireshark-users_miroR_q_bug_q.txt -rw-r--r-- 1 miro miro 1K 2017-04-30 17:16 wireshark-users_miroR_q_bug_q.txt miro@g0n ~ $ mv -vi wireshark-users_miroR_q_bug_q.txtgzip_buggy.txt 'wireshark-users_miroR_q_bug_q.txt' -> 'gzip_buggy.txt' miro@g0n ~ $ tar czf gzip_buggy.txt.tar.gz gzip_buggy.txt miro@g0n ~ $ tar czf gzip_buggy.txt_1.tar.gz gzip_buggy.txt miro@g0n ~ $ tar czf gzip_buggy.txt_2.tar.gz gzip_buggy.txt miro@g0n ~ $ sha256sum gzip_buggy.txt*.tar.gz 26bcad15b4a81a3606859fdd4560af0e2c4cd875f1271808afb4569c50bfb5ec gzip_buggy.txt_1.tar.gz bd8f51bcd3274bbee4f73fa7ba9e1a2acd5db5d27f5643f72de10c52e224c22a gzip_buggy.txt_2.tar.gz 251a57e83ebaf4f07179f4a6500be80612a39ac6d2e97e482b30fb0a4de66f56 gzip_buggy.txt.tar.gz miro@g0n ~ $ You may check attachments: gzip_buggy.txt_1.tar.gz gzip_buggy.txt_2.tar.gz gzip_buggy.txt.tar.gz I think this is not like it should be. But the important question, really, is: is this the case in my machine only, or is it Gentoo-userdom wide? I fear it's the former, but like I asked in that Wireshark thread ( And it was amazing to see how little people care... there are always plenty of readers of Wireshark ML, and while not all can understand that thread --it took me years since I first used Wireshark to get to be able to minimally contribute there... lots of readers there such--, and of course those that can not really understand, need not reply... but lots of readers of Wireshark ML can easily repeat my steps that show in which exact fashion my Wireshark misbehaves... I'll be thankful is some Gentoo reader fills in this huge gap of carelessness.. ) [but like I asked in that Wireshark thread,] I ask here on Gentoo users ML (or readers from elsewhere). Pls. somebody confirm if this is, or is not, the case in their Gentoo machines. By simply decompressing the tar-gzip'd archive are do the tar-gzip compression as I did, and checking the sums of them. If you don't like my text file, you can use any file (like the ASCII file below, which I hope should be simple and fine enough), and reproduce what I tried with it. So let's try /usr/bin/eix-installed-after It's ASCII, it's Gentoo, everybody has it on this ML, but for visitors, it's in You may check attachments: eix-installed-after_1.tar.gz eix-installed-after_2.tar.gz eix-installed-after.tar.gz In my Gentoo, I copied it in my homedir and... miro@g0n ~ $ tar czf eix-installed-after.tar.gz eix-installed-after miro@g0n ~ $ tar czf eix-installed-after_1.tar.gz eix-installed-after miro@g0n ~ $ tar czf eix-installed-after_2.tar.gz eix-installed-after miro@g0n ~ $ sha256sum eix-installed-after*.tar.gz 80d004b1ecc6e6c2aec7cd453d05c7b4e4cc4092b9cef6b7c3fb31d148eb2310 eix-installed-after_1.tar.gz f4c30c6d4ccc90d1ec2cc6df2ac10cae2545d04a85c7ff39207d93129cd0ec78 eix-installed-after_2.tar.gz a8161c5771b661476966250af31d2b8bd80581a3e0c52d535b401524ee541ae7 eix-installed-after.tar.gz miro@g0n ~ $ And let me go further. Inspecting those...: miro@g0n ~ $ ls -l eix-installed-after*.tar.gz -rw-r--r-- 1 miro miro 3771 2017-04-30 17:39 eix-installed-after_1.tar.gz -rw-r--r-- 1 miro miro 3771 2017-04-30 17:39 eix-installed-after_2.tar.gz -rw-r--r-- 1 miro miro 3771 2017-04-30 17:39 eix-installed-after.tar.gz miro@g0n ~ $ ls -l --block-size=K eix-installed-after*.tar.gz
Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
Miroslav Rovis wrote: gzip apparently inconsistent behavior occupies the most part of the report on inconsistencies here (esp. the script make_gzip_archives_consistent.sh). Checked on my system, same behaviour, looking inside the gzip file you see why. I used shed but strings is easier: $ strings eix-installed-after_1.gz eix-installed-after_1 ... $ strings eix-installed-after_2.gz eix-installed-after_2 ... gzip stores the filename in the compressed file so the files differ. But you get different results even if you use the same file name, so digging into the file format (e.g. http://www.zlib.org/rfc-gzip.html#file-format) you find that gzip stores the MTIME (Modification TIME) in the file header, so even equally-named files will also differ. HTH, I did not have the time to go through your long email completely. raffaele
Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
On 170502-10:33+0200, Raffaele Belardi wrote: > Miroslav Rovis wrote: > > gzip apparently inconsistent behavior occupies the most part of the report > > on > > inconsistencies here (esp. the script make_gzip_archives_consistent.sh). > > Checked on my system, same behaviour, looking inside the gzip file you see > why. I used > shed but strings is easier: > > $ strings eix-installed-after_1.gz > eix-installed-after_1 > ... > > $ strings eix-installed-after_2.gz > eix-installed-after_2 > ... > > gzip stores the filename in the compressed file so the files differ. No, it doesn't, on my system. Did you really check the files: https://lists.gt.net/engine?do=post_attachment;postatt_id=51651;list=gentoo https://lists.gt.net/engine?do=post_attachment;postatt_id=51652;list=gentoo (these should download as eix-installed-after_1.gz former and eix-installed-after_2.gz the latter)? And they have these SHA256: fff6f3f0f07c863fee6962379f063f742578569fd13fcee3df9161b4a6d99aa7 eix-installed-after_1.tar.gz b88cd07885fbdc2235c9c64be7d02aa9ace7661cc2fce07909355e369366b408 eix-installed-after_2.tar.gz If you did check those files, and there are the strings you say, at what byte, the start, and the end... Really don't know how you got that... > But you get different results even if you use the same file name, so digging > into the file > format (e.g. http://www.zlib.org/rfc-gzip.html#file-format) you find that > gzip stores the > MTIME (Modification TIME) in the file header, so even equally-named files > will also differ. > > HTH, I did not have the time to go through your long email completely. > > raffaele And for easier insight into this plight of mine with these inconsistencies/issues, I am about to send another, I hope much clearer email --but no gzip issue in the new email, if gzip to discuss, pls, this sub-thread should better be used-- I resend a different email because I need the old quotes, removed in your reply... Thanks for caring! -- Miroslav Rovis Zagreb, Croatia https://www.CroatiaFidelis.hr signature.asc Description: Digital signature
Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
I've received one reply, and thanks again, but I had better remove the gzip-"inconsistency" related bloat from my own previous email... I need the previous text to make the remaining three important parts/issues/inconsistencies clearer and easier to check, and reply to, any of the three. I will also reorder my quotes to get them easier to skip or skip to, since they are separate issues/inconsistencies. On 170501-18:17+0200, Miroslav Rovis wrote: ... First issue === (All first issue-related text have been removed here from all quotes from my previous message) ... Second issue > Another part is actually on Wireshark mailing list. Pls. see: > > Filtering on (negated) frame.time_relative filters out wrong frame.number > https://www.wireshark.org/lists/wireshark-users/201704/msg00037.html > as well as my study at: > https://www.croatiafidelis.hr/foss/cap/cap-170313-git-devuan-mail/git-devuan-mail-4.php That page has just been updated with clearer instructions. > (and the previous ones there, but I gave the last as it is simplest/fastest > to check) > > There is information that any advanced reader can easily provide by retracing > some of my steps there, and which would clear some uncertainties here. ... > ... That's a serious bug or a > serious malfunction in my Gentoo, the latter being most likely... > > And if it is the latter, it can only be one or the other way. One: the cause > is in some Gentoo packge. Two: it is an attack by some unknown means. > > ( > If Air-Gapped is some info, I did try and editcap (and the whole > Wireshark) behave in the same wrong way in my Air-Gapped too. > ... > ) > Third issue == The text it too much because the command line in which bash throw strange error is a long for loop. The main point is marked with short new text below. > This is one of a series of commands that I used to check one of the backups, > in three different instances of tar-gzip'd archive I checked (such as the > /root directory tar-gzip'd today), and which showed faultless upon > decompression in all the three instances, despite the three instances of > tar-gzip'd archives not being identical (as their SHA256 sums show): > > # for i in $(ls -1d root_170430_g0n*.d/); do sum=$(echo $i|sed > 's/\.d\//\.sum/'); echo $sum ; read FAKE ; j=$(echo $i | sed > 's/\.d\//\.tar.gz/'); ls -l $j $i ; cd $i; pwd ; read FAKE ; for file in > $(find ./ -name '*'); do if [ -f "$file" ]; then sha256sum $file >> ../$sum ; > fi; done ; cd - ; done ; > > Now if I just place the cursor, by moving with Alt-F (skipping "words") and > Ctrl-F (skipping 1 char) to just after: > > "for i in $(ls -1d root_170430_g0n*.d/" in that command, > > and if I then hit Tab for completion on the experssion there, I get (and I'm > sorry for the mess, but that's what I get): > > g0n ~ # for i in $(ls -1d root_170430_g0n*.dbash: unexpected EOF while > looking for matching `)'bash: syntax error: unexpected end of > file//\.tar.gz/'); ls -l $j $i ; cd $i; pwd ; read FAKE ; for file in $(find > ./ -name '*'); do if [ -f "$file" ]; then sha256sum $file >> ../$sum ; fi; > done ; cd - ; done ; > > NOTE (at proofreading time): rechecked, I do get that same behavior the day > after (wrote most of this yesterday, still to send this morning). > > [[ > NOTE (before delayed sending): In fact, it is only this clone that exibits the > above Bash malfunctioning. I just checked the same for loop command (some six > paragraphs above) in my Air-Gapped master [1] (never any internet it sees, The [1] is important for understanding, especially this Bash issue in my Gentoo instance. Because in my Air-Gapped Gentoo instance that issue does not show at all. > longer workaround/detailed checking before updating it with stuff from > internet, sneakernet or optical media), and it is just fine. That line, simply > gave what it should: > > # for i in $(ls -1d root_170430_g0n*.d/); do sum=$(echo $i|sed > 's/\.d\//\.sum/'); echo $sum ; read FAKE ; j=$(echo $i | sed > 's/\.d\//\.tar.gz/'); ls -l $j $i ; cd $i; pwd ; read FAKE ; for file in > $(find ./ -name '*'); do if [ -f "$file" ]; then sha256sum $file >> ../$sum ; > fi; done ; cd - ; done > root_170430_g0n_1.d// root_170430_g0n_2.d// root_170430_g0n.d// > # [[and the same command line was back here]] > > under exact same conditions/circumstances as the clone of my Air-Gapped. And > it's similar with some other completion issues: they seem non-existent in my > Air-Gapped. > ]] This is the main point (in my clone that I use for online): > IOW, first, Bash sullied the entire line, which is not very considerate of > Her, and second that's not some usual error. Just for clarity, it wrote this: > The error: > bash: unexpected EOF while looking for matching `)'bash: syntax error: > unexpected end of file > > (and it wrote it by overwriting, which I never used to see in Bash) ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ | | | | | | | | | | | | Anyone else su
Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
Miroslav Rovis wrote: On 170502-10:33+0200, Raffaele Belardi wrote: Miroslav Rovis wrote: gzip apparently inconsistent behavior occupies the most part of the report on inconsistencies here (esp. the script make_gzip_archives_consistent.sh). Checked on my system, same behaviour, looking inside the gzip file you see why. I used shed but strings is easier: $ strings eix-installed-after_1.gz eix-installed-after_1 ... $ strings eix-installed-after_2.gz eix-installed-after_2 ... gzip stores the filename in the compressed file so the files differ. No, it doesn't, on my system. Did you really check the files: https://lists.gt.net/engine?do=post_attachment;postatt_id=51651;list=gentoo https://lists.gt.net/engine?do=post_attachment;postatt_id=51652;list=gentoo (these should download as eix-installed-after_1.gz former and eix-installed-after_2.gz the latter)? And they have these SHA256: fff6f3f0f07c863fee6962379f063f742578569fd13fcee3df9161b4a6d99aa7 eix-installed-after_1.tar.gz b88cd07885fbdc2235c9c64be7d02aa9ace7661cc2fce07909355e369366b408 eix-installed-after_2.tar.gz If you did check those files, and there are the strings you say, at what byte, the start, and the end... Really don't know how you got that... I did not use your files, I re-generated them on my system based on the /usr/bin/eix-installed-after installed on my system, as you suggested. The command I used was plain gzip, not tar, since the difference in the files appears to come from the gzip execution. I just checked your files: $ cmp -bl gzip_buggy.txt_1.tar.gz gzip_buggy.txt_2.tar.gz 5 7 ^G12 ^J They differ in byte 5 which, according to the link I posted, is inside the MTIME field. Looks to me that this gzip issue is a non-issue. Regarding the other issues, maybe someone else will have the time to go through the complete email, even the abridged one you re-sent is too much for me. Or maybe if you could concentrate on one issue at a time only... raffaele
RE: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
Regarding the fourth issue: > g0n ~ # eix memtest86+ > * sys-apps/memtest86 > Available versions: 4.3.7 (~)4.3.7-r1 {serial} > Homepage:http://www.memtest86.com/ > Description: A stand alone memory test for x86 computers ... > > Found 2 matches > Received SIGSEGV - you probably found a bug in eix. ... > Anyone else gets this too? Not here (note the results of your "eix memtest86+" appears to be a match for " eix memtest86" on my system): # eix memtest86+ [I] sys-apps/memtest86+ Available versions: 2.01^t 4.00^t 4.20-r1 ~4.20-r3 5.01-r2 ~5.01-r3 {floppy iso serial} Installed versions: 5.01-r2(11:23:03 AM 03/18/2017)(-floppy -iso -serial) Homepage:http://www.memtest.org/ Description: Memory tester based on memtest86 # eix memtest86 * sys-apps/memtest86 Available versions: 4.3.7 ~4.3.7-r1 {serial} Homepage:http://www.memtest86.com/ Description: A stand alone memory test for x86 computers [I] sys-apps/memtest86+ Available versions: 2.01^t 4.00^t 4.20-r1 ~4.20-r3 5.01-r2 ~5.01-r3 {floppy iso serial} Installed versions: 5.01-r2(11:23:03 AM 03/18/2017)(-floppy -iso -serial) Homepage:http://www.memtest.org/ Description: Memory tester based on memtest86 Found 2 matches #
Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
On 170502-17:51+0200, Raffaele Belardi wrote: > Miroslav Rovis wrote: > > On 170502-10:33+0200, Raffaele Belardi wrote: > >> Miroslav Rovis wrote: > >>> gzip apparently inconsistent behavior occupies the most part of the > >>> report on > >>> inconsistencies here (esp. the script make_gzip_archives_consistent.sh). > >> > >> Checked on my system, same behaviour, looking inside the gzip file you see > >> why. I used > >> shed but strings is easier: > >> > >> $ strings eix-installed-after_1.gz > >> eix-installed-after_1 > >> ... > >> > >> $ strings eix-installed-after_2.gz > >> eix-installed-after_2 > >> ... > >> > >> gzip stores the filename in the compressed file so the files differ. > > > > No, it doesn't, on my system. Did you really check the files: > > https://lists.gt.net/engine?do=post_attachment;postatt_id=51651;list=gentoo > > https://lists.gt.net/engine?do=post_attachment;postatt_id=51652;list=gentoo > > (these should download as eix-installed-after_1.gz former and > > eix-installed-after_2.gz the latter)? > > > > And they have these SHA256: > > > > fff6f3f0f07c863fee6962379f063f742578569fd13fcee3df9161b4a6d99aa7 > > eix-installed-after_1.tar.gz > > b88cd07885fbdc2235c9c64be7d02aa9ace7661cc2fce07909355e369366b408 > > eix-installed-after_2.tar.gz > > > > If you did check those files, and there are the strings you say, at what > > byte, the start, and the end... Really don't know how you got that... > > I did not use your files, I re-generated them on my system based on the > /usr/bin/eix-installed-after installed on my system, as you suggested. The > command I used > was plain gzip, not tar, since the difference in the files appears to come > from the gzip > execution. which then is not dealing with the same issue. > I just checked your files: > > $ cmp -bl gzip_buggy.txt_1.tar.gz gzip_buggy.txt_2.tar.gz >5 7 ^G12 ^J Didn't know about cmp. Thanks for a fine example! But cmp found the same which I found upon visual inspecting with hexdump, and which differences (but it was a futile non-necessary exercize) I removed with the script I gave in the first email. > They differ in byte 5 which, according to the link I posted, is inside the > MTIME field. > Looks to me that this gzip issue is a non-issue. Yes, and thanks for the confirmation. > Regarding the other issues, maybe someone else will have the time to go > through the > complete email, even the abridged one you re-sent is too much for me. Or > maybe if you > could concentrate on one issue at a time only... > > raffaele It is now (likely) only two (2) issues left to go of the four (4) there from the abridged email, because I got a reply for another issue in the meantime. But sadly more trouble looming with my system (looks actually from a bigger subject, the first onw on the way and it's shadow)... Regards! -- Miroslav Rovis Zagreb, Croatia https://www.CroatiaFidelis.hr signature.asc Description: Digital signature
Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
On 170502-22:19-0400, Bobby Kent wrote: > Regarding the fourth issue: > > g0n ~ # eix memtest86+ > > * sys-apps/memtest86 > > Available versions: 4.3.7 (~)4.3.7-r1 {serial} > > Homepage:http://www.memtest86.com/ > > Description: A stand alone memory test for x86 computers > ... > > > > Found 2 matches > > Received SIGSEGV - you probably found a bug in eix. > ... > > Anyone else gets this too? This below is a nice catch: > Not here (note the results of your "eix memtest86+" appears to be a match > for " eix memtest86" on my system): > > # eix memtest86+ > [I] sys-apps/memtest86+ > Available versions: 2.01^t 4.00^t 4.20-r1 ~4.20-r3 5.01-r2 ~5.01-r3 > {floppy iso serial} > Installed versions: 5.01-r2(11:23:03 AM 03/18/2017)(-floppy -iso > -serial) > Homepage:http://www.memtest.org/ > Description: Memory tester based on memtest86 > > # eix memtest86 > * sys-apps/memtest86 > Available versions: 4.3.7 ~4.3.7-r1 {serial} > Homepage:http://www.memtest86.com/ > Description: A stand alone memory test for x86 computers > > [I] sys-apps/memtest86+ > Available versions: 2.01^t 4.00^t 4.20-r1 ~4.20-r3 5.01-r2 ~5.01-r3 > {floppy iso serial} > Installed versions: 5.01-r2(11:23:03 AM 03/18/2017)(-floppy -iso > -serial) > Homepage:http://www.memtest.org/ > Description: Memory tester based on memtest86 > > Found 2 matches > # If you look up my first email, I do have both memtest86+ and memtest86 like you, and I do have the same versions available as you so I just wrongly abrdidged that second email. Sorry. But you don't have my issue with eix. Thanks for reporting. Two issues left to go of the ones I presented (and there are more, in slow time). The Wireshark and the Bash. Regards! -- Miroslav Rovis Zagreb, Croatia https://www.CroatiaFidelis.hr signature.asc Description: Digital signature
Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
On 170503-07:03+0200, Miroslav Rovis wrote: > On 170502-22:19-0400, Bobby Kent wrote: > > Regarding the fourth issue: > > > g0n ~ # eix memtest86+ > > > * sys-apps/memtest86 > > > Available versions: 4.3.7 (~)4.3.7-r1 {serial} > > > Homepage:http://www.memtest86.com/ > > > Description: A stand alone memory test for x86 computers > > ... > > > > > > Found 2 matches > > > Received SIGSEGV - you probably found a bug in eix. > > ... ... > Two issues left to go of the ones I presented (and there are more, in > slow time). The Wireshark and the Bash. > I would believe that what can be seen and read here: Strange script planted with Bash https://www.croatiafidelis.hr/foss/cap/cap-170504-strange-bash/ should make for some thinking... It's in the logs ( https://www.croatiafidelis.hr/foss/cap/cap-170504-strange-bash/messages_170504_2155_g0n [link is at bottom of page, under "messages_170504_2155_g0n"] ). I've studied similar logs, but previous, for hours, but decided to post this as quickly as I can. It's much more easily credible if not much later I post it publicly. I'll think more about it and try and ask questions, but there are some questions there that are obvious, I would believe... And the issue I would think is undeniable now... And also not too hard to see (just a quick careful glance at it, you are bound to see some trouble there). Regards! -- Miroslav Rovis Zagreb, Croatia https://www.CroatiaFidelis.hr signature.asc Description: Digital signature
RE: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
Hi Miroslav, Attempting to reproduce third issue: # mkdir wibble1_1 # mkdir wibble2_1 # mkdir wibble3_1 # mkdir wibble4_1 # mkdir wibble5_1 # for d in wibble*_1 ; do mkdir $d/wobble ; done # ls -1d wibble*_1 wibble1_1 wibble2_1 wibble3_1 wibble4_1 wibble5_1 Then hit tab after positioning cursor after the / below: # for i in $(ls -1d wibble*_1/) ; do echo $i ; done And the results are an attempt to autocomplete: wibble1_1// wibble2_1// wibble3_1// wibble4_1// wibble5_1// Perhaps the test oversimplified the issue, though maybe you could provide the simplest way to reproduce what you see. Thanks. -Original Message- From: Miroslav Rovis [mailto:miro.ro...@croatiafidelis.hr] Sent: Tuesday, May 02, 2017 10:13 To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance I've received one reply, and thanks again, but I had better remove the gzip-"inconsistency" related bloat from my own previous email... I need the previous text to make the remaining three important parts/issues/inconsistencies clearer and easier to check, and reply to, any of the three. I will also reorder my quotes to get them easier to skip or skip to, since they are separate issues/inconsistencies. On 170501-18:17+0200, Miroslav Rovis wrote: ... First issue === (All first issue-related text have been removed here from all quotes from my previous message) ... Second issue > Another part is actually on Wireshark mailing list. Pls. see: > > Filtering on (negated) frame.time_relative filters out wrong > frame.number > https://www.wireshark.org/lists/wireshark-users/201704/msg00037.html > as well as my study at: > https://www.croatiafidelis.hr/foss/cap/cap-170313-git-devuan-mail/git- > devuan-mail-4.php That page has just been updated with clearer instructions. > (and the previous ones there, but I gave the last as it is > simplest/fastest to check) > > There is information that any advanced reader can easily provide by > retracing some of my steps there, and which would clear some uncertainties here. ... > ... That's a serious bug or a > serious malfunction in my Gentoo, the latter being most likely... > > And if it is the latter, it can only be one or the other way. One: the > cause is in some Gentoo packge. Two: it is an attack by some unknown means. > > ( > If Air-Gapped is some info, I did try and editcap (and the whole > Wireshark) behave in the same wrong way in my Air-Gapped too. > ... > ) > Third issue == The text it too much because the command line in which bash throw strange error is a long for loop. The main point is marked with short new text below. > This is one of a series of commands that I used to check one of the > backups, in three different instances of tar-gzip'd archive I checked > (such as the /root directory tar-gzip'd today), and which showed > faultless upon decompression in all the three instances, despite the > three instances of tar-gzip'd archives not being identical (as their SHA256 sums show): > > # for i in $(ls -1d root_170430_g0n*.d/); do sum=$(echo $i|sed > 's/\.d\//\.sum/'); echo $sum ; read FAKE ; j=$(echo $i | sed > 's/\.d\//\.tar.gz/'); ls -l $j $i ; cd $i; pwd ; read FAKE ; for file > in $(find ./ -name '*'); do if [ -f "$file" ]; then sha256sum $file >> > ../$sum ; fi; done ; cd - ; done ; > > Now if I just place the cursor, by moving with Alt-F (skipping "words") and Ctrl-F (skipping 1 char) to just after: > > "for i in $(ls -1d root_170430_g0n*.d/" in that command, > > and if I then hit Tab for completion on the experssion there, I get > (and I'm sorry for the mess, but that's what I get): > > g0n ~ # for i in $(ls -1d root_170430_g0n*.dbash: unexpected EOF while > looking for matching `)'bash: syntax error: unexpected end of > file//\.tar.gz/'); ls -l $j $i ; cd $i; pwd ; read FAKE ; for file in > $(find ./ -name '*'); do if [ -f "$file" ]; then sha256sum $file >> > ../$sum ; fi; done ; cd - ; done ; > > NOTE (at proofreading time): rechecked, I do get that same behavior > the day after (wrote most of this yesterday, still to send this morning). > > [[ > NOTE (before delayed sending): In fact, it is only this clone that > exibits the above Bash malfunctioning. I just checked the same for > loop command (some six paragraphs above) in my Air-Gapped master [1] > (never any internet it sees, The [1] is important for understanding, especially this Bash issue in my Gentoo instance. Because in my Air-Gapped Gentoo instance that issue does not show at all. > longer workaround/detailed checking before updating it with stuff from > internet, sneakernet or op
Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
Hi Bobby! Pls. see also: Tab (no exec) triggers script on Bash on grsec admin https://forums.grsecurity.net/viewtopic.php?f=3&t=4700 as well as the other email that I sent some 7 or so hours ago. NOTE: if I'm away, it's because I'm a little worried... I'm afraid my system may be vulnerable because of these issues. Patience pls. (no more but only my sig in bottom) On 170504-21:15-0400, Bobby Kent wrote: > Hi Miroslav, > > Attempting to reproduce third issue: > > # mkdir wibble1_1 > # mkdir wibble2_1 > # mkdir wibble3_1 > # mkdir wibble4_1 > # mkdir wibble5_1 > # for d in wibble*_1 ; do mkdir $d/wobble ; done > # ls -1d wibble*_1 > wibble1_1 > wibble2_1 > wibble3_1 > wibble4_1 > wibble5_1 > > Then hit tab after positioning cursor after the / below: > # for i in $(ls -1d wibble*_1/) ; do echo $i ; done > > And the results are an attempt to autocomplete: > wibble1_1// wibble2_1// wibble3_1// wibble4_1// wibble5_1// > > Perhaps the test oversimplified the issue, though maybe you could provide > the simplest way to reproduce what you see. > > Thanks. > > > -Original Message- > From: Miroslav Rovis [mailto:miro.ro...@croatiafidelis.hr] > Sent: Tuesday, May 02, 2017 10:13 > To: gentoo-user@lists.gentoo.org > Subject: Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance > > I've received one reply, and thanks again, but I had better remove the > gzip-"inconsistency" related bloat from my own previous email... I need the > previous text to make the remaining three important > parts/issues/inconsistencies clearer and easier to check, and reply to, any > of the three. > > I will also reorder my quotes to get them easier to skip or skip to, since > they are separate issues/inconsistencies. > > On 170501-18:17+0200, Miroslav Rovis wrote: > ... > First issue > === > (All first issue-related text have been removed here from all quotes from my > previous message) ... > > Second issue > > > Another part is actually on Wireshark mailing list. Pls. see: > > > > Filtering on (negated) frame.time_relative filters out wrong > > frame.number > > https://www.wireshark.org/lists/wireshark-users/201704/msg00037.html > > as well as my study at: > > https://www.croatiafidelis.hr/foss/cap/cap-170313-git-devuan-mail/git- > > devuan-mail-4.php > That page has just been updated with clearer instructions. > > > (and the previous ones there, but I gave the last as it is > > simplest/fastest to check) > > > > There is information that any advanced reader can easily provide by > > retracing some of my steps there, and which would clear some uncertainties > here. > ... > > ... That's a serious bug or a > > serious malfunction in my Gentoo, the latter being most likely... > > > > And if it is the latter, it can only be one or the other way. One: the > > cause is in some Gentoo packge. Two: it is an attack by some unknown > means. > > > > ( > > If Air-Gapped is some info, I did try and editcap (and the whole > > Wireshark) behave in the same wrong way in my Air-Gapped too. > > ... > > ) > > > > > Third issue > == > > The text it too much because the command line in which bash throw strange > error is a long for loop. The main point is marked with short new text > below. > > This is one of a series of commands that I used to check one of the > > backups, in three different instances of tar-gzip'd archive I checked > > (such as the /root directory tar-gzip'd today), and which showed > > faultless upon decompression in all the three instances, despite the > > three instances of tar-gzip'd archives not being identical (as their > SHA256 sums show): > > > > # for i in $(ls -1d root_170430_g0n*.d/); do sum=$(echo $i|sed > > 's/\.d\//\.sum/'); echo $sum ; read FAKE ; j=$(echo $i | sed > > 's/\.d\//\.tar.gz/'); ls -l $j $i ; cd $i; pwd ; read FAKE ; for file > > in $(find ./ -name '*'); do if [ -f "$file" ]; then sha256sum $file >> > > ../$sum ; fi; done ; cd - ; done ; > > > > Now if I just place the cursor, by moving with Alt-F (skipping "words") > and Ctrl-F (skipping 1 char) to just after: > > > > "for i in $(ls -1d root_170430_g0n*.d/" in that command, > > > > and if I then hit Tab for completion on the experssion there, I get > > (and I'm sorry for the mess, but that's what I get): > > > > g0n ~ # for i in $(ls -1d root_170430_g0n*.dbash: unexp
RE: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
Looks like there are two things that concern you. Firstly, how bash tab expansion appears to work (the ls, etc. commands executed when you hit the tab key) on your system. Secondly, the "bash: unexpected EOF while looking for matching `)'bash: syntax error: unexpected end of file" messages generated when a particular tab expansion fails. Is that second issue generated by hitting tab at the end of the command: ls -1d root_170430_g0n*.d ? If so, perhaps there's something unusual with the items that match the pattern "root_170430_g0n*.d*" that results in the error ... Regarding your diagnosis: > That's a serious bug or a serious malfunction in my Gentoo, the latter being most likely... > > And if it is the latter, it can only be one or the other way. > One: the cause is in some Gentoo packge. > Two: it is an attack by some unknown means. Before declaring bug / broken package / attack, it might be an idea to see whether the issue is reproducible, and under what circumstances. Note, tab expansion can be modified (see, for example, http://www.linuxjournal.com/content/more-using-bash-complete-command). -Original Message- From: Miroslav Rovis [mailto:miro.ro...@croatiafidelis.hr] Sent: Friday, May 05, 2017 01:02 To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance Hi Bobby! Pls. see also: Tab (no exec) triggers script on Bash on grsec admin https://forums.grsecurity.net/viewtopic.php?f=3&t=4700 as well as the other email that I sent some 7 or so hours ago. NOTE: if I'm away, it's because I'm a little worried... I'm afraid my system may be vulnerable because of these issues. Patience pls. (no more but only my sig in bottom) On 170504-21:15-0400, Bobby Kent wrote: > Hi Miroslav, > > Attempting to reproduce third issue: > > # mkdir wibble1_1 > # mkdir wibble2_1 > # mkdir wibble3_1 > # mkdir wibble4_1 > # mkdir wibble5_1 > # for d in wibble*_1 ; do mkdir $d/wobble ; done # ls -1d wibble*_1 > wibble1_1 > wibble2_1 > wibble3_1 > wibble4_1 > wibble5_1 > > Then hit tab after positioning cursor after the / below: > # for i in $(ls -1d wibble*_1/) ; do echo $i ; done > > And the results are an attempt to autocomplete: > wibble1_1// wibble2_1// wibble3_1// wibble4_1// wibble5_1// > > Perhaps the test oversimplified the issue, though maybe you could > provide the simplest way to reproduce what you see. > > Thanks. > > > -Original Message- > From: Miroslav Rovis [mailto:miro.ro...@croatiafidelis.hr] > Sent: Tuesday, May 02, 2017 10:13 > To: gentoo-user@lists.gentoo.org > Subject: Re: [gentoo-user] Inconsistent behavior in my Gentoo OS > instance > > I've received one reply, and thanks again, but I had better remove the > gzip-"inconsistency" related bloat from my own previous email... I > need the previous text to make the remaining three important > parts/issues/inconsistencies clearer and easier to check, and reply > to, any of the three. > > I will also reorder my quotes to get them easier to skip or skip to, > since they are separate issues/inconsistencies. > > On 170501-18:17+0200, Miroslav Rovis wrote: > ... > First issue > === > (All first issue-related text have been removed here from all quotes > from my previous message) ... > > Second issue > > > Another part is actually on Wireshark mailing list. Pls. see: > > > > Filtering on (negated) frame.time_relative filters out wrong > > frame.number > > https://www.wireshark.org/lists/wireshark-users/201704/msg00037.html > > as well as my study at: > > https://www.croatiafidelis.hr/foss/cap/cap-170313-git-devuan-mail/gi > > t- > > devuan-mail-4.php > That page has just been updated with clearer instructions. > > > (and the previous ones there, but I gave the last as it is > > simplest/fastest to check) > > > > There is information that any advanced reader can easily provide by > > retracing some of my steps there, and which would clear some > > uncertainties > here. > ... > > ... That's a serious bug or a > > serious malfunction in my Gentoo, the latter being most likely... > > > > And if it is the latter, it can only be one or the other way. One: > > the cause is in some Gentoo packge. Two: it is an attack by some > > unknown > means. > > > > ( > > If Air-Gapped is some info, I did try and editcap (and the whole > > Wireshark) behave in the same wrong way in my Air-Gapped too. > > ... > > ) > > > > > Third issue > == > > The text it too much because the command line in
Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
opened two days ago... and I don't even know if I received any replies there: ( Tab (no exec) triggers script on Bash on grsec admin https://forums.grsecurity.net/viewtopic.php?f=3&t=4700 ) ... And I don't know if I will be able to... First dhcpcd would crash on any attempt to run a bridge which I have run without any issues for months now, witness all the pages and screencasts and PCAPs at https://www.croatiafidelis.hr/foss/cap/ ( select by the timestamp, the later the better; I even got a really nice note of appreciation from Devuan devs when my analysis helped them to fix a trivial but urgent network issue on 2017-04-23 which timestamp I shorten to 170423 and so the link is: BAD sig on Devuan ISO https://www.croatiafidelis.hr/foss/cap/cap-170423-devuan-iso-sig/ )... And since this morning even plain one only ether device connection failed without any segfaults to anything or any " denied " errors... (the bridge would always get segfaults for dhcpcd). Back to the script seen in its action only. I spent hours trying to figure out what the lines of the script that does that should look like, but more hours I would need to be able to reconstruct any. I saw those entries in awk and I know sed that well, but it's more skills needed to reconstruct that script... and to hopefully locate it in the system partition dump. Thanks if anybody is able to better analyze those (and maybe help locate it). So that it be quicker at hand, I attach a gzip'ed archive of https://www.croatiafidelis.hr/foss/cap/cap-170504-strange-bash/messages_170504_2155_g0n messages_170504_2155_g0n.gz to this email as well (it's just over 1K). But I strongly believed it was a potential risk to keep running that system, and what I did is, while completely offline, I thoroughly checked the frozen clone and also the Air-Gapped (which only has the Wireshark inconsistency, and never had this Tab-triggers-Bash-script in (grsecurity RBAC) role admin). And then I updated my Air-Gapped and cloned my for-online system from it. In this system, [stop...] Haha! actually *only* in the software of this system, there are no traces that would indicate any Tab-triggers-a-script behavior, but I certainly don't know if anything was planted in my hardware... It's not Open Hardware,[5] so even if I knew how to check firware and stuff, I couldn't check much of it, let alone all of it... > -Original Message- > From: Miroslav Rovis [mailto:miro.ro...@croatiafidelis.hr] > Sent: Friday, May 05, 2017 01:02 > To: gentoo-user@lists.gentoo.org > Subject: Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance > > Hi Bobby! > > Pls. see also: > > Tab (no exec) triggers script on Bash on grsec admin > https://forums.grsecurity.net/viewtopic.php?f=3&t=4700 > > as well as the other email that I sent some 7 or so hours ago. > > NOTE: if I'm away, it's because I'm a little worried... I'm afraid my system > may be vulnerable because of these issues. Patience pls. > > (no more but only my sig in bottom) > > On 170504-21:15-0400, Bobby Kent wrote: > > Hi Miroslav, > > > > Attempting to reproduce third issue: > > > > # mkdir wibble1_1 > > # mkdir wibble2_1 > > # mkdir wibble3_1 > > # mkdir wibble4_1 > > # mkdir wibble5_1 > > # for d in wibble*_1 ; do mkdir $d/wobble ; done # ls -1d wibble*_1 > > wibble1_1 > > wibble2_1 > > wibble3_1 > > wibble4_1 > > wibble5_1 > > > > Then hit tab after positioning cursor after the / below: > > # for i in $(ls -1d wibble*_1/) ; do echo $i ; done > > > > And the results are an attempt to autocomplete: > > wibble1_1// wibble2_1// wibble3_1// wibble4_1// wibble5_1// > > > > Perhaps the test oversimplified the issue, though maybe you could > > provide the simplest way to reproduce what you see. > > > > Thanks. I do get this normal behavior that you explain above in my Air-Gapped. And generally in my cloned system. The erratic behavior that I caught a revealing glimse of was only ever happening in my clone that goes online. > > -Original Message- > > From: Miroslav Rovis [mailto:miro.ro...@croatiafidelis.hr] > > Sent: Tuesday, May 02, 2017 10:13 > > To: gentoo-user@lists.gentoo.org > > Subject: Re: [gentoo-user] Inconsistent behavior in my Gentoo OS > > instance ... > > > > Third issue > > == ... > > > [[ > > > NOTE (before delayed sending): In fact, it is only this clone that > > > exibits the above Bash malfunctioning. I just checked the same for > > > loop command (some six paragraphs above) in my Air-Gapped master [1] > > > (never any internet it sees, > > The [1] is important for understanding, espe
RE: [gentoo-user] Inconsistent behavior in my Gentoo OS instance
Apologies for conflating the Wireshark related "bug / broken package / attack" comment with the bash issue. Good luck resolving the issues. -Original Message- From: Miroslav Rovis [mailto:miro.ro...@croatiafidelis.hr] Sent: Sunday, May 07, 2017 09:42 To: gentoo-user@lists.gentoo.org Subject: Re: [gentoo-user] Inconsistent behavior in my Gentoo OS instance On 170505-22:40-0400, Bobby Kent wrote: > Looks like there are two things that concern you. Firstly, how bash > tab expansion appears to work (the ls, etc. commands executed when you > hit the tab key) on your system. Secondly, the "bash: unexpected EOF > while looking for matching `)'bash: syntax error: unexpected end of > file" messages generated when a particular tab expansion fails. > > Is that second issue generated by hitting tab at the end of the command: > > ls -1d root_170430_g0n*.d > > ? If so, perhaps there's something unusual with the items that match > the pattern "root_170430_g0n*.d*" that results in the error ... Well then there should have been somthing unusual with a plain rsync command and simple direcories in the link that I gave in other email, as I said in the mail to which the above is your reply, and which you quoted further below... > Regarding your diagnosis: > > > That's a serious bug or a serious malfunction in my Gentoo, the > > latter being most likely... > > > > And if it is the latter, it can only be one or the other way. > > One: the cause is in some Gentoo packge. > > Two: it is an attack by some unknown means. > > Before declaring But the whole paragraph originally, in the top of the thread (construing citation): > > Wireshark! Look at that! That's not a shadow. That's a serious bug > > or a serious malfunction in my Gentoo, the latter being most likely... And also in the abridged email it is under: > > Second issue > > So it refers to Wireshark only :) So pls. note that the above is not declaring it such about Bash... But I didn't modified the Bash completion. And esp. I would never modify it to be sed'ing and awk'ing on my /etc/ssh/ssh_config. ;-) ... So the above *could* apply to Bash, if I had (which I didn't) written it about Bash, but I would only word it to the level of suspicion. And suspicion it remains... > bug / broken package / attack, it might be an idea to see whether the > issue is reproducible, and under what circumstances. > > Note, tab expansion can be modified (see, for example, > http://www.linuxjournal.com/content/more-using-bash-complete-command). Which is a great link! Thanks! But again, while it could be some monkeys from space (of that kind of monkeys that write Bibles and so invent God[2], but these might be extraterrestrial monkeys, and maybe invisible, that can reach with their hands into computers without anybody realizing...). Oh, sorry for my irony. But this must have been something/someone with a purpose, that the purpose had been a prank/denial/subversion/... There is no event that can materialize out of nothing and without a cause, else physics and logic go to dusbin. And the event was pretty complex in this case. See below for the links to the script in action that I sent in the other email. And nobody expected that script to come to the fore. Thanks to Mr Linux[3], grsecurity in not widespread, and not so well known, and not even the shadows are familiar with all of its features. That script (in its action, I don't know where it resided in my machine[4]) only came to the fore because of the exec_logging feature of grsecurity-hardened kernel. Only because I had exec_logging turned on in my grsecurity-hardened kernel, I was able to show you the undeniable fact of what was executed at my hitting the Tab at that particular five or so seconds period of time in my real life. I need to remind the readers here that Bobby maybe refers here to what I gave in the other email, as I said I would (but the top posting that he uses, along with my peculiar slow and clumsy style, makes it a bit of a mess, sorry!). For my reference, see my quoted email further below, which I otherwise cut shorter. And from that other email I'm construing the links that I gave as if it was a reply, except for the links, I want them in the clear: > > Strange script planted with Bash https://www.croatiafidelis.hr/foss/cap/cap-170504-strange-bash/ > > should make for some thinking... > > It's in the logs > > ( https://www.croatiafidelis.hr/foss/cap/cap-170504-strange-bash/messages_1705 04_2155_g0n > > [link is at bottom of page, under "messages_170504_2155_g0n"] ). It has complicated further. On top of lots of time spent in analysis of my systems, I have had much difficulty connecting to the internet since I sent