Re: Perpetual Data (SL6 ssh fail)
On Tue, Jan 10, 2023 at 6:52 PM Konstantin Olchanski wrote: > > Keith, thanks. In practice, experiment data is not "perpetual", > life time of typical physics experiment is about 5-10 years after > the last data taking. Afterwards, people tend to dissipate (retire, > graduate and move on, just move on, lose interest, join different > project, etc). after the last person who was involved with > the experiment is "gone", experiment data is "orphaned" and > nothing much can be done with it, even if the physical media > and the data format is still readable and if the analysis > software still can be compiled and still runs. I can vouch for the difficulty of access to old data, and of preserving data from unique experiments. Updates and recoveries of old data have paid quite a bit of my wages, for years. And getting data out of people's proprietized setups, with their own internalized databases? And especially the CTO devised custom source control systems? And data stored on media that can no longer be read? I helped an astronomer friend recover data about a unique occultation of Pluto from mag tape that he could not find a reader for *anywhere* at MIT or Harvard: my friends in the lab next door still had one, and we used to recover data before it was decommissioned. And I remember transferring human experimental data to Exabyte, *and* CD, to make sure it would be recoverable for a dozen more years because you can't necessarily repeat data involving human nerves. > That said, CERN support the "open data" approach, where general public > has access to experiment data. But not to raw data, without calibrations > and interpretations, raw data is dangerous, john q. public can easily > "analyse it wrong". this is a hot debate topic in the physics community. > https://opendata.cern.ch/ Thanks for the pointer. I might poke my nose in. And yeah, raw data without information about the rest of the experimental setup can be... misleading. Meticulous data, collected with careful observation, is worth its weight in anti-matter. At last look, that's about 62 trillion USD/gram? > Re the TRIUMF photo, that's not my scope, I have a much newer one, but > for some tasks, old scopes are better. > > The photo is taken in the walkway of the meson hall. Cyclotron is dead ahead. > On the left side, attached to the He dewar is the brand new > He compressor (makes liquid He: in Canada, we recycle!). Further left beyound > the He compressor > is the experimental hall with the cold neutron (0K cold, not just cold) > machine > and the muon beam lines for materials science experiments (MuSR). Outside > of the picture (dead ahead behind the cyclotron) is the brand new state > of the art e- accelerator. To the right (in a different building) is > the unstable atom production (transmutation) machine (no, not lead to gold, > we are not CERN!) and experiments that study these rare unstable atoms. Dead > behind > is a small cyclotron, it makes unstable atoms for medical use (radioactive > tracers > for medical imaging). This concludes your virtual tour of TRIUMF. You are > welcome. > > > K.O. Thanks. Sometimes it's nice to hear what folks have been up to.
Re: Perpetual Data (SL6 ssh fail)
If you want to see what is involved in preserving old particle physics data, read the story of how JADE data from the 1980s was re-analysed in the 2000s thanks to incredible efforts by a few collaboration members. https://urldefense.proofpoint.com/v2/url?u=https-3A__link.springer.com_article_10.1140_epjh_s13129-2D022-2D00047-2D8&d=DwICaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=IJVw5OIiG01FIuxjjKcB_GyLlKhvuX90s8YJSsTEr9ca9o40pWm_FkEOTzQZs2--&s=_AiIVOkjHoA93uPoKhwpdlesDaN771GrfFyNuAviQsc&e= https://urldefense.proofpoint.com/v2/url?u=https-3A__www.mpp.mpg.de_en_research_data-2Dpreservation_jade&d=DwICaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=IJVw5OIiG01FIuxjjKcB_GyLlKhvuX90s8YJSsTEr9ca9o40pWm_FkEOTzQZs2--&s=XRUWasVEN5VuhnCtVGEJIV8oL-YN98J8rk-_HM9HbQ8&e= Alex On 10/01/2023 19:49, Keith Lofstrom wrote: From: Konstantin Olchanski Subject: Re: SL6 ssh fail ... It looks like my remaining option is to build openssh from OpenBSD "portable" sources. ... - "so old" - like a grand-father's axe, most our SL6 machines hardware was upgraded 2-3 times by now, they run from SSDs on DDR3/DDR4 RAM machines. - exception is VME processors I'm on Konstantin's side here - although it is a side many light-years wide, with MANY of us spread thinly across it. While I do not have my grandfather's axe, I still use my great-grandfather's carpentry toolbox, which my grandfather brought from Sweden in 1911 (I also have my grandfather's steamship ticket, and his Swedish-to-English dictionary). https://urldefense.proofpoint.com/v2/url?u=https-3A__eur02.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttp-2D3A-5F-5Fwiki.keithl.com-5FJohanSigfridLofstrom-2526d-253DDwIBAg-2526c-253DgRgGjJ3BkIsb5y6s49QqsA-2526r-253Dgd8BzeSQcySVxr0gDWSEbN-2DP-2DpgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A-2526m-253DiqT8zmlP56N56Jq9YP-5Fa6cjE90PVa3LlHNdlKR14LBh4UY7CFKqQzSC6tQwZud2d-2526s-253D-5FbHbAaGb3b436-2DGEoRYnWCwPRLp6V7b-5FtiSALqhmBzY-2526e-253D-26data-3D05-257C01-257Cfincha-2540live.lancs.ac.uk-257Cf9e8f61e32044bf0af9a08daf3449569-257C9c9bcd11977a4e9ca9a0bc734090164a-257C0-257C0-257C638089773137567708-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26sdata-3DLM0VUaRN5QZVD-252FkJ8rIyrE-252F-252FXK8BCBONrMx4DM6n-252BOY-253D-26reserved-3D0&d=DwICaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=IJVw5OIiG01FIuxjjKcB_GyLlKhvuX90s8YJSsTEr9ca9o40pWm_FkEOTzQZs2--&s=-ekk06zgRkwYDqOinEGNEjW8st5aRlsUt6TaHT_efME&e= I use those tools to build the gizmos that help me imagine space technology evolution into the 22nd century (and read emails from my Swedish fourth-cousins). Science has plucked almost all of the low-hanging fruit; future discovery lies in subtle manipulations of vast amounts of both new and archived measurements made by vast amounts of hardware accumulated over many decades. The huge problem with archived measurements is their origin in imperfect and evolving hardware, software, procedures, theories, and people. Those inputs color the data; new data collected with new hardware, software, etc. can be incommensurate with old data. This is a good reason for keeping the old hardware/software sets alive, so you can measure twice, with your great-grandfather's ruler and with your laser interferometer, and cross-calibrate the data taken both ways. Konstantin contributes to TRIUMF, Canada's premiere particle accelerator. I am amused that the photo associated with the TRIUMF Wikipedia page shows a Tektronix oscilloscope designed in the 1960s: https://urldefense.proofpoint.com/v2/url?u=https-3A__eur02.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Furldefense.proofpoint.com-252Fv2-252Furl-253Fu-253Dhttp-2D3A-5F-5Fen.wikipedia.org-5Fwiki-5FTRIUMF-2D23-5Fmedia-5FFile-2D3ACanadian-2D5FScience-2D5F-2D2D-2D5FTRIUMF-2D5Fcyclotron-2D5F-2D2D-2D5FFlickr-2D5F-2D2D-2D5FCargo-2D5FCult-2D5F-2D2821-2D29.jpg-2526d-253DDwIBAg-2526c-253DgRgGjJ3BkIsb5y6s49QqsA-2526r-253Dgd8BzeSQcySVxr0gDWSEbN-2DP-2DpgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A-2526m-253DiqT8zmlP56N56Jq9YP-5Fa6cjE90PVa3LlHNdlKR14LBh4UY7CFKqQzSC6tQwZud2d-2526s-253DVoPCz-5FdAeUSdH6dEptF53yurEpghrR-2DJZvyRjGJ0Sj0-2526e-253D-26data-3D05-257C01-257Cfincha-2540live.lancs.ac.uk-257Cf9e8f61e32044bf0af9a08daf3449569-257C9c9bcd11977a4e9ca9a0bc734090164a-257C0-257C0-257C638089773137567708-257CUnknown-257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0-253D-257C3000-257C-257C-257C-26sdata-3DiCg8AGbKud1aTD5W-252FBA2x-252Ba3UvY6lPAN9fPASBxX8I4-253D-26reserved-3D0&d=DwICaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=IJVw5OIiG01FIuxjjKcB_GyLlKhvuX90s8YJSsTEr9ca9o40pWm_FkEOTzQZs2--&s=iG753yvG6UG1NkdZOdgI42JK-GBTHivNURXzvfkaxbY&e=
Re: Perpetual Data (SL6 ssh fail)
On Tue, Jan 10, 2023 at 04:06:53PM -0500, Larry Linder wrote: > We used SL from 4.1 ... We started much earlier, 1998 or so, Red Hat Linux installed from floppy disks (soon from CDs). Choice of Debian vs Red Hat was settled by the very nice Red Hat graphical installer (featuring the sadly-now-defunct "install everything" button). > On of our dislikes is systemd and package manager. To give "them" credit, Ubuntu LTS 20 and 22 systemd is mostly problem free, it does try to take over most system functions (cron, NTP, DNS, etc) but does a 50% job at that (it works, but config files are bad and diagnostics are non-existant). > >From an industrial vantage point we really miss SL 5.11 and 6.5. I would say those were the best vintage of Linux. Since then, only one big improvement (ZFS) and many degradations. unfriendly UIs, systemd, grub, network manager, disfunctional boot system - "please shutdown! no! I must wait for all users to logout and all NFSes to unmount! (no matter that we are in the middle of a power outage and network is already dead). > We have basiacally abandoned RH and its following. They seam to pick up > the worst ideas and go with it. I feel opposite. RH abandoned us. They defaced linux with all kinds of laptop-centric crud (wayland, network manager, containers that do not work on NFS, etc) and then went "enterprise" where all this laptop-centric stuff only gets in the way. > Our security is pretty simple. - don't laugh too hard. We cron to turn > off network after 5:30 and turn it back on at 7 AM. good one! too bad we are a 24/7 shop... > Odd Notes. We hade two Toshiba high end laptops we were ready to junk. > After looking at them in detail. We replace the slow 125G HD with a 1 T SSD. So true, lots of old machines get a new life just be replacing SATA HDD with SATA SSD, unfortunately RAM limits (2G max, 4 GB max, 8 GB max) and limited CPU GHz takes their toll... > One of the real sad thing that has happened to our Engineering Comunity > is the during the China Virus shut down the students didn't learn much. So true. My favourite, I point physics student at a wall power outlet and ask "which plug is plus, which plug is minus?". You try it! > They are pretty good a running windows but they don't know how anything > works or how to connect a scope etc. The isolation is a disaster for > the new science / engineers who worked remotely. My contribution is > that we hire one or two bright students and teach them the real world > applications. TRIUMF has a big COOP and Summer student program. Lots of kids come through and learn how to use scope, screw driver, soldering iron, etc xttps://www.triumf.ca/academic-programs/undergraduate-program/coop-education > I encourage each when they are ready to explore other > oportunities. So far we have had 14 young men and ladies find their > lifes endever. The comments I get from companies is how does a student > get 4 years of engineering experience when they get their Deploma. > When I worked at GE one of my jobs was to train new engineers in the art > of digital control. The digital engine control (gray box) on side of > the CFM56 engines worked well and was certified in 1987. CFM56, legendary. Best the new generation could do is build and certify the 737-MAX airplane with a "fly down, down, down" feature in the control system (MCAS). Redundant sensors? Input sanity checks? Never heard of him! Discovered and fixed the hard way. K.O. > Another condecending Unix / Linux user. > By the way my great,great,great grandfather was a Hessian solder and > decided he like the country, jumped ship, married a lady and had 12 > Children. > > Larry Linder > > > > On Tue, 2023-01-10 at 11:49 -0800, Keith Lofstrom wrote: > > > From: Konstantin Olchanski > > > Subject: Re: SL6 ssh fail > > ... > > > It looks like my remaining option is to build openssh from OpenBSD > > > "portable" sources. > > ... > > > - "so old" - like a grand-father's axe, most our SL6 machines hardware > > > was upgraded 2-3 times by now, they run from SSDs on DDR3/DDR4 RAM > > > machines. > > > - exception is VME processors > > > > I'm on Konstantin's side here - although it is a side many > > light-years wide, with MANY of us spread thinly across it. > > > > While I do not have my grandfather's axe, I still use my > > great-grandfather's carpentry toolbox, which my grandfather > > brought from Sweden in 1911 (I also have my grandfather's > > steamship ticket, and his Swedish-to-English dictionary). > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__wiki.keithl.com_JohanSigfridLofstrom&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=iqT8zmlP56N56Jq9YP_a6cjE90PVa3LlHNdlKR14LBh4UY7CFKqQzSC6tQwZud2d&s=_bHbAaGb3b436-GEoRYnWCwPRLp6V7b_tiSALqhmBzY&e= > > > > > > I use those tools to build the gizmos that help me imagine > > space technology evolution into the 22nd century (and read > > emails f
Re: Perpetual Data (SL6 ssh fail)
Keith, thanks. In practice, experiment data is not "perpetual", life time of typical physics experiment is about 5-10 years after the last data taking. Afterwards, people tend to dissipate (retire, graduate and move on, just move on, lose interest, join different project, etc). after the last person who was involved with the experiment is "gone", experiment data is "orphaned" and nothing much can be done with it, even if the physical media and the data format is still readable and if the analysis software still can be compiled and still runs. That said, CERN support the "open data" approach, where general public has access to experiment data. But not to raw data, without calibrations and interpretations, raw data is dangerous, john q. public can easily "analyse it wrong". this is a hot debate topic in the physics community. https://opendata.cern.ch/ Re the TRIUMF photo, that's not my scope, I have a much newer one, but for some tasks, old scopes are better. The photo is taken in the walkway of the meson hall. Cyclotron is dead ahead. On the left side, attached to the He dewar is the brand new He compressor (makes liquid He: in Canada, we recycle!). Further left beyound the He compressor is the experimental hall with the cold neutron (0K cold, not just cold) machine and the muon beam lines for materials science experiments (MuSR). Outside of the picture (dead ahead behind the cyclotron) is the brand new state of the art e- accelerator. To the right (in a different building) is the unstable atom production (transmutation) machine (no, not lead to gold, we are not CERN!) and experiments that study these rare unstable atoms. Dead behind is a small cyclotron, it makes unstable atoms for medical use (radioactive tracers for medical imaging). This concludes your virtual tour of TRIUMF. You are welcome. K.O. On Tue, Jan 10, 2023 at 11:49:49AM -0800, Keith Lofstrom wrote: > > From: Konstantin Olchanski > > Subject: Re: SL6 ssh fail > ... > > It looks like my remaining option is to build openssh from OpenBSD > > "portable" sources. > ... > > - "so old" - like a grand-father's axe, most our SL6 machines hardware was > > upgraded 2-3 times by now, they run from SSDs on DDR3/DDR4 RAM machines. > > - exception is VME processors > > I'm on Konstantin's side here - although it is a side many > light-years wide, with MANY of us spread thinly across it. > > While I do not have my grandfather's axe, I still use my > great-grandfather's carpentry toolbox, which my grandfather > brought from Sweden in 1911 (I also have my grandfather's > steamship ticket, and his Swedish-to-English dictionary). > > https://urldefense.proofpoint.com/v2/url?u=http-3A__wiki.keithl.com_JohanSigfridLofstrom&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=iqT8zmlP56N56Jq9YP_a6cjE90PVa3LlHNdlKR14LBh4UY7CFKqQzSC6tQwZud2d&s=_bHbAaGb3b436-GEoRYnWCwPRLp6V7b_tiSALqhmBzY&e= > > > I use those tools to build the gizmos that help me imagine > space technology evolution into the 22nd century (and read > emails from my Swedish fourth-cousins). > > Science has plucked almost all of the low-hanging fruit; > future discovery lies in subtle manipulations of vast > amounts of both new and archived measurements made by > vast amounts of hardware accumulated over many decades. > > The huge problem with archived measurements is their origin > in imperfect and evolving hardware, software, procedures, > theories, and people. Those inputs color the data; new > data collected with new hardware, software, etc. can be > incommensurate with old data. This is a good reason for > keeping the old hardware/software sets alive, so you can > measure twice, with your great-grandfather's ruler and > with your laser interferometer, and cross-calibrate the > data taken both ways. > > Konstantin contributes to TRIUMF, Canada's premiere > particle accelerator. I am amused that the photo associated > with the TRIUMF Wikipedia page shows a Tektronix oscilloscope > designed in the 1960s: > > https://urldefense.proofpoint.com/v2/url?u=http-3A__en.wikipedia.org_wiki_TRIUMF-23_media_File-3ACanadian-5FScience-5F-2D-5FTRIUMF-5Fcyclotron-5F-2D-5FFlickr-5F-2D-5FCargo-5FCult-5F-2821-29.jpg&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=iqT8zmlP56N56Jq9YP_a6cjE90PVa3LlHNdlKR14LBh4UY7CFKqQzSC6tQwZud2d&s=VoPCz_dAeUSdH6dEptF53yurEpghrR-JZvyRjGJ0Sj0&e= > > > Also a large pipe and a huge dewar labeled "HELIUM", which > will probably be all used up and dissipated to outer space > by 2160. Data measured with instruments consuming large > amounts of helium may be non-repeatable in 2160. > Yet somehow, data wranglers like Konstantin must "pay data > forward" so that 2160 scientists can evaluate 2023 data > (and 1968 data, TRIUMF's founding) in an accurate context. > > > > I began using Scientific Linux because I assumed that > Fermilabs would maintain it
Re: Perpetual Data (SL6 ssh fail)
my two cents worth. We used SL from 4.1 and still have a number of boxes with SL 6.5. There is still a 7.6 running and never updated as the update killed the ability to connect with our network. On of our dislikes is system d and package manager. RH 8 - none of our cad tools worked. A bright spot in the Linux mess is Devuan. A non system D Debian. It works: All of our cad tools work and you can use apt-get and use most Debian applications. Several boxes have gone a year with out a reboot and were shut down for a clean up and restarted. The installation is pretty easy. The only real down side is the add users and groups function is retarded. I told them to just copy SL 6.5 add users and get on with the show. There is a lot of things 6.5 did well but are not in Devuan. >From an industrial vantage point we really miss SL 5.11 and 6.5. We have basiacally abandoned RH and its following. They seam to pick up the worst ideas and go with it. Our security is pretty simple. - don't laugh too hard. We cron to turn off network after 5:30 and turn it back on at 7 AM. This reduces the time hackers have to pound on it. The other thing is that your IP can be long lived unless you reboot your router. We reboot the router before 7 AM and get a new IP. For industrial use Devuan is a good answere. Odd Notes. We hade two Toshiba high end laptops we were ready to junk. After looking at them in detail. We replace the slow 125G HD with a 1 T SSD. Installed the 386 version of Devuan and we have two new laptops. Performance is outstanding. Not like my 8 core machine with 32G ram undr my desk but the performace is acceptable. Compared to the Windows Dog slow boot it is amazing. Everything works. Save a few K bucks. For a couple of CNC machines in the shop we need Windows. Installed VMWare and loaded Windows 10 and you can almost get your second cup of tea while it is booting. Enjoy the conversation and I really miss SL 6.5 but you have to move on or quit. So here I am 80 yeas old and still working. Get up at 5:30, go to the life maintence program at the local hospital work out for an hr. run (fast old folks shuff) for 10 laps, lift 15 in each hand and 120 reps, drive home get fed and work the next 12 hr. The SW development keeps the neurons connected. One of the real sad thing that has happened to our Engineering Comunity is the during the China Virus shut down the students didn't learn much. They are pretty good a running windows but they don't know how anything works or how to connect a scope etc. The isolation is a disaster for the new science / engineers who worked remotely. My contribution is that we hire one or two bright students and teach them the real world applications. I encourage each when they are ready to explore other oportunities. So far we have had 14 young men and ladies find their lifes endever. The comments I get from companies is how does a student get 4 years of engineering experience when they get their Deploma. When I worked at GE one of my jobs was to train new engineers in the art of digital control. The digital engine control (gray box) on side of the CFM56 engines worked well and was certified in 1987. Another condecending Unix / Linux user. By the way my great,great,great grandfather was a Hessian solder and decided he like the country, jumped ship, married a lady and had 12 Children. Larry Linder On Tue, 2023-01-10 at 11:49 -0800, Keith Lofstrom wrote: > > From: Konstantin Olchanski > > Subject: Re: SL6 ssh fail > ... > > It looks like my remaining option is to build openssh from OpenBSD > > "portable" sources. > ... > > - "so old" - like a grand-father's axe, most our SL6 machines hardware was > > upgraded 2-3 times by now, they run from SSDs on DDR3/DDR4 RAM machines. > > - exception is VME processors > > I'm on Konstantin's side here - although it is a side many > light-years wide, with MANY of us spread thinly across it. > > While I do not have my grandfather's axe, I still use my > great-grandfather's carpentry toolbox, which my grandfather > brought from Sweden in 1911 (I also have my grandfather's > steamship ticket, and his Swedish-to-English dictionary). > > https://urldefense.proofpoint.com/v2/url?u=http-3A__wiki.keithl.com_JohanSigfridLofstrom&d=DwIBAg&c=gRgGjJ3BkIsb5y6s49QqsA&r=gd8BzeSQcySVxr0gDWSEbN-P-pgDXkdyCtaMqdCgPPdW1cyL5RIpaIYrCn8C5x2A&m=iqT8zmlP56N56Jq9YP_a6cjE90PVa3LlHNdlKR14LBh4UY7CFKqQzSC6tQwZud2d&s=_bHbAaGb3b436-GEoRYnWCwPRLp6V7b_tiSALqhmBzY&e= > > > I use those tools to build the gizmos that help me imagine > space technology evolution into the 22nd century (and read > emails from my Swedish fourth-cousins). > > Science has plucked almost all of the low-hanging fruit; > future discovery lies in subtle manipulations of vast > amounts of both new and archived measurements made by > vast amounts of hardware accumulated over many decades. > > The huge problem with archived measurements is their origin > in imperfect and evolving
