Hi all, Based on the poll, it looks like 3-4 PM tomorrow works best. I've sent out a Teams meeting invite to those who responded to the poll. For anyone else who would like to join us, please use the link below.
https://teams.microsoft.com/l/meetup-join/19%3ameeting_OTE0NThjNmQtOTlmNC00MmJkLTk2MDEtNzAwODYyNzY3ZmQw%40thread.v2/0?context=%7b%22Tid%22%3a%22545921e0-10ef-4398-8713-9832ac563dad%22%2c%22Oid%22%3a%2249fff893-efa2-46e2-a36f-6fa0aa88f656%22%7d Meeting ID: 212 118 580 909 Passcode: Hn3rz649 Thanks, Nga On Thu, Nov 21, 2024 at 8:35 AM Nga Chung <nch...@apache.org> wrote: > Hi Marc, > > I've completed the poll. > > When I tested 1.4.0 on MacOS, I did not deviate from the quickstart. I > also ran https://sdap-nexus.readthedocs.io/en/1.4.0/test.html. > > Hopefully we can resolve the issues you're encountering. > > Best, > Nga > > On Wed, Nov 20, 2024 at 1:10 PM Marc A. Cruz <marcc...@cpp.edu> wrote: > >> Good afternoon! >> >> Thank you again for yesterday's meeting! Reaching out again in regard to >> setting up a follow-up meeting! Once the meeting time is confirmed, we’ll >> provide more details regarding the issues we’re encountering as well as the >> progress we’ve made. To facilitate scheduling, please use the following >> Doodle link: >> >> https://doodle.com/meeting/participate/id/e1QmZlme >> >> Regarding the issue we discussed yesterday—while all containers are >> running, Cassandra is not connecting with the Granule Ingester despite >> indications of processing. After multiple attempts to pinpoint the problem, >> we’re still coming up short. We did try local build but encountered new >> issues there as well. For context, we’re primarily focusing on MacOS for >> the time being. >> >> Lastly, a question for Nga Chung: When testing version 1.4.0, was there >> any deviation from the Quick Start guide? Were there alternative procedures >> used to test the new release, or were any regression tests conducted >> specifically for the new releases? >> >> Looking forward to your thoughts and the opportunity to connect again >> soon. >> >> Best regards, >> >> 1. Marc Cruz >> >> >> >> >> >> ------------------------------ >> *From:* Riley Kuttruff <r...@apache.org> >> *Sent:* Tuesday, November 19, 2024 3:53 PM >> *To:* Marc A. Cruz <marcc...@cpp.edu> >> *Cc:* Truc N. Nguyen <trucngu...@cpp.edu>; Riley Kuttruff <r...@apache.org>; >> dev@sdap.apache.org <dev@sdap.apache.org> >> *Subject:* Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting >> with SDAP Team >> >> You don't often get email from r...@apache.org. Learn why this is >> important <https://aka.ms/LearnAboutSenderIdentification> >> No worries. Please let us know how you progress with your work or if you >> encounter any more issues! >> >> -Riley >> >> On Tue, Nov 19, 2024 at 2:47 PM Marc A. Cruz <marcc...@cpp.edu> wrote: >> >> Good afternoon, >> >> I appreciate this morning's meeting to help troubleshoot the problems we >> had! My partner and I wanted to let you know that we cannot attend the >> community meeting today. We will continue working on this and will keep you >> posted. >> >> Thank you for your time, >> Marc Cruz >> ------------------------------ >> *From:* Truc N. Nguyen <trucngu...@cpp.edu> >> *Sent:* Monday, November 18, 2024 10:23:51 AM >> *To:* Riley Kuttruff <r...@apache.org>; Marc A. Cruz <marcc...@cpp.edu> >> *Subject:* Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting >> with SDAP Team >> >> Hi! >> >> Thank you for letting us know. Just did it. >> >> Sincerely, >> Truc Nguyen >> California State Polytechnic University, Pomona >> trucngu...@cpp.edu >> >> Get Outlook for iOS <https://aka.ms/o0ukef> >> ------------------------------ >> *From:* Riley Kuttruff <r...@apache.org> >> *Sent:* Monday, November 18, 2024 10:20:05 AM >> *To:* Truc N. Nguyen <trucngu...@cpp.edu>; Marc A. Cruz <marcc...@cpp.edu >> > >> *Subject:* Fwd: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting >> with SDAP Team >> >> You don't often get email from r...@apache.org. Learn why this is >> important <https://aka.ms/LearnAboutSenderIdentification> >> Hi, I'm forwarding this response directly since I'm not sure if you've >> subscribed to the SDAP mailing list which is what I sent my reply to >> initially. >> >> If you want to subscribe, send an email here >> <dev-subscr...@sdap.apache.org?subject=subscribe>. I don't think you >> need to put anything in the message body. You should get a reply to confirm. >> >> ---------- Forwarded message --------- >> From: Riley Kuttruff <r...@apache.org> >> Date: Mon, Nov 18, 2024 at 9:48 AM >> Subject: Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting with >> SDAP Team >> To: <dev@sdap.apache.org> >> >> >> Thanks and sorry you ran into so many issues with the quickstart - I >> wrote it a couple years ago as an intern and have only recreated the >> Cassandra/ZK/Solr/RMQ containers once (generally, you can just stop >> them and restart them no problem). I've been seeing more and more >> issues with using host.docker.internal / localhost and, given that >> using the container name as the hostname seems to be what needs to be >> done, you may wish to propose that as a change to the documentation. >> For the missing data in Jupyter: have you verified the data completed >> ingestion? You can check in the logs of the ingester containers. You >> can also open Solr admin console at localhost:8983 in your browser, >> select the nexustiles collection in the dropdown menu on the left, and >> query all tiles to verify they are there. For the documentation >> questions: the incubator-sdap-nexus RTD is obsolete. It's from when we >> were in ASF Incubator and we are unable to edit the RTD project >> subdomain so we migrated to sdap-nexus. I believe there should be a >> way to set it up to redirect, but we must have forgotten to configure >> that. Finally you do not need kubernetes to run the quickstart, though >> if you wish to try deploying to k8s, we have a Helm chart though I've >> always had to go through a bit of troubleshooting when deploying it >> fresh (that said, the k8s clusters I've deployed to were locked down >> permissions-wise). >> >> You asked about the code block of version tags: "With the code block >> of all version tags, would the given work? As we are not familiar with >> how sensitive each container is to new changes found in other >> containers." What is this in reference to? If you posted a screenshot, >> the ASF mailing lists do not support images so you'll need to paste >> plain text. Sorry if I misunderstood what you're asking there though. >> >> Finally, I'm definitely concerned about the crashes on Debian 12. I've >> never used it myself (I've really only used MacOS for running this - >> outside of k8s that is) but we should definitely be ensuring that we >> can run on common platforms. >> >> -Riley >> >> On Mon, Nov 18, 2024 at 9:26 AM Nga Chung <nch...@apache.org> wrote: >> > >> > Forwarding to dev@sdap >> > >> > ---------- Forwarded message --------- >> > From: Truc N. Nguyen <trucngu...@cpp.edu> >> > Date: Mon, Nov 18, 2024 at 12:43 AM >> > Subject: Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting >> > with SDAP Team >> > To: Nga Chung <nch...@apache.org>, Marc A. Cruz <marcc...@cpp.edu> >> > >> > >> > Good morning SDAP team, >> > >> > Thank you for your quick response! For the issues, here is the trail >> > we blazed thus far while working through the SDAP QuickStart process, >> > as well as a few questions regarding the documentation. >> > Progress and Challenges >> > >> > For myself: >> > >> > RabbitMQ issues: >> > I encountered an error (ACCESS_REFUSED - Login was refused using >> > authentication mechanism PLAIN) when trying to run the two ingester >> > containers on macOS. I resolved this by modifying RABBITMQ_HOST to rmq >> > in the granule-ingester.env file, though I’m unsure if this is the >> > recommended solution. >> > >> > Jupyter Notebook Data Issue: >> > While I successfully got the Jupyter Notebook running, I encountered a >> > problem where the data wasn’t found when running the Time Series >> > Examples, despite verifying that the data had been downloaded >> > correctly. >> > >> > For Marc, my partner: >> > >> > Initially, we attempted to use version 1.3.0 of the Docker images, but >> > documentation references version 1.2.0. Due to issues with older >> > images not being available, we proceeded with version 1.4.0. >> > Encountered Docker Desktop issues on Windows, prompting a switch to >> > macOS and Debian 12. >> > >> > For Debian 12: >> > >> > The granule ingester failed to connect to Cassandra, requiring us to >> > troubleshoot by replacing host.docker.internal to localhost ,or other >> > container-specific names (e.g., solr for History URL in connection >> > manager). >> > Updated the granule-ingester.env file as follows: >> > >> > CASSANDRA_CONTACT_POINTS=172.18.0.3 >> > CASSANDRA_USERNAME=cassandra >> > CASSANDRA_PASSWORD=cassandra >> > Here, the CASSANDRA_CONTACT_POINTS value was manually set to the >> > container’s IP address using docker inspect cassandra. >> > Created a new RabbitMQ user with admin access and set the virtual host >> to /. >> > >> > After getting granule ingester to not abort due to connection with >> > cassandra and rabbitmq. collection-manager was able to run and then >> > Debian 12 docker desktop crashed. Tried with default settings for >> > resources and also near maximum resources. Both cases of docker >> > desktop configuration crashed. We apologize that we can not provide >> > too many details of the crash as Docker Desktop premium is required to >> > determine exact causes of the crashes. >> > >> > To conclude, with all the mentioned troubleshooting, I was able to >> > complete all the steps then a crash occurs. >> > >> > >> > Questions Regarding Documentation >> > >> > For documentation, there seems to be two sdap-nexus.readthedocs.io & >> > incubator-sdap-nexus readthedocs.io. Can you confirm whether these are >> > the same or serve different purposes? >> > >> > For example, >> > https://sdap-nexus.readthedocs.io/en/1.4.0/quickstart.html >> > export COLLECTION_MANAGER_VERSION=1.4.0 >> > export GRANULE_INGESTER_VERSION=1.4.0 >> > >> > https://incubator-sdap-nexus.readthedocs.io/en/latest/quickstart.html >> > export COLLECTION_MANAGER_VERSION=1.2.0 >> > export GRANULE_INGESTER_VERSION=1.2.0 >> > >> > With the code block of all version tags, would the given work? As we >> > are not familiar with how sensitive each container is to new changes >> > found in other containers. >> > >> > Was Kubernetes a pre-requisite to run docker QuickStart? >> > >> > >> > Thank you for the help and support, we also filled out the doodle for >> > meeting time so we're looking forward to seeing everyone! >> > >> > Thank you for your time. >> > >> > Sincerely, >> > Truc Nguyen >> > California State Polytechnic University, Pomona >> > trucngu...@cpp.edu >> > ________________________________ >> > From: Nga Chung <nch...@apache.org> >> > Sent: Sunday, November 17, 2024 9:11 PM >> > To: Truc N. Nguyen <trucngu...@cpp.edu>; Marc A. Cruz <marcc...@cpp.edu >> > >> > Subject: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting with >> SDAP Team >> > >> > [You don't often get email from nch...@apache.org. Learn why this is >> > important at https://aka.ms/LearnAboutSenderIdentification ] >> > >> > Third time's the charm... Sorry by default when we reply to dev@ >> > mailing list it only includes dev@. >> > >> > Looking forward to speaking to you both. >> > >> > Best, >> > Nga >> > >> > On Sun, Nov 17, 2024 at 9:09 PM Nga Chung <nch...@apache.org> wrote: >> > > >> > > Sorry, forget to reply all! Truc, please see my message below. >> > > >> > > Thanks, >> > > Nga >> > > >> > > >> > > >> > > On Sun, Nov 17, 2024 at 9:07 PM Nga Chung <nch...@apache.org> wrote: >> > > > >> > > > Hi Truc, >> > > > >> > > > We'd be happy to meet with you. As Riley mentioned, it would be >> great >> > > > if you can please provide a brief description of the challenges, so >> we >> > > > can come to the meeting prepared. >> > > > >> > > > I've also created a doodle poll, so for those who would like to join >> > > > us, please fill out the poll and we will find a time that works for >> > > > everyone. >> > > > >> > > > Link to poll: >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoodle.com%2Fmeeting%2Fparticipate%2Fid%2FaQvQZAlb&data=05%7C02%7Ctrucnguyen%40cpp.edu%7Cf93cdbddbf854256e15608dd078f846b%7C164ba61e39ec4f5d89ffaa1f00a521b4%7C0%7C0%7C638675035179090142%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=JD0kfSP6L3poS3Vl1FbfJC4d0SisdY9lKjp60sWBL8U%3D&reserved=0 >> <https://doodle.com/meeting/participate/id/aQvQZAlb> >> > > > >> > > > Thanks, >> > > > Nga >> > > > >> > > > On Fri, Nov 15, 2024 at 1:06 PM Riley Kuttruff <r...@apache.org> >> wrote: >> > > > > >> > > > > Hi! >> > > > > >> > > > > I'd be happy to help in any way I can. Could you perhaps provide >> a brief >> > > > > description of the technical challenges you've been facing? Just >> so I can >> > > > > have a little time to prepare to make the most of our time. >> > > > > >> > > > > I'm happy to meet next week though my calendar is a bit full so >> we may need >> > > > > to iterate a little to find a suitable time. What times work for >> you? >> > > > > >> > > > > Also, I think @nchung may also wish to attend. >> > > > > >> > > > > -Riley >> > > > > >> > > > > On Fri, Nov 15, 2024, 12:52 PM Truc N. Nguyen <trucngu...@cpp.edu> >> wrote: >> > > > > >> > > > > > Good afternoon, SDAP Team, >> > > > > > >> > > > > > My partner and I are students in Dr. Thomas Huang’s Parallel >> Processing >> > > > > > class, and we are working on a project about Apache SDAP ETL >> pipeline using >> > > > > > Apache Airflow as part of our class presentation. >> > > > > > >> > > > > > Dr. Huang recommended reaching out to your team for guidance. >> We are >> > > > > > currently facing some technical challenges and would greatly >> appreciate any >> > > > > > support or advice you could provide. >> > > > > > >> > > > > > Would it be possible to schedule a meeting with your team >> sometime next >> > > > > > week? We are also aware of your monthly meeting scheduled for >> next Tuesday, >> > > > > > and while we’re unsure if we can attend, we will make an effort >> to join if >> > > > > > it's possible. >> > > > > > >> > > > > > Thank you for your time and look forward to hearing from you >> soon. >> > > > > > >> > > > > > Sincerely, >> > > > > > Truc Nguyen >> > > > > > California State Polytechnic University, Pomona >> > > > > > trucngu...@cpp.edu >> > > > > > >> > CAUTION: This email was not sent from a Cal Poly Pomona service. >> > Exercise caution when clicking links or opening attachments. Please >> > forward suspicious email to >> > suspectem...@cpp.edu<mailto:suspectem...@cpp.edu>. >> >> *CAUTION:* This email was not sent from a Cal Poly Pomona service. >> Exercise caution when clicking links or opening attachments. Please forward >> suspicious email to suspectem...@cpp.edu. >> >> >> *CAUTION:* This email was not sent from a Cal Poly Pomona service. >> Exercise caution when clicking links or opening attachments. Please forward >> suspicious email to suspectem...@cpp.edu. >> >> >>
