Hey Marc, I don't think you can use attachments for messages sent to ASF mailing lists, hence the bounce from dev@sdap.apache.org. But since you included our direct [at]apache.org emails it seems to have gone through to me and Nga.
-Riley On Fri, Nov 22, 2024 at 12:20 PM Marc A. Cruz <marcc...@cpp.edu> wrote: > Good afternoon, > > Earlier we sent an email at 12:04 PM and wanted to double check if it went > through as MAILER-DAEMON seemed to not be able to deliver our email. I > assume this is due to the zip file I attached. Please let us know if it was > not received to your emails Riley and Nga. Then where to send a zip file > containing docker logs and inspect details. > > Best, > Marc > ------------------------------ > *From:* Marc A. Cruz <marcc...@cpp.edu> > *Sent:* Friday, November 22, 2024 12:04 PM > *To:* Nga Chung <nch...@apache.org>; Riley Kuttruff <r...@apache.org> > *Cc:* Truc N. Nguyen <trucngu...@cpp.edu>; dev@sdap.apache.org < > dev@sdap.apache.org> > *Subject:* Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting > with SDAP Team > > Good morning SDAP team, > > Thank you, Nga, for setting up the Teams meeting—it’s greatly appreciated! > > To help make today’s meeting more effective, I’ve attached a zip file > containing documentation on progress for macOS and Linux (Debian 12). The > zip also includes a folder of .txt files detailing Linux attempts & MacOS > attempts. I’d like to suggest prioritizing the discussion and > troubleshooting of the macOS issues we’ve encountered at the start of the > meeting. > > Thank you for taking the time to meet with us today! > > Best regards, > Marc Cruz > ------------------------------ > *From:* Nga Chung <nch...@apache.org> > *Sent:* Thursday, November 21, 2024 12:40 PM > *To:* Marc A. Cruz <marcc...@cpp.edu> > *Cc:* Riley Kuttruff <r...@apache.org>; Truc N. Nguyen <trucngu...@cpp.edu>; > dev@sdap.apache.org <dev@sdap.apache.org> > *Subject:* Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting > with SDAP Team > > You don't often get email from nch...@apache.org. Learn why this is > important <https://aka.ms/LearnAboutSenderIdentification> > Hi all, > > Based on the poll, it looks like 3-4 PM tomorrow works best. I've sent out > a Teams meeting invite to those who responded to the poll. For anyone else > who would like to join us, please use the link below. > > > https://teams.microsoft.com/l/meetup-join/19%3ameeting_OTE0NThjNmQtOTlmNC00MmJkLTk2MDEtNzAwODYyNzY3ZmQw%40thread.v2/0?context=%7b%22Tid%22%3a%22545921e0-10ef-4398-8713-9832ac563dad%22%2c%22Oid%22%3a%2249fff893-efa2-46e2-a36f-6fa0aa88f656%22%7d > Meeting ID: 212 118 580 909 > Passcode: Hn3rz649 > > Thanks, > Nga > > On Thu, Nov 21, 2024 at 8:35 AM Nga Chung <nch...@apache.org> wrote: > > Hi Marc, > > I've completed the poll. > > When I tested 1.4.0 on MacOS, I did not deviate from the quickstart. I > also ran https://sdap-nexus.readthedocs.io/en/1.4.0/test.html. > > Hopefully we can resolve the issues you're encountering. > > Best, > Nga > > On Wed, Nov 20, 2024 at 1:10 PM Marc A. Cruz <marcc...@cpp.edu> wrote: > > Good afternoon! > > Thank you again for yesterday's meeting! Reaching out again in regard to > setting up a follow-up meeting! Once the meeting time is confirmed, we’ll > provide more details regarding the issues we’re encountering as well as the > progress we’ve made. To facilitate scheduling, please use the following > Doodle link: > > https://doodle.com/meeting/participate/id/e1QmZlme > > Regarding the issue we discussed yesterday—while all containers are > running, Cassandra is not connecting with the Granule Ingester despite > indications of processing. After multiple attempts to pinpoint the problem, > we’re still coming up short. We did try local build but encountered new > issues there as well. For context, we’re primarily focusing on MacOS for > the time being. > > Lastly, a question for Nga Chung: When testing version 1.4.0, was there > any deviation from the Quick Start guide? Were there alternative procedures > used to test the new release, or were any regression tests conducted > specifically for the new releases? > > Looking forward to your thoughts and the opportunity to connect again soon. > > Best regards, > > 1. Marc Cruz > > > > > > ------------------------------ > *From:* Riley Kuttruff <r...@apache.org> > *Sent:* Tuesday, November 19, 2024 3:53 PM > *To:* Marc A. Cruz <marcc...@cpp.edu> > *Cc:* Truc N. Nguyen <trucngu...@cpp.edu>; Riley Kuttruff <r...@apache.org>; > dev@sdap.apache.org <dev@sdap.apache.org> > *Subject:* Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting > with SDAP Team > > You don't often get email from r...@apache.org. Learn why this is important > <https://aka.ms/LearnAboutSenderIdentification> > No worries. Please let us know how you progress with your work or if you > encounter any more issues! > > -Riley > > On Tue, Nov 19, 2024 at 2:47 PM Marc A. Cruz <marcc...@cpp.edu> wrote: > > Good afternoon, > > I appreciate this morning's meeting to help troubleshoot the problems we > had! My partner and I wanted to let you know that we cannot attend the > community meeting today. We will continue working on this and will keep you > posted. > > Thank you for your time, > Marc Cruz > ------------------------------ > *From:* Truc N. Nguyen <trucngu...@cpp.edu> > *Sent:* Monday, November 18, 2024 10:23:51 AM > *To:* Riley Kuttruff <r...@apache.org>; Marc A. Cruz <marcc...@cpp.edu> > *Subject:* Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting > with SDAP Team > > Hi! > > Thank you for letting us know. Just did it. > > Sincerely, > Truc Nguyen > California State Polytechnic University, Pomona > trucngu...@cpp.edu > > Get Outlook for iOS <https://aka.ms/o0ukef> > ------------------------------ > *From:* Riley Kuttruff <r...@apache.org> > *Sent:* Monday, November 18, 2024 10:20:05 AM > *To:* Truc N. Nguyen <trucngu...@cpp.edu>; Marc A. Cruz <marcc...@cpp.edu> > *Subject:* Fwd: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting > with SDAP Team > > You don't often get email from r...@apache.org. Learn why this is important > <https://aka.ms/LearnAboutSenderIdentification> > Hi, I'm forwarding this response directly since I'm not sure if you've > subscribed to the SDAP mailing list which is what I sent my reply to > initially. > > If you want to subscribe, send an email here > <dev-subscr...@sdap.apache.org?subject=subscribe>. I don't think you need > to put anything in the message body. You should get a reply to confirm. > > ---------- Forwarded message --------- > From: Riley Kuttruff <r...@apache.org> > Date: Mon, Nov 18, 2024 at 9:48 AM > Subject: Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting with > SDAP Team > To: <dev@sdap.apache.org> > > > Thanks and sorry you ran into so many issues with the quickstart - I > wrote it a couple years ago as an intern and have only recreated the > Cassandra/ZK/Solr/RMQ containers once (generally, you can just stop > them and restart them no problem). I've been seeing more and more > issues with using host.docker.internal / localhost and, given that > using the container name as the hostname seems to be what needs to be > done, you may wish to propose that as a change to the documentation. > For the missing data in Jupyter: have you verified the data completed > ingestion? You can check in the logs of the ingester containers. You > can also open Solr admin console at localhost:8983 in your browser, > select the nexustiles collection in the dropdown menu on the left, and > query all tiles to verify they are there. For the documentation > questions: the incubator-sdap-nexus RTD is obsolete. It's from when we > were in ASF Incubator and we are unable to edit the RTD project > subdomain so we migrated to sdap-nexus. I believe there should be a > way to set it up to redirect, but we must have forgotten to configure > that. Finally you do not need kubernetes to run the quickstart, though > if you wish to try deploying to k8s, we have a Helm chart though I've > always had to go through a bit of troubleshooting when deploying it > fresh (that said, the k8s clusters I've deployed to were locked down > permissions-wise). > > You asked about the code block of version tags: "With the code block > of all version tags, would the given work? As we are not familiar with > how sensitive each container is to new changes found in other > containers." What is this in reference to? If you posted a screenshot, > the ASF mailing lists do not support images so you'll need to paste > plain text. Sorry if I misunderstood what you're asking there though. > > Finally, I'm definitely concerned about the crashes on Debian 12. I've > never used it myself (I've really only used MacOS for running this - > outside of k8s that is) but we should definitely be ensuring that we > can run on common platforms. > > -Riley > > On Mon, Nov 18, 2024 at 9:26 AM Nga Chung <nch...@apache.org> wrote: > > > > Forwarding to dev@sdap > > > > ---------- Forwarded message --------- > > From: Truc N. Nguyen <trucngu...@cpp.edu> > > Date: Mon, Nov 18, 2024 at 12:43 AM > > Subject: Re: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting > > with SDAP Team > > To: Nga Chung <nch...@apache.org>, Marc A. Cruz <marcc...@cpp.edu> > > > > > > Good morning SDAP team, > > > > Thank you for your quick response! For the issues, here is the trail > > we blazed thus far while working through the SDAP QuickStart process, > > as well as a few questions regarding the documentation. > > Progress and Challenges > > > > For myself: > > > > RabbitMQ issues: > > I encountered an error (ACCESS_REFUSED - Login was refused using > > authentication mechanism PLAIN) when trying to run the two ingester > > containers on macOS. I resolved this by modifying RABBITMQ_HOST to rmq > > in the granule-ingester.env file, though I’m unsure if this is the > > recommended solution. > > > > Jupyter Notebook Data Issue: > > While I successfully got the Jupyter Notebook running, I encountered a > > problem where the data wasn’t found when running the Time Series > > Examples, despite verifying that the data had been downloaded > > correctly. > > > > For Marc, my partner: > > > > Initially, we attempted to use version 1.3.0 of the Docker images, but > > documentation references version 1.2.0. Due to issues with older > > images not being available, we proceeded with version 1.4.0. > > Encountered Docker Desktop issues on Windows, prompting a switch to > > macOS and Debian 12. > > > > For Debian 12: > > > > The granule ingester failed to connect to Cassandra, requiring us to > > troubleshoot by replacing host.docker.internal to localhost ,or other > > container-specific names (e.g., solr for History URL in connection > > manager). > > Updated the granule-ingester.env file as follows: > > > > CASSANDRA_CONTACT_POINTS=172.18.0.3 > > CASSANDRA_USERNAME=cassandra > > CASSANDRA_PASSWORD=cassandra > > Here, the CASSANDRA_CONTACT_POINTS value was manually set to the > > container’s IP address using docker inspect cassandra. > > Created a new RabbitMQ user with admin access and set the virtual host > to /. > > > > After getting granule ingester to not abort due to connection with > > cassandra and rabbitmq. collection-manager was able to run and then > > Debian 12 docker desktop crashed. Tried with default settings for > > resources and also near maximum resources. Both cases of docker > > desktop configuration crashed. We apologize that we can not provide > > too many details of the crash as Docker Desktop premium is required to > > determine exact causes of the crashes. > > > > To conclude, with all the mentioned troubleshooting, I was able to > > complete all the steps then a crash occurs. > > > > > > Questions Regarding Documentation > > > > For documentation, there seems to be two sdap-nexus.readthedocs.io & > > incubator-sdap-nexus readthedocs.io. Can you confirm whether these are > > the same or serve different purposes? > > > > For example, > > https://sdap-nexus.readthedocs.io/en/1.4.0/quickstart.html > > export COLLECTION_MANAGER_VERSION=1.4.0 > > export GRANULE_INGESTER_VERSION=1.4.0 > > > > https://incubator-sdap-nexus.readthedocs.io/en/latest/quickstart.html > > export COLLECTION_MANAGER_VERSION=1.2.0 > > export GRANULE_INGESTER_VERSION=1.2.0 > > > > With the code block of all version tags, would the given work? As we > > are not familiar with how sensitive each container is to new changes > > found in other containers. > > > > Was Kubernetes a pre-requisite to run docker QuickStart? > > > > > > Thank you for the help and support, we also filled out the doodle for > > meeting time so we're looking forward to seeing everyone! > > > > Thank you for your time. > > > > Sincerely, > > Truc Nguyen > > California State Polytechnic University, Pomona > > trucngu...@cpp.edu > > ________________________________ > > From: Nga Chung <nch...@apache.org> > > Sent: Sunday, November 17, 2024 9:11 PM > > To: Truc N. Nguyen <trucngu...@cpp.edu>; Marc A. Cruz <marcc...@cpp.edu> > > Subject: Ex: Re: CS 3700 Re: SDAP Airflow Questions and Meeting with > SDAP Team > > > > [You don't often get email from nch...@apache.org. Learn why this is > > important at https://aka.ms/LearnAboutSenderIdentification ] > > > > Third time's the charm... Sorry by default when we reply to dev@ > > mailing list it only includes dev@. > > > > Looking forward to speaking to you both. > > > > Best, > > Nga > > > > On Sun, Nov 17, 2024 at 9:09 PM Nga Chung <nch...@apache.org> wrote: > > > > > > Sorry, forget to reply all! Truc, please see my message below. > > > > > > Thanks, > > > Nga > > > > > > > > > > > > On Sun, Nov 17, 2024 at 9:07 PM Nga Chung <nch...@apache.org> wrote: > > > > > > > > Hi Truc, > > > > > > > > We'd be happy to meet with you. As Riley mentioned, it would be great > > > > if you can please provide a brief description of the challenges, so > we > > > > can come to the meeting prepared. > > > > > > > > I've also created a doodle poll, so for those who would like to join > > > > us, please fill out the poll and we will find a time that works for > > > > everyone. > > > > > > > > Link to poll: > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoodle.com%2Fmeeting%2Fparticipate%2Fid%2FaQvQZAlb&data=05%7C02%7Ctrucnguyen%40cpp.edu%7Cf93cdbddbf854256e15608dd078f846b%7C164ba61e39ec4f5d89ffaa1f00a521b4%7C0%7C0%7C638675035179090142%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=JD0kfSP6L3poS3Vl1FbfJC4d0SisdY9lKjp60sWBL8U%3D&reserved=0 > <https://doodle.com/meeting/participate/id/aQvQZAlb> > > > > > > > > Thanks, > > > > Nga > > > > > > > > On Fri, Nov 15, 2024 at 1:06 PM Riley Kuttruff <r...@apache.org> > wrote: > > > > > > > > > > Hi! > > > > > > > > > > I'd be happy to help in any way I can. Could you perhaps provide a > brief > > > > > description of the technical challenges you've been facing? Just > so I can > > > > > have a little time to prepare to make the most of our time. > > > > > > > > > > I'm happy to meet next week though my calendar is a bit full so we > may need > > > > > to iterate a little to find a suitable time. What times work for > you? > > > > > > > > > > Also, I think @nchung may also wish to attend. > > > > > > > > > > -Riley > > > > > > > > > > On Fri, Nov 15, 2024, 12:52 PM Truc N. Nguyen <trucngu...@cpp.edu> > wrote: > > > > > > > > > > > Good afternoon, SDAP Team, > > > > > > > > > > > > My partner and I are students in Dr. Thomas Huang’s Parallel > Processing > > > > > > class, and we are working on a project about Apache SDAP ETL > pipeline using > > > > > > Apache Airflow as part of our class presentation. > > > > > > > > > > > > Dr. Huang recommended reaching out to your team for guidance. We > are > > > > > > currently facing some technical challenges and would greatly > appreciate any > > > > > > support or advice you could provide. > > > > > > > > > > > > Would it be possible to schedule a meeting with your team > sometime next > > > > > > week? We are also aware of your monthly meeting scheduled for > next Tuesday, > > > > > > and while we’re unsure if we can attend, we will make an effort > to join if > > > > > > it's possible. > > > > > > > > > > > > Thank you for your time and look forward to hearing from you > soon. > > > > > > > > > > > > Sincerely, > > > > > > Truc Nguyen > > > > > > California State Polytechnic University, Pomona > > > > > > trucngu...@cpp.edu > > > > > > > > CAUTION: This email was not sent from a Cal Poly Pomona service. > > Exercise caution when clicking links or opening attachments. Please > > forward suspicious email to > > suspectem...@cpp.edu<mailto:suspectem...@cpp.edu>. > > *CAUTION:* This email was not sent from a Cal Poly Pomona service. > Exercise caution when clicking links or opening attachments. Please forward > suspicious email to suspectem...@cpp.edu. > > > *CAUTION:* This email was not sent from a Cal Poly Pomona service. > Exercise caution when clicking links or opening attachments. Please forward > suspicious email to suspectem...@cpp.edu. > > > *CAUTION:* This email was not sent from a Cal Poly Pomona service. > Exercise caution when clicking links or opening attachments. Please forward > suspicious email to suspectem...@cpp.edu. > > >
