Hi Dimuthu, I did try the steps mentioned in [10]. I forgot to mention that when doing so, I wouldn’t get any connection at all. However, if I were to simply leave it as:
docker-compose exec pga getent hosts I could then copy the last address (which wasn’t a loopback) and use the next `exec` function to set the container’s hosts file, allowing me to connect to the web interface presented by the container. I should note that we have one developer doing this on native Ubuntu Linux and he faced the same issue. Thank you! - Daniel Shchur From: DImuthu Upeksha<mailto:[email protected]> Sent: Monday, October 19, 2020 5:29 PM To: Airavata Dev<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]>; Evan Palmer<mailto:[email protected]> Subject: Re: HTCondor Extension Help Hi Daniel, Did you follow the steps mentioned in [10]?. I recently added couple of instructions that is specific to windows deployments docker-compose exec pga getent hosts host.docker.internal docker-compose exec pga /bin/sh -c "echo '<host-machine ip> airavata.host' >> /etc/hosts" [10] https://github.com/apache/airavata/tree/develop/modules/ide-integration#optional-starting-super-admin-portal-pga<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_tree_develop_modules_ide-2Dintegration-23optional-2Dstarting-2Dsuper-2Dadmin-2Dportal-2Dpga&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=Ypkf_DmUlrtRhSp_fDiIv6WQo6FaPTI9Qqf6Dutchw0&s=t9cv4PIElFkkZUmTp4QOg_j6bhsIQasdi6gKCIxrmzM&e=> Thanks Dimuthu On Sun, Oct 18, 2020 at 7:26 PM Daniel Shchur <[email protected]<mailto:[email protected]>> wrote: Hi Dimuthu, We had some trouble initially getting the changes pulled in, but we eventually did and were able to run the portal. However, when running as recommended, the PGA portal is not able to communicate with the database stack with Keystore and so on, so we weren’t able to get to the login. The error we receive is: curl_exec() failed. Error1: Failed to connect to airavata.host port 8443: Connection refused One theory I had was that the containers aren’t communicating with each other so I added an external network within their respective docker-compose files might mitigate the issue, and it did. I was able to get into the login portal without the issue above given the resources stack and pga container were all on the same docker internal network. However, now when trying to log in, the portal redirects to the home page noting that the “Airavata servers are currently down”. Any help with this would be greatly appreciated. Thanks! - Daniel Shchur From: DImuthu Upeksha<mailto:[email protected]> Sent: Wednesday, October 14, 2020 12:33 AM To: Airavata Dev<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]>; Evan Palmer<mailto:[email protected]> Subject: Re: HTCondor Extension Help Hi Daniel, We only have tested ide integration on MacOS and Ubuntu. However I just managed to reproduce your issue in Windows 10 with WSL2 enabled and commits [5-9] done to the develop branch should fix those issues. You might have to merge those changes to your fork to make it effective and I recommend getting a fresh clone to avoid unexpected behaviors due to temp files. Let us know if it works for you. [5] https://github.com/apache/airavata/commit/f9b9e203fc268db468977680ab511f88a3eadc99<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_commit_f9b9e203fc268db468977680ab511f88a3eadc99&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=J6UxM_Wu9xQcmoXxxvGTjmFqbH5Za77xBEF4pTynXj8&s=jXtAEOkym95skPFsXd9YW4PxKg8ko3fmd2lHgphKA2M&e=> [6] https://github.com/apache/airavata/commit/3794421f01fdb4ef5c0199fdfc6cfc15c25d2e7a<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_commit_3794421f01fdb4ef5c0199fdfc6cfc15c25d2e7a&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=J6UxM_Wu9xQcmoXxxvGTjmFqbH5Za77xBEF4pTynXj8&s=avw7ECyJ3UUo0YYlphKm8LWxDrf_wHmaoUsMSSx_CyU&e=> [7] https://github.com/apache/airavata/commit/f581b61d44eb9a848b866456acd63f29e2bbfc8e<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_commit_f581b61d44eb9a848b866456acd63f29e2bbfc8e&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=J6UxM_Wu9xQcmoXxxvGTjmFqbH5Za77xBEF4pTynXj8&s=_LrHNm7vMt75JMADyCyHs7VWKutPjKZCjw8YLnZF0xQ&e=> [8] https://github.com/apache/airavata/commit/9e30529d7545a1df0b9d94c2783d7d11c32d301d<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_commit_9e30529d7545a1df0b9d94c2783d7d11c32d301d&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=J6UxM_Wu9xQcmoXxxvGTjmFqbH5Za77xBEF4pTynXj8&s=aO43YTRpZ8PLuZ84xcB_yR-5yT8GOKXtfwHKxAgdkak&e=> [9] https://github.com/apache/airavata/commit/b11d81182f915491187dcf9f3ede5beeb5ea94a2<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_commit_b11d81182f915491187dcf9f3ede5beeb5ea94a2&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=J6UxM_Wu9xQcmoXxxvGTjmFqbH5Za77xBEF4pTynXj8&s=k_GXTpaj3wcjCgsjj8PIzxxFISaGmYekMoTg2rk5aM0&e=> Thanks Dimuthu On Tue, Oct 13, 2020 at 6:37 PM Daniel Shchur <[email protected]<mailto:[email protected]>> wrote: Hi Dimuthu, Windows 10 2004 19041.508 Docker v19.03.13 running on WSL2 - Ubuntu 20.04 - Daniel Shchur From: DImuthu Upeksha<mailto:[email protected]> Sent: Tuesday, October 13, 2020 5:34 PM To: Airavata Dev<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]>; Evan Palmer<mailto:[email protected]> Subject: Re: HTCondor Extension Help Hi Daniel, I'm trying to reproduce the error locally. Can you please send me the following information of your local setup. OS / version Docker version Thanks Dimuthu On Tue, Oct 13, 2020 at 5:32 PM Daniel Shchur <[email protected]<mailto:[email protected]>> wrote: Hi Dimunth, We have been trying to get the PGA running; however, it seems to fail building currently during the optimize portion. This is after having started the resources stack and starting the JobEngineStarter and AiravataAPI. Those parts start up without a hitch. I also added in the hosts file changes. The error is: > php artisan optimize Generating optimized class loader Compiling common classes Compiling views [ErrorException] file_put_contents(/var/www/portals/default/app/storage/views/16ac8ea2e3f88487e87226a07d300af0): failed to open stream: No such file or directory optimize [--force] [--psr] Script php artisan optimize handling the post-install-cmd event returned with error code 1 Any help would be greatly appreciated. Thanks! - Daniel Shchur From: DImuthu Upeksha<mailto:[email protected]> Sent: Friday, October 9, 2020 8:46 AM To: Airavata Dev<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]>; Evan Palmer<mailto:[email protected]> Subject: Re: HTCondor Extension Help Hi Daniel, Please send you PR to airavata:htcondor branch which I created yesterday. Then I also can help you to speed up the development process. Thanks Dimuthu On Thu, Oct 8, 2020 at 5:59 PM Daniel Shchur <[email protected]<mailto:[email protected]>> wrote: Hi Dimuthu, That clears up a lot of confusion. Thank you! - Daniel Shchur From: DImuthu Upeksha<mailto:[email protected]> Sent: Thursday, October 8, 2020 4:58 PM To: Airavata Dev<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]>; Evan Palmer<mailto:[email protected]> Subject: Re: HTCondor Extension Help Hi Daniel, That's correct, we assume all job manager configurations have unique set of command types like job submission, cancellation, status checking etc. It's up to the end user to specify the exact command for each command type. For example, SLURM job submission command is sbatch. So the user has to specify that though the portal which will eventually be stored in the database. [cid:17542f724f9197e1881] Thanks Dimuthu On Thu, Oct 8, 2020 at 5:14 PM Daniel Shchur <[email protected]<mailto:[email protected]>> wrote: Hi Dimuthu, So, to be clear, The relevant job submission commands are registered at the client-side? As in, the front end interface and are not hardcoded into the backend? - Daniel Shchur From: DImuthu Upeksha<mailto:[email protected]> Sent: Thursday, October 8, 2020 4:12 PM To: Airavata Dev<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]>; Evan Palmer<mailto:[email protected]> Subject: Re: HTCondor Extension Help Hi Daniel, If you look at [2], it's where the protocol specific configuration object is created. Each implementation is a subclass of the JobManagerConfiguration interface [3]. So each implementation might have different execution commands and those are stored in the database and you can register them through PGA. Please look at the screenshot I sent earlier. Once helix figured out the right Configuration implementation, it invokes it at the [4]. Please let me know if this is clear [2] https://github.com/apache/airavata/blob/develop/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/task/submission/config/JobFactory.java#L106<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_blob_develop_modules_airavata-2Dhelix_helix-2Dspectator_src_main_java_org_apache_airavata_helix_impl_task_submission_config_JobFactory.java-23L106&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=TIv6Hm841JVIH_TucpIxiNWYVrZ9o8BO-aAKlXeLNCU&s=jOokB292No_5nhnM9IQCJ3f4HgYCBysW7cYy7r_iwns&e=> [3] https://github.com/apache/airavata/blob/develop/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/task/submission/config/JobManagerConfiguration.java<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_blob_develop_modules_airavata-2Dhelix_helix-2Dspectator_src_main_java_org_apache_airavata_helix_impl_task_submission_config_JobManagerConfiguration.java&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=TIv6Hm841JVIH_TucpIxiNWYVrZ9o8BO-aAKlXeLNCU&s=E9HjTKXLL3YhkdLhA98G3r0fH7lpD5WKlJldHoFI32E&e=> [4] https://github.com/apache/airavata/blob/develop/modules/airavata-helix/helix-spectator/src/main/java/org/apache/airavata/helix/impl/task/submission/JobSubmissionTask.java#L78<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_blob_develop_modules_airavata-2Dhelix_helix-2Dspectator_src_main_java_org_apache_airavata_helix_impl_task_submission_JobSubmissionTask.java-23L78&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=TIv6Hm841JVIH_TucpIxiNWYVrZ9o8BO-aAKlXeLNCU&s=yuHKbjJZT93zcEiQHAIPBNZMq6aKiKLmzkSep5i2-j8&e=> Thanks Dimuthu On Thu, Oct 8, 2020 at 4:58 PM Daniel Shchur <[email protected]<mailto:[email protected]>> wrote: Hi all, We have been trying to figure out where the helix system actually specifies job submission protocol commands (e.g. sbatch, squeue, scancel). The only direct references we could find are part of the client-sdk “registerSampleApplication”. Everything else, when referencing “ResourceJobConfiguration” seems to be fairly circular, or goes to dead-end files that may not be used anymore. Question: is there a direct command mapping for the “Map<JobManagerCommand, string>” that is called within the code, or is this set elsewhere? We haven’t found any references to where Airavata would know how to submit a SLURM job so that we can let it know how to submit an HTCondor job. Any help on this would be greatly appreciated. Thanks! - Daniel Shchur From: DImuthu Upeksha<mailto:[email protected]> Sent: Monday, October 5, 2020 9:50 AM To: Airavata Dev<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]>; Evan Palmer<mailto:[email protected]> Subject: Re: HTCondor Extension Help Sorry I didn't send the image I mentioned earlier On Sun, Oct 4, 2020 at 10:51 PM DImuthu Upeksha <[email protected]<mailto:[email protected]>> wrote: Hi Daniel, Your PR looks good to me. I think the best place for you to start is by setting up the PGA portal (the old portal we used before the django one) and adding a resource manager type for the compute resource as you can see in the attached image. You can set up the pga locally using this [1] guide. Once you can register the resource manager type using the pga portal, you can use the django portal to submit jobs. I believe that you don't need to change any code in the django portal. [1] https://github.com/apache/airavata/tree/master/modules/ide-integration#optional-starting-super-admin-portal-pga<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_tree_master_modules_ide-2Dintegration-23optional-2Dstarting-2Dsuper-2Dadmin-2Dportal-2Dpga&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=tuYZSoSxpr0vpOY_QMnBJNwCbUSViATLYySXRyD33aw&s=8A5-Br28vywXcU42WH9bl0Ad0b5YMnXNNIplSl3r9J8&e=> Thanks Dimuthu On Sun, Oct 4, 2020 at 6:12 PM Daniel Shchur <[email protected]<mailto:[email protected]>> wrote: Hi Dimuthu and Suresh, Thanks for getting back to us. We went ahead and made a work in progress pull request<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_pull_263&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=tuYZSoSxpr0vpOY_QMnBJNwCbUSViATLYySXRyD33aw&s=axraS0UJlcrcJlRwGIeswZKPC0cHEDu13ADapD1Xz5A&e=> for our changes we have created so far. We also took a look at the job-monitor section and will work towards creating additions to support HTCondor. Do any of you have any suggestions on what the best way is to test submissions to HTCondor? We looked into using the Airavata API and the Django portal and are curious what you would recommend, and how to do it using your recommendation. Thanks! - Daniel Shchur From: DImuthu Upeksha<mailto:[email protected]> Sent: Friday, October 2, 2020 5:22 PM To: Airavata Dev<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]>; Evan Palmer<mailto:[email protected]> Subject: Re: HTCondor Extension Help Hi Daniel, This is great progress. I think you are on the right track. Did you manage to set up the ide integration for the development of Airavata. If you have already done that, you can test these features. However I recommend you to send pull requests to a separate branch (htcondor?) and then I can provide you feedback on the next steps. Thanks Dimuthu On Thu, Oct 1, 2020 at 4:57 PM Suresh Marru <[email protected]<mailto:[email protected]>> wrote: Hi Daniel, Thanks for this nice summary for keeping the dev list apprised of your progress. One additional item is to discuss and plan on job monitoring approaches, you can look at current references here - https://github.com/apache/airavata/tree/master/modules/job-monitor<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_airavata_tree_master_modules_job-2Dmonitor&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=lmrRGQ0q0u8q8eCyoDDHtDCxKniLMvrktlVw7qnV3Fo&s=zpJDdIRgv9hm0QkGZuHhDOYvc1_zMhUmH9TF7mx9A-0&e=> Suresh On Oct 1, 2020, at 4:52 PM, Daniel Shchur <[email protected]<mailto:[email protected]>> wrote: Hi all, We are working on the HTCondor extension for Airavata. We have gone through the code and started to implement additions based on the other job workflow systems such as SLURM. We forked<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_docquantum_airavata_tree_htcondor&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=lmrRGQ0q0u8q8eCyoDDHtDCxKniLMvrktlVw7qnV3Fo&s=14ShokmbcQHBZlmV67fTrNoK8ARktUQERu6wRQQTlWc&e=> the project and created a new branch (htcondor) to host our changes. We will make a pull request into the main Airavata repository for it once we have finished the implementation. A relevant issue has also been created here<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_AIRAVATA-2D3377&d=DwMFaQ&c=Cu5g146wZdoqVuKpTNsYHeFX_rg6kWhlkLF8Eft-wwo&r=SfMtyOx08LDuEVnPLH5bLf0tBX7T0FnnGw0oDSWu8Es&m=lmrRGQ0q0u8q8eCyoDDHtDCxKniLMvrktlVw7qnV3Fo&s=ZjE4skBTqEHdJqLBat2-uLoS1w6_Ksh_c5d4VPKjnS8&e=>. We would like help to know exactly where additions need to be made in case we missed a few spots. At this point, we know that: * A groovy template needs to be made (finished) * A job configuration class needs to be made (WIP) * A job parser class needs to be made (WIP) * And enums need to be modified throughout the project so that systems know of HTCondor as an option. (finished) If any of you could direct us to the right places, that would be greatly appreciated. Also, if there’s more documentation or information about the code, such as how the command map should work or should be implemented (configuration class?), that would be appreciated. Thank you! - Daniel Shchur
