[GitHub] [incubator-sdap-nexus] RKuttruff commented on a diff in pull request #185: SDAP-399: Quickstart update - Docker

[GitBox]

RKuttruff commented on code in PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#discussion_r984136730


##
docs/quickstart.rst:
##
@@ -18,43 +18,59 @@ This quickstart guide will walk you through how to install 
and run NEXUS on your
 Prerequisites
 ==
 
-* Docker (tested on v18.03.1-ce)
+* Docker (tested on v20.10.17)
 * Internet Connection
-* bash
+* bash or zsh
 * cURL
-* 500 MB of disk space
+* 10.5 GB of disk space

Review Comment:
   Lot of docker images that are quite sizable.
   
   + a little margin for granule data &c so I just rounded up to a half GB or so



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on a diff in pull request #185: SDAP-399: Quickstart update - Docker

[GitBox]

RKuttruff commented on code in PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#discussion_r984136329


##
CHANGELOG.md:
##
@@ -41,9 +41,9 @@ and this project adheres to [Semantic 
Versioning](https://semver.org/spec/v2.0.0
 - Fixed issue where satellite to satellite matchups with the same dataset 
don't return the expected result
 - Fixed CSV and NetCDF matchup output bug
 - Fixed NetCDF output switching latitude and longitude
+- SDAP-399: Updated quickstart guide for standalone docker deployment of SDAP.
+- SDAP-399: Updated quickstart Jupyter notebook
 - Fixed import error causing `/timeSeriesSpark` queries to fail.
 - Fixed bug where domsresults no longer worked after successful matchup
 - Fixed certificate error in Dockerfile
-### Security

Review Comment:
   I think I removed that line by accident. Oops



##
CHANGELOG.md:
##
@@ -41,9 +41,9 @@ and this project adheres to [Semantic 
Versioning](https://semver.org/spec/v2.0.0
 - Fixed issue where satellite to satellite matchups with the same dataset 
don't return the expected result
 - Fixed CSV and NetCDF matchup output bug
 - Fixed NetCDF output switching latitude and longitude
+- SDAP-399: Updated quickstart guide for standalone docker deployment of SDAP.
+- SDAP-399: Updated quickstart Jupyter notebook
 - Fixed import error causing `/timeSeriesSpark` queries to fail.
 - Fixed bug where domsresults no longer worked after successful matchup
 - Fixed certificate error in Dockerfile
-### Security

Review Comment:
   I think I removed that line by accident. Oops.
   
   I'll put it back



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on a diff in pull request #185: SDAP-399: Quickstart update - Docker

[GitBox]

RKuttruff commented on code in PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#discussion_r984136094


##
docker/jupyter/Dockerfile:
##
@@ -29,7 +29,8 @@ ENV CHOWN_HOME_OPTS='-R'
 ENV REBUILD_CODE=true
 
 ARG APACHE_NEXUS=https://github.com/apache/incubator-sdap-nexus.git
-ARG APACHE_NEXUS_BRANCH=master
+ARG APACHE_NEXUS_COMMIT=be19c1d567301b09269e851cc5b5af55fea02c5d

Review Comment:
   I checked out this commit because I was unaware that the code for nexuscli 
had moved (I thought it was removed) so I grabbed a commit that would've been 
what was used to create the old docker image. I'll retry with the current 
version.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #202: SDAP-402: Update matchOnce logic

[GitBox]

RKuttruff commented on PR #202:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/202#issuecomment-1262998052

   I actually don't know exactly how many matches L2 -> Saildrone _should_ be 
returning. I based the current test case on how many were returning from the 
AWS deployment. However, the current deployment is returning the 'expected' 
number of matches while the new changes have it returning 2 extra. One thing to 
note about these extra matches is that the primary points are actually 
duplicated in the matchup output. 
   
   An example for one of the extra matches:
   
   
Common point 
   
   ```json
   {
"platform": "orbiting satellite",
"device": "scatterometers",
"lon": "-129.94937",
"lat": "26.56576",
"point": "Point(-129.94937 26.56576)",
"time": 1522695015,
"depth": null,
"fileurl": 
"ascat_20180402_180900_metopb_28742_eps_o_coa_2401_ovw.l2.nc",
"id": "44d09fab-f85e-3de5-9495-835b8607d8e2[[9 9 9]]",
"source": "ASCATB-L2-Coastal",
"primary": [
{
"variable_name": "wind_speed",
"cf_variable_name": "wind_speed",
"variable_value": 2.480190734863,
"variable_unit": null
},
{
"variable_name": "wind_dir",
"cf_variable_name": "wind_to_direction",
"variable_value": 283.338964844,
"variable_unit": null
}
],
"matches": [
{
"platform": "3B",
"device": "130",
"lon": "-130.0027776",
"lat": "26.35588",
"point": "Point(-130.0027776 26.35588)",
"time": 1522691460,
"depth": 0.5,
"fileurl": null,
"id": null,
"source": "shark-2018",
"secondary": [
{
"variable_name": 
"sea_water_practical_salinity",
"cf_variable_name": 
"sea_water_practical_salinity",
"variable_value": 34.681,
"variable_unit": "1"
},
{
"variable_name": 
"sea_water_practical_salinity_quality",
"cf_variable_name": 
"sea_water_practical_salinity_quality",
"variable_value": 2,
"variable_unit": null
},
{
"variable_name": 
"sea_water_temperature",
"cf_variable_name": 
"sea_water_temperature",
"variable_value": 19.314,
"variable_unit": "C"
},
{
"variable_name": 
"sea_water_temperature_quality",
"cf_variable_name": 
"sea_water_temperature_quality",
"variable_value": 2,
"variable_unit": null
}
]
}
]
   }
   
   ```
   
   
   
Extra point (from modified SDAP) 
   
   ```json
   {
"platform": "orbiting satellite",
"device": "scatterometers",
"lon": "-129.94937",
"lat": "26.56576",
"point": "Point(-129.94937 26.56576)",
"time": 1522695015,
"depth": null,
"fileurl": 
"ascat_20180402_180900_metopb_28742_eps_o_coa_2401_ovw.l2.nc",
"id": "44d09fab-f85e-3de5-9495-835b8607d8e2[[9 9 9]]",
"source": "ASCATB-L2-Coastal",
"primary": [
{
"variable_name": "wind_speed",
"cf_variable_name": "wind_speed",
"variable_value": 2.480190734863,
"variable_unit": null
},
{
"variable_name": "wind_dir",
"cf_variable_name": "wind_to_direction",
"variable_value": 283.338964844,
"variable_unit": null
}
],
"matches": [
{
"platform": "3B",
"device": "113",
"lon": "-130.0174208",
"lat": "26.3396",
"point": "Point(-

[GitHub] [incubator-sdap-nexus] skorper commented on a diff in pull request #185: SDAP-399: Quickstart update - Docker

[GitBox]

skorper commented on code in PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#discussion_r984133710


##
docker/jupyter/requirements.txt:
##
@@ -1,4 +1,3 @@
-shapely
-requests
-numpy
-cassandra-driver==3.9.0

Review Comment:
   I'm curious why we were able to remove cassandra-driver? Or why it was 
needed before but not now?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper commented on a diff in pull request #185: SDAP-399: Quickstart update - Docker

[GitBox]

skorper commented on code in PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#discussion_r984133683


##
docs/quickstart.rst:
##
@@ -18,43 +18,59 @@ This quickstart guide will walk you through how to install 
and run NEXUS on your
 Prerequisites
 ==
 
-* Docker (tested on v18.03.1-ce)
+* Docker (tested on v20.10.17)
 * Internet Connection
-* bash
+* bash or zsh
 * cURL
-* 500 MB of disk space
+* 10.5 GB of disk space
 
 Prepare
 
 
-Start downloading the Docker images and data files.
+Start downloading the Docker images and set up the Docker bridge network.
 
 .. _quickstart-step1:
 
 Pull Docker Images
 ---
 
-Pull the necessary Docker images from the `SDAP repository 
`_ on Docker Hub. Please check the repository 
for the latest version tag.
+Pull the necessary Docker images from the `NEXUS JPL repository 
`_ on Docker Hub. Please check the 
repository for the latest version tag.
 
 .. code-block:: bash
 
-  export VERSION=1.0.0-rc1
+  export CASSANDRA_VERSION=3.11.6-debian-10-r138
+  export RMQ_VERSION=3.8.9-debian-10-r37
+  export COLLECTION_MANAGER_VERSION=0.1.6a14
+  export GRANULE_INGESTER_VERSION=0.1.6a30
+  export WEBAPP_VERSION=distributed.0.4.5a49
+  export SOLR_VERSION=8.11.1
+  export SOLR_CLOUD_INIT_VERSION=1.0.2
+  export ZK_VERSION=3.5.5
+
+  export JUPYTER_VERSION=1.2
 
 .. code-block:: bash
 
-  docker pull sdap/ningester:${VERSION}
-  docker pull sdap/solr-singlenode:${VERSION}
-  docker pull sdap/cassandra:${VERSION}
-  docker pull sdap/nexus-webapp:standalone.${VERSION}
+  docker pull bitnami/cassandra:${CASSANDRA_VERSION}
+  docker pull bitnami/rabbitmq:${RMQ_VERSION}
+  docker pull nexusjpl/collection-manager:${COLLECTION_MANAGER_VERSION}
+  docker pull nexusjpl/granule-ingester:${GRANULE_INGESTER_VERSION}
+  docker pull nexusjpl/nexus-webapp:${WEBAPP_VERSION}
+  docker pull nexusjpl/solr:${SOLR_VERSION}
+  docker pull nexusjpl/solr-cloud-init:${SOLR_CLOUD_INIT_VERSION}
+  docker pull zookeeper:${ZK_VERSION}
+
+  # docker pull nexusjpl/jupyter:${JUPYTER_VERSION}

Review Comment:
   let's update this back to nexusjpl before we merge this



##
CHANGELOG.md:
##
@@ -41,9 +41,9 @@ and this project adheres to [Semantic 
Versioning](https://semver.org/spec/v2.0.0
 - Fixed issue where satellite to satellite matchups with the same dataset 
don't return the expected result
 - Fixed CSV and NetCDF matchup output bug
 - Fixed NetCDF output switching latitude and longitude
+- SDAP-399: Updated quickstart guide for standalone docker deployment of SDAP.
+- SDAP-399: Updated quickstart Jupyter notebook
 - Fixed import error causing `/timeSeriesSpark` queries to fail.
 - Fixed bug where domsresults no longer worked after successful matchup
 - Fixed certificate error in Dockerfile
-### Security

Review Comment:
   you can leave this section in empty in case we make a security change



##
docker/jupyter/Dockerfile:
##
@@ -29,7 +29,8 @@ ENV CHOWN_HOME_OPTS='-R'
 ENV REBUILD_CODE=true
 
 ARG APACHE_NEXUS=https://github.com/apache/incubator-sdap-nexus.git
-ARG APACHE_NEXUS_BRANCH=master
+ARG APACHE_NEXUS_COMMIT=be19c1d567301b09269e851cc5b5af55fea02c5d

Review Comment:
   This commit is from 2019... ideally, I think we'd like this to run on the 
latest code? What issues did you run when running this on master?



##
docker/jupyter/requirements.txt:
##
@@ -1,4 +1,3 @@
-shapely
-requests
-numpy
-cassandra-driver==3.9.0

Review Comment:
   I'm curious why we were able to remove this? Or why it was needed before but 
not now?



##
docs/quickstart.rst:
##
@@ -64,181 +80,240 @@ The network we will be using for this quickstart will be 
called ``sdap-net``. Cr
 
 .. _quickstart-step3:
 
-Download Sample Data
--
+Start Ingester Components and Ingest Some Science Data
+
 
-The data we will be downloading is part of the `AVHRR OI dataset 
`_ which 
measures sea surface temperature. We will download 1 month of data and ingest 
it into a local Solr and Cassandra instance.
+Create Data Directory
+
+
+Let's start by creating the directory to hold the science data to ingest.
 
 Choose a location that is mountable by Docker (typically needs to be under the 
User's home directory) to download the data files to.
 
 .. code-block:: bash
 
-  export DATA_DIRECTORY=~/nexus-quickstart/data/avhrr-granules
-  mkdir -p ${DATA_DIRECTORY}
+export DATA_DIRECTORY=~/nexus-quickstart/data/avhrr-granules
+mkdir -p ${DATA_DIRECTORY}
 
-Then go ahead and download 1 month worth of AVHRR netCDF files.
+Now we can start up the data storage components. We will be using Solr and 
Cassandra to store the tile metadata and data respectively.
 
-.. code-block:: bash
+.. _quic

[GitHub] [incubator-sdap-nexus] skorper commented on pull request #202: SDAP-402: Update matchOnce logic

[GitBox]

skorper commented on PR #202:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/202#issuecomment-1262983746

   Note, L2 -> Saildrone is returning more matches than expected for both my 
changes and the version deployed to AWS (Riley correct me if I'm wrong) so this 
discrepancy is maybe unrelated to this change?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] kevinmarlis opened a new pull request, #203: SDAP-403

[GitBox]

kevinmarlis opened a new pull request, #203:
URL: https://github.com/apache/incubator-sdap-nexus/pull/203

   
   On the AQ deployment we found a redirect bug that caused the AQ 
`/nexus/list` to crash when attempting to connect to a remote SDAP. The 
redirect code was catching `requests.exceptions.ConnectTimeout` exceptions but 
`requests.exceptions.ReadTimeout` exceptions could also be raised when the 
remote `/nexus/list` endpoint was unreachable.  This PR addresses that by 
including `requests.exceptions.ReadTimeout` in the except block. 
   
   This PR also includes a fix to the HofMoeller algorithms. There was a change 
made to the name of the nexus point data_val variable (from point.data_val to 
point.data_vals) in nexus model.py's nexus_point_generator. The change in 
variable name was not updated in the HofMoeller algorithms, so this change is 
included in the PR.
   
   The fix to the HofMoeller algorithms has been tested in SLCP's sit 
environment.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #202: SDAP-402: Update matchOnce logic

[GitBox]

RKuttruff commented on PR #202:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/202#issuecomment-1261347211

   As a followup: All the (currently implemented) test cases are passing with 
the exception of L2 -> Saildrone (returning more matches than expected)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #202: SDAP-402: Update matchOnce logic

[GitBox]

RKuttruff commented on PR #202:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/202#issuecomment-1261314820

   Testing to see if this affects my regression tests. It seems that the same 
query is returning different numbers of matches.
   
   The query I'm using:
   
`http://localhost:8083/match_spark?primary=MUR25-JPL-L4-GLOB-v04.2&startTime=2018-08-17T01%3A00%3A00Z&endTime=2018-08-27T06%3A00%3A00Z&tt=43200&rt=1000&b=-90.375,27.625,-86.125,28.125&depthMin=-20&depthMax=10&matchOnce=true&secondary=ICOADS%20Release%203.0&resultSizeLimit=7000&platforms=42`
   
   I've seen it return 3, 4, and 5 matches for different runs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper opened a new pull request, #202: SDAP-402: Update matchOnce logic

[GitBox]

skorper opened a new pull request, #202:
URL: https://github.com/apache/incubator-sdap-nexus/pull/202

   [SDAP-402](https://issues.apache.org/jira/browse/SDAP-402)
   
   - Fixed matchOnce logic. Previously, only a single secondary point was 
matched to a primary point. Now, we are allowing multiple matches so long as 
they share the same time and space as the closest point. 
   - Fixed the failing matchup unit tests
   - Added a new unit test case for this change
   - Fixed bug where `depth` was being overwritten with a null value (maybe 
will fix our depth problems?)
   
   Tested this locally with the following request (from the google doc):
   
   ```
   
{{big_data_url}}/match_spark?tt=43200&primary=JPL-L4-MRVA-CHLA-GLOB-v3.0&secondary=shark-2018&startTime=2018-04-01T00%3A00%3A00Z&endTime=2018-04-01T12%3A59%3A59Z&rt=25000&b=-140%2C10%2C-110%2C40&platforms=3B&depthMin=-5&depthMax=5&matchOnce=true
   ```
   
   
 This results in the following json output:
   
   ```json
   {
   "executionId": "e4fb1b4f-c002-4c68-a651-94db87e96c2f",
   "data": [
   {
   "platform": "orbiting satellite",
   "device": "radiometers",
   "lon": "-129.875",
   "lat": "27.125",
   "point": "Point(-129.875 27.125)",
   "time": 1522584000,
   "depth": null,
   "fileurl": "2018040112-JPL-CHL25-fv03.nc",
   "id": "96bc2f4b-fd78-3c41-a3cb-cc6b2d7197a3[[0, 18, 20]]",
   "source": "JPL-L4-MRVA-CHLA-GLOB-v3.0",
   "primary": [
   {
   "variable_name": "CHLA_analysis",
   "cf_variable_name": 
"mass_concentration_of_chlorophyll_a_in_sea_water",
   "variable_value": 0.0518328994512558,
   "variable_unit": null
   }
   ],
   "matches": [
   {
   "platform": "3B",
   "device": "113",
   "lon": "-130.015296",
   "lat": "27.0813824",
   "point": "Point(-130.015296 27.0813824)",
   "time": 1522570620,
   "depth": 0.5,
   "fileurl": null,
   "id": null,
   "source": "shark-2018",
   "secondary": [
   {
   "variable_name": 
"mass_concentration_of_chlorophyll_in_sea_water",
   "cf_variable_name": 
"mass_concentration_of_chlorophyll_in_sea_water",
   "variable_value": 0.11,
   "variable_unit": "kg m-3"
   },
   {
   "variable_name": 
"mass_concentration_of_chlorophyll_in_sea_water_quality",
   "cf_variable_name": 
"mass_concentration_of_chlorophyll_in_sea_water_quality",
   "variable_value": 2,
   "variable_unit": null
   }
   ]
   },
   {
   "platform": "3B",
   "device": "130",
   "lon": "-130.015296",
   "lat": "27.0813824",
   "point": "Point(-130.015296 27.0813824)",
   "time": 1522570620,
   "depth": 0.5,
   "fileurl": null,
   "id": null,
   "source": "shark-2018",
   "secondary": [
   {
   "variable_name": "sea_water_practical_salinity",
   "cf_variable_name": 
"sea_water_practical_salinity",
   "variable_value": 34.628,
   "variable_unit": "1"
   },
   {
   "variable_name": 
"sea_water_practical_salinity_quality",
   "cf_variable_name": 
"sea_water_practical_salinity_quality",
   "variable_value": 2,
   "variable_unit": null
   },
   {
   "variable_name": "sea_water_temperature",
   "cf_variable_name": "sea_water_temperature",
   "variable_value": 18.822,
   "variable_unit": "C"
   },
   {
   "variable_name": "sea_water_temperature_quality",
   "cf_variable_name": 
"sea_water_temperature_quality",
   "variable_value": 2,
   "variabl

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #201: Bumped ingress timeouts to reflect increased AWS gateway timeout

[GitBox]

RKuttruff opened a new pull request, #201:
URL: https://github.com/apache/incubator-sdap-nexus/pull/201

   Potential fix for duplicate request issue


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexusproto] dependabot[bot] opened a new pull request, #10: Bump protobuf from 3.2.0 to 3.18.3 in /src/main/python/nexusproto

[GitBox]

dependabot[bot] opened a new pull request, #10:
URL: https://github.com/apache/incubator-sdap-nexusproto/pull/10

   Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 3.2.0 to 
3.18.3.
   
   Release notes
   Sourced from https://github.com/protocolbuffers/protobuf/releases";>protobuf's 
releases.
   
   Protocol Buffers v3.18.3
   C++
   
   Reduce memory consumption of MessageSet parsing
   This release addresses a https://github.com/protocolbuffers/protobuf/security/advisories/GHSA-8gq9-2x98-w8hf";>Security
 Advisory for C++ and Python users
   
   Protocol Buffers v3.16.1
   Java
   
   Improve performance characteristics of UnknownFieldSet parsing (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/9371";>#9371)
   
   Protocol Buffers v3.18.2
   Java
   
   Improve performance characteristics of UnknownFieldSet parsing (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/9371";>#9371)
   
   Protocol Buffers v3.18.1
   Python
   
   Update setup.py to reflect that we now require at least Python 3.5 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8989";>#8989)
   Performance fix for DynamicMessage: force GetRaw() to be inlined (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/9023";>#9023)
   
   Ruby
   
   Update ruby_generator.cc to allow proto2 imports in proto3 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/9003";>#9003)
   
   Protocol Buffers v3.18.0
   C++
   
   Fix warnings raised by clang 11 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8664";>#8664)
   Make StringPiece constructible from std::string_view (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8707";>#8707)
   Add missing capability attributes for LLVM 12 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8714";>#8714)
   Stop using std::iterator (deprecated in C++17). (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8741";>#8741)
   Move field_access_listener from libprotobuf-lite to libprotobuf (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8775";>#8775)
   Fix https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/7047";>#7047
 Safely handle setlocale (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8735";>#8735)
   Remove deprecated version of SetTotalBytesLimit() (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8794";>#8794)
   Support arena allocation of google::protobuf::AnyMetadata (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8758";>#8758)
   Fix undefined symbol error around SharedCtor() (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8827";>#8827)
   Fix default value of enum(int) in json_util with proto2 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8835";>#8835)
   Better Smaller ByteSizeLong
   Introduce event filters for inject_field_listener_events
   Reduce memory usage of DescriptorPool
   For lazy fields copy serialized form when allowed.
   Re-introduce the InlinedStringField class
   v2 access listener
   Reduce padding in the proto's ExtensionRegistry map.
   GetExtension performance optimizations
   Make tracker a static variable rather than call static functions
   Support extensions in field access listener
   Annotate MergeFrom for field access listener
   Fix incomplete types for field access listener
   Add map_entry/new_map_entry to SpecificField in MessageDifferencer. They
   record the map items which are different in MessageDifferencer's 
reporter.
   Reduce binary size due to fieldless proto messages
   TextFormat: ParseInfoTree supports getting field end location in 
addition to
   start.
   
   
   
   ... (truncated)
   
   
   Commits
   
   https://github.com/protocolbuffers/protobuf/commit/a902b39270841beafc307dfa709610aa1cac2f06";>a902b39
 No-op whitespace change
   https://github.com/protocolbuffers/protobuf/commit/ae62acd7293e5a80378faeaac67b3baadba810d5";>ae62acd
 Updating version.json and repo version numbers to: 18.3
   https://github.com/protocolbuffers/protobuf/commit/f43ac49b91007501ce1683967b04dcfb47183478";>f43ac49
 Merge pull request https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10542";>#10542
 from deannagarcia/3.18.x
   https://github.com/protocolbuffers/protobuf/commit/9efdf55814add154d3c2646652f527a51d4a21ea";>9efdf55
 Add missing includes
   https://github.com/protocolbuffers/protobuf/commit/d1635e1496f51e0d5653d856211e8821bc47adc4";>d1635e1
 Apply patch
   https://github.com/protocolbuffers/protobuf/commit/5b37c91d62d7fe097253ac64518b993b096e9386";>5b37c91
 Update version.json with "lts": true (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/10534";>#10534)
   https://github.com/protocolbuffers/protobuf/commit/c39d622dba8343c50876770e6c2cc580781f6264

[GitHub] [incubator-sdap-nexusproto] dependabot[bot] closed pull request #9: Bump protobuf from 3.2.0 to 3.15.0 in /src/main/python/nexusproto

[GitBox]

dependabot[bot] closed pull request #9: Bump protobuf from 3.2.0 to 3.15.0 in 
/src/main/python/nexusproto
URL: https://github.com/apache/incubator-sdap-nexusproto/pull/9


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexusproto] dependabot[bot] commented on pull request #9: Bump protobuf from 3.2.0 to 3.15.0 in /src/main/python/nexusproto

[GitBox]

dependabot[bot] commented on PR #9:
URL: 
https://github.com/apache/incubator-sdap-nexusproto/pull/9#issuecomment-1256645505

   Superseded by #10.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper merged pull request #198: SDAP-400 Added NCAR insitu api to matchup

[GitBox]

skorper merged PR #198:
URL: https://github.com/apache/incubator-sdap-nexus/pull/198


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #200: SDAP-401: Added JSON output formatting for COVERAGE plotting

[GitBox]

RKuttruff commented on PR #200:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/200#issuecomment-1251679363

   Added implementation for `/domsresults` endpoint.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #200: SDAP-401: Added JSON output formatting for COVERAGE plotting

[GitBox]

RKuttruff opened a new pull request, #200:
URL: https://github.com/apache/incubator-sdap-nexus/pull/200

   Added output formatter for COVERAGE plotting to be used by the 
`/match_spark` endpoint.
   
   Outputs support the following plots:
   - Time series (primary + secondary)
   - Scatter (primary vs secondary)
   - Primary histogram
   - Secondary histogram
   - Difference map (primary - secondary)
   
   To use this formatter, set parameter `output=CAML`
   
   Parameters:
   
   |  **Parameter** |   

   **Description**  


|
   
|:--:|:-:|
   |   camlPrimary  | **Required:** Variable name from the primary dataset that 
is being plotted.   



|
   |  camlSecondary | **Required:** Variable name from the secondary dataset 
that is being plotted. Note that this will override the `parameter` parameter.  



   |
   | camlChartTypes | Comma-separated list of plot types to 
output. Default: Scatter, histograms, difference 
map.Values:time_series: Time series plotscatter: 
Scatter plothistogram_primary: Primary 
histogramhistogram_secondary: Secondary histogramtrajectory: 
Difference map |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] dependabot[bot] opened a new pull request, #199: Bump mako from 1.0.7 to 1.2.2 in /integrations/python-client

[GitBox]

dependabot[bot] opened a new pull request, #199:
URL: https://github.com/apache/incubator-sdap-nexus/pull/199

   Bumps [mako](https://github.com/sqlalchemy/mako) from 1.0.7 to 1.2.2.
   
   Release notes
   Sourced from https://github.com/sqlalchemy/mako/releases";>mako's releases.
   
   1.2.2
   Released: Mon Aug 29 2022
   bug
   
   
   [bug] [lexer] Fixed issue in lexer where the regexp used 
to match tags would not
   correctly interpret quoted sections individually. While this parsing issue
   still produced the same expected tag structure later on, the mis-handling
   of quoted sections was also subject to a regexp crash if a tag had a large
   number of quotes within its quoted sections.
   References: https://github-redirect.dependabot.com/sqlalchemy/mako/issues/366";>#366
   
   
   1.2.1
   Released: Thu Jun 30 2022
   bug
   
   
   [bug] [tests] Various fixes to the test suite in the 
area of exception message rendering
   to accommodate for variability in Python versions as well as Pygments.
   References: https://github-redirect.dependabot.com/sqlalchemy/mako/issues/360";>#360
   
   
   misc
   
   
   [performance] Optimized some codepaths within the 
lexer/Python code generation process,
   improving performance for generation of templates prior to their being
   cached. Pull request courtesy Takuto Ikuta.
   References: https://github-redirect.dependabot.com/sqlalchemy/mako/issues/361";>#361
   
   
   1.2.0
   Released: Thu Mar 10 2022
   changed
   
   
   [changed] [py3k] Corrected "universal wheel" 
directive in setup.cfg so that building a
   wheel does not target Python 2.
   References: https://github-redirect.dependabot.com/sqlalchemy/mako/issues/351";>#351
   
   
   [changed] [py3k] The bytestring_passthrough 
template argument is removed, as this
   flag only applied to Python 2.
   
   
   
   
   ... (truncated)
   
   
   Commits
   
   See full diff in https://github.com/sqlalchemy/mako/commits";>compare view
   
   
   
   
   
   [![Dependabot compatibility 
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=mako&package-manager=pip&previous-version=1.0.7&new-version=1.2.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
   
   Dependabot will resolve any conflicts with this PR as long as you don't 
alter it yourself. You can also trigger a rebase manually by commenting 
`@dependabot rebase`.
   
   [//]: # (dependabot-automerge-start)
   [//]: # (dependabot-automerge-end)
   
   ---
   
   
   Dependabot commands and options
   
   
   You can trigger Dependabot actions by commenting on this PR:
   - `@dependabot rebase` will rebase this PR
   - `@dependabot recreate` will recreate this PR, overwriting any edits that 
have been made to it
   - `@dependabot merge` will merge this PR after your CI passes on it
   - `@dependabot squash and merge` will squash and merge this PR after your CI 
passes on it
   - `@dependabot cancel merge` will cancel a previously requested merge and 
block automerging
   - `@dependabot reopen` will reopen this PR if it is closed
   - `@dependabot close` will close this PR and stop Dependabot recreating it. 
You can achieve the same result by closing it manually
   - `@dependabot ignore this major version` will close this PR and stop 
Dependabot creating any more for this major version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this minor version` will close this PR and stop 
Dependabot creating any more for this minor version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this dependency` will close this PR and stop 
Dependabot creating any more for this dependency (unless you reopen the PR or 
upgrade to it yourself)
   - `@dependabot use these labels` will set the current labels as the default 
for future PRs for this repo and language
   - `@dependabot use these reviewers` will set the current reviewers as the 
default for future PRs for this repo and language
   - `@dependabot use these assignees` will set the current assignees as the 
default for future PRs for this repo and language
   - `@dependabot use this milestone` will set the current milestone as the 
default for future PRs for this repo and language
   
   You can disable automated security fix PRs for this repo from the [Security 
Alerts page](https://github.com/apache/incubator-sdap-nexus/network/alerts).
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-in-situ-data-services] briansaildrone commented on pull request #9: EKS IRSA Clarification

[GitBox]

briansaildrone commented on PR #9:
URL: 
https://github.com/apache/incubator-sdap-in-situ-data-services/pull/9#issuecomment-1240931286

   I've also added further clarification for enabling EKS IRSA


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-in-situ-data-services] briansaildrone opened a new pull request, #9: EKS IRSA Typo Fix

[GitBox]

briansaildrone opened a new pull request, #9:
URL: https://github.com/apache/incubator-sdap-in-situ-data-services/pull/9

   Accidentally copied/pasted the existing credential-based s3 auth provider 
into the EKS IRSA documentation.  This PR fixes that by adding the correct auth 
provider.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper merged pull request #197: Matchup domsresults bugfix

[GitBox]

skorper merged PR #197:
URL: https://github.com/apache/incubator-sdap-nexus/pull/197


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper commented on pull request #197: Matchup domsresults bugfix

[GitBox]

skorper commented on PR #197:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/197#issuecomment-1235907553

   Added provider info to openapi spec:
   
   ![Screen Shot 2022-09-02 at 2 20 49 
PM](https://user-images.githubusercontent.com/11022336/188237092-df7c489a-bdf2-4f1a-86c1-63310da9f513.png)
   ![Screen Shot 2022-09-02 at 2 20 39 
PM](https://user-images.githubusercontent.com/11022336/188237094-5f7a916c-0adb-4a4b-8068-f5fd05e4761a.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper opened a new pull request, #197: Matchup domsresults bugfix

[GitBox]

skorper opened a new pull request, #197:
URL: https://github.com/apache/incubator-sdap-nexus/pull/197

   * Fixed bug introduced in the last PR where `domsresults` no longer worked. 
This was because the `details` field name were updated, but the code that 
inserts results into the database was not. I've updated the code that inserts 
the details into the database to look for the new field names. `None` is being 
used for `stats["numGriddedChecked"]` and `stats["numInsituChecked"]` because 
otherwise we'd need to change the db schema which would cause issues with many 
existing SDAP deployments. 
   * Added link to CDMS Schema endpoint in Swagger in the `parameter` and 
`platforms` field descriptions
   * Added env var `REQUESTS_CA_BUNDLE` env var to Dockerfile. Building the 
docker image suddenly stopped working the other day with error `Could not find 
a suitable TLS CA certificate bundle`, so this is a fix for that. Both myself 
and @kevinmarlis  have tested this fix.
   
   These changes are deployed to the CDMS AWS `nexus-sit` environment. 
   
   I tested the `domsresults` endpoint with both CDMS and DOMS matchup requests 
and was able to retrieve CSV results for both.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #196: Address matchup errors and integrate SDAP Proxy changes

[GitBox]

ngachung merged PR #196:
URL: https://github.com/apache/incubator-sdap-nexus/pull/196


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper opened a new pull request, #196: Address matchup errors and integrate SDAP Proxy changes

[GitBox]

skorper opened a new pull request, #196:
URL: https://github.com/apache/incubator-sdap-nexus/pull/196

   Prior to this branch, I have been cherry-picking out the SDAP Proxy changes, 
as they were breaking things in matchup.
   
   This PR addresses those changes, as well as implements feedback provided by 
the CDMS team.
   
   SDAP Proxy integration fixes:
   
   - Fixed error where matchup openapi spec was no longer being server via 
tornado
   - Fixed `domslist` endpoint which no longer worked
   - Fixed `cdmssubset` endpoint which no longer worked
   
   Feedback from team changes:
   
   - Remove numInSituRecords and numGriddedChecked from output
   - Change numInSituMatched to numSecondaryMatched and numGriddedMatched to 
numPrimaryMatched
   - Update parameter value to check against valid variables from in situ schema
   - Added `/domsresults` endpoint to openapi spec
   - Added validation to `parameter` matchup endpoint. `parameter` is validated 
by using the insitu schema endpoint. Example output if invalid param is 
provided:
   
   ```
   {
   "error": "Parameter test not supported. Must be one of ['air_pressure', 
'air_temperature', 'dew_point_temperature', 'downwelling_longwave_flux_in_air', 
'downwelling_longwave_radiance_in_air', 'downwelling_shortwave_flux_in_air', 
'mass_concentration_of_chlorophyll_in_sea_water', 'rainfall_rate', 
'relative_humidity', 'sea_surface_salinity', 'sea_surface_skin_temperature', 
'sea_surface_subskin_temperature', 'sea_surface_temperature', 
'sea_water_density', 'sea_water_electrical_conductivity', 
'sea_water_practical_salinity', 'sea_water_salinity', 'sea_water_temperature', 
'surface_downwelling_photosynthetic_photon_flux_in_air', 
'wet_bulb_temperature', 'wind_speed', 'wind_from_direction', 
'wind_to_direction', 'eastward_wind', 'northward_wind']",
   "code": 400
   }
   ```
   
   All of the above changes, except for the openAPI spec change, are deployed 
to the CDMS AWS SDAP deployment


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-ingester] MrFok opened a new pull request, #60: Preliminary Zarr Implementation + Tests

[GitBox]

MrFok opened a new pull request, #60:
URL: https://github.com/apache/incubator-sdap-ingester/pull/60

   - Generated separate files for Zarr implementation (without disturbing 
existing pipeline)
   - Local test generated for Lazy Load Zarr Aggregation (LLZA)
   - Minor dependency changes, bug fixes (needs checking), commenting
   - Preliminary methods for multiple parts in collection-manager and 
granule-ingester
   
   ### Changes in Proposed Pipeline
   - [x] Dataset metadata push for new, YAML format (needs dynamic public 
bucket boolean implementation)
   - [ ] Streamlined processors for xarray Dataset instead of nexus processing 
   - [ ] Direct Lazy Load Zarr Aggregation write into S3 bucket instead of tile 
storage in Cassandra
   
   **NOTE**: All changes have NOT been completely tested in a full run of the 
entire pipeline.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #185: SDAP-399: Quickstart update

[GitBox]

RKuttruff commented on PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#issuecomment-1214321592

   TODO: Check if AVHRR migration to Earthdata Cloud will affect granule 
download step.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #185: SDAP-399: Quickstart update

[GitBox]

RKuttruff commented on PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#issuecomment-1212779246

   Jupyter image verified to be working. 
   
   Now awaiting validation & runtime observations from others.
   
   PR will be ready when this happens.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #185: SDAP-399: Quickstart update

[GitBox]

RKuttruff commented on PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#issuecomment-1212307051

   I instead plan to have the k8s deployment in another PR. Once I get this one 
sorted & validated, I will make it ready for review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #185: SDAP-399: Quickstart update

[GitBox]

RKuttruff commented on PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#issuecomment-1211551329

   ```
   ---
   TypeError Traceback (most recent call last)
   Input In [3], in ()
20 start_time = datetime(2015, 11, 1)
21 end_time = datetime(2015, 11, 30)
   ---> 22 ts = nexuscli.time_series(datasets, bbox, start_time, end_time, 
spark=True)
24 print("Time Series took {} seconds to 
generate".format(time.perf_counter() - start))
26 # Plot the resulting time series using the helper method show_plot
   
   File 
/opt/conda/lib/python3.10/site-packages/nexuscli-1.0-py3.10.egg/nexuscli/nexuscli.py:212,
 in time_series(datasets, bounding_box, start_datetime, end_datetime, spark)
   206 time_series_data = np.array([tuple(each.values()) for each in 
[entry for entry in data if entry['ds'] == i]])
   208 if len(time_series_data) > 0:
   209 time_series_result.append(
   210 TimeSeries(
   211 dataset=response['meta'][i]['shortName'],
   --> 212 
time=np.array([datetime.utcfromtimestamp(t).replace(tzinfo=UTC) for t in
   213time_series_data[:, 
key_to_index['time']]]),
   214 mean=time_series_data[:, key_to_index['mean']],
   215 standard_deviation=time_series_data[:, 
key_to_index['std']],
   216 count=time_series_data[:, key_to_index['cnt']],
   217 minimum=time_series_data[:, key_to_index['min']],
   218 maximum=time_series_data[:, key_to_index['max']],
   219 )
   220 )
   222 return time_series_result
   
   File 
/opt/conda/lib/python3.10/site-packages/nexuscli-1.0-py3.10.egg/nexuscli/nexuscli.py:212,
 in (.0)
   206 time_series_data = np.array([tuple(each.values()) for each in 
[entry for entry in data if entry['ds'] == i]])
   208 if len(time_series_data) > 0:
   209 time_series_result.append(
   210 TimeSeries(
   211 dataset=response['meta'][i]['shortName'],
   --> 212 
time=np.array([datetime.utcfromtimestamp(t).replace(tzinfo=UTC) for t in
   213time_series_data[:, 
key_to_index['time']]]),
   214 mean=time_series_data[:, key_to_index['mean']],
   215 standard_deviation=time_series_data[:, 
key_to_index['std']],
   216 count=time_series_data[:, key_to_index['cnt']],
   217 minimum=time_series_data[:, key_to_index['min']],
   218 maximum=time_series_data[:, key_to_index['max']],
   219 )
   220 )
   222 return time_series_result
   
   TypeError: 'numpy.str_' object cannot be interpreted as an integer```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] dependabot[bot] commented on pull request #190: Bump lxml from 4.6.5 to 4.9.1 in /tools/cdms

[GitBox]

dependabot[bot] commented on PR #190:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/190#issuecomment-1211342374

   OK, I won't notify you again about this release, but will get in touch when 
a new version is available. If you'd rather skip all updates until the next 
major or minor version, let me know by commenting `@dependabot ignore this 
major version` or `@dependabot ignore this minor version`.
   
   If you change your mind, just re-open this PR and I'll resolve any conflicts 
on it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] dependabot[bot] commented on pull request #192: Bump numpy from 1.20.1 to 1.22.0 in /tools/cdms

[GitBox]

dependabot[bot] commented on PR #192:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/192#issuecomment-1211342387

   OK, I won't notify you again about this release, but will get in touch when 
a new version is available. If you'd rather skip all updates until the next 
major or minor version, let me know by commenting `@dependabot ignore this 
major version` or `@dependabot ignore this minor version`.
   
   If you change your mind, just re-open this PR and I'll resolve any conflicts 
on it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] dependabot[bot] commented on pull request #191: Bump pillow from 8.1.2 to 9.0.1 in /tools/cdms

[GitBox]

dependabot[bot] commented on PR #191:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/191#issuecomment-1211342379

   OK, I won't notify you again about this release, but will get in touch when 
a new version is available. If you'd rather skip all updates until the next 
major or minor version, let me know by commenting `@dependabot ignore this 
major version` or `@dependabot ignore this minor version`.
   
   If you change your mind, just re-open this PR and I'll resolve any conflicts 
on it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] dependabot[bot] commented on pull request #193: Bump protobuf from 3.14.0 to 3.15.0 in /tools/cdms

[GitBox]

dependabot[bot] commented on PR #193:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/193#issuecomment-1211342395

   OK, I won't notify you again about this release, but will get in touch when 
a new version is available. If you'd rather skip all updates until the next 
major or minor version, let me know by commenting `@dependabot ignore this 
major version` or `@dependabot ignore this minor version`.
   
   If you change your mind, just re-open this PR and I'll resolve any conflicts 
on it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung closed pull request #193: Bump protobuf from 3.14.0 to 3.15.0 in /tools/cdms

[GitBox]

ngachung closed pull request #193: Bump protobuf from 3.14.0 to 3.15.0 in 
/tools/cdms
URL: https://github.com/apache/incubator-sdap-nexus/pull/193


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung closed pull request #191: Bump pillow from 8.1.2 to 9.0.1 in /tools/cdms

[GitBox]

ngachung closed pull request #191: Bump pillow from 8.1.2 to 9.0.1 in 
/tools/cdms
URL: https://github.com/apache/incubator-sdap-nexus/pull/191


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung closed pull request #190: Bump lxml from 4.6.5 to 4.9.1 in /tools/cdms

[GitBox]

ngachung closed pull request #190: Bump lxml from 4.6.5 to 4.9.1 in /tools/cdms
URL: https://github.com/apache/incubator-sdap-nexus/pull/190


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung closed pull request #189: Bump mistune from 0.8.4 to 2.0.3 in /tools/cdms

[GitBox]

ngachung closed pull request #189: Bump mistune from 0.8.4 to 2.0.3 in 
/tools/cdms
URL: https://github.com/apache/incubator-sdap-nexus/pull/189


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung closed pull request #192: Bump numpy from 1.20.1 to 1.22.0 in /tools/cdms

[GitBox]

ngachung closed pull request #192: Bump numpy from 1.20.1 to 1.22.0 in 
/tools/cdms
URL: https://github.com/apache/incubator-sdap-nexus/pull/192


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] dependabot[bot] commented on pull request #189: Bump mistune from 0.8.4 to 2.0.3 in /tools/cdms

[GitBox]

dependabot[bot] commented on PR #189:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/189#issuecomment-1211342368

   OK, I won't notify you again about this release, but will get in touch when 
a new version is available. If you'd rather skip all updates until the next 
major or minor version, let me know by commenting `@dependabot ignore this 
major version` or `@dependabot ignore this minor version`.
   
   If you change your mind, just re-open this PR and I'll resolve any conflicts 
on it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung closed pull request #188: Bump notebook from 6.4.11 to 6.4.12 in /tools/cdms

[GitBox]

ngachung closed pull request #188: Bump notebook from 6.4.11 to 6.4.12 in 
/tools/cdms
URL: https://github.com/apache/incubator-sdap-nexus/pull/188


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #194: SDAP-390 Updated CDMS reader tool requirements.txt

[GitBox]

ngachung merged PR #194:
URL: https://github.com/apache/incubator-sdap-nexus/pull/194


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-in-situ-data-services] dependabot[bot] opened a new pull request, #8: chore(deps): bump pyspark from 3.1.2 to 3.1.3

[GitBox]

dependabot[bot] opened a new pull request, #8:
URL: https://github.com/apache/incubator-sdap-in-situ-data-services/pull/8

   Bumps [pyspark](https://github.com/apache/spark) from 3.1.2 to 3.1.3.
   
   Commits
   
   https://github.com/apache/spark/commit/d1f8a503a26bcfb4e466d9accc5fa241a7933667";>d1f8a50
 Preparing Spark release v3.1.3-rc4
   https://github.com/apache/spark/commit/7540421a2f26154be22852e876b9acfd004a85d3";>7540421
 Preparing development version 3.1.4-SNAPSHOT
   https://github.com/apache/spark/commit/b8c0799a8cef22c56132d94033759c9f82b0cc86";>b8c0799
 Preparing Spark release v3.1.3-rc3
   https://github.com/apache/spark/commit/0a7eda329bc33116677d0263b881495ae1aab409";>0a7eda3
 [SPARK-38075][SQL][3.1] Fix hasNext in 
HiveScriptTransformationExec's pro...
   https://github.com/apache/spark/commit/91db9a36a9ed74845908f14d21227d5267591653";>91db9a3
 Preparing development version 3.1.4-SNAPSHOT
   https://github.com/apache/spark/commit/fd2d694d783637e496a72166445b1853625c4cbd";>fd2d694
 Preparing Spark release v3.1.3-rc2
   https://github.com/apache/spark/commit/830d5b650ce9ac00f2a64bbf3e7fe9d31b02e51d";>830d5b6
 [SPARK-37860][UI] Fix taskindex in the stage page task event timeline
   https://github.com/apache/spark/commit/1a275396da176d9b0436066f0173ec24bc128ab9";>1a27539
 Preparing development version 3.1.4-SNAPSHOT
   https://github.com/apache/spark/commit/df89eb2d748c99665256d3ef297fef774db1014a";>df89eb2
 Preparing Spark release v3.1.3-rc1
   https://github.com/apache/spark/commit/94a69ffe0d26369004dcca8d9cb30057d2dcbf96";>94a69ff
 Revert "[SPARK-37779][SQL] Make ColumnarToRowExec plan canonicalizable 
after ...
   Additional commits viewable in https://github.com/apache/spark/compare/v3.1.2...v3.1.3";>compare 
view
   
   
   
   
   
   [![Dependabot compatibility 
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pyspark&package-manager=pip&previous-version=3.1.2&new-version=3.1.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
   
   Dependabot will resolve any conflicts with this PR as long as you don't 
alter it yourself. You can also trigger a rebase manually by commenting 
`@dependabot rebase`.
   
   [//]: # (dependabot-automerge-start)
   [//]: # (dependabot-automerge-end)
   
   ---
   
   
   Dependabot commands and options
   
   
   You can trigger Dependabot actions by commenting on this PR:
   - `@dependabot rebase` will rebase this PR
   - `@dependabot recreate` will recreate this PR, overwriting any edits that 
have been made to it
   - `@dependabot merge` will merge this PR after your CI passes on it
   - `@dependabot squash and merge` will squash and merge this PR after your CI 
passes on it
   - `@dependabot cancel merge` will cancel a previously requested merge and 
block automerging
   - `@dependabot reopen` will reopen this PR if it is closed
   - `@dependabot close` will close this PR and stop Dependabot recreating it. 
You can achieve the same result by closing it manually
   - `@dependabot ignore this major version` will close this PR and stop 
Dependabot creating any more for this major version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this minor version` will close this PR and stop 
Dependabot creating any more for this minor version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this dependency` will close this PR and stop 
Dependabot creating any more for this dependency (unless you reopen the PR or 
upgrade to it yourself)
   - `@dependabot use these labels` will set the current labels as the default 
for future PRs for this repo and language
   - `@dependabot use these reviewers` will set the current reviewers as the 
default for future PRs for this repo and language
   - `@dependabot use these assignees` will set the current assignees as the 
default for future PRs for this repo and language
   - `@dependabot use this milestone` will set the current milestone as the 
default for future PRs for this repo and language
   
   You can disable automated security fix PRs for this repo from the [Security 
Alerts 
page](https://github.com/apache/incubator-sdap-in-situ-data-services/network/alerts).
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper merged pull request #195: Fix changelog

[GitBox]

skorper merged PR #195:
URL: https://github.com/apache/incubator-sdap-nexus/pull/195


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #195: Fix changelog

[GitBox]

RKuttruff opened a new pull request, #195:
URL: https://github.com/apache/incubator-sdap-nexus/pull/195

   Minor formatting error + entry moved to correct section.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #183: SDAP-398 - CDMS Regression Test Script

[GitBox]

ngachung merged PR #183:
URL: https://github.com/apache/incubator-sdap-nexus/pull/183


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung opened a new pull request, #194: SDAP-390 Updated CDMS reader tool requirements.txt

[GitBox]

ngachung opened a new pull request, #194:
URL: https://github.com/apache/incubator-sdap-nexus/pull/194

   GitHub dependabot is flagging a lot of the specific dependencies. To run the 
tool we really only need netCDF4 library.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] dependabot[bot] opened a new pull request, #193: Bump protobuf from 3.14.0 to 3.15.0 in /tools/cdms

[GitBox]

dependabot[bot] opened a new pull request, #193:
URL: https://github.com/apache/incubator-sdap-nexus/pull/193

   Bumps [protobuf](https://github.com/protocolbuffers/protobuf) from 3.14.0 to 
3.15.0.
   
   Release notes
   Sourced from https://github.com/protocolbuffers/protobuf/releases";>protobuf's 
releases.
   
   Protocol Buffers v3.15.0
   Protocol Compiler
   
   Optional fields for proto3 are enabled by default, and no longer require
   the --experimental_allow_proto3_optional flag.
   
   C++
   
   MessageDifferencer: fixed bug when using custom ignore with multiple
   unknown fields
   Use init_seg in MSVC to push initialization to an earlier phase.
   Runtime no longer triggers -Wsign-compare warnings.
   Fixed -Wtautological-constant-out-of-range-compare warning.
   DynamicCastToGenerated works for nullptr input for even if RTTI is 
disabled
   Arena is refactored and optimized.
   Clarified/specified that the exact value of Arena::SpaceAllocated() is an
   implementation detail users must not rely on. It should not be used in
   unit tests.
   Change the signature of Any::PackFrom() to return false on error.
   Add fast reflection getter API for strings.
   Constant initialize the global message instances
   Avoid potential for missed wakeup in UnknownFieldSet
   Now Proto3 Oneof fields have "has" methods for checking their 
presence in
   C++.
   Bugfix for NVCC
   Return early in _InternalSerialize for empty maps.
   Adding functionality for outputting map key values in proto path logging
   output (does not affect comparison logic) and stop printing 'value' in the
   path. The modified print functionality is in the
   MessageDifferencer::StreamReporter.
   Fixed https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8129";>protocolbuffers/protobuf#8129
   Ensure that null char symbol, package and file names do not result in a
   crash.
   Constant initialize the global message instances
   Pretty print 'max' instead of numeric values in reserved ranges.
   Removed remaining instances of std::is_pod, which is deprecated in 
C++20.
   Changes to reduce code size for unknown field handling by making uncommon
   cases out of line.
   Fix std::is_pod deprecated in C++20 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/7180";>#7180)
   Fix some -Wunused-parameter warnings (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8053";>#8053)
   Fix detecting file as directory on zOS issue https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8051";>#8051
 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8052";>#8052)
   Don't include sys/param.h for _BYTE_ORDER (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8106";>#8106)
   remove CMAKE_THREAD_LIBS_INIT from pkgconfig CFLAGS (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8154";>#8154)
   Fix TextFormatMapTest.DynamicMessage issue#5136 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8159";>#8159)
   Fix for compiler warning issue#8145 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8160";>#8160)
   fix: support deprecated enums for GCC < 6 (https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8164";>#8164)
   Fix some warning when compiling with Visual Studio 2019 on x64 target 
(https://github-redirect.dependabot.com/protocolbuffers/protobuf/issues/8125";>#8125)
   
   Python
   
   Provided an override for the reverse() method that will reverse the 
internal
   collection directly instead of using the other methods of the 
BaseContainer.
   MessageFactory.CreateProtoype can be overridden to customize class 
creation.
   
   
   
   ... (truncated)
   
   
   Commits
   
   https://github.com/protocolbuffers/protobuf/commit/ae50d9b9902526efd6c7a1907d09739f959c6297";>ae50d9b
 Update protobuf version
   https://github.com/protocolbuffers/protobuf/commit/8260126500f073894c38b2211d383442eb7e58d5";>8260126
 Update protobuf version
   https://github.com/protocolbuffers/protobuf/commit/c741c4660443da46bfaf6c96fd1f01e743f97b56";>c741c46
 Resovled issue in the .pb.cc files
   https://github.com/protocolbuffers/protobuf/commit/eef276412ec417e2f0563c1c51d59a968cb2a6db";>eef2764
 Resolved an issue where NO_DESTROY and CONSTINIT were in incorrect order
   https://github.com/protocolbuffers/protobuf/commit/0040102e6f001724194d9c4c6bcd5effcec52bf1";>0040102
 Updated collect_all_artifacts.sh for Ubuntu Xenial
   https://github.com/protocolbuffers/protobuf/commit/26cb6a7a6d7973657f09fc44b201287c046da62e";>26cb6a7
 Delete root-owned files in Kokoro builds
   https://github.com/protocolbuffers/protobuf/commit/1e924efa90b476d23157426ad1eb17301bc4df41";>1e924ef
 Update port_def.inc
   https://github.com/protocolbuffers/protobuf/commit/9a80cf12254dec1af833d9fb78a80673dce45dc4";>9a80cf1
 Update coded_stream.h
   https://github.com/protocolbuffers/protobuf/com

[GitHub] [incubator-sdap-nexus] dependabot[bot] opened a new pull request, #192: Bump numpy from 1.20.1 to 1.22.0 in /tools/cdms

[GitBox]

dependabot[bot] opened a new pull request, #192:
URL: https://github.com/apache/incubator-sdap-nexus/pull/192

   Bumps [numpy](https://github.com/numpy/numpy) from 1.20.1 to 1.22.0.
   
   Release notes
   Sourced from https://github.com/numpy/numpy/releases";>numpy's releases.
   
   v1.22.0
   NumPy 1.22.0 Release Notes
   NumPy 1.22.0 is a big release featuring the work of 153 contributors
   spread over 609 pull requests. There have been many improvements,
   highlights are:
   
   Annotations of the main namespace are essentially complete. Upstream
   is a moving target, so there will likely be further improvements,
   but the major work is done. This is probably the most user visible
   enhancement in this release.
   A preliminary version of the proposed Array-API is provided. This is
   a step in creating a standard collection of functions that can be
   used across application such as CuPy and JAX.
   NumPy now has a DLPack backend. DLPack provides a common interchange
   format for array (tensor) data.
   New methods for quantile, percentile, and 
related functions. The
   new methods provide a complete set of the methods commonly found in
   the literature.
   A new configurable allocator for use by downstream projects.
   
   These are in addition to the ongoing work to provide SIMD support for
   commonly used functions, improvements to F2PY, and better documentation.
   The Python versions supported in this release are 3.8-3.10, Python 3.7
   has been dropped. Note that 32 bit wheels are only provided for Python
   3.8 and 3.9 on Windows, all other wheels are 64 bits on account of
   Ubuntu, Fedora, and other Linux distributions dropping 32 bit support.
   All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should 
fix
   the occasional problems encountered by folks using truly huge arrays.
   Expired deprecations
   Deprecated numeric style dtype strings have been removed
   Using the strings "Bytes0", 
"Datetime64", "Str0", 
"Uint32",
   and "Uint64" as a dtype will now raise a 
TypeError.
   (https://github-redirect.dependabot.com/numpy/numpy/pull/19539";>gh-19539)
   Expired deprecations for loads, ndfromtxt, and 
mafromtxt in npyio
   numpy.loads was deprecated in v1.15, with the recommendation 
that
   users use pickle.loads instead. ndfromtxt and 
mafromtxt were both
   deprecated in v1.17 - users should use numpy.genfromtxt instead 
with
   the appropriate value for the usemask parameter.
   (https://github-redirect.dependabot.com/numpy/numpy/pull/19615";>gh-19615)
   
   
   ... (truncated)
   
   
   Commits
   
   https://github.com/numpy/numpy/commit/4adc87dff15a247e417d50f10cc4def8e1c17a03";>4adc87d
 Merge pull request https://github-redirect.dependabot.com/numpy/numpy/issues/20685";>#20685
 from charris/prepare-for-1.22.0-release
   https://github.com/numpy/numpy/commit/fd66547557f57c430d41be2fc0764f74a62e8ccf";>fd66547
 REL: Prepare for the NumPy 1.22.0 release.
   https://github.com/numpy/numpy/commit/125304b035effcd82e366e601b102e7347eaa9ba";>125304b
 wip
   https://github.com/numpy/numpy/commit/c283859128b1a4b57014581570a23ed7950a24ea";>c283859
 Merge pull request https://github-redirect.dependabot.com/numpy/numpy/issues/20682";>#20682
 from charris/backport-20416
   https://github.com/numpy/numpy/commit/5399c03d4a069fe81a1616be0184c9749d7271ee";>5399c03
 Merge pull request https://github-redirect.dependabot.com/numpy/numpy/issues/20681";>#20681
 from charris/backport-20954
   https://github.com/numpy/numpy/commit/f9c45f8ebf31340b1a5a0371bfca25afcfc4794e";>f9c45f8
 Merge pull request https://github-redirect.dependabot.com/numpy/numpy/issues/20680";>#20680
 from charris/backport-20663
   https://github.com/numpy/numpy/commit/794b36f7e1bf2a8c42774ab0db86a74bd32f674b";>794b36f
 Update armccompiler.py
   https://github.com/numpy/numpy/commit/d93b14e3d7abaa1d837825e51671f817788e120f";>d93b14e
 Update test_public_api.py
   https://github.com/numpy/numpy/commit/7662c0789cc6a70d5ad4d950ee2e95f3afef7df6";>7662c07
 Update init.py
   https://github.com/numpy/numpy/commit/311ab52488a7d096ac3bc4c2de0fdae17ecd13ef";>311ab52
 Update armccompiler.py
   Additional commits viewable in https://github.com/numpy/numpy/compare/v1.20.1...v1.22.0";>compare 
view
   
   
   
   
   
   [![Dependabot compatibility 
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=numpy&package-manager=pip&previous-version=1.20.1&new-version=1.22.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
   
   Dependabot will resolve any conflicts with this PR as long as you don't 
alter it yourself. You can also trigger a rebase manually by commenting 
`@dependabot rebase`.
   
   [//]: # (dependabot-automerge-start)
   [//]: # (dependabot-automerge-end)
   
   ---
   
   
   Dependabot commands and options
   
   
   You can trigger Dependabot actions by commenting on this PR:
   

[GitHub] [incubator-sdap-nexus] dependabot[bot] opened a new pull request, #190: Bump lxml from 4.6.5 to 4.9.1 in /tools/cdms

[GitBox]

dependabot[bot] opened a new pull request, #190:
URL: https://github.com/apache/incubator-sdap-nexus/pull/190

   Bumps [lxml](https://github.com/lxml/lxml) from 4.6.5 to 4.9.1.
   
   Changelog
   Sourced from https://github.com/lxml/lxml/blob/master/CHANGES.txt";>lxml's 
changelog.
   
   4.9.1 (2022-07-01)
   Bugs fixed
   
   A crash was resolved when using iterwalk() (or 
canonicalize())
   after parsing certain incorrect input.  Note that iterwalk() 
can crash
   on valid input parsed with the same parser after failing 
to parse the
   incorrect input.
   
   4.9.0 (2022-06-01)
   Bugs fixed
   
   GH#341: The mixin inheritance order in lxml.html was 
corrected.
   Patch by xmo-odoo.
   
   Other changes
   
   
   Built with Cython 0.29.30 to adapt to changes in Python 3.11 and 3.12.
   
   
   Wheels include zlib 1.2.12, libxml2 2.9.14 and libxslt 1.1.35
   (libxml2 2.9.12+ and libxslt 1.1.34 on Windows).
   
   
   GH#343: Windows-AArch64 build support in Visual Studio.
   Patch by Steve Dower.
   
   
   4.8.0 (2022-02-17)
   Features added
   
   
   GH#337: Path-like objects are now supported throughout the API instead of 
just strings.
   Patch by Henning Janssen.
   
   
   The ElementMaker now supports QName values as 
tags, which always override
   the default namespace of the factory.
   
   
   Bugs fixed
   
   GH#338: In lxml.objectify, the XSI float annotation "nan" and 
"inf" were spelled in
   lower case, whereas XML Schema datatypes define them as "NaN" and 
"INF" respectively.
   
   
   
   ... (truncated)
   
   
   Commits
   
   https://github.com/lxml/lxml/commit/d01872ccdf7e1e5e825b6c6292b43e7d27ae5fc4";>d01872c
 Prevent parse failure in new test from leaking into later test runs.
   https://github.com/lxml/lxml/commit/d65e63229e8958bc08344a85cd3f09ceeef933c3";>d65e632
 Prepare release of lxml 4.9.1.
   https://github.com/lxml/lxml/commit/86368e9cf70a0ad23cccd5ee32de847149af0c6f";>86368e9
 Fix a crash when incorrect parser input occurs together with usages of 
iterwa...
   https://github.com/lxml/lxml/commit/50c276412880c1a3dde8a6d6c909e3ed8ef47e43";>50c2764
 Delete unused Travis CI config and reference in docs (https://github-redirect.dependabot.com/lxml/lxml/issues/345";>GH-345)
   https://github.com/lxml/lxml/commit/8f0bf2d158f2dd3f98d410c8a38fcd536fd11b53";>8f0bf2d
 Try to speed up the musllinux AArch64 build by splitting the different 
CPytho...
   https://github.com/lxml/lxml/commit/b9f7074430594b95824059eef931dfbb27a7645e";>b9f7074
 Remove debug print from test.
   https://github.com/lxml/lxml/commit/b224e0f69dde58425d1077e07d193d19d3f803a9";>b224e0f
 Try to install 'xz' in wheel builds, if available, since it's now needed to 
e...
   https://github.com/lxml/lxml/commit/897ebfa002fe5ec773ffe8851721047fedcc6928";>897ebfa
 Update macOS deployment target version from 10.14 to 10.15 since 10.14 
starts...
   https://github.com/lxml/lxml/commit/853c9e9cbf1c82d1ad3c096362372a048108905e";>853c9e9
 Prepare release of 4.9.0.
   https://github.com/lxml/lxml/commit/d3f77e678a8394559331d27257714e8aa4b082f2";>d3f77e6
 Add a test for https://bugs.launchpad.net/lxml/+bug/1965070";>https://bugs.launchpad.net/lxml/+bug/1965070
 leaving out the a...
   Additional commits viewable in https://github.com/lxml/lxml/compare/lxml-4.6.5...lxml-4.9.1";>compare 
view
   
   
   
   
   
   [![Dependabot compatibility 
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=lxml&package-manager=pip&previous-version=4.6.5&new-version=4.9.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
   
   Dependabot will resolve any conflicts with this PR as long as you don't 
alter it yourself. You can also trigger a rebase manually by commenting 
`@dependabot rebase`.
   
   [//]: # (dependabot-automerge-start)
   [//]: # (dependabot-automerge-end)
   
   ---
   
   
   Dependabot commands and options
   
   
   You can trigger Dependabot actions by commenting on this PR:
   - `@dependabot rebase` will rebase this PR
   - `@dependabot recreate` will recreate this PR, overwriting any edits that 
have been made to it
   - `@dependabot merge` will merge this PR after your CI passes on it
   - `@dependabot squash and merge` will squash and merge this PR after your CI 
passes on it
   - `@dependabot cancel merge` will cancel a previously requested merge and 
block automerging
   - `@dependabot reopen` will reopen this PR if it is closed
   - `@dependabot close` will close this PR and stop Dependabot recreating it. 
You can achieve the same result by closing it manually
   - `@dependabot ignore this major version` will close this PR and stop 
Dependabot creating any more for this major version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this minor version` will close this PR and stop 
Dependabot creating any more for this minor version (unless you reopen the PR

[GitHub] [incubator-sdap-nexus] dependabot[bot] opened a new pull request, #188: Bump notebook from 6.4.11 to 6.4.12 in /tools/cdms

[GitBox]

dependabot[bot] opened a new pull request, #188:
URL: https://github.com/apache/incubator-sdap-nexus/pull/188

   Bumps [notebook](http://jupyter.org) from 6.4.11 to 6.4.12.
   
   
   [![Dependabot compatibility 
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=notebook&package-manager=pip&previous-version=6.4.11&new-version=6.4.12)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
   
   Dependabot will resolve any conflicts with this PR as long as you don't 
alter it yourself. You can also trigger a rebase manually by commenting 
`@dependabot rebase`.
   
   [//]: # (dependabot-automerge-start)
   [//]: # (dependabot-automerge-end)
   
   ---
   
   
   Dependabot commands and options
   
   
   You can trigger Dependabot actions by commenting on this PR:
   - `@dependabot rebase` will rebase this PR
   - `@dependabot recreate` will recreate this PR, overwriting any edits that 
have been made to it
   - `@dependabot merge` will merge this PR after your CI passes on it
   - `@dependabot squash and merge` will squash and merge this PR after your CI 
passes on it
   - `@dependabot cancel merge` will cancel a previously requested merge and 
block automerging
   - `@dependabot reopen` will reopen this PR if it is closed
   - `@dependabot close` will close this PR and stop Dependabot recreating it. 
You can achieve the same result by closing it manually
   - `@dependabot ignore this major version` will close this PR and stop 
Dependabot creating any more for this major version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this minor version` will close this PR and stop 
Dependabot creating any more for this minor version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this dependency` will close this PR and stop 
Dependabot creating any more for this dependency (unless you reopen the PR or 
upgrade to it yourself)
   - `@dependabot use these labels` will set the current labels as the default 
for future PRs for this repo and language
   - `@dependabot use these reviewers` will set the current reviewers as the 
default for future PRs for this repo and language
   - `@dependabot use these assignees` will set the current assignees as the 
default for future PRs for this repo and language
   - `@dependabot use this milestone` will set the current milestone as the 
default for future PRs for this repo and language
   
   You can disable automated security fix PRs for this repo from the [Security 
Alerts page](https://github.com/apache/incubator-sdap-nexus/network/alerts).
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] dependabot[bot] opened a new pull request, #191: Bump pillow from 8.1.2 to 9.0.1 in /tools/cdms

[GitBox]

dependabot[bot] opened a new pull request, #191:
URL: https://github.com/apache/incubator-sdap-nexus/pull/191

   Bumps [pillow](https://github.com/python-pillow/Pillow) from 8.1.2 to 9.0.1.
   
   Release notes
   Sourced from https://github.com/python-pillow/Pillow/releases";>pillow's 
releases.
   
   9.0.1
   https://pillow.readthedocs.io/en/stable/releasenotes/9.0.1.html";>https://pillow.readthedocs.io/en/stable/releasenotes/9.0.1.html
   Changes
   
   In show_file, use os.remove to remove temporary images. CVE-2022-24303 
https://github-redirect.dependabot.com/python-pillow/Pillow/issues/6010";>#6010
 [https://github.com/radarhere";>@​radarhere, https://github.com/hugovk";>@​hugovk]
   Restrict builtins within lambdas for ImageMath.eval. CVE-2022-22817 https://github-redirect.dependabot.com/python-pillow/Pillow/issues/6009";>#6009
 [radarhere]
   
   9.0.0
   https://pillow.readthedocs.io/en/stable/releasenotes/9.0.0.html";>https://pillow.readthedocs.io/en/stable/releasenotes/9.0.0.html
   Changes
   
   Restrict builtins for ImageMath.eval() https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5923";>#5923
 [https://github.com/radarhere";>@​radarhere]
   Ensure JpegImagePlugin stops at the end of a truncated file https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5921";>#5921
 [https://github.com/radarhere";>@​radarhere]
   Fixed ImagePath.Path array handling https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5920";>#5920
 [https://github.com/radarhere";>@​radarhere]
   Remove consecutive duplicate tiles that only differ by their offset https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5919";>#5919
 [https://github.com/radarhere";>@​radarhere]
   Removed redundant part of condition https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5915";>#5915
 [https://github.com/radarhere";>@​radarhere]
   Explicitly enable strip chopping for large uncompressed TIFFs https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5517";>#5517
 [https://github.com/kmilos";>@​kmilos]
   Use the Windows method to get TCL functions on Cygwin https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5807";>#5807
 [https://github.com/DWesl";>@​DWesl]
   Changed error type to allow for incremental WebP parsing https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5404";>#5404
 [https://github.com/radarhere";>@​radarhere]
   Improved I;16 operations on big endian https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5901";>#5901
 [https://github.com/radarhere";>@​radarhere]
   Ensure that BMP pixel data offset does not ignore palette https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5899";>#5899
 [https://github.com/radarhere";>@​radarhere]
   Limit quantized palette to number of colors https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5879";>#5879
 [https://github.com/radarhere";>@​radarhere]
   Use latin1 encoding to decode bytes https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5870";>#5870
 [https://github.com/radarhere";>@​radarhere]
   Fixed palette index for zeroed color in FASTOCTREE quantize https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5869";>#5869
 [https://github.com/radarhere";>@​radarhere]
   When saving RGBA to GIF, make use of first transparent palette entry https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5859";>#5859
 [https://github.com/radarhere";>@​radarhere]
   Pass SAMPLEFORMAT to libtiff https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5848";>#5848
 [https://github.com/radarhere";>@​radarhere]
   Added rounding when converting P and PA https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5824";>#5824
 [https://github.com/radarhere";>@​radarhere]
   Improved putdata() documentation and data handling https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5910";>#5910
 [https://github.com/radarhere";>@​radarhere]
   Exclude carriage return in PDF regex to help prevent ReDoS https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5912";>#5912
 [https://github.com/radarhere";>@​radarhere]
   Image.NONE is only used for resampling and dithers https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5908";>#5908
 [https://github.com/radarhere";>@​radarhere]
   Fixed freeing pointer in ImageDraw.Outline.transform https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5909";>#5909
 [https://github.com/radarhere";>@​radarhere]
   Add Tidelift alignment action and badge https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5763";>#5763
 [https://github.com/aclark4life";>@​aclark4life]
   Replaced further direct invocations of setup.py https://github-redirect.dependabot.com/python-pillow/Pillow/issues/5906";>#5906
 [https://github.com/radarhere";>@​radarhere]
   Added ImageShow support for xdg-open https://github-r

[GitHub] [incubator-sdap-nexus] dependabot[bot] opened a new pull request, #189: Bump mistune from 0.8.4 to 2.0.3 in /tools/cdms

[GitBox]

dependabot[bot] opened a new pull request, #189:
URL: https://github.com/apache/incubator-sdap-nexus/pull/189

   Bumps [mistune](https://github.com/lepture/mistune) from 0.8.4 to 2.0.3.
   
   Release notes
   Sourced from https://github.com/lepture/mistune/releases";>mistune's 
releases.
   
   Version 2.0.2
   Fix escape_url  via https://github-redirect.dependabot.com/lepture/mistune/pull/295";>lepture/mistune#295
   Version 2.0.1
   Fix XSS for image link syntax.
   Version 2.0.0
   First release of Mistune v2.
   Version 2.0.0 RC1
   In this release, we have a Security Fix for harmful 
links.
   Version 2.0.0 Alpha 1
   This is the first release of v2. An alpha version for users to have a 
preview of the new mistune.
   
   
   
   Changelog
   Sourced from https://github.com/lepture/mistune/blob/master/docs/changes.rst";>mistune's
 changelog.
   
   Changelog
   Here is the full history of mistune v2.
   Version 2.0.4
   
   Released on Jul 15, 2022
   
   Fix url plugin in <a> tag
   Fix * formatting
   
   Version 2.0.3
   
   Released on Jun 27, 2022
   
   Fix table plugin
   Security fix for CVE-2022-34749
   
   Version 2.0.2
   
   Released on Jan 14, 2022
   Fix escape_url
   Version 2.0.1
   
   Released on Dec 30, 2021
   XSS fix for image link syntax.
   Version 2.0.0
   
   Released on Dec 5, 2021
   This is the first non-alpha release of mistune v2.
   Version 2.0.0rc1
   
   Released on Feb 16, 2021
   Version 2.0.0a6
   
    
   
   
   ... (truncated)
   
   
   Commits
   
   https://github.com/lepture/mistune/commit/3f422f1e84edae0f39756c45be453ecde534b755";>3f422f1
 Version bump 2.0.3
   https://github.com/lepture/mistune/commit/a6d43215132fe4f3d93f8d7e90ba83b16a0838b2";>a6d4321
 Fix asteris emphasis regex CVE-2022-34749
   https://github.com/lepture/mistune/commit/5638e460459cb59ceb20e4ce4716c802d4d73c53";>5638e46
 Merge pull request https://github-redirect.dependabot.com/lepture/mistune/issues/307";>#307
 from jieter/patch-1
   https://github.com/lepture/mistune/commit/0eba47196a81453bafe1f2492748a87475063dff";>0eba471
 Fix typo in guide.rst
   https://github.com/lepture/mistune/commit/61e9337884e20f9f8fdc0b7788d319afdd259729";>61e9337
 Fix table plugin
   https://github.com/lepture/mistune/commit/76dec68c4514c2612ef9263b49c6ec7f4d77bd14";>76dec68
 Add documentation for renderer heading when TOC enabled
   https://github.com/lepture/mistune/commit/799cd118cc5e664b72e98410ce1b68645f1a38c0";>799cd11
 Version bump 2.0.2
   https://github.com/lepture/mistune/commit/babb0cfa57a983ead615286a2b7c8f6885c46721";>babb0cf
 Merge pull request https://github-redirect.dependabot.com/lepture/mistune/issues/295";>#295
 from dairiki/bug.escape_url
   https://github.com/lepture/mistune/commit/fc2cd53d7698e432ab5b250ffac53458263a49e2";>fc2cd53
 Make mistune.util.escape_url less aggressive
   https://github.com/lepture/mistune/commit/3e8d35215120ac82176f300dd5e20c0bea5464ea";>3e8d352
 Version bump 2.0.1
   Additional commits viewable in https://github.com/lepture/mistune/compare/v0.8.4...v2.0.3";>compare 
view
   
   
   
   
   
   [![Dependabot compatibility 
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=mistune&package-manager=pip&previous-version=0.8.4&new-version=2.0.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
   
   Dependabot will resolve any conflicts with this PR as long as you don't 
alter it yourself. You can also trigger a rebase manually by commenting 
`@dependabot rebase`.
   
   [//]: # (dependabot-automerge-start)
   [//]: # (dependabot-automerge-end)
   
   ---
   
   
   Dependabot commands and options
   
   
   You can trigger Dependabot actions by commenting on this PR:
   - `@dependabot rebase` will rebase this PR
   - `@dependabot recreate` will recreate this PR, overwriting any edits that 
have been made to it
   - `@dependabot merge` will merge this PR after your CI passes on it
   - `@dependabot squash and merge` will squash and merge this PR after your CI 
passes on it
   - `@dependabot cancel merge` will cancel a previously requested merge and 
block automerging
   - `@dependabot reopen` will reopen this PR if it is closed
   - `@dependabot close` will close this PR and stop Dependabot recreating it. 
You can achieve the same result by closing it manually
   - `@dependabot ignore this major version` will close this PR and stop 
Dependabot creating any more for this major version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this minor version` will close this PR and stop 
Dependabot creating any more for this minor version (unless you reopen the PR 
or upgrade to it yourself)
   - `@dependabot ignore this dependency` will close this PR and stop 
Dependabot creating any more for this dependency (unless you reopen the PR or 
upgrade to it yourself)
   - `@dependabot use these labels` will 

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #178: SDAP-390 Update NetCDF reader tool for data match-up

[GitBox]

ngachung merged PR #178:
URL: https://github.com/apache/incubator-sdap-nexus/pull/178


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #186: Fixed error causing timeSeriesSpark to fail

[GitBox]

ngachung merged PR #186:
URL: https://github.com/apache/incubator-sdap-nexus/pull/186


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #185: CDMS-159: Quickstart update

[GitBox]

RKuttruff commented on PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#issuecomment-1205848293

   Currently investigating deployment with kubernetes. Will write second guide 
for that and add to this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] tloubrieu-jpl opened a new pull request, #187: add a script to generate collection configuration from a remote SDAP

[GitBox]

tloubrieu-jpl opened a new pull request, #187:
URL: https://github.com/apache/incubator-sdap-nexus/pull/187

   This pull request adds a small script to generate a piece of 
collection-config configmap to present remote SDAP collections in the local 
SDAP server.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff commented on pull request #185: CDMS-159: Quickstart update

[GitBox]

RKuttruff commented on PR #185:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/185#issuecomment-1205649728

   Currently uses an older version of the nexus webapp image. Changes made to 
sdap after this image was made causes the `/timeSeriesSpark` endpoint to fail 
(This endpoint is the one used in the demo Jupyter notebook). I have 
implemented a patch for this issue in PR #186, pending approval, merging, and 
build + push of new image. If that happens, I will update the quickstart guide 
to use a more updated image.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #186: Fixed error in timeSeriesSpark

[GitBox]

RKuttruff opened a new pull request, #186:
URL: https://github.com/apache/incubator-sdap-nexus/pull/186

   It appeared to have been caused by an import issue in RedirectHandler.py
   
   Updated import statements and tested with local SDAP. Verified results 
against SDAP image preceding the error.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #185: CDMS-159: Quickstart update

[GitBox]

RKuttruff opened a new pull request, #185:
URL: https://github.com/apache/incubator-sdap-nexus/pull/185

   Updated quickstart guide for standalone SDAP deployment with Docker.
   
   Draft PR since this guide still needs to be verified and tested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #165: Fix matchup bugs

[GitBox]

ngachung merged PR #165:
URL: https://github.com/apache/incubator-sdap-nexus/pull/165


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung commented on a diff in pull request #178: SDAP-390 Update NetCDF reader tool for data match-up

[GitBox]

ngachung commented on code in PR #178:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/178#discussion_r935888956


##
tools/cdms/cdms_reader.py:
##
@@ -0,0 +1,250 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import string
+from netCDF4 import Dataset, num2date
+import sys
+import datetime
+import csv
+from collections import OrderedDict
+import logging
+
+#TODO: Get rid of numpy errors?
+#TODO: Update big SDAP README
+
+LOGGER =  logging.getLogger("cdms_reader")
+
+def assemble_matches(filename):
+"""
+Read a CDMS netCDF file and return a list of matches.
+
+Parameters
+--
+filename : str
+The CDMS netCDF file name.
+
+Returns
+---
+matches : list
+List of matches. Each list element is a dictionary.
+For match m, netCDF group GROUP (SatelliteData or InsituData), and

Review Comment:
   It looks like in a recent change we've replaced SatelliteData / InsituData 
groups with PrimaryData and SecondaryData. See 
https://github.com/apache/incubator-sdap-nexus/blob/master/analysis/webservice/algorithms/doms/BaseDomsHandler.py#L317-L318
 Can you please update references here and in README.md accordingly? Thank you!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung commented on pull request #166: CDMS - 150 NETCDF Reader Updates

[GitBox]

ngachung commented on PR #166:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/166#issuecomment-1202991601

   This PR has been superseded by 
https://github.com/apache/incubator-sdap-nexus/pull/178 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung closed pull request #166: CDMS - 150 NETCDF Reader Updates

[GitBox]

ngachung closed pull request #166: CDMS - 150 NETCDF Reader Updates
URL: https://github.com/apache/incubator-sdap-nexus/pull/166


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #181: Preserve insitu schema

[GitBox]

ngachung merged PR #181:
URL: https://github.com/apache/incubator-sdap-nexus/pull/181


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #182: Fix NetCDF output

[GitBox]

ngachung merged PR #182:
URL: https://github.com/apache/incubator-sdap-nexus/pull/182


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] dependabot[bot] opened a new pull request, #184: Bump urllib3 from 1.26.2 to 1.26.5 in /data-access

[GitBox]

dependabot[bot] opened a new pull request, #184:
URL: https://github.com/apache/incubator-sdap-nexus/pull/184

   Bumps [urllib3](https://github.com/urllib3/urllib3) from 1.26.2 to 1.26.5.
   
   Release notes
   Sourced from https://github.com/urllib3/urllib3/releases";>urllib3's 
releases.
   
   1.26.5
   :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 
2: https://urllib3.readthedocs.io/en/latest/v2-roadmap.html";>Read more in 
the v2.0 Roadmap
   
   Fixed deprecation warnings emitted in Python 3.10.
   Updated vendored six library to 1.16.0.
   Improved performance of URL parser when splitting the authority 
component.
   
   If you or your organization rely on urllib3 consider supporting 
us via https://github.com/sponsors/urllib3";>GitHub 
Sponsors
   1.26.4
   :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 
2: https://urllib3.readthedocs.io/en/latest/v2-roadmap.html";>Read more in 
the v2.0 Roadmap
   
   Changed behavior of the default SSLContext when connecting 
to HTTPS proxy during HTTPS requests. The default SSLContext now 
sets check_hostname=True.
   
   If you or your organization rely on urllib3 consider supporting 
us via https://github.com/sponsors/urllib3";>GitHub 
Sponsors
   1.26.3
   :warning: IMPORTANT: urllib3 v2.0 will drop support for Python 
2: https://urllib3.readthedocs.io/en/latest/v2-roadmap.html";>Read more in 
the v2.0 Roadmap
   
   
   Fixed bytes and string comparison issue with headers (Pull https://github-redirect.dependabot.com/urllib3/urllib3/issues/2141";>#2141)
   
   
   Changed ProxySchemeUnknown error message to be more 
actionable if the user supplies a proxy URL without a scheme (Pull https://github-redirect.dependabot.com/urllib3/urllib3/issues/2107";>#2107)
   
   
   If you or your organization rely on urllib3 consider supporting 
us via https://github.com/sponsors/urllib3";>GitHub 
Sponsors
   
   
   
   Changelog
   Sourced from https://github.com/urllib3/urllib3/blob/main/CHANGES.rst";>urllib3's 
changelog.
   
   1.26.5 (2021-05-26)
   
   Fixed deprecation warnings emitted in Python 3.10.
   Updated vendored six library to 1.16.0.
   Improved performance of URL parser when splitting
   the authority component.
   
   1.26.4 (2021-03-15)
   
   Changed behavior of the default SSLContext when connecting 
to HTTPS proxy
   during HTTPS requests. The default SSLContext now sets 
check_hostname=True.
   
   1.26.3 (2021-01-26)
   
   
   Fixed bytes and string comparison issue with headers (Pull https://github-redirect.dependabot.com/urllib3/urllib3/issues/2141";>#2141)
   
   
   Changed ProxySchemeUnknown error message to be
   more actionable if the user supplies a proxy URL without
   a scheme. (Pull https://github-redirect.dependabot.com/urllib3/urllib3/issues/2107";>#2107)
   
   
   
   
   
   Commits
   
   https://github.com/urllib3/urllib3/commit/d1616473df94b94f0f5ad19d2a6608cfe93b7cdf";>d161647
 Release 1.26.5
   https://github.com/urllib3/urllib3/commit/2d4a3fee6de2fa45eb82169361918f759269b4ec";>2d4a3fe
 Improve performance of sub-authority splitting in URL
   https://github.com/urllib3/urllib3/commit/2698537d52f8ff1f0bbb1d45cf018b118e91f637";>2698537
 Update vendored six to 1.16.0
   https://github.com/urllib3/urllib3/commit/07bed791e9c391d8bf12950f76537dc3c6f90550";>07bed79
 Fix deprecation warnings for Python 3.10 ssl module
   https://github.com/urllib3/urllib3/commit/d725a9b56bb8baf87c9e6eee0e9edf010034b63b";>d725a9b
 Add Python 3.10 to GitHub Actions
   https://github.com/urllib3/urllib3/commit/339ad34c677c98fd9ad008de1d8bbeb9dbf34381";>339ad34
 Use pytest==6.2.4 on Python 3.10+
   https://github.com/urllib3/urllib3/commit/f271c9c3149e20d7feffb6429b135bbb6c09ddf4";>f271c9c
 Apply latest Black formatting
   https://github.com/urllib3/urllib3/commit/1884878aac87ef0494b282e940c32c24ee917d52";>1884878
 [1.26] Properly proxy EOF on the SSLTransport test suite
   https://github.com/urllib3/urllib3/commit/a8913042b676c510e94fc2b097f6b514ae11a537";>a891304
 Release 1.26.4
   https://github.com/urllib3/urllib3/commit/8d65ea1ecf6e2cdc27d42124e587c1b83a3118b0";>8d65ea1
 Merge pull request from GHSA-5phf-pp7p-vc2r
   Additional commits viewable in https://github.com/urllib3/urllib3/compare/1.26.2...1.26.5";>compare 
view
   
   
   
   
   
   [![Dependabot compatibility 
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=pip&previous-version=1.26.2&new-version=1.26.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
   
   Dependabot will resolve any conflicts with this PR as long as you don't 
alter it yourself. You can also trigger a rebase manually by commenting 
`@dependabot rebase`.
   
   [//]: # (dependabot-automerge-start)
   [//]: # (dependabot-automerge-end)
   
   ---
   
   
   Dependabot commands and options
   
   
   You can trigger Dependabot actions by co

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #173: Add metadata to remote coll

[GitBox]

ngachung merged PR #173:
URL: https://github.com/apache/incubator-sdap-nexus/pull/173


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] tloubrieu-jpl commented on pull request #173: Add metadata to remote coll

[GitBox]

tloubrieu-jpl commented on PR #173:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/173#issuecomment-1194753764

   Hello @nga, @skorper, 
   
   Can you review this pull request ? I am afraid it gets buried under the 
other pull requests which came after.
   I need the pull request to be merged so that I can have this feature 
deployed at one of the partner's SDAP using an official SDAP docker image.
   
   Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #183: CDMS-151 - Regression Test Script

[GitBox]

RKuttruff opened a new pull request, #183:
URL: https://github.com/apache/incubator-sdap-nexus/pull/183

   Regression test script that tests a number of SDAP endpoints + the insitu 
endpoint. SDAP host and insitu endpoint can be configured through environment 
variables. Tests can also be manually skipped through environment variables.
   
   The test for `/domsresults` makes use of the CDMSReader provided in PR #178. 
For now, the script from that PR has been copied to the same directory, but 
should be replaced with a link when the CDMSReader PR is merged.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #182: Fix NetCDF output

[GitBox]

RKuttruff opened a new pull request, #182:
URL: https://github.com/apache/incubator-sdap-nexus/pull/182

   Latitude and longitude were switched.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #181: Preserve insitu schema

[GitBox]

RKuttruff opened a new pull request, #181:
URL: https://github.com/apache/incubator-sdap-nexus/pull/181

   Matchup makes repeated calls to `query_insitu_schema()` in the insitu 
module, which results in a severe performance penalty due to repeatedly hitting 
the insitu API for the same data. This pr is to fix that by preserving the 
return value from the first call and using that value to replace subsequent 
calls to minimize API utilization.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #179: SDAP-396: saildrone insitu

[GitBox]

ngachung merged PR #179:
URL: https://github.com/apache/incubator-sdap-nexus/pull/179


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #175: Fix CSV/NetCDF bug

[GitBox]

ngachung merged PR #175:
URL: https://github.com/apache/incubator-sdap-nexus/pull/175


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #180: SDAP-397: Requested matchup updates

[GitBox]

ngachung merged PR #180:
URL: https://github.com/apache/incubator-sdap-nexus/pull/180


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #176: Fix satellite to satellite matchup

[GitBox]

ngachung merged PR #176:
URL: https://github.com/apache/incubator-sdap-nexus/pull/176


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung commented on pull request #179: SDAP-396: saildrone insitu

[GitBox]

ngachung commented on PR #179:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/179#issuecomment-1190587866

   Thank you! That's expected as I filtered out only matches that contained 
mass_concentration_of_chlorophyll_in_sea_water.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper commented on pull request #179: SDAP-396: saildrone insitu

[GitBox]

skorper commented on PR #179:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/179#issuecomment-1190504757

   @ngachung Ran this same request locally using this branch -- it seemed to 
match to more than 6 points. See output below.
   
   I ran this through a script I wrote to verify the matchup results are within 
the expected tolerances/bounds and they are. Perhaps Saildrone has more data 
ingested than the doms insitu system did.
   
   
 Matchup output
 
   ```json
   {
   "executionId": "64f16c3f-faf9-4e18-be2a-978d020d6001",
   "data": [
   {
   "platform": "orbiting satellite",
   "device": "radiometers",
   "lon": "-129.875",
   "lat": "26.125",
   "point": "Point(-129.875 26.125)",
   "time": 1522584000,
   "fileurl": "2018040112-JPL-CHL25-fv03.nc",
   "id": "96bc2f4b-fd78-3c41-a3cb-cc6b2d7197a3[[0, 14, 20]]",
   "source": "JPL-L4-MRVA-CHLA-GLOB-v3.0",
   "primary": [
   {
   "variable_name": "CHLA_analysis",
   "cf_variable_name": 
"mass_concentration_of_chlorophyll_a_in_sea_water",
   "variable_value": 0.05919817090034485
   }
   ],
   "matches": [
   {
   "platform": "3B",
   "device": "130",
   "lon": "-130.012864",
   "lat": "26.4345008",
   "point": "Point(-130.012864 26.4345008)",
   "time": 1522670340,
   "fileurl": null,
   "id": "2018-04-02T11:59:00Z:-130.012864:26.4345008",
   "source": "shark-2018",
   "secondary": [
   {
   "variable_name": "sea_water_practical_salinity",
   "cf_variable_name": null,
   "variable_value": 34.69
   },
   {
   "variable_name": 
"sea_water_practical_salinity_quality",
   "cf_variable_name": null,
   "variable_value": 2
   },
   {
   "variable_name": "sea_water_temperature",
   "cf_variable_name": null,
   "variable_value": 19.419
   },
   {
   "variable_name": "sea_water_temperature_quality",
   "cf_variable_name": null,
   "variable_value": 2
   }
   ]
   }
   ]
   },
   {
   "platform": "orbiting satellite",
   "device": "radiometers",
   "lon": "-130.125",
   "lat": "26.875",
   "point": "Point(-130.125 26.875)",
   "time": 1522584000,
   "fileurl": "2018040112-JPL-CHL25-fv03.nc",
   "id": "96bc2f4b-fd78-3c41-a3cb-cc6b2d7197a3[[0, 17, 19]]",
   "source": "JPL-L4-MRVA-CHLA-GLOB-v3.0",
   "primary": [
   {
   "variable_name": "CHLA_analysis",
   "cf_variable_name": 
"mass_concentration_of_chlorophyll_a_in_sea_water",
   "variable_value": 0.05785427987575531
   }
   ],
   "matches": [
   {
   "platform": "3B",
   "device": "130",
   "lon": "-129.9747584",
   "lat": "26.8175136",
   "point": "Point(-129.9747584 26.8175136)",
   "time": 1522608780,
   "fileurl": null,
   "id": "2018-04-01T18:53:00Z:-129.9747584:26.8175136",
   "source": "shark-2018",
   "secondary": [
   {
   "variable_name": "sea_water_practical_salinity",
   "cf_variable_name": null,
   "variable_value": 34.565
   },
   {
   "variable_name": 
"sea_water_practical_salinity_quality",
   "cf_variable_name": null,
   "variable_value": 2
   },
   {
   "variable_name": "sea_water_temperature",
   "cf_variable_name": null,
   "variable_value": 18.797
   },
   {

[GitHub] [incubator-sdap-nexus] skorper commented on pull request #180: SDAP-397: Requested matchup updates

[GitBox]

skorper commented on PR #180:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/180#issuecomment-1190486320

   Latest update:
   
   - Query insitu schema endpoint to get units for insitu observations
   - Use "meta" field to populate "id" field for insitu
   - Moved depth to top level alongside lat/lon/time
   - I recall this being requested -- I am using the insitu variable name for 
both "variable_name" and "cf_variable_name"
   
   Sample output (truncated):
   
   ```json
   {
   "executionId": "2cb5b591-9b47-41a7-8e15-a5386a4132f6",
   "data": [
   {
   "platform": "orbiting satellite",
   "device": "radiometers",
   "lon": "161.125",
   "lat": "-28.375",
   "point": "Point(161.125 -28.375)",
   "time": 1537952400,
   "depth": null,
   "fileurl": 
"2018092609-JPL-L4_GHRSST-SSTfnd-MUR25-GLOB-v02.0-fv04.2.nc",
   "id": "47be9823-0b9e-3984-b562-2221c0396ada[[0, 6, 14]]",
   "source": "MUR25-JPL-L4-GLOB-v04.2",
   "primary": [
   {
   "variable_name": "analysed_sst",
   "cf_variable_name": "sea_surface_foundation_temperature",
   "variable_value": 19.394012451171875,
   "variable_unit": null
   }
   ],
   "matches": [
   {
   "platform": "42",
   "device": null,
   "lon": "161.12",
   "lat": "-28.37",
   "point": "Point(161.12 -28.37)",
   "time": 1538784000,
   "depth": -9.0,
   "fileurl": null,
   "id": 
"https://rda.ucar.edu/php/icoadsuid.php?uid=PAWNVF";,
   "source": "ICOADS Release 3.0",
   "secondary": [
   {
   "variable_name": "sea_water_temperature",
   "cf_variable_name": "sea_water_temperature",
   "variable_value": 18.9,
   "variable_unit": "C"
   },
   {
   "variable_name": "sea_water_temperature_quality",
   "cf_variable_name": 
"sea_water_temperature_quality",
   "variable_value": 1,
   "variable_unit": null
   }
   ]
   }
   ]
   }
   ],
   "params": {
   "primary": "MUR25-JPL-L4-GLOB-v04.2",
   "matchup": "ICOADS Release 3.0",
   "startTime": 1537747200,
   "endTime": 1538092800,
   "bbox": "160,-29,165,-28",
   "timeTolerance": 2592000,
   "radiusTolerance": 1000.0,
   "platforms": "42",
   "parameter": null,
   "depthMin": 0.0,
   "depthMax": 5.0
   },
   "bounds": {},
   "count": 11,
   "details": {
   "timeToComplete": 105,
   "numInSituRecords": 0,
   "numInSituMatched": 11,
   "numGriddedChecked": 0,
   "numGriddedMatched": 11
   }
   }
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper commented on pull request #180: SDAP-397: Requested matchup updates

[GitBox]

skorper commented on PR #180:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/180#issuecomment-1190433207

   Thanks for your review @ngachung.
   
   - I wondered that as well. The depth field should probably be at the root 
level alongside lat/lon/time.
   - Thank you! 
   - Thanks! I didn't realize that endpoint was completed.
   
   I will get these changes in today.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung commented on pull request #180: SDAP-397: Requested matchup updates

[GitBox]

ngachung commented on PR #180:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/180#issuecomment-1190348199

   Thank you, @skorper.
   
   - We can discuss at our next team meeting, but I wonder if depth should just 
be added as an attribute similar to lat, lon for both primary and secondary. 
Though for satellite measurements depth would always default to 0 or TBD.
   - Sorry the request to add ID was vague. What we meant was to return the 
value from meta, e.g. for ICOADS "meta": 
"https://rda.ucar.edu/php/icoadsuid.php?uid=ORQ2YD";. This is for provenance so 
we can identify the original source of the measurement. See 
https://github.jpl.nasa.gov/access-cdms/in-situ-data-services/blob/master/in_situ_schema.json#L2054-L2061
 Note this can be a string or object.
   - For in situ units, we decided that the matchup API will retrieve units 
from the schema which is returned through 
https://doms.jpl.nasa.gov/insitu/1.0/cdms_schema Satellite units TBD


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung commented on pull request #179: SDAP-396: saildrone insitu

[GitBox]

ngachung commented on PR #179:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/179#issuecomment-1190328943

   Thank you, @skorper. This is a call we used for the notebook demo that 
returned 6 matches. 
https://doms.jpl.nasa.gov/match_spark?primary=JPL-L4-MRVA-CHLA-GLOB-v3.0&secondary=shark-2018&startTime=2018-04-01T00%3A00%3A00Z&endTime=2018-04-01T23%3A59%3A59Z&tt=86400&rt=5&b=-140%2C10%2C-110%2C40&platforms=3B&depthMin=-5&depthMax=5&matchOnce=true&resultSizeLimit=100
 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper opened a new pull request, #180: SDAP-397: Requested matchup updates

[GitBox]

skorper opened a new pull request, #180:
URL: https://github.com/apache/incubator-sdap-nexus/pull/180

   https://issues.apache.org/jira/browse/SDAP-397
   
   - Added depth to matchup insitu output
 - Added in the "data" field (see example below). Is this a desirable 
location for this value?
   - Pull ID from insitu api response for matchup
 - Using ID from insitu `platforms` dict. That value doesn't seem unique 
though, so I'm not sure if there's a better field.
   - Added null unit field for later use in matchup
 - This metadata doesn't exist in the insitu API or satellite metadata so 
just using null for now.
   
   Notice I don't have a change for the last bullet "numInSituRecords and 
numGriddedChecked are both zero in completed query...". Adding the behavior 
described from Doms would be a significant change, so I'd like to discuss with 
the team if this is indeed the desired behavior and if there's value in this 
field.
   
   Example request:
   ```
   
{{big_data_url}}/match_spark?primary=MUR25-JPL-L4-GLOB-v04.2&startTime=2018-09-24T00:00:00Z&endTime=2018-09-30T00:00:00Z&tt=2592000&rt=1000&b=160,-30,180,-25&depthMin=0&depthMax=5&matchOnce=true&secondary=ICOADS%20Release%203.0&resultSizeLimit=6000&platforms=42
   ```
   
   Example output (truncated):
   
   ```json
   {
   "executionId": "8943d0a6-2513-4df1-83fb-536f1050fde8",
   "data": [
   {
   "platform": "orbiting satellite",
   "device": "radiometers",
   "lon": "179.375",
   "lat": "-27.125",
   "point": "Point(179.375 -27.125)",
   "time": 1537779600,
   "fileurl": 
"2018092409-JPL-L4_GHRSST-SSTfnd-MUR25-GLOB-v02.0-fv04.2.nc",
   "id": "3e1ed916-5681-3a33-8688-d6a70ec080eb[[0, 11, 27]]",
   "source": "MUR25-JPL-L4-GLOB-v04.2",
   "primary": [
   {
   "variable_name": "analysed_sst",
   "cf_variable_name": "sea_surface_foundation_temperature",
   "variable_value": 20.381988525390625,
   "variable_unit": null
   }
   ],
   "matches": [
   {
   "platform": "42",
   "device": null,
   "lon": "179.38",
   "lat": "-27.12",
   "point": "Point(179.38 -27.12)",
   "time": 1537624800,
   "fileurl": null,
   "id": "55948",
   "source": "ICOADS Release 3.0",
   "secondary": [
   {
   "variable_name": "sea_water_temperature",
   "cf_variable_name": null,
   "variable_value": 19.8,
   "variable_unit": null
   },
   {
   "variable_name": "sea_water_temperature_quality",
   "cf_variable_name": null,
   "variable_value": 1,
   "variable_unit": null
   },
   {
   "variable_name": "depth",
   "cf_variable_name": null,
   "variable_value": -9.0,
   "variable_unit": null
   }
   ]
   }
   ]
   }
   ],
   "params": {
   "primary": "MUR25-JPL-L4-GLOB-v04.2",
   "matchup": "ICOADS Release 3.0",
   "startTime": 1537747200,
   "endTime": 1538265600,
   "bbox": "160,-30,180,-25",
   "timeTolerance": 2592000,
   "radiusTolerance": 1000.0,
   "platforms": "42",
   "parameter": null,
   "depthMin": 0.0,
   "depthMax": 5.0
   },
   "bounds": {},
   "count": 54,
   "details": {
   "timeToComplete": 101,
   "numInSituRecords": 0,
   "numInSituMatched": 54,
   "numGriddedChecked": 0,
   "numGriddedMatched": 54
   }
   }
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper opened a new pull request, #179: SDAP-396: saildrone insitu

[GitBox]

skorper opened a new pull request, #179:
URL: https://github.com/apache/incubator-sdap-nexus/pull/179

   https://issues.apache.org/jira/browse/SDAP-396
   
   Added saildrone insitu API.
   
   I haven't had much luck finding a request that results in a matchup. I'm 
trying to find overlap with a satellite dataset. insitu calls seem to be 
executed as expected, though. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] JordanGethers opened a new pull request, #178: SDAP-390 Update NetCDF reader tool for data match-up

[GitBox]

JordanGethers opened a new pull request, #178:
URL: https://github.com/apache/incubator-sdap-nexus/pull/178

   cdms_reader.py is the original doms_reader.py code with additional 
functionality provided for the user to make each run of the program more 
customizable. The user may now select, with option flags, whether or whether 
not they would like to create a CSV, log, or text (holds global attributes) 
files associated with their original NETCDF file.
   
   * The ReadMe associated with this file has been updated to reflect changes 
in the program.
   * A requirements.txt file has been added so that users can run the code with 
its necessary dependencies
   
   Note:
   When I run the code in my linux environment, there are some (seemingly) 
harmless warning messages (pertaining to numPy and occasionally variable 
conversion).
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #177: Fix domslist count

[GitBox]

RKuttruff opened a new pull request, #177:
URL: https://github.com/apache/incubator-sdap-nexus/pull/177

   Count equalling insitu datasets + satellite datasets
   
   Confirmed working with quick test on local SDAP instance


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper opened a new pull request, #176: Fix satellite to satellite matchup

[GitBox]

skorper opened a new pull request, #176:
URL: https://github.com/apache/incubator-sdap-nexus/pull/176

   - Nga was seeing a 500 error when doing satellite to satellite matchup
 - Fixed satellite to satellite matchup failure -- was due to ` 'platform': 
None,` line
   - Nga's request was returning "null" instead of data array, even after 
fixing above error
 - Changed behavior where, if > `resultsSizeLimit` results are found, the 
first `resultsSizeLimit` are returned, rather than returning "None"
   - Nga's request was not returning expected results. The primary/secondary 
points were not identical, as expected.
 - This is due to the behavior of `matchOnce`. If `matchOnce` is selected, 
only one secondary point is returned. The secondary point that is chosen is the 
point that is closest to the primary point, spatially.
 - Updated this to choose the point based on both space AND time. I'm 
trying to think if there are any scenarios this wouldn't work?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] tloubrieu-jpl commented on a diff in pull request #174: CDMS-122 - Limited Zarr Proxy

[GitBox]

tloubrieu-jpl commented on code in PR #174:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/174#discussion_r921396416


##
analysis/webservice/algorithms_spark/Matchup.py:
##
@@ -218,12 +218,17 @@ def calc(self, request, **args):
 
 self.log.debug("Querying for tiles in search domain")
 # Get tile ids in box
-tile_ids = [tile.tile_id for tile in
-
self._get_tile_service().find_tiles_in_polygon(bounding_polygon, 
primary_ds_name,
- 
start_seconds_from_epoch, end_seconds_from_epoch,
- fetch_data=False, 
fl='id',
- 
sort=['tile_min_time_dt asc', 'tile_min_lon asc',
-   
'tile_min_lat asc'], rows=5000)]
+if not self._get_tile_service().supports_direct_bounds_to_tile():
+tile_ids = [tile.tile_id for tile in
+
self._get_tile_service().find_tiles_in_polygon(bounding_polygon, 
primary_ds_name,
+ 
start_seconds_from_epoch, end_seconds_from_epoch,
+ 
fetch_data=False, fl='id',
+ 
sort=['tile_min_time_dt asc', 'tile_min_lon asc',
+   
'tile_min_lat asc'], rows=5000)]
+else:
+bb = bounding_polygon.bounds
+
+tile_ids = 
[self._get_tile_service().bounds_to_direct_tile_id(bb[1], bb[0], bb[3], bb[2], 
start_time, end_time)]

Review Comment:
   If I understand well, that is a smart way of still using SDAP framework 
without need to use the solr index when the collection is stored in ZARR. Great 
job. So that is how you made the performances better.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung commented on pull request #175: Fix CSV/NetCDF bug

[GitBox]

ngachung commented on PR #175:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/175#issuecomment-1183782964

   Lgtm. Update CHANGELOG.md please?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper opened a new pull request, #175: Fix CSV/NetCDF bug

[GitBox]

skorper opened a new pull request, #175:
URL: https://github.com/apache/incubator-sdap-nexus/pull/175

   CSV and NetCDF requests were failing for AWS Insitu endpoints because there 
is no entry for those datasets in the `METADATA_LINKS` dict. Updated the code 
so that is optional. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-in-situ-data-services] ngachung merged pull request #7: feat: SDAP-395: CDMS JSON Schema Endpoint

[GitBox]

ngachung merged PR #7:
URL: https://github.com/apache/incubator-sdap-in-situ-data-services/pull/7


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] RKuttruff opened a new pull request, #174: CDMS-122 - Limited Zarr Proxy

[GitBox]

RKuttruff opened a new pull request, #174:
URL: https://github.com/apache/incubator-sdap-nexus/pull/174

   Implementation of a proxy to fetch Nexus data stored in an AWS S3 bucket as 
a Zarr array. 
   
   Currently only supports the MUR25-JPL-L4-GLOB-v04.2 dataset and gridded data 
and is only supported by the `/matchup_spark` endpoint. 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung commented on pull request #166: CDMS - 150 NETCDF Reader Updates

[GitBox]

ngachung commented on PR #166:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/166#issuecomment-1181956713

   Sorry for the confusion @JordanGethers. Yes, the github repository is 
public, but the CDMS JIRA is restricted. Please push all your changes to your 
forked repo, but where you've referenced CDMS - 150, e.g. in the pull request 
title, please replace with SDAP-390: Match up NETCDF Reader Updates. Branch 
name of SDAP-XXX should be SDAP-390. Thank you!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] tloubrieu-jpl opened a new pull request, #173: Add metadata to remote coll

[GitBox]

tloubrieu-jpl opened a new pull request, #173:
URL: https://github.com/apache/incubator-sdap-nexus/pull/173

   This pull request completes ticket  
https://issues.apache.org/jira/browse/SDAP-388 by providing metadata of the 
remote collections in the /list end-point.
   
   The remote metadata is cached for 24 hours so to avoid latency on the user 
end. Note that the list result itself is cached for 1hour.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] JordanGethers commented on pull request #166: CDMS - 150 NETCDF Reader Updates

[GitBox]

JordanGethers commented on PR #166:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/166#issuecomment-1181826386

   Hello Nga,
   
   I made the adjustments you mentioned for SDAP-390, but I don't exactly 
understand where to put the code since you say that JIRA is private and not 
public. Do I still push to the GitHub like I did last time? The GitHub seems to 
be public, and I don't see an obvious place to put the code on the Apache 
website.
   
   Best,
   
   Jordan Gethers
   Florida State University | MS Computer Science
   ***@***.***
   
   From: Nga Chung ***@***.***>
   Sent: Friday, July 8, 2022 12:43 PM
   To: apache/incubator-sdap-nexus ***@***.***>
   Cc: Jordan Gethers ***@***.***>; Mention ***@***.***>
   Subject: Re: [apache/incubator-sdap-nexus] CDMS - 150 NETCDF Reader Updates 
(PR #166)
   
   
   Thank you, 
@JordanGethers!
   
   Please reference 
https://issues.apache.org/jira/projects/SDAP/issues/SDAP-390
 as CDMS Jira is restricted and not public.
   Please also update CHANGELOG.md
   It's great that the dependent libraries are provided in the readme, but 
please also include requirements.txt or conda-requirements.txt so that users 
can easily set up env for using the reader.
   
   —
   Reply to this email directly, view it on 
GitHub,
 or 
unsubscribe.
   You are receiving this because you were mentioned.Message ID: ***@***.***>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung commented on pull request #166: CDMS - 150 NETCDF Reader Updates

[GitBox]

ngachung commented on PR #166:
URL: 
https://github.com/apache/incubator-sdap-nexus/pull/166#issuecomment-1179182763

   Thank you, @JordanGethers!
   
   Please reference 
https://issues.apache.org/jira/projects/SDAP/issues/SDAP-390 as CDMS Jira is 
restricted and not public.
   Please also update CHANGELOG.md
   It's great that the dependent libraries are provided in the readme, but 
please also include requirements.txt or conda-requirements.txt so that users 
can easily set up env for using the reader.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] ngachung merged pull request #172: SDAP-371: Fix domssubset API

[GitBox]

ngachung merged PR #172:
URL: https://github.com/apache/incubator-sdap-nexus/pull/172


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper opened a new pull request, #172: SDAP-371: Fix domssubset API

[GitBox]

skorper opened a new pull request, #172:
URL: https://github.com/apache/incubator-sdap-nexus/pull/172

   https://issues.apache.org/jira/browse/SDAP-371
   
   
   ### Summary of changes
   
   - Renamed endpoint `/domssubset` -> `/cdmssubset`
   - Updated openapi spec with "cdms" terminology
   - Added unit tests for toZip and toCsv
   - Moved `query_edge` function into a new python module 
`analysis/webservice/algorithms/doms/insitu.py` and changed "edge" terminology 
to "insitu"
 - Updated matchup to use this, removed query_edge
   - Updated subsetter to, rather than querying SDAP, use existing functions to 
query satellite and insitu data
   - ZIP contains one CSV file per dataset. File is named like 
"\.csv"
   - Added default of 0/5 for min/max depth because AWS insitu API requires 
those fields
   
   
   ### Testing 
   
   In addition to unit tests also ran locally with the following request:
   
   ```
   
{{big_data_url}}/cdmssubset?dataset=ASCATB-L2-Coastal¶meter=sea_water_temperature&startTime=2018-09-24T00:00:00Z&endTime=2018-09-30T00:00:00Z&b=160,-30,180,-25&output=ZIP&insitu=ICOADS%20Release%203.0&platforms=42&depthMin=0&depthMax=5
   ```
   
   Response: 
[response.zip](https://github.com/apache/incubator-sdap-nexus/files/9067888/response.zip)
   
   Also tested matchup to make sure nothing was broken due to the new insitu 
module -- response was as expected. 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [incubator-sdap-nexus] skorper merged pull request #171: Fix for null count field in return for matchup_spark request

[GitBox]

skorper merged PR #171:
URL: https://github.com/apache/incubator-sdap-nexus/pull/171


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@sdap.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org