BaseX index queries and new website images.
Project: http://git-wip-us.apache.org/repos/asf/vxquery/repo Commit: http://git-wip-us.apache.org/repos/asf/vxquery/commit/1704272e Tree: http://git-wip-us.apache.org/repos/asf/vxquery/tree/1704272e Diff: http://git-wip-us.apache.org/repos/asf/vxquery/diff/1704272e Branch: refs/heads/prestonc/basex Commit: 1704272e60aa54d0ff93473b235f7ca6743a9fca Parents: 0fcd4df Author: Preston Carman <[email protected]> Authored: Tue Sep 22 16:48:56 2015 -0700 Committer: Preston Carman <[email protected]> Committed: Tue Sep 22 16:48:56 2015 -0700 ---------------------------------------------------------------------- src/site/apt/developer_get_started.apt | 8 ++-- src/site/apt/index.apt | 5 +- src/site/apt/user_cluster_installation.apt | 30 ++++++++---- src/site/resources/images/vxquery_cluster.png | Bin 0 -> 22024 bytes .../images/vxquery_cluster_with_hdfs.png | Bin 0 -> 31459 bytes src/site/resources/images/vxquery_stack.png | Bin 0 -> 13264 bytes .../other_systems/basex_index/q00.xq | 32 +++++++++++++ .../other_systems/basex_index/q01.xq | 25 ++++++++++ .../other_systems/basex_index/q02.xq | 31 ++++++++++++ .../other_systems/basex_index/q03.xq | 25 ++++++++++ .../other_systems/basex_index/q04.xq | 30 ++++++++++++ .../other_systems/basex_index/q05.xq | 33 +++++++++++++ .../other_systems/basex_index/q06.xq | 30 ++++++++++++ .../other_systems/basex_index/q07.xq | 32 +++++++++++++ .../basex_index/sensors_database.bxs | 2 + .../basex_index/stations_database.bxs | 2 + .../other_systems/basex_scripts/README | 9 ++++ .../basex_scripts/run_basex_index_tests.sh | 47 +++++++++++++++++++ .../basex_scripts/run_basex_tests.sh | 2 +- 19 files changed, 327 insertions(+), 16 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/src/site/apt/developer_get_started.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/developer_get_started.apt b/src/site/apt/developer_get_started.apt index 7a93d66..6d282f5 100644 --- a/src/site/apt/developer_get_started.apt +++ b/src/site/apt/developer_get_started.apt @@ -21,9 +21,9 @@ Get Started as a VXQuery Developer * Go through the community member steps. * {{{http://vxquery.apache.org/user_get_started.html}Community steps}}. - + * Setup your eclipse development environment. - + * {{{http://vxquery.apache.org/development_eclipse_setup.html}Setup instructions}}. * XQuery has a test suite to verify XQuery specifications. @@ -34,8 +34,8 @@ Get Started as a VXQuery Developer * Review the test structure. - * Code is found in the "VXQuery XTest" module. + * Code is found in the "VXQuery XTest" module. * Review open issues for the project. - + * {{{http://vxquery.apache.org/issue-tracking.html}Issues list}}. http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/src/site/apt/index.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/index.apt b/src/site/apt/index.apt index 8b5777a..3986f81 100644 --- a/src/site/apt/index.apt +++ b/src/site/apt/index.apt @@ -22,6 +22,7 @@ Apache VXQuery There are lots of large collections of relatively small documents like e.g. the {{{http://www.sec.gov/info/edgar/ednews/dissemin.htm}EDGAR dataset}} or the {{{http://wiki.openstreetmap.org/wiki/Download}OpenStreetMap dataset}}. However there are no XQuery processors available today that are capable of processing these datasets in parallel and making the contained information accessible. - - \ No newline at end of file + +[images/vxquery_stack.png] VXQuery Stack Diagram + http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/src/site/apt/user_cluster_installation.apt ---------------------------------------------------------------------- diff --git a/src/site/apt/user_cluster_installation.apt b/src/site/apt/user_cluster_installation.apt index 1342908..0e756b0 100644 --- a/src/site/apt/user_cluster_installation.apt +++ b/src/site/apt/user_cluster_installation.apt @@ -15,6 +15,18 @@ Cluster Installation +* Architecture + + The VXQuery cluster is made up of two parts: a single cluster controller (cc) and many node controllers (nc). + The VXQuery CLI is used to parse the query and compile the job for the VXQuery cluster to process. + The CLI passes the job to the cc which manages the job and returns the result to the CLI. + The following diagram depicts the cluster layout. + +[images/vxquery_cluster.png] VXQuery Cluster Diagram + + The XML document are distributed between the ncs. + The query's collection function will identify XML file path for the ncs. + * Requirements @@ -28,7 +40,7 @@ Cluster Installation * Steps * <<Export JAVA_HOME>> - + --- $ export JAVA_HOME=/usr/java/latest --- @@ -36,7 +48,7 @@ $ export JAVA_HOME=/usr/java/latest * <<Unzip and build VXQuery>> ---- +--- $ unzip apache-vxquery-X.Y-source-release.zip $ cd apache-vxquery-X.Y $ mvn package -DskipTests @@ -45,7 +57,7 @@ $ cd .. * <<Create configuration file>> - + Create a configuration xml file containing the information of the vxquery cluster.Here is an example of a VXQuery configuration file for a cluster with 1 master and 3 slaves. --- @@ -75,7 +87,7 @@ $ cd .. * Fields that are required: * name : name of the cluster - + * username : user that will execute commands in all the machines of the cluster. Preferably a user that has passwordless ssh access to the machines. * id : hostname of the node @@ -94,12 +106,12 @@ $ cd .. * <<Deploy cluster>> To deploy the cluster you need to execute this command in the vxquery installation directory - + --- $python cluster_cli.py -c ../conf/cluster.xml -a deploy -d /apache-vxquery/vxquery-server --- - * Arguments: + * Arguments: * -c : path to the configuration file you created @@ -110,16 +122,16 @@ $python cluster_cli.py -c ../conf/cluster.xml -a deploy -d /apache-vxquery/vxque * <<Start cluster>> - The command to start the cluster is + The command to start the cluster is --- $python cluster_cli.py -c ../conf/cluster.xml -a start --- - + * <<Stop cluster>> - The command to stop the cluster is + The command to stop the cluster is --- $python cluster_cli.py -c ../conf/cluster.xml -a stop http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/src/site/resources/images/vxquery_cluster.png ---------------------------------------------------------------------- diff --git a/src/site/resources/images/vxquery_cluster.png b/src/site/resources/images/vxquery_cluster.png new file mode 100644 index 0000000..50cb26b Binary files /dev/null and b/src/site/resources/images/vxquery_cluster.png differ http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/src/site/resources/images/vxquery_cluster_with_hdfs.png ---------------------------------------------------------------------- diff --git a/src/site/resources/images/vxquery_cluster_with_hdfs.png b/src/site/resources/images/vxquery_cluster_with_hdfs.png new file mode 100644 index 0000000..2ec42b8 Binary files /dev/null and b/src/site/resources/images/vxquery_cluster_with_hdfs.png differ http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/src/site/resources/images/vxquery_stack.png ---------------------------------------------------------------------- diff --git a/src/site/resources/images/vxquery_stack.png b/src/site/resources/images/vxquery_stack.png new file mode 100644 index 0000000..01549a5 Binary files /dev/null and b/src/site/resources/images/vxquery_stack.png differ http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q00.xq ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q00.xq b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q00.xq new file mode 100644 index 0000000..21fc54a --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q00.xq @@ -0,0 +1,32 @@ +(: Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. :) + +(: +XQuery Filter Query +------------------- +See historical data for Key West International Airport, FL (USW00012836) +station by selecting the weather readings for December 25 over the last +10 years. +:) +let $collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/" +for $r in collection($collection)/root/dataCollection/data +let $datetime := xs:dateTime(fn:data($r/date)) +where $r/station eq "GHCND:USW00012836" + and fn:year-from-dateTime($datetime) ge 2003 + and fn:month-from-dateTime($datetime) eq 12 + and fn:day-from-dateTime($datetime) eq 25 +return $r \ No newline at end of file http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q01.xq ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q01.xq b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q01.xq new file mode 100644 index 0000000..ba7f0f9 --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q01.xq @@ -0,0 +1,25 @@ +(: Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. :) + +(: XQuery Filter Query :) +(: Find all reading for hurricane force wind warning or extreme wind warning. :) +(: The warnings occur when the wind speed (AWND) exceeds 110 mph (49.1744 :) +(: meters per second). (Wind value is in tenth of a meter per second) :) +let $collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/" +for $r in fn:collection($collection)/root/dataCollection/data +where $r/dataType eq "AWND" and xs:decimal($r/value) gt 491.744 +return $r http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q02.xq ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q02.xq b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q02.xq new file mode 100644 index 0000000..b089348 --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q02.xq @@ -0,0 +1,31 @@ +(: Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. :) + +(: +XQuery Aggregate Query +---------------------- +Find the annual precipitation (PRCP) for a Syracuse, NY using the airport +weather station (USW00014771) report for 1999. +:) +fn:sum( + let $collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/" + for $r in collection($collection)/root/dataCollection/data + where $r/station eq "GHCND:USW00014771" + and $r/dataType eq "PRCP" + and fn:year-from-dateTime(xs:dateTime(fn:data($r/date))) eq 1999 + return $r/value +) div 10 http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q03.xq ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q03.xq b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q03.xq new file mode 100644 index 0000000..285bf21 --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q03.xq @@ -0,0 +1,25 @@ +(: Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. :) + +(: XQuery Aggregate Query :) +(: Find the highest recorded temperature (TMAX) in Celsius. :) +fn:max( + let $collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/" + for $r in collection($collection)/root/dataCollection/data + where $r/dataType eq "TMAX" + return $r/value +) div 10 http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q04.xq ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q04.xq b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q04.xq new file mode 100644 index 0000000..9b69303 --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q04.xq @@ -0,0 +1,30 @@ +(: Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. :) + +(: XQuery Join Query :) +(: Find all the weather readings for King county for a specific day :) +(: 1976/7/4. :) +let $sensor_collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/" +for $r in collection($sensor_collection)/root/dataCollection/data + +let $station_collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/stations/" +for $s in collection($station_collection)/root/stationCollection/station + +where $s/id eq $r/station + and (some $x in $s/locationLabels satisfies ($x/type eq "ST" and fn:upper-case(fn:data($x/displayName)) eq "WASHINGTON")) + and xs:dateTime(fn:data($r/date)) eq xs:dateTime("1976-07-04T00:00:00.000") +return $r http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q05.xq ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q05.xq b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q05.xq new file mode 100644 index 0000000..68eed3a --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q05.xq @@ -0,0 +1,33 @@ +(: Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. :) + +(: XQuery Join Aggregate Query :) +(: Find the lowest recorded temperature (TMIN) in the state of Oregon for :) +(: 2001. :) +fn:min( + let $sensor_collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/" + for $r in collection($sensor_collection)/root/dataCollection/data + + let $station_collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/stations/" + for $s in collection($station_collection)/root/stationCollection/station + + where $s/id eq $r/station + and (some $x in $s/locationLabels satisfies ($x/type eq "CNTRY" and $x/id eq "FIPS:US")) + and $r/dataType eq "TMIN" + and fn:year-from-dateTime(xs:dateTime(fn:data($r/date))) eq 2001 + return $r/value +) div 10 http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q06.xq ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q06.xq b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q06.xq new file mode 100644 index 0000000..1e3061c --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q06.xq @@ -0,0 +1,30 @@ +(: Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. :) + +(: XQuery Join Query :) +(: Find the highest recorded temperature (TMAX) for each station for each :) +(: day over the year 2000. :) +let $sensor_collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/" +for $r in collection($sensor_collection)/root/dataCollection/data + +let $station_collection := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/stations/" +for $s in collection($station_collection)/root/stationCollection/station + +where $s/id eq $r/station + and $r/dataType eq "TMAX" + and fn:year-from-dateTime(xs:dateTime(fn:data($r/date))) eq 2000 +return ($s/displayName, $r/date, $r/value) \ No newline at end of file http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q07.xq ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q07.xq b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q07.xq new file mode 100644 index 0000000..23d4358 --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/q07.xq @@ -0,0 +1,32 @@ +(: Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. :) + +(: XQuery Join Aggregate Query :) +(: Self join with all sensor readings after the year 2000. :) +fn:avg( +let $sensor_collection_min := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/" +for $r_min in collection($sensor_collection_min)/root/dataCollection/data + +let $sensor_collection_max := "../../../../../../../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/" +for $r_max in collection($sensor_collection_max)/root/dataCollection/data + +where $r_min/station eq $r_max/station + and $r_min/date eq $r_max/date + and $r_min/dataType eq "TMIN" + and $r_max/dataType eq "TMAX" +return ($r_max/value - $r_min/value) +) div 10 \ No newline at end of file http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/sensors_database.bxs ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/sensors_database.bxs b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/sensors_database.bxs new file mode 100644 index 0000000..0d2f897 --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/sensors_database.bxs @@ -0,0 +1,2 @@ +CREATE DB db_sesnors +ADD ../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/sensors/ http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/stations_database.bxs ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/stations_database.bxs b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/stations_database.bxs new file mode 100644 index 0000000..8523aa2 --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_index/stations_database.bxs @@ -0,0 +1,2 @@ +CREATE DB db_sesnors +ADD ../weather_data/dataset-tiny-local/data_links/local_speed_up/d0_p1_i0/stations/ http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/README ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/README b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/README new file mode 100644 index 0000000..1d371b5 --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/README @@ -0,0 +1,9 @@ +BaseX local configuration options for the weather benchmark. +The options attempt to match the processing and output of VXQuery. + + +# Local Options +CHOP = false +DBPATH = /home/ecarman/disk1/basex/BaseXData +REPOPATH = /home/ecarman/disk1/basex/BaseXRepo +WEBPATH = /home/ecarman/disk1/basex/BaseXWeb http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/run_basex_index_tests.sh ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/run_basex_index_tests.sh b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/run_basex_index_tests.sh new file mode 100755 index 0000000..9b35237 --- /dev/null +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/run_basex_index_tests.sh @@ -0,0 +1,47 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +NODES=${2} +REPEAT=${3} +DATASET=${4} +EMAIL=${5} + + +mkdir -p ~/logs/basex_index/ + +time JAVA_OPTS="-Xmx8g" java -cp BaseX823.jar org.basex.BaseX + +for j in $(find ${1} -name '*q??.xq') +do + date + echo "Running BaseX query: ${j}" + time JAVA_OPTS="-Xmx8g" java -cp BaseX823.jar org.basex.BaseX -V -x -w -r${REPEAT} ${j} >> ~/logs/basex_index/$(basename "${j}").log 2>&1 +done + + +if which programname >/dev/null; +then + echo "Sending out e-mail notification." + SUBJECT="BaseX Tests Finished (${DATASET})" + /bin/mail -s "${SUBJECT}" "${EMAIL}" <<EOM + Completed all BaseX tests on ${DATASET}. + EOM +else + echo "No mail command to use." +fi; http://git-wip-us.apache.org/repos/asf/vxquery/blob/1704272e/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/run_basex_tests.sh ---------------------------------------------------------------------- diff --git a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/run_basex_tests.sh b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/run_basex_tests.sh index cf9d2cd..1e91498 100755 --- a/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/run_basex_tests.sh +++ b/vxquery-benchmark/src/main/resources/noaa-ghcn-daily/other_systems/basex_scripts/run_basex_tests.sh @@ -29,7 +29,7 @@ for j in $(find ${1} -name '*q??.xq') do date echo "Running BaseX query: ${j}" - time for i in {1..${REPEAT}}; do JAVA_OPTS="-Xmx8g" java -cp BaseX823.jar org.basex.BaseX -v -r${REPEAT} ${j} >> ~/logs/basex/$(basename "${j}").log 2>&1; done; + time JAVA_OPTS="-Xmx8g" java -cp BaseX823.jar org.basex.BaseX -V -x -w -r${REPEAT} ${j} >> ~/logs/basex/$(basename "${j}").log 2>&1 done
