[GitHub] incubator-quickstep pull request #314: Added Vector Aggregation support in t...

[zuyu]

GitHub user zuyu opened a pull request:

https://github.com/apache/incubator-quickstep/pull/314

Added Vector Aggregation support in the distributed version.

Assigned to @jianqiao.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zuyu/incubator-quickstep dist-vector-aggr

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-quickstep/pull/314.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #314


commit 705ab574e606a859a1a1bfe8a2dd9a62f796b1bd
Author: Zuyu Zhang 
Date:   2017-08-04T22:03:34Z

Added Vector Aggregation support in the distributed version.




---

[GitHub] incubator-quickstep pull request #313: Relax the sort requirement in columns...

[zuyu]

GitHub user zuyu opened a pull request:

https://github.com/apache/incubator-quickstep/pull/313

Relax the sort requirement in columnstore.

This small PR allows to create a relation w/ column store, while not 
specifying a sort attribute.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zuyu/incubator-quickstep no-sort-in-cs

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-quickstep/pull/313.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #313


commit 2e0496c22a804da067a91caf8bc291b8c66a9c7b
Author: Zuyu Zhang 
Date:   2017-10-09T16:23:08Z

Relax the sort requirement in columnstore.




---

[GitHub] incubator-quickstep pull request #309: Add ProbabilityStore class

[zuyu]

Github user zuyu commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/309#discussion_r143527465
  
--- Diff: query_execution/ProbabilityStore.hpp ---
@@ -0,0 +1,274 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#ifndef QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+#define QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "utility/Macros.hpp"
+
+#include "glog/logging.h"
+
+namespace quickstep {
+
+/**
+ * @brief A class that stores the probabilities of objects. We use an 
integer field
+ *called "key" to identify each object.
+ *A probability is expressed as a fraction. All the objects share 
a common denominator.
+ **/
+class ProbabilityStore {
+ public:
+  /**
+   * @brief Constructor.
+   **/
+  ProbabilityStore()
+  : common_denominator_(1.0) {}
+
+  /**
+   * @brief Get the number of objects in the store.
+   **/
+  inline const std::size_t getNumObjects() const {
+DCHECK_EQ(individual_probabilities_.size(), 
cumulative_probabilities_.size());
+return individual_probabilities_.size();
+  }
+
+  /**
+   * @brief Get the common denominator.
+   */
+  inline const std::size_t getDenominator() const {
+return common_denominator_;
+  }
+
+  /**
+   * @brief Check if an object with a given key is present.
+   * @param key The key of the given object.
+   * @return True if the object is present, false otherwise.
+   */
+  inline bool hasObject(const std::size_t key) const {
+return (individual_probabilities_.find(key) != 
individual_probabilities_.end());
+  }
+
+  /**
+   * @brief Add individual (not cumulative) probability for a given object 
with
+   *updated denominator.
+   *
+   * @note This function leaves the cumulative probabilities in a 
consistent
+   *   state. An alternative lazy implementation should be written if 
cost
+   *   of calculating cumulative probabilities is high.
+   *
+   * @param key The key of the given object.
+   * @param numerator The numerator for the given object.
+   * @param new_denominator The updated denominator for the store.
+   **/
+  void addOrUpdateObjectNewDenominator(const std::size_t key,
+   const float numerator,
+   const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+DCHECK_LE(numerator, new_denominator);
+common_denominator_ = new_denominator;
+addOrUpdateObjectHelper(key, numerator);
+  }
+
+  /**
+   * @brief Add or update multiple objects with a new denominator.
+   * @param keys The keys to be added/updated.
+   * @param numerators The numerators to be added/updated.
+   * @param new_denominator The new denominator.
+   */
+  void addOrUpdateObjectsNewDenominator(
+  const std::vector ,
+  const std::vector ,
+  const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+common_denominator_ = new_denominator;
+addOrUpdateObjectsHelper(keys, numerators);
+  }
+
+  /**
+   * @brief Remove an object from the store.
+   *
+   * @note This function decrements the denominator with the value of the 
numerator being removed.
+   *
+   * @param key The key of the object to be removed.
+   **/
+  void removeObject(const std::size_t key) {
+auto individual_it = individual_probabilities_.find(key);
+DCHECK(individual_it != individual_probabilities_.end());
+const float new_denominator = common_denominator_ - 
individual_it->second;
+individual_probabilities_.erase(individual_it);
+common_denominator_ = new_denominator;
+updateCumulativeProbabilities();
+  }
+
+  /**
+   * @brief 

[GitHub] incubator-quickstep pull request #309: Add ProbabilityStore class

[zuyu]

Github user zuyu commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/309#discussion_r143522173
  
--- Diff: query_execution/ProbabilityStore.hpp ---
@@ -0,0 +1,274 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#ifndef QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+#define QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "utility/Macros.hpp"
+
+#include "glog/logging.h"
+
+namespace quickstep {
+
+/**
+ * @brief A class that stores the probabilities of objects. We use an 
integer field
+ *called "key" to identify each object.
+ *A probability is expressed as a fraction. All the objects share 
a common denominator.
+ **/
+class ProbabilityStore {
+ public:
+  /**
+   * @brief Constructor.
+   **/
+  ProbabilityStore()
+  : common_denominator_(1.0) {}
+
+  /**
+   * @brief Get the number of objects in the store.
+   **/
+  inline const std::size_t getNumObjects() const {
+DCHECK_EQ(individual_probabilities_.size(), 
cumulative_probabilities_.size());
--- End diff --

Does it follow the rule that the left side is the checked variable, and the 
right is the expected one?


---

[GitHub] incubator-quickstep pull request #309: Add ProbabilityStore class

[zuyu]

Github user zuyu commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/309#discussion_r143522609
  
--- Diff: query_execution/ProbabilityStore.hpp ---
@@ -0,0 +1,274 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#ifndef QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+#define QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "utility/Macros.hpp"
+
+#include "glog/logging.h"
+
+namespace quickstep {
+
+/**
+ * @brief A class that stores the probabilities of objects. We use an 
integer field
+ *called "key" to identify each object.
+ *A probability is expressed as a fraction. All the objects share 
a common denominator.
+ **/
+class ProbabilityStore {
+ public:
+  /**
+   * @brief Constructor.
+   **/
+  ProbabilityStore()
+  : common_denominator_(1.0) {}
+
+  /**
+   * @brief Get the number of objects in the store.
+   **/
+  inline const std::size_t getNumObjects() const {
+DCHECK_EQ(individual_probabilities_.size(), 
cumulative_probabilities_.size());
+return individual_probabilities_.size();
+  }
+
+  /**
+   * @brief Get the common denominator.
+   */
+  inline const std::size_t getDenominator() const {
+return common_denominator_;
+  }
+
+  /**
+   * @brief Check if an object with a given key is present.
+   * @param key The key of the given object.
+   * @return True if the object is present, false otherwise.
+   */
+  inline bool hasObject(const std::size_t key) const {
+return (individual_probabilities_.find(key) != 
individual_probabilities_.end());
+  }
+
+  /**
+   * @brief Add individual (not cumulative) probability for a given object 
with
+   *updated denominator.
+   *
+   * @note This function leaves the cumulative probabilities in a 
consistent
+   *   state. An alternative lazy implementation should be written if 
cost
+   *   of calculating cumulative probabilities is high.
+   *
+   * @param key The key of the given object.
+   * @param numerator The numerator for the given object.
+   * @param new_denominator The updated denominator for the store.
+   **/
+  void addOrUpdateObjectNewDenominator(const std::size_t key,
+   const float numerator,
+   const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+DCHECK_LE(numerator, new_denominator);
+common_denominator_ = new_denominator;
+addOrUpdateObjectHelper(key, numerator);
+  }
+
+  /**
+   * @brief Add or update multiple objects with a new denominator.
+   * @param keys The keys to be added/updated.
+   * @param numerators The numerators to be added/updated.
+   * @param new_denominator The new denominator.
+   */
+  void addOrUpdateObjectsNewDenominator(
+  const std::vector ,
+  const std::vector ,
+  const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+common_denominator_ = new_denominator;
+addOrUpdateObjectsHelper(keys, numerators);
+  }
+
+  /**
+   * @brief Remove an object from the store.
+   *
+   * @note This function decrements the denominator with the value of the 
numerator being removed.
+   *
+   * @param key The key of the object to be removed.
+   **/
+  void removeObject(const std::size_t key) {
+auto individual_it = individual_probabilities_.find(key);
+DCHECK(individual_it != individual_probabilities_.end());
+const float new_denominator = common_denominator_ - 
individual_it->second;
+individual_probabilities_.erase(individual_it);
+common_denominator_ = new_denominator;
+updateCumulativeProbabilities();
+  }
+
+  /**
+   * @brief 

[GitHub] incubator-quickstep pull request #309: Add ProbabilityStore class

[zuyu]

Github user zuyu commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/309#discussion_r143525719
  
--- Diff: query_execution/ProbabilityStore.hpp ---
@@ -0,0 +1,274 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#ifndef QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+#define QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "utility/Macros.hpp"
+
+#include "glog/logging.h"
+
+namespace quickstep {
+
+/**
+ * @brief A class that stores the probabilities of objects. We use an 
integer field
+ *called "key" to identify each object.
+ *A probability is expressed as a fraction. All the objects share 
a common denominator.
+ **/
+class ProbabilityStore {
+ public:
+  /**
+   * @brief Constructor.
+   **/
+  ProbabilityStore()
+  : common_denominator_(1.0) {}
+
+  /**
+   * @brief Get the number of objects in the store.
+   **/
+  inline const std::size_t getNumObjects() const {
+DCHECK_EQ(individual_probabilities_.size(), 
cumulative_probabilities_.size());
+return individual_probabilities_.size();
+  }
+
+  /**
+   * @brief Get the common denominator.
+   */
+  inline const std::size_t getDenominator() const {
+return common_denominator_;
+  }
+
+  /**
+   * @brief Check if an object with a given key is present.
+   * @param key The key of the given object.
+   * @return True if the object is present, false otherwise.
+   */
+  inline bool hasObject(const std::size_t key) const {
+return (individual_probabilities_.find(key) != 
individual_probabilities_.end());
+  }
+
+  /**
+   * @brief Add individual (not cumulative) probability for a given object 
with
+   *updated denominator.
+   *
+   * @note This function leaves the cumulative probabilities in a 
consistent
+   *   state. An alternative lazy implementation should be written if 
cost
+   *   of calculating cumulative probabilities is high.
+   *
+   * @param key The key of the given object.
+   * @param numerator The numerator for the given object.
+   * @param new_denominator The updated denominator for the store.
+   **/
+  void addOrUpdateObjectNewDenominator(const std::size_t key,
+   const float numerator,
+   const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+DCHECK_LE(numerator, new_denominator);
+common_denominator_ = new_denominator;
+addOrUpdateObjectHelper(key, numerator);
+  }
+
+  /**
+   * @brief Add or update multiple objects with a new denominator.
+   * @param keys The keys to be added/updated.
+   * @param numerators The numerators to be added/updated.
+   * @param new_denominator The new denominator.
+   */
+  void addOrUpdateObjectsNewDenominator(
+  const std::vector ,
+  const std::vector ,
+  const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+common_denominator_ = new_denominator;
+addOrUpdateObjectsHelper(keys, numerators);
+  }
+
+  /**
+   * @brief Remove an object from the store.
+   *
+   * @note This function decrements the denominator with the value of the 
numerator being removed.
+   *
+   * @param key The key of the object to be removed.
+   **/
+  void removeObject(const std::size_t key) {
+auto individual_it = individual_probabilities_.find(key);
+DCHECK(individual_it != individual_probabilities_.end());
+const float new_denominator = common_denominator_ - 
individual_it->second;
+individual_probabilities_.erase(individual_it);
+common_denominator_ = new_denominator;
+updateCumulativeProbabilities();
+  }
+
+  /**
+   * @brief 

[GitHub] incubator-quickstep pull request #309: Add ProbabilityStore class

[zuyu]

Github user zuyu commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/309#discussion_r143522055
  
--- Diff: query_execution/ProbabilityStore.hpp ---
@@ -0,0 +1,274 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#ifndef QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+#define QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "utility/Macros.hpp"
+
+#include "glog/logging.h"
+
+namespace quickstep {
+
+/**
+ * @brief A class that stores the probabilities of objects. We use an 
integer field
+ *called "key" to identify each object.
+ *A probability is expressed as a fraction. All the objects share 
a common denominator.
+ **/
+class ProbabilityStore {
+ public:
+  /**
+   * @brief Constructor.
+   **/
+  ProbabilityStore()
+  : common_denominator_(1.0) {}
+
+  /**
+   * @brief Get the number of objects in the store.
+   **/
+  inline const std::size_t getNumObjects() const {
+DCHECK_EQ(individual_probabilities_.size(), 
cumulative_probabilities_.size());
+return individual_probabilities_.size();
+  }
+
+  /**
+   * @brief Get the common denominator.
+   */
+  inline const std::size_t getDenominator() const {
--- End diff --

Remove `const` in the return type.


---

[GitHub] incubator-quickstep pull request #309: Add ProbabilityStore class

[zuyu]

Github user zuyu commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/309#discussion_r143528439
  
--- Diff: query_execution/tests/ProbabilityStore_unittest.cpp ---
@@ -0,0 +1,106 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#include 
+#include 
+
+#include "gtest/gtest.h"
+
+#include "query_execution/ProbabilityStore.hpp"
--- End diff --

Per header order, move it to the top.


---

[GitHub] incubator-quickstep pull request #309: Add ProbabilityStore class

[zuyu]

Github user zuyu commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/309#discussion_r143527518
  
--- Diff: query_execution/ProbabilityStore.hpp ---
@@ -0,0 +1,274 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#ifndef QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+#define QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "utility/Macros.hpp"
+
+#include "glog/logging.h"
+
+namespace quickstep {
+
+/**
+ * @brief A class that stores the probabilities of objects. We use an 
integer field
+ *called "key" to identify each object.
+ *A probability is expressed as a fraction. All the objects share 
a common denominator.
+ **/
+class ProbabilityStore {
+ public:
+  /**
+   * @brief Constructor.
+   **/
+  ProbabilityStore()
+  : common_denominator_(1.0) {}
+
+  /**
+   * @brief Get the number of objects in the store.
+   **/
+  inline const std::size_t getNumObjects() const {
+DCHECK_EQ(individual_probabilities_.size(), 
cumulative_probabilities_.size());
+return individual_probabilities_.size();
+  }
+
+  /**
+   * @brief Get the common denominator.
+   */
+  inline const std::size_t getDenominator() const {
+return common_denominator_;
+  }
+
+  /**
+   * @brief Check if an object with a given key is present.
+   * @param key The key of the given object.
+   * @return True if the object is present, false otherwise.
+   */
+  inline bool hasObject(const std::size_t key) const {
+return (individual_probabilities_.find(key) != 
individual_probabilities_.end());
+  }
+
+  /**
+   * @brief Add individual (not cumulative) probability for a given object 
with
+   *updated denominator.
+   *
+   * @note This function leaves the cumulative probabilities in a 
consistent
+   *   state. An alternative lazy implementation should be written if 
cost
+   *   of calculating cumulative probabilities is high.
+   *
+   * @param key The key of the given object.
+   * @param numerator The numerator for the given object.
+   * @param new_denominator The updated denominator for the store.
+   **/
+  void addOrUpdateObjectNewDenominator(const std::size_t key,
+   const float numerator,
+   const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+DCHECK_LE(numerator, new_denominator);
+common_denominator_ = new_denominator;
+addOrUpdateObjectHelper(key, numerator);
+  }
+
+  /**
+   * @brief Add or update multiple objects with a new denominator.
+   * @param keys The keys to be added/updated.
+   * @param numerators The numerators to be added/updated.
+   * @param new_denominator The new denominator.
+   */
+  void addOrUpdateObjectsNewDenominator(
+  const std::vector ,
+  const std::vector ,
+  const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+common_denominator_ = new_denominator;
+addOrUpdateObjectsHelper(keys, numerators);
+  }
+
+  /**
+   * @brief Remove an object from the store.
+   *
+   * @note This function decrements the denominator with the value of the 
numerator being removed.
+   *
+   * @param key The key of the object to be removed.
+   **/
+  void removeObject(const std::size_t key) {
+auto individual_it = individual_probabilities_.find(key);
+DCHECK(individual_it != individual_probabilities_.end());
+const float new_denominator = common_denominator_ - 
individual_it->second;
+individual_probabilities_.erase(individual_it);
+common_denominator_ = new_denominator;
+updateCumulativeProbabilities();
+  }
+
+  /**
+   * @brief 

[GitHub] incubator-quickstep pull request #309: Add ProbabilityStore class

[zuyu]

Github user zuyu commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/309#discussion_r143527323
  
--- Diff: query_execution/ProbabilityStore.hpp ---
@@ -0,0 +1,274 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#ifndef QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+#define QUICKSTEP_QUERY_EXECUTION_PROBABILITY_STORE_HPP_
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "utility/Macros.hpp"
+
+#include "glog/logging.h"
+
+namespace quickstep {
+
+/**
+ * @brief A class that stores the probabilities of objects. We use an 
integer field
+ *called "key" to identify each object.
+ *A probability is expressed as a fraction. All the objects share 
a common denominator.
+ **/
+class ProbabilityStore {
+ public:
+  /**
+   * @brief Constructor.
+   **/
+  ProbabilityStore()
+  : common_denominator_(1.0) {}
+
+  /**
+   * @brief Get the number of objects in the store.
+   **/
+  inline const std::size_t getNumObjects() const {
+DCHECK_EQ(individual_probabilities_.size(), 
cumulative_probabilities_.size());
+return individual_probabilities_.size();
+  }
+
+  /**
+   * @brief Get the common denominator.
+   */
+  inline const std::size_t getDenominator() const {
+return common_denominator_;
+  }
+
+  /**
+   * @brief Check if an object with a given key is present.
+   * @param key The key of the given object.
+   * @return True if the object is present, false otherwise.
+   */
+  inline bool hasObject(const std::size_t key) const {
+return (individual_probabilities_.find(key) != 
individual_probabilities_.end());
+  }
+
+  /**
+   * @brief Add individual (not cumulative) probability for a given object 
with
+   *updated denominator.
+   *
+   * @note This function leaves the cumulative probabilities in a 
consistent
+   *   state. An alternative lazy implementation should be written if 
cost
+   *   of calculating cumulative probabilities is high.
+   *
+   * @param key The key of the given object.
+   * @param numerator The numerator for the given object.
+   * @param new_denominator The updated denominator for the store.
+   **/
+  void addOrUpdateObjectNewDenominator(const std::size_t key,
+   const float numerator,
+   const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+DCHECK_LE(numerator, new_denominator);
+common_denominator_ = new_denominator;
+addOrUpdateObjectHelper(key, numerator);
+  }
+
+  /**
+   * @brief Add or update multiple objects with a new denominator.
+   * @param keys The keys to be added/updated.
+   * @param numerators The numerators to be added/updated.
+   * @param new_denominator The new denominator.
+   */
+  void addOrUpdateObjectsNewDenominator(
+  const std::vector ,
+  const std::vector ,
+  const float new_denominator) {
+CHECK_GT(new_denominator, 0u);
+common_denominator_ = new_denominator;
+addOrUpdateObjectsHelper(keys, numerators);
+  }
+
+  /**
+   * @brief Remove an object from the store.
+   *
+   * @note This function decrements the denominator with the value of the 
numerator being removed.
+   *
+   * @param key The key of the object to be removed.
+   **/
+  void removeObject(const std::size_t key) {
+auto individual_it = individual_probabilities_.find(key);
+DCHECK(individual_it != individual_probabilities_.end());
+const float new_denominator = common_denominator_ - 
individual_it->second;
+individual_probabilities_.erase(individual_it);
+common_denominator_ = new_denominator;
+updateCumulativeProbabilities();
+  }
+
+  /**
+   * @brief 

[GitHub] incubator-quickstep pull request #304: Added a new set API for TupleIdSequen...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-quickstep/pull/304


---

[GitHub] incubator-quickstep issue #304: Added a new set API for TupleIdSequence.

[jianqiao]

Github user jianqiao commented on the issue:

https://github.com/apache/incubator-quickstep/pull/304
  
LGTM! Merging.


---

[GitHub] incubator-quickstep pull request #307: Moved InsertDestination::getTouchedBl...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-quickstep/pull/307


---

[GitHub] incubator-quickstep pull request #310: Removed the virtual function call in ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-quickstep/pull/310


---

[GitHub] incubator-quickstep pull request #312: Fixed a flaky case in Catalog test.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-quickstep/pull/312


---

[GitHub] incubator-quickstep issue #310: Removed the virtual function call in InvokeO...

[hbdeshmukh]

Github user hbdeshmukh commented on the issue:

https://github.com/apache/incubator-quickstep/pull/310
  
Looks good. 


---

[GitHub] incubator-quickstep issue #312: Fixed a flaky case in Catalog test.

[hbdeshmukh]

Github user hbdeshmukh commented on the issue:

https://github.com/apache/incubator-quickstep/pull/312
  
Looks good to me. 


---

[GitHub] incubator-quickstep issue #311: Fixed the distributed version

[hbdeshmukh]

Github user hbdeshmukh commented on the issue:

https://github.com/apache/incubator-quickstep/pull/311
  
Looks good to me. 


---

[GitHub] incubator-quickstep pull request #312: Fixed a flaky case in Catalog test.

[hbdeshmukh]

Github user hbdeshmukh commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/312#discussion_r143515729
  
--- Diff: catalog/tests/Catalog_unittest.cpp ---
@@ -552,13 +552,15 @@ TEST_F(CatalogTest, CatalogIndexTest) {
   IndexSubBlockDescription index_description;
   index_description.set_sub_block_type(IndexSubBlockDescription::CSB_TREE);
   
index_description.add_indexed_attribute_ids(rel->getAttributeByName("attr_idx1")->getID());
+  IndexSubBlockDescription index_description_copy;
+  index_description_copy.MergeFrom(index_description);
 
   EXPECT_TRUE(rel->addIndex("idx1", std::move(index_description)));
   EXPECT_TRUE(rel->hasIndexWithName("idx1"));
   // Adding an index with duplicate name should return false.
   EXPECT_FALSE(rel->addIndex("idx1", std::move(index_description)));
--- End diff --

By the same logic, we should create another copy for this move instruction. 


---