[GitHub] incubator-pirk pull request #105: [PIRK 71] Move Querier creation logic into...
Github user jryancarr commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/105#discussion_r81340968
--- Diff:
src/main/java/org/apache/pirk/querier/wideskies/QuerierFactory.java ---
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pirk.querier.wideskies;
+
+import org.apache.pirk.encryption.Paillier;
+import org.apache.pirk.querier.wideskies.encrypt.EncryptQuery;
+import org.apache.pirk.query.wideskies.QueryInfo;
+import org.apache.pirk.schema.query.QuerySchemaRegistry;
+import org.apache.pirk.utils.PIRException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.math.BigInteger;
+import java.util.List;
+import java.util.Properties;
+import java.util.UUID;
+
+/**
+ * Handles encrypting a query and constructing a {@link Querier} given a
{@link EncryptionPropertiesBuilder}.
+ *
+ * Created by ryan on 9/28/16.
--- End diff --
Sorry! Will be more careful next time. Looks like @ellisonanne took care of
this already as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #105: [PIRK 71] Move Querier creation logic into...
Github user jryancarr commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/105#discussion_r8133
--- Diff: src/main/java/org/apache/pirk/querier/wideskies/QuerierCLI.java
---
@@ -131,12 +125,28 @@ private boolean parseOptions()
// Validate properties
valid = QuerierProps.validateQuerierProperties();
+// Load the new local query and data schemas
+if (valid)
+{
+ logger.info("loading schemas: dataSchemas = " +
SystemConfiguration.getProperty("data.schemas") + " querySchemas = " +
SystemConfiguration
+ .getProperty("query.schemas"));
+ try
+ {
+DataSchemaLoader.initialize();
+QuerySchemaLoader.initialize();
+
+ } catch (Exception e)
--- End diff --
This code was moved out of the old QuerierProps.validateQuerierProperties()
method. I decided it would be safe to move out since this was the only place
that method was ever called, and this code doing initialization, not
validation. But I'm not familiar enough yet with what the schema loaders are
doing to understand how to handle those exceptions differently. Any suggestions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
Re: [GitHub] incubator-pirk pull request #105: [PIRK 71] Move Querier creation logic into...
I will take off the author info and push - I meant to do this yesterday
when I reformatted...
On Fri, Sep 30, 2016 at 5:31 AM, tellison wrote:
> Github user tellison commented on a diff in the pull request:
>
> https://github.com/apache/incubator-pirk/pull/105#discussion_r81305132
>
> --- Diff: src/main/java/org/apache/pirk/querier/wideskies/QuerierCLI.java
> ---
> @@ -131,12 +125,28 @@ private boolean parseOptions()
> // Validate properties
> valid = QuerierProps.validateQuerierProperties();
>
> +// Load the new local query and data schemas
> +if (valid)
> +{
> + logger.info("loading schemas: dataSchemas = " +
> SystemConfiguration.getProperty("data.schemas") + " querySchemas = " +
> SystemConfiguration
> + .getProperty("query.schemas"));
> + try
> + {
> +DataSchemaLoader.initialize();
> +QuerySchemaLoader.initialize();
> +
> + } catch (Exception e)
> --- End diff --
>
> It would be much better to handle the exception, or re-throw it,
> rather than consume and continue.
> Why catch all Exception types?
>
>
> ---
> If your project is set up for it, you can reply to this email and have your
> reply appear on GitHub as well. If your project does not have this feature
> enabled and wishes so, or if the feature is enabled but not working, please
> contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
> with INFRA.
> ---
>
[GitHub] incubator-pirk pull request #105: [PIRK 71] Move Querier creation logic into...
Github user tellison commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/105#discussion_r81306398
--- Diff: src/main/java/org/apache/pirk/test/utils/StandaloneQuery.java ---
@@ -77,25 +76,16 @@
logger.info("fileQuerier = " + fileQuerier.getAbsolutePath() + "
fileQuery = " + fileQuery.getAbsolutePath() + " responseFile = "
+ fileResponse.getAbsolutePath() + " fileFinalResults = " +
fileFinalResults.getAbsolutePath());
-boolean embedSelector =
SystemConfiguration.getBooleanProperty("pirTest.embedSelector", false);
-boolean useExpLookupTable =
SystemConfiguration.getBooleanProperty("pirTest.useExpLookupTable", false);
-boolean useHDFSExpLookupTable =
SystemConfiguration.getBooleanProperty("pirTest.useHDFSExpLookupTable", false);
-
-// Set the necessary objects
-QueryInfo queryInfo = new QueryInfo(BaseTests.queryIdentifier,
selectors.size(), BaseTests.hashBitSize, BaseTests.hashKey,
BaseTests.dataPartitionBitSize,
-queryType, useExpLookupTable, embedSelector,
useHDFSExpLookupTable);
-
-if (SystemConfiguration.getBooleanProperty("pir.embedQuerySchema",
false))
-{
- queryInfo.addQuerySchema(qSchema);
-}
-
-Paillier paillier = new Paillier(BaseTests.paillierBitSize,
BaseTests.certainty);
+Properties baseTestEncryptionProperties =
EncryptionPropertiesBuilder.newBuilder()
+ .dataPartitionBitSize(BaseTests.dataPartitionBitSize)
+ .hashBitSize(BaseTests.hashBitSize)
+ .hashKey(BaseTests.hashKey)
+ .paillierBitSize(BaseTests.paillierBitSize)
+ .certainty(BaseTests.certainty)
+ .queryType(queryType)
+ .build();
--- End diff --
Nice.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #105: [PIRK 71] Move Querier creation logic into...
Github user tellison commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/105#discussion_r81304556
--- Diff:
src/main/java/org/apache/pirk/querier/wideskies/EncryptionPropertiesBuilder.java
---
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pirk.querier.wideskies;
+
+import org.apache.pirk.utils.PIRException;
+
+import java.util.Properties;
+
+import static org.apache.pirk.querier.wideskies.QuerierProps.*;
+
+/**
+ * Holds the various parameters related to creating a {@link Querier}.
+ *
+ * Created by ryan on 9/28/16.
--- End diff --
Please drop the author info.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #105: [PIRK 71] Move Querier creation logic into...
Github user tellison commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/105#discussion_r81305132
--- Diff: src/main/java/org/apache/pirk/querier/wideskies/QuerierCLI.java
---
@@ -131,12 +125,28 @@ private boolean parseOptions()
// Validate properties
valid = QuerierProps.validateQuerierProperties();
+// Load the new local query and data schemas
+if (valid)
+{
+ logger.info("loading schemas: dataSchemas = " +
SystemConfiguration.getProperty("data.schemas") + " querySchemas = " +
SystemConfiguration
+ .getProperty("query.schemas"));
+ try
+ {
+DataSchemaLoader.initialize();
+QuerySchemaLoader.initialize();
+
+ } catch (Exception e)
--- End diff --
It would be much better to handle the exception, or re-throw it, rather
than consume and continue.
Why catch all Exception types?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #105: [PIRK 71] Move Querier creation logic into...
Github user tellison commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/105#discussion_r81305924
--- Diff: src/main/java/org/apache/pirk/querier/wideskies/QuerierProps.java
---
@@ -60,154 +59,179 @@
// Decryption properties
static final String QUERIERFILE = "querier.querierFile";
- static final List PROPSLIST = Arrays.asList(ACTION, INPUTFILE,
OUTPUTFILE, QUERYTYPE, NUMTHREADS, EMBEDQUERYSCHEMA, HASHBITSIZE, HASHKEY,
- DATAPARTITIONSIZE, PAILLIERBITSIZE, BITSET, CERTAINTY, QUERYSCHEMAS,
DATASCHEMAS, EMBEDSELECTOR, USEMEMLOOKUPTABLE, USEHDFSLOOKUPTABLE, SR_ALGORITHM,
- SR_PROVIDER);
+ static final List PROPSLIST = Arrays
+ .asList(ACTION, INPUTFILE, OUTPUTFILE, QUERYTYPE, NUMTHREADS,
EMBEDQUERYSCHEMA, HASHBITSIZE, HASHKEY, DATAPARTITIONSIZE, PAILLIERBITSIZE,
BITSET,
+ CERTAINTY, QUERYSCHEMAS, DATASCHEMAS, EMBEDSELECTOR,
USEMEMLOOKUPTABLE, USEHDFSLOOKUPTABLE, SR_ALGORITHM, SR_PROVIDER);
- /**
- * Validates the querier properties
- *
- */
public static boolean validateQuerierProperties()
{
+setGeneralDefaults(SystemConfiguration.getProperties());
+
if(validateGeneralQuerierProperties(SystemConfiguration.getProperties())) {
+ String action =
SystemConfiguration.getProperty(ACTION).toLowerCase();
--- End diff --
Trivial: does not follow house formatting rules ;-)
I believe this has subsequently been fixed by @ellisonanne
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #105: [PIRK 71] Move Querier creation logic into...
Github user tellison commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/105#discussion_r81305311
--- Diff:
src/main/java/org/apache/pirk/querier/wideskies/QuerierFactory.java ---
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pirk.querier.wideskies;
+
+import org.apache.pirk.encryption.Paillier;
+import org.apache.pirk.querier.wideskies.encrypt.EncryptQuery;
+import org.apache.pirk.query.wideskies.QueryInfo;
+import org.apache.pirk.schema.query.QuerySchemaRegistry;
+import org.apache.pirk.utils.PIRException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.math.BigInteger;
+import java.util.List;
+import java.util.Properties;
+import java.util.UUID;
+
+/**
+ * Handles encrypting a query and constructing a {@link Querier} given a
{@link EncryptionPropertiesBuilder}.
+ *
+ * Created by ryan on 9/28/16.
--- End diff --
Please drop author info in comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #105: [PIRK 71] Move Querier creation logic into...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-pirk/pull/105 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
