[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[
https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565701#comment-15565701
]
ASF GitHub Bot commented on PIRK-45:
Github user tellison commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/107#discussion_r82813992
--- Diff: src/main/java/org/apache/pirk/query/wideskies/Query.java ---
@@ -26,41 +26,51 @@
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Consumer;
+import com.google.gson.annotations.Expose;
import org.apache.pirk.encryption.ModPowAbstraction;
import org.apache.pirk.serialization.Storable;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Class to hold the PIR query vectors
- *
*/
+
public class Query implements Serializable, Storable
{
- private static final long serialVersionUID = 1L;
+ public static final long querySerialVersionUID = 1L;
+
+ // So that we can serialize the version number in gson.
+ @Expose public final long queryVersion = querySerialVersionUID;
private static final Logger logger =
LoggerFactory.getLogger(Query.class);
- private final QueryInfo queryInfo; // holds all query info
+ @Expose private final QueryInfo queryInfo; // holds all query info
- private final SortedMap queryElements; // query
elements - ordered on insertion
+ @Expose private final SortedMap queryElements; //
query elements - ordered on insertion
// lookup table for exponentiation of query vectors - based on
dataPartitionBitSize
// element ->
private Map> expTable = new
ConcurrentHashMap<>();
// File based lookup table for modular exponentiation
// element hash -> filename containing it's modular exponentiations
- private Map expFileBasedLookup = new HashMap<>();
+ @Expose private Map expFileBasedLookup = new HashMap<>();
--- End diff --
Should the hash -> filename map be serialized?
AFAIK it is only used by the responder to compute the results, so I don't
think it needs serializing.
> Remove dependency on Java serialization
> ---
>
> Key: PIRK-45
> URL: https://issues.apache.org/jira/browse/PIRK-45
> Project: PIRK
> Issue Type: Improvement
>Reporter: Tim Ellison
>Assignee: Walter Ray-Dulany
>
> Pirk should not depend upon Java serialization as a persistent object format.
> Maintaining support for a variety of versions of Java serialized form can be
> difficult, this includes both the querier and responder sides of a PIR.
> Alternative formats such as XML and JSON are more forgiving/extensible.
> Furthermore, and despite Pirk's trust between querier and responder, there
> are potential implications for loading the binary representation of Java's
> serialized instances as a vector for security vulnerabilities.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[
https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565700#comment-15565700
]
ASF GitHub Bot commented on PIRK-45:
Github user tellison commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/107#discussion_r82814228
--- Diff: src/main/java/org/apache/pirk/query/wideskies/Query.java ---
@@ -114,8 +124,7 @@ public void generateExpTable()
queryElements.values().parallelStream().forEach(new
Consumer()
{
- @Override
- public void accept(BigInteger element)
+ @Override public void accept(BigInteger element)
--- End diff --
IDE formatting wars ;-)
> Remove dependency on Java serialization
> ---
>
> Key: PIRK-45
> URL: https://issues.apache.org/jira/browse/PIRK-45
> Project: PIRK
> Issue Type: Improvement
>Reporter: Tim Ellison
>Assignee: Walter Ray-Dulany
>
> Pirk should not depend upon Java serialization as a persistent object format.
> Maintaining support for a variety of versions of Java serialized form can be
> difficult, this includes both the querier and responder sides of a PIR.
> Alternative formats such as XML and JSON are more forgiving/extensible.
> Furthermore, and despite Pirk's trust between querier and responder, there
> are potential implications for loading the binary representation of Java's
> serialized instances as a vector for security vulnerabilities.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565699#comment-15565699 ] ASF GitHub Bot commented on PIRK-45: Github user tellison commented on a diff in the pull request: https://github.com/apache/incubator-pirk/pull/107#discussion_r82813654 --- Diff: src/main/java/org/apache/pirk/query/wideskies/Query.java --- @@ -26,41 +26,51 @@ import java.util.concurrent.ConcurrentHashMap; import java.util.function.Consumer; +import com.google.gson.annotations.Expose; import org.apache.pirk.encryption.ModPowAbstraction; import org.apache.pirk.serialization.Storable; import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** * Class to hold the PIR query vectors - * */ + public class Query implements Serializable, Storable --- End diff -- I assume we can also drop ```implements Serializable``` now too, right? This instance, and friends elsewhere. > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562451#comment-15562451 ] ASF GitHub Bot commented on PIRK-45: Github user asfgit closed the pull request at: https://github.com/apache/incubator-pirk/pull/107 > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562405#comment-15562405 ] ASF GitHub Bot commented on PIRK-45: Github user ellisonanne commented on the issue: https://github.com/apache/incubator-pirk/pull/107 +1 - looks good to merge > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562084#comment-15562084 ] ASF GitHub Bot commented on PIRK-45: Github user wraydulany commented on the issue: https://github.com/apache/incubator-pirk/pull/107 Ok, changing to full PR. I should note for the record that the previously-working-for-me Pirk Releases also no longer pass the ES and ESS tests for me, another sign that It's Not My Fault. > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560695#comment-15560695 ] ASF GitHub Bot commented on PIRK-45: Github user wraydulany commented on the issue: https://github.com/apache/incubator-pirk/pull/107 Ok, I've expurgated references (import and comment) to jackson in the code, and I've run the style checker against the code i touched. Additionally, I've investigated the problems with the 1:ES and 1:ESS tests that I'm having, and I'm convinced that they are due to misconfiguration of the cluster I'm testing on. This passes the build tests and distributed tests 1:J, 1:JS, 1:JSS, 1:SS, and 1:E for me. If anyone has been able to successfully execute the 1:ES and/or 1:ESS tests with this PR *and* fail them, please comment and kill this PR. Otherwise, I'm going to move this from WIP to full PR in the late morning on the US East Coast tomorrow. > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15558302#comment-15558302 ] ASF GitHub Bot commented on PIRK-45: Github user ellisonanne commented on the issue: https://github.com/apache/incubator-pirk/pull/107 A couple of general comments - (1) need to remove the Jackson imports and dependency as it GSON is being used (2) need to format with the Pirk codestyle file. I'll take a look at the ES issues... > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15558296#comment-15558296 ] ASF GitHub Bot commented on PIRK-45: Github user ellisonanne commented on a diff in the pull request: https://github.com/apache/incubator-pirk/pull/107#discussion_r82505758 --- Diff: src/main/java/org/apache/pirk/query/wideskies/Query.java --- @@ -26,100 +26,103 @@ import java.util.concurrent.ConcurrentHashMap; import java.util.function.Consumer; +import com.fasterxml.jackson.annotation.JsonIgnore; --- End diff -- Need to remove Jackson imports > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15558293#comment-15558293 ] ASF GitHub Bot commented on PIRK-45: Github user ellisonanne commented on a diff in the pull request: https://github.com/apache/incubator-pirk/pull/107#discussion_r82505750 --- Diff: src/main/java/org/apache/pirk/querier/wideskies/QuerierDeserializer.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pirk.querier.wideskies; + +import com.fasterxml.jackson.databind.JsonNode; + +import com.google.gson.Gson; +import com.google.gson.JsonDeserializationContext; +import com.google.gson.JsonDeserializer; +import com.google.gson.JsonElement; +import com.google.gson.JsonObject; +import com.google.gson.JsonParseException; +import com.google.gson.reflect.TypeToken; +import org.apache.pirk.encryption.Paillier; +import org.apache.pirk.query.wideskies.Query; + + +import java.lang.reflect.Type; +import java.math.BigInteger; +import java.util.List; +import java.util.Map; + +/** + * Custom deserializer for Querier class for Jackson. --- End diff -- Need to remove Jackson reference since using GSON > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556261#comment-15556261 ] ASF GitHub Bot commented on PIRK-45: Github user wraydulany commented on the issue: https://github.com/apache/incubator-pirk/pull/107 Fixed the `import...*` and also removed some code garbage that i'd commented out, from an earlier attempt to use jackson, an attempt killed by spark and its ancient version of jackson. > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556211#comment-15556211 ] ASF GitHub Bot commented on PIRK-45: Github user wraydulany commented on a diff in the pull request: https://github.com/apache/incubator-pirk/pull/107#discussion_r82467453 --- Diff: src/main/java/org/apache/pirk/query/wideskies/QueryDeserializer.java --- @@ -0,0 +1,265 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pirk.query.wideskies; + +import com.fasterxml.jackson.core.type.TypeReference; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.DeserializationContext; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.deser.std.StdDeserializer; +import com.google.gson.*; +import com.google.gson.reflect.TypeToken; +import org.apache.pirk.schema.query.QuerySchema; +import org.apache.pirk.schema.query.filter.DataFilter; +import org.apache.pirk.schema.query.filter.FilterFactory; +import org.apache.pirk.utils.PIRException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.IOException; +import java.lang.reflect.Type; +import java.math.BigInteger; +import java.util.*; --- End diff -- Yes, indeed, sorry; my IDE has been doing that automatically. Let me go fix all of those. > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556216#comment-15556216 ] ASF GitHub Bot commented on PIRK-45: Github user wraydulany commented on a diff in the pull request: https://github.com/apache/incubator-pirk/pull/107#discussion_r82467602 --- Diff: src/main/java/org/apache/pirk/query/wideskies/QueryDeserializer.java --- @@ -0,0 +1,265 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pirk.query.wideskies; + +import com.fasterxml.jackson.core.type.TypeReference; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.DeserializationContext; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.deser.std.StdDeserializer; +import com.google.gson.*; +import com.google.gson.reflect.TypeToken; +import org.apache.pirk.schema.query.QuerySchema; +import org.apache.pirk.schema.query.filter.DataFilter; +import org.apache.pirk.schema.query.filter.FilterFactory; +import org.apache.pirk.utils.PIRException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.IOException; +import java.lang.reflect.Type; +import java.math.BigInteger; +import java.util.*; --- End diff -- Yeah, there are 8 of those. Fixing. > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556202#comment-15556202 ] ASF GitHub Bot commented on PIRK-45: Github user smarthi commented on a diff in the pull request: https://github.com/apache/incubator-pirk/pull/107#discussion_r82467067 --- Diff: src/main/java/org/apache/pirk/querier/wideskies/QuerierDeserializer.java --- @@ -0,0 +1,117 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pirk.querier.wideskies; + +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.core.type.TypeReference; +import com.fasterxml.jackson.databind.DeserializationContext; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.deser.std.StdDeserializer; +import com.google.gson.*; --- End diff -- Please avoid using import *. > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556205#comment-15556205 ] ASF GitHub Bot commented on PIRK-45: Github user smarthi commented on a diff in the pull request: https://github.com/apache/incubator-pirk/pull/107#discussion_r82467151 --- Diff: src/main/java/org/apache/pirk/query/wideskies/QueryDeserializer.java --- @@ -0,0 +1,265 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pirk.query.wideskies; + +import com.fasterxml.jackson.core.type.TypeReference; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.core.JsonParser; +import com.fasterxml.jackson.core.JsonProcessingException; +import com.fasterxml.jackson.databind.DeserializationContext; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.deser.std.StdDeserializer; +import com.google.gson.*; +import com.google.gson.reflect.TypeToken; +import org.apache.pirk.schema.query.QuerySchema; +import org.apache.pirk.schema.query.filter.DataFilter; +import org.apache.pirk.schema.query.filter.FilterFactory; +import org.apache.pirk.utils.PIRException; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.IOException; +import java.lang.reflect.Type; +import java.math.BigInteger; +import java.util.*; --- End diff -- avoid using import * > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PIRK-45) Remove dependency on Java serialization
[ https://issues.apache.org/jira/browse/PIRK-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15556158#comment-15556158 ] ASF GitHub Bot commented on PIRK-45: GitHub user wraydulany opened a pull request: https://github.com/apache/incubator-pirk/pull/107 [WIP] PIRK-45 Remove dependency on Java serialization This is a WIP (do not merge) for adding in JSON serialization. I can't get this WIP to pass the elasticsearch spark streaming tests on my cluster. I'm fairly sure that this is a configuration problem on my end (the errors I get appear to be such), but I'm having little luck resolving them, and would love it if some of the community could give it a test. I've left in Java Serialization for now, but I've made JSON serialization the default. You can merge this pull request into a Git repository by running: $ git pull https://github.com/wraydulany/incubator-pirk PIRK-45 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-pirk/pull/107.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #107 commit c2c6af2465e21cd706c8cfd0dc4f95f7746d86f0 Author: Walter Ray-Dulany Date: 2016-09-28T15:13:12Z Initial start on moving away from Java object serialization. commit 66235a9c6ac07e3cec6a94c0874b40bf6624192e Author: Walter Ray-Dulany Date: 2016-09-29T17:02:17Z Initial work. commit 03b4964c2b25538aaecfdcd56a408bcba8154688 Author: Walter Ray-Dulany Date: 2016-10-01T17:20:09Z Merge branch 'master' of https://github.com/wraydulany/incubator-pirk into PIRK-45 commit de01c5ad1e670200cefa6c7dfc2b678cb245eaf6 Author: Walter Ray-Dulany Date: 2016-10-01T17:21:27Z Merge branch 'master' of https://github.com/wraydulany/incubator-pirk into PIRK-45 commit 7b221de870f7efc8b41ec52fd5d05af06360c405 Author: Walter Ray-Dulany Date: 2016-10-04T18:24:18Z Added deserializer for query; I think I should delete the serializer for QueryInfo. commit fdeb7d77117691f7dda63a2a180d805861216893 Author: Walter Ray-Dulany Date: 2016-10-04T19:21:22Z Query serialization seems to be working now; at least it doesn't fail to get recalled. commit b31afcfe7f01ebda6a19e27b5e651f8dc8763f16 Author: Walter Ray-Dulany Date: 2016-10-04T19:42:20Z Response deserializer compiles. commit 42e2bb6082a601e5e503e76170499baff37fc478 Author: Walter Ray-Dulany Date: 2016-10-05T12:31:51Z Custom deserializers complete, as are class amendments to make them work. commit 603fcaa777f754dbf60836a8a81b7239b39ae494 Author: Walter Ray-Dulany Date: 2016-10-05T14:29:21Z Merge remote-tracking branch 'upstream/master' into PIRK-45 commit fbebc4adafefa640e5b0339d2423abdabed9b0b3 Author: Walter Ray-Dulany Date: 2016-10-05T15:59:55Z Needed to add a new constructor for QueryInfo so as to not depend on the existence of a populated QuerySchemaRegistry. commit b9caf260f14405705864ba200d993057a7040416 Author: Walter Ray-Dulany Date: 2016-10-06T12:16:34Z Move to gson breaks hdfs? commit dc9fcfd848c89bf59310c0c12284b5bbb72caec4 Author: Walter Ray-Dulany Date: 2016-10-06T12:49:45Z Working with Gson on two tests; time for distributed tests. commit 99cddd1981f76569d63add70a6351ba744ca13c9 Author: Walter Ray-Dulany Date: 2016-10-07T20:19:48Z Merge remote-tracking branch 'upstream/master' into PIRK-45 > Remove dependency on Java serialization > --- > > Key: PIRK-45 > URL: https://issues.apache.org/jira/browse/PIRK-45 > Project: PIRK > Issue Type: Improvement >Reporter: Tim Ellison >Assignee: Walter Ray-Dulany > > Pirk should not depend upon Java serialization as a persistent object format. > Maintaining support for a variety of versions of Java serialized form can be > difficult, this includes both the querier and responder sides of a PIR. > Alternative formats such as XML and JSON are more forgiving/extensible. > Furthermore, and despite Pirk's trust between querier and responder, there > are potential implications for loading the binary representation of Java's > serialized instances as a vector for security vulnerabilities. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
