Re: Quickstep Network Mode and C++ Sockets

[Jianqiao]

Yes. NetworkCliClient is quite standalone. The dependencies are:

(1) #include 
(2) #include "cli/NetworkCli.grpc.pb.h"
(3) #include "cli/NetworkCli.pb.h"
(4) #include "utility/Macros.hpp"


To write your own client:

(A) Setup grpc so that you can include the header files and link to it.

(B) Grab NetworkCli.proto (https://github.com/apache/
incubator-quickstep/blob/master/cli/NetworkCli.proto), change the package
name if necessary (originally quickstep).
Either
(B.1) use grpc/protobuf tools to compile NetworkCli.proto to generate (2)
and (3) -- see https://github.com/apache/incubator-quickstep/blob/
master/cli/CMakeLists.txt#L53
or
(B.2) compile quickstep and grab the files from build/cli/

(C) Copy the NetworkCliClient class from QS into your client code.


(A)/(B) may be somehow annoying to handle as you need to search through
various documentations ...


Best,
Jianqiao

2017-11-30 17:07 GMT-06:00 Dylan Bacon :

> So NetworkCliClient should be something I'm able to include in my program
> along with the appropriate dependencies and use as the API? I was thinking
> about needing to do that but I wasn't sure if that was a standalone API QS
> has implemented or a core part of the system. Unless I'm being mistaken and
> you're talking about something from gRPC. This is my first time working
> with it.
>
>
>
> On 11/30/17 4:58 PM, Jianqiao wrote:
>
>> Hi Dylan,
>>
>> Currently the network mode is using gRPC, so you probably need to use the
>> corresponding API (see
>> https://github.com/apache/incubator-quickstep/blob/master/
>> cli/NetworkCliClientMain.cpp#L42
>> as an example). The raw socket connection won't work unless you hack
>> gRPC's
>> message exchange protocol ..
>>
>> Best,
>> Jianqiao
>>
>> 2017-11-30 16:49 GMT-06:00 Dylan Bacon :
>>
>> Hello,
>>>
>>> I am attempting to interface with Quickstep using its NetworkCliClient
>>> and
>>> it's not working as I would expect. I have the default port and IP set to
>>> 3000 and 0.0.0.0 and am attempting to send single queries to be processed
>>> over in my test harness. From what I could tell of the code when QS is in
>>> network mode it accepts a socket connection and string input from that
>>> function and processes it in NetworkCliClient.hpp and
>>> NetworkCliClientMain.cpp, and yet this is not happening with my test
>>> code.
>>> The connection is being established but Quickstep does not seem to be
>>> doing
>>> anything with the queries that come in.
>>>
>>> Attached is the test code that I am using. test is just a table by that
>>> name, I'm selecting a literal from it so the contents shouldn't matter.
>>> I've also attempted to create a table with this but Quickstep did not
>>> process that.
>>>
>>> --
>>> Regards,
>>>
>>> Dylan Bacon
>>> University of Wisconsin - Madison
>>> Department of Computer Sciences
>>> dba...@wisc.edu
>>>
>>>
>>>
>

[GitHub] incubator-quickstep pull request #326: QUICKSTEP-112 Get the list of referen...

[jianqiao]

Github user jianqiao commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/326#discussion_r154230379
  
--- Diff: query_optimizer/rules/ReferencedBaseRelations.hpp ---
@@ -0,0 +1,78 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#ifndef QUICKSTEP_QUERY_OPTIMIZER_RULES_REFERENCED_BASE_RELATIONS_HPP_
+#define QUICKSTEP_QUERY_OPTIMIZER_RULES_REFERENCED_BASE_RELATIONS_HPP_
+
+#include 
+#include 
+
+#include "catalog/CatalogTypedefs.hpp"
+#include "query_optimizer/logical/Logical.hpp"
+#include "query_optimizer/rules/DFSTraversal.hpp"
+#include "utility/Macros.hpp"
+
+namespace quickstep {
+
+class CatalogRelation;
+
+namespace optimizer {
+
+class OptimizerContext;
+
+class ReferencedBaseRelations : public DFSTraversal {
--- End diff --

Since this class overrides the `apply` method, we can just inherit 
`Rule`.


---

[GitHub] incubator-quickstep pull request #326: QUICKSTEP-112 Get the list of referen...

[jianqiao]

Github user jianqiao commented on a diff in the pull request:

https://github.com/apache/incubator-quickstep/pull/326#discussion_r154229622
  
--- Diff: query_optimizer/rules/ReferencedBaseRelations.hpp ---
@@ -0,0 +1,78 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ **/
+
+#ifndef QUICKSTEP_QUERY_OPTIMIZER_RULES_REFERENCED_BASE_RELATIONS_HPP_
+#define QUICKSTEP_QUERY_OPTIMIZER_RULES_REFERENCED_BASE_RELATIONS_HPP_
+
+#include 
+#include 
+
+#include "catalog/CatalogTypedefs.hpp"
+#include "query_optimizer/logical/Logical.hpp"
+#include "query_optimizer/rules/DFSTraversal.hpp"
+#include "utility/Macros.hpp"
+
+namespace quickstep {
+
+class CatalogRelation;
+
+namespace optimizer {
+
+class OptimizerContext;
+
+class ReferencedBaseRelations : public DFSTraversal {
+ public:
+  /**
+   * @brief Constructor
+   * @param optimizer_context The optimizer context.
+   */
+  explicit ReferencedBaseRelations(OptimizerContext *optimizer_context)
+  : optimizer_context_(optimizer_context) {
+  }
+
+  std::string getName() const override { return "ReferencedBaseRelations"; 
}
+
+  TreeNodePtr apply(const TreeNodePtr ) override;
+
+  /**
+   * @brief Get the base relations referenced in a query.
+   */
+  const std::vector getReferencedBaseRelations() 
const {
--- End diff --

Better remove the beginning `const`, as the method returns a temporary 
object.


---

Re: Quickstep Network Mode and C++ Sockets

[Dylan Bacon]

So NetworkCliClient should be something I'm able to include in my 
program along with the appropriate dependencies and use as the API? I 
was thinking about needing to do that but I wasn't sure if that was a 
standalone API QS has implemented or a core part of the system. Unless 
I'm being mistaken and you're talking about something from gRPC. This is 
my first time working with it.



On 11/30/17 4:58 PM, Jianqiao wrote:

Hi Dylan,

Currently the network mode is using gRPC, so you probably need to use the
corresponding API (see
https://github.com/apache/incubator-quickstep/blob/master/cli/NetworkCliClientMain.cpp#L42
as an example). The raw socket connection won't work unless you hack gRPC's
message exchange protocol ..

Best,
Jianqiao

2017-11-30 16:49 GMT-06:00 Dylan Bacon :


Hello,

I am attempting to interface with Quickstep using its NetworkCliClient and
it's not working as I would expect. I have the default port and IP set to
3000 and 0.0.0.0 and am attempting to send single queries to be processed
over in my test harness. From what I could tell of the code when QS is in
network mode it accepts a socket connection and string input from that
function and processes it in NetworkCliClient.hpp and
NetworkCliClientMain.cpp, and yet this is not happening with my test code.
The connection is being established but Quickstep does not seem to be doing
anything with the queries that come in.

Attached is the test code that I am using. test is just a table by that
name, I'm selecting a literal from it so the contents shouldn't matter.
I've also attempted to create a table with this but Quickstep did not
process that.

--
Regards,

Dylan Bacon
University of Wisconsin - Madison
Department of Computer Sciences
dba...@wisc.edu

Re: Quickstep Network Mode and C++ Sockets

[Jianqiao]

Hi Dylan,

Currently the network mode is using gRPC, so you probably need to use the
corresponding API (see
https://github.com/apache/incubator-quickstep/blob/master/cli/NetworkCliClientMain.cpp#L42
as an example). The raw socket connection won't work unless you hack gRPC's
message exchange protocol ..

Best,
Jianqiao

2017-11-30 16:49 GMT-06:00 Dylan Bacon :

> Hello,
>
> I am attempting to interface with Quickstep using its NetworkCliClient and
> it's not working as I would expect. I have the default port and IP set to
> 3000 and 0.0.0.0 and am attempting to send single queries to be processed
> over in my test harness. From what I could tell of the code when QS is in
> network mode it accepts a socket connection and string input from that
> function and processes it in NetworkCliClient.hpp and
> NetworkCliClientMain.cpp, and yet this is not happening with my test code.
> The connection is being established but Quickstep does not seem to be doing
> anything with the queries that come in.
>
> Attached is the test code that I am using. test is just a table by that
> name, I'm selecting a literal from it so the contents shouldn't matter.
> I've also attempted to create a table with this but Quickstep did not
> process that.
>
> --
> Regards,
>
> Dylan Bacon
> University of Wisconsin - Madison
> Department of Computer Sciences
> dba...@wisc.edu
>
>

Quickstep Network Mode and C++ Sockets

[Dylan Bacon]

Hello,

I am attempting to interface with Quickstep using its NetworkCliClient 
and it's not working as I would expect. I have the default port and IP 
set to 3000 and 0.0.0.0 and am attempting to send single queries to be 
processed over in my test harness. From what I could tell of the code 
when QS is in network mode it accepts a socket connection and string 
input from that function and processes it in NetworkCliClient.hpp and 
NetworkCliClientMain.cpp, and yet this is not happening with my test 
code. The connection is being established but Quickstep does not seem to 
be doing anything with the queries that come in.


Attached is the test code that I am using. test is just a table by that 
name, I'm selecting a literal from it so the contents shouldn't matter. 
I've also attempted to create a table with this but Quickstep did not 
process that.


--
Regards,

Dylan Bacon
University of Wisconsin - Madison
Department of Computer Sciences
dba...@wisc.edu

// Client side C/C++ program to demonstrate Socket programming
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#define PORT 3000
  
int main(int argc, char const *argv[])
{
struct sockaddr_in address;
int sock = 0, valread;
struct sockaddr_in serv_addr;
char *query = "SELECT 1 FROM test;";
char buffer[1024] = {0};
if ((sock = socket(AF_INET, SOCK_STREAM, 0)) < 0)
{
printf("\n Socket creation error \n");
return -1;
}
  
memset(_addr, '0', sizeof(serv_addr));
  
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons(PORT);
  
// Convert IPv4 and IPv6 addresses from text to binary form
if(inet_pton(AF_INET, "0.0.0.0", _addr.sin_addr)<=0) 
{
printf("\nInvalid address/ Address not supported \n");
return -1;
}
  
if (connect(sock, (struct sockaddr *)_addr, sizeof(serv_addr)) < 0)
{
printf("\nConnection Failed.\n");
return -1;
}
send(sock, query, strlen(query), 0);
printf("Query sent.\n");
valread = read(sock, buffer, 1024);
printf("%s\n",buffer );
return 0;
}