subject:"\[GitHub\] \[skywalking\] kezhenxu94 commented on a change in pull request #4239\: Provide influxdb as a new storage plugin"

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-17 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r380048045
 
 

 ##
 File path: 
test/e2e/e2e-ttl/e2e-ttl-influxdb/src/test/java/org/apache/skywalking/e2e/StorageTTLITCase.java
 ##
 @@ -0,0 +1,285 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.skywalking.e2e;
+
+import io.grpc.ManagedChannel;
+import io.grpc.ManagedChannelBuilder;
+import io.grpc.internal.DnsNameResolverProvider;
+import io.grpc.netty.NettyChannelBuilder;
+import io.grpc.stub.StreamObserver;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.DetectPoint;
+import org.apache.skywalking.apm.network.servicemesh.MeshProbeDownstream;
+import org.apache.skywalking.apm.network.servicemesh.Protocol;
+import org.apache.skywalking.apm.network.servicemesh.ServiceMeshMetric;
+import 
org.apache.skywalking.apm.network.servicemesh.ServiceMeshMetricServiceGrpc;
+import org.apache.skywalking.e2e.metrics.AllOfMetricsMatcher;
+import org.apache.skywalking.e2e.metrics.AtLeastOneOfMetricsMatcher;
+import org.apache.skywalking.e2e.metrics.Metrics;
+import org.apache.skywalking.e2e.metrics.MetricsQuery;
+import org.apache.skywalking.e2e.metrics.MetricsValueMatcher;
+import org.apache.skywalking.e2e.service.Service;
+import org.apache.skywalking.e2e.service.ServicesQuery;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.util.List;
+import java.util.concurrent.CountDownLatch;
+
+import static org.apache.skywalking.e2e.metrics.MetricsQuery.SERVICE_RESP_TIME;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/**
+ * @author kezhenxu94
+ */
 
 Review comment:
   > Why isn't this detected by CI process?
   
   CI belongs to the root module and its submodules, e2e and plugin tests are 
neither of them, maybe set up the same Checkstyle plugin in e2e and plugin 
tests?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-17 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r380045349
 
 

 ##
 File path: 
test/e2e/e2e-ttl/e2e-ttl-influxdb/src/test/java/org/apache/skywalking/e2e/StorageTTLITCase.java
 ##
 @@ -0,0 +1,285 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.skywalking.e2e;
+
+import io.grpc.ManagedChannel;
+import io.grpc.ManagedChannelBuilder;
+import io.grpc.internal.DnsNameResolverProvider;
+import io.grpc.netty.NettyChannelBuilder;
+import io.grpc.stub.StreamObserver;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.DetectPoint;
+import org.apache.skywalking.apm.network.servicemesh.MeshProbeDownstream;
+import org.apache.skywalking.apm.network.servicemesh.Protocol;
+import org.apache.skywalking.apm.network.servicemesh.ServiceMeshMetric;
+import 
org.apache.skywalking.apm.network.servicemesh.ServiceMeshMetricServiceGrpc;
+import org.apache.skywalking.e2e.metrics.AllOfMetricsMatcher;
+import org.apache.skywalking.e2e.metrics.AtLeastOneOfMetricsMatcher;
+import org.apache.skywalking.e2e.metrics.Metrics;
+import org.apache.skywalking.e2e.metrics.MetricsQuery;
+import org.apache.skywalking.e2e.metrics.MetricsValueMatcher;
+import org.apache.skywalking.e2e.service.Service;
+import org.apache.skywalking.e2e.service.ServicesQuery;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.util.List;
+import java.util.concurrent.CountDownLatch;
+
+import static org.apache.skywalking.e2e.metrics.MetricsQuery.SERVICE_RESP_TIME;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/**
+ * @author kezhenxu94
+ */
 
 Review comment:
   remove this, and those in other files


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-17 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r380045349
 
 

 ##
 File path: 
test/e2e/e2e-ttl/e2e-ttl-influxdb/src/test/java/org/apache/skywalking/e2e/StorageTTLITCase.java
 ##
 @@ -0,0 +1,285 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.skywalking.e2e;
+
+import io.grpc.ManagedChannel;
+import io.grpc.ManagedChannelBuilder;
+import io.grpc.internal.DnsNameResolverProvider;
+import io.grpc.netty.NettyChannelBuilder;
+import io.grpc.stub.StreamObserver;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.DetectPoint;
+import org.apache.skywalking.apm.network.servicemesh.MeshProbeDownstream;
+import org.apache.skywalking.apm.network.servicemesh.Protocol;
+import org.apache.skywalking.apm.network.servicemesh.ServiceMeshMetric;
+import 
org.apache.skywalking.apm.network.servicemesh.ServiceMeshMetricServiceGrpc;
+import org.apache.skywalking.e2e.metrics.AllOfMetricsMatcher;
+import org.apache.skywalking.e2e.metrics.AtLeastOneOfMetricsMatcher;
+import org.apache.skywalking.e2e.metrics.Metrics;
+import org.apache.skywalking.e2e.metrics.MetricsQuery;
+import org.apache.skywalking.e2e.metrics.MetricsValueMatcher;
+import org.apache.skywalking.e2e.service.Service;
+import org.apache.skywalking.e2e.service.ServicesQuery;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.util.List;
+import java.util.concurrent.CountDownLatch;
+
+import static org.apache.skywalking.e2e.metrics.MetricsQuery.SERVICE_RESP_TIME;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/**
+ * @author kezhenxu94
+ */
 
 Review comment:
   remove this, and other files with the `@author`


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-17 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r380045349
 
 

 ##
 File path: 
test/e2e/e2e-ttl/e2e-ttl-influxdb/src/test/java/org/apache/skywalking/e2e/StorageTTLITCase.java
 ##
 @@ -0,0 +1,285 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.skywalking.e2e;
+
+import io.grpc.ManagedChannel;
+import io.grpc.ManagedChannelBuilder;
+import io.grpc.internal.DnsNameResolverProvider;
+import io.grpc.netty.NettyChannelBuilder;
+import io.grpc.stub.StreamObserver;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.DetectPoint;
+import org.apache.skywalking.apm.network.servicemesh.MeshProbeDownstream;
+import org.apache.skywalking.apm.network.servicemesh.Protocol;
+import org.apache.skywalking.apm.network.servicemesh.ServiceMeshMetric;
+import 
org.apache.skywalking.apm.network.servicemesh.ServiceMeshMetricServiceGrpc;
+import org.apache.skywalking.e2e.metrics.AllOfMetricsMatcher;
+import org.apache.skywalking.e2e.metrics.AtLeastOneOfMetricsMatcher;
+import org.apache.skywalking.e2e.metrics.Metrics;
+import org.apache.skywalking.e2e.metrics.MetricsQuery;
+import org.apache.skywalking.e2e.metrics.MetricsValueMatcher;
+import org.apache.skywalking.e2e.service.Service;
+import org.apache.skywalking.e2e.service.ServicesQuery;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.time.LocalDateTime;
+import java.time.ZoneOffset;
+import java.util.List;
+import java.util.concurrent.CountDownLatch;
+
+import static org.apache.skywalking.e2e.metrics.MetricsQuery.SERVICE_RESP_TIME;
+import static org.assertj.core.api.Assertions.assertThat;
+
+/**
+ * @author kezhenxu94
+ */
 
 Review comment:
   remove this


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-16 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379902233
 
 

 ##
 File path: .github/workflows/e2e.yaml
 ##
 @@ -71,15 +73,19 @@ jobs:
   ./mvnw --batch-mode -Dcheckstyle.skip -Drat.skip -T2 
-Dmaven.compile.fork -Dmaven.compiler.maxmem=3072 -DskipTests clean install
   ./mvnw --batch-mode -f test/e2e/pom.xml -pl e2e-base clean install
   - name: Cluster Tests (ES6/ZK/JDK8)
-run: export E2E_VERSION=jdk8-1.3 && bash -x test/e2e/run.sh 
e2e-cluster/e2e-cluster-test-runner
+run: export E2E_VERSION=jdk8-1.5 && bash -x test/e2e/run.sh 
e2e-cluster/e2e-cluster-test-runner --storage=elasticsearch
+  - name: Cluster Tests (InfluxDB/ZK/JDK8)
+run: export E2E_VERSION=jdk8-1.5 && bash -x test/e2e/run.sh 
e2e-cluster/e2e-cluster-test-runner --storage=influxdb
   - name: Cluster With Gateway Tests (ES6/ZK/JDK8)
-run: export E2E_VERSION=jdk8-1.3 && bash -x test/e2e/run.sh 
e2e-cluster-with-gateway/e2e-cluster-with-gateway-test-runner
+run: export E2E_VERSION=jdk8-1.5 && bash -x test/e2e/run.sh 
e2e-cluster-with-gateway/e2e-cluster-with-gateway-test-runner
 
 Review comment:
   > Why this doesn't have `--storage=elasticsearch`? What is the default? Or 
what happens if no `--storage=`?
   
   Not every e2e case is aware of the parameter `--storage`, 
`e2e-cluster-with-gateway` is one of them, the parameter is introduced recently 
and only `e2e-cluster` and `e2e-profile` are refactored then, IIRC
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-16 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379902233
 
 

 ##
 File path: .github/workflows/e2e.yaml
 ##
 @@ -71,15 +73,19 @@ jobs:
   ./mvnw --batch-mode -Dcheckstyle.skip -Drat.skip -T2 
-Dmaven.compile.fork -Dmaven.compiler.maxmem=3072 -DskipTests clean install
   ./mvnw --batch-mode -f test/e2e/pom.xml -pl e2e-base clean install
   - name: Cluster Tests (ES6/ZK/JDK8)
-run: export E2E_VERSION=jdk8-1.3 && bash -x test/e2e/run.sh 
e2e-cluster/e2e-cluster-test-runner
+run: export E2E_VERSION=jdk8-1.5 && bash -x test/e2e/run.sh 
e2e-cluster/e2e-cluster-test-runner --storage=elasticsearch
+  - name: Cluster Tests (InfluxDB/ZK/JDK8)
+run: export E2E_VERSION=jdk8-1.5 && bash -x test/e2e/run.sh 
e2e-cluster/e2e-cluster-test-runner --storage=influxdb
   - name: Cluster With Gateway Tests (ES6/ZK/JDK8)
-run: export E2E_VERSION=jdk8-1.3 && bash -x test/e2e/run.sh 
e2e-cluster-with-gateway/e2e-cluster-with-gateway-test-runner
+run: export E2E_VERSION=jdk8-1.5 && bash -x test/e2e/run.sh 
e2e-cluster-with-gateway/e2e-cluster-with-gateway-test-runner
 
 Review comment:
   > Why this doesn't have `--storage=elasticsearch`? What is the default? Or 
what happens if no `--storage=`?
   
   Not every e2e case is aware of the parameter `--storage`, 
`e2e-cluster-with-gateway` is one of them, it's intruduced recently and only 
`e2e-cluster` and `e2e-profile` are refactored then, IIRC
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379867511
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/TraceQuery.java
 ##
 @@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.List;
+import lombok.extern.slf4j.Slf4j;
+import 
org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.query.entity.BasicTrace;
+import org.apache.skywalking.oap.server.core.query.entity.QueryOrder;
+import org.apache.skywalking.oap.server.core.query.entity.Span;
+import org.apache.skywalking.oap.server.core.query.entity.TraceBrief;
+import org.apache.skywalking.oap.server.core.query.entity.TraceState;
+import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.base.RecordDAO;
+import org.elasticsearch.common.Strings;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.SelectQueryImpl;
+import org.influxdb.querybuilder.WhereQueryImpl;
+import org.influxdb.querybuilder.clauses.Clause;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.eq;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.gte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.lte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.regex;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+@Slf4j
+public class TraceQuery implements ITraceQueryDAO {
+private final InfluxClient client;
+
+public TraceQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public TraceBrief queryBasicTraces(long startSecondTB,
+   long endSecondTB,
+   long minDuration,
+   long maxDuration,
+   String endpointName,
+   int serviceId,
+   int serviceInstanceId,
+   int endpointId,
+   String traceId,
+   int limit,
+   int from,
+   TraceState traceState,
+   QueryOrder queryOrder)
+throws IOException {
+
+String orderBy = SegmentRecord.START_TIME;
+if (queryOrder == QueryOrder.BY_DURATION) {
+orderBy = SegmentRecord.LATENCY;
+}
+
+WhereQueryImpl recallQuery = select()
+.function("top", orderBy, limit + from)
+.column(SegmentRecord.SEGMENT_ID)
+.column(SegmentRecord.START_TIME)
+.column(SegmentRecord.ENDPOINT_NAME)
+.column(SegmentRecord.LATENCY)
+.column(SegmentRecord.IS_ERROR)
+.column(SegmentRecord.TRACE_ID)
+.from(client.getDatabase(), SegmentRecord.INDEX_NAME)
+.where();
+
+if (startSecondTB != 0 && endSecondTB != 0) {
+recallQuery.and(gte(SegmentRecord.TIME_BUCKET, startSecondTB))
+   .and(lte(SegmentRecord.TIME_BUCKET, endSecondTB));
+}
+if (minDuration != 0) {
+recallQuery.and(gte(SegmentRecord.LATENCY, minDuration));
+}
+if (maxDuration != 0) {
+recallQuery.and(lte(SegmentRecord.LATENCY, maxDuration));
+}
+if (!Strings.isNullOrEmpty(endpointName)) {
+recallQuery.and(regex(SegmentRecord.

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379867129
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/TraceQuery.java
 ##
 @@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.List;
+import lombok.extern.slf4j.Slf4j;
+import 
org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.query.entity.BasicTrace;
+import org.apache.skywalking.oap.server.core.query.entity.QueryOrder;
+import org.apache.skywalking.oap.server.core.query.entity.Span;
+import org.apache.skywalking.oap.server.core.query.entity.TraceBrief;
+import org.apache.skywalking.oap.server.core.query.entity.TraceState;
+import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.base.RecordDAO;
+import org.elasticsearch.common.Strings;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.SelectQueryImpl;
+import org.influxdb.querybuilder.WhereQueryImpl;
+import org.influxdb.querybuilder.clauses.Clause;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.eq;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.gte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.lte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.regex;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+@Slf4j
+public class TraceQuery implements ITraceQueryDAO {
+private final InfluxClient client;
+
+public TraceQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public TraceBrief queryBasicTraces(long startSecondTB,
+   long endSecondTB,
+   long minDuration,
+   long maxDuration,
+   String endpointName,
+   int serviceId,
+   int serviceInstanceId,
+   int endpointId,
+   String traceId,
+   int limit,
+   int from,
+   TraceState traceState,
+   QueryOrder queryOrder)
+throws IOException {
+
+String orderBy = SegmentRecord.START_TIME;
+if (queryOrder == QueryOrder.BY_DURATION) {
+orderBy = SegmentRecord.LATENCY;
+}
+
+WhereQueryImpl recallQuery = select()
+.function("top", orderBy, limit + from)
+.column(SegmentRecord.SEGMENT_ID)
+.column(SegmentRecord.START_TIME)
+.column(SegmentRecord.ENDPOINT_NAME)
+.column(SegmentRecord.LATENCY)
+.column(SegmentRecord.IS_ERROR)
+.column(SegmentRecord.TRACE_ID)
+.from(client.getDatabase(), SegmentRecord.INDEX_NAME)
+.where();
+
+if (startSecondTB != 0 && endSecondTB != 0) {
+recallQuery.and(gte(SegmentRecord.TIME_BUCKET, startSecondTB))
+   .and(lte(SegmentRecord.TIME_BUCKET, endSecondTB));
+}
+if (minDuration != 0) {
+recallQuery.and(gte(SegmentRecord.LATENCY, minDuration));
+}
+if (maxDuration != 0) {
+recallQuery.and(lte(SegmentRecord.LATENCY, maxDuration));
+}
+if (!Strings.isNullOrEmpty(endpointName)) {
+recallQuery.and(regex(SegmentRecord.

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379867032
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/TraceQuery.java
 ##
 @@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.List;
+import lombok.extern.slf4j.Slf4j;
+import 
org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.query.entity.BasicTrace;
+import org.apache.skywalking.oap.server.core.query.entity.QueryOrder;
+import org.apache.skywalking.oap.server.core.query.entity.Span;
+import org.apache.skywalking.oap.server.core.query.entity.TraceBrief;
+import org.apache.skywalking.oap.server.core.query.entity.TraceState;
+import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.base.RecordDAO;
+import org.elasticsearch.common.Strings;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.SelectQueryImpl;
+import org.influxdb.querybuilder.WhereQueryImpl;
+import org.influxdb.querybuilder.clauses.Clause;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.eq;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.gte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.lte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.regex;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+@Slf4j
+public class TraceQuery implements ITraceQueryDAO {
+private final InfluxClient client;
+
+public TraceQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public TraceBrief queryBasicTraces(long startSecondTB,
+   long endSecondTB,
+   long minDuration,
+   long maxDuration,
+   String endpointName,
+   int serviceId,
+   int serviceInstanceId,
+   int endpointId,
+   String traceId,
+   int limit,
+   int from,
+   TraceState traceState,
+   QueryOrder queryOrder)
+throws IOException {
+
+String orderBy = SegmentRecord.START_TIME;
+if (queryOrder == QueryOrder.BY_DURATION) {
+orderBy = SegmentRecord.LATENCY;
+}
+
+WhereQueryImpl recallQuery = select()
+.function("top", orderBy, limit + from)
+.column(SegmentRecord.SEGMENT_ID)
+.column(SegmentRecord.START_TIME)
+.column(SegmentRecord.ENDPOINT_NAME)
+.column(SegmentRecord.LATENCY)
+.column(SegmentRecord.IS_ERROR)
+.column(SegmentRecord.TRACE_ID)
+.from(client.getDatabase(), SegmentRecord.INDEX_NAME)
+.where();
+
+if (startSecondTB != 0 && endSecondTB != 0) {
+recallQuery.and(gte(SegmentRecord.TIME_BUCKET, startSecondTB))
+   .and(lte(SegmentRecord.TIME_BUCKET, endSecondTB));
+}
+if (minDuration != 0) {
+recallQuery.and(gte(SegmentRecord.LATENCY, minDuration));
+}
+if (maxDuration != 0) {
+recallQuery.and(lte(SegmentRecord.LATENCY, maxDuration));
+}
+if (!Strings.isNullOrEmpty(endpointName)) {
+recallQuery.and(regex(SegmentRecord.

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379832725
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/TraceQuery.java
 ##
 @@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.List;
+import lombok.extern.slf4j.Slf4j;
+import 
org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.query.entity.BasicTrace;
+import org.apache.skywalking.oap.server.core.query.entity.QueryOrder;
+import org.apache.skywalking.oap.server.core.query.entity.Span;
+import org.apache.skywalking.oap.server.core.query.entity.TraceBrief;
+import org.apache.skywalking.oap.server.core.query.entity.TraceState;
+import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.base.RecordDAO;
+import org.elasticsearch.common.Strings;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.SelectQueryImpl;
+import org.influxdb.querybuilder.WhereQueryImpl;
+import org.influxdb.querybuilder.clauses.Clause;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.eq;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.gte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.lte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.regex;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+@Slf4j
+public class TraceQuery implements ITraceQueryDAO {
+private final InfluxClient client;
+
+public TraceQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public TraceBrief queryBasicTraces(long startSecondTB,
+   long endSecondTB,
+   long minDuration,
+   long maxDuration,
+   String endpointName,
+   int serviceId,
+   int serviceInstanceId,
+   int endpointId,
+   String traceId,
+   int limit,
+   int from,
+   TraceState traceState,
+   QueryOrder queryOrder)
+throws IOException {
+
+String orderBy = SegmentRecord.START_TIME;
+if (queryOrder == QueryOrder.BY_DURATION) {
+orderBy = SegmentRecord.LATENCY;
+}
+
+WhereQueryImpl recallQuery = select()
+.function("top", orderBy, limit + from)
+.column(SegmentRecord.SEGMENT_ID)
+.column(SegmentRecord.START_TIME)
+.column(SegmentRecord.ENDPOINT_NAME)
+.column(SegmentRecord.LATENCY)
+.column(SegmentRecord.IS_ERROR)
+.column(SegmentRecord.TRACE_ID)
+.from(client.getDatabase(), SegmentRecord.INDEX_NAME)
+.where();
+
+if (startSecondTB != 0 && endSecondTB != 0) {
+recallQuery.and(gte(SegmentRecord.TIME_BUCKET, startSecondTB))
+   .and(lte(SegmentRecord.TIME_BUCKET, endSecondTB));
+}
+if (minDuration != 0) {
+recallQuery.and(gte(SegmentRecord.LATENCY, minDuration));
+}
+if (maxDuration != 0) {
+recallQuery.and(lte(SegmentRecord.LATENCY, maxDuration));
+}
+if (!Strings.isNullOrEmpty(endpointName)) {
+recallQuery.and(regex(SegmentRecord.

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379832246
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/ProfileThreadSnapshotQuery.java
 ##
 @@ -0,0 +1,165 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.base.Joiner;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.LinkedList;
+import java.util.List;
+import org.apache.skywalking.apm.util.StringUtil;
+import 
org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import 
org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+import org.apache.skywalking.oap.server.core.query.entity.BasicTrace;
+import 
org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.WhereQueryImpl;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.contains;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.eq;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.gte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.lte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+public class ProfileThreadSnapshotQuery implements 
IProfileThreadSnapshotQueryDAO {
+private final InfluxClient client;
+
+public ProfileThreadSnapshotQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public List queryProfiledSegments(String taskId) throws 
IOException {
+WhereQueryImpl query = select(ProfileThreadSnapshotRecord.SEGMENT_ID)
+.from(client.getDatabase(), ProfileThreadSnapshotRecord.INDEX_NAME)
+.where()
+.and(eq(ProfileThreadSnapshotRecord.TASK_ID, taskId))
+.and(eq(ProfileThreadSnapshotRecord.SEQUENCE, 0));
+
+final LinkedList segments = new LinkedList<>();
+QueryResult.Series series = client.queryForSingleSeries(query);
+if (series == null) {
+return Collections.emptyList();
+}
+series.getValues().forEach(values -> {
+segments.add((String) values.get(1));
+});
+
+if (segments.isEmpty()) {
+return Collections.emptyList();
+}
+
+query = select()
+.function("bottom", SegmentRecord.START_TIME, segments.size())
+.column(SegmentRecord.SEGMENT_ID)
+.column(SegmentRecord.START_TIME)
+.column(SegmentRecord.ENDPOINT_NAME)
+.column(SegmentRecord.LATENCY)
+.column(SegmentRecord.IS_ERROR)
+.column(SegmentRecord.TRACE_ID)
+.from(client.getDatabase(), SegmentRecord.INDEX_NAME)
+.where()
+.and(contains(SegmentRecord.SEGMENT_ID, 
Joiner.on("|").join(segments)));
+
+ArrayList result = 
Lists.newArrayListWithCapacity(segments.size());
+client.queryForSingleSeries(query)
+  .getValues()
+  .stream()
+  .sorted((a, b) -> Long.compare(((Number) b.get(1)).longValue(), 
((Number) a.get(1)).longValue()))
+  .forEach(values -> {
+  BasicTrace basicTrace = new BasicTrace();
+
+  basicTrace.setSegmentId((String) values.get(2));
+  basicTrace.setStart(String.valueOf(values.get(3)));
+  basicTrace.getEndpointNames().add((String) values.get(4));
+  basicTrace.setDuration((int) values.get(5));
+  basicTrace.setError(BooleanUtils.valueToBoolean((int) 
values.get(6)));
+  String traceIds = (String) values.get(7);
+  basicTrace.

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379832871
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/TraceQuery.java
 ##
 @@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.List;
+import lombok.extern.slf4j.Slf4j;
+import 
org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.query.entity.BasicTrace;
+import org.apache.skywalking.oap.server.core.query.entity.QueryOrder;
+import org.apache.skywalking.oap.server.core.query.entity.Span;
+import org.apache.skywalking.oap.server.core.query.entity.TraceBrief;
+import org.apache.skywalking.oap.server.core.query.entity.TraceState;
+import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.base.RecordDAO;
+import org.elasticsearch.common.Strings;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.SelectQueryImpl;
+import org.influxdb.querybuilder.WhereQueryImpl;
+import org.influxdb.querybuilder.clauses.Clause;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.eq;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.gte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.lte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.regex;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+@Slf4j
+public class TraceQuery implements ITraceQueryDAO {
+private final InfluxClient client;
+
+public TraceQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public TraceBrief queryBasicTraces(long startSecondTB,
+   long endSecondTB,
+   long minDuration,
+   long maxDuration,
+   String endpointName,
+   int serviceId,
+   int serviceInstanceId,
+   int endpointId,
+   String traceId,
+   int limit,
+   int from,
+   TraceState traceState,
+   QueryOrder queryOrder)
+throws IOException {
+
+String orderBy = SegmentRecord.START_TIME;
+if (queryOrder == QueryOrder.BY_DURATION) {
+orderBy = SegmentRecord.LATENCY;
+}
+
+WhereQueryImpl recallQuery = select()
+.function("top", orderBy, limit + from)
+.column(SegmentRecord.SEGMENT_ID)
+.column(SegmentRecord.START_TIME)
+.column(SegmentRecord.ENDPOINT_NAME)
+.column(SegmentRecord.LATENCY)
+.column(SegmentRecord.IS_ERROR)
+.column(SegmentRecord.TRACE_ID)
+.from(client.getDatabase(), SegmentRecord.INDEX_NAME)
+.where();
+
+if (startSecondTB != 0 && endSecondTB != 0) {
+recallQuery.and(gte(SegmentRecord.TIME_BUCKET, startSecondTB))
+   .and(lte(SegmentRecord.TIME_BUCKET, endSecondTB));
+}
+if (minDuration != 0) {
+recallQuery.and(gte(SegmentRecord.LATENCY, minDuration));
+}
+if (maxDuration != 0) {
+recallQuery.and(lte(SegmentRecord.LATENCY, maxDuration));
+}
+if (!Strings.isNullOrEmpty(endpointName)) {
+recallQuery.and(regex(SegmentRecord.

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379833534
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/LogQuery.java
 ##
 @@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.collect.Maps;
+import java.io.IOException;
+import java.util.List;
+import java.util.Map;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.core.Const;
+import 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord;
+import org.apache.skywalking.oap.server.core.query.entity.ContentType;
+import org.apache.skywalking.oap.server.core.query.entity.Log;
+import org.apache.skywalking.oap.server.core.query.entity.LogState;
+import org.apache.skywalking.oap.server.core.query.entity.Logs;
+import org.apache.skywalking.oap.server.core.query.entity.Pagination;
+import org.apache.skywalking.oap.server.core.storage.query.ILogQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.base.RecordDAO;
+import org.elasticsearch.common.Strings;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.SelectQueryImpl;
+import org.influxdb.querybuilder.WhereQueryImpl;
+import org.influxdb.querybuilder.clauses.ConjunctionClause;
+
+import static 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord.CONTENT;
+import static 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord.CONTENT_TYPE;
+import static 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord.ENDPOINT_ID;
+import static 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord.IS_ERROR;
+import static 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord.SERVICE_ID;
+import static 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord.SERVICE_INSTANCE_ID;
+import static 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord.STATUS_CODE;
+import static 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord.TIMESTAMP;
+import static 
org.apache.skywalking.oap.server.core.analysis.manual.log.AbstractLogRecord.TRACE_ID;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.eq;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.gte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.lte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+@Slf4j
+public class LogQuery implements ILogQueryDAO {
+private final InfluxClient client;
+
+public LogQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public Logs queryLogs(String metricName, int serviceId, int 
serviceInstanceId, int endpointId, String traceId,
+  LogState state, String stateCode, Pagination paging, 
int from, int limit,
+  long startTB, long endTB) throws IOException {
+WhereQueryImpl recallQuery = 
select().regex("*::field")
+.from(client.getDatabase(), metricName)
+.where();
+if (serviceId != Const.NONE) {
+recallQuery.and(eq(RecordDAO.TAG_SERVICE_ID, 
String.valueOf(serviceId)));
+}
+if (serviceInstanceId != Const.NONE) {
+recallQuery.and(eq(SERVICE_INSTANCE_ID, serviceInstanceId));
+}
+if (endpointId != Const.NONE) {
+recallQuery.and(eq(ENDPOINT_ID, endpointId));
+}
+if (!Strings.isNullOrEmpty(traceId)) {
+recallQuery.and(eq(TRACE_ID, traceId));
+}
+switch (state) {
+case ERROR: {
+recallQuery.and(eq(IS_ERROR, true));
+break;
+}
+case SUCCESS: {
+recallQuery.and(eq(IS_ERROR, fals

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379832246
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/ProfileThreadSnapshotQuery.java
 ##
 @@ -0,0 +1,165 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.base.Joiner;
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.LinkedList;
+import java.util.List;
+import org.apache.skywalking.apm.util.StringUtil;
+import 
org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import 
org.apache.skywalking.oap.server.core.profile.ProfileThreadSnapshotRecord;
+import org.apache.skywalking.oap.server.core.query.entity.BasicTrace;
+import 
org.apache.skywalking.oap.server.core.storage.profile.IProfileThreadSnapshotQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.WhereQueryImpl;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.contains;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.eq;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.gte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.lte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+public class ProfileThreadSnapshotQuery implements 
IProfileThreadSnapshotQueryDAO {
+private final InfluxClient client;
+
+public ProfileThreadSnapshotQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public List queryProfiledSegments(String taskId) throws 
IOException {
+WhereQueryImpl query = select(ProfileThreadSnapshotRecord.SEGMENT_ID)
+.from(client.getDatabase(), ProfileThreadSnapshotRecord.INDEX_NAME)
+.where()
+.and(eq(ProfileThreadSnapshotRecord.TASK_ID, taskId))
+.and(eq(ProfileThreadSnapshotRecord.SEQUENCE, 0));
+
+final LinkedList segments = new LinkedList<>();
+QueryResult.Series series = client.queryForSingleSeries(query);
+if (series == null) {
+return Collections.emptyList();
+}
+series.getValues().forEach(values -> {
+segments.add((String) values.get(1));
+});
+
+if (segments.isEmpty()) {
+return Collections.emptyList();
+}
+
+query = select()
+.function("bottom", SegmentRecord.START_TIME, segments.size())
+.column(SegmentRecord.SEGMENT_ID)
+.column(SegmentRecord.START_TIME)
+.column(SegmentRecord.ENDPOINT_NAME)
+.column(SegmentRecord.LATENCY)
+.column(SegmentRecord.IS_ERROR)
+.column(SegmentRecord.TRACE_ID)
+.from(client.getDatabase(), SegmentRecord.INDEX_NAME)
+.where()
+.and(contains(SegmentRecord.SEGMENT_ID, 
Joiner.on("|").join(segments)));
+
+ArrayList result = 
Lists.newArrayListWithCapacity(segments.size());
+client.queryForSingleSeries(query)
+  .getValues()
+  .stream()
+  .sorted((a, b) -> Long.compare(((Number) b.get(1)).longValue(), 
((Number) a.get(1)).longValue()))
+  .forEach(values -> {
+  BasicTrace basicTrace = new BasicTrace();
+
+  basicTrace.setSegmentId((String) values.get(2));
+  basicTrace.setStart(String.valueOf(values.get(3)));
+  basicTrace.getEndpointNames().add((String) values.get(4));
+  basicTrace.setDuration((int) values.get(5));
+  basicTrace.setError(BooleanUtils.valueToBoolean((int) 
values.get(6)));
+  String traceIds = (String) values.get(7);
+  basicTrace.

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379833111
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/base/MetricsDAO.java
 ##
 @@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.base;
+
+import com.google.common.base.Joiner;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
+import org.apache.skywalking.oap.server.core.storage.StorageBuilder;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.model.ModelColumn;
+import org.apache.skywalking.oap.server.core.storage.type.StorageDataType;
+import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.SelectQueryImpl;
+import org.influxdb.querybuilder.WhereQueryImpl;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.contains;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+public class MetricsDAO implements IMetricsDAO {
+public static final String TAG_ENTITY_ID = "_entity_id";
+
+private final StorageBuilder storageBuilder;
+private final InfluxClient client;
+
+public MetricsDAO(InfluxClient client, StorageBuilder 
storageBuilder) {
+this.client = client;
+this.storageBuilder = storageBuilder;
+}
+
+@Override
+public List multiGet(Model model, List ids) throws 
IOException {
+WhereQueryImpl query = select()
+.regex("*::field")
 
 Review comment:
   did not find related documentation, but will `.raw("*::field")` be more 
efficient?
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379828752
 
 

 ##
 File path: 
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/TimeBucket.java
 ##
 @@ -32,10 +32,119 @@ public static long getRecordTimeBucket(long time) {
 return getTimeBucket(time, Downsampling.Second);
 }
 
+/**
+ * Record time bucket format in Minute Unit.
+ *
+ * @param time Timestamp
+ * @return time in minute format.
+ */
 public static long getMinuteTimeBucket(long time) {
 return getTimeBucket(time, Downsampling.Minute);
 }
 
+/**
+ * Convert TimeBucket to Timestamp in millisecond.
+ *
+ * @param timeBucket long
+ * @return timestamp in millisecond unit
+ */
+public static long getTimestamp(long timeBucket) {
+if (isSecondBucket(timeBucket)) {
+return getTimestamp(timeBucket, Downsampling.Second);
+} else if (isMinuteBucket(timeBucket)) {
+return getTimestamp(timeBucket, Downsampling.Minute);
+} else if (isHourBucket(timeBucket)) {
+return getTimestamp(timeBucket, Downsampling.Hour);
+} else if (isDayBucket(timeBucket)) {
+return getTimestamp(timeBucket, Downsampling.Day);
+} else if (isMonthBucket(timeBucket)) {
+return getTimestamp(timeBucket, Downsampling.Month);
+} else {
+throw new UnexpectedException("Unknown downsampling value.");
+}
+}
+
+/**
+ * The format of timeBucket in minute Unit is "MMddHHmmss", so which 
means the TimeBucket mush between
 
 Review comment:
   `mush` -> `must be`


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379828221
 
 

 ##
 File path: docker/oap-es7/docker-entrypoint.sh
 ##
 @@ -149,6 +149,37 @@ storage:
 EOT
 }
 
+generateStorageInfluxDB() {
+cat <> ${var_application_file}
+storage:
+  influx:
+# Metadata storage provider configuration
+metabaseType: ${SW_STORAGE_METABASE_TYPE:H2} # There are 2 options as 
Metabase provider, H2 or MySQL.
+h2Props:
+  dataSourceClassName: 
${SW_STORAGE_METABASE_DRIVER:org.h2.jdbcx.JdbcDataSource}
+  dataSource.url: ${SW_STORAGE_METABASE_URL:jdbc:h2:mem:skywalking-oap-db}
+  dataSource.user: ${SW_STORAGE_METABASE_USER:sa}
+  dataSource.password: ${SW_STORAGE_METABASE_PASSWORD:}
+mysqlProps:
+  jdbcUrl: ${SW_STORAGE_METABASE_URL:"jdbc:mysql://localhost:3306/swtest"}
+  dataSource.user: ${SW_STORAGE_METABASE_USER:root}
+  dataSource.password: ${SW_STORAGE_METABASE_PASSWORD:root@1234}
+  dataSource.cachePrepStmts: ${SW_STORAGE_METABASE_CACHE_PREP_STMTS:true}
+  dataSource.prepStmtCacheSize: 
${SW_STORAGE_METABASE_PREP_STMT_CACHE_SQL_SIZE:250}
+  dataSource.prepStmtCacheSqlLimit: 
${SW_STORAGE_METABASE_PREP_STMT_CACHE_SQL_LIMIT:2048}
+  dataSource.useServerPrepStmts: 
${SW_STORAGE_METABASE_USE_SERVER_PREP_STMTS:true}
+metadataQueryMaxSize: ${SW_STORAGE_METABASE_QUERY_MAX_SIZE:5000}
+# InfluxDB configuration
+url: ${SW_STORAGE_INFLUXDB_URL:http://localhost:8086}
+user: ${SW_STORAGE_INFLUXDB_USER:root}
+password: ${SW_STORAGE_INFLUXDB_PASSWORD:}
+database: ${SW_STORAGE_INFLUXDB_DATABASE:skywalking}
+actions: ${SW_STORAGE_INFLUXDB_ACTIONS:1000} # the number of actions to 
collect
+duration: ${SW_STORAGE_INFLUXDB_DURATION:1000} # the time to wait at most 
(milliseconds)
+fetchTaskLogMaxSize: ${SW_STORAGE_INFLUXDB_FETCH_TASK_LOG_MAX_SIZE:5000} # 
the max number of fetch task log in a request
 
 Review comment:
   Should escape the `$` character


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379832725
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/TraceQuery.java
 ##
 @@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.collect.Lists;
+import java.io.IOException;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.List;
+import lombok.extern.slf4j.Slf4j;
+import 
org.apache.skywalking.oap.server.core.analysis.manual.segment.SegmentRecord;
+import org.apache.skywalking.oap.server.core.query.entity.BasicTrace;
+import org.apache.skywalking.oap.server.core.query.entity.QueryOrder;
+import org.apache.skywalking.oap.server.core.query.entity.Span;
+import org.apache.skywalking.oap.server.core.query.entity.TraceBrief;
+import org.apache.skywalking.oap.server.core.query.entity.TraceState;
+import org.apache.skywalking.oap.server.core.storage.query.ITraceQueryDAO;
+import org.apache.skywalking.oap.server.library.util.BooleanUtils;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.base.RecordDAO;
+import org.elasticsearch.common.Strings;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.SelectQueryImpl;
+import org.influxdb.querybuilder.WhereQueryImpl;
+import org.influxdb.querybuilder.clauses.Clause;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.eq;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.gte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.lte;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.regex;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+@Slf4j
+public class TraceQuery implements ITraceQueryDAO {
+private final InfluxClient client;
+
+public TraceQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public TraceBrief queryBasicTraces(long startSecondTB,
+   long endSecondTB,
+   long minDuration,
+   long maxDuration,
+   String endpointName,
+   int serviceId,
+   int serviceInstanceId,
+   int endpointId,
+   String traceId,
+   int limit,
+   int from,
+   TraceState traceState,
+   QueryOrder queryOrder)
+throws IOException {
+
+String orderBy = SegmentRecord.START_TIME;
+if (queryOrder == QueryOrder.BY_DURATION) {
+orderBy = SegmentRecord.LATENCY;
+}
+
+WhereQueryImpl recallQuery = select()
+.function("top", orderBy, limit + from)
+.column(SegmentRecord.SEGMENT_ID)
+.column(SegmentRecord.START_TIME)
+.column(SegmentRecord.ENDPOINT_NAME)
+.column(SegmentRecord.LATENCY)
+.column(SegmentRecord.IS_ERROR)
+.column(SegmentRecord.TRACE_ID)
+.from(client.getDatabase(), SegmentRecord.INDEX_NAME)
+.where();
+
+if (startSecondTB != 0 && endSecondTB != 0) {
+recallQuery.and(gte(SegmentRecord.TIME_BUCKET, startSecondTB))
+   .and(lte(SegmentRecord.TIME_BUCKET, endSecondTB));
+}
+if (minDuration != 0) {
+recallQuery.and(gte(SegmentRecord.LATENCY, minDuration));
+}
+if (maxDuration != 0) {
+recallQuery.and(lte(SegmentRecord.LATENCY, maxDuration));
+}
+if (!Strings.isNullOrEmpty(endpointName)) {
+recallQuery.and(regex(SegmentRecord.

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-15 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379828284
 
 

 ##
 File path: docker/oap/docker-entrypoint.sh
 ##
 @@ -150,6 +150,37 @@ storage:
 EOT
 }
 
+generateStorageInfluxDB() {
+cat <> ${var_application_file}
+storage:
+  influx:
+# Metadata storage provider configuration
+metabaseType: ${SW_STORAGE_METABASE_TYPE:H2} # There are 2 options as 
Metabase provider, H2 or MySQL.
+h2Props:
+  dataSourceClassName: 
${SW_STORAGE_METABASE_DRIVER:org.h2.jdbcx.JdbcDataSource}
+  dataSource.url: ${SW_STORAGE_METABASE_URL:jdbc:h2:mem:skywalking-oap-db}
+  dataSource.user: ${SW_STORAGE_METABASE_USER:sa}
+  dataSource.password: ${SW_STORAGE_METABASE_PASSWORD:}
+mysqlProps:
+  jdbcUrl: ${SW_STORAGE_METABASE_URL:"jdbc:mysql://localhost:3306/swtest"}
+  dataSource.user: ${SW_STORAGE_METABASE_USER:root}
+  dataSource.password: ${SW_STORAGE_METABASE_PASSWORD:root@1234}
+  dataSource.cachePrepStmts: ${SW_STORAGE_METABASE_CACHE_PREP_STMTS:true}
+  dataSource.prepStmtCacheSize: 
${SW_STORAGE_METABASE_PREP_STMT_CACHE_SQL_SIZE:250}
+  dataSource.prepStmtCacheSqlLimit: 
${SW_STORAGE_METABASE_PREP_STMT_CACHE_SQL_LIMIT:2048}
+  dataSource.useServerPrepStmts: 
${SW_STORAGE_METABASE_USE_SERVER_PREP_STMTS:true}
+metadataQueryMaxSize: ${SW_STORAGE_METABASE_QUERY_MAX_SIZE:5000}
+# InfluxDB configuration
+url: ${SW_STORAGE_INFLUXDB_URL:http://localhost:8086}
+user: ${SW_STORAGE_INFLUXDB_USER:root}
+password: ${SW_STORAGE_INFLUXDB_PASSWORD:}
+database: ${SW_STORAGE_INFLUXDB_DATABASE:skywalking}
+actions: ${SW_STORAGE_INFLUXDB_ACTIONS:1000} # the number of actions to 
collect
+duration: ${SW_STORAGE_INFLUXDB_DURATION:1000} # the time to wait at most 
(milliseconds)
+fetchTaskLogMaxSize: ${SW_STORAGE_INFLUXDB_FETCH_TASK_LOG_MAX_SIZE:5000} # 
the max number of fetch task log in a request
 
 Review comment:
   Should escape the `$` character
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-13 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r379220907
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/InfluxClient.java
 ##
 @@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb;
+
+import java.io.IOException;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import lombok.extern.slf4j.Slf4j;
+import okhttp3.OkHttpClient;
+import org.apache.skywalking.oap.server.core.analysis.Downsampling;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.library.client.Client;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.BatchPoints;
+import org.influxdb.dto.Point;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.time.TimeInterval;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.ti;
+
+/**
+ * InfluxDB connection maintainer, provides base data write/query API.
+ */
+@Slf4j
+public class InfluxClient implements Client {
+private InfluxStorageConfig config;
+private InfluxDB influx;
+
+/**
+ * A constant, the name of time field in Time-series database.
+ */
+public static final String TIME = "time";
+/**
+ * A constant, the name of tag.
+ */
+public static final String TAG_ENTITY_ID = "entity_id";
+
+private final String database;
+
+public InfluxClient(InfluxStorageConfig config) {
+this.config = config;
+this.database = config.getDatabase();
+}
+
+public final String getDatabase() {
+return database;
+}
+
+@Override
+public void connect() {
+influx = InfluxDBFactory.connect(config.getUrl(), config.getUser(), 
config.getPassword(),
+ new 
OkHttpClient.Builder().readTimeout(3, TimeUnit.MINUTES)
+   
.writeTimeout(3, TimeUnit.MINUTES),
+ InfluxDB.ResponseFormat.MSGPACK
+);
+influx.query(new Query("CREATE DATABASE " + database));
+
+influx.enableBatch(config.getActions(), config.getDuration(), 
TimeUnit.MILLISECONDS);
+influx.setDatabase(database);
+}
+
+/**
+ * To get a connection of InfluxDB.
+ *
+ * @return InfluxDB's connection
+ */
+private InfluxDB getInflux() {
+return influx;
+}
+
+/**
+ * Execute a query against InfluxDB and return a set of {@link 
QueryResult.Result}s. Normally, InfluxDB supports
+ * combining multiple statements into one query, so that we do get 
multi-results.
+ *
+ * @param query Query
+ * @return a set of {@link QueryResult.Result}s.
+ * @throws IOException if there is an error on the InfluxDB server or 
communication error.
+ */
+public List query(Query query) throws IOException {
+if (log.isDebugEnabled()) {
+log.debug("SQL Statement: {}", query.getCommand());
+}
+
+try {
+QueryResult result = getInflux().query(query);
+if (result.hasError()) {
+throw new IOException(result.getError());
+}
+return result.getResults();
+} catch (Exception e) {
+throw new IOException(e.getMessage() + System.lineSeparator() + 
"SQL Statement: " + query.getCommand(), e);
+}
+}
+
+/**
+ * Execute a query against InfluxDB with a single statement.
+ *
+ * @param query Query
+ * @return a set of {@link QueryResult.Series}s
+ * @throws IOException if there is an error on the InfluxDB server or 
communication error
+ */
+public List queryForSeries(Query query) throws 
IOException {
+List results = query(query);
+
+if (CollectionUtils.isEmpty(results)) {
+r

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-11 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r378009081
 
 

 ##
 File path: .github/workflows/e2e.yaml
 ##
 @@ -39,19 +39,21 @@ jobs:
   ./mvnw --batch-mode -Dcheckstyle.skip -Drat.skip -T2 
-Dmaven.compile.fork -Dmaven.compiler.maxmem=3072 -DskipTests clean install
   ./mvnw --batch-mode -f test/e2e/pom.xml -pl e2e-base clean install
   - name: Single Node Tests(JDK8)
-run: export E2E_VERSION=jdk8-1.3 && bash -x test/e2e/run.sh 
e2e-single-service
+run: export E2E_VERSION=jdk8-1.4 && bash -x test/e2e/run.sh 
e2e-single-service
+  - name: Single Node Tests(InfluxDB/JDK8)
+run: export E2E_VERSION=jdk8-1.4 && bash -x test/e2e/run.sh 
e2e-influxdb
   - name: Single Node Tests(MySQL/JDK8)
-run: export E2E_VERSION=jdk8-1.3 && bash -x test/e2e/run.sh e2e-mysql
+run: export E2E_VERSION=jdk8-1.4 && bash -x test/e2e/run.sh e2e-mysql
   - name: Single Node Tests(JDK9)
-run: export E2E_VERSION=jdk9-1.3 && bash -x test/e2e/run.sh 
e2e-single-service
+run: export E2E_VERSION=jdk9-1.4 && bash -x test/e2e/run.sh 
e2e-single-service
   - name: Single Node Tests(JDK11)
-run: export E2E_VERSION=jdk11-1.3 && bash -x test/e2e/run.sh 
e2e-single-service
+run: export E2E_VERSION=jdk11-1.4 && bash -x test/e2e/run.sh 
e2e-single-service
   - name: Single Node Tests(JDK12)
-run: export E2E_VERSION=jdk12-1.3 && bash -x test/e2e/run.sh 
e2e-single-service
+run: export E2E_VERSION=jdk12-1.4 && bash -x test/e2e/run.sh 
e2e-single-service
   - name: Agent Reboot Tests(JDK8)
-run: export E2E_VERSION=jdk8-1.3 && bash -x test/e2e/run.sh 
e2e-agent-reboot
+run: export E2E_VERSION=jdk8-1.4 && bash -x test/e2e/run.sh 
e2e-agent-reboot
 
-  Cluster:
+  Cluster_N_TTL:
 
 Review comment:
   > What do you mean N_TTL?
   
   Possibly `N` == `And`, we don't need to save 2 letters to cause confusion, 
no?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-04 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r374619657
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/InfluxClient.java
 ##
 @@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.List;
+import okhttp3.OkHttpClient;
+import org.apache.skywalking.oap.server.core.analysis.Downsampling;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.library.client.Client;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.BatchPoints;
+import org.influxdb.dto.Point;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.time.TimeInterval;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.ti;
+
+public class InfluxClient implements Client {
+private static final Logger logger = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+private InfluxStorageConfig config;
+private InfluxDB influx;
+
+public static final String TIME = "time";
+public static final String TAG_ENTITY_ID = "entity_id";
+private final String database;
+
+public InfluxClient(InfluxStorageConfig config) {
+this.config = config;
+this.database = config.getDatabase();
+}
+
+public final String getDatabase() {
+return database;
+}
+
+@Override
+public void connect() {
+influx = InfluxDBFactory.connect(config.getUrl(), config.getUser(), 
config.getPassword(),
+new OkHttpClient.Builder(), InfluxDB.ResponseFormat.MSGPACK);
+influx.query(new Query("CREATE DATABASE " + database));
+
+influx.setDatabase(database);
+influx.enableBatch();
+}
+
+public InfluxDB getInflux() {
+return influx;
+}
+
+public List query(Query query) throws IOException {
+if (logger.isDebugEnabled()) {
+logger.debug("SQL Statement: {}", query.getCommand());
+}
+
+try {
+QueryResult result = getInflux().query(query);
+if (result.hasError()) {
+throw new IOException(result.getError());
+}
+return result.getResults();
+} catch (Exception e) {
+throw new IOException(e.getMessage() + "\r\nSQL Statement: " + 
query.getCommand(), e);
 
 Review comment:
   use `System.lineSeparator()` instead of hardcoded `\r\n`


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-04 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r374611069
 
 

 ##
 File path: test/e2e/e2e-influxdb/src/docker/rc.d/rc0-prepare.sh
 ##
 @@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+# Licensed to the SkyAPM under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+echo "InfluxDB with H2 database is storage provider..."
+
+# Modify application.yml to set MySQL as storage provider.
 
 Review comment:
   ```suggestion
   # Modify application.yml to set InfluxDB as storage provider.
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-04 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r374598967
 
 

 ##
 File path: oap-server/server-storage-plugin/storage-influxdb-plugin/pom.xml
 ##
 @@ -0,0 +1,50 @@
+
+
+
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
+server-storage-plugin
+org.apache.skywalking
+7.0.0-SNAPSHOT
+
+4.0.0
+
+storage-influxdb-plugin
+jar
+
+
+
+org.apache.skywalking
+storage-elasticsearch-plugin
+${project.version}
+
 
 Review comment:
   why does influx plugin depend on ElasticSearch plugin


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-04 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r374623410
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/base/MetricsDAO.java
 ##
 @@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.base;
+
+import com.google.common.base.Joiner;
+import com.google.common.collect.Lists;
+import com.google.common.collect.Maps;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+import org.apache.skywalking.oap.server.core.storage.IMetricsDAO;
+import org.apache.skywalking.oap.server.core.storage.StorageBuilder;
+import org.apache.skywalking.oap.server.core.storage.model.Model;
+import org.apache.skywalking.oap.server.core.storage.model.ModelColumn;
+import org.apache.skywalking.oap.server.core.storage.type.StorageDataType;
+import org.apache.skywalking.oap.server.library.client.request.InsertRequest;
+import org.apache.skywalking.oap.server.library.client.request.UpdateRequest;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.WhereQueryImpl;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.contains;
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.select;
+
+public class MetricsDAO implements IMetricsDAO {
+private final StorageBuilder storageBuilder;
+private final InfluxClient client;
+
+public MetricsDAO(InfluxClient client, StorageBuilder 
storageBuilder) {
+this.client = client;
+this.storageBuilder = storageBuilder;
+}
+
+@Override
+public List multiGet(Model model, List ids) throws 
IOException {
+WhereQueryImpl query = select("*::field")
 
 Review comment:
   Raw use of parameterized class `WhereQueryImpl`


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-04 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r374601890
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/query/AggregationQuery.java
 ##
 @@ -0,0 +1,138 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb.query;
+
+import com.google.common.collect.Lists;
+import org.apache.skywalking.oap.server.core.analysis.Downsampling;
+import org.apache.skywalking.oap.server.core.analysis.metrics.Metrics;
+import org.apache.skywalking.oap.server.core.query.entity.Order;
+import org.apache.skywalking.oap.server.core.query.entity.TopNEntity;
+import org.apache.skywalking.oap.server.core.register.EndpointInventory;
+import org.apache.skywalking.oap.server.core.register.ServiceInstanceInventory;
+import org.apache.skywalking.oap.server.core.storage.model.ModelName;
+import 
org.apache.skywalking.oap.server.core.storage.query.IAggregationQueryDAO;
+import org.apache.skywalking.oap.server.storage.plugin.influxdb.InfluxClient;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.SelectQueryImpl;
+import org.influxdb.querybuilder.SelectSubQueryImpl;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.List;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.*;
+
+public class AggregationQuery implements IAggregationQueryDAO {
+private static final Logger LOG = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
+private final InfluxClient client;
+
+public AggregationQuery(InfluxClient client) {
+this.client = client;
+}
+
+@Override
+public List getServiceTopN(String indName, String valueCName, 
int topN, Downsampling downsampling,
+   long startTB, long endTB, Order 
order) throws IOException {
+return getTopNEntity(downsampling, indName, subQuery(indName, 
valueCName, startTB, endTB), order, topN);
+}
+
+@Override
+public List getAllServiceInstanceTopN(String indName, String 
valueCName, int topN, Downsampling downsampling,
+  long startTB, long 
endTB, Order order) throws IOException {
+return getTopNEntity(downsampling, indName, subQuery(indName, 
valueCName, startTB, endTB), order, topN);
+}
+
+@Override
+public List getServiceInstanceTopN(int serviceId, String 
indName, String valueCName, int topN, Downsampling downsampling,
+   long startTB, long endTB, 
Order order) throws IOException {
+return getTopNEntity(downsampling, indName, 
subQuery(ServiceInstanceInventory.SERVICE_ID, serviceId, indName, valueCName, 
startTB, endTB), order, topN);
+}
+
+@Override
+public List getAllEndpointTopN(String indName, String 
valueCName, int topN, Downsampling downsampling,
+   long startTB, long endTB, Order 
order) throws IOException {
+return getTopNEntity(downsampling, indName, subQuery(indName, 
valueCName, startTB, endTB), order, topN);
+}
+
+@Override
+public List getEndpointTopN(int serviceId, String indName, 
String valueCName, int topN, Downsampling downsampling,
+long startTB, long endTB, Order 
order) throws IOException {
+return getTopNEntity(downsampling, indName, 
subQuery(EndpointInventory.SERVICE_ID, serviceId, indName, valueCName, startTB, 
endTB), order, topN);
+}
+
+private final List getTopNEntity(Downsampling downsampling, 
String name, SelectSubQueryImpl subQuery, Order order, int 
topN) throws IOException {
 
 Review comment:
   As private methods cannot be meaningfully overridden, declaring them final 
is redundant.


This is an automated messag

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-04 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r374619003
 
 

 ##
 File path: oap-server/server-storage-plugin/storage-influxdb-plugin/pom.xml
 ##
 @@ -0,0 +1,50 @@
+
+
+
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+
+server-storage-plugin
+org.apache.skywalking
+7.0.0-SNAPSHOT
+
+4.0.0
+
+storage-influxdb-plugin
+jar
+
+
+
+org.apache.skywalking
+storage-elasticsearch-plugin
+${project.version}
+
+
+
+org.apache.skywalking
+storage-jdbc-hikaricp-plugin
+${project.version}
+
+
+
+org.influxdb
+influxdb-java
+2.15
 
 Review comment:
   make it a property


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-04 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r374611296
 
 

 ##
 File path: 
test/e2e/e2e-influxdb/src/main/java/org/apache/skywalking/e2e/sample/client/SampleClientApplication.java
 ##
 @@ -0,0 +1,34 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.e2e.sample.client;
+
+import org.springframework.boot.SpringApplication;
+import org.springframework.boot.autoconfigure.SpringBootApplication;
+import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
+
+/**
+ * @author kezhenxu94
+ */
 
 Review comment:
   remove the `@author`s please


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-04 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r374597179
 
 

 ##
 File path: 
oap-server/server-storage-plugin/storage-influxdb-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/influxdb/InfluxClient.java
 ##
 @@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.storage.plugin.influxdb;
+
+import java.io.IOException;
+import java.lang.invoke.MethodHandles;
+import java.util.List;
+import okhttp3.OkHttpClient;
+import org.apache.skywalking.oap.server.core.analysis.Downsampling;
+import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
+import org.apache.skywalking.oap.server.library.client.Client;
+import org.influxdb.InfluxDB;
+import org.influxdb.InfluxDBFactory;
+import org.influxdb.dto.BatchPoints;
+import org.influxdb.dto.Point;
+import org.influxdb.dto.Query;
+import org.influxdb.dto.QueryResult;
+import org.influxdb.querybuilder.time.TimeInterval;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static org.influxdb.querybuilder.BuiltQuery.QueryBuilder.ti;
+
+public class InfluxClient implements Client {
+private static final Logger logger = 
LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
 
 Review comment:
   I don't really recommend to use this kind of codes to get the current class 
just for logging, `java.lang.invoke.MethodHandles#lookup` creates new object 
for every call, which I don't think is necessary, and if you are trying to 
avoid "copy-and-paste" mistakes, why not just use the `@Slf4j` annotation of 
Lombok, WDYT @wu-sheng 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

2020-02-04 Thread GitBox

kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a 
new storage plugin
URL: https://github.com/apache/skywalking/pull/4239#discussion_r374612571
 
 

 ##
 File path: 
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/storage/model/Model.java
 ##
 @@ -44,5 +48,11 @@ public Model(String name, List columns, 
boolean capableOfTimeSeries
 this.scopeId = scopeId;
 this.name = ModelName.build(downsampling, name);
 this.record = record;
+this.storageColumns = Maps.newTreeMap();
+columns.forEach(column -> { 
storageColumns.put(column.getColumnName().getStorageName(), column); });
+}
+
+public ModelColumn getColumnByStorageCName(String storageCName) {
 
 Review comment:
   why add unused method 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

[GitHub] [skywalking] kezhenxu94 commented on a change in pull request #4239: Provide influxdb as a new storage plugin

30 matches

Site Navigation

Mail list logo

Footer information