[GitHub] keith-turner commented on a change in pull request #5: Fixes #3 Check if application is running before starting
keith-turner commented on a change in pull request #5: Fixes #3 Check if application is running before starting URL: https://github.com/apache/fluo-yarn/pull/5#discussion_r133576196 ## File path: README.md ## @@ -82,21 +82,25 @@ After you obtain a Fluo YARN distribution tarball, follow these steps to install ## Start Fluo application in YARN -Follow the instructions below to start a Fluo application which contain an oracle and multiple workers. +Before you can launch a Fluo application in YARN, you should follow Fluo's [install] and [application] instructions Review comment: Why repeat this info? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #913: Fluo 912
keith-turner commented on a change in pull request #913: Fluo 912
URL: https://github.com/apache/fluo/pull/913#discussion_r133574855
##
File path:
modules/command/src/main/java/org/apache/fluo/command/FluoStatus.java
##
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
contributor license
+ * agreements. See the NOTICE file distributed with this work for additional
information regarding
+ * copyright ownership. The ASF licenses this file to you under the Apache
License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express
+ * or implied. See the License for the specific language governing permissions
and limitations under
+ * the License.
+ */
+
+package org.apache.fluo.command;
+
+import java.io.File;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+
+import com.google.common.base.Preconditions;
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.fluo.api.config.FluoConfiguration;
+import org.apache.fluo.core.client.FluoAdminImpl;
+import org.apache.fluo.core.util.CuratorUtil;
+
+public class FluoStatus {
+
+ public static void main(String[] args) throws Exception {
+if (args.length != 2) {
+ System.err.println("Usage: FluoStatus
");
+ System.exit(-1);
+}
+String connectionPropsPath = args[0];
+String applicationName = args[1];
+Objects.requireNonNull(connectionPropsPath);
+File connectionPropsFile = new File(connectionPropsPath);
+Preconditions.checkArgument(connectionPropsFile.exists(),
connectionPropsPath
++ " does not exist");
+
+FluoConfiguration config = new FluoConfiguration(connectionPropsFile);
+config.setApplicationName(applicationName);
+
+try (CuratorFramework curator = CuratorUtil.newFluoCurator(config)) {
+ curator.start();
+ if (curator.checkExists().forPath("/" + applicationName) == null) {
+System.out.println("NOT_FOUND");
+return;
+ }
+}
+
+if (FluoAdminImpl.oracleExists(config)) {
Review comment:
what about workers?
--------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] mikewalch opened a new issue #914: Allow configuration to be passed to 'fluo oracle' & 'fluo worker' commands
mikewalch opened a new issue #914: Allow configuration to be passed to 'fluo oracle' & 'fluo worker' commands URL: https://github.com/apache/fluo/issues/914 The `fluo oracle` and `fluo worker` should be able to support the following syntax to override the configuration read from `fluo-conn.properties`: ``` fluo oracle -p key1=value1,key2=value2 fluo oracle myapp -p fluo.connection.zookepers=localhost/fluo,fluo.connection.zookeeper.secret=mysecret ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed issue #2: Print yarn app id
mikewalch closed issue #2: Print yarn app id URL: https://github.com/apache/fluo-yarn/issues/2 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on issue #2: Print yarn app id
mikewalch commented on issue #2: Print yarn app id URL: https://github.com/apache/fluo-yarn/issues/2#issuecomment-322837270 Fixed by #1 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new pull request #5: Fixes #3 Check if application is running before starting
mikewalch opened a new pull request #5: Fixes #3 Check if application is running before starting URL: https://github.com/apache/fluo-yarn/pull/5 * Also improved README.md This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new pull request #913: Fluo 912
mikewalch opened a new pull request #913: Fluo 912 URL: https://github.com/apache/fluo/pull/913 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjmctague commented on issue #879: CollisionIT test failed on Travis
cjmctague commented on issue #879: CollisionIT test failed on Travis URL: https://github.com/apache/fluo/issues/879#issuecomment-322834046 ``` Running org.apache.fluo.integration.impl.CollisionIT DEBUG oldestTs : 6839 recentTS : 6839 DEBUG 0 stat:processed [] 6764-WRITE 6755 DEBUG 0 stat:processed [] 6755-DATA 181 DEBUG 0 stat:total [] 6499-TX_DONE DEBUG 0 stat:total [] 6499-WRITE 6472 PRIMARY DEBUG 0 stat:total [] 6472-DATA 181 DEBUG 1 stat:processed [] 6831-TX_DONE DEBUG 1 stat:processed [] 6831-WRITE 6826 PRIMARY DEBUG 1 stat:processed [] 6826-DATA 223 DEBUG 1 stat:total [] 6821-TX_DONE DEBUG 1 stat:total [] 6821-WRITE 6820 PRIMARY DEBUG 1 stat:total [] 6820-DATA 223 DEBUG 2 stat:processed [] 6681-TX_DONE DEBUG 2 stat:processed [] 6681-WRITE 6655 PRIMARY DEBUG 2 stat:processed [] 6655-DATA 190 DEBUG 2 stat:total [] 6552-TX_DONE DEBUG 2 stat:total [] 6552-WRITE 6540 PRIMARY DEBUG 2 stat:total [] 6540-DATA 190 DEBUG 3 stat:processed [] 6798-WRITE 6797 DEBUG 3 stat:processed [] 6797-DATA 201 DEBUG 3 stat:total [] 6739-TX_DONE DEBUG 3 stat:total [] 6739-WRITE 6707 PRIMARY DEBUG 3 stat:total [] 6707-DATA 201 DEBUG 4 stat:processed [] 6813-WRITE 6805 DEBUG 4 stat:processed [] 6805-DATA 205 DEBUG 4 stat:total [] 6792-TX_DONE DEBUG 4 stat:total [] 6792-WRITE 6778 PRIMARY DEBUG 4 stat:total [] 6778-DATA 205 DEBUG all stat:total [] 6831-WRITE 6826 DEBUG all stat:total [] 140-DEL_LOCK ABORT PRIMARY 0 DEBUG all stat:total [] 140-DEL_LOCK ABORT PRIMARY 0 DEBUG all stat:total [] 6826-DATA1000 Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 32.547 sec <<< FAILURE! - in org.apache.fluo.integration.impl.CollisionIT testLotsOfCollisions(org.apache.fluo.integration.impl.CollisionIT) Time elapsed: 32.517 sec <<< FAILURE! java.lang.AssertionError: Duplicate row col all:stat:total:2000 at org.apache.fluo.integration.impl.CollisionIT.testLotsOfCollisions(CollisionIT.java:174) ``` https://travis-ci.org/apache/fluo/jobs/265159869 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on issue #912: Create 'fluo status' command
mikewalch commented on issue #912: Create 'fluo status' command URL: https://github.com/apache/fluo/issues/912#issuecomment-322778409 I think `NOT_FOUND` is better than `UNINITIALIZED` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed pull request #910: Fixes #909 Move netty jar to lib/twill
mikewalch closed pull request #910: Fixes #909 Move netty jar to lib/twill URL: https://github.com/apache/fluo/pull/910 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed issue #909: Move netty jar to lib/twill
mikewalch closed issue #909: Move netty jar to lib/twill URL: https://github.com/apache/fluo/issues/909 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed pull request #4: Removed unused dependencies & add travis.yml
mikewalch closed pull request #4: Removed unused dependencies & add travis.yml URL: https://github.com/apache/fluo-yarn/pull/4 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #4: Removed unused dependencies & add travis.yml
mikewalch commented on a change in pull request #4: Removed unused dependencies & add travis.yml URL: https://github.com/apache/fluo-yarn/pull/4#discussion_r133451206 ## File path: .travis.yml ## @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more contributor license +# agreements. See the NOTICE file distributed with this work for additional information regarding +# copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance with the License. You may obtain a +# copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software distributed under the License +# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express +# or implied. See the License for the specific language governing permissions and limitations under +# the License. + +language: java +jdk: + - oraclejdk8 +script: mvn -U verify +sudo: false Review comment: Ok, I'll remove This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #4: Removed unused dependencies & add travis.yml
ctubbsii commented on a change in pull request #4: Removed unused dependencies & add travis.yml URL: https://github.com/apache/fluo-yarn/pull/4#discussion_r12451 ## File path: .travis.yml ## @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more contributor license +# agreements. See the NOTICE file distributed with this work for additional information regarding +# copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance with the License. You may obtain a +# copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software distributed under the License +# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express +# or implied. See the License for the specific language governing permissions and limitations under +# the License. + +language: java +jdk: + - oraclejdk8 +script: mvn -U verify +sudo: false Review comment: I think `sudo: false` is default, but in any case, I don't think it's necessary to specify one way or the other, because this build probably does not depend on either. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii closed pull request #911: Preserve license header comment formatting
ctubbsii closed pull request #911: Preserve license header comment formatting URL: https://github.com/apache/fluo/pull/911 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii closed pull request #905: Add Accumulo API checks to checkstyle rules
ctubbsii closed pull request #905: Add Accumulo API checks to checkstyle rules URL: https://github.com/apache/fluo/pull/905 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on issue #4: Removed unused dependencies & add travis.yml
mikewalch commented on issue #4: Removed unused dependencies & add travis.yml URL: https://github.com/apache/fluo-yarn/pull/4#issuecomment-322593909 @cjmctague, I created a [infra ticket](https://issues.apache.org/jira/browse/INFRA-14884) to enable the repo. The PR can merged before this work is done. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjmctague commented on issue #4: Removed unused dependencies & add travis.yml
cjmctague commented on issue #4: Removed unused dependencies & add travis.yml URL: https://github.com/apache/fluo-yarn/pull/4#issuecomment-322592008 Someone with admin rights is gonna need to activate the Travis repo first https://travis-ci.org/profile/apache This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjmctague commented on issue #4: Removed unused dependencies & add travis.yml
cjmctague commented on issue #4: Removed unused dependencies & add travis.yml URL: https://github.com/apache/fluo-yarn/pull/4#issuecomment-322592008 Someone with admin right is gonna need to activate the Travis repo first https://travis-ci.org/profile/apache This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new issue #912: Create 'fluo status' command
mikewalch opened a new issue #912: Create 'fluo status' command URL: https://github.com/apache/fluo/issues/912 There should be a `fluo status ` command to check status (i.e `UNINITIALIZED`, `RUNNING`, & `STOPPED`) of Fluo application. This would be helpful for downstream launchers. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new pull request #4: Removed unused dependencies & add travis.yml
mikewalch opened a new pull request #4: Removed unused dependencies & add travis.yml URL: https://github.com/apache/fluo-yarn/pull/4 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii opened a new pull request #911: Preserve license header comment formatting
ctubbsii opened a new pull request #911: Preserve license header comment formatting URL: https://github.com/apache/fluo/pull/911 This change makes the formatter stop formatting header comments at the top of Java source files. This ensures the licenses are formatted in the way they are supposed to, for easier readability. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed pull request #1: Initial implementation of Apache Fluo YARN launcher
mikewalch closed pull request #1: Initial implementation of Apache Fluo YARN launcher URL: https://github.com/apache/fluo-yarn/pull/1 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher
mikewalch commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher URL: https://github.com/apache/fluo-yarn/pull/1#discussion_r133212438 ## File path: pom.xml ## @@ -69,6 +69,11 @@ 13.0.1 +io.netty Review comment: Added This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher
mikewalch commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher URL: https://github.com/apache/fluo-yarn/pull/1#discussion_r133212287 ## File path: distribution/bin/fluo-yarn ## @@ -36,6 +36,7 @@ if [ ! -f "$conf/fluo-yarn-env.sh" ]; then fi source "$conf/fluo-yarn-env.sh" export CLASSPATH=$LAUNCHER_CLASSPATH +export HADOOP_HOME=$HADOOP_PREFIX Review comment: It's not needed so I removed it This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new pull request #910: Fixes #909 Move netty jar to lib/twill
mikewalch opened a new pull request #910: Fixes #909 Move netty jar to lib/twill URL: https://github.com/apache/fluo/pull/910 * Also put log4j & logback jars together This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher
keith-turner commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher URL: https://github.com/apache/fluo-yarn/pull/1#discussion_r133194842 ## File path: distribution/bin/fluo-yarn ## @@ -36,6 +36,7 @@ if [ ! -f "$conf/fluo-yarn-env.sh" ]; then fi source "$conf/fluo-yarn-env.sh" export CLASSPATH=$LAUNCHER_CLASSPATH +export HADOOP_HOME=$HADOOP_PREFIX Review comment: why set this? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher
keith-turner commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher URL: https://github.com/apache/fluo-yarn/pull/1#discussion_r133195373 ## File path: pom.xml ## @@ -69,6 +69,11 @@ 13.0.1 +io.netty Review comment: could add a comment This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new issue #3: Check if application is running before starting in YARN
mikewalch opened a new issue #3: Check if application is running before starting in YARN URL: https://github.com/apache/fluo-yarn/issues/3 The `fluo-yarn` command currently has no check for running application and will start a second Fluo application This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner closed pull request #899: fixes #893 added ability to secure zookeeper
keith-turner closed pull request #899: fixes #893 added ability to secure zookeeper URL: https://github.com/apache/fluo/pull/899 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner closed issue #893: Support locking down properties in zookeeper
keith-turner closed issue #893: Support locking down properties in zookeeper URL: https://github.com/apache/fluo/issues/893 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner closed pull request #902: fixes #894 merge SimpleConfiguration Objects
keith-turner closed pull request #902: fixes #894 merge SimpleConfiguration Objects URL: https://github.com/apache/fluo/pull/902 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on issue #900: Document properties in FluoConfiguration javadoc
keith-turner commented on issue #900: Document properties in FluoConfiguration javadoc URL: https://github.com/apache/fluo/issues/900#issuecomment-322202177 @kpm1985 I opened kpm1985/fluo#1 and made some comments. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kpm1985 commented on issue #900: Document properties in FluoConfiguration javadoc
kpm1985 commented on issue #900: Document properties in FluoConfiguration javadoc URL: https://github.com/apache/fluo/issues/900#issuecomment-322037041 This is still a work in progress, but it would be appreciated if you could give a quick review to make sure I'm not over/under documenting anything so far. I plan on finishing this up tomorrow, and making the suggested changes as soon as I get them :) https://github.com/kpm1985/fluo/blob/fluo-900/modules/api/src/main/java/org/apache/fluo/api/config/FluoConfiguration.java This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher
mikewalch commented on a change in pull request #1: Initial implementation of
Apache Fluo YARN launcher
URL: https://github.com/apache/fluo-yarn/pull/1#discussion_r132734280
##
File path: distribution/lib/fetch.sh
##
@@ -0,0 +1,81 @@
+#! /usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
contributor license
+# agreements. See the NOTICE file distributed with this work for additional
information regarding
+# copyright ownership. The ASF licenses this file to you under the Apache
License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the License.
You may obtain a
+# copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
distributed under the License
+# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express
+# or implied. See the License for the specific language governing permissions
and limitations under
+# the License.
+
+lib_dir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
+maven_prefix=https://repo1.maven.org/maven2
+
+function download {
+ IFS=':' read -ra DEP <<< "$1"
+ dir=$lib_dir/
+ if [ -n "$2" ]; then
+dir=$lib_dir/$2
+if [ ! -d $dir ]; then
+ mkdir $dir
+fi
+ fi
+ group=${DEP[0]}
+ artifact=${DEP[1]}
+ ftype=${DEP[2]}
+ version=${DEP[3]}
+ fn=$artifact-$version.$ftype
+ path="${group//.//}/$artifact/$version/$fn"
+ download_url=$maven_prefix/$path
+
+ if [ -f $dir/$fn ]; then
+echo "SUCCESS: Dependency exists - $dir/$fn"
+ else
+wget -q $download_url -P $dir
+if [ $? == 0 ]; then
+ echo "SUCCESS: Dependency downloaded from $download_url"
+else
+ echo "ERROR: Dependency download failed - $download_url"
+fi
+ fi
+}
+
+echo "Fetching Fluo YARN launcher dependencies"
+download aopalliance:aopalliance:jar:1.0
+download ch.qos.logback:logback-classic:jar:1.1.3
+download ch.qos.logback:logback-core:jar:1.1.3
+download com.101tec:zkclient:jar:0.3
+download com.google.code.findbugs:jsr305:jar:2.0.1
+download com.google.code.gson:gson:jar:2.8.0
+download com.google.guava:guava:jar:13.0.1
+download com.yammer.metrics:metrics-annotation:jar:2.2.0
+download com.yammer.metrics:metrics-core:jar:2.2.0
+download net.sf.jopt-simple:jopt-simple:jar:3.2
+download org.apache.kafka:kafka_2.10:jar:0.8.0
+download org.apache.twill:twill-api:jar:0.11.0
+download org.apache.twill:twill-common:jar:0.11.0
+download org.apache.twill:twill-core:jar:0.11.0
+download org.apache.twill:twill-discovery-api:jar:0.11.0
+download org.apache.twill:twill-discovery-core:jar:0.11.0
+download org.apache.twill:twill-ext:jar:0.11.0
+download org.apache.twill:twill-yarn:jar:0.11.0
+download org.apache.twill:twill-zookeeper:jar:0.11.0
Review comment:
fixed. `fluo-yarn-env.sh` now adds jar to classpath given ZOOKEEPER_HOME
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] keith-turner opened a new issue #80: Improve Fluo's front page and short description.
keith-turner opened a new issue #80: Improve Fluo's front page and short description. URL: https://github.com/apache/fluo-website/issues/80 Recent events have caused discussion in the community about how best to quickly describe Fluo. As a result we have a few different short descriptions floating around. Description from Github readme ``` Apache Fluo is an open source implementation of Percolator (which populates Google's search index) for Apache Accumulo. Fluo makes it possible to update the results of a large-scale computation, index, or analytic as new data is discovered. Check out the Fluo project website for news and general information. ``` Description on website ``` Apache Fluo is an open source implementation of Percolator (which populates Google's search index) for Apache Accumulo. With Fluo, users can continuously join new data into large existing data sets without reprocessing all data. Unlike batch and streaming frameworks, Fluo offers much lower latency and can operate on extremely large data sets. ``` Description in August 2017 board report. ``` - Apache Fluo is a distributed processing system built on Apache Accumulo. Fluo users can easily setup workflows that execute cross node transactions when data changes. These workflows enable users to continuously join new data into large existing data sets with low latency while avoiding reprocessing all data. ``` Below are some of the concepts these short descriptions are trying to communicate. What else needs to be in this outline? Can we improve the front page of the website to be more informative and succinct? The front page does not have to touch on all aspects, it could possibly link out for more details or omit some aspects. * History * Based on Percolator design. * What capabilities it offers to users * Continuously join new data into large existing data sets without reprocessing all data * Keep multiple dependent derived data sets (similar to materialized views) * Emit changes in derived data sets to external systems * Continually keep a large index up to date as new data arrives. * Update external analytic systems. * How it works * Cross node transactions * Notifications * Observers that execute based on notifications * Context, how does it compare to other technologies. Explaining Fluo relative to other technologies may help people understand Fluo more quickly. * lower latency than batch * larger data sets than streaming This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner opened a new issue #908: Discuss get jars behavior
keith-turner opened a new issue #908: Discuss get jars behavior URL: https://github.com/apache/fluo/issues/908 For get jars command add in #907 the command expects the target directory to exist but does not care if the directory is non-empty. I found myself wondering if the behavior should change or if the docs should be improved to clearly state the behavior for a non-empty dir while looking at the PR. I was not sure. Below are some thoughts I had. * Should the get jars command create the dir? * If the directory is not empty should the get jars command fail? Their are two use cases to consider. * A user intentionally adds jars to a dir before calling get jars and does not expect it to empty. * A user unintentionally reuses a dir and expects the dir to be empty when its not and ends ups extra/unexpected jars in the dir. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed pull request #907: Fixes #906 - Create 'fluo get-jars' command
mikewalch closed pull request #907: Fixes #906 - Create 'fluo get-jars' command URL: https://github.com/apache/fluo/pull/907 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed issue #906: Create 'fluo get-jars' command
mikewalch closed issue #906: Create 'fluo get-jars' command URL: https://github.com/apache/fluo/issues/906 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii closed pull request #1: Provide initial README
ctubbsii closed pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #1: Provide initial README
ctubbsii commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132544784 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and +other projects' APIs.** + +## Features and Goals + +This project aims to fill a void in Java, by providing convenient objects to +represent a string of bytes and associated utility classes for situations when +a raw byte array is not appropriate. + +Specifically, it provides a `ByteSequence` interface, analogous to Java's +`CharSequence`, an immutable `Bytes` implementation analogous to Java's +`String`, and a corresponding `BytesBuilder` analogous to Java's +`StringBuilder`. + +The provided classes have appropriate methods for serialization, and proper +equals and hashCode implementations, as well as a comparator for +`ByteSequence`, so they will be suitable for use in `Set`s and as keys in +`Map`s. + +An immutable bytes implementation makes it possible to pass data between APIs +without the need for performance-killing protective copies. This benefit is +compounded if this library is used by multiple projects, as the need to make +protective copies while passing data between a project and its dependency's API +is eliminated. + +This project aims to provide a fluent and intuitive API, with support for +conversions to/from other common types, such as `ByteBuffer`, `byte[]`, and +`CharSequence`/`String`. + +This project requires at least Java 8, and supports `Stream` and functional +APIs where appropriate. + +See this [blog post][blog] for some additional background. + +## Safe for APIs + +Using an external library in a project's API poses some risks to that project, +especially if it and its dependencies depend on different versions of that +library. This project attempts to mitigate those risks, so that it can be used +safely by other projects. + +This project is made safe for reuse in other projects' APIs by adopting the +following principles: + +* Using [Semantic Versioning 2.0.0][semver] to make strong declarations about + backwards-compatibility +* Strongly avoid breaking changes (avoid major version bumps), so that projects Review comment: Yeah, removal is not explicitly mentioned, but if it's "dangerous" or if a "better alternative exists", it seems appropriate to assume it is "subject to" removal, even if removal is deferred indefinitely because of other factors (stability). Part of the problem with the new flag is that "subject to" is not well-defined. I hear "subject to" and think "could possibly be, at the determination of the developers"... but that's basically *always the case*. -------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher
keith-turner commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher URL: https://github.com/apache/fluo-yarn/pull/1#discussion_r132476195 ## File path: distribution/conf/fluo-yarn.properties ## @@ -20,8 +20,11 @@ # Properties that have no default are uncommented and must be set by # the user. -## General YARN properties -## --- +## General properties +## -- +## Zookeeper connection string specifying host and chroot where Fluo YARN launcher stores data. +## A chroot directory suff must be specified but doesn't need to be named '/fluo-yarn'. Review comment: Is the chroot dir limited to depth 1? If so that would be useful to mention. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Provide initial README
keith-turner commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132464158 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is a simple library with the goal of providing an extremely +stable API for handling bytes, suitable for use in [Apache Fluo][fluo] and +other projects' APIs.** + +## Features and Goals + +This project aims to fill a void in Java, by providing convenient objects to +represent a sequence of bytes and associated utility classes for situations +when a raw byte array is not appropriate. + +Specifically, it provides a `ByteSequence` interface, analogous to Java's +`CharSequence`, an immutable `Bytes` implementation analogous to Java's +`String`, and a corresponding `BytesBuilder` analogous to Java's +`StringBuilder`. + +The provided classes have appropriate methods for serialization, and proper +equals and hashCode implementations, as well as a comparator for +`ByteSequence`, so they will be suitable for use in `Set`s and as keys in +`Map`s. + +An immutable bytes implementation makes it possible to pass data between APIs +without the need for performance-killing protective copies. This benefit is +compounded if this library is used by multiple projects, as the need to make +protective copies while passing data between a project and its dependency's API +is eliminated. + +This project aims to provide a fluent and intuitive API, with support for +conversions to/from other common types, such as `ByteBuffer`, `byte[]`, and +`CharSequence`/`String`. + +This project requires at least Java 8, and supports `Stream` and functional +APIs where appropriate. + +See this [blog post][blog] for some additional background. + +## Safe for APIs + +Using an external library in a project's API poses some risks to that project, +especially if it and its dependencies depend on different versions of that +library. This project attempts to mitigate those risks, so that it can be used +safely by other projects. + +This project is made safe for reuse in other projects' APIs by adopting the +following principles: + +* Using [Semantic Versioning 2.0.0][semver] to make strong declarations about + backwards-compatibility +* Strongly avoid breaking changes (avoid major version bumps), so that projects + can converge on the latest version required by their code and that of their Review comment: When I read `the latest version required` I found myself wondering if `the latest version of Fluo Bytes required` would be better. I like it because its more precise. I don't like that it adds words I think people could infer (but not completely sure because it all makes perfect sense to me). I am not advocating for this change, it was just a nagging thought I had when reading this. This document frequently uses `this project`. When thinking about the above I started mulling over whether replacing `this project` with `Fluo Bytes` would be a good thing. I have not come to any conclusion. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Provide initial README
keith-turner commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132451240 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and +other projects' APIs.** + +## Features and Goals + +This project aims to fill a void in Java, by providing convenient objects to +represent a string of bytes and associated utility classes for situations when +a raw byte array is not appropriate. + +Specifically, it provides a `ByteSequence` interface, analogous to Java's +`CharSequence`, an immutable `Bytes` implementation analogous to Java's +`String`, and a corresponding `BytesBuilder` analogous to Java's +`StringBuilder`. + +The provided classes have appropriate methods for serialization, and proper +equals and hashCode implementations, as well as a comparator for +`ByteSequence`, so they will be suitable for use in `Set`s and as keys in +`Map`s. + +An immutable bytes implementation makes it possible to pass data between APIs +without the need for performance-killing protective copies. This benefit is +compounded if this library is used by multiple projects, as the need to make +protective copies while passing data between a project and its dependency's API +is eliminated. + +This project aims to provide a fluent and intuitive API, with support for +conversions to/from other common types, such as `ByteBuffer`, `byte[]`, and +`CharSequence`/`String`. + +This project requires at least Java 8, and supports `Stream` and functional +APIs where appropriate. + +See this [blog post][blog] for some additional background. + +## Safe for APIs + +Using an external library in a project's API poses some risks to that project, +especially if it and its dependencies depend on different versions of that +library. This project attempts to mitigate those risks, so that it can be used +safely by other projects. + +This project is made safe for reuse in other projects' APIs by adopting the +following principles: + +* Using [Semantic Versioning 2.0.0][semver] to make strong declarations about + backwards-compatibility +* Strongly avoid breaking changes (avoid major version bumps), so that projects Review comment: > I think all deprecations are subject to removal I think this depends on the project. I realized I have never looked at the javadoc for the annotation, it says the following. ``` A program element annotated @Deprecated is one that programmers are discouraged from using, typically because it is dangerous, or because a better alternative exists. Compilers warn when a deprecated program element is used or overridden in non-deprecated code. ``` Removal is not mentioned, so the new parameter will mesh well with the current javadoc. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Provide initial README
keith-turner commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132444629 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is a simple library with the goal of providing an extremely +stable API for handling bytes, suitable for use in [Apache Fluo][fluo] and +other projects' APIs.** + +## Features and Goals + +This project aims to fill a void in Java, by providing convenient objects to +represent a sequence of bytes and associated utility classes for situations +when a raw byte array is not appropriate. + +Specifically, it provides a `ByteSequence` interface, analogous to Java's +`CharSequence`, an immutable `Bytes` implementation analogous to Java's +`String`, and a corresponding `BytesBuilder` analogous to Java's +`StringBuilder`. + +The provided classes have appropriate methods for serialization, and proper +equals and hashCode implementations, as well as a comparator for +`ByteSequence`, so they will be suitable for use in `Set`s and as keys in +`Map`s. + +An immutable bytes implementation makes it possible to pass data between APIs +without the need for performance-killing protective copies. This benefit is +compounded if this library is used by multiple projects, as the need to make +protective copies while passing data between a project and its dependency's API +is eliminated. + +This project aims to provide a fluent and intuitive API, with support for +conversions to/from other common types, such as `ByteBuffer`, `byte[]`, and +`CharSequence`/`String`. + +This project requires at least Java 8, and supports `Stream` and functional +APIs where appropriate. + +See this [blog post][blog] for some additional background. + +## Safe for APIs + +Using an external library in a project's API poses some risks to that project, +especially if it and its dependencies depend on different versions of that +library. This project attempts to mitigate those risks, so that it can be used +safely by other projects. + +This project is made safe for reuse in other projects' APIs by adopting the +following principles: + +* Using [Semantic Versioning 2.0.0][semver] to make strong declarations about + backwards-compatibility +* Strongly avoid breaking changes (avoid major version bumps), so that projects + can converge on the latest version required by their code and that of their + dependencies, without risk of incompatibility +* No runtime dependencies to use the library, to eliminate any potential Review comment: `No runtime dependencies to use the library` this seems a bit off. To use this library a runtime dep on this library will be needed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Provide initial README
keith-turner commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132443633 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and Review comment: I like your wording better. One thing I didn't like in my wording was `bytes library`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #1: Provide initial README
ctubbsii commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132365844 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and +other projects' APIs.** + +## Features and Goals + +This project aims to fill a void in Java, by providing convenient objects to +represent a string of bytes and associated utility classes for situations when +a raw byte array is not appropriate. + +Specifically, it provides a `ByteSequence` interface, analogous to Java's +`CharSequence`, an immutable `Bytes` implementation analogous to Java's +`String`, and a corresponding `BytesBuilder` analogous to Java's +`StringBuilder`. + +The provided classes have appropriate methods for serialization, and proper +equals and hashCode implementations, as well as a comparator for +`ByteSequence`, so they will be suitable for use in `Set`s and as keys in +`Map`s. + +An immutable bytes implementation makes it possible to pass data between APIs +without the need for performance-killing protective copies. This benefit is +compounded if this library is used by multiple projects, as the need to make +protective copies while passing data between a project and its dependency's API +is eliminated. + +This project aims to provide a fluent and intuitive API, with support for +conversions to/from other common types, such as `ByteBuffer`, `byte[]`, and +`CharSequence`/`String`. + +This project requires at least Java 8, and supports `Stream` and functional +APIs where appropriate. + +See this [blog post][blog] for some additional background. + +## Safe for APIs + +Using an external library in a project's API poses some risks to that project, +especially if it and its dependencies depend on different versions of that +library. This project attempts to mitigate those risks, so that it can be used +safely by other projects. + +This project is made safe for reuse in other projects' APIs by adopting the +following principles: + +* Using [Semantic Versioning 2.0.0][semver] to make strong declarations about + backwards-compatibility +* Strongly avoid breaking changes (avoid major version bumps), so that projects Review comment: Ugh, I hate that new "feature" to Java 9, especially since it defaults to "false". Personally, I think all deprecations are subject to removal. I really don't want to start putting "forRemoval = true" in all my deprecation annotations just so users don't think "this isn't going away". I'd rather they'd have created a new annotation for things which are expected to remain, but should not be used by users, which generated similar compiler warnings as Deprecated. This one is really going to cause confusion with defaulting to false. A lot of them are going to be set to false simply because nobody's explicitly setting them to true. Users are going to be disappointed when they disappear. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #1: Provide initial README
ctubbsii commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132365435 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and +other projects' APIs.** + +## Features and Goals + +This project aims to fill a void in Java, by providing convenient objects to +represent a string of bytes and associated utility classes for situations when +a raw byte array is not appropriate. Review comment: I did try to write more on this initially, but everything I came up with was too long and seemed to drift off topic. I boiled it down to what you see here, mentioning this library is useful when byte arrays aren't appropriate, and then immediately following that statement by some of the features that make it so, naming all of the things you list here. I don't know that I'd really want more details in the README than that. I certainly don't want to repeat a blog post, or make the README a big wall of text that provides so many details, it's hard to get the gist. But, if you have specific suggestions which can improve this, I'm happy to work it in. I, however, am near my editing limit on this subject. :) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #1: Provide initial README
ctubbsii commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132364598 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and +other projects' APIs.** + +## Features and Goals + +This project aims to fill a void in Java, by providing convenient objects to +represent a string of bytes and associated utility classes for situations when Review comment: Fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #1: Provide initial README
ctubbsii commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132364588 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and Review comment: I didn't see your exact wording above, because it didn't show correctly when I saw this comment in my email, but I pushed a change which basically applies the same fix, based on your feedback above the blockquote section. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new pull request #907: Fixes #906 - Create 'fluo get-jars' command
mikewalch opened a new pull request #907: Fixes #906 - Create 'fluo get-jars' command URL: https://github.com/apache/fluo/pull/907 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new issue #906: Create 'fluo get-jars' command
mikewalch opened a new issue #906: Create 'fluo get-jars' command URL: https://github.com/apache/fluo/issues/906 Command will retrieve application jars store in HDFS to a local directory This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #902: fixes #894 merge SimpleConfiguration Objects
keith-turner commented on a change in pull request #902: fixes #894 merge
SimpleConfiguration Objects
URL: https://github.com/apache/fluo/pull/902#discussion_r132312049
##
File path:
modules/api/src/test/java/org/apache/fluo/api/config/SimpleConfigurationTest.java
##
@@ -64,12 +64,20 @@ public void testMerge() {
diff.setProperty("set1", "value13");
diff.setProperty("set3", "value14");
+SimpleConfiguration noChange = new SimpleConfiguration(sc3);
+// check to make sure this is not being changed.
+sc3.orElse(sc1).orElse(sc2);
Review comment:
need to keep the return val like
```java
sc4 = sc3.orElse(sc1).orElse(sc2);
sc3.set(...)
//ensure sc4 did not change.
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] mikewalch closed issue #897: Add high level application lifecycle overview to docs
mikewalch closed issue #897: Add high level application lifecycle overview to docs URL: https://github.com/apache/fluo/issues/897 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed pull request #904: Fixes #897 added context to application docs and moved some docs from?
mikewalch closed pull request #904: Fixes #897 added context to application docs and moved some docs from? URL: https://github.com/apache/fluo/pull/904 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Provide initial README
keith-turner commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132304408 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and +other projects' APIs.** + +## Features and Goals + +This project aims to fill a void in Java, by providing convenient objects to +represent a string of bytes and associated utility classes for situations when Review comment: could say sequence of bytes to since in java String is so strongly associated with char This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Provide initial README
keith-turner commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132304118 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and Review comment: A few things I found unclear. * What is an `API project`? * What does `stable library` mean? Below is my attempt to clear those things up. My main goal was to state its a library with a stable API. ``` Apache Fluo Bytes is a project with the goal of providing a bytes library with an extremely stable API that is suitable for use in other projects' APIs. ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Provide initial README
keith-turner commented on a change in pull request #1: Provide initial README URL: https://github.com/apache/fluo-bytes/pull/1#discussion_r132306642 ## File path: README.md ## @@ -0,0 +1,102 @@ + + +# Apache Fluo Bytes + +[![Build Status][ti]][tl] [![Apache License][li]][ll] [![Maven Central][mi]][ml] [![Javadoc][ji]][jl] + +**Apache Fluo Bytes is an API project with the goal of providing an extremely +stable library for handling bytes, suitable for use in [Apache Fluo][fluo] and +other projects' APIs.** + +## Features and Goals + +This project aims to fill a void in Java, by providing convenient objects to +represent a string of bytes and associated utility classes for situations when +a raw byte array is not appropriate. + +Specifically, it provides a `ByteSequence` interface, analogous to Java's +`CharSequence`, an immutable `Bytes` implementation analogous to Java's +`String`, and a corresponding `BytesBuilder` analogous to Java's +`StringBuilder`. + +The provided classes have appropriate methods for serialization, and proper +equals and hashCode implementations, as well as a comparator for +`ByteSequence`, so they will be suitable for use in `Set`s and as keys in +`Map`s. + +An immutable bytes implementation makes it possible to pass data between APIs +without the need for performance-killing protective copies. This benefit is +compounded if this library is used by multiple projects, as the need to make +protective copies while passing data between a project and its dependency's API +is eliminated. + +This project aims to provide a fluent and intuitive API, with support for +conversions to/from other common types, such as `ByteBuffer`, `byte[]`, and +`CharSequence`/`String`. + +This project requires at least Java 8, and supports `Stream` and functional +APIs where appropriate. + +See this [blog post][blog] for some additional background. + +## Safe for APIs + +Using an external library in a project's API poses some risks to that project, +especially if it and its dependencies depend on different versions of that +library. This project attempts to mitigate those risks, so that it can be used +safely by other projects. + +This project is made safe for reuse in other projects' APIs by adopting the +following principles: + +* Using [Semantic Versioning 2.0.0][semver] to make strong declarations about + backwards-compatibility +* Strongly avoid breaking changes (avoid major version bumps), so that projects Review comment: It will be so nice in Java 9 when the deprecated annotation has the forRemoval parameter. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from?
This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from?
mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from? URL: https://github.com/apache/fluo/pull/904#discussion_r132262720 ## File path: docs/applications.md ## @@ -140,12 +135,123 @@ To create an observer, follow these steps: 3. Build a jar containing these classes and include this jar in the `lib/` directory of your Fluo application. -4. Configure your Fluo instance to use this observer provider by modifying the Application section of -[fluo-app.properties]. -5. Initialize Fluo. During initialization Fluo will obtain the observed columns from the -ObserverProvider and persist the columns in Zookeeper. These columns persisted in Zookeeper -are used by transactions to know when to trigger observers. -6. Start your Fluo instance so that your Fluo workers load the new observer. +4. Configure your Fluo application to use this observer provider by modifying the Application section of +[fluo-app.properties]. Set `fluo.observer.provider` to the observer provider class name. +5. Initialize your Fluo application as described in the next section. During initialization Fluo +will obtain the observed columns from the ObserverProvider and persist the columns in Zookeeper. +These columns persisted in Zookeeper are used by transactions to know when to trigger observers. + +## Initializing a Fluo Application + +Before a Fluo Application can run, it must be initiaized. Below is an overview of what +initialization does and some of the properties that may be set for initialization. + + * **Initialize ZooKeeper** : Each application has its own area in ZooKeeper used for configuration, + Oracle state, and worker coordination. All properties, except `fluo.connections.*`, are copied + into ZooKeeper. For example, if `fluo.worker.num.threads=128` was set, then when a worker process + starts it will read this from ZooKeeper. + * **Copy Observer jars to DFS** : Fluo workers processes need the jars containing observers. These + are provided in one of the following ways. + * Set the property `fluo.observer.init.dir` to a local directory containing observer jars. The + jars in this directory are copied to DFS under `/`. When a worker is + started, the jars are pulled from DFS and added to its classpath. + * Set the property `fluo.observer.jars.url` to a directory in DFS containing observer jars. No + copying is done. When a worker is started, the jars are pulled from this location and added to + its classpath. + * Do not set any of the properties above and have the mechanism that starts the worker process + add the needed jars to the classpath. + * **Create Accumulo table** : Each Fluo application creates and configures an Accumulo table. The + `fluo.accumulo.*` properties determine which Accumulo instance is used. For performance reasons, + Fluo runs its own code in Accumulo tablet servers. Fluo attempts to copy Fluo jars into DFS and + configure Accumulo to use them. Fluo first checks the property `fluo.accumulo.jars` and if set, + copies the jars listed there. If that property is not set, then Fluo looks on the classpath to + find jars. Jars are copied to a location under `/`. + +Below are the steps to initialize an application from the command line. It is also possible to +initialize an application using Fluo's Java API. + +1. Create a copy of [fluo-app.properties] for your Fluo application. + +cp $FLUO_HOME/conf/fluo-app.properties /path/to/myapp/fluo-app.properties + +2. Edit your copy of [fluo-app.properties] and make sure to set the following: + +* Class name of your ObserverProvider +* Paths to your Fluo observer jars +* Accumulo configuration +* DFS configuration + + When configuring the observer section of fluo-app.properties, you can configure your instance for the + [phrasecount] application if you have not created your own application. See the [phrasecount] Review comment: No link for phrasecount This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from?
mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from? URL: https://github.com/apache/fluo/pull/904#discussion_r132269526 ## File path: docs/applications.md ## @@ -17,8 +17,18 @@ limitations under the License. # Fluo Applications -Once you have Fluo installed and running on your cluster, you can now run Fluo applications which -consist of clients and observers. +Once you have Fluo installed and running on your cluster, you can run Fluo applications consisting +of clients and observers. A client is a user started process that uses the Fluo API. An Observer +is user provided code run by Fluo Workers in response to notifications. Notifications are set by Review comment: Could link to architecture.md here if users want to better understand components of an application. This text could be incorporated into architecture.md to improve it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #902: fixes #894 merge SimpleConfiguration Objects
keith-turner commented on a change in pull request #902: fixes #894 merge
SimpleConfiguration Objects
URL: https://github.com/apache/fluo/pull/902#discussion_r132267259
##
File path:
modules/api/src/main/java/org/apache/fluo/api/config/SimpleConfiguration.java
##
@@ -250,6 +250,43 @@ public SimpleConfiguration subset(String prefix) {
return new SimpleConfiguration(internalConfig.subset(prefix));
}
+ /**
+ * @param fallback SimpleConfiguration to join together
+ * @return a new simple configuration that contains all of the current
properties from this plus
+ * the properties from fallback that are not present in this.
+ *
+ * @since 1.2.0
+ */
+ public SimpleConfiguration orElse(SimpleConfiguration fallback) {
+for (Map.Entry<String, String> entry : fallback.toMap().entrySet()) {
Review comment:
This implementation modifies this, but the javadoc says returns a new. I
think it I would prefer returning a copy. May be able to do
`SimpleConfiguration copy = new SimpleConfiguration(this)` and then use `copy`
instead of `this` in method.
Would be nice to have a test that verifies following :
```java
sc1 = ...
sc2 = ...
sc3 = sc1.orElse(sc2)
//set something on sc1 and sc2
// verify that sc3 did not change
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #905: Add Accumulo API checks to checkstyle rules
ctubbsii commented on a change in pull request #905: Add Accumulo API checks to checkstyle rules URL: https://github.com/apache/fluo/pull/905#discussion_r132264201 ## File path: src/main/resources/org/apache/fluo/resources/java-checkstyle.xml ## @@ -170,5 +170,11 @@ + Review comment: I copied from the Fluo pom, but did not do any additional testing. I did, however, manually inspect to ensure I understood the expression and that it made sense, as a sanity check for what already existed. The only thing I changed was to make `\.` become `[.]`, because I think that construction is more clear (escapes are hard to reason about). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from?
keith-turner commented on a change in pull request #904: Fixes #897 added
context to application docs and moved some docs from?
URL: https://github.com/apache/fluo/pull/904#discussion_r132261694
##
File path: docs/applications.md
##
@@ -70,31 +80,6 @@ FluoClient client = FluoFactory.newClient(config)
It may help to reference the [API javadocs][API] while you are learning the
Fluo API.
-## Running application code
-
-The `fluo exec {arguments}` provides an easy way to execute
application code. It
-will execute a class with a main method if a jar containing the class is
placed in the lib directory
-of the application. When the class is run, Fluo classes and dependencies will
be on the classpath.
-The `fluo exec` command can inject the applications configuration if the class
is written in the
-following way. Defining the injection point is optional.
-
-```java
-import javax.inject.Inject;
-
-public class AppCommand {
-
- //when run with fluo exec command, the applications configuration will be
injected
- @Inject
- private static FluoConfiguration fluoConfig;
-
- public static void main(String[] args) throws Exception {
-try(FluoClient fluoClient = FluoFactory.newClient(fluoConfig)) {
- //do stuff with Fluo
-}
- }
-}
-```
-
## Creating a Fluo observer
To create an observer, follow these steps:
Review comment:
I didn't understand why you wanted to make this change and didn't change
anything.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] keith-turner commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from?
keith-turner commented on a change in pull request #904: Fixes #897 added
context to application docs and moved some docs from?
URL: https://github.com/apache/fluo/pull/904#discussion_r132261218
##
File path: docs/applications.md
##
@@ -70,31 +80,6 @@ FluoClient client = FluoFactory.newClient(config)
It may help to reference the [API javadocs][API] while you are learning the
Fluo API.
-## Running application code
-
-The `fluo exec {arguments}` provides an easy way to execute
application code. It
-will execute a class with a main method if a jar containing the class is
placed in the lib directory
-of the application. When the class is run, Fluo classes and dependencies will
be on the classpath.
-The `fluo exec` command can inject the applications configuration if the class
is written in the
-following way. Defining the injection point is optional.
-
-```java
-import javax.inject.Inject;
-
-public class AppCommand {
-
- //when run with fluo exec command, the applications configuration will be
injected
- @Inject
- private static FluoConfiguration fluoConfig;
-
- public static void main(String[] args) throws Exception {
-try(FluoClient fluoClient = FluoFactory.newClient(fluoConfig)) {
- //do stuff with Fluo
-}
- }
-}
-```
-
## Creating a Fluo observer
Review comment:
Based on this comment I removed some redundant info at the end of this
section. Is that what you were thinking of?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] mikewalch commented on a change in pull request #899: fixes #893 added ability to secure zookeeper
mikewalch commented on a change in pull request #899: fixes #893 added ability
to secure zookeeper
URL: https://github.com/apache/fluo/pull/899#discussion_r132257442
##
File path:
modules/integration/src/test/java/org/apache/fluo/integration/impl/ZKSecretIT.java
##
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
contributor license
+ * agreements. See the NOTICE file distributed with this work for additional
information regarding
+ * copyright ownership. The ASF licenses this file to you under the Apache
License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express
+ * or implied. See the License for the specific language governing permissions
and limitations under
+ * the License.
+ */
+
+package org.apache.fluo.integration.impl;
+
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.util.Arrays;
+import java.util.concurrent.TimeUnit;
+
+import com.google.common.util.concurrent.Uninterruptibles;
+import org.apache.fluo.accumulo.util.ZookeeperPath;
+import org.apache.fluo.api.client.FluoClient;
+import org.apache.fluo.api.client.FluoFactory;
+import org.apache.fluo.api.client.Snapshot;
+import org.apache.fluo.api.client.Transaction;
+import org.apache.fluo.api.config.FluoConfiguration;
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.observer.ObserverProvider;
+import org.apache.fluo.integration.ITBaseMini;
+import org.apache.zookeeper.KeeperException.NoAuthException;
+import org.apache.zookeeper.ZooKeeper;
+import org.junit.Assert;
+import org.junit.Test;
+
+import static org.apache.fluo.api.observer.Observer.NotificationType.STRONG;
+
+public class ZKSecretIT extends ITBaseMini {
+
+ public static class MyObserverProvider implements ObserverProvider {
+
+@Override
+public void provide(Registry or, Context ctx) {
+ or.forColumn(new Column("edge", "forward"), STRONG).useObserver((tx,
row, col) -> {
+tx.set(tx.get(row, col), new Column("edge", "reverese"), row);
+ });
+}
+
+ }
+
+ @Override
+ protected void setConfig(FluoConfiguration config) {
+config.setZookeeperSecret("are3");
+config.setObserverProvider(MyObserverProvider.class);
+ }
+
+ private ZooKeeper getZookeeper() throws IOException {
+ZooKeeper zk = new ZooKeeper(config.getAppZookeepers(), 3, null);
+
+long start = System.currentTimeMillis();
+while (!zk.getState().isConnected() && System.currentTimeMillis() - start
< 3) {
+ Uninterruptibles.sleepUninterruptibly(10, TimeUnit.MILLISECONDS);
+}
+
+return zk;
+ }
+
+ @Test
+ public void testZKAcls() throws Exception {
+
+// verify basic functionality works when password is set in ZK
+try (FluoClient client =
FluoFactory.newClient(miniFluo.getClientConfiguration())) {
Review comment:
What happens if a user a passes in an incorrect Zookeeper secret? It would
be good to make sure there is a good exception or error message for this case.
A unit test could also be written to verify this behavior.
--------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] mikewalch commented on a change in pull request #902: fixes #894 merge SimpleConfiguration Objects
mikewalch commented on a change in pull request #902: fixes #894 merge
SimpleConfiguration Objects
URL: https://github.com/apache/fluo/pull/902#discussion_r132228853
##
File path:
modules/api/src/main/java/org/apache/fluo/api/config/SimpleConfiguration.java
##
@@ -250,6 +250,40 @@ public SimpleConfiguration subset(String prefix) {
return new SimpleConfiguration(internalConfig.subset(prefix));
}
+ /**
+ * @param fallback SimpleConfiguration to join together
+ * @return a new simple configuration that contains all of the current
properties from this plus
+ * the properties from fallback that are not present in this.
+ *
+ * @since 1.2.0
+ */
+ public SimpleConfiguration orElse(SimpleConfiguration fallback) {
+for (Map.Entry<String, String> entry : fallback.toMap().entrySet()) {
+ if (!this.containsKey(entry.getKey())) {
+this.setProperty(entry.getKey(), entry.getValue());
+ }
+}
+return this;
+ }
+
+ @Override
+ public int hashCode() {
+return Objects.hashCode(this.toString());
+ }
+
+ @Override
+ public boolean equals(Object o) {
+if (o == this) {
+ return true;
+}
+
+if (o instanceof SimpleConfiguration) {
+ SimpleConfiguration sc = (SimpleConfiguration) o;
+ return this.toString().equals(sc.toString());
Review comment:
I am not sure about checking for equality by comparing toString(). It
relies on toString() method to be very consistent. Can properties be printed in
a different order? If so, comparing two identical configuration objects could
return false. For this comparison, you could convert each simple configuration
to map, verify that the maps contain same number of entries, and verify that
all properties & values in on sc matches the other.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii opened a new pull request #905: Add Accumulo API checks to checkstyle rules
ctubbsii opened a new pull request #905: Add Accumulo API checks to checkstyle rules URL: https://github.com/apache/fluo/pull/905 Prevent use of non-public Accumulo APIs being imported into Fluo classes. This removes the need to have a separate execution of checkstyle to do Accumulo API checks in Fluo projects. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch opened a new pull request #78: Remove incubator references on website
mikewalch opened a new pull request #78: Remove incubator references on website URL: https://github.com/apache/fluo-website/pull/78 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed issue #843: Rename FluoConfiguration properties
mikewalch closed issue #843: Rename FluoConfiguration properties URL: https://github.com/apache/fluo/issues/843 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed issue #851: Create FluoList command
mikewalch closed issue #851: Create FluoList command URL: https://github.com/apache/fluo/issues/851 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch closed issue #848: Create FluoOracle command
mikewalch closed issue #848: Create FluoOracle command URL: https://github.com/apache/fluo/issues/848 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on issue #845: Create FluoInit command
mikewalch commented on issue #845: Create FluoInit command URL: https://github.com/apache/fluo/issues/845#issuecomment-321079261 Closed by #883 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on issue #851: Create FluoList command
mikewalch commented on issue #851: Create FluoList command URL: https://github.com/apache/fluo/issues/851#issuecomment-321079167 Closed by #883 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #902: fixes #894 merge SimpleConfiguration Objects
ctubbsii commented on a change in pull request #902: fixes #894 merge
SimpleConfiguration Objects
URL: https://github.com/apache/fluo/pull/902#discussion_r132011418
##
File path:
modules/api/src/main/java/org/apache/fluo/api/config/SimpleConfiguration.java
##
@@ -250,6 +277,22 @@ public SimpleConfiguration subset(String prefix) {
return new SimpleConfiguration(internalConfig.subset(prefix));
}
+ /**
+ * @param SimpleConfigs SimpleConfiguration's to be merged
+ * @return SimpleConfiguration
+ *
+ * @since 1.2.0
+ */
+ public static SimpleConfiguration merge(SimpleConfiguration...
simpleConfigs) {
+SimpleConfiguration mrg = new SimpleConfiguration();
+for (SimpleConfiguration sc : simpleConfigs) {
+ for (Map.Entry<String, String> entry : sc.toMap().entrySet()) {
+mrg.addProperty(entry.getKey(), entry.getValue());
Review comment:
@keith-turner I was thinking it would just hang off of SimpleConfiguration,
and produce a new SimpleConfiguration, like a linked list.
@cjmctague 'conjoin' probably isn't the best term to use. Also,
grammatically, I believe the preposition to use with 'conjoin' is 'with'
(mutual relationship) instead of 'to' (directional relationship), as in
'conjoinWith' instead of 'conjoinTo'.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] keith-turner commented on a change in pull request #77: update links in pages/ to https
keith-turner commented on a change in pull request #77: update links in pages/ to https URL: https://github.com/apache/fluo-website/pull/77#discussion_r131980892 ## File path: pages/people.md ## @@ -27,13 +27,13 @@ based on commits. | Name | Organization | Timezone | ---|-|| | [Eric Newton](https://github.com/ericnewton) | | [ET][et] | -| [Garvit Bansal](https://github.com/Garvit244)| [Flipkart](http://www.flipkart.com) | [IST][ist] | +| [Garvit Bansal](https://github.com/Garvit244)| [Flipkart](https://www.flipkart.com) | [IST][ist] | | [Michael Wall](https://github.com/mjwall)| | [ET][et] | [tc]: http://www.tetraconcepts.com/ Review comment: @cjnolet do you have any opinion on this? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjmctague commented on a change in pull request #902: fixes #894 merge SimpleConfiguration Objects
cjmctague commented on a change in pull request #902: fixes #894 merge
SimpleConfiguration Objects
URL: https://github.com/apache/fluo/pull/902#discussion_r131969482
##
File path:
modules/api/src/main/java/org/apache/fluo/api/config/SimpleConfiguration.java
##
@@ -250,6 +277,22 @@ public SimpleConfiguration subset(String prefix) {
return new SimpleConfiguration(internalConfig.subset(prefix));
}
+ /**
+ * @param SimpleConfigs SimpleConfiguration's to be merged
+ * @return SimpleConfiguration
+ *
+ * @since 1.2.0
+ */
+ public static SimpleConfiguration merge(SimpleConfiguration...
simpleConfigs) {
+SimpleConfiguration mrg = new SimpleConfiguration();
+for (SimpleConfiguration sc : simpleConfigs) {
+ for (Map.Entry<String, String> entry : sc.toMap().entrySet()) {
+mrg.addProperty(entry.getKey(), entry.getValue());
Review comment:
`conjoinTo(SimpleConfiguration)`
Means "join; combine"
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] keith-turner commented on a change in pull request #902: fixes #894 merge SimpleConfiguration Objects
keith-turner commented on a change in pull request #902: fixes #894 merge
SimpleConfiguration Objects
URL: https://github.com/apache/fluo/pull/902#discussion_r131937967
##
File path:
modules/api/src/main/java/org/apache/fluo/api/config/SimpleConfiguration.java
##
@@ -250,6 +277,22 @@ public SimpleConfiguration subset(String prefix) {
return new SimpleConfiguration(internalConfig.subset(prefix));
}
+ /**
+ * @param SimpleConfigs SimpleConfiguration's to be merged
+ * @return SimpleConfiguration
+ *
+ * @since 1.2.0
+ */
+ public static SimpleConfiguration merge(SimpleConfiguration...
simpleConfigs) {
+SimpleConfiguration mrg = new SimpleConfiguration();
+for (SimpleConfiguration sc : simpleConfigs) {
+ for (Map.Entry<String, String> entry : sc.toMap().entrySet()) {
+mrg.addProperty(entry.getKey(), entry.getValue());
Review comment:
> Instead of merge, maybe something like `defaultsTo(SimpleConfiguration)`,
or `orElse(SimpleConfiguration)`, or something like that?
@ctubbsii `orElse(SimpleConfiguration)` sounds interesting, however it
seems like a Fluent API which implies it would hang off of something. What
where you thinking it would hang off of?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] keith-turner commented on issue #900: Document properties in FluoConfiguration javadoc
keith-turner commented on issue #900: Document properties in FluoConfiguration javadoc URL: https://github.com/apache/fluo/issues/900#issuecomment-320976896 > @kpm1985 I will assign the task to you @kpm1985 Github is not allowing me to assign the issue to you, however its all yours. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on issue #900: Document properties in FluoConfiguration javadoc
mikewalch commented on issue #900: Document properties in FluoConfiguration javadoc URL: https://github.com/apache/fluo/issues/900#issuecomment-320965130 Thanks @kpm1985! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from?
mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from? URL: https://github.com/apache/fluo/pull/904#discussion_r131913642 ## File path: docs/applications.md ## @@ -141,12 +126,123 @@ To create an observer, follow these steps: 3. Build a jar containing these classes and include this jar in the `lib/` directory of your Fluo application. 4. Configure your Fluo instance to use this observer provider by modifying the Application section of Review comment: Should use `application` instead of `instance` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from?
mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from? URL: https://github.com/apache/fluo/pull/904#discussion_r131913848 ## File path: docs/applications.md ## @@ -141,12 +126,123 @@ To create an observer, follow these steps: 3. Build a jar containing these classes and include this jar in the `lib/` directory of your Fluo application. 4. Configure your Fluo instance to use this observer provider by modifying the Application section of -[fluo-app.properties]. +[fluo-app.properties]. Set `fluo.observer.provider` to the observer provider class name. 5. Initialize Fluo. During initialization Fluo will obtain the observed columns from the ObserverProvider and persist the columns in Zookeeper. These columns persisted in Zookeeper are used by transactions to know when to trigger observers. 6. Start your Fluo instance so that your Fluo workers load the new observer. Review comment: Could be `Start your Fluo application so that` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from?
mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from? URL: https://github.com/apache/fluo/pull/904#discussion_r131913735 ## File path: docs/applications.md ## @@ -141,12 +126,123 @@ To create an observer, follow these steps: 3. Build a jar containing these classes and include this jar in the `lib/` directory of your Fluo application. 4. Configure your Fluo instance to use this observer provider by modifying the Application section of -[fluo-app.properties]. +[fluo-app.properties]. Set `fluo.observer.provider` to the observer provider class name. 5. Initialize Fluo. During initialization Fluo will obtain the observed columns from the Review comment: Could be `Initialize your Fluo application` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mikewalch commented on a change in pull request #904: Fixes #897 added context to application docs and moved some docs from?
mikewalch commented on a change in pull request #904: Fixes #897 added context
to application docs and moved some docs from?
URL: https://github.com/apache/fluo/pull/904#discussion_r131912795
##
File path: docs/applications.md
##
@@ -70,31 +80,6 @@ FluoClient client = FluoFactory.newClient(config)
It may help to reference the [API javadocs][API] while you are learning the
Fluo API.
-## Running application code
-
-The `fluo exec {arguments}` provides an easy way to execute
application code. It
-will execute a class with a main method if a jar containing the class is
placed in the lib directory
-of the application. When the class is run, Fluo classes and dependencies will
be on the classpath.
-The `fluo exec` command can inject the applications configuration if the class
is written in the
-following way. Defining the injection point is optional.
-
-```java
-import javax.inject.Inject;
-
-public class AppCommand {
-
- //when run with fluo exec command, the applications configuration will be
injected
- @Inject
- private static FluoConfiguration fluoConfig;
-
- public static void main(String[] args) throws Exception {
-try(FluoClient fluoClient = FluoFactory.newClient(fluoConfig)) {
- //do stuff with Fluo
-}
- }
-}
-```
-
## Creating a Fluo observer
To create an observer, follow these steps:
Review comment:
Could be `To create an observer, follow these steps (which are described in
more detail in the sections below):`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] keith-turner commented on a change in pull request #1: Initial implementation of Apache Fluo YARN launcher
keith-turner commented on a change in pull request #1: Initial implementation
of Apache Fluo YARN launcher
URL: https://github.com/apache/fluo-yarn/pull/1#discussion_r131766475
##
File path: distribution/bin/fluo-yarn
##
@@ -0,0 +1,103 @@
+#! /usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
contributor license
+# agreements. See the NOTICE file distributed with this work for additional
information regarding
+# copyright ownership. The ASF licenses this file to you under the Apache
License, Version 2.0 (the
+# "License"); you may not use this file except in compliance with the License.
You may obtain a
+# copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
distributed under the License
+# is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express
+# or implied. See the License for the specific language governing permissions
and limitations under
+# the License.
+
+SOURCE="${BASH_SOURCE[0]}"
+while [ -h "$SOURCE" ]; do
+ bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
+ SOURCE="$(readlink "$SOURCE")"
+ [[ $SOURCE != /* ]] && SOURCE="$bin/$SOURCE"
+done
+# Set up variables needed by fluo-env.sh
+export bin="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
+export basedir="$( cd -P ${bin}/.. && pwd )"
+export conf="$basedir/conf"
+export lib="$basedir/lib"
+export cmd="$1"
+case "$cmd" in
+ start) export app="$2" ;;
+esac
+export FLUO_YARN_VERSION=${project.version}
+
+if [ ! -f "$conf/fluo-yarn-env.sh" ]; then
+ echo "fluo-yarn-env.sh must exist in $conf"
+ exit 1
+fi
+source "$conf/fluo-yarn-env.sh"
+export CLASSPATH
+
+# stop if any command fails
+set -e
+
+function print_usage {
+ echo -e "Usage: fluo-yarn ( ...)\n"
+ echo -e "Possible commands:\n"
+ echo " classpath Prints the classpath setup by
fluo-yarn-env.sh"
+ echo " start Starts Fluo in YARN using
"
+ echo " version Prints the version of Fluo YARN launcher"
+ echo " "
+ exit 1
+}
+
+function build_bundled_jar {
+ app_dir=$lib/apps/$app
+ mkdir -p "$app_dir/lib"
+
+ java org.apache.fluo.command.FluoGetJars "$FLUO_CONN_PROPS" "$app"
"$app_dir/lib"
Review comment:
What happens if this fails for some reason?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] keith-turner commented on issue #891: Migrate to checkstyle 8.1 rules
keith-turner commented on issue #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#issuecomment-320780109 > Updated log message. Let me know if that's sufficient. Looks great. The description is nice. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules
keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#discussion_r131761990 ## File path: src/main/resources/org/apache/fluo/resources/java-checkstyle.xml ## @@ -30,6 +28,8 @@ + Review comment: > I thought that's basically what my log message said, but I can reword it if it's not clear. Before I looked at the code changes, based on the commit message I was expecting this change to change a pointer to an externally versioned resource. When I looked at the code changes I was not sure what was what, not sure if a better log messages would have helped. I don't care if you change the message, just made that comment about the commit message based on my first impression. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules
keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#discussion_r131761990 ## File path: src/main/resources/org/apache/fluo/resources/java-checkstyle.xml ## @@ -30,6 +28,8 @@ + Review comment: > I thought that's basically what my log message said, but I can reword it if it's not clear. Before I looked at the code changes, based on the commit message I was expecting this change to change a pointer to an externally versioned resource. When I looked at the code changes I was not sure what was what, not sure if a better log messages would have helped. I don't care if you change the message, just made that comment about the log message based on my first impression. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on issue #891: Migrate to checkstyle 8.1 rules
ctubbsii commented on issue #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#issuecomment-320778952 Updated log message. Let me know if that's sufficient. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #891: Migrate to checkstyle 8.1 rules
ctubbsii commented on a change in pull request #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#discussion_r131760968 ## File path: src/main/resources/org/apache/fluo/resources/java-checkstyle.xml ## @@ -30,6 +28,8 @@ + Review comment: I thought that's basically what my log message said, but I can reword it if it's not clear. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules
keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#discussion_r131760336 ## File path: src/main/resources/org/apache/fluo/resources/java-checkstyle.xml ## @@ -30,6 +28,8 @@ + Review comment: > In the parent-pom Ok I think I understand. So, the change of version for com.puppycrawl.tools:checkstyle in 9789f9f9e0649522d86bb1afc92424d7178e9f16 necessitates this change? A commit message of `Modified rules to work with checkstyle 8.1` would be a bit more informative. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #891: Migrate to checkstyle 8.1 rules
ctubbsii commented on a change in pull request #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#discussion_r131754175 ## File path: src/main/resources/org/apache/fluo/resources/java-checkstyle.xml ## @@ -30,6 +28,8 @@ + Review comment: In the parent-pom, which I want to update to use the newer version of this build-resources, after the checkstyle rules are updated to work with said newer version. Updating this build-resources allows us to use these rules. Otherwise, we'll be blocked from updating that plugin because the rules won't work. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules
keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#discussion_r131753294 ## File path: src/main/resources/org/apache/fluo/resources/java-checkstyle.xml ## @@ -30,6 +28,8 @@ + Review comment: > It had to be moved in order to work with the newer version of Checkstyle. Where did the version of Checkstyle change? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #902: fixes #894 merge SimpleConfiguration Objects
ctubbsii commented on a change in pull request #902: fixes #894 merge
SimpleConfiguration Objects
URL: https://github.com/apache/fluo/pull/902#discussion_r131751494
##
File path:
modules/api/src/main/java/org/apache/fluo/api/config/SimpleConfiguration.java
##
@@ -250,6 +277,22 @@ public SimpleConfiguration subset(String prefix) {
return new SimpleConfiguration(internalConfig.subset(prefix));
}
+ /**
+ * @param SimpleConfigs SimpleConfiguration's to be merged
+ * @return SimpleConfiguration
+ *
+ * @since 1.2.0
+ */
+ public static SimpleConfiguration merge(SimpleConfiguration...
simpleConfigs) {
+SimpleConfiguration mrg = new SimpleConfiguration();
+for (SimpleConfiguration sc : simpleConfigs) {
+ for (Map.Entry<String, String> entry : sc.toMap().entrySet()) {
+mrg.addProperty(entry.getKey(), entry.getValue());
Review comment:
`commons-configuration` has this concept as a [`CompositeConfiguration`][1];
they don't have the name "merge", since the idea of composition is explicit.
They have multiple configs in the constructor, and an `addConfiguration` method.
Instead of `merge`, maybe something like `defaultsTo(SimpleConfiguration)`,
or `orElse(SimpleConfiguration)`, or something like that?
[1]:
https://commons.apache.org/proper/commons-configuration/apidocs/org/apache/commons/configuration2/CompositeConfiguration.html
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #902: fixes #894 merge SimpleConfiguration Objects
ctubbsii commented on a change in pull request #902: fixes #894 merge
SimpleConfiguration Objects
URL: https://github.com/apache/fluo/pull/902#discussion_r131751494
##
File path:
modules/api/src/main/java/org/apache/fluo/api/config/SimpleConfiguration.java
##
@@ -250,6 +277,22 @@ public SimpleConfiguration subset(String prefix) {
return new SimpleConfiguration(internalConfig.subset(prefix));
}
+ /**
+ * @param SimpleConfigs SimpleConfiguration's to be merged
+ * @return SimpleConfiguration
+ *
+ * @since 1.2.0
+ */
+ public static SimpleConfiguration merge(SimpleConfiguration...
simpleConfigs) {
+SimpleConfiguration mrg = new SimpleConfiguration();
+for (SimpleConfiguration sc : simpleConfigs) {
+ for (Map.Entry<String, String> entry : sc.toMap().entrySet()) {
+mrg.addProperty(entry.getKey(), entry.getValue());
Review comment:
commons-configuration has this concept as a `[CompositeConfiguration][1]`;
they don't have the name "merge", since the idea of composition is explicit.
They have multiple configs in the constructor, and an `addConfiguration` method.
Instead of `merge`, maybe something like `defaultsTo(SimpleConfiguration)`,
or `orElse(SimpleConfiguration)`, or something like that?
[1]:
https://commons.apache.org/proper/commons-configuration/apidocs/org/apache/commons/configuration2/CompositeConfiguration.html
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ctubbsii commented on a change in pull request #891: Migrate to checkstyle 8.1 rules
ctubbsii commented on a change in pull request #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#discussion_r131749951 ## File path: src/main/resources/org/apache/fluo/resources/java-checkstyle.xml ## @@ -30,6 +28,8 @@ + Review comment: Checkstyle has different rules for processing Java files. Some just do their own thing, others inherit from some parent behavior, allowing them to perform their checks inside a particular context. I don't know the details of this module, or exactly how the TreeWalker works, but at some point in the newer version, this rule which we had been using was changed so that it only works within the TreeWalker. It had to be moved in order to work with the newer version of Checkstyle. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules
keith-turner commented on a change in pull request #891: Migrate to checkstyle 8.1 rules URL: https://github.com/apache/fluo/pull/891#discussion_r131748791 ## File path: src/main/resources/org/apache/fluo/resources/java-checkstyle.xml ## @@ -30,6 +28,8 @@ + Review comment: Can you provide a bit of context for this? I don't understand these changes. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on a change in pull request #903: Fluo 897
keith-turner commented on a change in pull request #903: Fluo 897 URL: https://github.com/apache/fluo/pull/903#discussion_r131735469 ## File path: docs/applications.md ## @@ -147,6 +128,102 @@ To create an observer, follow these steps: are used by transactions to know when to trigger observers. 6. Start your Fluo instance so that your Fluo workers load the new observer. +## Initialize Fluo application + +Below is an overview of what initialization does and some of the key properties it uses. + + * **Initialize ZooKeeper** : Each application has its own area in ZooKeeper that is used for application configuration, Oracle state, and worker coordination. All properties, except `fluo.connections.*`, are copied into ZooKeeper. For example, if `fluo.worker.num.threads=128` was set, then when a worker process starts it will read this from ZooKeeper. + * **Copy Observer jars to DFS** : Fluo workers processes need the jars containing observers. These are provided in one of the following ways. + * Set the property `fluo.observer.init.dir` to a local directory containing observer jars. The jars in this directory are copied to DFS under `/`. When a worker is started, the jars are pulled from DFS and added to its classpath. + * Set the property `fluo.observer.jars.url` to a directory in DFS containing observer jars. No copying is done. When a worker is started, the jars are pulled from this location and added to its classpath. + * Do not set any of the properties above and have the mechanism that starts the worker process add the needed jars to the classpath. + * **Create Accumulo table** : Each Fluo application creates and configures an Accumulo table. The `fluo.accumulo.*` properties determine which Accumulo instance is used. For performance reasons, Fluo runs its own code in Accumulo tablet servers. Fluo attempts to copy Fluo jars into DFS and configure Accumulo to use them. Fluo first checks the property `fluo.accumulo.jars` and if set, copies the jars listed there. If that property is not set, then Fluo looks on the classpath to find jars. Jars are copied to a location under `/`. + +Below are the steps to initialize an appliaction from the command line. It is also possible to initialize an appliaction using Fluo's Java API. Review comment: fixed that, thanks This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] keith-turner commented on issue #903: Fluo 897
keith-turner commented on issue #903: Fluo 897 URL: https://github.com/apache/fluo/pull/903#issuecomment-320749031 I am still working on this and accidentally submitted it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] cjmctague commented on a change in pull request #903: Fluo 897
cjmctague commented on a change in pull request #903: Fluo 897 URL: https://github.com/apache/fluo/pull/903#discussion_r131734286 ## File path: docs/applications.md ## @@ -147,6 +128,102 @@ To create an observer, follow these steps: are used by transactions to know when to trigger observers. 6. Start your Fluo instance so that your Fluo workers load the new observer. +## Initialize Fluo application + +Below is an overview of what initialization does and some of the key properties it uses. + + * **Initialize ZooKeeper** : Each application has its own area in ZooKeeper that is used for application configuration, Oracle state, and worker coordination. All properties, except `fluo.connections.*`, are copied into ZooKeeper. For example, if `fluo.worker.num.threads=128` was set, then when a worker process starts it will read this from ZooKeeper. + * **Copy Observer jars to DFS** : Fluo workers processes need the jars containing observers. These are provided in one of the following ways. + * Set the property `fluo.observer.init.dir` to a local directory containing observer jars. The jars in this directory are copied to DFS under `/`. When a worker is started, the jars are pulled from DFS and added to its classpath. + * Set the property `fluo.observer.jars.url` to a directory in DFS containing observer jars. No copying is done. When a worker is started, the jars are pulled from this location and added to its classpath. + * Do not set any of the properties above and have the mechanism that starts the worker process add the needed jars to the classpath. + * **Create Accumulo table** : Each Fluo application creates and configures an Accumulo table. The `fluo.accumulo.*` properties determine which Accumulo instance is used. For performance reasons, Fluo runs its own code in Accumulo tablet servers. Fluo attempts to copy Fluo jars into DFS and configure Accumulo to use them. Fluo first checks the property `fluo.accumulo.jars` and if set, copies the jars listed there. If that property is not set, then Fluo looks on the classpath to find jars. Jars are copied to a location under `/`. + +Below are the steps to initialize an appliaction from the command line. It is also possible to initialize an appliaction using Fluo's Java API. Review comment: Application s.p This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
