[jira] [Commented] (ACCUMULO-3364) Inconsistency between java and native map update within single mutation
[
https://issues.apache.org/jira/browse/ACCUMULO-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224681#comment-14224681
]
Christopher Tubbs commented on ACCUMULO-3364:
-
This issue seems part of a whole set of related issues, where we don't handle
identical keys well.
> Inconsistency between java and native map update within single mutation
> ---
>
> Key: ACCUMULO-3364
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3364
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
>Affects Versions: 1.6.1
>Reporter: Josh Elser
>Priority: Critical
>
> Noticed the following when I was poking around with
> {{ExamplesIT#testStatsCombiner}}:
> {code}
> String table = getUniqueNames(1)[0];
> c.tableOperations().create(table);
> is = new IteratorSetting(10, StatsCombiner.class);
> StatsCombiner.setCombineAllColumns(is, true);
> c.tableOperations().attachIterator(table, is);
> bw = c.createBatchWriter(table, bwc);
> Mutation m = new Mutation("foo");
> m.put("a", "b", "1");
> m.put("a", "b", "3");
> bw.addMutation(m);
> bw.flush();
> Iterator> iter = c.createScanner(table,
> Authorizations.EMPTY).iterator();
> assertTrue("Iterator had no results", iter.hasNext());
> Entry e = iter.next();
> assertEquals("Results ", "1,3,4,2", e.getValue().toString());
> {code}
> When run against the Java maps, the above succeeds as the test expects it to.
> However, this fails when the instance is using native maps.
> Talked to [~kturner] about this, and he noted that the Java maps [use a
> one-up counter when performing
> updates|https://github.com/apache/accumulo/blob/1.6.1/server/tserver/src/main/java/org/apache/accumulo/tserver/InMemoryMap.java#L404]
> whereas it does not appear that the NativeMap does this.
> The reason I got stuck on it is because I expected to see both updates
> because the StatsCombiner was running below the VersioningIterator, but I
> would only see a single Key-Value out of the scanner (a value of "3,3,3,1").
> My gut reaction is that we should have the same semantics across both Java
> and Native maps, although I'm not sure which one is correct/expected yet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-3364) Inconsistency between java and native map update within single mutation
[
https://issues.apache.org/jira/browse/ACCUMULO-3364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224740#comment-14224740
]
Josh Elser commented on ACCUMULO-3364:
--
Very likely so. Thanks for linking the extra issues.
> Inconsistency between java and native map update within single mutation
> ---
>
> Key: ACCUMULO-3364
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3364
> Project: Accumulo
> Issue Type: Bug
> Components: tserver
>Affects Versions: 1.6.1
>Reporter: Josh Elser
>Priority: Critical
>
> Noticed the following when I was poking around with
> {{ExamplesIT#testStatsCombiner}}:
> {code}
> String table = getUniqueNames(1)[0];
> c.tableOperations().create(table);
> is = new IteratorSetting(10, StatsCombiner.class);
> StatsCombiner.setCombineAllColumns(is, true);
> c.tableOperations().attachIterator(table, is);
> bw = c.createBatchWriter(table, bwc);
> Mutation m = new Mutation("foo");
> m.put("a", "b", "1");
> m.put("a", "b", "3");
> bw.addMutation(m);
> bw.flush();
> Iterator> iter = c.createScanner(table,
> Authorizations.EMPTY).iterator();
> assertTrue("Iterator had no results", iter.hasNext());
> Entry e = iter.next();
> assertEquals("Results ", "1,3,4,2", e.getValue().toString());
> {code}
> When run against the Java maps, the above succeeds as the test expects it to.
> However, this fails when the instance is using native maps.
> Talked to [~kturner] about this, and he noted that the Java maps [use a
> one-up counter when performing
> updates|https://github.com/apache/accumulo/blob/1.6.1/server/tserver/src/main/java/org/apache/accumulo/tserver/InMemoryMap.java#L404]
> whereas it does not appear that the NativeMap does this.
> The reason I got stuck on it is because I expected to see both updates
> because the StatsCombiner was running below the VersioningIterator, but I
> would only see a single Key-Value out of the scanner (a value of "3,3,3,1").
> My gut reaction is that we should have the same semantics across both Java
> and Native maps, although I'm not sure which one is correct/expected yet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-3176) Add ability to create a table with user specified initial properties
[ https://issues.apache.org/jira/browse/ACCUMULO-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224763#comment-14224763 ] Christopher Tubbs commented on ACCUMULO-3176: - There are 3 +1s for this patch (including mine and those on the review), which improves the table creation API to make it possible to set initial properties at table creation. I don't see any objections, other than [~busbey]'s suggestions for alternate solutions to some use cases, which this change does not preclude. As such, it is my intention to rebase and apply this patch today to 1.7, since it has been idle for more than a month. Objections should be represented as a formal veto, and brought to a vote, in accordance with our bylaws. I'm also going to move it to a top-level issue, because, while it was conceived of in the context of ACCUMULO-3089 and might benefit it, it is a completely independent improvement. The proposals for other improvements, represented by some of the above comments, should be made into their own issues if anybody intends to pursue, or consider pursuing, those proposals. > Add ability to create a table with user specified initial properties > > > Key: ACCUMULO-3176 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3176 > Project: Accumulo > Issue Type: Sub-task >Reporter: Jenna Huston >Assignee: Jenna Huston > > This change would allow for table properties to be set before the default > tablet is created. Instead of just adding a new create method, a > NewTableConfiguration class could be created and passed and the other create > methods deprecated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ACCUMULO-3176) Add ability to create a table with user specified initial properties
[ https://issues.apache.org/jira/browse/ACCUMULO-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-3176: Issue Type: Improvement (was: Sub-task) Parent: (was: ACCUMULO-3089) > Add ability to create a table with user specified initial properties > > > Key: ACCUMULO-3176 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3176 > Project: Accumulo > Issue Type: Improvement >Reporter: Jenna Huston >Assignee: Jenna Huston > > This change would allow for table properties to be set before the default > tablet is created. Instead of just adding a new create method, a > NewTableConfiguration class could be created and passed and the other create > methods deprecated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-3176) Add ability to create a table with user specified initial properties
[ https://issues.apache.org/jira/browse/ACCUMULO-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224814#comment-14224814 ] Sean Busbey commented on ACCUMULO-3176: --- [~ctubbsii], my objection to this change and intention to veto was [clearly stated|http://s.apache.org/4c8] on ACCUMULO-3089. If we are going to need a formal vote, please do not push the patch until after we go through the necessary consensus vote. The bylaws are ambiguous about what a "code change" is for veto (e.g. either this patch as it is here / rb or that patch once it is in the git repository). I ask that you not exploit this loophole to land code before attempting to bypass my veto. > Add ability to create a table with user specified initial properties > > > Key: ACCUMULO-3176 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3176 > Project: Accumulo > Issue Type: Improvement >Reporter: Jenna Huston >Assignee: Jenna Huston > > This change would allow for table properties to be set before the default > tablet is created. Instead of just adding a new create method, a > NewTableConfiguration class could be created and passed and the other create > methods deprecated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-3176) Add ability to create a table with user specified initial properties
[ https://issues.apache.org/jira/browse/ACCUMULO-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224861#comment-14224861 ] Eric Newton commented on ACCUMULO-3176: --- [~busbey] call a vote to either not commit or revert the change. > Add ability to create a table with user specified initial properties > > > Key: ACCUMULO-3176 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3176 > Project: Accumulo > Issue Type: Improvement >Reporter: Jenna Huston >Assignee: Jenna Huston > > This change would allow for table properties to be set before the default > tablet is created. Instead of just adding a new create method, a > NewTableConfiguration class could be created and passed and the other create > methods deprecated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-3176) Add ability to create a table with user specified initial properties
[ https://issues.apache.org/jira/browse/ACCUMULO-3176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224875#comment-14224875 ] Christopher Tubbs commented on ACCUMULO-3176: - I initiated a vote to the dev@ list. > Add ability to create a table with user specified initial properties > > > Key: ACCUMULO-3176 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3176 > Project: Accumulo > Issue Type: Improvement >Reporter: Jenna Huston >Assignee: Jenna Huston > > This change would allow for table properties to be set before the default > tablet is created. Instead of just adding a new create method, a > NewTableConfiguration class could be created and passed and the other create > methods deprecated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ACCUMULO-1817) Create a monitoring bridge similar to Hadoop's GangliaContext that can allow easy pluggable support
[ https://issues.apache.org/jira/browse/ACCUMULO-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated ACCUMULO-1817: - Assignee: Billie Rinaldi > Create a monitoring bridge similar to Hadoop's GangliaContext that can allow > easy pluggable support > --- > > Key: ACCUMULO-1817 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1817 > Project: Accumulo > Issue Type: New Feature >Reporter: Corey J. Nolet >Assignee: Billie Rinaldi >Priority: Trivial > Labels: proposed > Fix For: 1.7.0 > > > We currently expose JMX and it's possible (with external code) to bridge the > JMX to solutions like Ganglia. It would be ideal if the integration were > native and pluggable. > Turns out that Hadoop (hdfs, mapred) and HBase has "direct" metrics reporting > to Ganglia through some nice code provided in Hadoop. > Look into the GangliaContext to see if we can implement Ganglia metrics > reporting by Accumulo configuration alone. > References: http://wiki.apache.org/hadoop/GangliaMetrics, > http://hbase.apache.org/metrics.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-1817) Create a monitoring bridge similar to Hadoop's GangliaContext that can allow easy pluggable support
[ https://issues.apache.org/jira/browse/ACCUMULO-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224944#comment-14224944 ] Josh Elser commented on ACCUMULO-1817: -- [~billie.rinaldi] and I have started looking at porting things over. > Create a monitoring bridge similar to Hadoop's GangliaContext that can allow > easy pluggable support > --- > > Key: ACCUMULO-1817 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1817 > Project: Accumulo > Issue Type: New Feature >Reporter: Corey J. Nolet >Assignee: Billie Rinaldi >Priority: Trivial > Labels: proposed > Fix For: 1.7.0 > > > We currently expose JMX and it's possible (with external code) to bridge the > JMX to solutions like Ganglia. It would be ideal if the integration were > native and pluggable. > Turns out that Hadoop (hdfs, mapred) and HBase has "direct" metrics reporting > to Ganglia through some nice code provided in Hadoop. > Look into the GangliaContext to see if we can implement Ganglia metrics > reporting by Accumulo configuration alone. > References: http://wiki.apache.org/hadoop/GangliaMetrics, > http://hbase.apache.org/metrics.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ACCUMULO-1817) Create a monitoring bridge similar to Hadoop's GangliaContext that can allow easy pluggable support
[ https://issues.apache.org/jira/browse/ACCUMULO-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated ACCUMULO-1817: - Priority: Major (was: Trivial) > Create a monitoring bridge similar to Hadoop's GangliaContext that can allow > easy pluggable support > --- > > Key: ACCUMULO-1817 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1817 > Project: Accumulo > Issue Type: New Feature >Reporter: Corey J. Nolet >Assignee: Billie Rinaldi > Labels: proposed > Fix For: 1.7.0 > > > We currently expose JMX and it's possible (with external code) to bridge the > JMX to solutions like Ganglia. It would be ideal if the integration were > native and pluggable. > Turns out that Hadoop (hdfs, mapred) and HBase has "direct" metrics reporting > to Ganglia through some nice code provided in Hadoop. > Look into the GangliaContext to see if we can implement Ganglia metrics > reporting by Accumulo configuration alone. > References: http://wiki.apache.org/hadoop/GangliaMetrics, > http://hbase.apache.org/metrics.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ACCUMULO-1817) Create a monitoring bridge similar to Hadoop's GangliaContext that can allow easy pluggable support
[ https://issues.apache.org/jira/browse/ACCUMULO-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224944#comment-14224944 ] Josh Elser edited comment on ACCUMULO-1817 at 11/25/14 6:24 PM: [~billie.rinaldi] and I have started looking at porting things over to metrics2. was (Author: elserj): [~billie.rinaldi] and I have started looking at porting things over. > Create a monitoring bridge similar to Hadoop's GangliaContext that can allow > easy pluggable support > --- > > Key: ACCUMULO-1817 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1817 > Project: Accumulo > Issue Type: New Feature >Reporter: Corey J. Nolet >Assignee: Billie Rinaldi >Priority: Trivial > Labels: proposed > Fix For: 1.7.0 > > > We currently expose JMX and it's possible (with external code) to bridge the > JMX to solutions like Ganglia. It would be ideal if the integration were > native and pluggable. > Turns out that Hadoop (hdfs, mapred) and HBase has "direct" metrics reporting > to Ganglia through some nice code provided in Hadoop. > Look into the GangliaContext to see if we can implement Ganglia metrics > reporting by Accumulo configuration alone. > References: http://wiki.apache.org/hadoop/GangliaMetrics, > http://hbase.apache.org/metrics.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-1817) Create a monitoring bridge similar to Hadoop's GangliaContext that can allow easy pluggable support
[ https://issues.apache.org/jira/browse/ACCUMULO-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224975#comment-14224975 ] Corey Nolet commented on ACCUMULO-1817: --- Awesome! Given that people have been using the jmx wrapper for ganglia metrics, I didn't know if this was still going to be important for others. I can say it's still something I would like to see. > Create a monitoring bridge similar to Hadoop's GangliaContext that can allow > easy pluggable support > --- > > Key: ACCUMULO-1817 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1817 > Project: Accumulo > Issue Type: New Feature >Reporter: Corey J. Nolet >Assignee: Billie Rinaldi > Labels: proposed > Fix For: 1.7.0 > > > We currently expose JMX and it's possible (with external code) to bridge the > JMX to solutions like Ganglia. It would be ideal if the integration were > native and pluggable. > Turns out that Hadoop (hdfs, mapred) and HBase has "direct" metrics reporting > to Ganglia through some nice code provided in Hadoop. > Look into the GangliaContext to see if we can implement Ganglia metrics > reporting by Accumulo configuration alone. > References: http://wiki.apache.org/hadoop/GangliaMetrics, > http://hbase.apache.org/metrics.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-1817) Create a monitoring bridge similar to Hadoop's GangliaContext that can allow easy pluggable support
[ https://issues.apache.org/jira/browse/ACCUMULO-1817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225063#comment-14225063 ] Josh Elser commented on ACCUMULO-1817: -- In practice, it actually helps simplify a bit of our code, notably, removing the custom JMX/MBean registration code we have. The extra perk is that we also get all of the nice Sinks that come with metrics2. > Create a monitoring bridge similar to Hadoop's GangliaContext that can allow > easy pluggable support > --- > > Key: ACCUMULO-1817 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1817 > Project: Accumulo > Issue Type: New Feature >Reporter: Corey J. Nolet >Assignee: Billie Rinaldi > Labels: proposed > Fix For: 1.7.0 > > > We currently expose JMX and it's possible (with external code) to bridge the > JMX to solutions like Ganglia. It would be ideal if the integration were > native and pluggable. > Turns out that Hadoop (hdfs, mapred) and HBase has "direct" metrics reporting > to Ganglia through some nice code provided in Hadoop. > Look into the GangliaContext to see if we can implement Ganglia metrics > reporting by Accumulo configuration alone. > References: http://wiki.apache.org/hadoop/GangliaMetrics, > http://hbase.apache.org/metrics.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-1904) Add ability to create table in offline state
[
https://issues.apache.org/jira/browse/ACCUMULO-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225088#comment-14225088
]
Christopher Tubbs commented on ACCUMULO-1904:
-
This could be implemented in the API with the {{NewTableConfiguration}}
proposed in the patch for ACCUMULO-3176.
> Add ability to create table in offline state
>
>
> Key: ACCUMULO-1904
> URL: https://issues.apache.org/jira/browse/ACCUMULO-1904
> Project: Accumulo
> Issue Type: Improvement
> Components: client
>Reporter: Christopher Tubbs
>Assignee: Vicky Kak
> Fix For: 1.7.0
>
>
> Having the ability to create tables in the offline state means we have the
> ability to manipulate its initial configuration before it comes online. This
> would allow us to reduce the number of convenience methods for creating
> tables, and would simplify the API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ACCUMULO-3366) Ensure ITs can run against cluster with SSL
Josh Elser created ACCUMULO-3366: Summary: Ensure ITs can run against cluster with SSL Key: ACCUMULO-3366 URL: https://issues.apache.org/jira/browse/ACCUMULO-3366 Project: Accumulo Issue Type: Improvement Components: test Reporter: Josh Elser Assignee: Josh Elser Fix For: 1.6.2, 1.7.0 Need to make sure that there isn't anything extra that needs to be implemented to run the ITs against an existing cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ACCUMULO-3367) Lift alter-and-reset-configuration paradigm in common location
Josh Elser created ACCUMULO-3367: Summary: Lift alter-and-reset-configuration paradigm in common location Key: ACCUMULO-3367 URL: https://issues.apache.org/jira/browse/ACCUMULO-3367 Project: Accumulo Issue Type: Improvement Components: test Reporter: Josh Elser Assignee: Josh Elser Fix For: 1.6.2, 1.7.0 In a number of tests, they alter the system configuration, restart a server if necessary, run the test, and then return the configuration back to its initial state. This can be encapsulated into its own method, likely in AccumuloClusterIT, which would reduce repetition across the tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ACCUMULO-2806) Accumulo init should ensure wals and tables are not world readable
[
https://issues.apache.org/jira/browse/ACCUMULO-2806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey reassigned ACCUMULO-2806:
-
Assignee: (was: Sean Busbey)
> Accumulo init should ensure wals and tables are not world readable
> --
>
> Key: ACCUMULO-2806
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2806
> Project: Accumulo
> Issue Type: Bug
>Affects Versions: 1.5.0, 1.5.1, 1.6.0
>Reporter: Sean Busbey
>Priority: Critical
> Fix For: 1.5.3, 1.6.2, 1.7.0
>
>
> Just did an init on a new 1.6.1-SNAP cluster, and noticed the following
> permissions:
> {noformat}
> dfs -ls /
> Found 4 items
> drwxr-xr-x - accumulo supergroup 0 2014-05-14 09:48 /accumulo
> drwxr-xr-x - hdfs supergroup 0 2014-05-14 08:10 /jobtracker
> drwxrwxrwx - hdfs supergroup 0 2014-05-14 08:10 /tmp
> drwxr-xr-x - hdfs supergroup 0 2014-05-14 09:48 /user
> -bash-4.1$ hdfs dfs -ls /accumulo
> Found 3 items
> drwxr-xr-x - accumulo supergroup 0 2014-05-14 09:55
> /accumulo/instance_id
> drwxr-xr-x - accumulo supergroup 0 2014-05-14 09:55
> /accumulo/tables
> drwxr-xr-x - accumulo supergroup 0 2014-05-14 09:55
> /accumulo/version
> {noformat}
> I previously set up /accumulo as 755, under the understanding that clients
> need access to /accumulo/instance_id
> things to fix
> # make init chmod tables and wals to 700, as a defensive measure to avoid
> data leaks
> # maybe also make sure if the trash is enabled that our user directory is
> also not world readable
> # If clients don't need access to instance_id, include a check that the data
> dir is not world readable
> Workaround: manually change permissions after init
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ACCUMULO-3252) SslITs OOME tserver on master
[ https://issues.apache.org/jira/browse/ACCUMULO-3252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Tubbs updated ACCUMULO-3252: Resolution: Fixed Status: Resolved (was: Patch Available) Fixed by patch in ACCUMULO-3199 > SslITs OOME tserver on master > - > > Key: ACCUMULO-3252 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3252 > Project: Accumulo > Issue Type: Bug >Reporter: Josh Elser >Assignee: Christopher Tubbs >Priority: Blocker > Fix For: 1.7.0 > > Attachments: ssl.diff > > Time Spent: 10m > Remaining Estimate: 0h > > I think something is broken with SSL in 1.7. For a week and a half or so, > I've been seeing SslIT and SslWithClientAuthIT fail occasionally because a > server process failed due to an OOME. > I haven't seen it on 1.6 and the brief poking around and bisect has not been > fruitful in actually reproducing the issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ACCUMULO-3199) RPC options in client configuration are passed to the RPC layer poorly
[
https://issues.apache.org/jira/browse/ACCUMULO-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christopher Tubbs updated ACCUMULO-3199:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> RPC options in client configuration are passed to the RPC layer poorly
> --
>
> Key: ACCUMULO-3199
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3199
> Project: Accumulo
> Issue Type: Task
> Components: client
>Reporter: Christopher Tubbs
>Assignee: Christopher Tubbs
> Fix For: 1.7.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The client configuration options for SSL, timeout, and (possibly) other
> RPC-related options are wrapped into an AccumuloConfiguration object to pass
> down to the RPC layer. This is done in a bit of a convoluted way, by
> leveraging the deprecated {{Instance.getConfiguration()}} method and relying
> on the fact that the {{Instance}} object is already passed through, instead
> of passing it through the {{Connector}} and internally to the the RPC layer
> where it's needed.
> The problem with this is that only the {{ZooKeeperInstance}} holds client
> configuration for this purpose, and simply calling
> {{instance.getConfiguration()}} will not always work as expected if you don't
> know what kind of {{Instance}} object you have. Further, this configuration
> object is occasionally incorrectly used in server code to read the
> server-side site configuration, when {{new
> ServerConfigurationFactory(instance).getConfiguration()}} should be used
> instead. This can lead to unexpected behavior if you are relying on the
> {{HdfsZooInstance}} implementation to provide the site configuration, but
> instead you have a different {{Instance}}, because all you'll get back is the
> {{DefaultConfiguration}}.
> The code that uses {{Instance.getConfiguration()}} needs to be changed, and
> the RPC-related configuration needs to be passed more explicitly through to
> the RPC layer, so we are less prone to bugs which make assumptions about
> which configuration is being retrieved with the deprecated API.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ACCUMULO-3178) Create example preferred volumes chooser
[ https://issues.apache.org/jira/browse/ACCUMULO-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225327#comment-14225327 ] Christopher Tubbs commented on ACCUMULO-3178: - [~busbey] Were you going to review this or ACCUMULO-3177 at all? > Create example preferred volumes chooser > > > Key: ACCUMULO-3178 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3178 > Project: Accumulo > Issue Type: Sub-task >Reporter: Jenna Huston >Assignee: Jenna Huston > > This example VolumeChooser chooses a volume from a user specified list of > preferred volumes for that specific table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-3177) Create a per table volume chooser
[
https://issues.apache.org/jira/browse/ACCUMULO-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225340#comment-14225340
]
Christopher Tubbs commented on ACCUMULO-3177:
-
[~hustjl22]: Can you rebase this patch onto the latest master? In particular, I
think some of the calls to the choose method violate the goal of ACCUMULO-3181
and should be updated (some of the use of {{ServerConstants.get\*()}}
convenience methods seem to be forcing the chooser to choose paths, not volumes
again). Additionally, I think some more problematic conflicts were introduced
in {{Initialize.java}} since the last time the patch was updated, a few more
{{choose}} methods exist, some substituting {{"/"}} with {{Path.SEPARATOR}},
and maybe a few more changes.
Thanks for the work on this! Your contributions are appreciated.
> Create a per table volume chooser
> -
>
> Key: ACCUMULO-3177
> URL: https://issues.apache.org/jira/browse/ACCUMULO-3177
> Project: Accumulo
> Issue Type: Sub-task
>Reporter: Jenna Huston
>Assignee: Jenna Huston
>
> This chooser would decide on a per table basis what VolumeChooser to use when
> creating new tablets.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ACCUMULO-3368) Port SimpleMacIT and ConfigurableMacIT that only exist in master
Josh Elser created ACCUMULO-3368: Summary: Port SimpleMacIT and ConfigurableMacIT that only exist in master Key: ACCUMULO-3368 URL: https://issues.apache.org/jira/browse/ACCUMULO-3368 Project: Accumulo Issue Type: Task Components: test Reporter: Josh Elser Assignee: Josh Elser Fix For: 1.7.0 There are a few new ITs in master. Port them to the new IT infrasturcture if relevant. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ACCUMULO-3368) Port SimpleMacIT and ConfigurableMacIT that only exist in master
[ https://issues.apache.org/jira/browse/ACCUMULO-3368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14225489#comment-14225489 ] Josh Elser commented on ACCUMULO-3368: -- * DeletedTablesDontFlushIT * ArbitraryTablePropertiesIT * SplitCancelsMajCIT * StatusCombinerMacIT > Port SimpleMacIT and ConfigurableMacIT that only exist in master > > > Key: ACCUMULO-3368 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3368 > Project: Accumulo > Issue Type: Task > Components: test >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 1.7.0 > > > There are a few new ITs in master. Port them to the new IT infrasturcture if > relevant. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ACCUMULO-3167) Decouple MiniAccumuloCluster from integration test base class
[ https://issues.apache.org/jira/browse/ACCUMULO-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser resolved ACCUMULO-3167. -- Resolution: Fixed Submitted -- a couple of follow on tasks already identified. > Decouple MiniAccumuloCluster from integration test base class > - > > Key: ACCUMULO-3167 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3167 > Project: Accumulo > Issue Type: Improvement > Components: mini, test >Reporter: Josh Elser >Assignee: Josh Elser > Fix For: 1.6.2, 1.7.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > (apologies if I already had a ticket for this somewhere, I couldn't find it) > Our integration tests are very nice and automated at the moment because we > can use MiniAccumuloCluster to "provision" an Accumulo instance (or used a > shared instance), and run a test against it. For the most part, this works > well and provides an accurate test harness. > Thus, to run integration tests, you need a sufficiently beefy machine since > the same host will be running all of Accumulo as well as performing any > client work. When resources are available to use, it would be nice to > leverage them -- whether these are yarn, mesos, a vanila installation, etc. > In addition to the additional computational power from using extra hardware, > it also encourages us to use the public API as much as possible instead of > relying on "hidden" impl methods in MiniAccumuloCluster. > I propose making changes to the IT test base (AbstractMacIT, SimpleMacIT, > ConfigurableMacIT) to add an extra step between them an test classes to allow > "injection" of a more generic Accumulo "cluster" that is not associated with > MAC. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
