[jira] [Commented] (RYA-94) Remove from secondary indexes when deleting from DAO
[ https://issues.apache.org/jira/browse/RYA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427389#comment-15427389 ] ASF GitHub Bot commented on RYA-94: --- Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/73 hey david, Two things: 1. Merge to develop not master 2. Mark the Pull request as WIP if its not ready to be merged > Remove from secondary indexes when deleting from DAO > > > Key: RYA-94 > URL: https://issues.apache.org/jira/browse/RYA-94 > Project: Rya > Issue Type: Bug >Reporter: Andrew Smith >Assignee: David W. Lotts >Priority: Blocker > > Currently, if a delete is made from the DAO, the secondary indexers are not > explicitly notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya issue #73: RYA-94 Remove from secondary indexes when deleting ...
Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/73 hey david, Two things: 1. Merge to develop not master 2. Mark the Pull request as WIP if its not ready to be merged --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-rya pull request #73: Rya 94 rm index
GitHub user DLotts opened a pull request: https://github.com/apache/incubator-rya/pull/73 Rya 94 rm index Not ready for review yet. More commits to come. You can merge this pull request into a Git repository by running: $ git pull https://github.com/DLotts/incubator-rya RYA-94-rm-index Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/73.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #73 commit d6635321e4dcd6a29c15e850a065c74f2d946bf9 Author: pujav65 Date: 2015-12-18T15:52:18Z RYA-24 issue with hashes and regex queries commit 8df9ac7680dc26265cc41aa2d3193398660f196a Author: pujav65 Date: 2015-12-21T14:10:10Z RYA-23 exposing triple pattern strategy from rya context commit faae178c004c2e96b216192b8ee76892b671d44b Author: pujav65 Date: 2015-12-21T15:12:30Z RYA-21 adding input format and sample mappers/reducers commit 2b30478cdbd0207bfd48d2e2c10c26438fb9b353 Author: Aaron Mihalik Date: 2015-12-23T16:23:31Z RYA-25 Adding Apache-RAT Eclipse Exclusion commit e530aec63ea99f601ccaa6431029023a5325c94a Author: pujav65 Date: 2016-01-04T17:50:43Z Merge branch 'RYA-24' of https://github.com/pujav65/incubator-rya into RYA-24 commit a7d266ea343742921bc5b30ecc276f6297aaf32a Author: pujav65 Date: 2016-01-04T18:19:30Z Merge branch 'RYA-23' of https://github.com/pujav65/incubator-rya into RYA-23 commit 4ca2b241f6dd3394e99f7f49476d6216b400994a Author: pujav65 Date: 2016-01-04T19:37:47Z Merge branch 'RYA-21' of https://github.com/pujav65/incubator-rya into RYA-21 commit db96c4584815fec6b67f31db2753d12c1bb23f64 Author: pujav65 Date: 2016-01-04T21:31:24Z RYA-28 adding additional life cycle dependencies for secondary indices commit 4b7bd4f4ecc7a2d2e074ef6394452145138ebbca Author: pujav65 Date: 2016-01-06T19:01:40Z RYA-28 fixing build after incomplete check in commit 22f82c7abdb53cd16a6c6f77cf605cc8fae5de7e Author: Aaron Mihalik Date: 2015-11-22T22:50:19Z RYA-4 Adding Vagrant Example and Documentation commit c12f58f46be0aed3c88eff7504d4bfd4e4f295e9 Author: Caleb Meier Date: 2016-01-29T16:40:59Z RYA-32 Improve how metadata and values are written to Accumulo PCJ tables commit 6cd8aeb71ee28252023965d2acbffc19eb32296b Author: Aaron Mihalik Date: 2016-02-07T18:24:38Z RYA-32 Adding License Headers commit 19e2e438c9e6aef2585d72a5696b336b7bd7ab8b Author: Aaron Mihalik Date: 2016-02-07T19:15:27Z RYA-4 Adding Vagrant Example and Documentation commit 3157bc466a4faf87814ad9c81166c6f2139fc49c Author: Kevin Chilton Date: 2016-02-14T20:20:15Z RYA-35 - Added a class that converts BindingSet objects into Strings and back again. commit 6b2944693ac5bc4ef0ea9e067715c63c9b78bf88 Author: Mike Pfaffenberger Date: 2016-02-18T14:47:09Z RYA-37 Bump GeoMesa version to 1.2.0 commit 8db5bbc3aa0256426c1c9ec925a31bf9e177ff3b Author: Kevin Chilton Date: 2016-02-19T22:22:15Z RYA-41 Fixed a bug where AccumuloIndexSet assumes the first node of a ParsedQuery is always a Projection node. commit 1a3839607d61be7ddd0124f6e50f7c0ae35367c8 Author: Aaron Mihalik Date: 2016-02-22T19:39:14Z Merge branch 'geomesa_version_bump' of https://github.com/mpfaffenberger/incubator-rya into develop commit 39c39429019937284d2e100ce0cffd214ea726e3 Author: AmilaWijayarathna Date: 2016-03-02T14:44:47Z RYA-44 testMilliSecondsNoZone test case fails when running from +0530 time zone commit 188791306474f5d45f9bb42f3f424bc6709c5eb0 Author: Aaron Mihalik Date: 2016-03-02T23:37:28Z RYA-7 Applying Correct License commit d5202aa52309ab914fb561d73015b22234c0d53f Author: Aaron Mihalik Date: 2016-03-04T02:40:07Z RYA-45 Pushing apache-rat constraints to submodules commit c4d44ebae60c1ac3db5ea91d3f05ed9a588cf9b2 Author: pujav65 Date: 2016-01-13T16:21:52Z RYA-11 initial implementation of free text for mongo backend commit efcae829c97e78b122b0d27586dd2a74e5c3 Author: pujav65 Date: 2016-02-29T20:15:52Z RYA-34 fixing inference with mongo db commit 358c13b83dc04887f95afee1c326971602d31647 Author: pujav65 Date: 2016-03-14T14:55:31Z RYA-11 fixing compile errors commit commit a150595e2184caa57e93e6e1c18c4cac6477e8cb Author: jej2003 Date: 2016-03-17T18:04:01Z update so that regular expressions are applied in cases where the strategy is defaulted to SPO commit 4a73be7ab8afe24faf9b871f5e11c9c3ca5f034a Author: craighahnparsons Date: 2016-03-17T18:31:36Z Added data type in tuple creation commit b6d271c6c85d45d0bc8993bcbf19503d299ba17a Author: jej2003 Date: 2016-03-17T21:08:39Z move delete mutation creation to RyaTableMutationsFactory so it can be reused (i.e. applications needing to migrate data within Rya) commit 15ec5d5faa670f9a65de52caa907eebdfa5ae757 Author: Kev
[jira] [Commented] (RYA-149) Pom Modifications after 3.2.10 release
[ https://issues.apache.org/jira/browse/RYA-149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427135#comment-15427135 ] ASF GitHub Bot commented on RYA-149: Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/69 looks good. I won't be able to merge this today, but I don't see why you couldn't merge it since this change was already discussed on the dev list. > Pom Modifications after 3.2.10 release > -- > > Key: RYA-149 > URL: https://issues.apache.org/jira/browse/RYA-149 > Project: Rya > Issue Type: Improvement > Components: build >Affects Versions: 3.2.10 >Reporter: Aaron Mihalik >Priority: Trivial > > There are a couple changes that we should implement to make the release > slightly easier. > 1. Change the default behavior for the maven release plug: The default > behavior should be "deploy" or "deploy-site" but we override it to only be > "install". We should remove the override from the parent pom. > 2. Change the tag name format to be the project version number: > https://maven.apache.org/maven-release/maven-release-plugin/examples/prepare-release.html#Overriding_the_default_tag_name_format > If there are any other changes I'd like to see, I'll add it to the comments -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya issue #69: RYA-149 Updating pom to make the release easier
Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/69 looks good. I won't be able to merge this today, but I don't see why you couldn't merge it since this change was already discussed on the dev list. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (RYA-156) Add a Pull Request Template
[ https://issues.apache.org/jira/browse/RYA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Smith reassigned RYA-156: Assignee: Andrew Smith > Add a Pull Request Template > --- > > Key: RYA-156 > URL: https://issues.apache.org/jira/browse/RYA-156 > Project: Rya > Issue Type: Wish > Components: docs >Reporter: Aaron Mihalik >Assignee: Andrew Smith > > I really like how Andrew described this Pull Request > [here|https://github.com/apache/incubator-rya/pull/72] > This task is to capture this outline and create a "Pull Request > [Template|https://github.com/blog/2111-issue-and-pull-request-templates]"; for > instance, here is the one for > [Spark|https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (RYA-133) Mongo Secondary indexers are broken
[ https://issues.apache.org/jira/browse/RYA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427104#comment-15427104 ] ASF GitHub Bot commented on RYA-133: Github user isper3at commented on the issue: https://github.com/apache/incubator-rya/pull/72 I already have it templated up in a bookmarklett. We can actually add something to the .git folder so that the template is auto-added to the PR. I'll assign the ticket to myself and set it up > Mongo Secondary indexers are broken > --- > > Key: RYA-133 > URL: https://issues.apache.org/jira/browse/RYA-133 > Project: Rya > Issue Type: Sub-task >Reporter: Andrew Smith >Assignee: Andrew Smith > Fix For: 3.2.10 > > > With the creation of integration tests for mongo support in the rya admin > shell, it was discovered that the refactoring of the mongo secondary indexers > broke some of the functionality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya issue #72: RYA-133 Fix broken Mongo Secondary Indexers
Github user isper3at commented on the issue: https://github.com/apache/incubator-rya/pull/72 I already have it templated up in a bookmarklett. We can actually add something to the .git folder so that the template is auto-added to the PR. I'll assign the ticket to myself and set it up --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-133) Mongo Secondary indexers are broken
[ https://issues.apache.org/jira/browse/RYA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427101#comment-15427101 ] ASF GitHub Bot commented on RYA-133: Github user amihalik commented on the issue: https://github.com/apache/incubator-rya/pull/72 No comments on your PR commit, but I really like your description. I think we should do this: [RYA-156 - Add a Pull Request Template](https://issues.apache.org/jira/browse/RYA-156) > Mongo Secondary indexers are broken > --- > > Key: RYA-133 > URL: https://issues.apache.org/jira/browse/RYA-133 > Project: Rya > Issue Type: Sub-task >Reporter: Andrew Smith >Assignee: Andrew Smith > Fix For: 3.2.10 > > > With the creation of integration tests for mongo support in the rya admin > shell, it was discovered that the refactoring of the mongo secondary indexers > broke some of the functionality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (RYA-156) Add a Pull Request Template
[ https://issues.apache.org/jira/browse/RYA-156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Mihalik updated RYA-156: -- Description: I really like how Andrew described this Pull Request [here|https://github.com/apache/incubator-rya/pull/72] This task is to capture this outline and create a "Pull Request [Template|https://github.com/blog/2111-issue-and-pull-request-templates]"; for instance, here is the one for [Spark|https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE] was: I really like how Andrew described this Pull Request [here|https://github.com/apache/incubator-rya/pull/72] This task is to capture this outline and create a "Pull Request [Template|https://github.com/blog/2111-issue-and-pull-request-templates] " for instance, here is [Sparks|https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE] > Add a Pull Request Template > --- > > Key: RYA-156 > URL: https://issues.apache.org/jira/browse/RYA-156 > Project: Rya > Issue Type: Wish > Components: docs >Reporter: Aaron Mihalik > > I really like how Andrew described this Pull Request > [here|https://github.com/apache/incubator-rya/pull/72] > This task is to capture this outline and create a "Pull Request > [Template|https://github.com/blog/2111-issue-and-pull-request-templates]"; for > instance, here is the one for > [Spark|https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya issue #72: RYA-133 Fix broken Mongo Secondary Indexers
Github user amihalik commented on the issue: https://github.com/apache/incubator-rya/pull/72 No comments on your PR commit, but I really like your description. I think we should do this: [RYA-156 - Add a Pull Request Template](https://issues.apache.org/jira/browse/RYA-156) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-133) Mongo Secondary indexers are broken
[ https://issues.apache.org/jira/browse/RYA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427100#comment-15427100 ] ASF GitHub Bot commented on RYA-133: Github user isper3at commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/72#discussion_r75383666 --- Diff: extras/indexing/src/main/java/mvm/rya/indexing/FilterFunctionOptimizer.java --- @@ -101,23 +102,23 @@ public void setConf(final Configuration conf) { init = false; try { init(); -} catch (final NumberFormatException | UnknownHostException e) { +} catch (final NumberFormatException | IOException e) { LOG.error("Unable to update to use new config, falling back to the old config.", e); init = true; } } -private synchronized void init() throws NumberFormatException, UnknownHostException { +private synchronized void init() throws NumberFormatException, IOException { if (!init) { if (ConfigUtils.getUseMongo(conf)) { try { final MongoClient mongoClient = MongoConnectorFactory.getMongoClient(conf); -geoIndexer = new MongoGeoIndexer(mongoClient); -geoIndexer.setConf(conf); -freeTextIndexer = new MongoFreeTextIndexer(mongoClient); -freeTextIndexer.setConf(conf); -temporalIndexer = new MongoTemporalIndexer(mongoClient); -temporalIndexer.setConf(conf); +geoIndexer = new MongoGeoIndexer(); +((AbstractMongoIndexer) geoIndexer).initIndexer(conf, mongoClient); --- End diff -- that might be a problem in the other initIndexer() The reason there is a seperate initIndexer() function is because init() does things I don't want to happen quite yet. > Mongo Secondary indexers are broken > --- > > Key: RYA-133 > URL: https://issues.apache.org/jira/browse/RYA-133 > Project: Rya > Issue Type: Sub-task >Reporter: Andrew Smith >Assignee: Andrew Smith > Fix For: 3.2.10 > > > With the creation of integration tests for mongo support in the rya admin > shell, it was discovered that the refactoring of the mongo secondary indexers > broke some of the functionality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #72: RYA-133 Fix broken Mongo Secondary Indexers
Github user isper3at commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/72#discussion_r75383666 --- Diff: extras/indexing/src/main/java/mvm/rya/indexing/FilterFunctionOptimizer.java --- @@ -101,23 +102,23 @@ public void setConf(final Configuration conf) { init = false; try { init(); -} catch (final NumberFormatException | UnknownHostException e) { +} catch (final NumberFormatException | IOException e) { LOG.error("Unable to update to use new config, falling back to the old config.", e); init = true; } } -private synchronized void init() throws NumberFormatException, UnknownHostException { +private synchronized void init() throws NumberFormatException, IOException { if (!init) { if (ConfigUtils.getUseMongo(conf)) { try { final MongoClient mongoClient = MongoConnectorFactory.getMongoClient(conf); -geoIndexer = new MongoGeoIndexer(mongoClient); -geoIndexer.setConf(conf); -freeTextIndexer = new MongoFreeTextIndexer(mongoClient); -freeTextIndexer.setConf(conf); -temporalIndexer = new MongoTemporalIndexer(mongoClient); -temporalIndexer.setConf(conf); +geoIndexer = new MongoGeoIndexer(); +((AbstractMongoIndexer) geoIndexer).initIndexer(conf, mongoClient); --- End diff -- that might be a problem in the other initIndexer() The reason there is a seperate initIndexer() function is because init() does things I don't want to happen quite yet. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (RYA-156) Add a Pull Request Template
Aaron Mihalik created RYA-156: - Summary: Add a Pull Request Template Key: RYA-156 URL: https://issues.apache.org/jira/browse/RYA-156 Project: Rya Issue Type: Wish Components: docs Reporter: Aaron Mihalik I really like how Andrew described this Pull Request [here|https://github.com/apache/incubator-rya/pull/72] This task is to capture this outline and create a "Pull Request [Template|https://github.com/blog/2111-issue-and-pull-request-templates] " for instance, here is [Sparks|https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (RYA-133) Mongo Secondary indexers are broken
[ https://issues.apache.org/jira/browse/RYA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427095#comment-15427095 ] ASF GitHub Bot commented on RYA-133: Github user DLotts commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/72#discussion_r75383276 --- Diff: extras/indexing/src/main/java/mvm/rya/indexing/FilterFunctionOptimizer.java --- @@ -101,23 +102,23 @@ public void setConf(final Configuration conf) { init = false; try { init(); -} catch (final NumberFormatException | UnknownHostException e) { +} catch (final NumberFormatException | IOException e) { LOG.error("Unable to update to use new config, falling back to the old config.", e); init = true; } } -private synchronized void init() throws NumberFormatException, UnknownHostException { +private synchronized void init() throws NumberFormatException, IOException { if (!init) { if (ConfigUtils.getUseMongo(conf)) { try { final MongoClient mongoClient = MongoConnectorFactory.getMongoClient(conf); -geoIndexer = new MongoGeoIndexer(mongoClient); -geoIndexer.setConf(conf); -freeTextIndexer = new MongoFreeTextIndexer(mongoClient); -freeTextIndexer.setConf(conf); -temporalIndexer = new MongoTemporalIndexer(mongoClient); -temporalIndexer.setConf(conf); +geoIndexer = new MongoGeoIndexer(); +((AbstractMongoIndexer) geoIndexer).initIndexer(conf, mongoClient); --- End diff -- GeoIndexer has null fields: storageStrategy , predicates. The implementation init is not called? In the code I see from develop, init does this predicates = ConfigUtils.getGeoPredicates(conf); storageStrategy = ... > Mongo Secondary indexers are broken > --- > > Key: RYA-133 > URL: https://issues.apache.org/jira/browse/RYA-133 > Project: Rya > Issue Type: Sub-task >Reporter: Andrew Smith >Assignee: Andrew Smith > Fix For: 3.2.10 > > > With the creation of integration tests for mongo support in the rya admin > shell, it was discovered that the refactoring of the mongo secondary indexers > broke some of the functionality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #72: RYA-133 Fix broken Mongo Secondary Indexers
Github user DLotts commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/72#discussion_r75383276 --- Diff: extras/indexing/src/main/java/mvm/rya/indexing/FilterFunctionOptimizer.java --- @@ -101,23 +102,23 @@ public void setConf(final Configuration conf) { init = false; try { init(); -} catch (final NumberFormatException | UnknownHostException e) { +} catch (final NumberFormatException | IOException e) { LOG.error("Unable to update to use new config, falling back to the old config.", e); init = true; } } -private synchronized void init() throws NumberFormatException, UnknownHostException { +private synchronized void init() throws NumberFormatException, IOException { if (!init) { if (ConfigUtils.getUseMongo(conf)) { try { final MongoClient mongoClient = MongoConnectorFactory.getMongoClient(conf); -geoIndexer = new MongoGeoIndexer(mongoClient); -geoIndexer.setConf(conf); -freeTextIndexer = new MongoFreeTextIndexer(mongoClient); -freeTextIndexer.setConf(conf); -temporalIndexer = new MongoTemporalIndexer(mongoClient); -temporalIndexer.setConf(conf); +geoIndexer = new MongoGeoIndexer(); +((AbstractMongoIndexer) geoIndexer).initIndexer(conf, mongoClient); --- End diff -- GeoIndexer has null fields: storageStrategy , predicates. The implementation init is not called? In the code I see from develop, init does this predicates = ConfigUtils.getGeoPredicates(conf); storageStrategy = ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-133) Mongo Secondary indexers are broken
[ https://issues.apache.org/jira/browse/RYA-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427065#comment-15427065 ] ASF GitHub Bot commented on RYA-133: GitHub user isper3at opened a pull request: https://github.com/apache/incubator-rya/pull/72 RYA-133 Fix broken Mongo Secondary Indexers ## Description >What changed? The secondary indexers need an empty constructor. Moved back to an init() call instead, leave empty constructor open for the reflexive calls. ### Links [Jira](https://issues.apache.org/jira/browse/RYA-133) ### Checklist - [ ] Code Review - [ x] Squash commits People To Review @dlotts You can merge this pull request into a Git repository by running: $ git pull https://github.com/isper3at/incubator-rya RYA-133_MongoFix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/72.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #72 commit 5845536b5e0c8036658751010874879b308049b4 Author: isper3at Date: 2016-07-28T17:02:35Z RYA-133 Fix broken Mongo Secondary Indexers The refactoring of the mongo secondary indexers broke some functionality. The reflective way the indexers are created requires an empty constructor. The refactor was altered to allow the changes to persist while gaining the empty constructor. > Mongo Secondary indexers are broken > --- > > Key: RYA-133 > URL: https://issues.apache.org/jira/browse/RYA-133 > Project: Rya > Issue Type: Sub-task >Reporter: Andrew Smith >Assignee: Andrew Smith > Fix For: 3.2.10 > > > With the creation of integration tests for mongo support in the rya admin > shell, it was discovered that the refactoring of the mongo secondary indexers > broke some of the functionality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #72: RYA-133 Fix broken Mongo Secondary Indexers
GitHub user isper3at opened a pull request: https://github.com/apache/incubator-rya/pull/72 RYA-133 Fix broken Mongo Secondary Indexers ## Description >What changed? The secondary indexers need an empty constructor. Moved back to an init() call instead, leave empty constructor open for the reflexive calls. ### Links [Jira](https://issues.apache.org/jira/browse/RYA-133) ### Checklist - [ ] Code Review - [ x] Squash commits People To Review @dlotts You can merge this pull request into a Git repository by running: $ git pull https://github.com/isper3at/incubator-rya RYA-133_MongoFix Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/72.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #72 commit 5845536b5e0c8036658751010874879b308049b4 Author: isper3at Date: 2016-07-28T17:02:35Z RYA-133 Fix broken Mongo Secondary Indexers The refactoring of the mongo secondary indexers broke some functionality. The reflective way the indexers are created requires an empty constructor. The refactor was altered to allow the changes to persist while gaining the empty constructor. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-151) Rya Test Bed
[ https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427033#comment-15427033 ] ASF GitHub Bot commented on RYA-151: Github user kchilton2 commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75374195 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- So the two parameters that I was planning on having you configure are: 1. What SPARQL query to execute. 2. The number of results to read from the query. > Rya Test Bed > > > Key: RYA-151 > URL: https://issues.apache.org/jira/browse/RYA-151 > Project: Rya > Issue Type: Wish > Components: clients, dao >Reporter: Puja Valiyil >Assignee: Kevin Chilton > > It would be nice if there was a test bed checked in with Rya that could be > used for benchmark testing. This test bed should include options for > uploading data to a rya instance, then issuing some benchmark queries (and > timing them in a way that everyone agrees is sound). It could be a command > line tool. It should be dao agnostic (work on either mongo, accumulo, etc.). > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #68: [WIP] RYA-151 Rya Query benchmark tool imple...
Github user kchilton2 commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75374195 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- So the two parameters that I was planning on having you configure are: 1. What SPARQL query to execute. 2. The number of results to read from the query. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-151) Rya Test Bed
[ https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427030#comment-15427030 ] ASF GitHub Bot commented on RYA-151: Github user kchilton2 commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75374018 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- The data is reported by the JMH framework for every parameter you tweak. So it will say which query the results are for. You can use the benchmark tool's output to generate those reports. > Rya Test Bed > > > Key: RYA-151 > URL: https://issues.apache.org/jira/browse/RYA-151 > Project: Rya > Issue Type: Wish > Components: clients, dao >Reporter: Puja Valiyil >Assignee: Kevin Chilton > > It would be nice if there was a test bed checked in with Rya that could be > used for benchmark testing. This test bed should include options for > uploading data to a rya instance, then issuing some benchmark queries (and > timing them in a way that everyone agrees is sound). It could be a command > line tool. It should be dao agnostic (work on either mongo, accumulo, etc.). > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #68: [WIP] RYA-151 Rya Query benchmark tool imple...
Github user kchilton2 commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75374018 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- The data is reported by the JMH framework for every parameter you tweak. So it will say which query the results are for. You can use the benchmark tool's output to generate those reports. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (RYA-94) Remove from secondary indexes when deleting from DAO
[ https://issues.apache.org/jira/browse/RYA-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David W. Lotts updated RYA-94: -- Priority: Blocker (was: Critical) > Remove from secondary indexes when deleting from DAO > > > Key: RYA-94 > URL: https://issues.apache.org/jira/browse/RYA-94 > Project: Rya > Issue Type: Bug >Reporter: Andrew Smith >Assignee: David W. Lotts >Priority: Blocker > > Currently, if a delete is made from the DAO, the secondary indexers are not > explicitly notified. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (RYA-155) Merge Tool - Separate out config elements for different datastores
Eric White created RYA-155: -- Summary: Merge Tool - Separate out config elements for different datastores Key: RYA-155 URL: https://issues.apache.org/jira/browse/RYA-155 Project: Rya Issue Type: Task Components: clients Affects Versions: 3.2.10 Reporter: Eric White Assignee: Eric White Config elements for each datastore should be stored in their own package. (i.e. Accumulo specific config properties should be in the accumulo export package) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (RYA-109) Add Spark support
[ https://issues.apache.org/jira/browse/RYA-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426924#comment-15426924 ] ASF GitHub Bot commented on RYA-109: Github user Ejgood21 closed the pull request at: https://github.com/apache/incubator-rya/pull/70 > Add Spark support > - > > Key: RYA-109 > URL: https://issues.apache.org/jira/browse/RYA-109 > Project: Rya > Issue Type: New Feature > Components: clients >Affects Versions: 3.2.10 >Reporter: Andrew Smith >Assignee: Evan Good > Fix For: 3.2.10 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #70: [WIP] RYA-109 Add Spark Support
Github user Ejgood21 closed the pull request at: https://github.com/apache/incubator-rya/pull/70 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-151) Rya Test Bed
[ https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426900#comment-15426900 ] ASF GitHub Bot commented on RYA-151: Github user amihalik commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75355985 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- We should have query level granularity for the benchmark (i.e. we should have timing results for each query in the benchmark), so that we can have pretty graphs like this: ![image](https://cloud.githubusercontent.com/assets/4710489/17784519/c838a508-654a-11e6-8d65-e4043177dc6a.png) Also, it would be nice if the benchmark printed out the name of the query for the results. So we can have a table like this: ![image](https://cloud.githubusercontent.com/assets/4710489/17784596/14630158-654b-11e6-8dbb-23325619d3d7.png) (Note: I ripped off these graphics from Roshan's Paper) > Rya Test Bed > > > Key: RYA-151 > URL: https://issues.apache.org/jira/browse/RYA-151 > Project: Rya > Issue Type: Wish > Components: clients, dao >Reporter: Puja Valiyil >Assignee: Kevin Chilton > > It would be nice if there was a test bed checked in with Rya that could be > used for benchmark testing. This test bed should include options for > uploading data to a rya instance, then issuing some benchmark queries (and > timing them in a way that everyone agrees is sound). It could be a command > line tool. It should be dao agnostic (work on either mongo, accumulo, etc.). > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #68: [WIP] RYA-151 Rya Query benchmark tool imple...
Github user amihalik commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75355985 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- We should have query level granularity for the benchmark (i.e. we should have timing results for each query in the benchmark), so that we can have pretty graphs like this: ![image](https://cloud.githubusercontent.com/assets/4710489/17784519/c838a508-654a-11e6-8d65-e4043177dc6a.png) Also, it would be nice if the benchmark printed out the name of the query for the results. So we can have a table like this: ![image](https://cloud.githubusercontent.com/assets/4710489/17784596/14630158-654b-11e6-8dbb-23325619d3d7.png) (Note: I ripped off these graphics from Roshan's Paper) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-151) Rya Test Bed
[ https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426879#comment-15426879 ] ASF GitHub Bot commented on RYA-151: Github user kchilton2 commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75354404 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- What do you want to do with those attributes? If they aren't used within the benchmark, that sort of information could easily just be XML comments. {code} {code} > Rya Test Bed > > > Key: RYA-151 > URL: https://issues.apache.org/jira/browse/RYA-151 > Project: Rya > Issue Type: Wish > Components: clients, dao >Reporter: Puja Valiyil >Assignee: Kevin Chilton > > It would be nice if there was a test bed checked in with Rya that could be > used for benchmark testing. This test bed should include options for > uploading data to a rya instance, then issuing some benchmark queries (and > timing them in a way that everyone agrees is sound). It could be a command > line tool. It should be dao agnostic (work on either mongo, accumulo, etc.). > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #68: [WIP] RYA-151 Rya Query benchmark tool imple...
Github user kchilton2 commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75354404 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- What do you want to do with those attributes? If they aren't used within the benchmark, that sort of information could easily just be XML comments. {code} {code} --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-109) Add Spark support
[ https://issues.apache.org/jira/browse/RYA-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426768#comment-15426768 ] ASF GitHub Bot commented on RYA-109: GitHub user Ejgood21 opened a pull request: https://github.com/apache/incubator-rya/pull/71 [WIP] RYA-109 Feature/spark graph x You can merge this pull request into a Git repository by running: $ git pull https://github.com/Ejgood21/incubator-rya feature/sparkGraphX Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/71.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #71 commit 621cce8164e8e5322bb667143d363381bf8df2c2 Author: Evan Good Date: 2016-08-18T15:15:45Z initial spark support > Add Spark support > - > > Key: RYA-109 > URL: https://issues.apache.org/jira/browse/RYA-109 > Project: Rya > Issue Type: New Feature > Components: clients >Affects Versions: 3.2.10 >Reporter: Andrew Smith >Assignee: Evan Good > Fix For: 3.2.10 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #71: [WIP] RYA-109 Feature/spark graph x
GitHub user Ejgood21 opened a pull request: https://github.com/apache/incubator-rya/pull/71 [WIP] RYA-109 Feature/spark graph x You can merge this pull request into a Git repository by running: $ git pull https://github.com/Ejgood21/incubator-rya feature/sparkGraphX Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/71.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #71 commit 621cce8164e8e5322bb667143d363381bf8df2c2 Author: Evan Good Date: 2016-08-18T15:15:45Z initial spark support --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-76) Organize and document MapReduce support
[ https://issues.apache.org/jira/browse/RYA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426726#comment-15426726 ] ASF GitHub Bot commented on RYA-76: --- Github user jessehatfield closed the pull request at: https://github.com/apache/incubator-rya/pull/55 > Organize and document MapReduce support > --- > > Key: RYA-76 > URL: https://issues.apache.org/jira/browse/RYA-76 > Project: Rya > Issue Type: Improvement >Reporter: Jesse Hatfield >Assignee: Jesse Hatfield > > Rya has some MapReduce support (e.g. a Writable, an InputFormat -- RYA-21), > but this could be more clearly structured and documented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #55: RYA-76 Consolidated MapReduce API and applic...
Github user jessehatfield closed the pull request at: https://github.com/apache/incubator-rya/pull/55 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-109) Add Spark support
[ https://issues.apache.org/jira/browse/RYA-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426654#comment-15426654 ] ASF GitHub Bot commented on RYA-109: Github user amihalik commented on the issue: https://github.com/apache/incubator-rya/pull/70 Something is screwy with Jesse's commit that you pulled in. The commit you have has hash 1855213 and the one in the apache/develop branch has 01489ef. This needs to be fixed before we can pull in your PR, and it's probably easier to fix now rather than later. Let me know if you need any assistance with this and I can walk you through it. > Add Spark support > - > > Key: RYA-109 > URL: https://issues.apache.org/jira/browse/RYA-109 > Project: Rya > Issue Type: New Feature > Components: clients >Affects Versions: 3.2.10 >Reporter: Andrew Smith >Assignee: Evan Good > Fix For: 3.2.10 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya issue #70: [WIP] RYA-109 Add Spark Support
Github user amihalik commented on the issue: https://github.com/apache/incubator-rya/pull/70 Something is screwy with Jesse's commit that you pulled in. The commit you have has hash 1855213 and the one in the apache/develop branch has 01489ef. This needs to be fixed before we can pull in your PR, and it's probably easier to fix now rather than later. Let me know if you need any assistance with this and I can walk you through it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-rya pull request #70: Feature/spark
GitHub user Ejgood21 opened a pull request: https://github.com/apache/incubator-rya/pull/70 Feature/spark Spark GraphX Support - WIP You can merge this pull request into a Git repository by running: $ git pull https://github.com/Ejgood21/incubator-rya feature/spark Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/70.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #70 commit 18552132e8bfecec33b57aae302e25d93c28e7e3 Author: Jesse Hatfield Date: 2016-06-30T19:50:08Z Consolidated MapReduce API and applications into toplevel project. Changes include: - Made RdfFileInputFormat threaded, allowing it to handle all RDF input - Added entity-centric indexing to RyaOutputFormat - Added methods for using Rya input and output formats to AbstractAccumuloMRTool (renamed setupInputFormat to setupAccumuloInput since there are now multiple input options) - Removed *NullIndexer classes, which were only used in RyaOutputFormat - Removed StatementWritable, changed to RyaStatementWritable where applicable (for consistency) - Fixed hashCode/compareTo/equals methods in RyaStatementWritable, RyaType, and RyaURI - Minor renaming and repackaging (standalone tools go in "tools", "fileinput" and "utils" removed) - Minor tweaks to AccumuloHDFSFileInputFormat, which seemed to be broken with Accumulo 1.6 - Documented code and added a MapReduce page to the manual - Added "examples" package with one example - Removed RyaStatementMapper and RyaStatementReducer: these simply insert records into Rya. This functionality can be achieved with RyaOutputFormat and the default mapper/reducer, so these two classes seem redundant. - Removed BulkNtripsInputTool, BulkNtripsInputToolIndexing, RdfFileInputByLineTool, and RyaBatchWriterInputTool: Fixes to RdfFileInputFormat now allow RdfFileInputTool to now handle all the file input use cases (configurable secondary indexers, handles any format, scales), rendering the other file import tools redundant. (Previously, all five tools had largely overlapping but subtly different behavior.) commit dd727af8d6d97f273ca168637a15f02f5eb1a605 Author: Evan Good Date: 2016-08-18T15:15:45Z initial spark support --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Github user DLotts commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/67#discussion_r75326137 --- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java --- @@ -41,26 +47,77 @@ * @param conf The {@link Configuration} defining how to construct the MongoClient. * @return A {@link MongoClient}. This client is lazy loaded and the same one * is used throughout the lifecycle of the application. - * @throws NumberFormatException - Thrown if the configured port is not a valid number - * @throws UnknownHostException - The configured host cannot be found. + * @throws IOException - if MongodForTestsFactory constructor has an io exception. + * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing. + * @throws UnknownHostException - if MongoDB host name is not found. */ -public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException { -if(mongoClient == null) { -final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE); -final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT)); -final ServerAddress server = new ServerAddress(host, port); - -//check for authentication credentials -if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) { -final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER); -final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME); -final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray(); -final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd); -mongoClient = new MongoClient(server, Arrays.asList(cred)); +public static synchronized MongoClient getMongoClient(final Configuration conf) +throws ConfigurationRuntimeException, MongoException { +if (mongoClient == null) { +// The singleton client has not yet created, is it a test/mock instance, or a service? +if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) { --- End diff -- Private methods called from this method, correct? Great idea. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-149) Pom Modifications after 3.2.10 release
[ https://issues.apache.org/jira/browse/RYA-149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426467#comment-15426467 ] ASF GitHub Bot commented on RYA-149: GitHub user amihalik opened a pull request: https://github.com/apache/incubator-rya/pull/69 RYA-149 Updating pom to make the release easier You can merge this pull request into a Git repository by running: $ git pull https://github.com/amihalik/incubator-rya RYA-149 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/69.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #69 commit 134554bd4073aa1629f33c96994b41ab75267c32 Author: Aaron Mihalik Date: 2016-08-18T13:29:31Z RYA-149 Updating pom to make the release easier > Pom Modifications after 3.2.10 release > -- > > Key: RYA-149 > URL: https://issues.apache.org/jira/browse/RYA-149 > Project: Rya > Issue Type: Improvement > Components: build >Affects Versions: 3.2.10 >Reporter: Aaron Mihalik >Priority: Trivial > > There are a couple changes that we should implement to make the release > slightly easier. > 1. Change the default behavior for the maven release plug: The default > behavior should be "deploy" or "deploy-site" but we override it to only be > "install". We should remove the override from the parent pom. > 2. Change the tag name format to be the project version number: > https://maven.apache.org/maven-release/maven-release-plugin/examples/prepare-release.html#Overriding_the_default_tag_name_format > If there are any other changes I'd like to see, I'll add it to the comments -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #69: RYA-149 Updating pom to make the release eas...
GitHub user amihalik opened a pull request: https://github.com/apache/incubator-rya/pull/69 RYA-149 Updating pom to make the release easier You can merge this pull request into a Git repository by running: $ git pull https://github.com/amihalik/incubator-rya RYA-149 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/69.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #69 commit 134554bd4073aa1629f33c96994b41ab75267c32 Author: Aaron Mihalik Date: 2016-08-18T13:29:31Z RYA-149 Updating pom to make the release easier --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-151) Rya Test Bed
[ https://issues.apache.org/jira/browse/RYA-151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426397#comment-15426397 ] ASF GitHub Bot commented on RYA-151: Github user amihalik commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75303476 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- Can we add attributes to this? for instance ```xml ``` > Rya Test Bed > > > Key: RYA-151 > URL: https://issues.apache.org/jira/browse/RYA-151 > Project: Rya > Issue Type: Wish > Components: clients, dao >Reporter: Puja Valiyil >Assignee: Kevin Chilton > > It would be nice if there was a test bed checked in with Rya that could be > used for benchmark testing. This test bed should include options for > uploading data to a rya instance, then issuing some benchmark queries (and > timing them in a way that everyone agrees is sound). It could be a command > line tool. It should be dao agnostic (work on either mongo, accumulo, etc.). > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #68: [WIP] RYA-151 Rya Query benchmark tool imple...
Github user amihalik commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/68#discussion_r75303476 --- Diff: extras/rya.benchmark/src/test/java/org/apache/rya/benchmark/query/BenchmarkQueriesReaderIT.java --- @@ -0,0 +1,55 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.benchmark.query; + +import static org.junit.Assert.assertEquals; + +import java.io.ByteArrayInputStream; +import java.io.InputStream; + +import javax.xml.bind.JAXBException; + +import org.apache.rya.benchmark.query.QueryBenchmark.BenchmarkQueriesReader; +import org.junit.Test; +import org.xml.sax.SAXException; + +import com.google.common.base.Charsets; + +/** + * Tests the methods of {@link BenchmarkQueriesReader}. + */ +public class BenchmarkQueriesReaderIT { + +@Test +public void load() throws JAXBException, SAXException { +// Unmarshal some XML. +final String xml = +"\n" + +"\n" + +"" + --- End diff -- Can we add attributes to this? for instance ```xml ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-105) Fluo Integration with Rya
[ https://issues.apache.org/jira/browse/RYA-105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426381#comment-15426381 ] ASF GitHub Bot commented on RYA-105: Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/58 @meiercaleb can you close this pull request? > Fluo Integration with Rya > - > > Key: RYA-105 > URL: https://issues.apache.org/jira/browse/RYA-105 > Project: Rya > Issue Type: Bug > Components: dao >Affects Versions: 3.2.10 >Reporter: Caleb Meier >Assignee: Caleb Meier > Fix For: 3.2.10 > > > The Fluo table for incrementally updating precomputed joins is not fully > integrated with Rya. The PrecomputedJoinIndexer (which is an implementation > of the AccumuloIndexer interface) is not being added to the list of indexers > when the PCJ flag is set. So the DAO is never adding to Fluo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya issue #58: RYA-105: Fluo Integration
Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/58 @meiercaleb can you close this pull request? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-130) Rya Mongo Fail on Multi Bindingset Joins
[ https://issues.apache.org/jira/browse/RYA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426370#comment-15426370 ] ASF GitHub Bot commented on RYA-130: Github user amihalik closed the pull request at: https://github.com/apache/incubator-rya/pull/57 > Rya Mongo Fail on Multi Bindingset Joins > > > Key: RYA-130 > URL: https://issues.apache.org/jira/browse/RYA-130 > Project: Rya > Issue Type: Bug > Components: dao >Affects Versions: 3.2.9 >Reporter: Aaron Mihalik >Assignee: Aaron Mihalik > > Rya Mongo fails on a fairly straightforward query: > {code} > DATA = > . > . > . > QUERY = > select ?s ?foo where { > ?s ?foo . > ?s } > {code} > This is a bug in how Mongo Rya solves joins. The first Statement Pattern > will return multiple binding sets, and try to join it with a single > statement. The Mongo Rya Query Engine can't handle this case. > Full Code at: > https://gist.github.com/amihalik/61cf870650b7443389e7be7600f8502c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #46: Adding Visibilties to Rya Web
Github user amihalik closed the pull request at: https://github.com/apache/incubator-rya/pull/46 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-130) Rya Mongo Fail on Multi Bindingset Joins
[ https://issues.apache.org/jira/browse/RYA-130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426369#comment-15426369 ] ASF GitHub Bot commented on RYA-130: Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/57 @amihalik can you close this pull request? > Rya Mongo Fail on Multi Bindingset Joins > > > Key: RYA-130 > URL: https://issues.apache.org/jira/browse/RYA-130 > Project: Rya > Issue Type: Bug > Components: dao >Affects Versions: 3.2.9 >Reporter: Aaron Mihalik >Assignee: Aaron Mihalik > > Rya Mongo fails on a fairly straightforward query: > {code} > DATA = > . > . > . > QUERY = > select ?s ?foo where { > ?s ?foo . > ?s } > {code} > This is a bug in how Mongo Rya solves joins. The first Statement Pattern > will return multiple binding sets, and try to join it with a single > statement. The Mongo Rya Query Engine can't handle this case. > Full Code at: > https://gist.github.com/amihalik/61cf870650b7443389e7be7600f8502c -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya pull request #57: RYA-130 Fixed Rya Mongo Multi Binding Set Jo...
Github user amihalik closed the pull request at: https://github.com/apache/incubator-rya/pull/57 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-76) Organize and document MapReduce support
[ https://issues.apache.org/jira/browse/RYA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426366#comment-15426366 ] ASF GitHub Bot commented on RYA-76: --- Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/55 @jessehatfield can you please close this pull request? > Organize and document MapReduce support > --- > > Key: RYA-76 > URL: https://issues.apache.org/jira/browse/RYA-76 > Project: Rya > Issue Type: Improvement >Reporter: Jesse Hatfield >Assignee: Jesse Hatfield > > Rya has some MapReduce support (e.g. a Writable, an InputFormat -- RYA-21), > but this could be more clearly structured and documented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] incubator-rya issue #57: RYA-130 Fixed Rya Mongo Multi Binding Set Join
Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/57 @amihalik can you close this pull request? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-rya issue #55: RYA-76 Consolidated MapReduce API and applications ...
Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/55 @jessehatfield can you please close this pull request? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-rya issue #46: Adding Visibilties to Rya Web
Github user pujav65 commented on the issue: https://github.com/apache/incubator-rya/pull/46 @amihalik can you please close this pull request? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---