date:20150915

[jira] [Created] (LENS-775) NPE on closing HadoopFileFormatter

2015-09-15 Thread Deepak Barr (JIRA)

Deepak Barr created LENS-775:


 Summary: NPE on closing HadoopFileFormatter
 Key: LENS-775
 URL: https://issues.apache.org/jira/browse/LENS-775
 Project: Apache Lens
  Issue Type: Bug
Reporter: Deepak Barr
Assignee: Deepak Barr
Priority: Minor


In HadoopFileFormatter, If rowWriter intialization fails because of,say, HDFS 
permission issue. The close() method (called in finally block in 
ResultFormatter class) throws an NPE.

Code snippet - 

 public void setupOutputs() throws IOException {
String pathStr = ctx.getResultSetParentDir();
if (StringUtils.isBlank(pathStr)) {
  throw new IllegalArgumentException("No output path specified");
}
String outputPathStr = Strings.isNullOrEmpty(ctx.getQueryName()) ? ""
  : LensFileOutputFormat.getValidOutputFileName(ctx.getQueryName()) + "-";
outputPath = new Path(pathStr, outputPathStr + 
ctx.getQueryHandle().toString());
Path tmpWorkPath = new Path(pathStr, ctx.getQueryHandle().toString() + 
".tmp");
try {
  rowWriter = LensFileOutputFormat.createRecordWriter(ctx.getConf(), 
tmpWorkPath, Reporter.NULL,
ctx.getCompressOutput(), ctx.getOuptutFileExtn(), 
ctx.getResultEncoding());
  numRows=0;
} catch (IOException e) {
  throw new IllegalArgumentException("Could not create tmp path");
}
  }

@Override
  public void close() throws IOException {
rowWriter.close(Reporter.NULL);
  }




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (LENS-774) Unable to override "lens.server.session.expiry.service.interval.secs" because of spelling mistake

2015-09-15 Thread Deepak Barr (JIRA)

Deepak Barr created LENS-774:


 Summary: Unable to override 
"lens.server.session.expiry.service.interval.secs" because of spelling mistake
 Key: LENS-774
 URL: https://issues.apache.org/jira/browse/LENS-774
 Project: Apache Lens
  Issue Type: Bug
Reporter: Deepak Barr
Assignee: Deepak Barr
Priority: Minor


Property String in LensConfConstants.java

public static final String SESSION_EXPIRY_SERVICE_INTERVAL_IN_SECS = SERVER_PFX
+ "session.expiry.serivce.interval.secs";




Property in lensserver-default.xml

 
lens.server.session.expiry.service.interval.secs
3600
Interval at which lens session expiry service 
runs
  




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

2015-09-15 Thread Amareshwari Sriramadasu



> On Sept. 15, 2015, 2 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java, lines 
> > 586-587
> > 
> >
> > only null check needed.
> 
> Amareshwari Sriramadasu wrote:
> why? we should check if the column are queried from that table as well.
> 
> Rajat Khandelwal wrote:
> the empty case will be taken care while iterating over collection of zero 
> elements.

The check is not just for iteration, it is doing more along with iteration.


> On Sept. 15, 2015, 2 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java, line 
> > 585
> > 
> >
> > Among the approaches: 
> > 
> > 1. Relying on a flag to append "select" or "join"
> > 2. Making a list and later doing "select" + StringUtils.join("join", 
> > list)
> > 
> > the second seems more intuitive. And would be easier to debug later on.
> 
> Amareshwari Sriramadasu wrote:
> Did not understand the comment fully.
> 
> It is not simply prepending select or join to alist. If join needs to be 
> converted to select from a bridge table - it becomes a full fledged query 
> with select, from, groupby, filters and optionally more joins.
> 
> Rajat Khandelwal wrote:
> Yes. I hadn't understood the code fully. Can you please add docs 
> explaining the steps.

Sure. Will update with comments.


- Amareshwari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38402/#review99024
---


On Sept. 15, 2015, 12:48 p.m., Amareshwari Sriramadasu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38402/
> ---
> 
> (Updated Sept. 15, 2015, 12:48 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Changes include :
> - Ability to specify if the destination link in a join chain has 
> many-many-relationship
> - Ability to configure at query to say flatten the fields coming from 
> bridge-table relations
> - Enhance cube query rewriter (mainly the join clause) to add join for bridge 
> table to flatten them
> 
> Pending :
> - Add a test with multi fact query
> - Update documentation with feature addition
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/resources/cube-0.1.xsd 58f68f5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/JoinChain.java 
> e394e20 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 
> bf27b99 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/SchemaGraph.java 
> 1a37e80 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TableReference.java 
> 31fd97b 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
> aab2488 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java 
> c7f1e2a 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 7f56292 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestJoinResolver.java 
> cb63fad 
>   lens-server/src/main/java/org/apache/lens/server/metastore/JAXBUtils.java 
> ed472f6 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java
>  f9be6e6 
> 
> Diff: https://reviews.apache.org/r/38402/diff/
> 
> 
> Testing
> ---
> 
> Added testcases for bridge tables and queries on them.
> 
> All tests in lens-cube module pass. Will update full test results once done.
> 
> 
> Thanks,
> 
> Amareshwari Sriramadasu
> 
>

[jira] [Commented] (LENS-733) Add Helper Method for lens-regression

2015-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746889#comment-14746889
 ] 

Hadoop QA commented on LENS-733:


Applied patch: 
[rb37526.patch|https://issues.apache.org/jira/secure/attachment/12756011/rb37526.patch]
 and ran command: mvn clean install. Result: Success. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/62/

> Add Helper Method for lens-regression
> -
>
> Key: LENS-733
> URL: https://issues.apache.org/jira/browse/LENS-733
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: regression
>Reporter: Arshad Matin
>Assignee: Arshad Matin
> Attachments: rb37526.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

2015-09-15 Thread Rajat Khandelwal



> On Sept. 15, 2015, 7:30 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java, lines 
> > 586-587
> > 
> >
> > only null check needed.
> 
> Amareshwari Sriramadasu wrote:
> why? we should check if the column are queried from that table as well.

the empty case will be taken care while iterating over collection of zero 
elements.


> On Sept. 15, 2015, 7:30 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java, line 
> > 585
> > 
> >
> > Among the approaches: 
> > 
> > 1. Relying on a flag to append "select" or "join"
> > 2. Making a list and later doing "select" + StringUtils.join("join", 
> > list)
> > 
> > the second seems more intuitive. And would be easier to debug later on.
> 
> Amareshwari Sriramadasu wrote:
> Did not understand the comment fully.
> 
> It is not simply prepending select or join to alist. If join needs to be 
> converted to select from a bridge table - it becomes a full fledged query 
> with select, from, groupby, filters and optionally more joins.

Yes. I hadn't understood the code fully. Can you please add docs explaining the 
steps.


- Rajat


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38402/#review99024
---


On Sept. 15, 2015, 6:18 p.m., Amareshwari Sriramadasu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38402/
> ---
> 
> (Updated Sept. 15, 2015, 6:18 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Changes include :
> - Ability to specify if the destination link in a join chain has 
> many-many-relationship
> - Ability to configure at query to say flatten the fields coming from 
> bridge-table relations
> - Enhance cube query rewriter (mainly the join clause) to add join for bridge 
> table to flatten them
> 
> Pending :
> - Add a test with multi fact query
> - Update documentation with feature addition
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/resources/cube-0.1.xsd 58f68f5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/JoinChain.java 
> e394e20 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 
> bf27b99 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/SchemaGraph.java 
> 1a37e80 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TableReference.java 
> 31fd97b 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
> aab2488 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java 
> c7f1e2a 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 7f56292 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestJoinResolver.java 
> cb63fad 
>   lens-server/src/main/java/org/apache/lens/server/metastore/JAXBUtils.java 
> ed472f6 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java
>  f9be6e6 
> 
> Diff: https://reviews.apache.org/r/38402/diff/
> 
> 
> Testing
> ---
> 
> Added testcases for bridge tables and queries on them.
> 
> All tests in lens-cube module pass. Will update full test results once done.
> 
> 
> Thanks,
> 
> Amareshwari Sriramadasu
> 
>

[jira] [Commented] (LENS-392) Look ahead timerange should not look for all finer granularity partitions to be present

2015-09-15 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746869#comment-14746869
 ] 

Hudson commented on LENS-392:
-

SUCCESS: Integrated in Lens-Commit #1080 (See 
[https://builds.apache.org/job/Lens-Commit/1080/])
LENS-392 : Look ahead timerange should not look for all finer granularity 
partitions to be present (amareshwari: rev 
08135aa693658aa454e55a2c1e81c9e0fa19444b)
* lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java
* lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java
* lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/FactPartition.java
* 
lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java
* lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRange.java
* 
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java


> Look ahead timerange should not look for all finer granularity partitions to 
> be present
> ---
>
> Key: LENS-392
> URL: https://issues.apache.org/jira/browse/LENS-392
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: cube
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
> Fix For: 2.4
>
> Attachments: LENS-392.05.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

2015-09-15 Thread Amareshwari Sriramadasu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38402/#review99170
---



lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java (line 582)


We should make this function configurable and can pass different function 
names from drivers.


- Amareshwari Sriramadasu


On Sept. 15, 2015, 12:48 p.m., Amareshwari Sriramadasu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38402/
> ---
> 
> (Updated Sept. 15, 2015, 12:48 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Changes include :
> - Ability to specify if the destination link in a join chain has 
> many-many-relationship
> - Ability to configure at query to say flatten the fields coming from 
> bridge-table relations
> - Enhance cube query rewriter (mainly the join clause) to add join for bridge 
> table to flatten them
> 
> Pending :
> - Add a test with multi fact query
> - Update documentation with feature addition
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/resources/cube-0.1.xsd 58f68f5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/JoinChain.java 
> e394e20 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 
> bf27b99 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/SchemaGraph.java 
> 1a37e80 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TableReference.java 
> 31fd97b 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
> aab2488 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java 
> c7f1e2a 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 7f56292 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestJoinResolver.java 
> cb63fad 
>   lens-server/src/main/java/org/apache/lens/server/metastore/JAXBUtils.java 
> ed472f6 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java
>  f9be6e6 
> 
> Diff: https://reviews.apache.org/r/38402/diff/
> 
> 
> Testing
> ---
> 
> Added testcases for bridge tables and queries on them.
> 
> All tests in lens-cube module pass. Will update full test results once done.
> 
> 
> Thanks,
> 
> Amareshwari Sriramadasu
> 
>

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

2015-09-15 Thread Amareshwari Sriramadasu



> On Sept. 15, 2015, 2 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java, line 
> > 579
> > 
> >
> > blankness of `bridgeFromClause`

from clause would never be empty. This is for appending userfilter if it is not 
null


> On Sept. 15, 2015, 2 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java, lines 
> > 586-587
> > 
> >
> > only null check needed.

why? we should check if the column are queried from that table as well.


> On Sept. 15, 2015, 2 p.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java, line 
> > 585
> > 
> >
> > Among the approaches: 
> > 
> > 1. Relying on a flag to append "select" or "join"
> > 2. Making a list and later doing "select" + StringUtils.join("join", 
> > list)
> > 
> > the second seems more intuitive. And would be easier to debug later on.

Did not understand the comment fully.

It is not simply prepending select or join to alist. If join needs to be 
converted to select from a bridge table - it becomes a full fledged query with 
select, from, groupby, filters and optionally more joins.


- Amareshwari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38402/#review99024
---


On Sept. 15, 2015, 12:48 p.m., Amareshwari Sriramadasu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38402/
> ---
> 
> (Updated Sept. 15, 2015, 12:48 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Changes include :
> - Ability to specify if the destination link in a join chain has 
> many-many-relationship
> - Ability to configure at query to say flatten the fields coming from 
> bridge-table relations
> - Enhance cube query rewriter (mainly the join clause) to add join for bridge 
> table to flatten them
> 
> Pending :
> - Add a test with multi fact query
> - Update documentation with feature addition
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/resources/cube-0.1.xsd 58f68f5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/JoinChain.java 
> e394e20 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 
> bf27b99 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/SchemaGraph.java 
> 1a37e80 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TableReference.java 
> 31fd97b 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
> aab2488 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java 
> c7f1e2a 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 7f56292 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestJoinResolver.java 
> cb63fad 
>   lens-server/src/main/java/org/apache/lens/server/metastore/JAXBUtils.java 
> ed472f6 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java
>  f9be6e6 
> 
> Diff: https://reviews.apache.org/r/38402/diff/
> 
> 
> Testing
> ---
> 
> Added testcases for bridge tables and queries on them.
> 
> All tests in lens-cube module pass. Will update full test results once done.
> 
> 
> Thanks,
> 
> Amareshwari Sriramadasu
> 
>

[jira] [Updated] (LENS-392) Look ahead timerange should not look for all finer granularity partitions to be present

2015-09-15 Thread Amareshwari Sriramadasu (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated LENS-392:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed.. Thanks [~prongs]

> Look ahead timerange should not look for all finer granularity partitions to 
> be present
> ---
>
> Key: LENS-392
> URL: https://issues.apache.org/jira/browse/LENS-392
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: cube
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
> Fix For: 2.4
>
> Attachments: LENS-392.05.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-392) Look ahead timerange should not look for all finer granularity partitions to be present

2015-09-15 Thread Amareshwari Sriramadasu (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated LENS-392:
-
Summary: Look ahead timerange should not look for all finer granularity 
partitions to be present  (was: Look ahead should take how much time to look 
ahead, instead of just a number)

> Look ahead timerange should not look for all finer granularity partitions to 
> be present
> ---
>
> Key: LENS-392
> URL: https://issues.apache.org/jira/browse/LENS-392
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: cube
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
> Fix For: 2.4
>
> Attachments: LENS-392.05.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

2015-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745792#comment-14745792
 ] 

Hadoop QA commented on LENS-392:


Applied patch: 
[LENS-392.05.patch|https://issues.apache.org/jira/secure/attachment/12755995/LENS-392.05.patch]
 and ran command: mvn clean install. Result: Success. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/61/

> Look ahead should take how much time to look ahead, instead of just a number
> 
>
> Key: LENS-392
> URL: https://issues.apache.org/jira/browse/LENS-392
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: cube
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
> Fix For: 2.4
>
> Attachments: LENS-392.05.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-742) Saved query and parameterization

2015-09-15 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745716#comment-14745716
 ] 

Hudson commented on LENS-742:
-

SUCCESS: Integrated in Lens-Commit #1079 (See 
[https://builds.apache.org/job/Lens-Commit/1079/])
LENS-742 : Adds query feature : Saved query and parameterization (amareshwari: 
rev 4e81ef4ddfef2d8c89c48576e474c2094cbbc56b)
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/exception/ValueEncodeException.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/param/ParameterCollectionTypeEncoder.java
* lens-server/src/test/resources/lens-site.xml
* 
lens-api/src/main/java/org/apache/lens/api/query/save/ParameterParserResponse.java
* lens-server/src/main/java/org/apache/lens/server/query/save/SavedQueryApp.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/SavedQueryService.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/exception/ParameterValueException.java
* lens-server/src/main/java/org/apache/lens/server/query/save/SavedQueryDao.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
* lens-api/src/main/java/org/apache/lens/api/query/save/SavedQuery.java
* src/site/apt/user/index.apt
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/exception/ParameterCollectionException.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/exception/SavedQueryNotFound.java
* lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java
* lens-api/src/main/java/org/apache/lens/api/error/LensCommonErrorCode.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/param/ParameterParser.java
* 
lens-server-api/src/test/java/org/apache/lens/server/api/query/save/TestParameterResolution.java
* lens-server/src/main/resources/lensserver-default.xml
* lens-api/src/main/java/org/apache/lens/api/query/save/Parameter.java
* lens-server/enunciate.xml
* lens-api/src/main/java/org/apache/lens/api/query/save/ParameterDataType.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/exception/MissingParameterException.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/SavedQueryHelper.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/param/ParameterDataTypeEncoder.java
* 
lens-server-api/src/test/java/org/apache/lens/server/api/query/save/TestParameterParser.java
* 
lens-api/src/main/java/org/apache/lens/api/query/save/ParameterCollectionType.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/param/ParameterResolver.java
* src/site/apt/admin/config.apt
* lens-api/src/main/resources/lens-errors.conf
* 
lens-server/src/main/java/org/apache/lens/server/query/save/SavedQueryServiceImpl.java
* 
lens-server/src/main/java/org/apache/lens/server/query/save/SavedQueryResource.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/save/exception/PrivilegeException.java
* 
lens-api/src/main/java/org/apache/lens/api/query/save/ResourceModifiedResponse.java
* lens-api/src/main/java/org/apache/lens/api/query/save/ListResponse.java
* 
lens-server/src/test/java/org/apache/lens/server/query/save/TestSavedQueryService.java


> Saved query and parameterization
> 
>
> Key: LENS-742
> URL: https://issues.apache.org/jira/browse/LENS-742
> Project: Apache Lens
>  Issue Type: New Feature
>Reporter: Amruth S
>Assignee: Amruth S
> Fix For: 2.4
>
> Attachments: LENS-742-REV-8.patch, LENS-742-new.patch
>
>
> - User should be able to
>  * save a query parameterising parts of it.
>  * list all saved queries that are created by him and shared with him.
>  * share a query with other people with privileges.
>  * execute a saved query if his privilege allows (READ, EXECUTE).
>  * clone a saved query (READ).
> - All of these operations should be supported from CLI as well as service
> User flow (from UI)
> Saving
>  -> User authors a query and clicks on SAVE.
>  -> Client calls the helper api /parameters to get info about the parameters 
> existing in the query. 
>  -> For each parameter in the response, these details are obtained from the 
> user (from a pop up UI)
>   * DATA TYPE
>   * COLLECTION TYPE
> (scroll down to the end to see why we would need these details)
>  -> User enters all the details and clicks on SAVE again. A final payload 
> containing the query and all the parameter details is sent.
> {
>   "name": "query_name",
>   "description": "description",
>   "query": "select * from table where col1 = :param1 and col2 in :param2 and 
> col3 = :param3 and col4 in :param4 and col5 = :param5 and col6 in :param6",
>   "parameters": [
> {
>   "name": "param1",
>   "displayName": "Param1",
>

[jira] [Commented] (LENS-753) Queue number for queries

2015-09-15 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745717#comment-14745717
 ] 

Hudson commented on LENS-753:
-

SUCCESS: Integrated in Lens-Commit #1079 (See 
[https://builds.apache.org/job/Lens-Commit/1079/])
LENS-753: Queue number for queries in submitted queue (rajatgupta59: rev 
4addd7b62ed1db8fe1af498a6baf55ee35ad692a)
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java
* lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java
* 
lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java
* lens-cli/src/main/java/org/apache/lens/cli/commands/LensQueryCommands.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
* 
lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java
* 
lens-server-api/src/main/java/org/apache/lens/server/api/driver/DriverQueryStatus.java
* 
lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java
* 
lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java
* 
lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java
* 
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
* 
lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java


> Queue number for queries
> 
>
> Key: LENS-753
> URL: https://issues.apache.org/jira/browse/LENS-753
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Akshay Goyal
>Assignee: Akshay Goyal
> Fix For: 2.4
>
> Attachments: LENS-753.02.patch, LENS-753.03.patch, LENS-753.04.patch
>
>
> This is to add queue number for queries after they were launched, in order to 
> give users an idea of the sequence in which their queries were launched. Or 
> in other words, to give them an idea of how many queries were launched before 
> them which are still competing for resources with their query. 
> Giving queue numbers from waiting query collection may not be correct as they 
> are not consistent. When a candidate query is picked up from waiting state, 
> it is removed from the collection and checked for cost constraints. If it 
> fails the constraints, the query is added back to the waiting query 
> collection. This causes inconsistency in terms of indices of queries within 
> the collection. Because of this inconsistency, user might see their queue 
> number increasing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-733) Add Helper Method for lens-regression

2015-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745688#comment-14745688
 ] 

Hadoop QA commented on LENS-733:


Applied patch: 
[rb37526.patch|https://issues.apache.org/jira/secure/attachment/12756011/rb37526.patch]
 and ran command: mvn clean install. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/59/

> Add Helper Method for lens-regression
> -
>
> Key: LENS-733
> URL: https://issues.apache.org/jira/browse/LENS-733
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: regression
>Reporter: Arshad Matin
>Assignee: Arshad Matin
> Attachments: rb37526.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

2015-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745677#comment-14745677
 ] 

Hadoop QA commented on LENS-392:


Applied patch: 
[LENS-392.05.patch|https://issues.apache.org/jira/secure/attachment/12755995/LENS-392.05.patch]
 and ran command: mvn clean install. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/60/

> Look ahead should take how much time to look ahead, instead of just a number
> 
>
> Key: LENS-392
> URL: https://issues.apache.org/jira/browse/LENS-392
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: cube
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
> Fix For: 2.4
>
> Attachments: LENS-392.05.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37516: LENS-734: Query names in persisted output result files

2015-09-15 Thread Nitin Gupta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37516/
---

(Updated Sept. 15, 2015, 2:41 p.m.)


Review request for lens and Amareshwari Sriramadasu.


Repository: lens


Description
---

Added query name in output files generated through HadoopFileFormatter and 
ZipFileFormatter


Diffs
-

  
lens-query-lib/src/main/java/org/apache/lens/lib/query/HadoopFileFormatter.java 
6f55c79 
  
lens-query-lib/src/main/java/org/apache/lens/lib/query/LensFileOutputFormat.java
 034b88f 
  lens-query-lib/src/main/java/org/apache/lens/lib/query/ZipFileFormatter.java 
a3614fe 
  
lens-query-lib/src/test/java/org/apache/lens/lib/query/TestAbstractFileFormatter.java
 307e075 
  
lens-query-lib/src/test/java/org/apache/lens/lib/query/TestFilePersistentFormatter.java
 7617e06 

Diff: https://reviews.apache.org/r/37516/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules . SUCCESS [  6.581 s]
[INFO] Lens .. SUCCESS [  4.412 s]
[INFO] Lens API .. SUCCESS [ 14.188 s]
[INFO] Lens API for server and extensions  SUCCESS [ 18.008 s]
[INFO] Lens Cube . SUCCESS [05:59 min]
[INFO] Lens DB storage ... SUCCESS [ 16.070 s]
[INFO] Lens Query Library  SUCCESS [ 10.633 s]
[INFO] Lens Hive Driver .. SUCCESS [03:31 min]
[INFO] Lens Driver for JDBC .. SUCCESS [ 30.486 s]
[INFO] Lens Elastic Search Driver  SUCCESS [  9.347 s]
[INFO] Lens Server ... SUCCESS [10:47 min]
[INFO] Lens client ... SUCCESS [ 31.356 s]
[INFO] Lens CLI .. SUCCESS [03:47 min]
[INFO] Lens Examples . SUCCESS [  2.398 s]
[INFO] Lens Distribution . SUCCESS [ 13.890 s]
[INFO] Lens ML Lib ... SUCCESS [01:54 min]
[INFO] Lens ML Ext Distribution .. SUCCESS [ 12.358 s]
[INFO] Lens Regression ... SUCCESS [  2.135 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 28:53 min
[INFO] Finished at: 2015-08-17T03:28:45+05:30
[INFO] Final Memory: 163M/806M
[INFO] 


Thanks,

Nitin Gupta

Re: Review Request 37526: LENS-733 : Add library for lens regression

2015-09-15 Thread Rajat Khandelwal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37526/#review99028
---



lens-regression/src/main/java/org/apache/lens/regression/util/Util.java (line 
401)


JaxbUtils has the same method.


- Rajat Khandelwal


On Sept. 14, 2015, 6:40 p.m., Arshad Matin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37526/
> ---
> 
> (Updated Sept. 14, 2015, 6:40 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add Helper Method for lens-regression
> 
> 
> Diffs
> -
> 
>   lens-regression/pom.xml 48c5b0c8887f5210cccdc6dccb078f90a7398509 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/QueryInventory.java
>  PRE-CREATION 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/LensServerHelper.java
>  8b41dbae3a31fcc86794b751c324609da1c3ac92 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/MetastoreHelper.java
>  aff0e8ea48fc6cb5d6edb31428ab32ef53ab870d 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/QueryHelper.java
>  70cad28fbf0722c34b2fe8c041d0570e86bbabf6 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/ServiceManagerHelper.java
>  b14b00bda417830a50b48c81e5f2cbe2735896b6 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/SessionHelper.java
>  e7bc700c063e914d5c305d6f468f4d7f4ff9cbd4 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/testHelper/BaseTestClass.java
>  2985af46374606bef388e876c28fbcaaa43adc22 
>   
> lens-regression/src/main/java/org/apache/lens/regression/util/AssertUtil.java 
> 62bd1608dbd1ffcbdc931a1f024b5d73a696bb25 
>   
> lens-regression/src/main/java/org/apache/lens/regression/util/HadoopUtil.java 
> PRE-CREATION 
>   lens-regression/src/main/java/org/apache/lens/regression/util/Util.java 
> 0e531e02718ba553868c52d0c0c4ad25a4498e48 
> 
> Diff: https://reviews.apache.org/r/37526/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules .. SUCCESS [  2.903 
> s]
> [INFO] Lens ... SUCCESS [  3.844 
> s]
> [INFO] Lens API ... SUCCESS [ 29.872 
> s]
> [INFO] Lens API for server and extensions . SUCCESS [ 30.727 
> s]
> [INFO] Lens Cube .. SUCCESS [ 52.871 
> s]
> [INFO] Lens DB storage  SUCCESS [ 12.098 
> s]
> [INFO] Lens Query Library . SUCCESS [ 16.444 
> s]
> [INFO] Lens Hive Driver ... SUCCESS [ 23.530 
> s]
> [INFO] Lens Driver for JDBC ... SUCCESS [ 25.832 
> s]
> [INFO] Lens Elastic Search Driver . SUCCESS [ 20.170 
> s]
> [INFO] Lens Server  SUCCESS [01:14 
> min]
> [INFO] Lens client  SUCCESS [ 27.819 
> s]
> [INFO] Lens CLI ... SUCCESS [ 26.542 
> s]
> [INFO] Lens Examples .. SUCCESS [ 21.779 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 16.608 
> s]
> [INFO] Lens ML Lib  SUCCESS [ 35.379 
> s]
> [INFO] Lens ML Ext Distribution ... SUCCESS [  5.195 
> s]
> [INFO] Lens Regression  SUCCESS [ 23.066 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 07:29 min
> [INFO] Finished at: 2015-09-14T16:14:22+05:30
> [INFO] Final Memory: 191M/1455M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Arshad Matin
> 
>

Re: Review Request 37526: LENS-733 : Add library for lens regression

2015-09-15 Thread Rajat Khandelwal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37526/#review99026
---



lens-regression/src/main/java/org/apache/lens/regression/core/helpers/QueryHelper.java
 (line 84)


`LensAPIResult` is a generic class. Is it possible to return 
`LensAPIResult`?



lens-regression/src/main/java/org/apache/lens/regression/core/helpers/QueryHelper.java
 (line 87)


instead of string version of LensConf, is it possible to take a LensConf 
isntance?



lens-regression/src/main/java/org/apache/lens/regression/core/helpers/QueryHelper.java
 (line 129)


casting won't be necessary if the `getObject` return type is `T` instead of 
`Object` as it is now.



lens-regression/src/main/java/org/apache/lens/regression/core/testHelper/BaseTestClass.java
 (lines 47 - 61)


can be replaced by `@Getter` annotations.



lens-regression/src/main/java/org/apache/lens/regression/util/Util.java (line 
166)


return type can be `T` now that explicit cast is happening. Note that it's 
object in line 157.



lens-regression/src/main/java/org/apache/lens/regression/util/Util.java (line 
174)


return type.


- Rajat Khandelwal


On Sept. 14, 2015, 6:40 p.m., Arshad Matin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37526/
> ---
> 
> (Updated Sept. 14, 2015, 6:40 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add Helper Method for lens-regression
> 
> 
> Diffs
> -
> 
>   lens-regression/pom.xml 48c5b0c8887f5210cccdc6dccb078f90a7398509 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/QueryInventory.java
>  PRE-CREATION 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/LensServerHelper.java
>  8b41dbae3a31fcc86794b751c324609da1c3ac92 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/MetastoreHelper.java
>  aff0e8ea48fc6cb5d6edb31428ab32ef53ab870d 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/QueryHelper.java
>  70cad28fbf0722c34b2fe8c041d0570e86bbabf6 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/ServiceManagerHelper.java
>  b14b00bda417830a50b48c81e5f2cbe2735896b6 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/SessionHelper.java
>  e7bc700c063e914d5c305d6f468f4d7f4ff9cbd4 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/testHelper/BaseTestClass.java
>  2985af46374606bef388e876c28fbcaaa43adc22 
>   
> lens-regression/src/main/java/org/apache/lens/regression/util/AssertUtil.java 
> 62bd1608dbd1ffcbdc931a1f024b5d73a696bb25 
>   
> lens-regression/src/main/java/org/apache/lens/regression/util/HadoopUtil.java 
> PRE-CREATION 
>   lens-regression/src/main/java/org/apache/lens/regression/util/Util.java 
> 0e531e02718ba553868c52d0c0c4ad25a4498e48 
> 
> Diff: https://reviews.apache.org/r/37526/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules .. SUCCESS [  2.903 
> s]
> [INFO] Lens ... SUCCESS [  3.844 
> s]
> [INFO] Lens API ... SUCCESS [ 29.872 
> s]
> [INFO] Lens API for server and extensions . SUCCESS [ 30.727 
> s]
> [INFO] Lens Cube .. SUCCESS [ 52.871 
> s]
> [INFO] Lens DB storage  SUCCESS [ 12.098 
> s]
> [INFO] Lens Query Library . SUCCESS [ 16.444 
> s]
> [INFO] Lens Hive Driver ... SUCCESS [ 23.530 
> s]
> [INFO] Lens Driver for JDBC ... SUCCESS [ 25.832 
> s]
> [INFO] Lens Elastic Search Driver . SUCCESS [ 20.170 
> s]
> [INFO] Lens Server  SUCCESS [01:14 
> min]
> [INFO] Lens client  SUCCESS [ 27.819 
> s]
> [INFO] Lens CLI ... SUCCESS [ 26.542 
> s]
> [INFO

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

2015-09-15 Thread Rajat Khandelwal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38402/#review99024
---



lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java (line 556)


Let's check blankness of `bridgeFilterClause` instead of checking blankness 
of the only thing appended to `bridgeFilterClause`.



lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java (line 572)


blankness of `bridgeFromClause`



lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java (line 578)


Among the approaches: 

1. Relying on a flag to append "select" or "join"
2. Making a list and later doing "select" + StringUtils.join("join", list)

the second seems more intuitive. And would be easier to debug later on.



lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java (lines 579 
- 580)


only null check needed.


- Rajat Khandelwal


On Sept. 15, 2015, 6:18 p.m., Amareshwari Sriramadasu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38402/
> ---
> 
> (Updated Sept. 15, 2015, 6:18 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Changes include :
> - Ability to specify if the destination link in a join chain has 
> many-many-relationship
> - Ability to configure at query to say flatten the fields coming from 
> bridge-table relations
> - Enhance cube query rewriter (mainly the join clause) to add join for bridge 
> table to flatten them
> 
> Pending :
> - Add a test with multi fact query
> - Update documentation with feature addition
> 
> 
> Diffs
> -
> 
>   lens-api/src/main/resources/cube-0.1.xsd 58f68f5 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/JoinChain.java 
> e394e20 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 
> bf27b99 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/SchemaGraph.java 
> 1a37e80 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TableReference.java 
> 31fd97b 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
> aab2488 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java 
> c7f1e2a 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 7f56292 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestJoinResolver.java 
> cb63fad 
>   lens-server/src/main/java/org/apache/lens/server/metastore/JAXBUtils.java 
> ed472f6 
>   
> lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java
>  f9be6e6 
> 
> Diff: https://reviews.apache.org/r/38402/diff/
> 
> 
> Testing
> ---
> 
> Added testcases for bridge tables and queries on them.
> 
> All tests in lens-cube module pass. Will update full test results once done.
> 
> 
> Thanks,
> 
> Amareshwari Sriramadasu
> 
>

[jira] [Updated] (LENS-733) Add Helper Method for lens-regression

2015-09-15 Thread Raghavendra Singh (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raghavendra Singh updated LENS-733:
---
Assignee: Arshad Matin

> Add Helper Method for lens-regression
> -
>
> Key: LENS-733
> URL: https://issues.apache.org/jira/browse/LENS-733
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: regression
>Reporter: Arshad Matin
>Assignee: Arshad Matin
> Attachments: rb37526.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-733) Add Helper Method for lens-regression

2015-09-15 Thread Arshad Matin (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arshad Matin updated LENS-733:
--
Attachment: rb37526.patch

> Add Helper Method for lens-regression
> -
>
> Key: LENS-733
> URL: https://issues.apache.org/jira/browse/LENS-733
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: regression
>Reporter: Arshad Matin
> Attachments: rb37526.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-733) Add Helper Method for lens-regression

2015-09-15 Thread Arshad Matin (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arshad Matin updated LENS-733:
--
Status: Patch Available  (was: Open)

> Add Helper Method for lens-regression
> -
>
> Key: LENS-733
> URL: https://issues.apache.org/jira/browse/LENS-733
> Project: Apache Lens
>  Issue Type: Sub-task
>  Components: regression
>Reporter: Arshad Matin
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 37526: LENS-733 : Add library for lens regression

2015-09-15 Thread Raghavendra Singh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37526/#review99023
---

Ship it!


Ship It!

- Raghavendra Singh


On Sept. 14, 2015, 1:10 p.m., Arshad Matin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37526/
> ---
> 
> (Updated Sept. 14, 2015, 1:10 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add library for lens regression
> 
> 
> LENS-733 : Add Helper Method for lens-regression
> 
> 
> Diffs
> -
> 
>   lens-regression/pom.xml 48c5b0c8887f5210cccdc6dccb078f90a7398509 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/constants/QueryInventory.java
>  PRE-CREATION 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/LensServerHelper.java
>  8b41dbae3a31fcc86794b751c324609da1c3ac92 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/MetastoreHelper.java
>  aff0e8ea48fc6cb5d6edb31428ab32ef53ab870d 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/QueryHelper.java
>  70cad28fbf0722c34b2fe8c041d0570e86bbabf6 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/ServiceManagerHelper.java
>  b14b00bda417830a50b48c81e5f2cbe2735896b6 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/helpers/SessionHelper.java
>  e7bc700c063e914d5c305d6f468f4d7f4ff9cbd4 
>   
> lens-regression/src/main/java/org/apache/lens/regression/core/testHelper/BaseTestClass.java
>  2985af46374606bef388e876c28fbcaaa43adc22 
>   
> lens-regression/src/main/java/org/apache/lens/regression/util/AssertUtil.java 
> 62bd1608dbd1ffcbdc931a1f024b5d73a696bb25 
>   
> lens-regression/src/main/java/org/apache/lens/regression/util/HadoopUtil.java 
> PRE-CREATION 
>   lens-regression/src/main/java/org/apache/lens/regression/util/Util.java 
> 0e531e02718ba553868c52d0c0c4ad25a4498e48 
> 
> Diff: https://reviews.apache.org/r/37526/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules .. SUCCESS [  2.903 
> s]
> [INFO] Lens ... SUCCESS [  3.844 
> s]
> [INFO] Lens API ... SUCCESS [ 29.872 
> s]
> [INFO] Lens API for server and extensions . SUCCESS [ 30.727 
> s]
> [INFO] Lens Cube .. SUCCESS [ 52.871 
> s]
> [INFO] Lens DB storage  SUCCESS [ 12.098 
> s]
> [INFO] Lens Query Library . SUCCESS [ 16.444 
> s]
> [INFO] Lens Hive Driver ... SUCCESS [ 23.530 
> s]
> [INFO] Lens Driver for JDBC ... SUCCESS [ 25.832 
> s]
> [INFO] Lens Elastic Search Driver . SUCCESS [ 20.170 
> s]
> [INFO] Lens Server  SUCCESS [01:14 
> min]
> [INFO] Lens client  SUCCESS [ 27.819 
> s]
> [INFO] Lens CLI ... SUCCESS [ 26.542 
> s]
> [INFO] Lens Examples .. SUCCESS [ 21.779 
> s]
> [INFO] Lens Distribution .. SUCCESS [ 16.608 
> s]
> [INFO] Lens ML Lib  SUCCESS [ 35.379 
> s]
> [INFO] Lens ML Ext Distribution ... SUCCESS [  5.195 
> s]
> [INFO] Lens Regression  SUCCESS [ 23.066 
> s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 07:29 min
> [INFO] Finished at: 2015-09-14T16:14:22+05:30
> [INFO] Final Memory: 191M/1455M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Arshad Matin
> 
>

[jira] [Updated] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

2015-09-15 Thread Rajat Khandelwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajat Khandelwal updated LENS-392:
--
Status: Patch Available  (was: In Progress)

> Look ahead should take how much time to look ahead, instead of just a number
> 
>
> Key: LENS-392
> URL: https://issues.apache.org/jira/browse/LENS-392
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: cube
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
> Fix For: 2.4
>
> Attachments: LENS-392.05.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

2015-09-15 Thread Rajat Khandelwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajat Khandelwal updated LENS-392:
--
Attachment: LENS-392.05.patch

> Look ahead should take how much time to look ahead, instead of just a number
> 
>
> Key: LENS-392
> URL: https://issues.apache.org/jira/browse/LENS-392
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: cube
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
> Fix For: 2.4
>
> Attachments: LENS-392.05.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

2015-09-15 Thread Rajat Khandelwal (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745394#comment-14745394
 ] 

Rajat Khandelwal commented on LENS-392:
---

Taking patch from reviewboard and attaching

> Look ahead should take how much time to look ahead, instead of just a number
> 
>
> Key: LENS-392
> URL: https://issues.apache.org/jira/browse/LENS-392
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: cube
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
> Fix For: 2.4
>
> Attachments: LENS-392.05.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38212: LENS-392: Look ahead not working when not all process time partitions on lower granularity are present

2015-09-15 Thread Rajat Khandelwal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38212/
---

(Updated Sept. 15, 2015, 6:22 p.m.)


Review request for lens.


Bugs: LENS-392
https://issues.apache.org/jira/browse/LENS-392


Repository: lens


Description
---

e.g. for a monthly query with look ahead = 1, if next month partition is not 
present, it looks for process time partitions for all days of the next month. 
And all of them might not be present. So we should proceed with whichever ones 
are present.


Diffs (updated)
-

  lens-cube/src/main/java/org/apache/lens/cube/metadata/FactPartition.java 
fc2d85b89e9809a61c982ca78a7d18ffacee76f6 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java 
d52f1686254ab0b9c977af84731888778edacb8a 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRange.java 
f5f8d4c8ec4177b2d83f5f567386a0422958c69d 
  
lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java
 1b9a44a6c8ac0280bc79d7e62753d692e8e4cd57 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java 
2caea569a6af1f717bef49491b024441808b5448 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
58d0fa7b3d4b4ffd503b3606c3a42d4713d7ef11 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
7f5629200f82f92e00fccd423097666383b06e18 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
7e5184c472b230cbee0c633191283a0368a82949 

Diff: https://reviews.apache.org/r/38212/diff/


Testing
---

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] Lens Checkstyle Rules . SUCCESS [2.285s]
[INFO] Lens .. SUCCESS [2.853s]
[INFO] Lens API .. SUCCESS [21.064s]
[INFO] Lens API for server and extensions  SUCCESS [20.780s]
[INFO] Lens Cube . SUCCESS [5:27.193s]
[INFO] Lens DB storage ... SUCCESS [18.947s]
[INFO] Lens Query Library  SUCCESS [16.214s]
[INFO] Lens Hive Driver .. SUCCESS [2:54.010s]
[INFO] Lens Driver for JDBC .. SUCCESS [36.208s]
[INFO] Lens Elastic Search Driver  SUCCESS [17.168s]
[INFO] Lens Server ... SUCCESS [5:37.934s]
[INFO] Lens client ... SUCCESS [38.626s]
[INFO] Lens CLI .. SUCCESS [2:39.371s]
[INFO] Lens Examples . SUCCESS [8.829s]
[INFO] Lens Distribution . SUCCESS [9.163s]
[INFO] Lens ML Lib ... SUCCESS [1:21.146s]
[INFO] Lens ML Ext Distribution .. SUCCESS [2.040s]
[INFO] Lens Regression ... SUCCESS [10.560s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 21:25.417s
[INFO] Finished at: Mon Sep 14 12:59:16 UTC 2015
[INFO] Final Memory: 194M/2078M
[INFO] 

Added a test case


Thanks,

Rajat Khandelwal

Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

2015-09-15 Thread Amareshwari Sriramadasu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38402/
---

Review request for lens.


Repository: lens


Description
---

Changes include :
- Ability to specify if the destination link in a join chain has 
many-many-relationship
- Ability to configure at query to say flatten the fields coming from 
bridge-table relations
- Enhance cube query rewriter (mainly the join clause) to add join for bridge 
table to flatten them

Pending :
- Add a test with multi fact query
- Update documentation with feature addition


Diffs
-

  lens-api/src/main/resources/cube-0.1.xsd 58f68f5 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/JoinChain.java e394e20 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreUtil.java 
bf27b99 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/SchemaGraph.java 
1a37e80 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/TableReference.java 
31fd97b 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
aab2488 
  lens-cube/src/main/java/org/apache/lens/cube/parse/JoinResolver.java c7f1e2a 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 7f56292 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestJoinResolver.java 
cb63fad 
  lens-server/src/main/java/org/apache/lens/server/metastore/JAXBUtils.java 
ed472f6 
  
lens-server/src/test/java/org/apache/lens/server/metastore/TestMetastoreService.java
 f9be6e6 

Diff: https://reviews.apache.org/r/38402/diff/


Testing
---

Added testcases for bridge tables and queries on them.

All tests in lens-cube module pass. Will update full test results once done.


Thanks,

Amareshwari Sriramadasu

[jira] [Updated] (LENS-742) Saved query and parameterization

2015-09-15 Thread Amareshwari Sriramadasu (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amareshwari Sriramadasu updated LENS-742:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed. Thanks [~amrk7]

> Saved query and parameterization
> 
>
> Key: LENS-742
> URL: https://issues.apache.org/jira/browse/LENS-742
> Project: Apache Lens
>  Issue Type: New Feature
>Reporter: Amruth S
>Assignee: Amruth S
> Fix For: 2.4
>
> Attachments: LENS-742-REV-8.patch, LENS-742-new.patch
>
>
> - User should be able to
>  * save a query parameterising parts of it.
>  * list all saved queries that are created by him and shared with him.
>  * share a query with other people with privileges.
>  * execute a saved query if his privilege allows (READ, EXECUTE).
>  * clone a saved query (READ).
> - All of these operations should be supported from CLI as well as service
> User flow (from UI)
> Saving
>  -> User authors a query and clicks on SAVE.
>  -> Client calls the helper api /parameters to get info about the parameters 
> existing in the query. 
>  -> For each parameter in the response, these details are obtained from the 
> user (from a pop up UI)
>   * DATA TYPE
>   * COLLECTION TYPE
> (scroll down to the end to see why we would need these details)
>  -> User enters all the details and clicks on SAVE again. A final payload 
> containing the query and all the parameter details is sent.
> {
>   "name": "query_name",
>   "description": "description",
>   "query": "select * from table where col1 = :param1 and col2 in :param2 and 
> col3 = :param3 and col4 in :param4 and col5 = :param5 and col6 in :param6",
>   "parameters": [
> {
>   "name": "param1",
>   "displayName": "Param1",
>   "defaultValue": "x",
>   "dataType": "STRING",
>   "collectionType": "SINGLE"
> },
> {
>   "name": "param2",
>   "displayName": "Param2",
>   "defaultValue": "x",
>   "dataType": "STRING",
>   "collectionType": "MULTIPLE"
> },
> {
>   "name": "param3",
>   "displayName": "Param3",
>   "defaultValue": "1.0",
>   "dataType": "NUMBER",
>   "collectionType": "SINGLE"
> },
> {
>   "name": "param4",
>   "displayName": "Param4",
>   "defaultValue": "1.0",
>   "dataType": "NUMBER",
>   "collectionType": "MULTIPLE"
> },
> {
>   "name": "param5",
>   "displayName": "Param5",
>   "defaultValue": "true",
>   "dataType": "BOOLEAN",
>   "collectionType": "SINGLE"
> },
> {
>   "name": "param6",
>   "displayName": "Param6",
>   "defaultValue": "true",
>   "dataType": "BOOLEAN",
>   "collectionType": "MULTIPLE"
> }
>   ]
> }
> Execution
> - User selects a saved query from a list of saved queries and clicks on RUN
> - A pop is shown asking for parameter values
> - User enters all param values (client side validation happens on datatype 
> and collection type) and clicks RUN. A query handle is returned.
> User flow - From CLI
> Saving
> - create savedquery 'path to the final json payload'
> Executing
> - savedquery execute  
> Why do we need these DATATYPE and COLLECTION TYPE details in parameters??
> A parameterised query would look like this
> select col1, col2 from table where col1 = :param1 and col in :param2 limit 
> :param3
> param1, param2, param3 are parameters here. 
> While resolving values for the parameters datatype would be required else SQL 
> injection could happen.
> Eg. param1 value could be : 'val2 or param1 like '%''. We would not know if 
> we can single quote it unless its a String.
> Data types could be inferred from the native tables but it is not applicable 
> to all the drivers. 
> Also depending on the operator the parameter is associated with, the 
> collection type could differ. Eg. IN versus =. Ideally this intelligence 
> should exist with the code. This would require a great deal of query parsing. 
> (The existing HQLParser - ASTNode is not able to parse an ANSI compatible 
> parameterised query)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-742) Saved query and parameterization

2015-09-15 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745339#comment-14745339
 ] 

Hadoop QA commented on LENS-742:


Applied patch: 
[LENS-742-new.patch|https://issues.apache.org/jira/secure/attachment/12755903/LENS-742-new.patch]
 and ran command: mvn clean install. Result: Success. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/58/

> Saved query and parameterization
> 
>
> Key: LENS-742
> URL: https://issues.apache.org/jira/browse/LENS-742
> Project: Apache Lens
>  Issue Type: New Feature
>Reporter: Amruth S
>Assignee: Amruth S
> Fix For: 2.4
>
> Attachments: LENS-742-REV-8.patch, LENS-742-new.patch
>
>
> - User should be able to
>  * save a query parameterising parts of it.
>  * list all saved queries that are created by him and shared with him.
>  * share a query with other people with privileges.
>  * execute a saved query if his privilege allows (READ, EXECUTE).
>  * clone a saved query (READ).
> - All of these operations should be supported from CLI as well as service
> User flow (from UI)
> Saving
>  -> User authors a query and clicks on SAVE.
>  -> Client calls the helper api /parameters to get info about the parameters 
> existing in the query. 
>  -> For each parameter in the response, these details are obtained from the 
> user (from a pop up UI)
>   * DATA TYPE
>   * COLLECTION TYPE
> (scroll down to the end to see why we would need these details)
>  -> User enters all the details and clicks on SAVE again. A final payload 
> containing the query and all the parameter details is sent.
> {
>   "name": "query_name",
>   "description": "description",
>   "query": "select * from table where col1 = :param1 and col2 in :param2 and 
> col3 = :param3 and col4 in :param4 and col5 = :param5 and col6 in :param6",
>   "parameters": [
> {
>   "name": "param1",
>   "displayName": "Param1",
>   "defaultValue": "x",
>   "dataType": "STRING",
>   "collectionType": "SINGLE"
> },
> {
>   "name": "param2",
>   "displayName": "Param2",
>   "defaultValue": "x",
>   "dataType": "STRING",
>   "collectionType": "MULTIPLE"
> },
> {
>   "name": "param3",
>   "displayName": "Param3",
>   "defaultValue": "1.0",
>   "dataType": "NUMBER",
>   "collectionType": "SINGLE"
> },
> {
>   "name": "param4",
>   "displayName": "Param4",
>   "defaultValue": "1.0",
>   "dataType": "NUMBER",
>   "collectionType": "MULTIPLE"
> },
> {
>   "name": "param5",
>   "displayName": "Param5",
>   "defaultValue": "true",
>   "dataType": "BOOLEAN",
>   "collectionType": "SINGLE"
> },
> {
>   "name": "param6",
>   "displayName": "Param6",
>   "defaultValue": "true",
>   "dataType": "BOOLEAN",
>   "collectionType": "MULTIPLE"
> }
>   ]
> }
> Execution
> - User selects a saved query from a list of saved queries and clicks on RUN
> - A pop is shown asking for parameter values
> - User enters all param values (client side validation happens on datatype 
> and collection type) and clicks RUN. A query handle is returned.
> User flow - From CLI
> Saving
> - create savedquery 'path to the final json payload'
> Executing
> - savedquery execute  
> Why do we need these DATATYPE and COLLECTION TYPE details in parameters??
> A parameterised query would look like this
> select col1, col2 from table where col1 = :param1 and col in :param2 limit 
> :param3
> param1, param2, param3 are parameters here. 
> While resolving values for the parameters datatype would be required else SQL 
> injection could happen.
> Eg. param1 value could be : 'val2 or param1 like '%''. We would not know if 
> we can single quote it unless its a String.
> Data types could be inferred from the native tables but it is not applicable 
> to all the drivers. 
> Also depending on the operator the parameter is associated with, the 
> collection type could differ. Eg. IN versus =. Ideally this intelligence 
> should exist with the code. This would require a great deal of query parsing. 
> (The existing HQLParser - ASTNode is not able to parse an ANSI compatible 
> parameterised query)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-753) Queue number for queries

2015-09-15 Thread Rajat Khandelwal (JIRA)


 [ 
https://issues.apache.org/jira/browse/LENS-753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajat Khandelwal updated LENS-753:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Queue number for queries
> 
>
> Key: LENS-753
> URL: https://issues.apache.org/jira/browse/LENS-753
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Akshay Goyal
>Assignee: Akshay Goyal
> Fix For: 2.4
>
> Attachments: LENS-753.02.patch, LENS-753.03.patch, LENS-753.04.patch
>
>
> This is to add queue number for queries after they were launched, in order to 
> give users an idea of the sequence in which their queries were launched. Or 
> in other words, to give them an idea of how many queries were launched before 
> them which are still competing for resources with their query. 
> Giving queue numbers from waiting query collection may not be correct as they 
> are not consistent. When a candidate query is picked up from waiting state, 
> it is removed from the collection and checked for cost constraints. If it 
> fails the constraints, the query is added back to the waiting query 
> collection. This causes inconsistency in terms of indices of queries within 
> the collection. Because of this inconsistency, user might see their queue 
> number increasing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-753) Queue number for queries

2015-09-15 Thread Rajat Khandelwal (JIRA)


[ 
https://issues.apache.org/jira/browse/LENS-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745269#comment-14745269
 ] 

Rajat Khandelwal commented on LENS-753:
---

Committed. Thanks [~akshaygoyal]

> Queue number for queries
> 
>
> Key: LENS-753
> URL: https://issues.apache.org/jira/browse/LENS-753
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Akshay Goyal
>Assignee: Akshay Goyal
> Fix For: 2.4
>
> Attachments: LENS-753.02.patch, LENS-753.03.patch, LENS-753.04.patch
>
>
> This is to add queue number for queries after they were launched, in order to 
> give users an idea of the sequence in which their queries were launched. Or 
> in other words, to give them an idea of how many queries were launched before 
> them which are still competing for resources with their query. 
> Giving queue numbers from waiting query collection may not be correct as they 
> are not consistent. When a candidate query is picked up from waiting state, 
> it is removed from the collection and checked for cost constraints. If it 
> fails the constraints, the query is added back to the waiting query 
> collection. This causes inconsistency in terms of indices of queries within 
> the collection. Because of this inconsistency, user might see their queue 
> number increasing.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 38212: LENS-392: Look ahead not working when not all process time partitions on lower granularity are present

2015-09-15 Thread Amareshwari Sriramadasu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38212/#review99009
---

Ship it!


Ship It!

- Amareshwari Sriramadasu


On Sept. 14, 2015, 1:05 p.m., Rajat Khandelwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38212/
> ---
> 
> (Updated Sept. 14, 2015, 1:05 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-392
> https://issues.apache.org/jira/browse/LENS-392
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> e.g. for a monthly query with look ahead = 1, if next month partition is not 
> present, it looks for process time partitions for all days of the next month. 
> And all of them might not be present. So we should proceed with whichever 
> ones are present.
> 
> 
> Diffs
> -
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/FactPartition.java 
> fc2d85b89e9809a61c982ca78a7d18ffacee76f6 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartition.java 
> d52f1686254ab0b9c977af84731888778edacb8a 
>   
> lens-cube/src/main/java/org/apache/lens/cube/metadata/TimePartitionRange.java 
> f5f8d4c8ec4177b2d83f5f567386a0422958c69d 
>   
> lens-cube/src/main/java/org/apache/lens/cube/metadata/timeline/RangesPartitionTimeline.java
>  1b9a44a6c8ac0280bc79d7e62753d692e8e4cd57 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/AbridgedTimeRangeWriter.java
>  2caea569a6af1f717bef49491b024441808b5448 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
> 58d0fa7b3d4b4ffd503b3606c3a42d4713d7ef11 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 7f5629200f82f92e00fccd423097666383b06e18 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
> 7e5184c472b230cbee0c633191283a0368a82949 
> 
> Diff: https://reviews.apache.org/r/38212/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO] 
> [INFO] Lens Checkstyle Rules . SUCCESS [2.285s]
> [INFO] Lens .. SUCCESS [2.853s]
> [INFO] Lens API .. SUCCESS [21.064s]
> [INFO] Lens API for server and extensions  SUCCESS [20.780s]
> [INFO] Lens Cube . SUCCESS [5:27.193s]
> [INFO] Lens DB storage ... SUCCESS [18.947s]
> [INFO] Lens Query Library  SUCCESS [16.214s]
> [INFO] Lens Hive Driver .. SUCCESS [2:54.010s]
> [INFO] Lens Driver for JDBC .. SUCCESS [36.208s]
> [INFO] Lens Elastic Search Driver  SUCCESS [17.168s]
> [INFO] Lens Server ... SUCCESS [5:37.934s]
> [INFO] Lens client ... SUCCESS [38.626s]
> [INFO] Lens CLI .. SUCCESS [2:39.371s]
> [INFO] Lens Examples . SUCCESS [8.829s]
> [INFO] Lens Distribution . SUCCESS [9.163s]
> [INFO] Lens ML Lib ... SUCCESS [1:21.146s]
> [INFO] Lens ML Ext Distribution .. SUCCESS [2.040s]
> [INFO] Lens Regression ... SUCCESS [10.560s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 21:25.417s
> [INFO] Finished at: Mon Sep 14 12:59:16 UTC 2015
> [INFO] Final Memory: 194M/2078M
> [INFO] 
> 
> 
> Added a test case
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>

[jira] [Created] (LENS-775) NPE on closing HadoopFileFormatter

[jira] [Created] (LENS-774) Unable to override "lens.server.session.expiry.service.interval.secs" because of spelling mistake

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

[jira] [Commented] (LENS-733) Add Helper Method for lens-regression

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

[jira] [Commented] (LENS-392) Look ahead timerange should not look for all finer granularity partitions to be present

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

[jira] [Updated] (LENS-392) Look ahead timerange should not look for all finer granularity partitions to be present

[jira] [Updated] (LENS-392) Look ahead timerange should not look for all finer granularity partitions to be present

[jira] [Commented] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

[jira] [Commented] (LENS-742) Saved query and parameterization

[jira] [Commented] (LENS-753) Queue number for queries

[jira] [Commented] (LENS-733) Add Helper Method for lens-regression

[jira] [Commented] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

Re: Review Request 37516: LENS-734: Query names in persisted output result files

Re: Review Request 37526: LENS-733 : Add library for lens regression

Re: Review Request 37526: LENS-733 : Add library for lens regression

Re: Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

[jira] [Updated] (LENS-733) Add Helper Method for lens-regression

[jira] [Updated] (LENS-733) Add Helper Method for lens-regression

[jira] [Updated] (LENS-733) Add Helper Method for lens-regression

Re: Review Request 37526: LENS-733 : Add library for lens regression

[jira] [Updated] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

[jira] [Updated] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

[jira] [Commented] (LENS-392) Look ahead should take how much time to look ahead, instead of just a number

Re: Review Request 38212: LENS-392: Look ahead not working when not all process time partitions on lower granularity are present

Review Request 38402: LENS-752 : Support flattening of columns selected through bridge-tables(many-to-many relationships)

[jira] [Updated] (LENS-742) Saved query and parameterization

[jira] [Commented] (LENS-742) Saved query and parameterization

[jira] [Updated] (LENS-753) Queue number for queries

[jira] [Commented] (LENS-753) Queue number for queries

Re: Review Request 38212: LENS-392: Look ahead not working when not all process time partitions on lower granularity are present

33 matches

Site Navigation

Mail list logo

Footer information