[jira] [Updated] (LENS-1371) Add a substring based retry policy for failures

2016-11-08 Thread Lavkesh Lahngir (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lavkesh Lahngir updated LENS-1371:
--
Attachment: 0001-Retry-policy.patch

> Add a substring based retry policy for failures
> ---
>
> Key: LENS-1371
> URL: https://issues.apache.org/jira/browse/LENS-1371
> Project: Apache Lens
>  Issue Type: New Feature
>Reporter: Lavkesh Lahngir
>Assignee: Lavkesh Lahngir
> Attachments: 0001-Retry-patch.patch, 0001-Retry-policy.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

2016-11-08 Thread Sushil Mohanty 


> On Oct. 26, 2016, 6:01 a.m., Amareshwari Sriramadasu wrote:
> > lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java,
> >  line 169
> > 
> >
> > not sure how its corresponding local jar file is added to classloader. 
> > If hdfs jar is getting added, it wont work.
> > 
> > might have to pass the corresponding local jar as the location value 
> > for ResourceEntry.
> 
> Sushil Mohanty wrote:
> In case of base dir is hdfs, first copying the files to local directory 
> and setting local as base directory and respective jar files are picked from 
> here.
> 
>if 
> (resTopDirPath.getFileSystem(getHiveConf()).getScheme().equals("hdfs")) {
>   localResTopDir = 
> getHiveConf().get(LensConfConstants.DATABASE_LOCAL_RESOURCE_DIR,
>   LensConfConstants.DEFAULT_LOCAL_DATABASE_RESOURCE_DIR);
>   downloadJarFilesFromHDFS(resTopDir, localResTopDir);
>   resTopDir = localResTopDir;
> }
> 
> Amareshwari Sriramadasu wrote:
> We cannot change resTopDir which is hdfs to a local path.

Addressed.


- Sushil


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review153851
---


On Nov. 3, 2016, 5:31 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Nov. 3, 2016, 5:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
> 7fc8438 
>   lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f9db2a 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
>  20ec686 
>   lens-server/pom.xml d24dc1e 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
>  21e2a62 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   
> lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
> 63eea63 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/main/resources/lensserver-default.xml a00048b 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
>  PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml d96659f 
>   pom.xml 29c59d3 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver .

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

2016-11-08 Thread Sushil Mohanty 


> On Nov. 4, 2016, 6:35 a.m., Amareshwari Sriramadasu wrote:
> > lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java, 
> > line 97
> > 
> >
> > Database does not exists, but jar can be added?

It was being added to the default db. Changed it to work with specified db.


- Sushil


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/#review154869
---


On Nov. 3, 2016, 5:31 p.m., Sushil Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51964/
> ---
> 
> (Updated Nov. 3, 2016, 5:31 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-317
> https://issues.apache.org/jira/browse/LENS-317
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Server side api call to update database jar without restarting lens server. 
> More details can be found in LENS-317.
> 
> 
> Diffs
> -
> 
>   
> lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
> c6ae02b 
>   lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
> 7fc8438 
>   lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
>   lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
>   lens-client/src/main/java/org/apache/lens/client/LensConnection.java 
> bb15b23 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
>  8f9db2a 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
>  28b9d22 
>   
> lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
>  20ec686 
>   lens-server/pom.xml d24dc1e 
>   
> lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
>  8b10d1d 
>   
> lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
>  511e4cf 
>   
> lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
>  21e2a62 
>   
> lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
> 34c901c 
>   
> lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
> 63eea63 
>   lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
> e48eab4 
>   lens-server/src/main/resources/lensserver-default.xml a00048b 
>   lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 
> 7cccf30 
>   lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
> 67cee57 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
>  2bc3712 
>   
> lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
>  PRE-CREATION 
>   lens-server/src/test/resources/lens-site.xml d96659f 
>   pom.xml 29c59d3 
> 
> Diff: https://reviews.apache.org/r/51964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install.
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
> [INFO] Lens .. SUCCESS [7.286s]
> [INFO] Lens API .. SUCCESS [31.546s]
> [INFO] Lens API for server and extensions  SUCCESS [25.681s]
> [INFO] Lens Cube . SUCCESS 
> [17:55.255s]
> [INFO] Lens DB storage ... SUCCESS [25.650s]
> [INFO] Lens Query Library  SUCCESS [21.646s]
> [INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
> [INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
> [INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
> [INFO] Lens Server ... SUCCESS 
> [18:19.588s]
> [INFO] Lens client ... SUCCESS [2:02.590s]
> [INFO] Lens CLI .. SUCCESS [1:54.985s]
> [INFO] Lens Examples . SUCCESS [13.790s]
> [INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
> [INFO] Lens Distribution . SUCCESS [24.869s]
> [INFO] Lens ML Lib ... SUCCESS [1:54.426s]
> [INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
> [INFO] Lens Regression ... SUCCESS [20.677s]
> [INFO] Lens UI ... SUCCESS [19.454s]
> [INFO] 
> --

Re: Review Request 51964: LENS-317: Server side calls for updating database jars

2016-11-08 Thread Sushil Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51964/
---

(Updated Nov. 8, 2016, 10:18 a.m.)


Review request for lens.


Changes
---

- Addressed review.


Bugs: LENS-317
https://issues.apache.org/jira/browse/LENS-317


Repository: lens


Description
---

Server side api call to update database jar without restarting lens server. 
More details can be found in LENS-317.


Diffs (updated)
-

  lens-cli/pom.xml 8e5e3eb 
  lens-cli/src/main/java/org/apache/lens/cli/commands/LensDatabaseCommands.java 
c6ae02b 
  lens-cli/src/test/java/org/apache/lens/cli/TestLensDatabaseCommands.java 
7fc8438 
  lens-cli/src/test/resources/schema/jars/test_db_resource.jar PRE-CREATION 
  lens-client/src/main/java/org/apache/lens/client/LensClient.java e936798 
  lens-client/src/main/java/org/apache/lens/client/LensConnection.java bb15b23 
  
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java 
8f9db2a 
  
lens-server-api/src/main/java/org/apache/lens/server/api/metastore/CubeMetastoreService.java
 28b9d22 
  
lens-server-api/src/main/java/org/apache/lens/server/api/session/SessionService.java
 20ec686 
  lens-server/pom.xml d24dc1e 
  
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
 8b10d1d 
  
lens-server/src/main/java/org/apache/lens/server/session/DatabaseResourceService.java
 511e4cf 
  
lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
 21e2a62 
  lens-server/src/main/java/org/apache/lens/server/session/LensSessionImpl.java 
34c901c 
  lens-server/src/main/java/org/apache/lens/server/session/SessionResource.java 
63eea63 
  lens-server/src/main/java/org/apache/lens/server/util/ScannedPaths.java 
e48eab4 
  lens-server/src/main/resources/lensserver-default.xml a00048b 
  lens-server/src/test/java/org/apache/lens/server/LensJerseyTest.java 7cccf30 
  lens-server/src/test/java/org/apache/lens/server/LensServerTestUtil.java 
67cee57 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseResourceService.java
 2bc3712 
  
lens-server/src/test/java/org/apache/lens/server/session/TestDatabaseService.java
 PRE-CREATION 
  
lens-server/src/test/java/org/apache/lens/server/session/TestSessionClassLoaders.java
 d66de4c 
  lens-server/src/test/resources/lens-site.xml d96659f 
  pom.xml 29c59d3 

Diff: https://reviews.apache.org/r/51964/diff/


Testing
---

mvn clean install.

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules . SUCCESS [3.302s]
[INFO] Lens .. SUCCESS [7.286s]
[INFO] Lens API .. SUCCESS [31.546s]
[INFO] Lens API for server and extensions  SUCCESS [25.681s]
[INFO] Lens Cube . SUCCESS [17:55.255s]
[INFO] Lens DB storage ... SUCCESS [25.650s]
[INFO] Lens Query Library  SUCCESS [21.646s]
[INFO] Lens Hive Driver .. SUCCESS [2:11.167s]
[INFO] Lens Driver for JDBC .. SUCCESS [1:03.464s]
[INFO] Lens Elastic Search Driver  SUCCESS [54.798s]
[INFO] Lens Server ... SUCCESS [18:19.588s]
[INFO] Lens client ... SUCCESS [2:02.590s]
[INFO] Lens CLI .. SUCCESS [1:54.985s]
[INFO] Lens Examples . SUCCESS [13.790s]
[INFO] Lens Ship Jars to Distributed Cache ... SUCCESS [2.061s]
[INFO] Lens Distribution . SUCCESS [24.869s]
[INFO] Lens ML Lib ... SUCCESS [1:54.426s]
[INFO] Lens ML Ext Distribution .. SUCCESS [11.053s]
[INFO] Lens Regression ... SUCCESS [20.677s]
[INFO] Lens UI ... SUCCESS [19.454s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 49:44.002s
[INFO] Finished at: Sat Sep 17 00:12:08 IST 2016
[INFO] Final Memory: 178M/2490M
[INFO] 


Thanks,

Sushil Mohanty

Re: Review Request 52514: Add data completeness checker

2016-11-08 Thread Narayan Periwal 


> On Nov. 4, 2016, 5:45 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java,
> >  line 538
> > 
> >
> > Can you cross check context.getQueriedMsrs() is actually resolving 
> > expression fields as well ? I'm not fully sure.

We will have to use cubeql.getQueriedExprsWithMeasures and parse measures from 
here.


> On Nov. 4, 2016, 5:45 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java,
> >  line 599
> > 
> >
> > Instead of measureFromTag should be we put actual queried phrase column 
> > here ?

The measures from phrases already gets included in the 
cubeql.queriedMeasures(). For the expressions, we will put the expression name 
here instead of the measures that make up the expressions.


> On Nov. 4, 2016, 5:45 a.m., Amareshwari Sriramadasu wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java,
> >  lines 603-605
> > 
> >
> > Should be moved to outer while loop and should be moved inside the 
> > check for isFactDataIncomplete flag for removing there.

Agreed, Will fix it.


- Narayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52514/#review154865
---


On Nov. 3, 2016, 11:13 a.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52514/
> ---
> 
> (Updated Nov. 3, 2016, 11:13 a.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-1333
> https://issues.apache.org/jira/browse/LENS-1333
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Though lens has partition registration being done whenever data is available, 
> there is no guarantee the partition registered is complete. There can be 
> different ways to know if the data is complete for partition. One option 
> could be to have a partition property saying whether it is complete or not. 
> Other could be to do a http call to another hosted service and more.
> 
> Proposal here is to add an interface for DataCompletenessChecker and do the 
> check while resolving partitions.
> 
> Here are some of the capabilities we would like to add in Lens :
> # Lens will check partition existence first, if it exists, then check the 
> completeness percentage. If the completeness percentage is less than a 
> configured threshold (default should be 98, 99 or even 100), Lens will fail 
> the query.
> # Lens's accept query on partial data will accept on incomplete data as well.
> # Lens will also option to override the completeness percentage threshold 
> value at query level
> # Lens will still have look ahead capability of daily being incomplete, then 
> it will union with hourly. 
> # If same measure is there in two different facts , Lens will we pick the one 
> with higher availability.
> # In case of completeness percentage threshold missed, Lens will respond back 
> with available percentage.
> 
> 
> Diffs
> -
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java 
> fb958c3 
>   
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
>  e14c43f 
>   
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 
> 4585ef7 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 
> 5dc9dc9 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java
>  78fb21d 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
> f20f105 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultChecker.java 
> PRE-CREATION 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/MaxCoveringFactResolver.java
>  13f1aa4 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
> db26718 
>   lens-cube/src/main/resources/olap-query-conf.xml 0c888ca 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 0c43cb5 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/FieldsCannotBeQueriedTogetherTest.java
>  7afa32e 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/MockCompletenessChecker.java
>  PRE-CREATION 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestBaseCubeQueries.java 
> f6cec1b 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
> b90d4d3 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestQueryRewrite.java 
> 0aa31f4 
>   lens-server/src/main/resources/lensses

[jira] [Commented] (LENS-1371) Add a substring based retry policy for failures

2016-11-08 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647399#comment-15647399
 ] 

Hadoop QA commented on LENS-1371:
-

Applied patch: 
[0001-Retry-policy.patch|https://issues.apache.org/jira/secure/attachment/12837930/0001-Retry-policy.patch]
 and ran command: mvn clean install -fae. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/1162/

> Add a substring based retry policy for failures
> ---
>
> Key: LENS-1371
> URL: https://issues.apache.org/jira/browse/LENS-1371
> Project: Apache Lens
>  Issue Type: New Feature
>Reporter: Lavkesh Lahngir
>Assignee: Lavkesh Lahngir
> Attachments: 0001-Retry-patch.patch, 0001-Retry-policy.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-968) Monitor run time for queries and terminate long running queries

2016-11-08 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647447#comment-15647447
 ] 

Hadoop QA commented on LENS-968:


Applied patch: 
[LENS-968.3.patch|https://issues.apache.org/jira/secure/attachment/12836776/LENS-968.3.patch]
 and ran command: mvn clean install -fae. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/1163/

> Monitor run time for queries and terminate long running queries
> ---
>
> Key: LENS-968
> URL: https://issues.apache.org/jira/browse/LENS-968
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: server
>Reporter: Puneet Gupta
>Assignee: Amareshwari Sriramadasu
> Fix For: 2.7
>
> Attachments: LENS-968.3.patch, LENS-968.patch
>
>
> server should be able to kill queries that are taking more than x number of 
> hours in RUNNING state and intimate the user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-968) Monitor run time for queries and terminate long running queries

2016-11-08 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647467#comment-15647467
 ] 

Hadoop QA commented on LENS-968:


Applied patch: 
[LENS-968.3.patch|https://issues.apache.org/jira/secure/attachment/12836776/LENS-968.3.patch]
 and ran command: mvn clean install -fae. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/1164/

> Monitor run time for queries and terminate long running queries
> ---
>
> Key: LENS-968
> URL: https://issues.apache.org/jira/browse/LENS-968
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: server
>Reporter: Puneet Gupta
>Assignee: Amareshwari Sriramadasu
> Fix For: 2.7
>
> Attachments: LENS-968.3.patch, LENS-968.patch
>
>
> server should be able to kill queries that are taking more than x number of 
> hours in RUNNING state and intimate the user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1369) Druid rewriter is excluding both "from" and "to" date of lens query

2016-11-08 Thread Rajitha R (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajitha R updated LENS-1369:

Status: Patch Available  (was: Open)

> Druid rewriter is excluding both "from" and "to" date of lens query
> ---
>
> Key: LENS-1369
> URL: https://issues.apache.org/jira/browse/LENS-1369
> Project: Apache Lens
>  Issue Type: Bug
>  Components: driver-jdbc
>Reporter: Archana H
>Assignee: Rajitha R
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1369) Druid rewriter is excluding both "from" and "to" date of lens query

2016-11-08 Thread Rajitha R (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajitha R updated LENS-1369:

Status: Open  (was: Patch Available)

> Druid rewriter is excluding both "from" and "to" date of lens query
> ---
>
> Key: LENS-1369
> URL: https://issues.apache.org/jira/browse/LENS-1369
> Project: Apache Lens
>  Issue Type: Bug
>  Components: driver-jdbc
>Reporter: Archana H
>Assignee: Rajitha R
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1369) Druid rewriter is excluding both "from" and "to" date of lens query

2016-11-08 Thread Rajitha R (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajitha R updated LENS-1369:

Status: Patch Available  (was: Open)

> Druid rewriter is excluding both "from" and "to" date of lens query
> ---
>
> Key: LENS-1369
> URL: https://issues.apache.org/jira/browse/LENS-1369
> Project: Apache Lens
>  Issue Type: Bug
>  Components: driver-jdbc
>Reporter: Archana H
>Assignee: Rajitha R
> Attachments: LENS-new.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1369) Druid rewriter is excluding both "from" and "to" date of lens query

2016-11-08 Thread Rajitha R (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajitha R updated LENS-1369:

Attachment: LENS-new.patch

> Druid rewriter is excluding both "from" and "to" date of lens query
> ---
>
> Key: LENS-1369
> URL: https://issues.apache.org/jira/browse/LENS-1369
> Project: Apache Lens
>  Issue Type: Bug
>  Components: driver-jdbc
>Reporter: Archana H
>Assignee: Rajitha R
> Attachments: LENS-new.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 52514: Add data completeness checker

2016-11-08 Thread Narayan Periwal 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52514/
---

(Updated Nov. 8, 2016, 1:11 p.m.)


Review request for lens.


Bugs: LENS-1333
https://issues.apache.org/jira/browse/LENS-1333


Repository: lens


Description
---

Though lens has partition registration being done whenever data is available, 
there is no guarantee the partition registered is complete. There can be 
different ways to know if the data is complete for partition. One option could 
be to have a partition property saying whether it is complete or not. Other 
could be to do a http call to another hosted service and more.

Proposal here is to add an interface for DataCompletenessChecker and do the 
check while resolving partitions.

Here are some of the capabilities we would like to add in Lens :
# Lens will check partition existence first, if it exists, then check the 
completeness percentage. If the completeness percentage is less than a 
configured threshold (default should be 98, 99 or even 100), Lens will fail the 
query.
# Lens's accept query on partial data will accept on incomplete data as well.
# Lens will also option to override the completeness percentage threshold value 
at query level
# Lens will still have look ahead capability of daily being incomplete, then it 
will union with hourly. 
# If same measure is there in two different facts , Lens will we pick the one 
with higher availability.
# In case of completeness percentage threshold missed, Lens will respond back 
with available percentage.


Diffs (updated)
-

  lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java 
fb958c3 
  
lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java 
e14c43f 
  lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 
4585ef7 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 5dc9dc9 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java
 78fb21d 
  lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
f20f105 
  lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultChecker.java 
PRE-CREATION 
  
lens-cube/src/main/java/org/apache/lens/cube/parse/MaxCoveringFactResolver.java 
13f1aa4 
  lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
db26718 
  lens-cube/src/main/resources/olap-query-conf.xml 0c888ca 
  lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 0c43cb5 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/FieldsCannotBeQueriedTogetherTest.java
 7afa32e 
  
lens-cube/src/test/java/org/apache/lens/cube/parse/MockCompletenessChecker.java 
PRE-CREATION 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestBaseCubeQueries.java 
f6cec1b 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
b90d4d3 
  lens-cube/src/test/java/org/apache/lens/cube/parse/TestQueryRewrite.java 
0aa31f4 
  lens-server/src/main/resources/lenssession-default.xml 9a8f9da 
  src/site/apt/admin/session-config.apt d480f88 
  src/site/apt/user/olap-query-conf.apt ce35ec4 

Diff: https://reviews.apache.org/r/52514/diff/


Testing
---


Thanks,

Narayan Periwal

Re: Review Request 52514: Add data completeness checker

2016-11-08 Thread Narayan Periwal 


> On Oct. 21, 2016, 9:12 a.m., Rajat Khandelwal wrote:
> > lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java,
> >  lines 118-121
> > 
> >
> > StorageTableResolver is invoked in each rewrite step, which happens 
> > #drivers times for each query. Using reflection to create an instance this 
> > often will be harmful for performance.

Agreed. Added this in Hive Conf.


- Narayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52514/#review153515
---


On Nov. 8, 2016, 1:11 p.m., Narayan Periwal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52514/
> ---
> 
> (Updated Nov. 8, 2016, 1:11 p.m.)
> 
> 
> Review request for lens.
> 
> 
> Bugs: LENS-1333
> https://issues.apache.org/jira/browse/LENS-1333
> 
> 
> Repository: lens
> 
> 
> Description
> ---
> 
> Though lens has partition registration being done whenever data is available, 
> there is no guarantee the partition registered is complete. There can be 
> different ways to know if the data is complete for partition. One option 
> could be to have a partition property saying whether it is complete or not. 
> Other could be to do a http call to another hosted service and more.
> 
> Proposal here is to add an interface for DataCompletenessChecker and do the 
> check while resolving partitions.
> 
> Here are some of the capabilities we would like to add in Lens :
> # Lens will check partition existence first, if it exists, then check the 
> completeness percentage. If the completeness percentage is less than a 
> configured threshold (default should be 98, 99 or even 100), Lens will fail 
> the query.
> # Lens's accept query on partial data will accept on incomplete data as well.
> # Lens will also option to override the completeness percentage threshold 
> value at query level
> # Lens will still have look ahead capability of daily being incomplete, then 
> it will union with hourly. 
> # If same measure is there in two different facts , Lens will we pick the one 
> with higher availability.
> # In case of completeness percentage threshold missed, Lens will respond back 
> with available percentage.
> 
> 
> Diffs
> -
> 
>   lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeFactTable.java 
> fb958c3 
>   
> lens-cube/src/main/java/org/apache/lens/cube/metadata/CubeMetastoreClient.java
>  e14c43f 
>   
> lens-cube/src/main/java/org/apache/lens/cube/metadata/MetastoreConstants.java 
> 4585ef7 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateFact.java 
> 5dc9dc9 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/CandidateTablePruneCause.java
>  78fb21d 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/CubeQueryConfUtil.java 
> f20f105 
>   lens-cube/src/main/java/org/apache/lens/cube/parse/DefaultChecker.java 
> PRE-CREATION 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/MaxCoveringFactResolver.java
>  13f1aa4 
>   
> lens-cube/src/main/java/org/apache/lens/cube/parse/StorageTableResolver.java 
> db26718 
>   lens-cube/src/main/resources/olap-query-conf.xml 0c888ca 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/CubeTestSetup.java 
> 0c43cb5 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/FieldsCannotBeQueriedTogetherTest.java
>  7afa32e 
>   
> lens-cube/src/test/java/org/apache/lens/cube/parse/MockCompletenessChecker.java
>  PRE-CREATION 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestBaseCubeQueries.java 
> f6cec1b 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestCubeRewriter.java 
> b90d4d3 
>   lens-cube/src/test/java/org/apache/lens/cube/parse/TestQueryRewrite.java 
> 0aa31f4 
>   lens-server/src/main/resources/lenssession-default.xml 9a8f9da 
>   src/site/apt/admin/session-config.apt d480f88 
>   src/site/apt/user/olap-query-conf.apt ce35ec4 
> 
> Diff: https://reviews.apache.org/r/52514/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Narayan Periwal
> 
>

[jira] [Updated] (LENS-1333) Add data completeness checker

2016-11-08 Thread Narayan Periwal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Narayan Periwal updated LENS-1333:
--
Status: Open  (was: Patch Available)

> Add data completeness checker
> -
>
> Key: LENS-1333
> URL: https://issues.apache.org/jira/browse/LENS-1333
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: cube
>Reporter: Amareshwari Sriramadasu
>Assignee: Narayan Periwal
> Attachments: LENS-1333-v1.patch, LENS-1333-v2.patch, 
> LENS-1333-v3.patch, LENS-1333-v4.patch
>
>
> Though lens has partition registration being done whenever data is available, 
> there is no guarantee the partition registered is complete. There can be 
> different ways to know if the data is complete for partition. One option 
> could be to have a partition property saying whether it is complete or not. 
> Other could be to do a http call to another hosted service and more.
> Proposal here is to add an interface for DataCompletenessChecker and do the 
> check while resolving partitions.
> Here are some of the capabilities we would like to add in Lens :
> # Lens will check partition existence first, if it exists, then check the 
> completeness percentage. If the completeness percentage is less than a 
> configured threshold (default should be 98, 99 or even 100), Lens will fail 
> the query.
> # Lens's accept query on partial data will accept on incomplete data as well.
> # Lens will also option to override the completeness percentage threshold 
> value at query level
> # Lens will still have look ahead capability of daily being incomplete, then 
> it will union with hourly. 
> # If same measure is there in two different facts , Lens will we pick the one 
> with higher availability.
> # In case of completeness percentage threshold missed, Lens will respond back 
> with available percentage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (LENS-1333) Add data completeness checker

2016-11-08 Thread Narayan Periwal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on LENS-1333 started by Narayan Periwal.
-
> Add data completeness checker
> -
>
> Key: LENS-1333
> URL: https://issues.apache.org/jira/browse/LENS-1333
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: cube
>Reporter: Amareshwari Sriramadasu
>Assignee: Narayan Periwal
> Attachments: LENS-1333-v1.patch, LENS-1333-v2.patch, 
> LENS-1333-v3.patch, LENS-1333-v4.patch
>
>
> Though lens has partition registration being done whenever data is available, 
> there is no guarantee the partition registered is complete. There can be 
> different ways to know if the data is complete for partition. One option 
> could be to have a partition property saying whether it is complete or not. 
> Other could be to do a http call to another hosted service and more.
> Proposal here is to add an interface for DataCompletenessChecker and do the 
> check while resolving partitions.
> Here are some of the capabilities we would like to add in Lens :
> # Lens will check partition existence first, if it exists, then check the 
> completeness percentage. If the completeness percentage is less than a 
> configured threshold (default should be 98, 99 or even 100), Lens will fail 
> the query.
> # Lens's accept query on partial data will accept on incomplete data as well.
> # Lens will also option to override the completeness percentage threshold 
> value at query level
> # Lens will still have look ahead capability of daily being incomplete, then 
> it will union with hourly. 
> # If same measure is there in two different facts , Lens will we pick the one 
> with higher availability.
> # In case of completeness percentage threshold missed, Lens will respond back 
> with available percentage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1333) Add data completeness checker

2016-11-08 Thread Narayan Periwal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Narayan Periwal updated LENS-1333:
--
Attachment: LENS-1333-v4.patch

> Add data completeness checker
> -
>
> Key: LENS-1333
> URL: https://issues.apache.org/jira/browse/LENS-1333
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: cube
>Reporter: Amareshwari Sriramadasu
>Assignee: Narayan Periwal
> Attachments: LENS-1333-v1.patch, LENS-1333-v2.patch, 
> LENS-1333-v3.patch, LENS-1333-v4.patch
>
>
> Though lens has partition registration being done whenever data is available, 
> there is no guarantee the partition registered is complete. There can be 
> different ways to know if the data is complete for partition. One option 
> could be to have a partition property saying whether it is complete or not. 
> Other could be to do a http call to another hosted service and more.
> Proposal here is to add an interface for DataCompletenessChecker and do the 
> check while resolving partitions.
> Here are some of the capabilities we would like to add in Lens :
> # Lens will check partition existence first, if it exists, then check the 
> completeness percentage. If the completeness percentage is less than a 
> configured threshold (default should be 98, 99 or even 100), Lens will fail 
> the query.
> # Lens's accept query on partial data will accept on incomplete data as well.
> # Lens will also option to override the completeness percentage threshold 
> value at query level
> # Lens will still have look ahead capability of daily being incomplete, then 
> it will union with hourly. 
> # If same measure is there in two different facts , Lens will we pick the one 
> with higher availability.
> # In case of completeness percentage threshold missed, Lens will respond back 
> with available percentage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1333) Add data completeness checker

2016-11-08 Thread Narayan Periwal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Narayan Periwal updated LENS-1333:
--
Status: Patch Available  (was: Open)

> Add data completeness checker
> -
>
> Key: LENS-1333
> URL: https://issues.apache.org/jira/browse/LENS-1333
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: cube
>Reporter: Amareshwari Sriramadasu
>Assignee: Narayan Periwal
> Attachments: LENS-1333-v1.patch, LENS-1333-v2.patch, 
> LENS-1333-v3.patch, LENS-1333-v4.patch
>
>
> Though lens has partition registration being done whenever data is available, 
> there is no guarantee the partition registered is complete. There can be 
> different ways to know if the data is complete for partition. One option 
> could be to have a partition property saying whether it is complete or not. 
> Other could be to do a http call to another hosted service and more.
> Proposal here is to add an interface for DataCompletenessChecker and do the 
> check while resolving partitions.
> Here are some of the capabilities we would like to add in Lens :
> # Lens will check partition existence first, if it exists, then check the 
> completeness percentage. If the completeness percentage is less than a 
> configured threshold (default should be 98, 99 or even 100), Lens will fail 
> the query.
> # Lens's accept query on partial data will accept on incomplete data as well.
> # Lens will also option to override the completeness percentage threshold 
> value at query level
> # Lens will still have look ahead capability of daily being incomplete, then 
> it will union with hourly. 
> # If same measure is there in two different facts , Lens will we pick the one 
> with higher availability.
> # In case of completeness percentage threshold missed, Lens will respond back 
> with available percentage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work stopped] (LENS-1333) Add data completeness checker

2016-11-08 Thread Narayan Periwal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on LENS-1333 stopped by Narayan Periwal.
-
> Add data completeness checker
> -
>
> Key: LENS-1333
> URL: https://issues.apache.org/jira/browse/LENS-1333
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: cube
>Reporter: Amareshwari Sriramadasu
>Assignee: Narayan Periwal
> Attachments: LENS-1333-v1.patch, LENS-1333-v2.patch, 
> LENS-1333-v3.patch, LENS-1333-v4.patch
>
>
> Though lens has partition registration being done whenever data is available, 
> there is no guarantee the partition registered is complete. There can be 
> different ways to know if the data is complete for partition. One option 
> could be to have a partition property saying whether it is complete or not. 
> Other could be to do a http call to another hosted service and more.
> Proposal here is to add an interface for DataCompletenessChecker and do the 
> check while resolving partitions.
> Here are some of the capabilities we would like to add in Lens :
> # Lens will check partition existence first, if it exists, then check the 
> completeness percentage. If the completeness percentage is less than a 
> configured threshold (default should be 98, 99 or even 100), Lens will fail 
> the query.
> # Lens's accept query on partial data will accept on incomplete data as well.
> # Lens will also option to override the completeness percentage threshold 
> value at query level
> # Lens will still have look ahead capability of daily being incomplete, then 
> it will union with hourly. 
> # If same measure is there in two different facts , Lens will we pick the one 
> with higher availability.
> # In case of completeness percentage threshold missed, Lens will respond back 
> with available percentage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-1369) Druid rewriter is excluding both "from" and "to" date of lens query

2016-11-08 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-1369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647588#comment-15647588
 ] 

Hadoop QA commented on LENS-1369:
-

Applied patch: 
[LENS-new.patch|https://issues.apache.org/jira/secure/attachment/12837967/LENS-new.patch]
 and ran command: mvn clean install -fae. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/1165/

> Druid rewriter is excluding both "from" and "to" date of lens query
> ---
>
> Key: LENS-1369
> URL: https://issues.apache.org/jira/browse/LENS-1369
> Project: Apache Lens
>  Issue Type: Bug
>  Components: driver-jdbc
>Reporter: Archana H
>Assignee: Rajitha R
> Attachments: LENS-new.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1371) Add a substring based retry policy for failures

2016-11-08 Thread Lavkesh Lahngir (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lavkesh Lahngir updated LENS-1371:
--
Attachment: 0001-Retry-policy.patch

> Add a substring based retry policy for failures
> ---
>
> Key: LENS-1371
> URL: https://issues.apache.org/jira/browse/LENS-1371
> Project: Apache Lens
>  Issue Type: New Feature
>Reporter: Lavkesh Lahngir
>Assignee: Lavkesh Lahngir
> Attachments: 0001-Retry-patch.patch, 0001-Retry-policy.patch, 
> 0001-Retry-policy.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-1333) Add data completeness checker

2016-11-08 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647599#comment-15647599
 ] 

Hadoop QA commented on LENS-1333:
-

Applied patch: 
[LENS-1333-v4.patch|https://issues.apache.org/jira/secure/attachment/12837972/LENS-1333-v4.patch]
 and ran command: mvn clean install -fae. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/1166/

> Add data completeness checker
> -
>
> Key: LENS-1333
> URL: https://issues.apache.org/jira/browse/LENS-1333
> Project: Apache Lens
>  Issue Type: New Feature
>  Components: cube
>Reporter: Amareshwari Sriramadasu
>Assignee: Narayan Periwal
> Attachments: LENS-1333-v1.patch, LENS-1333-v2.patch, 
> LENS-1333-v3.patch, LENS-1333-v4.patch
>
>
> Though lens has partition registration being done whenever data is available, 
> there is no guarantee the partition registered is complete. There can be 
> different ways to know if the data is complete for partition. One option 
> could be to have a partition property saying whether it is complete or not. 
> Other could be to do a http call to another hosted service and more.
> Proposal here is to add an interface for DataCompletenessChecker and do the 
> check while resolving partitions.
> Here are some of the capabilities we would like to add in Lens :
> # Lens will check partition existence first, if it exists, then check the 
> completeness percentage. If the completeness percentage is less than a 
> configured threshold (default should be 98, 99 or even 100), Lens will fail 
> the query.
> # Lens's accept query on partial data will accept on incomplete data as well.
> # Lens will also option to override the completeness percentage threshold 
> value at query level
> # Lens will still have look ahead capability of daily being incomplete, then 
> it will union with hourly. 
> # If same measure is there in two different facts , Lens will we pick the one 
> with higher availability.
> # In case of completeness percentage threshold missed, Lens will respond back 
> with available percentage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1371) Add a substring based retry policy for failures

2016-11-08 Thread Lavkesh Lahngir (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lavkesh Lahngir updated LENS-1371:
--
Status: Open  (was: Patch Available)

> Add a substring based retry policy for failures
> ---
>
> Key: LENS-1371
> URL: https://issues.apache.org/jira/browse/LENS-1371
> Project: Apache Lens
>  Issue Type: New Feature
>Reporter: Lavkesh Lahngir
>Assignee: Lavkesh Lahngir
> Attachments: 0001-Retry-patch.patch, 0001-Retry-policy.patch, 
> 0001-Retry-policy.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1371) Add a substring based retry policy for failures

2016-11-08 Thread Lavkesh Lahngir (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lavkesh Lahngir updated LENS-1371:
--
Status: Patch Available  (was: Open)

> Add a substring based retry policy for failures
> ---
>
> Key: LENS-1371
> URL: https://issues.apache.org/jira/browse/LENS-1371
> Project: Apache Lens
>  Issue Type: New Feature
>Reporter: Lavkesh Lahngir
>Assignee: Lavkesh Lahngir
> Attachments: 0001-Retry-patch.patch, 0001-Retry-policy.patch, 
> 0001-Retry-policy.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-1371) Add a substring based retry policy for failures

2016-11-08 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-1371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647726#comment-15647726
 ] 

Hadoop QA commented on LENS-1371:
-

Applied patch: 
[0001-Retry-policy.patch|https://issues.apache.org/jira/secure/attachment/12837977/0001-Retry-policy.patch]
 and ran command: mvn clean install -fae. Result: Failure. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/1167/

> Add a substring based retry policy for failures
> ---
>
> Key: LENS-1371
> URL: https://issues.apache.org/jira/browse/LENS-1371
> Project: Apache Lens
>  Issue Type: New Feature
>Reporter: Lavkesh Lahngir
>Assignee: Lavkesh Lahngir
> Attachments: 0001-Retry-patch.patch, 0001-Retry-policy.patch, 
> 0001-Retry-policy.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (LENS-1372) QueryRetryTest is flaky

2016-11-08 Thread Ted Yu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated LENS-1372:
-
Description: 
>From 
>https://builds.apache.org/job/PreCommit-Lens-Build/1145/testReport/junit/org.apache.lens.server.query.retry/QueryRetryTest/testRestartWhileRetry/
> :
{code}
java.lang.NullPointerException: null
at 
org.apache.lens.server.query.retry.QueryRetryTest.testRestartWhileRetry(QueryRetryTest.java:201)
{code}
Here is related code:
{code}
ctx = getQueryService().getQueryContext(handle);
while (!ctx.getStatus().finished()) {
{code}

The test should handle the case where status is null.

  was:
>From 
>https://builds.apache.org/job/PreCommit-Lens-Build/1145/testReport/junit/org.apache.lens.server.query.retry/QueryRetryTest/testRestartWhileRetry/
> :
{code}
java.lang.NullPointerException: null
at 
org.apache.lens.server.query.retry.QueryRetryTest.testRestartWhileRetry(QueryRetryTest.java:201)
{code}
Here is related code:
{code}
ctx = getQueryService().getQueryContext(handle);
while (!ctx.getStatus().finished()) {
{code}
The test should handle the case where status is null.


> QueryRetryTest is flaky
> ---
>
> Key: LENS-1372
> URL: https://issues.apache.org/jira/browse/LENS-1372
> Project: Apache Lens
>  Issue Type: Test
>Reporter: Ted Yu
>Priority: Minor
>
> From 
> https://builds.apache.org/job/PreCommit-Lens-Build/1145/testReport/junit/org.apache.lens.server.query.retry/QueryRetryTest/testRestartWhileRetry/
>  :
> {code}
> java.lang.NullPointerException: null
>   at 
> org.apache.lens.server.query.retry.QueryRetryTest.testRestartWhileRetry(QueryRetryTest.java:201)
> {code}
> Here is related code:
> {code}
> ctx = getQueryService().getQueryContext(handle);
> while (!ctx.getStatus().finished()) {
> {code}
> The test should handle the case where status is null.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-968) Monitor run time for queries and terminate long running queries

2016-11-08 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649821#comment-15649821
 ] 

Hadoop QA commented on LENS-968:


Applied patch: 
[LENS-968.3.patch|https://issues.apache.org/jira/secure/attachment/12836776/LENS-968.3.patch]
 and ran command: mvn clean install -fae. Result: Success. Build Job: 
https://builds.apache.org/job/PreCommit-Lens-Build/1168/

> Monitor run time for queries and terminate long running queries
> ---
>
> Key: LENS-968
> URL: https://issues.apache.org/jira/browse/LENS-968
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: server
>Reporter: Puneet Gupta
>Assignee: Amareshwari Sriramadasu
> Fix For: 2.7
>
> Attachments: LENS-968.3.patch, LENS-968.patch
>
>
> server should be able to kill queries that are taking more than x number of 
> hours in RUNNING state and intimate the user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

November 2016 report

2016-11-08 Thread Amareshwari Sriramdasu 
Hello Lens dev,

Have put up report for November month at
https://cwiki.apache.org/confluence/display/LENS/November%2C2016.

Please review and let me know your comments.

Will post the report to board EOD.

Thanks
Amareshwari

[jira] [Updated] (LENS-968) Monitor run time for queries and terminate long running queries

2016-11-08 Thread Raju Bairishetti (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raju Bairishetti updated LENS-968:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed patch. Thanks [~amareshwari]

> Monitor run time for queries and terminate long running queries
> ---
>
> Key: LENS-968
> URL: https://issues.apache.org/jira/browse/LENS-968
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: server
>Reporter: Puneet Gupta
>Assignee: Amareshwari Sriramadasu
> Fix For: 2.7
>
> Attachments: LENS-968.3.patch, LENS-968.patch
>
>
> server should be able to kill queries that are taking more than x number of 
> hours in RUNNING state and intimate the user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-968) Monitor run time for queries and terminate long running queries

2016-11-08 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649970#comment-15649970
 ] 

Hudson commented on LENS-968:
-

UNSTABLE: Integrated in Jenkins build Lens-Commit-Java8 #289 (See 
[https://builds.apache.org/job/Lens-Commit-Java8/289/])
LENS-968: Configure timeouts for queries and terminate queries in case (raju: 
rev 4ae48c7497ef46319b1b5f39d21531b11a5afba8)
* (edit) src/site/apt/admin/jdbcdriver-config.apt
* (edit) lens-server/src/main/resources/lensserver-default.xml
* (edit) src/site/apt/admin/session-config.apt
* (edit) src/site/apt/admin/esdriver-config.apt
* (edit) 
lens-server/src/test/java/org/apache/lens/server/common/FailingQueryDriver.java
* (edit) lens-server/src/test/resources/logback.xml
* (edit) lens-server/src/main/resources/lenssession-default.xml
* (edit) lens-driver-es/src/main/resources/esdriver-default.xml
* (edit) 
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
* (edit) 
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
* (edit) lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml
* (edit) lens-server/src/test/resources/lens-site.xml
* (edit) src/site/apt/admin/config.apt
* (edit) 
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java
* (edit) 
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java
* (edit) 
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java


> Monitor run time for queries and terminate long running queries
> ---
>
> Key: LENS-968
> URL: https://issues.apache.org/jira/browse/LENS-968
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: server
>Reporter: Puneet Gupta
>Assignee: Amareshwari Sriramadasu
> Fix For: 2.7
>
> Attachments: LENS-968.3.patch, LENS-968.patch
>
>
> server should be able to kill queries that are taking more than x number of 
> hours in RUNNING state and intimate the user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: November 2016 report

2016-11-08 Thread Puneet Gupta 
+1

Thanks,
Puneet Gupta

On Wed, Nov 9, 2016 at 11:06 AM, Amareshwari Sriramdasu <
amareshw...@apache.org> wrote:

> Hello Lens dev,
>
> Have put up report for November month at
> https://cwiki.apache.org/confluence/display/LENS/November%2C2016.
>
> Please review and let me know your comments.
>
> Will post the report to board EOD.
>
> Thanks
> Amareshwari
>

-- 
_
The information contained in this communication is intended solely for the 
use of the individual or entity to whom it is addressed and others 
authorized to receive it. It may contain confidential or legally privileged 
information. If you are not the intended recipient you are hereby notified 
that any disclosure, copying, distribution or taking any action in reliance 
on the contents of this information is strictly prohibited and may be 
unlawful. If you have received this communication in error, please notify 
us immediately by responding to this email and then delete it from your 
system. The firm is neither liable for the proper and complete transmission 
of the information contained in this communication nor for any delay in its 
receipt.

[jira] [Assigned] (LENS-1302) Add launch rejection reason to query status message

2016-11-08 Thread Rajat Khandelwal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajat Khandelwal reassigned LENS-1302:
--

Assignee: Rajat Khandelwal

> Add launch rejection reason to query status message
> ---
>
> Key: LENS-1302
> URL: https://issues.apache.org/jira/browse/LENS-1302
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (LENS-1302) Add launch rejection reason to query status message

2016-11-08 Thread Rajat Khandelwal (JIRA) 

 [ 
https://issues.apache.org/jira/browse/LENS-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajat Khandelwal resolved LENS-1302.

Resolution: Fixed

Fixed alongwith LENS-743

> Add launch rejection reason to query status message
> ---
>
> Key: LENS-1302
> URL: https://issues.apache.org/jira/browse/LENS-1302
> Project: Apache Lens
>  Issue Type: Improvement
>Reporter: Rajat Khandelwal
>Assignee: Rajat Khandelwal
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (LENS-968) Monitor run time for queries and terminate long running queries

2016-11-08 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/LENS-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15650085#comment-15650085
 ] 

Hudson commented on LENS-968:
-

UNSTABLE: Integrated in Jenkins build Lens-Commit #1371 (See 
[https://builds.apache.org/job/Lens-Commit/1371/])
LENS-968: Configure timeouts for queries and terminate queries in case (raju: 
rev 4ae48c7497ef46319b1b5f39d21531b11a5afba8)
* (edit) 
lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java
* (edit) lens-server/src/test/resources/logback.xml
* (edit) 
lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java
* (edit) lens-driver-es/src/main/resources/esdriver-default.xml
* (edit) 
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
* (edit) src/site/apt/admin/esdriver-config.apt
* (edit) 
lens-server/src/test/java/org/apache/lens/server/common/FailingQueryDriver.java
* (edit) 
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java
* (edit) src/site/apt/admin/config.apt
* (edit) src/site/apt/admin/session-config.apt
* (edit) 
lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java
* (edit) lens-server/src/test/resources/lens-site.xml
* (edit) lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml
* (edit) lens-server/src/main/resources/lenssession-default.xml
* (edit) src/site/apt/admin/jdbcdriver-config.apt
* (edit) lens-server/src/main/resources/lensserver-default.xml


> Monitor run time for queries and terminate long running queries
> ---
>
> Key: LENS-968
> URL: https://issues.apache.org/jira/browse/LENS-968
> Project: Apache Lens
>  Issue Type: Improvement
>  Components: server
>Reporter: Puneet Gupta
>Assignee: Amareshwari Sriramadasu
> Fix For: 2.7
>
> Attachments: LENS-968.3.patch, LENS-968.patch
>
>
> server should be able to kill queries that are taking more than x number of 
> hours in RUNNING state and intimate the user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: November 2016 report

2016-11-08 Thread Rajat Khandelwal 
+1

On Wed, Nov 9, 2016 at 12:28 PM Puneet Gupta 
wrote:

> +1
>
> Thanks,
> Puneet Gupta
>
> On Wed, Nov 9, 2016 at 11:06 AM, Amareshwari Sriramdasu <
> amareshw...@apache.org> wrote:
>
> > Hello Lens dev,
> >
> > Have put up report for November month at
> > https://cwiki.apache.org/confluence/display/LENS/November%2C2016.
> >
> > Please review and let me know your comments.
> >
> > Will post the report to board EOD.
> >
> > Thanks
> > Amareshwari
> >
>
> --
> _
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify
> us immediately by responding to this email and then delete it from your
> system. The firm is neither liable for the proper and complete transmission
> of the information contained in this communication nor for any delay in its
> receipt.
>