[jira] [Created] (PHOENIX-2367) Change PhoenixRecordWriter to use execute instead of executeBatch

2015-11-03 Thread Siddhi Mehta (JIRA) 
Siddhi Mehta created PHOENIX-2367:
-

 Summary: Change PhoenixRecordWriter to use execute instead of 
executeBatch
 Key: PHOENIX-2367
 URL: https://issues.apache.org/jira/browse/PHOENIX-2367
 Project: Phoenix
  Issue Type: Improvement
Reporter: Siddhi Mehta
Assignee: Siddhi Mehta


Hey All,

I wanted to add a notion of skipping invalid rows for PhoenixHbaseStorage 
similar to how the CSVBulkLoad tool has an option of ignoring the bad rows.I 
did some work on the apache pig code that allows Storers to have a notion of 
Customizable/Configurable Errors PIG-4704.

I wanted to plug this behavior for PhoenixHbaseStorage and propose certain 
changes for the same.

Current Behavior/Problem:

PhoenixRecordWriter makes use of executeBatch() to process rows once batch size 
is reached. If there are any client side validation/syntactical errors like 
data not fitting the column size, executeBatch() throws an exception and there 
is no-way to retrieve the valid rows from the batch and retry them. We discard 
the whole batch or fail the job without errorhandling.

With auto commit set to false execute() also servers the purpose of not making 
any rpc calls  but does a bunch of validation client side and adds it to the 
client cache of mutation.

On conn.commit() we make a rpc call.

Proposed Change

To be able to use Configurable ErrorHandling and ignore only the failed records 
instead of discarding the whole batch I want to propose changing the behavior 
in PhoenixRecordWriter from execute to executeBatch() or having a configuration 
to toggle between the 2 behaviors 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: PhoenixHbaseStorage to Skip invalid rows

2015-11-03 Thread Siddhi Mehta 
Jira Created: https://issues.apache.org/jira/browse/PHOENIX-2367
I will submit a patch for review soon.

On Tue, Nov 3, 2015 at 10:52 AM, Jan Fernando 
wrote:

> +1 on making this change. Can you file a JIRA for it?
>
> On Mon, Nov 2, 2015 at 4:31 PM, Siddhi Mehta  wrote:
>
> > Hey All,
> >
> > I wanted to add a notion of skipping invalid rows for PhoenixHbaseStorage
> > similar to how the CSVBulkLoad tool has an option of ignoring the bad
> > rows.I did some work on the apache pig code that allows Storers to have a
> > notion of Customizable/Configurable Errors PIG-4704
> > .
> >
> > I wanted to plug this behavior for PhoenixHbaseStorage and propose
> certain
> > changes for the same.
> >
> > *Current Behavior/Problem:*
> >
> > PhoenixRecordWriter makes use of executeBatch() to process rows once
> batch
> > size is reached. If there are any client side validation/syntactical
> errors
> > like data not fitting the column size, executeBatch() throws an exception
> > and there is no-way to retrieve the valid rows from the batch and retry
> > them. We discard the whole batch or fail the job without errorhandling.
> >
> > With auto commit set to false execute() also servers the purpose of not
> > making any rpc calls  but does a bunch of validation client side and adds
> > it to the client cache of mutation.
> >
> > On conn.commit() we make a rpc call.
> >
> > *Proposed Change*
> >
> > To be able to use Configurable ErrorHandling and ignore only the failed
> > records instead of discarding the whole batch I want to propose changing
> > the behavior in PhoenixRecordWriter from execute to executeBatch() or
> > having a configuration to toggle between the 2 behaviors
> > Thoughts?
> >
>

[jira] [Resolved] (PHOENIX-2362) Rollback doesn't work when checkpointing is done

2015-11-03 Thread James Taylor (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor resolved PHOENIX-2362.
---
Resolution: Fixed

> Rollback doesn't work when checkpointing is done
> 
>
> Key: PHOENIX-2362
> URL: https://issues.apache.org/jira/browse/PHOENIX-2362
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: James Taylor
> Attachments: PHOENIX-2362.patch, PHOENIX-2362_v2.patch, 
> PHOENIX-2362_v3.patch, PHOENIX-2362_v4.patch, PHOENIX-2362_v5.patch, 
> PHOENIX-2362_v6.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (PHOENIX-2361) Use single byte value for empty key value

2015-11-03 Thread James Taylor (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor resolved PHOENIX-2361.
---
Resolution: Fixed

> Use single byte value for empty key value
> -
>
> Key: PHOENIX-2361
> URL: https://issues.apache.org/jira/browse/PHOENIX-2361
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: James Taylor
> Attachments: PHOENIX-2361.patch, PHOENIX-2361_addendum1.patch, 
> PHOENIX-2361_v2.patch
>
>
> To save a few bytes, we should use a single byte value for the empty key 
> value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Build failed in Jenkins: Phoenix-4.x-HBase-1.0 #206

2015-11-03 Thread James Taylor 
No, not related. It's the same mysterious NPE we've been getting every few
runs - I sent a note on the dev list. It'd be great if someone could figure
out how to get around it (or at least get a stack trace for it). Maybe a
newer version of the surefire plugin would solve it?

James

On Tue, Nov 3, 2015 at 2:51 PM, Cody Marcel  wrote:

> I don't think so. Tests are passing locally for me.
>
> On Tue, Nov 3, 2015 at 1:04 PM, Mujtaba Chohan  wrote:
>
> > Cody - Is this NPE in Pherf test related to your change?
> >
> >
> > On Mon, Nov 2, 2015 at 5:58 PM, Apache Jenkins Server <
> > jenk...@builds.apache.org> wrote:
> >
> >> See 
> >>
> >> Changes:
> >>
> >> [cmarcel] PHOENIX-2366 - Pherf NPE on Data loader
> >>
> >> --
> >> [...truncated 1873 lines...]
> >> Downloading:
> >>
> https://oss.sonatype.org/content/repositories/snapshots/commons-io/commons-io/maven-metadata.xml
> >> Downloading:
> >>
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/maven-metadata.xml
> >> 725/725 B
> >> 374/374 B   725/725 B
> >>
> >> Downloading:
> >>
> http://repository.apache.org/snapshots/commons-io/commons-io/maven-metadata.xml
> >>
> >> Downloaded:
> >>
> https://repository.apache.org/content/repositories/releases/commons-io/commons-io/maven-metadata.xml
> >> (725 B at 3.7 KB/sec)
> >> 374/374 B   350/350 B
> >> Downloading:
> >>
> http://repo.maven.apache.org/maven2/commons-io/commons-io/maven-metadata.xml
> >>
> >> Downloaded:
> >>
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/maven-metadata.xml
> >> (374 B at 1.8 KB/sec)
> >>
> >>
> >> Downloaded:
> >>
> http://repository.apache.org/snapshots/commons-io/commons-io/maven-metadata.xml
> >> (350 B at 4.2 KB/sec)
> >> 725/725 B
> >>
> >> Downloaded:
> >>
> http://repo.maven.apache.org/maven2/commons-io/commons-io/maven-metadata.xml
> >> (725 B at 13.4 KB/sec)
> >> Downloading:
> >>
> http://conjars.org/repo/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> >> Downloading:
> >>
> https://repository.apache.org/content/repositories/releases/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> >> Downloading:
> >>
> https://oss.sonatype.org/content/repositories/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> >> Downloading:
> >>
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> >>
> >> 3/3 KB
> >> Downloading:
> >>
> http://repository.apache.org/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> >> 3/3 KB   3/3 KB
> >>
> >> Downloaded:
> >>
> http://repository.apache.org/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> >> (3 KB at 30.6 KB/sec)
> >>
> >>
> >> Downloaded:
> >>
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> >> (3 KB at 16.6 KB/sec)
> >>
> >> Downloading:
> >>
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> >> Downloading:
> >>
> http://conjars.org/repo/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> >> Downloading:
> >>
> https://repository.apache.org/content/repositories/releases/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> >> Downloading:
> >>
> https://oss.sonatype.org/content/repositories/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> >>
> >> Downloading:
> >>
> http://repository.apache.org/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> >> 3/3 KB
> >> 3/3 KB   3/3 KB
> >>
> >> Downloaded:
> >>
> http://repository.apache.org/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> >> (3 KB at 27.2 KB/sec)
> >>
> >>
> >> Downloaded:
> >>
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> >> (3 KB at 10.6 KB/sec)
> >>
> >> [INFO]
> >> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ phoenix-pherf
> >> ---
> >> [INFO]
> >> [INFO] --- maven-checkstyle-plugin:2.13:check (validate) @ phoenix-pherf
> >> ---
> >> [INFO]
> >> [INFO] --- build-helper-maven-plugin:1.9.1:add-test-source
> >> (add-test-source) @ phoenix-pherf ---
> >> [INFO] Test Source directory: <
> >>
> https://builds.apache.org/job/Phoenix-4.x-HBase-1.0/ws/phoenix-pherf/src/it/java
> >
> >> added.
> >> [INFO]
> >> [INFO] --- build-helper-maven-plugin:1.9.1:add-test-resource
> >> (add-test-resource) @ phoenix-pherf ---
> >> [INFO]
> >> [INFO] --- maven-remote-resources-plugin:1.5:process (default) @
> >> phoenix-pherf ---
> >> [INFO] artifact junit:junit-dep: checking for updates from apache
> release
> >> [INFO] artifact junit:junit-dep: checking for updates from conjars.org
> >> [INFO] artifact junit:junit-dep: checking for updates from apache
> snapshot
> >> [INFO] artifact junit:junit-dep: checking for updates from
>

[jira] [Comment Edited] (PHOENIX-2267) Disambiguate server jar from phoenix-server module jar

2015-11-03 Thread James Taylor (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988472#comment-14988472
 ] 

James Taylor edited comment on PHOENIX-2267 at 11/3/15 11:33 PM:
-

Sure, that's an improvement. Maybe we should rename the other one to 
{{phoenix-regionserver}}?


was (Author: jamestaylor):
Sure, that's an improvement. Maybe we should rename the other one to 
{phoenix-regionserver}?

> Disambiguate server jar from phoenix-server module jar
> --
>
> Key: PHOENIX-2267
> URL: https://issues.apache.org/jira/browse/PHOENIX-2267
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Andrew Purtell
>Priority: Minor
>  Labels: beginner
>
> For installation of Phoenix into the RegionServer, the install documentation 
> says
> {noformat}
> Add the phoenix-[version]-server.jar to the classpath of all HBase region
> server and master and remove any previous version. An easy way to do
> this is to copy it into the HBase lib directory
> {noformat}
> This correctly identifies the server uber jar, but unfortunately the 
> phoenix-server module produces a much smaller jar missing all of the classes 
> from phoenix-core as phoenix-server-.jar. I know people can become 
> confused about this because I've seen that happen. Make the wrong choice 
> during an upgrade and all of the RegionServers will crash upon restart with 
> CNFEs, which is an unfortunate outcome to say the least.
> I'd suggest renaming the phoenix-server module to something else in order to 
> disambiguate between the two.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-2267) Disambiguate server jar from phoenix-server module jar

2015-11-03 Thread James Taylor (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-2267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988472#comment-14988472
 ] 

James Taylor commented on PHOENIX-2267:
---

Sure, that's an improvement. Maybe we should rename the other one to 
{phoenix-regionserver}?

> Disambiguate server jar from phoenix-server module jar
> --
>
> Key: PHOENIX-2267
> URL: https://issues.apache.org/jira/browse/PHOENIX-2267
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Andrew Purtell
>Priority: Minor
>  Labels: beginner
>
> For installation of Phoenix into the RegionServer, the install documentation 
> says
> {noformat}
> Add the phoenix-[version]-server.jar to the classpath of all HBase region
> server and master and remove any previous version. An easy way to do
> this is to copy it into the HBase lib directory
> {noformat}
> This correctly identifies the server uber jar, but unfortunately the 
> phoenix-server module produces a much smaller jar missing all of the classes 
> from phoenix-core as phoenix-server-.jar. I know people can become 
> confused about this because I've seen that happen. Make the wrong choice 
> during an upgrade and all of the RegionServers will crash upon restart with 
> CNFEs, which is an unfortunate outcome to say the least.
> I'd suggest renaming the phoenix-server module to something else in order to 
> disambiguate between the two.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-2365) PHOENIX-2248 sub-task: Remove Pherf uber jar and create python scripts to execute from Phoenix bin directory

2015-11-03 Thread James Taylor (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988465#comment-14988465
 ] 

James Taylor commented on PHOENIX-2365:
---

+1. Nice improvement, [~mujtabachohan]. You'll do the same for the other jars?

> PHOENIX-2248 sub-task: Remove Pherf uber jar and create python scripts to 
> execute from Phoenix bin directory
> 
>
> Key: PHOENIX-2365
> URL: https://issues.apache.org/jira/browse/PHOENIX-2365
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Mujtaba Chohan
>Priority: Minor
> Attachments: PHOENIX-2248.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Build failed in Jenkins: Phoenix-4.x-HBase-1.0 #206

2015-11-03 Thread Cody Marcel 
I don't think so. Tests are passing locally for me.

On Tue, Nov 3, 2015 at 1:04 PM, Mujtaba Chohan  wrote:

> Cody - Is this NPE in Pherf test related to your change?
>
>
> On Mon, Nov 2, 2015 at 5:58 PM, Apache Jenkins Server <
> jenk...@builds.apache.org> wrote:
>
>> See 
>>
>> Changes:
>>
>> [cmarcel] PHOENIX-2366 - Pherf NPE on Data loader
>>
>> --
>> [...truncated 1873 lines...]
>> Downloading:
>> https://oss.sonatype.org/content/repositories/snapshots/commons-io/commons-io/maven-metadata.xml
>> Downloading:
>> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/maven-metadata.xml
>> 725/725 B
>> 374/374 B   725/725 B
>>
>> Downloading:
>> http://repository.apache.org/snapshots/commons-io/commons-io/maven-metadata.xml
>>
>> Downloaded:
>> https://repository.apache.org/content/repositories/releases/commons-io/commons-io/maven-metadata.xml
>> (725 B at 3.7 KB/sec)
>> 374/374 B   350/350 B
>> Downloading:
>> http://repo.maven.apache.org/maven2/commons-io/commons-io/maven-metadata.xml
>>
>> Downloaded:
>> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/maven-metadata.xml
>> (374 B at 1.8 KB/sec)
>>
>>
>> Downloaded:
>> http://repository.apache.org/snapshots/commons-io/commons-io/maven-metadata.xml
>> (350 B at 4.2 KB/sec)
>> 725/725 B
>>
>> Downloaded:
>> http://repo.maven.apache.org/maven2/commons-io/commons-io/maven-metadata.xml
>> (725 B at 13.4 KB/sec)
>> Downloading:
>> http://conjars.org/repo/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
>> Downloading:
>> https://repository.apache.org/content/repositories/releases/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
>> Downloading:
>> https://oss.sonatype.org/content/repositories/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
>> Downloading:
>> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
>>
>> 3/3 KB
>> Downloading:
>> http://repository.apache.org/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
>> 3/3 KB   3/3 KB
>>
>> Downloaded:
>> http://repository.apache.org/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
>> (3 KB at 30.6 KB/sec)
>>
>>
>> Downloaded:
>> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
>> (3 KB at 16.6 KB/sec)
>>
>> Downloading:
>> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
>> Downloading:
>> http://conjars.org/repo/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
>> Downloading:
>> https://repository.apache.org/content/repositories/releases/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
>> Downloading:
>> https://oss.sonatype.org/content/repositories/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
>>
>> Downloading:
>> http://repository.apache.org/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
>> 3/3 KB
>> 3/3 KB   3/3 KB
>>
>> Downloaded:
>> http://repository.apache.org/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
>> (3 KB at 27.2 KB/sec)
>>
>>
>> Downloaded:
>> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
>> (3 KB at 10.6 KB/sec)
>>
>> [INFO]
>> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ phoenix-pherf
>> ---
>> [INFO]
>> [INFO] --- maven-checkstyle-plugin:2.13:check (validate) @ phoenix-pherf
>> ---
>> [INFO]
>> [INFO] --- build-helper-maven-plugin:1.9.1:add-test-source
>> (add-test-source) @ phoenix-pherf ---
>> [INFO] Test Source directory: <
>> https://builds.apache.org/job/Phoenix-4.x-HBase-1.0/ws/phoenix-pherf/src/it/java>
>> added.
>> [INFO]
>> [INFO] --- build-helper-maven-plugin:1.9.1:add-test-resource
>> (add-test-resource) @ phoenix-pherf ---
>> [INFO]
>> [INFO] --- maven-remote-resources-plugin:1.5:process (default) @
>> phoenix-pherf ---
>> [INFO] artifact junit:junit-dep: checking for updates from apache release
>> [INFO] artifact junit:junit-dep: checking for updates from conjars.org
>> [INFO] artifact junit:junit-dep: checking for updates from apache snapshot
>> [INFO] artifact junit:junit-dep: checking for updates from
>> sonatype-nexus-snapshots
>> [INFO] artifact junit:junit-dep: checking for updates from central
>> [INFO] artifact junit:junit: checking for updates from apache release
>> [INFO] artifact junit:junit: checking for updates from conjars.org
>> [INFO] artifact junit:junit: checking for updates from apache snapshot
>> [INFO] artifact junit:junit: checking for updates from
>> sonatype-nexus-snapshots
>> [INFO] artifact junit:junit: checking for updates from central
>> [INFO] artifact commons-io:commons-io: checking for updates from apache
>> release
>> [INFO] artifact commons-io:commons-io: checking for updates from
>> co

[jira] [Updated] (PHOENIX-2365) PHOENIX-2248 sub-task: Remove Pherf uber jar and create python scripts to execute from Phoenix bin directory

2015-11-03 Thread Mujtaba Chohan (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-2365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mujtaba Chohan updated PHOENIX-2365:

Attachment: PHOENIX-2248.patch

This patch removes the huge Pherf uber jar and it's associated zip package. 
Also it converts pherf cluster and standalone scripts to Python in Phoenix/bin 
directory for easier execution and will also get bundled in Phoenix release 
tars.

[~cody.mar...@gmail.com] or [~jamestaylor] mind taking a look? Thanks!

> PHOENIX-2248 sub-task: Remove Pherf uber jar and create python scripts to 
> execute from Phoenix bin directory
> 
>
> Key: PHOENIX-2365
> URL: https://issues.apache.org/jira/browse/PHOENIX-2365
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Mujtaba Chohan
>Priority: Minor
> Attachments: PHOENIX-2248.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Build failed in Jenkins: Phoenix-4.x-HBase-1.0 #206

2015-11-03 Thread Mujtaba Chohan 
Cody - Is this NPE in Pherf test related to your change?


On Mon, Nov 2, 2015 at 5:58 PM, Apache Jenkins Server <
jenk...@builds.apache.org> wrote:

> See 
>
> Changes:
>
> [cmarcel] PHOENIX-2366 - Pherf NPE on Data loader
>
> --
> [...truncated 1873 lines...]
> Downloading:
> https://oss.sonatype.org/content/repositories/snapshots/commons-io/commons-io/maven-metadata.xml
> Downloading:
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/maven-metadata.xml
> 725/725 B
> 374/374 B   725/725 B
>
> Downloading:
> http://repository.apache.org/snapshots/commons-io/commons-io/maven-metadata.xml
>
> Downloaded:
> https://repository.apache.org/content/repositories/releases/commons-io/commons-io/maven-metadata.xml
> (725 B at 3.7 KB/sec)
> 374/374 B   350/350 B
> Downloading:
> http://repo.maven.apache.org/maven2/commons-io/commons-io/maven-metadata.xml
>
> Downloaded:
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/maven-metadata.xml
> (374 B at 1.8 KB/sec)
>
>
> Downloaded:
> http://repository.apache.org/snapshots/commons-io/commons-io/maven-metadata.xml
> (350 B at 4.2 KB/sec)
> 725/725 B
>
> Downloaded:
> http://repo.maven.apache.org/maven2/commons-io/commons-io/maven-metadata.xml
> (725 B at 13.4 KB/sec)
> Downloading:
> http://conjars.org/repo/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> Downloading:
> https://repository.apache.org/content/repositories/releases/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> Downloading:
> https://oss.sonatype.org/content/repositories/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> Downloading:
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
>
> 3/3 KB
> Downloading:
> http://repository.apache.org/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> 3/3 KB   3/3 KB
>
> Downloaded:
> http://repository.apache.org/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> (3 KB at 30.6 KB/sec)
>
>
> Downloaded:
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.5-SNAPSHOT/maven-metadata.xml
> (3 KB at 16.6 KB/sec)
>
> Downloading:
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> Downloading:
> http://conjars.org/repo/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> Downloading:
> https://repository.apache.org/content/repositories/releases/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> Downloading:
> https://oss.sonatype.org/content/repositories/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
>
> Downloading:
> http://repository.apache.org/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> 3/3 KB
> 3/3 KB   3/3 KB
>
> Downloaded:
> http://repository.apache.org/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> (3 KB at 27.2 KB/sec)
>
>
> Downloaded:
> https://repository.apache.org/content/repositories/snapshots/commons-io/commons-io/2.6-SNAPSHOT/maven-metadata.xml
> (3 KB at 10.6 KB/sec)
>
> [INFO]
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ phoenix-pherf ---
> [INFO]
> [INFO] --- maven-checkstyle-plugin:2.13:check (validate) @ phoenix-pherf
> ---
> [INFO]
> [INFO] --- build-helper-maven-plugin:1.9.1:add-test-source
> (add-test-source) @ phoenix-pherf ---
> [INFO] Test Source directory: <
> https://builds.apache.org/job/Phoenix-4.x-HBase-1.0/ws/phoenix-pherf/src/it/java>
> added.
> [INFO]
> [INFO] --- build-helper-maven-plugin:1.9.1:add-test-resource
> (add-test-resource) @ phoenix-pherf ---
> [INFO]
> [INFO] --- maven-remote-resources-plugin:1.5:process (default) @
> phoenix-pherf ---
> [INFO] artifact junit:junit-dep: checking for updates from apache release
> [INFO] artifact junit:junit-dep: checking for updates from conjars.org
> [INFO] artifact junit:junit-dep: checking for updates from apache snapshot
> [INFO] artifact junit:junit-dep: checking for updates from
> sonatype-nexus-snapshots
> [INFO] artifact junit:junit-dep: checking for updates from central
> [INFO] artifact junit:junit: checking for updates from apache release
> [INFO] artifact junit:junit: checking for updates from conjars.org
> [INFO] artifact junit:junit: checking for updates from apache snapshot
> [INFO] artifact junit:junit: checking for updates from
> sonatype-nexus-snapshots
> [INFO] artifact junit:junit: checking for updates from central
> [INFO] artifact commons-io:commons-io: checking for updates from apache
> release
> [INFO] artifact commons-io:commons-io: checking for updates from
> conjars.org
> [INFO] artifact commons-io:commons-io: checking for updates from apache
> snapshot
> [INFO] artifact commons-io:commons-io: checking for updates from
> sonatype-nexus-snapshots
> [INFO] artifact commons-io:commons-io: checking

[jira] [Commented] (PHOENIX-2361) Use single byte value for empty key value

2015-11-03 Thread Thomas D'Silva (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988090#comment-14988090
 ] 

Thomas D'Silva commented on PHOENIX-2361:
-

+1 LGTM

> Use single byte value for empty key value
> -
>
> Key: PHOENIX-2361
> URL: https://issues.apache.org/jira/browse/PHOENIX-2361
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: James Taylor
> Attachments: PHOENIX-2361.patch, PHOENIX-2361_addendum1.patch, 
> PHOENIX-2361_v2.patch
>
>
> To save a few bytes, we should use a single byte value for the empty key 
> value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-2343) Pherf - Add ability to draw comparison chart with average query run time in addition to existing minimum query time

2015-11-03 Thread Cody Marcel (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988015#comment-14988015
 ] 

Cody Marcel commented on PHOENIX-2343:
--

+1

> Pherf - Add ability to draw comparison chart with average query run time in 
> addition to existing minimum query time
> ---
>
> Key: PHOENIX-2343
> URL: https://issues.apache.org/jira/browse/PHOENIX-2343
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Mujtaba Chohan
>Assignee: Mujtaba Chohan
>Priority: Minor
> Attachments: PHOENIX-2343.patch
>
>
> Pherf - Add ability to draw comparison chart with average query run time in 
> addition to existing minimum query time



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: PhoenixHbaseStorage to Skip invalid rows

2015-11-03 Thread Jan Fernando 
+1 on making this change. Can you file a JIRA for it?

On Mon, Nov 2, 2015 at 4:31 PM, Siddhi Mehta  wrote:

> Hey All,
>
> I wanted to add a notion of skipping invalid rows for PhoenixHbaseStorage
> similar to how the CSVBulkLoad tool has an option of ignoring the bad
> rows.I did some work on the apache pig code that allows Storers to have a
> notion of Customizable/Configurable Errors PIG-4704
> .
>
> I wanted to plug this behavior for PhoenixHbaseStorage and propose certain
> changes for the same.
>
> *Current Behavior/Problem:*
>
> PhoenixRecordWriter makes use of executeBatch() to process rows once batch
> size is reached. If there are any client side validation/syntactical errors
> like data not fitting the column size, executeBatch() throws an exception
> and there is no-way to retrieve the valid rows from the batch and retry
> them. We discard the whole batch or fail the job without errorhandling.
>
> With auto commit set to false execute() also servers the purpose of not
> making any rpc calls  but does a bunch of validation client side and adds
> it to the client cache of mutation.
>
> On conn.commit() we make a rpc call.
>
> *Proposed Change*
>
> To be able to use Configurable ErrorHandling and ignore only the failed
> records instead of discarding the whole batch I want to propose changing
> the behavior in PhoenixRecordWriter from execute to executeBatch() or
> having a configuration to toggle between the 2 behaviors
> Thoughts?
>

[jira] [Commented] (PHOENIX-2288) Phoenix-Spark: PDecimal precision and scale aren't carried through to Spark DataFrame

2015-11-03 Thread James Taylor (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987765#comment-14987765
 ] 

James Taylor commented on PHOENIX-2288:
---

Thanks, [~jmahonin]. Have you addressed this feedback yet? 
https://github.com/apache/phoenix/pull/124#discussion_r42823983

> Phoenix-Spark: PDecimal precision and scale aren't carried through to Spark 
> DataFrame
> -
>
> Key: PHOENIX-2288
> URL: https://issues.apache.org/jira/browse/PHOENIX-2288
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.5.2
>Reporter: Josh Mahonin
> Attachments: PHOENIX-2288.patch
>
>
> When loading a Spark dataframe from a Phoenix table with a 'DECIMAL' type, 
> the underlying precision and scale aren't carried forward to Spark.
> The Spark catalyst schema converter should load these from the underlying 
> column. These appear to be exposed in the ResultSetMetaData, but if there was 
> a way to expose these somehow through ColumnInfo, it would be cleaner.
> I'm not sure if Pig has the same issues or not, but I suspect it may.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-2288) Phoenix-Spark: PDecimal precision and scale aren't carried through to Spark DataFrame

2015-11-03 Thread Josh Mahonin (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Mahonin updated PHOENIX-2288:
--
Attachment: PHOENIX-2288.patch

Github PR + Spark unit test

> Phoenix-Spark: PDecimal precision and scale aren't carried through to Spark 
> DataFrame
> -
>
> Key: PHOENIX-2288
> URL: https://issues.apache.org/jira/browse/PHOENIX-2288
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.5.2
>Reporter: Josh Mahonin
> Attachments: PHOENIX-2288.patch
>
>
> When loading a Spark dataframe from a Phoenix table with a 'DECIMAL' type, 
> the underlying precision and scale aren't carried forward to Spark.
> The Spark catalyst schema converter should load these from the underlying 
> column. These appear to be exposed in the ResultSetMetaData, but if there was 
> a way to expose these somehow through ColumnInfo, it would be cleaner.
> I'm not sure if Pig has the same issues or not, but I suspect it may.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Announcing phoenix-for-cloudera 4.6.0

2015-11-03 Thread Andrew Purtell 
Today I pushed a new branch '4.6-HBase-1.0-cdh5' and the tag
'v4.6.0-cdh5.4.5' (58fcfa6) to
https://github.com/chiastic-security/phoenix-for-cloudera. This is the
Phoenix 4.6.0 release, modified to build against CDH 5.4.5 and possibly
(but not tested) subsequent CDH releases.

If you want release tarballs I built from this, get them here:

Binaries

http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.6.0-cdh5.4.5-bin.tar.gz

http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.6.0-cdh5.4.5-bin.tar.gz.asc
 (signature)

http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.6.0-cdh5.4.5-bin.tar.gz.md5
(MD5 sum)

http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.6.0-cdh5.4.5-bin.tar.gz.sha
 (SHA-1 sum)


Source

http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.6.0-cdh5.4.5-src.tar.gz

http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.6.0-cdh5.4.5-src.tar.gz.asc
 (signature)

http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.6.0-cdh5.4.5-src.tar.gz.md5
(MD5 sum)

http://apurtell.s3.amazonaws.com/phoenix/phoenix-4.6.0-cdh5.4.5-src.tar.gz.sha
 (SHA1-sum)


Signed with my code signing key D5365CCD.

​The source and these binaries incorporate changes from the Cloudera Labs
fork of Phoenix (https://github.com/cloudera-labs/phoenix), licensed under
the ASL v2, Neither the source or binary artifacts are in any way
"official" or supported by the Apache Phoenix project. The source and
artifacts are provided by me in a personal capacity for the convenience of
would-be Phoenix users that also use CDH. Please don't contact the Apache
Phoenix project for any issues regarding this source and these binaries.

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

[jira] [Updated] (PHOENIX-2362) Rollback doesn't work when checkpointing is done

2015-11-03 Thread James Taylor (JIRA) 

 [ 
https://issues.apache.org/jira/browse/PHOENIX-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-2362:
--
Summary: Rollback doesn't work when checkpointing is done  (was: Rollback 
doesn't work when checkpointing occurs)

> Rollback doesn't work when checkpointing is done
> 
>
> Key: PHOENIX-2362
> URL: https://issues.apache.org/jira/browse/PHOENIX-2362
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: James Taylor
> Attachments: PHOENIX-2362.patch, PHOENIX-2362_v2.patch, 
> PHOENIX-2362_v3.patch, PHOENIX-2362_v4.patch, PHOENIX-2362_v5.patch, 
> PHOENIX-2362_v6.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-2288) Phoenix-Spark: PDecimal precision and scale aren't carried through to Spark DataFrame

2015-11-03 Thread Josh Mahonin (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987453#comment-14987453
 ] 

Josh Mahonin commented on PHOENIX-2288:
---

And follow-up question, what branches should this go on? master, 4.x, 4.6?

> Phoenix-Spark: PDecimal precision and scale aren't carried through to Spark 
> DataFrame
> -
>
> Key: PHOENIX-2288
> URL: https://issues.apache.org/jira/browse/PHOENIX-2288
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.5.2
>Reporter: Josh Mahonin
>
> When loading a Spark dataframe from a Phoenix table with a 'DECIMAL' type, 
> the underlying precision and scale aren't carried forward to Spark.
> The Spark catalyst schema converter should load these from the underlying 
> column. These appear to be exposed in the ResultSetMetaData, but if there was 
> a way to expose these somehow through ColumnInfo, it would be cleaner.
> I'm not sure if Pig has the same issues or not, but I suspect it may.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-2288) Phoenix-Spark: PDecimal precision and scale aren't carried through to Spark DataFrame

2015-11-03 Thread Josh Mahonin (JIRA) 

[ 
https://issues.apache.org/jira/browse/PHOENIX-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987440#comment-14987440
 ] 

Josh Mahonin commented on PHOENIX-2288:
---

Any issues with this PR [~jamestaylor] [~maghamraviki...@gmail.com] ?

I've updated the ignored unit test and verifies the precision and scale are 
carried forward from schema creation to Spark.

> Phoenix-Spark: PDecimal precision and scale aren't carried through to Spark 
> DataFrame
> -
>
> Key: PHOENIX-2288
> URL: https://issues.apache.org/jira/browse/PHOENIX-2288
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.5.2
>Reporter: Josh Mahonin
>
> When loading a Spark dataframe from a Phoenix table with a 'DECIMAL' type, 
> the underlying precision and scale aren't carried forward to Spark.
> The Spark catalyst schema converter should load these from the underlying 
> column. These appear to be exposed in the ResultSetMetaData, but if there was 
> a way to expose these somehow through ColumnInfo, it would be cleaner.
> I'm not sure if Pig has the same issues or not, but I suspect it may.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)