[jira] [Commented] (PHOENIX-3185) Error: ERROR 514 (42892): A duplicate column name was detected in the object definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 (state=42892,code=514)
[ https://issues.apache.org/jira/browse/PHOENIX-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425944#comment-15425944 ] Kalyan commented on PHOENIX-3185: - Thanks William Yang , I am not aware of this issue then please make this issue as duplicate. James Taylor, please ignore my previous patch. sorry about my mistake. > Error: ERROR 514 (42892): A duplicate column name was detected in the object > definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 > (state=42892,code=514) > - > > Key: PHOENIX-3185 > URL: https://issues.apache.org/jira/browse/PHOENIX-3185 > Project: Phoenix > Issue Type: Bug >Reporter: Kalyan >Assignee: Kalyan > Fix For: 4.8.1 > > Attachments: image1.png, image2.png, image3.png, > phoenix_duplicate_column_check.patch > > > create a table with duplicate columns. > create table test_table (rowkey varchar primary key, c1 varchar, c2 integer, > c3 double, c1 bigint); > The below exception getting .. no issue with exception. But the problem is > phoenix is creating table with exception & later we are not able to delete > the table also. We need to fix this Bug > 0: jdbc:phoenix:localhost> create table test_table (rowkey varchar primary > key, c1 varchar, c2 integer, c3 double, c1 bigint); > Error: ERROR 514 (42892): A duplicate column name was detected in the object > definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 > (state=42892,code=514) > org.apache.phoenix.schema.ColumnAlreadyExistsException: ERROR 514 (42892): A > duplicate column name was detected in the object definition or ALTER TABLE > statement. columnName=TEST_TABLE.C1 > at org.apache.phoenix.schema.PTableImpl.init(PTableImpl.java:415) > at org.apache.phoenix.schema.PTableImpl.(PTableImpl.java:315) > at org.apache.phoenix.schema.PTableImpl.makePTable(PTableImpl.java:288) > at > org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2146) > at > org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:828) > at > org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:338) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:326) > at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:324) > at > org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1345) > at sqlline.Commands.execute(Commands.java:822) > at sqlline.Commands.sql(Commands.java:732) > at sqlline.SqlLine.dispatch(SqlLine.java:808) > at sqlline.SqlLine.begin(SqlLine.java:681) > at sqlline.SqlLine.start(SqlLine.java:398) > at sqlline.SqlLine.main(SqlLine.java:292) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3185) Error: ERROR 514 (42892): A duplicate column name was detected in the object definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 (state=42892,code=514)
[ https://issues.apache.org/jira/browse/PHOENIX-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425796#comment-15425796 ] William Yang commented on PHOENIX-3185: --- I'm afraid we already has an issue and a applicable patch for this bug: (PHOENIX-930)[https://issues.apache.org/jira/browse/PHOENIX-930] > Error: ERROR 514 (42892): A duplicate column name was detected in the object > definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 > (state=42892,code=514) > - > > Key: PHOENIX-3185 > URL: https://issues.apache.org/jira/browse/PHOENIX-3185 > Project: Phoenix > Issue Type: Bug >Reporter: Kalyan >Assignee: Kalyan > Fix For: 4.8.1 > > Attachments: image1.png, image2.png, image3.png, > phoenix_duplicate_column_check.patch > > > create a table with duplicate columns. > create table test_table (rowkey varchar primary key, c1 varchar, c2 integer, > c3 double, c1 bigint); > The below exception getting .. no issue with exception. But the problem is > phoenix is creating table with exception & later we are not able to delete > the table also. We need to fix this Bug > 0: jdbc:phoenix:localhost> create table test_table (rowkey varchar primary > key, c1 varchar, c2 integer, c3 double, c1 bigint); > Error: ERROR 514 (42892): A duplicate column name was detected in the object > definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 > (state=42892,code=514) > org.apache.phoenix.schema.ColumnAlreadyExistsException: ERROR 514 (42892): A > duplicate column name was detected in the object definition or ALTER TABLE > statement. columnName=TEST_TABLE.C1 > at org.apache.phoenix.schema.PTableImpl.init(PTableImpl.java:415) > at org.apache.phoenix.schema.PTableImpl.(PTableImpl.java:315) > at org.apache.phoenix.schema.PTableImpl.makePTable(PTableImpl.java:288) > at > org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2146) > at > org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:828) > at > org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:338) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:326) > at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:324) > at > org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1345) > at sqlline.Commands.execute(Commands.java:822) > at sqlline.Commands.sql(Commands.java:732) > at sqlline.SqlLine.dispatch(SqlLine.java:808) > at sqlline.SqlLine.begin(SqlLine.java:681) > at sqlline.SqlLine.start(SqlLine.java:398) > at sqlline.SqlLine.main(SqlLine.java:292) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PHOENIX-3194) Document Hive integration
[ https://issues.apache.org/jira/browse/PHOENIX-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Soldatov reassigned PHOENIX-3194: Assignee: Sergey Soldatov > Document Hive integration > - > > Key: PHOENIX-3194 > URL: https://issues.apache.org/jira/browse/PHOENIX-3194 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Sergey Soldatov > > We should document our Hive integration similar to the way we've documented > our Spark integration[1] and Pig integration[2]. This would focus on how to > use it (as opposed to how it was implemented), limitations, version > requirements, and include examples and any required/optional config > parameters or other setup required. > [1] http://phoenix.apache.org/phoenix_spark.html > [2] http://phoenix.apache.org/pig_integration.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3194) Document Hive integration
[ https://issues.apache.org/jira/browse/PHOENIX-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425763#comment-15425763 ] James Taylor commented on PHOENIX-3194: --- [~sergey.soldatov] - could you take a stab at this. We author our website typically using markdown. See http://phoenix.apache.org/building_website.html and look at phoenix_spark.md as a guide. The menus are driven by site.xml. > Document Hive integration > - > > Key: PHOENIX-3194 > URL: https://issues.apache.org/jira/browse/PHOENIX-3194 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor > > We should document our Hive integration similar to the way we've documented > our Spark integration[1] and Pig integration[2]. This would focus on how to > use it (as opposed to how it was implemented), limitations, version > requirements, and include examples and any required/optional config > parameters or other setup required. > [1] http://phoenix.apache.org/phoenix_spark.html > [2] http://phoenix.apache.org/pig_integration.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3194) Document Hive integration
James Taylor created PHOENIX-3194: - Summary: Document Hive integration Key: PHOENIX-3194 URL: https://issues.apache.org/jira/browse/PHOENIX-3194 Project: Phoenix Issue Type: Bug Reporter: James Taylor We should document our Hive integration similar to the way we've documented our Spark integration[1] and Pig integration[2]. This would focus on how to use it (as opposed to how it was implemented), limitations, version requirements, and include examples and any required/optional config parameters or other setup required. [1] http://phoenix.apache.org/phoenix_spark.html [2] http://phoenix.apache.org/pig_integration.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2807) Update logging and explain plan to show hbase tables instead of phoenix names
[ https://issues.apache.org/jira/browse/PHOENIX-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-2807: -- Issue Type: Improvement (was: Sub-task) Parent: (was: PHOENIX-1311) > Update logging and explain plan to show hbase tables instead of phoenix names > - > > Key: PHOENIX-2807 > URL: https://issues.apache.org/jira/browse/PHOENIX-2807 > Project: Phoenix > Issue Type: Improvement >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Minor > Fix For: 4.9.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-2835) Add documentation for schema constructs and upgrade utility
[ https://issues.apache.org/jira/browse/PHOENIX-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor resolved PHOENIX-2835. --- Resolution: Fixed > Add documentation for schema constructs and upgrade utility > --- > > Key: PHOENIX-2835 > URL: https://issues.apache.org/jira/browse/PHOENIX-2835 > Project: Phoenix > Issue Type: Sub-task >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Minor > Fix For: 4.8.0 > > Attachments: PHOENIX-2835.patch, PHOENIX-2835_v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-2835) Add documentation for schema constructs and upgrade utility
[ https://issues.apache.org/jira/browse/PHOENIX-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-2835: -- Fix Version/s: (was: 4.9.0) 4.8.0 > Add documentation for schema constructs and upgrade utility > --- > > Key: PHOENIX-2835 > URL: https://issues.apache.org/jira/browse/PHOENIX-2835 > Project: Phoenix > Issue Type: Sub-task >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Minor > Fix For: 4.8.0 > > Attachments: PHOENIX-2835.patch, PHOENIX-2835_v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-2835) Add documentation for schema constructs and upgrade utility
[ https://issues.apache.org/jira/browse/PHOENIX-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425616#comment-15425616 ] James Taylor commented on PHOENIX-2835: --- Looks good. I checked it in on your behalf. > Add documentation for schema constructs and upgrade utility > --- > > Key: PHOENIX-2835 > URL: https://issues.apache.org/jira/browse/PHOENIX-2835 > Project: Phoenix > Issue Type: Sub-task >Reporter: Ankit Singhal >Assignee: Ankit Singhal >Priority: Minor > Fix For: 4.9.0 > > Attachments: PHOENIX-2835.patch, PHOENIX-2835_v1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3188) Making renew leases of scanners more resilient
[ https://issues.apache.org/jira/browse/PHOENIX-3188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425601#comment-15425601 ] Andrew Purtell commented on PHOENIX-3188: - That sounds like a better approach than using protected member variables. Please let us know if you still need a blessed way to get a scanner ID. > Making renew leases of scanners more resilient > -- > > Key: PHOENIX-3188 > URL: https://issues.apache.org/jira/browse/PHOENIX-3188 > Project: Phoenix > Issue Type: Improvement >Reporter: Samarth Jain > Attachments: PHOENIX-3188_hack.patch > > > For renewing scanner leases, we need to perform an RPC per lease renewal. It > could happen that under load, these calls won't be picked up by region > handler threads soon enough which could cause queries to fail. There are a > couple of ways to solve this problem: > 1) Issue renew lease calls at a higher priority. This essentially causes a > different thread pool to be used on the region servers. One such example is > the metadata and index updates calls we make. This would scale well unless > these special thread pools themselves get saturated with requests. > 2) Batch up the the renewLease rpc calls. If HBase is able to provide us > scanner names and ids, then we can potentially renew leases for multiple > scanners by batching them up in one rpc. This would entail changes in both > client and server side of HBase. Client side - to expose scanner name. Server > side - to expose renewLease call on a scanner given a scanner name/id. > We still need to fix renewing leases for non-aggregate queries though. See > PHOENIX-1751 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-808) Create snapshot of system tables prior to upgrade and restore on any failure
[ https://issues.apache.org/jira/browse/PHOENIX-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425417#comment-15425417 ] Samarth Jain commented on PHOENIX-808: -- We can always attempt to restore. Like you said, if we can't restore SYSTEM.STATS, then we can always regenerate the stats. Either way, in case of restore failures, we won't be deleting the snapshot that we failed to restore. > Create snapshot of system tables prior to upgrade and restore on any failure > > > Key: PHOENIX-808 > URL: https://issues.apache.org/jira/browse/PHOENIX-808 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Samarth Jain > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3009) Estimated Size for PTable is ~20% greater than actual size
[
https://issues.apache.org/jira/browse/PHOENIX-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425419#comment-15425419
]
James Taylor commented on PHOENIX-3009:
---
[~tdsilva] - would you mind taking this one after PHOENIX-3148, as you
mentioned seeing the real size being more like 2500 bytes instead of 6000?
> Estimated Size for PTable is ~20% greater than actual size
> --
>
> Key: PHOENIX-3009
> URL: https://issues.apache.org/jira/browse/PHOENIX-3009
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Mujtaba Chohan
>Assignee: Thomas D'Silva
> Attachments: PHOENIX-3009.patch, ptable_retained.png
>
>
> {{PTable.estimatedSize}} returns size that is around 20% higher than actual
> PTable size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3009) Estimated Size for PTable is ~20% greater than actual size
[
https://issues.apache.org/jira/browse/PHOENIX-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-3009:
--
Assignee: Thomas D'Silva
> Estimated Size for PTable is ~20% greater than actual size
> --
>
> Key: PHOENIX-3009
> URL: https://issues.apache.org/jira/browse/PHOENIX-3009
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Mujtaba Chohan
>Assignee: Thomas D'Silva
> Attachments: PHOENIX-3009.patch, ptable_retained.png
>
>
> {{PTable.estimatedSize}} returns size that is around 20% higher than actual
> PTable size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-808) Create snapshot of system tables prior to upgrade and restore on any failure
[ https://issues.apache.org/jira/browse/PHOENIX-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425414#comment-15425414 ] James Taylor commented on PHOENIX-808: -- We could snapshot system.stats, but the data is more or less transient. Losing it just means you have to generate stats again. If we snapshot more than one table, it's a bit tricky to restore. What if we successfully restore one but not the other? > Create snapshot of system tables prior to upgrade and restore on any failure > > > Key: PHOENIX-808 > URL: https://issues.apache.org/jira/browse/PHOENIX-808 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Samarth Jain > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-808) Create snapshot of system tables prior to upgrade and restore on any failure
[
https://issues.apache.org/jira/browse/PHOENIX-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425406#comment-15425406
]
Samarth Jain commented on PHOENIX-808:
--
Agreed on not snapshotting the system.sequence table as we could end up losing
increments on sequence values. Any reason why we shouldn't SYSTEM.STATS table?
What if the upgrade for SYSTEM.STATS fails? For ex - when we changed the schema
back in 4.3 for the SYSTEM.STATS table:
{code}
metaConnection = addColumnsIfNotExists(
metaConnection,
SYSTEM_STATS_NAME,
MetaDataProtocol.MIN_SYSTEM_TABLE_TIMESTAMP,
PhoenixDatabaseMetaData.GUIDE_POSTS_ROW_COUNT + " "
+
PLong.INSTANCE.getSqlTypeName());
{code}
> Create snapshot of system tables prior to upgrade and restore on any failure
>
>
> Key: PHOENIX-808
> URL: https://issues.apache.org/jira/browse/PHOENIX-808
> Project: Phoenix
> Issue Type: Task
>Reporter: James Taylor
>Assignee: Samarth Jain
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-808) Create snapshot of system tables prior to upgrade and restore on any failure
[ https://issues.apache.org/jira/browse/PHOENIX-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425402#comment-15425402 ] James Taylor commented on PHOENIX-808: -- IMHO, we should create only a snapshot of the system.catalog. We definitely don't want to snapshot the system.stats and system.tracing. Snapshoting and restoring the system.sequence table sounds risky. Making a snapshot of the system.function table might be fine too. > Create snapshot of system tables prior to upgrade and restore on any failure > > > Key: PHOENIX-808 > URL: https://issues.apache.org/jira/browse/PHOENIX-808 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Samarth Jain > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (PHOENIX-808) Create snapshot of system tables prior to upgrade and restore on any failure
[ https://issues.apache.org/jira/browse/PHOENIX-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain reassigned PHOENIX-808: Assignee: Samarth Jain > Create snapshot of system tables prior to upgrade and restore on any failure > > > Key: PHOENIX-808 > URL: https://issues.apache.org/jira/browse/PHOENIX-808 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Samarth Jain > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [DISCUSS] RM for next release
Done. On Wed, Aug 17, 2016 at 1:38 PM, wrote: > Great. Can somebody make me an administrator for Phoenix in Jira?I am for > HBase but not for Phoenix. > -- Lars > From: James Taylor > To: "dev@phoenix.apache.org" ; lars hofhansl < > la...@apache.org> > Sent: Tuesday, August 16, 2016 10:45 AM > Subject: Re: [DISCUSS] RM for next release > > Thanks, Lars. I don't think we need a vote - you have everyone's > overwhelming support. If you're up for being both the 4.8.1 and 4.9 RM, > that's be awesome. I'm hoping we can do 4.8.1 sooner rather than later as > there are already a number of crucial fixes checked in. Can we document a > target date for the releases? Other comments inline below. > > On Mon, Aug 15, 2016 at 2:33 PM, wrote: > > > Should we put this up for a vote? > > There are a few "conditions" that need to be met, IMHO, to be successful > > with a monthly release cadence. > > 1. Ability to skip releases. When we do monthly releases we can not > expect > > user to follow along. So it has to be possible to upgrade straight from > > (say) 4.8.0 to 4.9.2. The upgrade paths would need to be defensively > > implemented and tested. > > > Yes, our upgrades support this and we have tests around this. They're run > internally at SFDC currently, but Mujtaba is working on open sourcing them > and having them run on ASF Jenkins jobs. > > 2. 100% stable test suite. Haven't looked at the Phoenix tests lately, but > > with many releases in fairly quick succession, we need to be able to > derive > > confidence from the test suite. > > > Yes, our tests are in good shape. We spent a lot of time on this over the > last couple of releases. The failures we get are typically hiccups in ASF > systems. > > > > 2a. wide test coverage. Corollary to #2. Automated confidence needs to be > > high. Even more so as we do many releases. I'd expect some of the > > committers to spend some time to add tests. > > > > We've improved this as well, but there's always more to do. One area we're > working on for 4.9 is improving the test suite run speed so that we can > potentially run more combinations of features. Also, there's been some good > new stress-related tests developed by the folks over at Hortonworks that > we're hoping will become open sources. > > > > 3. The community and the PMC needs to be behind monthly release. Every > > month there will be a release to test and to vote. At least 3 PMCs need > to > > +1. > > > > Agreed. I think we can handle it. > > > > 3a. committers are diligent about backporting fixes to earlier branches. > > Everybody wants to work on the newest and coolest. At the same time, the > RM > > will not have enough B/W to decide what should be backported and to do > the > > actual backport. > > > > Agreed. > > > > 4. A compatibility story. Similar to the HBase one ( > > http://hbase.apache.org/book.html#hbase.versioning). I am happy to help > > drive this. We may have a good compatibility story already, it's even > > better if it's written down. > > > Our compatibility story is documented here: > https://phoenix.apache.org/upgrading.html. We actually support upgrade all > the way back to 4.2. We can update as needed. > > > > 5. Code quality. What follows is my opinion. No "beta" features. Those > > should go into a newer version. No 1/2 baked code committed. Committers > > should also look out for good code quality (like good comments, etc). Not > > saying that is not the case now, btw. > > > > Agreed. > > > > > > That's all I can think of offhand. If we can in principle agree on these, > > I am volunteering to be RM for 4.9 and/or 4.8.1. > > -- Lars > > From: Enis Söztutar > > To: "dev@phoenix.apache.org" > > Sent: Wednesday, July 13, 2016 6:49 PM > > Subject: Re: [DISCUSS] RM for next release > > > > Great, I think there is some consensus on: > > > > 1. Doing 4.8.1 release. > > 2. Dropping HBase-1.0 for 4.9+ > > 3. Lars being RM for 4.9 (and 4.8.1?) > > > > I just created 4.8.1 as a release version. Let's use that to track issues > > to backport. Committers, once 4.8.0 RC is cut, please start committing > > issues to 4.8 branches as well, and don't forget to set the fixVersion > with > > 4.8.1 as well. Only bug fixes should be committed unless we need to do an > > exception. > > > > Enis > > > > On Sun, Jul 10, 2016 at 9:39 AM, Andrew Purtell < > andrew.purt...@gmail.com> > > wrote: > > > > > It's the monthly sweep of commits on later branches, backporting, and > > > testing each change. So yeah it's the other end of the spectrum and > good > > as > > > another a data point. Where a Phoenix RM would fall on that spectrum > will > > > correlate with how long the long term support for a branch has been in > > > effect. > > > > > > I am also planning to run ITBLL with active chaos agents for 24 hours > for > > > every HBase RC once I figure out Dima's clusterdock stuff. Speaking of > > > which: We should have something like this for Phoenix. For the most > part > > we > > > can reu
[jira] [Updated] (PHOENIX-808) Create snapshot of system tables prior to upgrade and restore on any failure
[ https://issues.apache.org/jira/browse/PHOENIX-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-808: - Summary: Create snapshot of system tables prior to upgrade and restore on any failure (was: Create snapshot of SYSTEM.CATALOG prior to upgrade and restore on any failure) > Create snapshot of system tables prior to upgrade and restore on any failure > > > Key: PHOENIX-808 > URL: https://issues.apache.org/jira/browse/PHOENIX-808 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [DISCUSS] RM for next release
Great. Can somebody make me an administrator for Phoenix in Jira?I am for HBase but not for Phoenix. -- Lars From: James Taylor To: "dev@phoenix.apache.org" ; lars hofhansl Sent: Tuesday, August 16, 2016 10:45 AM Subject: Re: [DISCUSS] RM for next release Thanks, Lars. I don't think we need a vote - you have everyone's overwhelming support. If you're up for being both the 4.8.1 and 4.9 RM, that's be awesome. I'm hoping we can do 4.8.1 sooner rather than later as there are already a number of crucial fixes checked in. Can we document a target date for the releases? Other comments inline below. On Mon, Aug 15, 2016 at 2:33 PM, wrote: > Should we put this up for a vote? > There are a few "conditions" that need to be met, IMHO, to be successful > with a monthly release cadence. > 1. Ability to skip releases. When we do monthly releases we can not expect > user to follow along. So it has to be possible to upgrade straight from > (say) 4.8.0 to 4.9.2. The upgrade paths would need to be defensively > implemented and tested. Yes, our upgrades support this and we have tests around this. They're run internally at SFDC currently, but Mujtaba is working on open sourcing them and having them run on ASF Jenkins jobs. 2. 100% stable test suite. Haven't looked at the Phoenix tests lately, but > with many releases in fairly quick succession, we need to be able to derive > confidence from the test suite. Yes, our tests are in good shape. We spent a lot of time on this over the last couple of releases. The failures we get are typically hiccups in ASF systems. > 2a. wide test coverage. Corollary to #2. Automated confidence needs to be > high. Even more so as we do many releases. I'd expect some of the > committers to spend some time to add tests. > We've improved this as well, but there's always more to do. One area we're working on for 4.9 is improving the test suite run speed so that we can potentially run more combinations of features. Also, there's been some good new stress-related tests developed by the folks over at Hortonworks that we're hoping will become open sources. > 3. The community and the PMC needs to be behind monthly release. Every > month there will be a release to test and to vote. At least 3 PMCs need to > +1. > Agreed. I think we can handle it. > 3a. committers are diligent about backporting fixes to earlier branches. > Everybody wants to work on the newest and coolest. At the same time, the RM > will not have enough B/W to decide what should be backported and to do the > actual backport. > Agreed. > 4. A compatibility story. Similar to the HBase one ( > http://hbase.apache.org/book.html#hbase.versioning). I am happy to help > drive this. We may have a good compatibility story already, it's even > better if it's written down. Our compatibility story is documented here: https://phoenix.apache.org/upgrading.html. We actually support upgrade all the way back to 4.2. We can update as needed. > 5. Code quality. What follows is my opinion. No "beta" features. Those > should go into a newer version. No 1/2 baked code committed. Committers > should also look out for good code quality (like good comments, etc). Not > saying that is not the case now, btw. > Agreed. > > That's all I can think of offhand. If we can in principle agree on these, > I am volunteering to be RM for 4.9 and/or 4.8.1. > -- Lars > From: Enis Söztutar > To: "dev@phoenix.apache.org" > Sent: Wednesday, July 13, 2016 6:49 PM > Subject: Re: [DISCUSS] RM for next release > > Great, I think there is some consensus on: > > 1. Doing 4.8.1 release. > 2. Dropping HBase-1.0 for 4.9+ > 3. Lars being RM for 4.9 (and 4.8.1?) > > I just created 4.8.1 as a release version. Let's use that to track issues > to backport. Committers, once 4.8.0 RC is cut, please start committing > issues to 4.8 branches as well, and don't forget to set the fixVersion with > 4.8.1 as well. Only bug fixes should be committed unless we need to do an > exception. > > Enis > > On Sun, Jul 10, 2016 at 9:39 AM, Andrew Purtell > wrote: > > > It's the monthly sweep of commits on later branches, backporting, and > > testing each change. So yeah it's the other end of the spectrum and good > as > > another a data point. Where a Phoenix RM would fall on that spectrum will > > correlate with how long the long term support for a branch has been in > > effect. > > > > I am also planning to run ITBLL with active chaos agents for 24 hours for > > every HBase RC once I figure out Dima's clusterdock stuff. Speaking of > > which: We should have something like this for Phoenix. For the most part > we > > can reuse HBase's chaos stuff. We'd want to write and verify the same > type > > of linked data, with all interesting features enabled like transactions > and > > local indexes. If we had this then doing it for Phoenix RCs would likely > > fall to the RM. > > > > > > > On Jul 10, 2016, at 2:42 AM, > > wrote: > > > > > > 0.94 never took that m
[jira] [Commented] (PHOENIX-2995) Write performance severely degrades with large number of views
[
https://issues.apache.org/jira/browse/PHOENIX-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425316#comment-15425316
]
Hudson commented on PHOENIX-2995:
-
SUCCESS: Integrated in Jenkins build Phoenix-master #1364 (See
[https://builds.apache.org/job/Phoenix-master/1364/])
PHOENIX-2995 Write performance severely degrades with large number of (tdsilva:
rev 386cbbbf7a5dd736888e8e0bfe16e513d54e215c)
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixConnection.java
> Write performance severely degrades with large number of views
> ---
>
> Key: PHOENIX-2995
> URL: https://issues.apache.org/jira/browse/PHOENIX-2995
> Project: Phoenix
> Issue Type: Bug
>Reporter: Mujtaba Chohan
>Assignee: Thomas D'Silva
> Labels: Argus
> Fix For: 4.8.1
>
> Attachments: PHOENIX-2995-v2.patch, PHOENIX-2995-v3.patch,
> PHOENIX-2995-v4.patch, PHOENIX-2995-v5.patch, PHOENIX-2995-v6.patch,
> PHOENIX-2995.patch, create_view_and_upsert.png, image.png, image2.png,
> image3.png, upsert_rate.png
>
>
> Write performance for each 1K batch degrades significantly when there are
> *10K* views being written in random with default
> {{phoenix.client.maxMetaDataCacheSize}}. With all views created, upsert rate
> remains around 25 seconds per 1K batch i.e. ~2K rows/min upsert rate.
> When {{phoenix.client.maxMetaDataCacheSize}} is increased to 100MB+ then view
> does not need to get re-resolved and upsert rate gets back to normal ~60K
> rows/min.
> With *100K* views and {{phoenix.client.maxMetaDataCacheSize}} set to 1GB, I
> wasn't able create all 100K views as upsert time for each 1K batch keeps on
> steadily increasing.
> Following graph shows 1K batch upsert rate over time with variation of number
> of views. Rows are upserted to random views {{CREATE VIEW IF NOT EXISTS ...
> APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY=90}} is executed before
> upsert statement.
> !upsert_rate.png!
> Base table is also created with {{APPEND_ONLY_SCHEMA = true,
> UPDATE_CACHE_FREQUENCY = 90, AUTO_PARTITION_SEQ}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3193) Tracing UI cleanup - final tasks before GSoC pull request
[ https://issues.apache.org/jira/browse/PHOENIX-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425155#comment-15425155 ] Mujtaba Chohan commented on PHOENIX-3193: - FYI [~nishani] > Tracing UI cleanup - final tasks before GSoC pull request > - > > Key: PHOENIX-3193 > URL: https://issues.apache.org/jira/browse/PHOENIX-3193 > Project: Phoenix > Issue Type: Improvement >Reporter: Mujtaba Chohan >Assignee: Nishani > > Points from GSoC presentation on tracing imporvements: > *Tracing UI* > * Remove line chart > * In list page, run query with description, start_time, (end_time-start_time) > duration from T where trace_id = ? > * More space for descriptions on bar chart. Wrap if necessary > * Label for X axis on timeline sometime again start from 0, if X axis is in > seconds then it should not rollover after 60 seconds unless minutes are also > shown > * X-axis labeled as Node on various charts, but should be Percentage > *Zipkin* > * Flip zipkin chart on vertical axis with arrows going other way. So start > from the top level root on the leftmost side and work toward children on the > right. > * Ask zipkin community if there's a way to tell it that date/time is in > milliseconds. > *Overall* > * Please put together a pull request to the phoenix project to add the > zipkiin work you've done to the OS project. Ideally, include the zipkin work > in the phoenix-tracing module and call it phoeix-tracing. Only if there is > some major hurdle, create a new module. > * Test with trace_ids that have multiple spans with duration (end_time - > start_time) > 5 ms and verify that UI and Zipkin output shows the correct > corresponding timeline > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3193) Tracing UI cleanup - final tasks before GSoC pull request
[ https://issues.apache.org/jira/browse/PHOENIX-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mujtaba Chohan updated PHOENIX-3193: Assignee: Nishani > Tracing UI cleanup - final tasks before GSoC pull request > - > > Key: PHOENIX-3193 > URL: https://issues.apache.org/jira/browse/PHOENIX-3193 > Project: Phoenix > Issue Type: Improvement >Reporter: Mujtaba Chohan >Assignee: Nishani > > Points from GSoC presentation on tracing imporvements: > *Tracing UI* > * Remove line chart > * In list page, run query with description, start_time, (end_time-start_time) > duration from T where trace_id = ? > * More space for descriptions on bar chart. Wrap if necessary > * Label for X axis on timeline sometime again start from 0, if X axis is in > seconds then it should not rollover after 60 seconds unless minutes are also > shown > * X-axis labeled as Node on various charts, but should be Percentage > *Zipkin* > * Flip zipkin chart on vertical axis with arrows going other way. So start > from the top level root on the leftmost side and work toward children on the > right. > * Ask zipkin community if there's a way to tell it that date/time is in > milliseconds. > *Overall* > * Please put together a pull request to the phoenix project to add the > zipkiin work you've done to the OS project. Ideally, include the zipkin work > in the phoenix-tracing module and call it phoeix-tracing. Only if there is > some major hurdle, create a new module. > * Test with trace_ids that have multiple spans with duration (end_time - > start_time) > 5 ms and verify that UI and Zipkin output shows the correct > corresponding timeline > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3193) Tracing UI cleanup - final tasks before GSoC pull request
Mujtaba Chohan created PHOENIX-3193: --- Summary: Tracing UI cleanup - final tasks before GSoC pull request Key: PHOENIX-3193 URL: https://issues.apache.org/jira/browse/PHOENIX-3193 Project: Phoenix Issue Type: Improvement Reporter: Mujtaba Chohan Points from GSoC presentation on tracing imporvements: *Tracing UI* * Remove line chart * In list page, run query with description, start_time, (end_time-start_time) duration from T where trace_id = ? * More space for descriptions on bar chart. Wrap if necessary * Label for X axis on timeline sometime again start from 0, if X axis is in seconds then it should not rollover after 60 seconds unless minutes are also shown * X-axis labeled as Node on various charts, but should be Percentage *Zipkin* * Flip zipkin chart on vertical axis with arrows going other way. So start from the top level root on the leftmost side and work toward children on the right. * Ask zipkin community if there's a way to tell it that date/time is in milliseconds. *Overall* * Please put together a pull request to the phoenix project to add the zipkiin work you've done to the OS project. Ideally, include the zipkin work in the phoenix-tracing module and call it phoeix-tracing. Only if there is some major hurdle, create a new module. * Test with trace_ids that have multiple spans with duration (end_time - start_time) > 5 ms and verify that UI and Zipkin output shows the correct corresponding timeline -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-2995) Write performance severely degrades with large number of views
[
https://issues.apache.org/jira/browse/PHOENIX-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425139#comment-15425139
]
Hudson commented on PHOENIX-2995:
-
FAILURE: Integrated in Jenkins build Phoenix-master #1363 (See
[https://builds.apache.org/job/Phoenix-master/1363/])
PHOENIX-2995 Write performance severely degrades with large number of (tdsilva:
rev 3130fa9919ba937d98853f9be8068a0a8a8a96f5)
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/query/MetaDataMutated.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/TransactionUtil.java
* (edit)
phoenix-core/src/test/java/org/apache/phoenix/schema/PMetaDataImplTest.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixConnection.java
* (add)
phoenix-core/src/main/java/org/apache/phoenix/schema/PSynchronizedMetaData.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/UpsertSelectIT.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/query/DelegateConnectionQueryServices.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaData.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaDataImpl.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/compile/CreateTableCompiler.java
> Write performance severely degrades with large number of views
> ---
>
> Key: PHOENIX-2995
> URL: https://issues.apache.org/jira/browse/PHOENIX-2995
> Project: Phoenix
> Issue Type: Bug
>Reporter: Mujtaba Chohan
>Assignee: Thomas D'Silva
> Labels: Argus
> Fix For: 4.8.1
>
> Attachments: PHOENIX-2995-v2.patch, PHOENIX-2995-v3.patch,
> PHOENIX-2995-v4.patch, PHOENIX-2995-v5.patch, PHOENIX-2995-v6.patch,
> PHOENIX-2995.patch, create_view_and_upsert.png, image.png, image2.png,
> image3.png, upsert_rate.png
>
>
> Write performance for each 1K batch degrades significantly when there are
> *10K* views being written in random with default
> {{phoenix.client.maxMetaDataCacheSize}}. With all views created, upsert rate
> remains around 25 seconds per 1K batch i.e. ~2K rows/min upsert rate.
> When {{phoenix.client.maxMetaDataCacheSize}} is increased to 100MB+ then view
> does not need to get re-resolved and upsert rate gets back to normal ~60K
> rows/min.
> With *100K* views and {{phoenix.client.maxMetaDataCacheSize}} set to 1GB, I
> wasn't able create all 100K views as upsert time for each 1K batch keeps on
> steadily increasing.
> Following graph shows 1K batch upsert rate over time with variation of number
> of views. Rows are upserted to random views {{CREATE VIEW IF NOT EXISTS ...
> APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY=90}} is executed before
> upsert statement.
> !upsert_rate.png!
> Base table is also created with {{APPEND_ONLY_SCHEMA = true,
> UPDATE_CACHE_FREQUENCY = 90, AUTO_PARTITION_SEQ}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3148) Reduce size of PTable so that more tables can be cached in the metada cache.
[ https://issues.apache.org/jira/browse/PHOENIX-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas D'Silva updated PHOENIX-3148: Fix Version/s: (was: 4.9.0) 4.8.1 > Reduce size of PTable so that more tables can be cached in the metada cache. > > > Key: PHOENIX-3148 > URL: https://issues.apache.org/jira/browse/PHOENIX-3148 > Project: Phoenix > Issue Type: Bug >Reporter: Thomas D'Silva >Assignee: Thomas D'Silva > Labels: argus > Fix For: 4.8.1 > > > According to PHOENIX-2995, the current size is 7KB per PTable which works out > to enabling 140K PTables per 1GB. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-2995) Write performance severely degrades with large number of views
[
https://issues.apache.org/jira/browse/PHOENIX-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas D'Silva resolved PHOENIX-2995.
-
Resolution: Fixed
> Write performance severely degrades with large number of views
> ---
>
> Key: PHOENIX-2995
> URL: https://issues.apache.org/jira/browse/PHOENIX-2995
> Project: Phoenix
> Issue Type: Bug
>Reporter: Mujtaba Chohan
>Assignee: Thomas D'Silva
> Labels: Argus
> Fix For: 4.8.1
>
> Attachments: PHOENIX-2995-v2.patch, PHOENIX-2995-v3.patch,
> PHOENIX-2995-v4.patch, PHOENIX-2995-v5.patch, PHOENIX-2995-v6.patch,
> PHOENIX-2995.patch, create_view_and_upsert.png, image.png, image2.png,
> image3.png, upsert_rate.png
>
>
> Write performance for each 1K batch degrades significantly when there are
> *10K* views being written in random with default
> {{phoenix.client.maxMetaDataCacheSize}}. With all views created, upsert rate
> remains around 25 seconds per 1K batch i.e. ~2K rows/min upsert rate.
> When {{phoenix.client.maxMetaDataCacheSize}} is increased to 100MB+ then view
> does not need to get re-resolved and upsert rate gets back to normal ~60K
> rows/min.
> With *100K* views and {{phoenix.client.maxMetaDataCacheSize}} set to 1GB, I
> wasn't able create all 100K views as upsert time for each 1K batch keeps on
> steadily increasing.
> Following graph shows 1K batch upsert rate over time with variation of number
> of views. Rows are upserted to random views {{CREATE VIEW IF NOT EXISTS ...
> APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY=90}} is executed before
> upsert statement.
> !upsert_rate.png!
> Base table is also created with {{APPEND_ONLY_SCHEMA = true,
> UPDATE_CACHE_FREQUENCY = 90, AUTO_PARTITION_SEQ}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (PHOENIX-3148) Reduce size of PTable so that more tables can be cached in the metada cache.
[ https://issues.apache.org/jira/browse/PHOENIX-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas D'Silva reassigned PHOENIX-3148: --- Assignee: Thomas D'Silva > Reduce size of PTable so that more tables can be cached in the metada cache. > > > Key: PHOENIX-3148 > URL: https://issues.apache.org/jira/browse/PHOENIX-3148 > Project: Phoenix > Issue Type: Bug >Reporter: Thomas D'Silva >Assignee: Thomas D'Silva > Labels: argus > Fix For: 4.8.1 > > > According to PHOENIX-2995, the current size is 7KB per PTable which works out > to enabling 140K PTables per 1GB. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3190) Creating Tracing Visualization Ducumentation
[ https://issues.apache.org/jira/browse/PHOENIX-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated PHOENIX-3190: Fix Version/s: (was: 4.8.0) > Creating Tracing Visualization Ducumentation > > > Key: PHOENIX-3190 > URL: https://issues.apache.org/jira/browse/PHOENIX-3190 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.8.0 >Reporter: Nishani >Assignee: Nishani > > Documentation will contain > -Improved Search Features > -How to start the Phoenix Zipkin Web App > -How to change the zookeeper host and port > -Features of Phoenix Zipkin Web App -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (PHOENIX-3189) HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url
[
https://issues.apache.org/jira/browse/PHOENIX-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated PHOENIX-3189:
Affects Version/s: 4.8.0
> HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url
>
>
> Key: PHOENIX-3189
> URL: https://issues.apache.org/jira/browse/PHOENIX-3189
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Blocker
> Fix For: 4.9.0, 4.8.1
>
>
> We've been doing some more testing after PHOENIX-3126 and, with the help of
> [~arpitgupta] and [~harsha_ch], we've found an issue in a test between Storm
> and Phoenix.
> Storm was configured to create a JDBC Bolt, specifying the principal and
> keytab in the JDBC URL, relying on PhoenixDriver to do the Kerberos login for
> them. After PHOENIX-3126, a ZK server blacklisted the host running the bolt,
> and we observed that there were over 140 active ZK threads in the JVM.
> This results in a subtle change where every time the client tries to get a
> new Connection, we end up getting a new UGI instance (because the
> {{ConnectionQueryServicesImpl#openConnection()}} always does a new login).
> If users are correctly caching Connections, there isn't an issue (best as I
> can presently tell). However, if users rely on the getting the same
> connection every time (the pre-PHOENIX-3126), they will saturate their local
> JVM with connections and crash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3189) HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url
[
https://issues.apache.org/jira/browse/PHOENIX-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated PHOENIX-3189:
Fix Version/s: 4.9.0
> HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url
>
>
> Key: PHOENIX-3189
> URL: https://issues.apache.org/jira/browse/PHOENIX-3189
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Blocker
> Fix For: 4.9.0, 4.8.1
>
>
> We've been doing some more testing after PHOENIX-3126 and, with the help of
> [~arpitgupta] and [~harsha_ch], we've found an issue in a test between Storm
> and Phoenix.
> Storm was configured to create a JDBC Bolt, specifying the principal and
> keytab in the JDBC URL, relying on PhoenixDriver to do the Kerberos login for
> them. After PHOENIX-3126, a ZK server blacklisted the host running the bolt,
> and we observed that there were over 140 active ZK threads in the JVM.
> This results in a subtle change where every time the client tries to get a
> new Connection, we end up getting a new UGI instance (because the
> {{ConnectionQueryServicesImpl#openConnection()}} always does a new login).
> If users are correctly caching Connections, there isn't an issue (best as I
> can presently tell). However, if users rely on the getting the same
> connection every time (the pre-PHOENIX-3126), they will saturate their local
> JVM with connections and crash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (PHOENIX-3189) HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url
[
https://issues.apache.org/jira/browse/PHOENIX-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425017#comment-15425017
]
ASF GitHub Bot commented on PHOENIX-3189:
-
GitHub user joshelser opened a pull request:
https://github.com/apache/phoenix/pull/191
PHOENIX-3189 Perform Kerberos login before ConnectionInfo is constructed
Now that ConnectionInfo has the current User/UGI stored inside, we must
make sure that any automatic Kerberos login occurs before the ConnectionInfo
object is constructed. Otherwise, we will have multiple instances of
ConnectionInfo that differ only by the User, which will leak HBase/ZK
connections in the connectionQueryServicesMap.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/joshelser/phoenix 3189-secure-cnxninfo
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/phoenix/pull/191.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #191
commit ac4184b9ec52ab261fc4d9f73edc6676eb421d79
Author: Josh Elser
Date: 2016-08-17T17:34:59Z
PHOENIX-3189 Perform Kerberos login before ConnectionInfo is constructed
Now that ConnectionInfo has the current User/UGI stored inside, we must
make sure that any automatic Kerberos login occurs before the ConnectionInfo
object is constructed. Otherwise, we will have multiple instances of
ConnectionInfo that differ only by the User, which will leak HBase/ZK
connections in the connectionQueryServicesMap.
> HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url
>
>
> Key: PHOENIX-3189
> URL: https://issues.apache.org/jira/browse/PHOENIX-3189
> Project: Phoenix
> Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Blocker
> Fix For: 4.8.1
>
>
> We've been doing some more testing after PHOENIX-3126 and, with the help of
> [~arpitgupta] and [~harsha_ch], we've found an issue in a test between Storm
> and Phoenix.
> Storm was configured to create a JDBC Bolt, specifying the principal and
> keytab in the JDBC URL, relying on PhoenixDriver to do the Kerberos login for
> them. After PHOENIX-3126, a ZK server blacklisted the host running the bolt,
> and we observed that there were over 140 active ZK threads in the JVM.
> This results in a subtle change where every time the client tries to get a
> new Connection, we end up getting a new UGI instance (because the
> {{ConnectionQueryServicesImpl#openConnection()}} always does a new login).
> If users are correctly caching Connections, there isn't an issue (best as I
> can presently tell). However, if users rely on the getting the same
> connection every time (the pre-PHOENIX-3126), they will saturate their local
> JVM with connections and crash.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] phoenix pull request #191: PHOENIX-3189 Perform Kerberos login before Connec...
GitHub user joshelser opened a pull request: https://github.com/apache/phoenix/pull/191 PHOENIX-3189 Perform Kerberos login before ConnectionInfo is constructed Now that ConnectionInfo has the current User/UGI stored inside, we must make sure that any automatic Kerberos login occurs before the ConnectionInfo object is constructed. Otherwise, we will have multiple instances of ConnectionInfo that differ only by the User, which will leak HBase/ZK connections in the connectionQueryServicesMap. You can merge this pull request into a Git repository by running: $ git pull https://github.com/joshelser/phoenix 3189-secure-cnxninfo Alternatively you can review and apply these changes as the patch at: https://github.com/apache/phoenix/pull/191.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #191 commit ac4184b9ec52ab261fc4d9f73edc6676eb421d79 Author: Josh Elser Date: 2016-08-17T17:34:59Z PHOENIX-3189 Perform Kerberos login before ConnectionInfo is constructed Now that ConnectionInfo has the current User/UGI stored inside, we must make sure that any automatic Kerberos login occurs before the ConnectionInfo object is constructed. Otherwise, we will have multiple instances of ConnectionInfo that differ only by the User, which will leak HBase/ZK connections in the connectionQueryServicesMap. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (PHOENIX-3192) phoenix-spark dataframe issue with combination of "column family + column name"
[
https://issues.apache.org/jira/browse/PHOENIX-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424957#comment-15424957
]
Josh Mahonin commented on PHOENIX-3192:
---
For the most part, the phoenix-spark code tries not to do to much w.r.t. column
names, as most of it is offloaded to the PhoenixConfigurationUtil class:
https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/PhoenixRDD.scala#L81
All I can suggest is to try keep any fixes in the utility classes, if possible,
so that Pig and the other MR integrations can make use of them as well. Unless
of course the issue is within phoenix-spark itself, then change whatever you
think is necessary.
> phoenix-spark dataframe issue with combination of "column family + column
> name"
> ---
>
> Key: PHOENIX-3192
> URL: https://issues.apache.org/jira/browse/PHOENIX-3192
> Project: Phoenix
> Issue Type: Bug
>Reporter: Kalyan
>Assignee: Kalyan
>
> 1. create table with different column families with same column name
> create table tbl_1 (rowkey varchar primary key, cf1.c1 varchar, cf1.c2
> integer, cf2.c1 double, cf2.c2 boolean, cf3.c1 bigint);
> 2. insert sample data into table
> 3. create dataframe using phoenix table with different column names
> val df1 = sqlContext.phoenixTableAsDataFrame("tbl_1", Array("CF1.C1",
> "CF2.C2"), conf = configuration )
> df1.show // this will work
> 4. create dataframe using phoenix table with same column names
> val df2 = sqlContext.phoenixTableAsDataFrame("tbl_1", Array("CF1.C1",
> "CF2.C1"), conf = configuration )
> df2.show// this will fail
> 5. reason currently we are not handled the dataframe solution fully (column
> family + column name).
> only works with (column name)
> Exception:
> scala> val df2 = sqlContext.phoenixTableAsDataFrame("tbl_1", Array("CF1.C1",
> "CF2.C1"), conf = configuration )
> df2: org.apache.spark.sql.DataFrame = [C1: string, C1: double]
> scala> df2.show
> 16/08/17 22:16:54 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
> scala.MatchError: 1.5 (of class java.lang.Double)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$StringConverter$.toCatalystImpl(CatalystTypeConverters.scala:295)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$StringConverter$.toCatalystImpl(CatalystTypeConverters.scala:294)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:102)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:260)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:250)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:102)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$$anonfun$createToCatalystConverter$2.apply(CatalystTypeConverters.scala:401)
> at
> org.apache.spark.sql.SQLContext$$anonfun$6.apply(SQLContext.scala:492)
> at
> org.apache.spark.sql.SQLContext$$anonfun$6.apply(SQLContext.scala:492)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
> at scala.collection.Iterator$$anon$10.next(Iterator.scala:312)
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> at
> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
> at
> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
> at
> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
> at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
> at scala.collection.AbstractIterator.to(Iterator.scala:1157)
> at
> scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
> at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
> at
> scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
> at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$5.apply(SparkPlan.scala:212)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$5.apply(SparkPlan.scala:212)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at or
[jira] [Commented] (PHOENIX-3192) phoenix-spark dataframe issue with combination of "column family + column name"
[
https://issues.apache.org/jira/browse/PHOENIX-3192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424914#comment-15424914
]
Kalyan commented on PHOENIX-3192:
-
I will be working on this bug .. planning to change the DataFrame creation with
schema mapping provided by user.
Otherwise we are not able to work with sql queries using sqlContext.sql()
operation.
Please share any suggestions to work with DataFrame fully (column family +
column name)
> phoenix-spark dataframe issue with combination of "column family + column
> name"
> ---
>
> Key: PHOENIX-3192
> URL: https://issues.apache.org/jira/browse/PHOENIX-3192
> Project: Phoenix
> Issue Type: Bug
>Reporter: Kalyan
>Assignee: Kalyan
>
> 1. create table with different column families with same column name
> create table tbl_1 (rowkey varchar primary key, cf1.c1 varchar, cf1.c2
> integer, cf2.c1 double, cf2.c2 boolean, cf3.c1 bigint);
> 2. insert sample data into table
> 3. create dataframe using phoenix table with different column names
> val df1 = sqlContext.phoenixTableAsDataFrame("tbl_1", Array("CF1.C1",
> "CF2.C2"), conf = configuration )
> df1.show // this will work
> 4. create dataframe using phoenix table with same column names
> val df2 = sqlContext.phoenixTableAsDataFrame("tbl_1", Array("CF1.C1",
> "CF2.C1"), conf = configuration )
> df2.show// this will fail
> 5. reason currently we are not handled the dataframe solution fully (column
> family + column name).
> only works with (column name)
> Exception:
> scala> val df2 = sqlContext.phoenixTableAsDataFrame("tbl_1", Array("CF1.C1",
> "CF2.C1"), conf = configuration )
> df2: org.apache.spark.sql.DataFrame = [C1: string, C1: double]
> scala> df2.show
> 16/08/17 22:16:54 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
> scala.MatchError: 1.5 (of class java.lang.Double)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$StringConverter$.toCatalystImpl(CatalystTypeConverters.scala:295)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$StringConverter$.toCatalystImpl(CatalystTypeConverters.scala:294)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:102)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:260)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:250)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:102)
> at
> org.apache.spark.sql.catalyst.CatalystTypeConverters$$anonfun$createToCatalystConverter$2.apply(CatalystTypeConverters.scala:401)
> at
> org.apache.spark.sql.SQLContext$$anonfun$6.apply(SQLContext.scala:492)
> at
> org.apache.spark.sql.SQLContext$$anonfun$6.apply(SQLContext.scala:492)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
> at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
> at scala.collection.Iterator$$anon$10.next(Iterator.scala:312)
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> at
> scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
> at
> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
> at
> scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
> at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
> at scala.collection.AbstractIterator.to(Iterator.scala:1157)
> at
> scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
> at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
> at
> scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
> at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$5.apply(SparkPlan.scala:212)
> at
> org.apache.spark.sql.execution.SparkPlan$$anonfun$5.apply(SparkPlan.scala:212)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
> at
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
> at org.apache.spark.scheduler.Task.run(Task.scala:89)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurre
[jira] [Commented] (PHOENIX-2938) HFile support for SparkSQL DataFrame saves
[ https://issues.apache.org/jira/browse/PHOENIX-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424898#comment-15424898 ] Kalyan commented on PHOENIX-2938: - Thanks Josh Mahonin for your suggestions. i will work on these changes .. after the below bug fix. PHOENIX-3192 : phoenix-spark dataframe issue with combination of "column family + column name" otherwise i need to refactor again. > HFile support for SparkSQL DataFrame saves > -- > > Key: PHOENIX-2938 > URL: https://issues.apache.org/jira/browse/PHOENIX-2938 > Project: Phoenix > Issue Type: Improvement >Reporter: Chris Tarnas >Assignee: Kalyan >Priority: Minor > > Currently when saving a DataFrame in Spark it is persisted as upserts. Having > an option to do saves natively via HFiles, as the MapReduce loader does, > would be a great performance improvement for large bulk loads. The current > work around to reduce the load on the regionservers would be to save to csv > from Spark then load via the MapReduce loader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (PHOENIX-3191) COUNT and other aggregate functions not working
[
https://issues.apache.org/jira/browse/PHOENIX-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor resolved PHOENIX-3191.
---
Resolution: Invalid
This is clearly an environmental issue. Would you mind asking on the dev or
user list and include the Phoenix version you're using? I recommend you check
your setting for the {{phoenix.spool.directory}} config property on the
server-side which in later versions defaults to
{{System.getProperty("java.io.tmpdir")}} through in earlier versions defaulted
to /tmp (which won't be valid in some environments). Try setting it to
something you know is valid.
> COUNT and other aggregate functions not working
> ---
>
> Key: PHOENIX-3191
> URL: https://issues.apache.org/jira/browse/PHOENIX-3191
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.6.0
> Environment: CentOS(Server), Windows(Client)
>Reporter: Vivek K T
>Priority: Critical
>
> Following are steps to reproduce the issue :
> 1) Create a dummy table BUDGET :
> create table BUDGET
> (
> id varchar (50) primary key,
> date timestamp,
> amount decimal,
> currency varchar (50)
> )
> 2) Insert data in BUDGET table :
> upsert into BUDGET (id,date,amount,currency) values
> ('1',current_date(),20,'INR');
> upsert into BUDGET (id,date,amount,currency) values
> ('2',current_date(),40,'INR');
> upsert into BUDGET (id,date,amount,currency) values
> ('3',current_date(),60,'INR');
> upsert into BUDGET (id,date,amount,currency) values
> ('4',current_date(),80,'INR');
> upsert into BUDGET (id,date,amount,currency) values
> ('5',current_date(),100,'INR');
> (PLEASE NOTE : TABLE CREATION AND DATA INSERTION SHOULD BE DONE FROM LINUX
> SERVER)
> 3) Fire following query from squirrel-sql on windows:
> select count(*) from BUDGET
> Note : Same error for other aggregate functions like sum,min,max etc.
> -> ERROR TRACE AS FOLLOW :
> [2016-08-17 21:32:45.925 IST] ERROR [] [] [] [] [] [] [] [] [] Thread-12
> net.sourceforge.squirrel_sql.client.session.SQLExecuterTask [126] - Error
> reading ResultSet for SQL: select count(*) from BUDGET
> org.apache.phoenix.exception.PhoenixIOException:
> org.apache.phoenix.exception.PhoenixIOException: The system cannot find the
> path specified
> at
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
> at
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:558)
> at
> org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
> at
> org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
> at
> org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
> at
> org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingResultIterator.java:64)
> at
> org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:771)
> at
> net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetWrapper._nextOnResultSet(ResultSetWrapper.java:95)
> at
> net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetWrapper.next(ResultSetWrapper.java:56)
> at
> net.sourceforge.squirrel_sql.fw.sql.ResultSetReader.readRow(ResultSetReader.java:182)
> at
> net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetDataSet.createRow(ResultSetDataSet.java:238)
> at
> net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetDataSet._setResultSet(ResultSetDataSet.java:204)
> at
> net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetDataSet.setSqlExecutionTabResultSet(ResultSetDataSet.java:127)
> at
> net.sourceforge.squirrel_sql.client.session.mainpanel.SQLExecutionHandler.sqlResultSetAvailable(SQLExecutionHandler.java:423)
> at
> net.sourceforge.squirrel_sql.client.session.SQLExecuterTask.processResultSet(SQLExecuterTask.java:549)
> at
> net.sourceforge.squirrel_sql.client.session.SQLExecuterTask.processQuery(SQLExecuterTask.java:414)
> at
> net.sourceforge.squirrel_sql.client.session.SQLExecuterTask.run(SQLExecuterTask.java:212)
> at
> net.sourceforge.squirrel_sql.fw.util.TaskExecuter.run(TaskExecuter.java:82)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.phoenix.exception.PhoenixIOException: The system cannot find the
> path specified
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:202)
> at
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:554)
> ... 18 more
[jira] [Created] (PHOENIX-3192) phoenix-spark dataframe issue with combination of "column family + column name"
Kalyan created PHOENIX-3192:
---
Summary: phoenix-spark dataframe issue with combination of "column
family + column name"
Key: PHOENIX-3192
URL: https://issues.apache.org/jira/browse/PHOENIX-3192
Project: Phoenix
Issue Type: Bug
Reporter: Kalyan
Assignee: Kalyan
1. create table with different column families with same column name
create table tbl_1 (rowkey varchar primary key, cf1.c1 varchar, cf1.c2 integer,
cf2.c1 double, cf2.c2 boolean, cf3.c1 bigint);
2. insert sample data into table
3. create dataframe using phoenix table with different column names
val df1 = sqlContext.phoenixTableAsDataFrame("tbl_1", Array("CF1.C1",
"CF2.C2"), conf = configuration )
df1.show // this will work
4. create dataframe using phoenix table with same column names
val df2 = sqlContext.phoenixTableAsDataFrame("tbl_1", Array("CF1.C1",
"CF2.C1"), conf = configuration )
df2.show// this will fail
5. reason currently we are not handled the dataframe solution fully (column
family + column name).
only works with (column name)
Exception:
scala> val df2 = sqlContext.phoenixTableAsDataFrame("tbl_1", Array("CF1.C1",
"CF2.C1"), conf = configuration )
df2: org.apache.spark.sql.DataFrame = [C1: string, C1: double]
scala> df2.show
16/08/17 22:16:54 ERROR Executor: Exception in task 0.0 in stage 0.0 (TID 0)
scala.MatchError: 1.5 (of class java.lang.Double)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$StringConverter$.toCatalystImpl(CatalystTypeConverters.scala:295)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$StringConverter$.toCatalystImpl(CatalystTypeConverters.scala:294)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:102)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:260)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$StructConverter.toCatalystImpl(CatalystTypeConverters.scala:250)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTypeConverters.scala:102)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$$anonfun$createToCatalystConverter$2.apply(CatalystTypeConverters.scala:401)
at
org.apache.spark.sql.SQLContext$$anonfun$6.apply(SQLContext.scala:492)
at
org.apache.spark.sql.SQLContext$$anonfun$6.apply(SQLContext.scala:492)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$$anon$10.next(Iterator.scala:312)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
at scala.collection.AbstractIterator.to(Iterator.scala:1157)
at
scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
at
scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$5.apply(SparkPlan.scala:212)
at
org.apache.spark.sql.execution.SparkPlan$$anonfun$5.apply(SparkPlan.scala:212)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at
org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1858)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
16/08/17 22:16:54 WARN TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0,
localhost): scala.MatchError: 1.5 (of class java.lang.Double)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$StringConverter$.toCatalystImpl(CatalystTypeConverters.scala:295)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$StringConverter$.toCatalystImpl(CatalystTypeConverters.scala:294)
at
org.apache.spark.sql.catalyst.CatalystTypeConverters$CatalystTypeConverter.toCatalyst(CatalystTy
[jira] [Commented] (PHOENIX-3185) Error: ERROR 514 (42892): A duplicate column name was detected in the object definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 (state=42892,code=514)
[
https://issues.apache.org/jira/browse/PHOENIX-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424873#comment-15424873
]
James Taylor commented on PHOENIX-3185:
---
Agree, [~kalyanhadoop], it's an important fix. Not worried about code format
yet. I'd just like to get a test run across all tests to make sure your change
doesn't introduce any regressions. That'll happen if you attach a patch like
this as described in the link I mentioned: {{git format-patch --stdout origin >
PHOENIX-{NUMBER}.patch}}
> Error: ERROR 514 (42892): A duplicate column name was detected in the object
> definition or ALTER TABLE statement. columnName=TEST_TABLE.C1
> (state=42892,code=514)
> -
>
> Key: PHOENIX-3185
> URL: https://issues.apache.org/jira/browse/PHOENIX-3185
> Project: Phoenix
> Issue Type: Bug
>Reporter: Kalyan
>Assignee: Kalyan
> Fix For: 4.8.1
>
> Attachments: image1.png, image2.png, image3.png,
> phoenix_duplicate_column_check.patch
>
>
> create a table with duplicate columns.
> create table test_table (rowkey varchar primary key, c1 varchar, c2 integer,
> c3 double, c1 bigint);
> The below exception getting .. no issue with exception. But the problem is
> phoenix is creating table with exception & later we are not able to delete
> the table also. We need to fix this Bug
> 0: jdbc:phoenix:localhost> create table test_table (rowkey varchar primary
> key, c1 varchar, c2 integer, c3 double, c1 bigint);
> Error: ERROR 514 (42892): A duplicate column name was detected in the object
> definition or ALTER TABLE statement. columnName=TEST_TABLE.C1
> (state=42892,code=514)
> org.apache.phoenix.schema.ColumnAlreadyExistsException: ERROR 514 (42892): A
> duplicate column name was detected in the object definition or ALTER TABLE
> statement. columnName=TEST_TABLE.C1
> at org.apache.phoenix.schema.PTableImpl.init(PTableImpl.java:415)
> at org.apache.phoenix.schema.PTableImpl.(PTableImpl.java:315)
> at org.apache.phoenix.schema.PTableImpl.makePTable(PTableImpl.java:288)
> at
> org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2146)
> at
> org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:828)
> at
> org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:338)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:326)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:324)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1345)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
> at sqlline.SqlLine.main(SqlLine.java:292)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (PHOENIX-3185) Error: ERROR 514 (42892): A duplicate column name was detected in the object definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 (state=42892,code=514)
[
https://issues.apache.org/jira/browse/PHOENIX-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424873#comment-15424873
]
James Taylor edited comment on PHOENIX-3185 at 8/17/16 4:44 PM:
Agree, [~kalyanhadoop], it's an important fix. Not worried about code format
yet. I'd just like to get a test run across all tests to make sure your change
doesn't introduce any regressions. That'll happen if you attach a patch like
this as described in the link I mentioned:
{code}
git format-patch --stdout origin > PHOENIX-{NUMBER}.patch
{code}
was (Author: jamestaylor):
Agree, [~kalyanhadoop], it's an important fix. Not worried about code format
yet. I'd just like to get a test run across all tests to make sure your change
doesn't introduce any regressions. That'll happen if you attach a patch like
this as described in the link I mentioned: {{git format-patch --stdout origin >
PHOENIX-{NUMBER}.patch}}
> Error: ERROR 514 (42892): A duplicate column name was detected in the object
> definition or ALTER TABLE statement. columnName=TEST_TABLE.C1
> (state=42892,code=514)
> -
>
> Key: PHOENIX-3185
> URL: https://issues.apache.org/jira/browse/PHOENIX-3185
> Project: Phoenix
> Issue Type: Bug
>Reporter: Kalyan
>Assignee: Kalyan
> Fix For: 4.8.1
>
> Attachments: image1.png, image2.png, image3.png,
> phoenix_duplicate_column_check.patch
>
>
> create a table with duplicate columns.
> create table test_table (rowkey varchar primary key, c1 varchar, c2 integer,
> c3 double, c1 bigint);
> The below exception getting .. no issue with exception. But the problem is
> phoenix is creating table with exception & later we are not able to delete
> the table also. We need to fix this Bug
> 0: jdbc:phoenix:localhost> create table test_table (rowkey varchar primary
> key, c1 varchar, c2 integer, c3 double, c1 bigint);
> Error: ERROR 514 (42892): A duplicate column name was detected in the object
> definition or ALTER TABLE statement. columnName=TEST_TABLE.C1
> (state=42892,code=514)
> org.apache.phoenix.schema.ColumnAlreadyExistsException: ERROR 514 (42892): A
> duplicate column name was detected in the object definition or ALTER TABLE
> statement. columnName=TEST_TABLE.C1
> at org.apache.phoenix.schema.PTableImpl.init(PTableImpl.java:415)
> at org.apache.phoenix.schema.PTableImpl.(PTableImpl.java:315)
> at org.apache.phoenix.schema.PTableImpl.makePTable(PTableImpl.java:288)
> at
> org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2146)
> at
> org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:828)
> at
> org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:338)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:326)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:324)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1345)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:808)
> at sqlline.SqlLine.begin(SqlLine.java:681)
> at sqlline.SqlLine.start(SqlLine.java:398)
> at sqlline.SqlLine.main(SqlLine.java:292)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (PHOENIX-3185) Error: ERROR 514 (42892): A duplicate column name was detected in the object definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 (state=42892,code=514)
[ https://issues.apache.org/jira/browse/PHOENIX-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-3185: -- Fix Version/s: 4.8.1 > Error: ERROR 514 (42892): A duplicate column name was detected in the object > definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 > (state=42892,code=514) > - > > Key: PHOENIX-3185 > URL: https://issues.apache.org/jira/browse/PHOENIX-3185 > Project: Phoenix > Issue Type: Bug >Reporter: Kalyan >Assignee: Kalyan > Fix For: 4.8.1 > > Attachments: image1.png, image2.png, image3.png, > phoenix_duplicate_column_check.patch > > > create a table with duplicate columns. > create table test_table (rowkey varchar primary key, c1 varchar, c2 integer, > c3 double, c1 bigint); > The below exception getting .. no issue with exception. But the problem is > phoenix is creating table with exception & later we are not able to delete > the table also. We need to fix this Bug > 0: jdbc:phoenix:localhost> create table test_table (rowkey varchar primary > key, c1 varchar, c2 integer, c3 double, c1 bigint); > Error: ERROR 514 (42892): A duplicate column name was detected in the object > definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 > (state=42892,code=514) > org.apache.phoenix.schema.ColumnAlreadyExistsException: ERROR 514 (42892): A > duplicate column name was detected in the object definition or ALTER TABLE > statement. columnName=TEST_TABLE.C1 > at org.apache.phoenix.schema.PTableImpl.init(PTableImpl.java:415) > at org.apache.phoenix.schema.PTableImpl.(PTableImpl.java:315) > at org.apache.phoenix.schema.PTableImpl.makePTable(PTableImpl.java:288) > at > org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2146) > at > org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:828) > at > org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:338) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:326) > at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:324) > at > org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1345) > at sqlline.Commands.execute(Commands.java:822) > at sqlline.Commands.sql(Commands.java:732) > at sqlline.SqlLine.dispatch(SqlLine.java:808) > at sqlline.SqlLine.begin(SqlLine.java:681) > at sqlline.SqlLine.start(SqlLine.java:398) > at sqlline.SqlLine.main(SqlLine.java:292) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (PHOENIX-3191) COUNT and other aggregate functions not working
Vivek K T created PHOENIX-3191:
--
Summary: COUNT and other aggregate functions not working
Key: PHOENIX-3191
URL: https://issues.apache.org/jira/browse/PHOENIX-3191
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.6.0
Environment: CentOS(Server), Windows(Client)
Reporter: Vivek K T
Priority: Critical
Following are steps to reproduce the issue :
1) Create a dummy table BUDGET :
create table BUDGET
(
id varchar (50) primary key,
date timestamp,
amount decimal,
currency varchar (50)
)
2) Insert data in BUDGET table :
upsert into BUDGET (id,date,amount,currency) values
('1',current_date(),20,'INR');
upsert into BUDGET (id,date,amount,currency) values
('2',current_date(),40,'INR');
upsert into BUDGET (id,date,amount,currency) values
('3',current_date(),60,'INR');
upsert into BUDGET (id,date,amount,currency) values
('4',current_date(),80,'INR');
upsert into BUDGET (id,date,amount,currency) values
('5',current_date(),100,'INR');
(PLEASE NOTE : TABLE CREATION AND DATA INSERTION SHOULD BE DONE FROM LINUX
SERVER)
3) Fire following query from squirrel-sql on windows:
select count(*) from BUDGET
Note : Same error for other aggregate functions like sum,min,max etc.
-> ERROR TRACE AS FOLLOW :
[2016-08-17 21:32:45.925 IST] ERROR [] [] [] [] [] [] [] [] [] Thread-12
net.sourceforge.squirrel_sql.client.session.SQLExecuterTask [126] - Error
reading ResultSet for SQL: select count(*) from BUDGET
org.apache.phoenix.exception.PhoenixIOException:
org.apache.phoenix.exception.PhoenixIOException: The system cannot find the
path specified
at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
at
org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:558)
at
org.apache.phoenix.iterate.ConcatResultIterator.getIterators(ConcatResultIterator.java:50)
at
org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:97)
at
org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
at
org.apache.phoenix.iterate.BaseGroupedAggregatingResultIterator.next(BaseGroupedAggregatingResultIterator.java:64)
at
org.apache.phoenix.iterate.UngroupedAggregatingResultIterator.next(UngroupedAggregatingResultIterator.java:39)
at
org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:771)
at
net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetWrapper._nextOnResultSet(ResultSetWrapper.java:95)
at
net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetWrapper.next(ResultSetWrapper.java:56)
at
net.sourceforge.squirrel_sql.fw.sql.ResultSetReader.readRow(ResultSetReader.java:182)
at
net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetDataSet.createRow(ResultSetDataSet.java:238)
at
net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetDataSet._setResultSet(ResultSetDataSet.java:204)
at
net.sourceforge.squirrel_sql.fw.datasetviewer.ResultSetDataSet.setSqlExecutionTabResultSet(ResultSetDataSet.java:127)
at
net.sourceforge.squirrel_sql.client.session.mainpanel.SQLExecutionHandler.sqlResultSetAvailable(SQLExecutionHandler.java:423)
at
net.sourceforge.squirrel_sql.client.session.SQLExecuterTask.processResultSet(SQLExecuterTask.java:549)
at
net.sourceforge.squirrel_sql.client.session.SQLExecuterTask.processQuery(SQLExecuterTask.java:414)
at
net.sourceforge.squirrel_sql.client.session.SQLExecuterTask.run(SQLExecuterTask.java:212)
at
net.sourceforge.squirrel_sql.fw.util.TaskExecuter.run(TaskExecuter.java:82)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.util.concurrent.ExecutionException:
org.apache.phoenix.exception.PhoenixIOException: The system cannot find the
path specified
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:202)
at
org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:554)
... 18 more
Caused by: org.apache.phoenix.exception.PhoenixIOException: The system cannot
find the path specified
at
org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:108)
at
org.apache.phoenix.iterate.SpoolingResultIterator.(SpoolingResultIterator.java:151)
at
org.apache.phoenix.iterate.SpoolingResultIterator.(SpoolingResultIterator.java:86)
at
org.apache.phoenix.iterate.SpoolingResultIterator.(SpoolingResultIterator.java:64)
at
org.apache.phoenix.iterate.SpoolingResultIterator$SpoolingResultIteratorFactory.newIterator(SpoolingResultIterator.java:81)
at
org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:109)
at
org.apache.phoenix.iterate.ParallelIterat
Re: [ANNOUNCE] Josh Elser joins Phoenix PMC
Congratulations! Well deserved recognition on your contribution to Phoenix! On Wed, Aug 10, 2016 at 11:51 PM, rajeshb...@apache.org < chrajeshbab...@gmail.com> wrote: > Congratulations Josh!!! > > On Thu, Aug 11, 2016 at 9:17 AM, Josh Elser wrote: > > > Thank you all! I look forward to working more closely with everyone :D > > > > > > James Taylor wrote: > > > >> On behalf of the Apache Phoenix PMC, I'm pleased to announce that Josh > >> Elser has accepted our invitation to become a member of the Apache > Phoenix > >> PMC. Recently he found and fixed licensing-related issues in our source > >> and > >> binary release, and prior to that he's been the force behind the Phoenix > >> Query Server (PQS). > >> > >> Really excited to have you onboard, Josh. Welcome, and looking forward > to > >> continued collaboration. > >> > >> Regards, > >> James > >> > >> >
[jira] [Commented] (PHOENIX-3185) Error: ERROR 514 (42892): A duplicate column name was detected in the object definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 (state=42892,code=514)
[ https://issues.apache.org/jira/browse/PHOENIX-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424782#comment-15424782 ] Kalyan commented on PHOENIX-3185: - Hi James Taylor, i think .. i need to follow the contributor guidelines for code format. my intention is to verify the patch only .. i feel it is a major bug. > Error: ERROR 514 (42892): A duplicate column name was detected in the object > definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 > (state=42892,code=514) > - > > Key: PHOENIX-3185 > URL: https://issues.apache.org/jira/browse/PHOENIX-3185 > Project: Phoenix > Issue Type: Bug >Reporter: Kalyan >Assignee: Kalyan > Attachments: image1.png, image2.png, image3.png, > phoenix_duplicate_column_check.patch > > > create a table with duplicate columns. > create table test_table (rowkey varchar primary key, c1 varchar, c2 integer, > c3 double, c1 bigint); > The below exception getting .. no issue with exception. But the problem is > phoenix is creating table with exception & later we are not able to delete > the table also. We need to fix this Bug > 0: jdbc:phoenix:localhost> create table test_table (rowkey varchar primary > key, c1 varchar, c2 integer, c3 double, c1 bigint); > Error: ERROR 514 (42892): A duplicate column name was detected in the object > definition or ALTER TABLE statement. columnName=TEST_TABLE.C1 > (state=42892,code=514) > org.apache.phoenix.schema.ColumnAlreadyExistsException: ERROR 514 (42892): A > duplicate column name was detected in the object definition or ALTER TABLE > statement. columnName=TEST_TABLE.C1 > at org.apache.phoenix.schema.PTableImpl.init(PTableImpl.java:415) > at org.apache.phoenix.schema.PTableImpl.(PTableImpl.java:315) > at org.apache.phoenix.schema.PTableImpl.makePTable(PTableImpl.java:288) > at > org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2146) > at > org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:828) > at > org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:183) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:338) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:326) > at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:324) > at > org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1345) > at sqlline.Commands.execute(Commands.java:822) > at sqlline.Commands.sql(Commands.java:732) > at sqlline.SqlLine.dispatch(SqlLine.java:808) > at sqlline.SqlLine.begin(SqlLine.java:681) > at sqlline.SqlLine.start(SqlLine.java:398) > at sqlline.SqlLine.main(SqlLine.java:292) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-2722) support mysql "limit,offset" clauses
[
https://issues.apache.org/jira/browse/PHOENIX-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424636#comment-15424636
]
Vishal Mahajan commented on PHOENIX-2722:
-
Can we also mention @since version in phoenix docs for this feature?
> support mysql "limit,offset" clauses
> -
>
> Key: PHOENIX-2722
> URL: https://issues.apache.org/jira/browse/PHOENIX-2722
> Project: Phoenix
> Issue Type: New Feature
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Minor
> Fix For: 4.8.0
>
> Attachments: PHOENIX-2722.patch, PHOENIX-2722_formatted.patch,
> PHOENIX-2722_v1_rebased.patch
>
>
> For serial query(query with “serial" hint or “limit" without "order by”), we
> can limit each scan(using page filter) to “limit+offset” instead of limit
> earlier.
> And then, for all queries, we can forward the relevant client iterators to
> the offset provided and then return the result.
> syntax
> {code}
> [ LIMIT { count } ]
> [ OFFSET start [ ROW | ROWS ] ]
> [ FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } ONLY ]
> {code}
> Some new keywords(OFFSET,FETCH,ROW, ROWS,ONLY) are getting introduced so
> users might need to see that they are not using them as column name or
> something.
> WDYT, [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] phoenix issue #186: cleanup sqlline*.py using python idioms.
Github user btbytes commented on the issue: https://github.com/apache/phoenix/pull/186 I'm looking into this. I tried splitting the triple quoted string on space but that isn't working. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (PHOENIX-3190) Creating Tracing Visualization Ducumentation
Nishani created PHOENIX-3190: - Summary: Creating Tracing Visualization Ducumentation Key: PHOENIX-3190 URL: https://issues.apache.org/jira/browse/PHOENIX-3190 Project: Phoenix Issue Type: Sub-task Affects Versions: 4.8.0 Reporter: Nishani Assignee: Nishani Fix For: 4.8.0 Documentation will contain -Improved Search Features -How to start the Phoenix Zipkin Web App -How to change the zookeeper host and port -Features of Phoenix Zipkin Web App -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (PHOENIX-3159) CachingHTableFactory may close HTable during eviction even if it is getting used for writing by another thread.
[
https://issues.apache.org/jira/browse/PHOENIX-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424082#comment-15424082
]
Hadoop QA commented on PHOENIX-3159:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12824096/PHOENIX-3159_v3.patch
against master branch at commit 657917bfb15ecd5bef4616458f2c8cd3ede0cf6b.
ATTACHMENT ID: 12824096
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
34 warning messages.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ synchronized (this) { // the whole operation of closing and
checking the count should be atomic
+ public CachingHTableFactory(HTableFactory tableFactory, Configuration conf,
RegionCoprocessorEnvironment env) {
+ public CachingHTableFactory(HTableFactory factory, int cacheSize,
RegionCoprocessorEnvironment env) {
+this.pool = ThreadPoolManager.getExecutor(new
ThreadPoolBuilder("CachedHtables", env.getConfiguration())
+ public HTableInterface getTable(ImmutableBytesPtr tablename, ExecutorService
pool) throws IOException {
+public HTableInterface getTable(ImmutableBytesPtr
tablename,ExecutorService pool) throws IOException {
+ public HTableInterface getTable(ImmutableBytesPtr tablename, ExecutorService
pool) throws IOException;
+ public static final String INDEX_WRITES_THREAD_MAX_PER_REGIONSERVER_KEY =
"phoenix.index.writes.threads.max";
+public static final String INDEX_WRITER_KEEP_ALIVE_TIME_CONF_KEY =
"index.writer.threads.keepalivetime";
+void setup(HTableFactory factory, ExecutorService pool, Abortable
abortable, Stoppable stop, int cacheSize, RegionCoprocessorEnvironment env) {
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/520//testReport/
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/520//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/520//console
This message is automatically generated.
> CachingHTableFactory may close HTable during eviction even if it is getting
> used for writing by another thread.
> ---
>
> Key: PHOENIX-3159
> URL: https://issues.apache.org/jira/browse/PHOENIX-3159
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3159.patch, PHOENIX-3159_v1.patch,
> PHOENIX-3159_v2.patch, PHOENIX-3159_v3.patch
>
>
> CachingHTableFactory may close HTable during eviction even if it is getting
> used for writing by another thread which results in writing thread to fail
> and index is disabled.
> LRU eviction closing HTable or underlying connection when cache is full and
> new HTable is requested.
> {code}
> 2016-08-04 13:45:21,109 DEBUG
> [nat-s11-4-ioss-phoenix-1-5.openstacklocal,16020,1470297472814-index-writer--pool11-t35]
> client.ConnectionManager$HConnectionImplementation: Closing HConnection
> (debugging purposes only)
> java.lang.Exception
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.internalClose(ConnectionManager.java:2423)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.close(ConnectionManager.java:2447)
> at
> org.apache.hadoop.hbase.client.CoprocessorHConnection.close(CoprocessorHConnection.java:41)
> at
> org.apache.hadoop.hbase.client.HTableWrapper.internalClose(HTableWrapper.java:91)
> at
> org.apache.hadoop.hbase.client.HTableWrapper.close(HTableWrapper.java:107)
> at
> org.apache.phoenix.hbase.index.table.CachingHTableFactory$HTableInterfaceLRUMap.removeLRU(CachingHTableFactory.java:61)
> at
> org.apache.commons.collections.map.LRUMap.addMapping(LRUMap.java:256)
> at
> org.apache.commons.collections.map.AbstractHashedMap.put(AbstractHashedMap.java:284)
> at
> org.apache.phoenix.hbase.index.table.CachingHTableFactory.getTable(CachingHTableFactory.java:100)
> at
> org.apache.phoenix.hbase.index.write.ParallelWriterIndexCommitter$1.call(ParallelWriterIndexCommitter.java:
[jira] [Commented] (PHOENIX-1751) Perform aggregations, sorting, etc, in the preScannerNext instead of postScannerOpen
[
https://issues.apache.org/jira/browse/PHOENIX-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15424019#comment-15424019
]
Hadoop QA commented on PHOENIX-1751:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12824086/PHOENIX-1751.patch
against master branch at commit 657917bfb15ecd5bef4616458f2c8cd3ede0cf6b.
ATTACHMENT ID: 12824086
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
34 warning messages.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SaltedViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantSpecificTablesDMLIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/519//testReport/
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/519//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/519//console
This message is automatically generated.
> Perform aggregations, sorting, etc, in the preScannerNext instead of
> postScannerOpen
>
>
> Key: PHOENIX-1751
> URL: https://issues.apache.org/jira/browse/PHOENIX-1751
> Project: Phoenix
> Issue Type: Bug
>Reporter: Lars Hofhansl
> Attachments: 1751-WIP-v2.txt, 1751-WIP-v2b.patch, 1751-WIP.txt,
> PHOENIX-1751-v2c.patch, PHOENIX-1751.patch
>
>
> HBase retains a lease for every scanner. Then lease expires the scan will no
> longer (be allowed to) work. The leases guard against the client going away,
> and allow cleaning up resources if that happens.
> At various points HBase "suspends" the lease while the region server are
> working on behalf of this scanner, so that the lease won't expire even though
> the server is working on it.
> HBase does that during the scanning process. Crucially it suspends the leaser
> after the scanner is opened, before next() is issued on it.
> The outcome of all this is that Phoenix executes aggregates, sorts, etc, with
> the lease in place, and hence if these take a bit the lease can expire even
> though the server was working on it.
> Phoenix should do this work in preScannerNext, being careful that the
> precalculation is only performed once.
> I'll attach a sample patch soon.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
