[jira] [Assigned] (PHOENIX-5220) Create table fails when using the same connection after schema upgrade
[ https://issues.apache.org/jira/browse/PHOENIX-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swaroopa Kadam reassigned PHOENIX-5220: --- Assignee: Swaroopa Kadam > Create table fails when using the same connection after schema upgrade > -- > > Key: PHOENIX-5220 > URL: https://issues.apache.org/jira/browse/PHOENIX-5220 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0, 4.14.1 >Reporter: Jacob Isaac >Assignee: Swaroopa Kadam >Priority: Major > Attachments: Screen Shot 2019-03-28 at 9.37.23 PM.png > > > Steps: > 1. Try to upgrade system.catalog from 4.10 to 4.13 > 2. Run Execute Upgrade > 3. Creating a table will fail with the following exception - > org.apache.phoenix.schema.ColumnNotFoundException: ERROR 504 (42703): > Undefined column. columnName=SYSTEM.CATALOG.USE_STATS_FOR_PARALLELIZATION > at > org.apache.phoenix.schema.PTableImpl.getColumnForColumnName(PTableImpl.java:828) > at > org.apache.phoenix.compile.FromCompiler$SingleTableColumnResolver.resolveColumn(FromCompiler.java:475) > at > org.apache.phoenix.compile.UpsertCompiler.compile(UpsertCompiler.java:450) > at > org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:755) > at > org.apache.phoenix.jdbc.PhoenixStatement$ExecutableUpsertStatement.compilePlan(PhoenixStatement.java:741) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:389) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:379) > at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:377) > at > org.apache.phoenix.jdbc.PhoenixStatement.access$700(PhoenixStatement.java:208) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:418) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:379) > at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:377) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:366) > at > org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:272) > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:172) > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:177) > at > org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:2665) > at > org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:1097) > at > org.apache.phoenix.compile.CreateTableCompiler$1.execute(CreateTableCompiler.java:192) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:396) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:379) > at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:377) > at > org.apache.phoenix.jdbc.PhoenixStatement.access$700(PhoenixStatement.java:208) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:418) > at > org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:379) > at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:377) > at > org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:366) > at > org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1775) > at sqlline.Commands.execute(Commands.java:822) > at sqlline.Commands.sql(Commands.java:732) > at sqlline.SqlLine.dispatch(SqlLine.java:807) > at sqlline.SqlLine.begin(SqlLine.java:681) > at sqlline.SqlLine.start(SqlLine.java:398) > at sqlline.SqlLine.main(SqlLine.java:292) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (PHOENIX-5067) Support for secure Phoenix cluster in Pherf
[
https://issues.apache.org/jira/browse/PHOENIX-5067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal reassigned PHOENIX-5067:
Assignee: Sandeep Pal (was: Biju Nair)
> Support for secure Phoenix cluster in Pherf
> ---
>
> Key: PHOENIX-5067
> URL: https://issues.apache.org/jira/browse/PHOENIX-5067
> Project: Phoenix
> Issue Type: Improvement
>Reporter: Biju Nair
>Assignee: Sandeep Pal
>Priority: Minor
> Attachments: PHOENIX-5067-4.x-HBase-1.1,
> PHOENIX-5067-4.x-HBase-1.1.patch
>
>
> Currently Phoenix performance and functional testing tool {{Pherf}} doesn't
> have options to pass in Kerberos principal and Keytab to connect to a secure
> (Kerberized) Phoenix cluster. This prevents running the tool against a
> Kerberized clusters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (PHOENIX-4743) ALTER TABLE ADD COLUMN for global index should not modify HBase metadata if failed
[
https://issues.apache.org/jira/browse/PHOENIX-4743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandeep Pal reassigned PHOENIX-4743:
Assignee: Sandeep Pal
> ALTER TABLE ADD COLUMN for global index should not modify HBase metadata if
> failed
> --
>
> Key: PHOENIX-4743
> URL: https://issues.apache.org/jira/browse/PHOENIX-4743
> Project: Phoenix
> Issue Type: Bug
>Reporter: Chinmay Kulkarni
>Assignee: Sandeep Pal
>Priority: Major
>
> When you issue an "ALTER TABLE" for a global index to add a column, Phoenix
> throws a SQLException, but the HBase metadata for the global index table is
> still modified.
> Steps to reproduce:
> * Create the base data table:
> {code:java}
> create table if not exists z_base_table (id INTEGER not null primary key,
> host VARCHAR(10), flag boolean);{code}
> * Create a global index on top of this table:
> {code:java}
> create index global_z_index on z_base_table(HOST);{code}
> * Add a column to the global index table:
> {code:java}
> alter table global_z_index add cf1.age INTEGER;{code}
> This will throw an exception in Phoenix, but HBase metadata for the global
> index table is still modified. Stack trace:
> {noformat}
> Error: ERROR 1010 (42M01): Not allowed to mutate table. Cannot add/drop
> column referenced by VIEW columnName=GLOBAL_Z_INDEX (state=42M01,code=1010)
> java.sql.SQLException: ERROR 1010 (42M01): Not allowed to mutate table.
> Cannot add/drop column referenced by VIEW columnName=GLOBAL_Z_INDEX
> at
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:494)
> at
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)
> at
> org.apache.phoenix.schema.MetaDataClient.processMutationResult(MetaDataClient.java:3049)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:3503)
> at
> org.apache.phoenix.schema.MetaDataClient.addColumn(MetaDataClient.java:3210)
> at
> org.apache.phoenix.jdbc.PhoenixStatement$ExecutableAddColumnStatement$1.execute(PhoenixStatement.java:1432)
> at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
> at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
> at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1825)
> at sqlline.Commands.execute(Commands.java:822)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:813)
> at sqlline.SqlLine.begin(SqlLine.java:686)
> at sqlline.SqlLine.start(SqlLine.java:398)
> at sqlline.SqlLine.main(SqlLine.java:291{noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (PHOENIX-5272) Support ALTER INDEX REBUILD ALL ASYNC to fully rebuild global indexes async
[ https://issues.apache.org/jira/browse/PHOENIX-5272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-5272: - Description: This Jira is part of PHOENIX-4703. On 4703, a flag to IndexTool is added so that we can delete all global indexes before rebuilding. This Jira is concentrated on ALTER INDEX REBUILD ALL sql syntax. (was: Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are deleted and the index is rebuilt synchronously. "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) So it seems currently the only way to fully rebuild is the drop the index, and recreate it. This is burdensome as it requires have the schema DDL. We should have an option to fully rebuild asynchronously, that has the same semantics as dropping and recreating the index. A further advantage of this is we can maintain the splits of the index table while dropping its data. We are currently seeing issues where rebuilding a large table via a MR job results in hotspotting due to all data regions writing to the same index region at the start. ) > Support ALTER INDEX REBUILD ALL ASYNC to fully rebuild global indexes async > --- > > Key: PHOENIX-5272 > URL: https://issues.apache.org/jira/browse/PHOENIX-5272 > Project: Phoenix > Issue Type: Bug >Reporter: Gokcen Iskender >Assignee: Gokcen Iskender >Priority: Major > Fix For: 4.15.0, 5.1.0 > > > This Jira is part of PHOENIX-4703. On 4703, a flag to IndexTool is added so > that we can delete all global indexes before rebuilding. This Jira is > concentrated on ALTER INDEX REBUILD ALL sql syntax. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-5272) Support ALTER INDEX REBUILD ALL ASYNC to fully rebuild global indexes async
Gokcen Iskender created PHOENIX-5272: Summary: Support ALTER INDEX REBUILD ALL ASYNC to fully rebuild global indexes async Key: PHOENIX-5272 URL: https://issues.apache.org/jira/browse/PHOENIX-5272 Project: Phoenix Issue Type: Bug Reporter: Gokcen Iskender Assignee: Gokcen Iskender Fix For: 4.15.0, 5.1.0 Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are deleted and the index is rebuilt synchronously. "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) So it seems currently the only way to fully rebuild is the drop the index, and recreate it. This is burdensome as it requires have the schema DDL. We should have an option to fully rebuild asynchronously, that has the same semantics as dropping and recreating the index. A further advantage of this is we can maintain the splits of the index table while dropping its data. We are currently seeing issues where rebuilding a large table via a MR job results in hotspotting due to all data regions writing to the same index region at the start. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4703) Provide an option to fully rebuild indexes asynchronously through SQL
[ https://issues.apache.org/jira/browse/PHOENIX-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-4703: - Comment: was deleted (was: This Jira is part of 4703. On 4703, a flag to IndexTool is added so that we can delete all global indexes before rebuilding. This Jira is concentrated on ALTER INDEX REBUILD ALL sql syntax.) > Provide an option to fully rebuild indexes asynchronously through SQL > - > > Key: PHOENIX-4703 > URL: https://issues.apache.org/jira/browse/PHOENIX-4703 > Project: Phoenix > Issue Type: Bug >Reporter: Vincent Poon >Assignee: Gokcen Iskender >Priority: Major > Fix For: 4.15.0, 5.1.0 > > Attachments: PHOENIX-4703-4.x.patch, PHOENIX-4703.patch > > Time Spent: 5h > Remaining Estimate: 0h > > Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are > deleted and the index is rebuilt synchronously. > "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial > rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) > So it seems currently the only way to fully rebuild is the drop the index, > and recreate it. This is burdensome as it requires have the schema DDL. > We should have an option to fully rebuild asynchronously, that has the same > semantics as dropping and recreating the index. A further advantage of this > is we can maintain the splits of the index table while dropping its data. We > are currently seeing issues where rebuilding a large table via a MR job > results in hotspotting due to all data regions writing to the same index > region at the start. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (PHOENIX-4703) Provide an option to fully rebuild indexes asynchronously through SQL
[ https://issues.apache.org/jira/browse/PHOENIX-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender closed PHOENIX-4703. This Jira is part of 4703. On 4703, a flag to IndexTool is added so that we can delete all global indexes before rebuilding. This Jira is concentrated on ALTER INDEX REBUILD ALL sql syntax. > Provide an option to fully rebuild indexes asynchronously through SQL > - > > Key: PHOENIX-4703 > URL: https://issues.apache.org/jira/browse/PHOENIX-4703 > Project: Phoenix > Issue Type: Bug >Reporter: Vincent Poon >Assignee: Gokcen Iskender >Priority: Major > Fix For: 4.15.0, 5.1.0 > > Attachments: PHOENIX-4703-4.x.patch, PHOENIX-4703.patch > > Time Spent: 5h > Remaining Estimate: 0h > > Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are > deleted and the index is rebuilt synchronously. > "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial > rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) > So it seems currently the only way to fully rebuild is the drop the index, > and recreate it. This is burdensome as it requires have the schema DDL. > We should have an option to fully rebuild asynchronously, that has the same > semantics as dropping and recreating the index. A further advantage of this > is we can maintain the splits of the index table while dropping its data. We > are currently seeing issues where rebuilding a large table via a MR job > results in hotspotting due to all data regions writing to the same index > region at the start. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5156) Consistent Mutable Global Indexes for Non-Transactional Tables
[ https://issues.apache.org/jira/browse/PHOENIX-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5156: --- Attachment: PHOENIX-5156.4.x-HBase-1.4.001.patch > Consistent Mutable Global Indexes for Non-Transactional Tables > -- > > Key: PHOENIX-5156 > URL: https://issues.apache.org/jira/browse/PHOENIX-5156 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0, 4.14.0, 5.0.0, 4.14.1 >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Attachments: PHOENIX-5156.4.x-HBase-1.4.001.patch, > PHOENIX-5156.master.001.patch, PHOENIX-5156.master.002.patch, > PHOENIX-5156.master.003.patch, PHOENIX-5156.master.004.patch, > PHOENIX-5156.master.005.patch, PHOENIX-5156.master.006.patch, > PHOENIX-5156.master.007.patch, PHOENIX-5156.master.008.patch, > PHOENIX-5156.master.009.patch, PHOENIX-5156.master.010.patch, > PHOENIX-5156.master.011.patch, PHOENIX-5156.master.012.patch > > Time Spent: 18.5h > Remaining Estimate: 0h > > Without transactional tables, the mutable global indexes can get easily out > of sync with their data tables in Phoenix. Transactional tables require a > separate transaction manager, have some restrictions and performance > penalties. This issue is to have consistent mutable global indexes without > the need for using transactional tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5156) Consistent Mutable Global Indexes for Non-Transactional Tables
[ https://issues.apache.org/jira/browse/PHOENIX-5156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-5156: --- Attachment: PHOENIX-5156.master.012.patch > Consistent Mutable Global Indexes for Non-Transactional Tables > -- > > Key: PHOENIX-5156 > URL: https://issues.apache.org/jira/browse/PHOENIX-5156 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.13.0, 4.14.0, 5.0.0, 4.14.1 >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Attachments: PHOENIX-5156.master.001.patch, > PHOENIX-5156.master.002.patch, PHOENIX-5156.master.003.patch, > PHOENIX-5156.master.004.patch, PHOENIX-5156.master.005.patch, > PHOENIX-5156.master.006.patch, PHOENIX-5156.master.007.patch, > PHOENIX-5156.master.008.patch, PHOENIX-5156.master.009.patch, > PHOENIX-5156.master.010.patch, PHOENIX-5156.master.011.patch, > PHOENIX-5156.master.012.patch > > Time Spent: 18.5h > Remaining Estimate: 0h > > Without transactional tables, the mutable global indexes can get easily out > of sync with their data tables in Phoenix. Transactional tables require a > separate transaction manager, have some restrictions and performance > penalties. This issue is to have consistent mutable global indexes without > the need for using transactional tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4703) Provide an option to fully rebuild indexes asynchronously through SQL
[ https://issues.apache.org/jira/browse/PHOENIX-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-4703: - Attachment: (was: PHOENIX-4703-4.x.patch) > Provide an option to fully rebuild indexes asynchronously through SQL > - > > Key: PHOENIX-4703 > URL: https://issues.apache.org/jira/browse/PHOENIX-4703 > Project: Phoenix > Issue Type: Bug >Reporter: Vincent Poon >Assignee: Gokcen Iskender >Priority: Major > Attachments: PHOENIX-4703-4.x.patch, PHOENIX-4703.patch > > Time Spent: 5h > Remaining Estimate: 0h > > Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are > deleted and the index is rebuilt synchronously. > "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial > rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) > So it seems currently the only way to fully rebuild is the drop the index, > and recreate it. This is burdensome as it requires have the schema DDL. > We should have an option to fully rebuild asynchronously, that has the same > semantics as dropping and recreating the index. A further advantage of this > is we can maintain the splits of the index table while dropping its data. We > are currently seeing issues where rebuilding a large table via a MR job > results in hotspotting due to all data regions writing to the same index > region at the start. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4703) Provide an option to fully rebuild indexes asynchronously through SQL
[ https://issues.apache.org/jira/browse/PHOENIX-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-4703: - Attachment: PHOENIX-4703-4.x.patch > Provide an option to fully rebuild indexes asynchronously through SQL > - > > Key: PHOENIX-4703 > URL: https://issues.apache.org/jira/browse/PHOENIX-4703 > Project: Phoenix > Issue Type: Bug >Reporter: Vincent Poon >Assignee: Gokcen Iskender >Priority: Major > Attachments: PHOENIX-4703-4.x.patch, PHOENIX-4703.patch > > Time Spent: 5h > Remaining Estimate: 0h > > Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are > deleted and the index is rebuilt synchronously. > "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial > rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) > So it seems currently the only way to fully rebuild is the drop the index, > and recreate it. This is burdensome as it requires have the schema DDL. > We should have an option to fully rebuild asynchronously, that has the same > semantics as dropping and recreating the index. A further advantage of this > is we can maintain the splits of the index table while dropping its data. We > are currently seeing issues where rebuilding a large table via a MR job > results in hotspotting due to all data regions writing to the same index > region at the start. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (PHOENIX-5271) Add b/w compatibility jenkins tests
Thomas D'Silva created PHOENIX-5271: --- Summary: Add b/w compatibility jenkins tests Key: PHOENIX-5271 URL: https://issues.apache.org/jira/browse/PHOENIX-5271 Project: Phoenix Issue Type: Test Reporter: Thomas D'Silva Checks basic functional backward compatibility between the last released major/minor Phoenix version with the next once that will be releasd. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5258) Add support to parse header from the input CSV file as input columns for CsvBulkLoadTool
[ https://issues.apache.org/jira/browse/PHOENIX-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Vithani updated PHOENIX-5258: -- Attachment: (was: PHOENIX-5258-master.patch) > Add support to parse header from the input CSV file as input columns for > CsvBulkLoadTool > > > Key: PHOENIX-5258 > URL: https://issues.apache.org/jira/browse/PHOENIX-5258 > Project: Phoenix > Issue Type: Improvement >Reporter: Prashant Vithani >Priority: Minor > Fix For: 4.15.0, 5.1.0 > > Attachments: PHOENIX-5258-4.x-HBase-1.4.patch, > PHOENIX-5258-master.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Currently, CsvBulkLoadTool does not support reading header from the input csv > and expects the content of the csv to match with the table schema. The > support for the header can be added to dynamically map the schema with the > header. > The proposed solution is to introduce another option for the tool `–header`. > If this option is passed, the input columns list is constructed by reading > the first line of the input CSV file. > * If there is only one file, read the header from the first line and > generate the `ColumnInfo` list. > * If there are multiple files, read the header from all the files, and throw > an error if the headers across files do not match. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5258) Add support to parse header from the input CSV file as input columns for CsvBulkLoadTool
[ https://issues.apache.org/jira/browse/PHOENIX-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Vithani updated PHOENIX-5258: -- Attachment: PHOENIX-5258-master.patch > Add support to parse header from the input CSV file as input columns for > CsvBulkLoadTool > > > Key: PHOENIX-5258 > URL: https://issues.apache.org/jira/browse/PHOENIX-5258 > Project: Phoenix > Issue Type: Improvement >Reporter: Prashant Vithani >Priority: Minor > Fix For: 4.15.0, 5.1.0 > > Attachments: PHOENIX-5258-4.x-HBase-1.4.patch, > PHOENIX-5258-master.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Currently, CsvBulkLoadTool does not support reading header from the input csv > and expects the content of the csv to match with the table schema. The > support for the header can be added to dynamically map the schema with the > header. > The proposed solution is to introduce another option for the tool `–header`. > If this option is passed, the input columns list is constructed by reading > the first line of the input CSV file. > * If there is only one file, read the header from the first line and > generate the `ColumnInfo` list. > * If there are multiple files, read the header from all the files, and throw > an error if the headers across files do not match. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[REPORT] Phoenix - May 2019
## Description:
- Apache Phoenix enables SQL-based OLTP and operational analytics for
Apache
Hadoop
## Issues:
- There are no issues requiring board attention at this time
## Activity:
- Continuing an excellent stream of new committers and PMC members
(details
below). Two more committers have been confirmed via a VOTE but are
pending
acceptance and clerical tasks to complete the addition.
- 4.14.2 is nearing completion. RC0 was voted on, but did not pass.
- 5.0.1 is getting close to an initial vote, but has some
stabilization/cleanup work yet to do. Appears to be naturally
queueing up
after 4.14.2 nicely.
- A community event ("NoSQL Day"), focusing on Accumulo, HBase, and
Phoenix
is scheduled for May 21st in Washington, D.C. We have half a dozen
Phoenix
specific talks accepted on the agenda for the event.
## Health report:
- Nothing notable to report. Trends are continuing per normal. Influx of
"new blood" is a boon to the community, reflected in a good number of
commits and reviews by "old" community members.
## PMC changes:
- Currently 30 PMC members.
- New PMC members:
- Chenglei (2019/04/09)
- Geoffrey Jacoby (2019/04/09)
## Committer base changes:
- Currently 42 committers.
- New committers:
- Mihir Monani (2019/04/27)
- Abhishek Singh Chouhan (2019/04/05)
## Releases:
- 4.14.1 was released on 2018/11/14
- 5.0.0 was released on 2018/07/14
## Mailing list activity:
- Activity is largely in-line with history. No significant changes to
trends
to be identified.
[jira] [Updated] (PHOENIX-5258) Add support to parse header from the input CSV file as input columns for CsvBulkLoadTool
[ https://issues.apache.org/jira/browse/PHOENIX-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Vithani updated PHOENIX-5258: -- Attachment: PHOENIX-5258-master.patch > Add support to parse header from the input CSV file as input columns for > CsvBulkLoadTool > > > Key: PHOENIX-5258 > URL: https://issues.apache.org/jira/browse/PHOENIX-5258 > Project: Phoenix > Issue Type: Improvement >Reporter: Prashant Vithani >Priority: Minor > Fix For: 4.15.0, 5.1.0 > > Attachments: PHOENIX-5258-4.x-HBase-1.4.patch, > PHOENIX-5258-master.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Currently, CsvBulkLoadTool does not support reading header from the input csv > and expects the content of the csv to match with the table schema. The > support for the header can be added to dynamically map the schema with the > header. > The proposed solution is to introduce another option for the tool `–header`. > If this option is passed, the input columns list is constructed by reading > the first line of the input CSV file. > * If there is only one file, read the header from the first line and > generate the `ColumnInfo` list. > * If there are multiple files, read the header from all the files, and throw > an error if the headers across files do not match. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5258) Add support to parse header from the input CSV file as input columns for CsvBulkLoadTool
[ https://issues.apache.org/jira/browse/PHOENIX-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Vithani updated PHOENIX-5258: -- Attachment: (was: PHOENIX-5258-4.x-HBase-1.4.patch) > Add support to parse header from the input CSV file as input columns for > CsvBulkLoadTool > > > Key: PHOENIX-5258 > URL: https://issues.apache.org/jira/browse/PHOENIX-5258 > Project: Phoenix > Issue Type: Improvement >Reporter: Prashant Vithani >Priority: Minor > Fix For: 4.15.0, 5.1.0 > > Attachments: PHOENIX-5258-4.x-HBase-1.4.patch, > PHOENIX-5258-master.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Currently, CsvBulkLoadTool does not support reading header from the input csv > and expects the content of the csv to match with the table schema. The > support for the header can be added to dynamically map the schema with the > header. > The proposed solution is to introduce another option for the tool `–header`. > If this option is passed, the input columns list is constructed by reading > the first line of the input CSV file. > * If there is only one file, read the header from the first line and > generate the `ColumnInfo` list. > * If there are multiple files, read the header from all the files, and throw > an error if the headers across files do not match. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5258) Add support to parse header from the input CSV file as input columns for CsvBulkLoadTool
[ https://issues.apache.org/jira/browse/PHOENIX-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Vithani updated PHOENIX-5258: -- Attachment: PHOENIX-5258-4.x-HBase-1.4.patch > Add support to parse header from the input CSV file as input columns for > CsvBulkLoadTool > > > Key: PHOENIX-5258 > URL: https://issues.apache.org/jira/browse/PHOENIX-5258 > Project: Phoenix > Issue Type: Improvement >Reporter: Prashant Vithani >Priority: Minor > Fix For: 4.15.0, 5.1.0 > > Attachments: PHOENIX-5258-4.x-HBase-1.4.patch, > PHOENIX-5258-master.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Currently, CsvBulkLoadTool does not support reading header from the input csv > and expects the content of the csv to match with the table schema. The > support for the header can be added to dynamically map the schema with the > header. > The proposed solution is to introduce another option for the tool `–header`. > If this option is passed, the input columns list is constructed by reading > the first line of the input CSV file. > * If there is only one file, read the header from the first line and > generate the `ColumnInfo` list. > * If there are multiple files, read the header from all the files, and throw > an error if the headers across files do not match. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5258) Add support to parse header from the input CSV file as input columns for CsvBulkLoadTool
[ https://issues.apache.org/jira/browse/PHOENIX-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Vithani updated PHOENIX-5258: -- Attachment: (was: PHOENIX-5258-master.patch) > Add support to parse header from the input CSV file as input columns for > CsvBulkLoadTool > > > Key: PHOENIX-5258 > URL: https://issues.apache.org/jira/browse/PHOENIX-5258 > Project: Phoenix > Issue Type: Improvement >Reporter: Prashant Vithani >Priority: Minor > Fix For: 4.15.0, 5.1.0 > > Attachments: PHOENIX-5258-4.x-HBase-1.4.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Currently, CsvBulkLoadTool does not support reading header from the input csv > and expects the content of the csv to match with the table schema. The > support for the header can be added to dynamically map the schema with the > header. > The proposed solution is to introduce another option for the tool `–header`. > If this option is passed, the input columns list is constructed by reading > the first line of the input CSV file. > * If there is only one file, read the header from the first line and > generate the `ColumnInfo` list. > * If there are multiple files, read the header from all the files, and throw > an error if the headers across files do not match. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (PHOENIX-5269) PhoenixAccessController should use AccessChecker instead of AccessControlClient for permission checks
[ https://issues.apache.org/jira/browse/PHOENIX-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned PHOENIX-5269: --- Assignee: Kiran Kumar Maturi (was: Andrew Purtell) > PhoenixAccessController should use AccessChecker instead of > AccessControlClient for permission checks > - > > Key: PHOENIX-5269 > URL: https://issues.apache.org/jira/browse/PHOENIX-5269 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.14.2 >Reporter: Andrew Purtell >Assignee: Kiran Kumar Maturi >Priority: Critical > > PhoenixAccessController should use AccessChecker instead of > AccessControlClient for permission checks. > In HBase, every RegionServer's AccessController maintains a local cache of > permissions. At startup time they are initialized from the ACL table. > Whenever the ACL table is changed (via grant or revoke) the AC on the ACL > table "broadcasts" the change via zookeeper, which updates the cache. This is > performed and managed by TableAuthManager but is exposed as API by > AccessChecker. AccessChecker is the result of a refactor that was committed > as far back as branch-1.4 I believe. > Phoenix implements its own access controller and is using the client API > AccessControlClient instead. AccessControlClient does not cache nor use the > ZK-based cache update mechanism, because it is designed for client side use. > The use of AccessControlClient instead of AccessChecker is not scalable. > Every permissions check will trigger a remote RPC to the ACL table, which is > generally going to be a single region hosted on a single RegionServer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4703) Provide an option to fully rebuild indexes asynchronously through SQL
[ https://issues.apache.org/jira/browse/PHOENIX-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-4703: - Attachment: PHOENIX-4703.patch > Provide an option to fully rebuild indexes asynchronously through SQL > - > > Key: PHOENIX-4703 > URL: https://issues.apache.org/jira/browse/PHOENIX-4703 > Project: Phoenix > Issue Type: Bug >Reporter: Vincent Poon >Assignee: Gokcen Iskender >Priority: Major > Attachments: PHOENIX-4703-4.x.patch, PHOENIX-4703.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are > deleted and the index is rebuilt synchronously. > "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial > rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) > So it seems currently the only way to fully rebuild is the drop the index, > and recreate it. This is burdensome as it requires have the schema DDL. > We should have an option to fully rebuild asynchronously, that has the same > semantics as dropping and recreating the index. A further advantage of this > is we can maintain the splits of the index table while dropping its data. We > are currently seeing issues where rebuilding a large table via a MR job > results in hotspotting due to all data regions writing to the same index > region at the start. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4703) Provide an option to fully rebuild indexes asynchronously through SQL
[ https://issues.apache.org/jira/browse/PHOENIX-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-4703: - Attachment: (was: PHOENIX-4703.patch) > Provide an option to fully rebuild indexes asynchronously through SQL > - > > Key: PHOENIX-4703 > URL: https://issues.apache.org/jira/browse/PHOENIX-4703 > Project: Phoenix > Issue Type: Bug >Reporter: Vincent Poon >Assignee: Gokcen Iskender >Priority: Major > Attachments: PHOENIX-4703-4.x.patch, PHOENIX-4703.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are > deleted and the index is rebuilt synchronously. > "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial > rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) > So it seems currently the only way to fully rebuild is the drop the index, > and recreate it. This is burdensome as it requires have the schema DDL. > We should have an option to fully rebuild asynchronously, that has the same > semantics as dropping and recreating the index. A further advantage of this > is we can maintain the splits of the index table while dropping its data. We > are currently seeing issues where rebuilding a large table via a MR job > results in hotspotting due to all data regions writing to the same index > region at the start. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4703) Provide an option to fully rebuild indexes asynchronously through SQL
[ https://issues.apache.org/jira/browse/PHOENIX-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-4703: - Attachment: PHOENIX-4703-4.x.patch > Provide an option to fully rebuild indexes asynchronously through SQL > - > > Key: PHOENIX-4703 > URL: https://issues.apache.org/jira/browse/PHOENIX-4703 > Project: Phoenix > Issue Type: Bug >Reporter: Vincent Poon >Assignee: Gokcen Iskender >Priority: Major > Attachments: PHOENIX-4703-4.x.patch, PHOENIX-4703.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are > deleted and the index is rebuilt synchronously. > "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial > rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) > So it seems currently the only way to fully rebuild is the drop the index, > and recreate it. This is burdensome as it requires have the schema DDL. > We should have an option to fully rebuild asynchronously, that has the same > semantics as dropping and recreating the index. A further advantage of this > is we can maintain the splits of the index table while dropping its data. We > are currently seeing issues where rebuilding a large table via a MR job > results in hotspotting due to all data regions writing to the same index > region at the start. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (PHOENIX-5261) Implement ALTER TABLE ADD COLUMN CASCADE
[ https://issues.apache.org/jira/browse/PHOENIX-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby reassigned PHOENIX-5261: Assignee: Swaroopa Kadam (was: Geoffrey Jacoby) > Implement ALTER TABLE ADD COLUMN CASCADE > > > Key: PHOENIX-5261 > URL: https://issues.apache.org/jira/browse/PHOENIX-5261 > Project: Phoenix > Issue Type: Improvement >Reporter: Geoffrey Jacoby >Assignee: Swaroopa Kadam >Priority: Major > > Phoenix currently gives no way to alter the columns in an index. This is > because adding a column to an index would require a full rebuild to cycle > through the parent table and add the values to the index. > There is a special case, however, when adding a column to an index would be > very useful and easy, and that is when adding a new nullable field to the > index's parent table. In this case it is safe to just go ahead and add the > column to the index, because at DDL time the field is known to be NULL on > each row. > I propose adding an optional parameter CASCADE to ALTER TABLE ADD COLUMN, > which when used will do the following: > 1. The new column will be automatically added to any child views. > 2. The new column will be automatically added as an INCLUDED column in any > secondary index belonging to the parent table. > Outstanding questions: > 1. Does splittable system catalog already take care of Item 1? > 2. What about tenant-owned views? > 3. Should there be a way to exclude adding the column to a child index to > allow for uncovered usage? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-4703) Provide an option to fully rebuild indexes asynchronously through SQL
[ https://issues.apache.org/jira/browse/PHOENIX-4703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gokcen Iskender updated PHOENIX-4703: - Attachment: PHOENIX-4703.patch > Provide an option to fully rebuild indexes asynchronously through SQL > - > > Key: PHOENIX-4703 > URL: https://issues.apache.org/jira/browse/PHOENIX-4703 > Project: Phoenix > Issue Type: Bug >Reporter: Vincent Poon >Assignee: Gokcen Iskender >Priority: Major > Attachments: PHOENIX-4703.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Currently if we run "ALTER INDEX ... REBUILD" , all the rows in the index are > deleted and the index is rebuilt synchronously. > "ALTER INEX ... REBUILD ASYNC" seems to be used for the IndexTool's partial > rebuild option, rebuilding from ASYNC_REBUILD_TIMESTAMP (PHOENIX-2890) > So it seems currently the only way to fully rebuild is the drop the index, > and recreate it. This is burdensome as it requires have the schema DDL. > We should have an option to fully rebuild asynchronously, that has the same > semantics as dropping and recreating the index. A further advantage of this > is we can maintain the splits of the index table while dropping its data. We > are currently seeing issues where rebuilding a large table via a MR job > results in hotspotting due to all data regions writing to the same index > region at the start. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (PHOENIX-5270) java.lang.AbstractMethodError: org.apache.phoenix.spark.DefaultSource.createRelation
[
https://issues.apache.org/jira/browse/PHOENIX-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ladislav Jech updated PHOENIX-5270:
---
Environment:
Spark env (localhost):
{code:java}
Welcome to
__
/ __/__ ___ _/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.4.2
/_/
Using Scala version 2.12.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_202)
Type in expressions to have them evaluated.
Type :help for more information.{code}
Hbase version (on cloudera cluster): HBase 1.2.0-cdh5.12.2
Phoenix distro used used: apache-phoenix-4.14.0-cdh5.14.2-bin and
apache-phoenix-4.14.1-HBase-1.2-bin i tried.
was:
Spark env (localhost):
{code:java}
Welcome to
__
/ __/__ ___ _/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.4.2
/_/
Using Scala version 2.12.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_202)
Type in expressions to have them evaluated.
Type :help for more information.{code}
Hbase version (on cloudera cluster): HBase 1.2.0-cdh5.12.2
Phoenix distro used used: apache-phoenix-4.14.0-cdh5.14.2-bin
> java.lang.AbstractMethodError:
> org.apache.phoenix.spark.DefaultSource.createRelation
>
>
> Key: PHOENIX-5270
> URL: https://issues.apache.org/jira/browse/PHOENIX-5270
> Project: Phoenix
> Issue Type: Bug
> Environment: Spark env (localhost):
>
> {code:java}
> Welcome to
> __
> / __/__ ___ _/ /__
> _\ \/ _ \/ _ `/ __/ '_/
> /___/ .__/\_,_/_/ /_/\_\ version 2.4.2
> /_/
> Using Scala version 2.12.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_202)
> Type in expressions to have them evaluated.
> Type :help for more information.{code}
>
>
> Hbase version (on cloudera cluster): HBase 1.2.0-cdh5.12.2
> Phoenix distro used used: apache-phoenix-4.14.0-cdh5.14.2-bin and
> apache-phoenix-4.14.1-HBase-1.2-bin i tried.
>Reporter: Ladislav Jech
>Priority: Major
>
> I get issue with Spark 2.4.2 Standalone binary and Phoenix:
> {code:java}
> py4j.protocol.Py4JJavaError: An error occurred while calling o134.save.
> : java.lang.AbstractMethodError:
> org.apache.phoenix.spark.DefaultSource.createRelation(Lorg/apache/spark/sql/SQLContext;Lorg/apache/spark/sql/SaveMode;Lscala/collection/immutable/Map;Lorg/apache/spark/sql/Dataset;)Lorg/apache/spark/sql/sources/BaseRelation;
> at
> org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
> at
> org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:131)
> at
> org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:155)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
> at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
> at
> org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80)
> at
> org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80)
> at
> org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:676)
> at
> org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:78)
> at
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
> at
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
> at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:676)
> at
> org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:290)
> at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:271)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> at py4j.Gateway.invoke(Gateway.java:282)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:238)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by
[jira] [Updated] (PHOENIX-5270) java.lang.AbstractMethodError: org.apache.phoenix.spark.DefaultSource.createRelation
[
https://issues.apache.org/jira/browse/PHOENIX-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ladislav Jech updated PHOENIX-5270:
---
Issue Type: Bug (was: Improvement)
> java.lang.AbstractMethodError:
> org.apache.phoenix.spark.DefaultSource.createRelation
>
>
> Key: PHOENIX-5270
> URL: https://issues.apache.org/jira/browse/PHOENIX-5270
> Project: Phoenix
> Issue Type: Bug
> Environment: Spark env (localhost):
>
> {code:java}
> Welcome to
> __
> / __/__ ___ _/ /__
> _\ \/ _ \/ _ `/ __/ '_/
> /___/ .__/\_,_/_/ /_/\_\ version 2.4.2
> /_/
> Using Scala version 2.12.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_202)
> Type in expressions to have them evaluated.
> Type :help for more information.{code}
>
>
> Hbase version (on cloudera cluster): HBase 1.2.0-cdh5.12.2
> Phoenix distro used used: apache-phoenix-4.14.0-cdh5.14.2-bin
>Reporter: Ladislav Jech
>Priority: Major
>
> I get issue with Spark 2.4.2 Standalone binary and Phoenix:
> {code:java}
> py4j.protocol.Py4JJavaError: An error occurred while calling o134.save.
> : java.lang.AbstractMethodError:
> org.apache.phoenix.spark.DefaultSource.createRelation(Lorg/apache/spark/sql/SQLContext;Lorg/apache/spark/sql/SaveMode;Lscala/collection/immutable/Map;Lorg/apache/spark/sql/Dataset;)Lorg/apache/spark/sql/sources/BaseRelation;
> at
> org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
> at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
> at
> org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:131)
> at
> org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:155)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
> at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
> at
> org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80)
> at
> org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80)
> at
> org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:676)
> at
> org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:78)
> at
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
> at
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
> at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:676)
> at
> org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:290)
> at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:271)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> at py4j.Gateway.invoke(Gateway.java:282)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:238)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (PHOENIX-5270) java.lang.AbstractMethodError: org.apache.phoenix.spark.DefaultSource.createRelation
Ladislav Jech created PHOENIX-5270:
--
Summary: java.lang.AbstractMethodError:
org.apache.phoenix.spark.DefaultSource.createRelation
Key: PHOENIX-5270
URL: https://issues.apache.org/jira/browse/PHOENIX-5270
Project: Phoenix
Issue Type: Improvement
Environment: Spark env (localhost):
{code:java}
Welcome to
__
/ __/__ ___ _/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.4.2
/_/
Using Scala version 2.12.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_202)
Type in expressions to have them evaluated.
Type :help for more information.{code}
Hbase version (on cloudera cluster): HBase 1.2.0-cdh5.12.2
Phoenix distro used used: apache-phoenix-4.14.0-cdh5.14.2-bin
Reporter: Ladislav Jech
I get issue with Spark 2.4.2 Standalone binary and Phoenix:
{code:java}
py4j.protocol.Py4JJavaError: An error occurred while calling o134.save.
: java.lang.AbstractMethodError:
org.apache.phoenix.spark.DefaultSource.createRelation(Lorg/apache/spark/sql/SQLContext;Lorg/apache/spark/sql/SaveMode;Lscala/collection/immutable/Map;Lorg/apache/spark/sql/Dataset;)Lorg/apache/spark/sql/sources/BaseRelation;
at
org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at
org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$execute$1(SparkPlan.scala:131)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:155)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at
org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80)
at
org.apache.spark.sql.DataFrameWriter.$anonfun$runCommand$1(DataFrameWriter.scala:676)
at
org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:78)
at
org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125)
at
org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:676)
at
org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:290)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:271)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
