[GitHub] [phoenix-omid] chia7712 commented on pull request #76: OMID-170 Configure checkstyle rules for all sub modules
chia7712 commented on pull request #76: URL: https://github.com/apache/phoenix-omid/pull/76#issuecomment-730967693 rebase code This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (PHOENIX-6229) When dataframe write Phoniex,saveMode will report an error only if it is not set to overwrite mode
[
https://issues.apache.org/jira/browse/PHOENIX-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
何军红 updated PHOENIX-6229:
-
Description:
org.apache.phoenix.spark.DefaultSource
Savemode will not report errors unless it is set to overwrite，But the message
to throw an exception is that overwrite is not supported
think it might be written like this,The logic is the opposite
if (mode.equals(SaveMode.Overwrite))
{ throw new Exception("SaveMode other than SaveMode.OverWrite is not
supported") }
!image-2020-11-19-17-17-43-669.png|width=648,height=149!
was:
org.apache.phoenix.spark.DefaultSource
只有saveMode是overwrite才不会抛异常，但抛异常的信息是不支持overwrite
应该是
if (mode.equals(SaveMode.Overwrite)) {
throw new Exception("SaveMode other than SaveMode.OverWrite is not supported")
}
!image-2020-11-19-17-17-43-669.png|width=648,height=149!
Environment:
jar-version
org.apache.phoenix
phoenix-spark
5.0.0-HBase-2.0
my code：
dataFrame.write
.format("org.apache.phoenix.spark")
.option("table", table)
.option("zkUrl", zkUrl)
.option("skipNormalizingIdentifier", true)
.save()
}
was:
版本
org.apache.phoenix
phoenix-spark
5.0.0-HBase-2.0
我的写法
dataFrame.write
.format("org.apache.phoenix.spark")
.mode("overwrite")
.option("table", table)
.option("zkUrl", zkUrl)
.option("skipNormalizingIdentifier", true)
.save()
}
Summary: When dataframe write Phoniex,saveMode will report an error
only if it is not set to overwrite mode (was:
dataframe写入phoniex时,saveMode设置为除了overwrite方式时都会报错)
> When dataframe write Phoniex,saveMode will report an error only if it is not
> set to overwrite mode
> --
>
> Key: PHOENIX-6229
> URL: https://issues.apache.org/jira/browse/PHOENIX-6229
> Project: Phoenix
> Issue Type: Bug
> Components: spark-connector
>Affects Versions: 5.0.0
> Environment: jar-version
>
> org.apache.phoenix
> phoenix-spark
> 5.0.0-HBase-2.0
>
> my code：
> dataFrame.write
> .format("org.apache.phoenix.spark")
> .option("table", table)
> .option("zkUrl", zkUrl)
> .option("skipNormalizingIdentifier", true)
> .save()
> }
>Reporter: 何军红
>Priority: Minor
> Attachments: image-2020-11-19-17-17-43-669.png
>
>
> org.apache.phoenix.spark.DefaultSource
> Savemode will not report errors unless it is set to overwrite，But the message
> to throw an exception is that overwrite is not supported
> think it might be written like this,The logic is the opposite
> if (mode.equals(SaveMode.Overwrite))
> { throw new Exception("SaveMode other than SaveMode.OverWrite is not
> supported") }
> !image-2020-11-19-17-17-43-669.png|width=648,height=149!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (PHOENIX-6170) PHOENIX_TTL spec should be in seconds instead of milliseconds
[ https://issues.apache.org/jira/browse/PHOENIX-6170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Geoffrey Jacoby updated PHOENIX-6170: - Fix Version/s: (was: 4.x) 4.16.0 > PHOENIX_TTL spec should be in seconds instead of milliseconds > - > > Key: PHOENIX-6170 > URL: https://issues.apache.org/jira/browse/PHOENIX-6170 > Project: Phoenix > Issue Type: Sub-task >Affects Versions: 4.x >Reporter: Jacob Isaac >Assignee: Jacob Isaac >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-6170.4.x.001.patch > > > When defining the PHOENIX_TTL spec it should be specified in seconds, which > is also how HBase TTL value is set. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-5920) Skip SYSTEM TABLE checks while creating phoenix connection if client has set the DoNotUpgrade config
[ https://issues.apache.org/jira/browse/PHOENIX-5920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neha Gupta updated PHOENIX-5920: Attachment: PHOENIX-5920.v5.patch > Skip SYSTEM TABLE checks while creating phoenix connection if client has set > the DoNotUpgrade config > > > Key: PHOENIX-5920 > URL: https://issues.apache.org/jira/browse/PHOENIX-5920 > Project: Phoenix > Issue Type: Bug >Affects Versions: 5.0.0, 4.15.0, 4.14.3 >Reporter: Neha Gupta >Assignee: Neha Gupta >Priority: Major > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-5920.patch, PHOENIX-5920.v1.patch, > PHOENIX-5920.v2.patch, PHOENIX-5920.v3.patch, PHOENIX-5920.v4.patch, > PHOENIX-5920.v5.patch > > Time Spent: 2h > Remaining Estimate: 0h > > Phoenix checks upgrade path even if client has set "DoNotUpgrade" config. > When HMaster failover happens and Phoenix client creating phoenix connection, > we get exception "org.apache.hadoop.hbase.PleaseHoldException: Master is > initializing" because phoenix calls HBase admin apis to check if SYSTEM > tables are in place and does not require an upgrade. > [CQSI.init()|https://github.com/apache/phoenix/blob/aad583670ea821286ab5e2460ce0ab7255d474c4/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L3169] > initiates creating systems table and > [CQSI.ensureTableCreated|https://github.com/apache/phoenix/blob/aad583670ea821286ab5e2460ce0ab7255d474c4/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java#L1246] > is checking if SYTEM table requires upgrade and calls hbaseAdminApis to > perform this operation which fails if HMaster is not running. Can we honor > the "DoNotUpgrade" config by client(By making an assumption that SYSTEM table > is already up-to-date and phoenix client shouldn't perform any upgrades) and > skip this upgrade checks ? > EnsureTableCreated performs bunch of client-server compatibility checks like: > 1.SYSTEM.CATALOG exists and its timestamp < MIN_SYSTEM_TABLE_TIMESTAMP > 2.SYSTEM.CATALOG exists, but client and server-side namespace mapping is > enabled so we need to migrate SYSTEM tables to the SYSTEM namespace > 3.When an end-user uses the vanilla PhoenixDriver to create a connection and > a requirement for upgrade is detected. In this case, the user should get a > connection on which they are only able to run "EXECUTE UPGRADE". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6207) Paged server side grouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-6207: --- Attachment: (was: PHOENIX-6211.master.002.patch) > Paged server side grouped aggregate operations > -- > > Key: PHOENIX-6207 > URL: https://issues.apache.org/jira/browse/PHOENIX-6207 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 5.0.0, 4.14.3 >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-6207.4.x.001.patch, PHOENIX-6207.4.x.002.patch, > PHOENIX-6207.4.x.003.patch, PHOENIX-6207.4.x.004.patch, > PHOENIX-6207.4.x.005.patch, PHOENIX-6207.4.x.006.patch, > PHOENIX-6207.master.001.patch, PHOENIX-6207.master.002.patch > > > Phoenix provides the option of performing query operations on the client or > server side. This is decided by the Phoenix optimizer based on configuration > parameters. For the server side option, the table operation is parallelized > such that multiple table regions are scanned. However, currently there is no > paging capability and the server side operation can take long enough lead to > HBase client timeouts. Putting a limit on the processing time within a single > RPC call (i.e., the next operation time on the aggregate scanner) on the > server side using a Phoenix level paging is highly desirable. A similar > paging mechanism has been already implemented for index rebuild and > verification operations and proven to be effective to prevent timeouts. This > Jira is for implementing this paging for the server side grouped aggregate > operations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6207) Paged server side grouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-6207: --- Attachment: PHOENIX-6207.master.002.patch > Paged server side grouped aggregate operations > -- > > Key: PHOENIX-6207 > URL: https://issues.apache.org/jira/browse/PHOENIX-6207 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 5.0.0, 4.14.3 >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-6207.4.x.001.patch, PHOENIX-6207.4.x.002.patch, > PHOENIX-6207.4.x.003.patch, PHOENIX-6207.4.x.004.patch, > PHOENIX-6207.4.x.005.patch, PHOENIX-6207.4.x.006.patch, > PHOENIX-6207.master.001.patch, PHOENIX-6207.master.002.patch > > > Phoenix provides the option of performing query operations on the client or > server side. This is decided by the Phoenix optimizer based on configuration > parameters. For the server side option, the table operation is parallelized > such that multiple table regions are scanned. However, currently there is no > paging capability and the server side operation can take long enough lead to > HBase client timeouts. Putting a limit on the processing time within a single > RPC call (i.e., the next operation time on the aggregate scanner) on the > server side using a Phoenix level paging is highly desirable. A similar > paging mechanism has been already implemented for index rebuild and > verification operations and proven to be effective to prevent timeouts. This > Jira is for implementing this paging for the server side grouped aggregate > operations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6230) IT suite hangs on ViewConcurrencyAndFailureIT
[ https://issues.apache.org/jira/browse/PHOENIX-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated PHOENIX-6230: -- Fix Version/s: 5.1.0 > IT suite hangs on ViewConcurrencyAndFailureIT > - > > Key: PHOENIX-6230 > URL: https://issues.apache.org/jira/browse/PHOENIX-6230 > Project: Phoenix > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Istvan Toth >Assignee: Chinmay Kulkarni >Priority: Critical > Fix For: 5.1.0 > > > The ASF Jenkins postcommit job timed out 5 times out of the last 6 runs on > the master branch. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6230) IT suite hangs on ViewConcurrencyAndFailureIT
[ https://issues.apache.org/jira/browse/PHOENIX-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni updated PHOENIX-6230: -- Affects Version/s: (was: 5.1.0) 5.0.0 > IT suite hangs on ViewConcurrencyAndFailureIT > - > > Key: PHOENIX-6230 > URL: https://issues.apache.org/jira/browse/PHOENIX-6230 > Project: Phoenix > Issue Type: Bug > Components: core >Affects Versions: 5.0.0 >Reporter: Istvan Toth >Assignee: Chinmay Kulkarni >Priority: Critical > > The ASF Jenkins postcommit job timed out 5 times out of the last 6 runs on > the master branch. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6207) Paged server side grouped aggregate operations
[ https://issues.apache.org/jira/browse/PHOENIX-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kadir OZDEMIR updated PHOENIX-6207: --- Attachment: PHOENIX-6211.master.002.patch > Paged server side grouped aggregate operations > -- > > Key: PHOENIX-6207 > URL: https://issues.apache.org/jira/browse/PHOENIX-6207 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 5.0.0, 4.14.3 >Reporter: Kadir OZDEMIR >Assignee: Kadir OZDEMIR >Priority: Major > Fix For: 4.16.0 > > Attachments: PHOENIX-6207.4.x.001.patch, PHOENIX-6207.4.x.002.patch, > PHOENIX-6207.4.x.003.patch, PHOENIX-6207.4.x.004.patch, > PHOENIX-6207.4.x.005.patch, PHOENIX-6207.4.x.006.patch, > PHOENIX-6207.master.001.patch, PHOENIX-6211.master.002.patch > > > Phoenix provides the option of performing query operations on the client or > server side. This is decided by the Phoenix optimizer based on configuration > parameters. For the server side option, the table operation is parallelized > such that multiple table regions are scanned. However, currently there is no > paging capability and the server side operation can take long enough lead to > HBase client timeouts. Putting a limit on the processing time within a single > RPC call (i.e., the next operation time on the aggregate scanner) on the > server side using a Phoenix level paging is highly desirable. A similar > paging mechanism has been already implemented for index rebuild and > verification operations and proven to be effective to prevent timeouts. This > Jira is for implementing this paging for the server side grouped aggregate > operations. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (PHOENIX-6230) IT suite hangs on ViewConcurrencyAndFailureIT
[ https://issues.apache.org/jira/browse/PHOENIX-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kulkarni reassigned PHOENIX-6230: - Assignee: Chinmay Kulkarni > IT suite hangs on ViewConcurrencyAndFailureIT > - > > Key: PHOENIX-6230 > URL: https://issues.apache.org/jira/browse/PHOENIX-6230 > Project: Phoenix > Issue Type: Bug > Components: core >Affects Versions: 5.1.0 >Reporter: Istvan Toth >Assignee: Chinmay Kulkarni >Priority: Critical > > The ASF Jenkins postcommit job timed out 5 times out of the last 6 runs on > the master branch. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6228) Admin resources are not closed in some places
[ https://issues.apache.org/jira/browse/PHOENIX-6228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated PHOENIX-6228: -- Attachment: PHOENIX-6228.master.000.patch > Admin resources are not closed in some places > - > > Key: PHOENIX-6228 > URL: https://issues.apache.org/jira/browse/PHOENIX-6228 > Project: Phoenix > Issue Type: Bug >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Fix For: 5.1.0, 4.16.0 > > Attachments: PHOENIX-6228.4.x.000.patch, PHOENIX-6228.master.000.patch > > > Came across some imp workflows where Admin resource is leaked. We should > properly close these resources as part of this Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (PHOENIX-6230) IT suite hangs on ViewConcurrencyAndFailureIT
[ https://issues.apache.org/jira/browse/PHOENIX-6230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated PHOENIX-6230: - Description: The ASF Jenkins postcommit job timed out 5 times out of the last 6 runs on the master branch. was: The ASF Jenkins postcommit job timed out 5 times out of the last 6 runs. > IT suite hangs on ViewConcurrencyAndFailureIT > - > > Key: PHOENIX-6230 > URL: https://issues.apache.org/jira/browse/PHOENIX-6230 > Project: Phoenix > Issue Type: Bug > Components: core >Affects Versions: 5.1.0 >Reporter: Istvan Toth >Priority: Critical > > The ASF Jenkins postcommit job timed out 5 times out of the last 6 runs on > the master branch. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (PHOENIX-6230) IT suite hangs on ViewConcurrencyAndFailureIT
Istvan Toth created PHOENIX-6230: Summary: IT suite hangs on ViewConcurrencyAndFailureIT Key: PHOENIX-6230 URL: https://issues.apache.org/jira/browse/PHOENIX-6230 Project: Phoenix Issue Type: Bug Components: core Affects Versions: 5.1.0 Reporter: Istvan Toth The ASF Jenkins postcommit job timed out 5 times out of the last 6 runs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (PHOENIX-6229) dataframe写入phoniex时,saveMode设置为除了overwrite方式时都会报错
何军红 created PHOENIX-6229:
Summary: dataframe写入phoniex时,saveMode设置为除了overwrite方式时都会报错
Key: PHOENIX-6229
URL: https://issues.apache.org/jira/browse/PHOENIX-6229
Project: Phoenix
Issue Type: Bug
Components: spark-connector
Affects Versions: 5.0.0
Environment: 版本
org.apache.phoenix
phoenix-spark
5.0.0-HBase-2.0
我的写法
dataFrame.write
.format("org.apache.phoenix.spark")
.mode("overwrite")
.option("table", table)
.option("zkUrl", zkUrl)
.option("skipNormalizingIdentifier", true)
.save()
}
Reporter: 何军红
Attachments: image-2020-11-19-17-17-43-669.png
org.apache.phoenix.spark.DefaultSource
只有saveMode是overwrite才不会抛异常，但抛异常的信息是不支持overwrite
应该是
if (mode.equals(SaveMode.Overwrite)) {
throw new Exception("SaveMode other than SaveMode.OverWrite is not supported")
}
!image-2020-11-19-17-17-43-669.png|width=648,height=149!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
