[jira] [Assigned] (PHOENIX-6354) Update to spark 3.0
[ https://issues.apache.org/jira/browse/PHOENIX-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal reassigned PHOENIX-6354: -- Assignee: (was: Ashwin Balasubramani) > Update to spark 3.0 > --- > > Key: PHOENIX-6354 > URL: https://issues.apache.org/jira/browse/PHOENIX-6354 > Project: Phoenix > Issue Type: Improvement > Components: spark-connector >Reporter: Alejandro Anadon >Priority: Major > > I am trying to use the phoenix spark connector with spark 3.0. > I tried to compile from trunk; but it is still configured with 2.4.0. > Is is not as easy (I am naive) as change the pon.xml from : > 2.4.0 > to > 3.0.1 > > I found that in class : > org.apache.phoenix.spark.datasource.v2.PhoenixDataSource is still using the > old DataSourcesV2 API; and I don't have the knowledge for updating it in a > brand. > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (PHOENIX-6632) Migrate connectors to Spark-3
[ https://issues.apache.org/jira/browse/PHOENIX-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal reassigned PHOENIX-6632: -- Assignee: Ashwin Balasubramani > Migrate connectors to Spark-3 > - > > Key: PHOENIX-6632 > URL: https://issues.apache.org/jira/browse/PHOENIX-6632 > Project: Phoenix > Issue Type: Improvement > Components: spark-connector >Affects Versions: connectors-6.0.0 >Reporter: Ashwin Balasubramani >Assignee: Ashwin Balasubramani >Priority: Major > > With Spark-3, the DatasourceV2 API has had major changes, where a new > TableProvider Interface has been introduced. These new changes bring in more > control to the data source developer and better integration with > spark-optimizer. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (PHOENIX-6354) Update to spark 3.0
[ https://issues.apache.org/jira/browse/PHOENIX-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Singhal reassigned PHOENIX-6354: -- Assignee: Ashwin Balasubramani > Update to spark 3.0 > --- > > Key: PHOENIX-6354 > URL: https://issues.apache.org/jira/browse/PHOENIX-6354 > Project: Phoenix > Issue Type: Improvement > Components: spark-connector >Reporter: Alejandro Anadon >Assignee: Ashwin Balasubramani >Priority: Major > > I am trying to use the phoenix spark connector with spark 3.0. > I tried to compile from trunk; but it is still configured with 2.4.0. > Is is not as easy (I am naive) as change the pon.xml from : > 2.4.0 > to > 3.0.1 > > I found that in class : > org.apache.phoenix.spark.datasource.v2.PhoenixDataSource is still using the > old DataSourcesV2 API; and I don't have the knowledge for updating it in a > brand. > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
Re: [DISCUSS] Releasing Phoenix-thirdparty 1.2 (or 2.0 ?)
+1, on bumping to 2.0 just to keep the version consistent in case someone wants to use it as a drop-in jar. PHOENIX-6641 also looks good to me, just gave +1 Regards, Ankit Singhal On Thu, 3 Feb 2022 at 04:35, Istvan Toth wrote: > Hi! > > I think that it is time to update phoenix-thirdparty. > There are only two changes: > >- PHOENIX-6575 Replace patched commons-cli with original one when a >release with CLI-254 is available > > which replaces the current patched commons-cli with the official 1.5.0 > release, which has the same fixes. > Unfortunately, the API that enables the fixes is a bit different, and > requires minor code changes in the downstream projects. > I'm not sure if we should bump the version to 2.0 because of that, or if > 1.2.0 is enough. > > The other change (not yet committed) is > >- PHOENIX-6641 Bump Guava to 31.0.1 in phoenix-thirdparty > > The current Guava version has CVE-2020-8908 . Now the vulnerability is not > really fixed in any later version, the problematic method is > just @deprecated . > Still, I guess it's better to keep up with the releases than to get stuck > on an old one, which is likely to cause problems later. > > Uncharacteristically, this Guava update does not seem to break any of our > code. > > As you can see, neither of the changes are critical, but I think both are > nice to have. > > Please let me know your opinion, if you agree, or if you agree. > Please also review PHOENIX-6641, if you have the time. > > regards > Istvan >
[GitHub] [phoenix-thirdparty] ankitsinghal commented on a change in pull request #4: PHOENIX-6641 Bump Guava to 31.0.1 in phoenix-thirdparty
ankitsinghal commented on a change in pull request #4: URL: https://github.com/apache/phoenix-thirdparty/pull/4#discussion_r802330352 ## File path: pom.xml ## @@ -68,7 +68,7 @@ org.apache.phoenix.thirdparty -29.0-android +31.0.1-android Review comment: Do we still want to stick to android not 31.0.1-jre because of java 7 compatibility? Not sure if we add any compat module in our third-party library for jdk7 separately though this is entirely unrelated to this fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [phoenix-thirdparty] ankitsinghal commented on a change in pull request #4: PHOENIX-6641 Bump Guava to 31.0.1 in phoenix-thirdparty
ankitsinghal commented on a change in pull request #4: URL: https://github.com/apache/phoenix-thirdparty/pull/4#discussion_r802330352 ## File path: pom.xml ## @@ -68,7 +68,7 @@ org.apache.phoenix.thirdparty -29.0-android +31.0.1-android Review comment: Do we still want to stick to android not 31.0.1-jre because of java 7 compatibility? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@phoenix.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (PHOENIX-6229) When dataframe write Phoniex,saveMode will report an error only if it is not set to overwrite mode
[
https://issues.apache.org/jira/browse/PHOENIX-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth resolved PHOENIX-6229.
--
Resolution: Not A Bug
The message and the check are consistent.
The intention is to throw an error if the SaveMode is NOT overwite.
The messages says: "SaveMode other than SaveMode.OverWrite is not supported"
Which is equivalent to:"Only SaveMode.OverWrite is supported"
There is a broader issue that Overwrite seems to imply that we drop the table,
and create a new one, which is not true, and we should really insist on using
SaveMode.append instead, which is closer to how the connector actually works.
At this point I wouldn't break existing client code that was written to use
SaveMode.OverWrite, so I'll just let that pass.
The new Spark3 API omits SaveMode.
> When dataframe write Phoniex,saveMode will report an error only if it is not
> set to overwrite mode
> --
>
> Key: PHOENIX-6229
> URL: https://issues.apache.org/jira/browse/PHOENIX-6229
> Project: Phoenix
> Issue Type: Bug
> Components: spark-connector
>Affects Versions: 5.0.0
> Environment: jar-version
>
> org.apache.phoenix
> phoenix-spark
> 5.0.0-HBase-2.0
>
> my code：
> dataFrame.write
> .format("org.apache.phoenix.spark")
> .option("table", table)
> .option("zkUrl", zkUrl)
> .option("skipNormalizingIdentifier", true)
> .save()
> }
>Reporter: 何军红
>Priority: Minor
> Attachments: image-2020-11-19-17-17-43-669.png
>
>
> org.apache.phoenix.spark.DefaultSource
> Savemode will not report errors unless it is set to overwrite，But the message
> to throw an exception is that overwrite is not supported
> think it might be written like this,The logic is the opposite
> if (mode.equals(SaveMode.Overwrite))
> { throw new Exception("SaveMode other than SaveMode.OverWrite is not
> supported") }
> !image-2020-11-19-17-17-43-669.png|width=648,height=149!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (PHOENIX-6352) Add doucmentation about HBase release support policy to webpage
[ https://issues.apache.org/jira/browse/PHOENIX-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved PHOENIX-6352. -- Resolution: Won't Do We could not agree on doing this. > Add doucmentation about HBase release support policy to webpage > --- > > Key: PHOENIX-6352 > URL: https://issues.apache.org/jira/browse/PHOENIX-6352 > Project: Phoenix > Issue Type: Task >Reporter: Istvan Toth >Priority: Major > > One we have reached consensus on the HBase release support policy, document > it on the web page. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Resolved] (PHOENIX-6354) Update to spark 3.0
[ https://issues.apache.org/jira/browse/PHOENIX-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth resolved PHOENIX-6354. -- Resolution: Duplicate PHOENIX-6632 includes a PR for the same issue. > Update to spark 3.0 > --- > > Key: PHOENIX-6354 > URL: https://issues.apache.org/jira/browse/PHOENIX-6354 > Project: Phoenix > Issue Type: Improvement > Components: spark-connector >Reporter: Alejandro Anadon >Priority: Major > > I am trying to use the phoenix spark connector with spark 3.0. > I tried to compile from trunk; but it is still configured with 2.4.0. > Is is not as easy (I am naive) as change the pon.xml from : > 2.4.0 > to > 3.0.1 > > I found that in class : > org.apache.phoenix.spark.datasource.v2.PhoenixDataSource is still using the > old DataSourcesV2 API; and I don't have the knowledge for updating it in a > brand. > > > -- This message was sent by Atlassian Jira (v8.20.1#820001)
