Hi Fabio,
I have the very same requiremnt in my enviroment . My way is to collect
all the data you need to delete and save them into one hbase table and then
issue the delete statment . following is the sample code , hope this can
help.
conn=sc._jvm.DriverManager.getConnection("jdbc:phoenix:namenode:2181:/hbase-unsecure")
stmt = conn.createStatement()
stmt.execute("delete from t_table1 where exists (select 1 from
t_table1_for_delete where t_table1_for_delete .ENTITYID=t_table1 .ENTITYID
and t_table1_for_delete .CLIENTMAC=t_table1 .CLIENTMAC
and t_table1_for_delete .ETIME=t_table1 .ETIME)")
#CLIENTMAC , ENTITYID and ETIME is the primary key .
Dalin
On Thu, Sep 29, 2016 at 9:30 AM, fabio ferrante <[email protected]>
wrote:
> Hi Josh,
> using a regular DELETE is not a bulk delete. Bulk operation use jdbc
> statement batch, collecting up to 1000 statements into a single execution
> reducing the amount of communication overhead and improving performance.
> What i really want to do is add a deleteFromPhoenix() method into
> ProductRDD to simplify the scenario where an RDD doesn't contains new data
> to save into but old data to delete from Phoenix.
>
> FF.
>
> ------------------------------
> *Da:* Josh Mahonin [mailto:[email protected]]
> *Inviato:* mercoledì 28 settembre 2016 20:29
> *A:* [email protected]
> *Oggetto:* Re: bulk-delete spark phoenix
>
> Hi Fabio,
>
> You could probably just execute a regular DELETE query from a JDBC call,
> which is generally safe to do either from the Spark driver or within an
> executor. As long as auto-commit is enabled, it's an entirely server side
> operation: https://phoenix.apache.org/language/#delete
>
> Josh
>
> On Wed, Sep 28, 2016 at 2:13 PM, fabio ferrante <[email protected]>
> wrote:
>
>> Hi,
>>
>> I would like to perform a bulk delete to HBase using Apache Phoenix from
>> Spark. Using Phoenix-Spark plugin i can successfully perform a bulk load
>> using saveToPhoenix method from PhoenixRDD but how i can perform a bulk
>> delete? There isn't a deleteFromPhoenix method in PhoenixRDD. Is that
>> correct? Implement such method is a trivial task?
>>
>> Thanks in advance,
>> Fabio.
>>
>
>
