suyash yadav created CARBONDATA-4207:
----------------------------------------
Summary: MV data getting lost
Key: CARBONDATA-4207
URL: https://issues.apache.org/jira/browse/CARBONDATA-4207
Project: CarbonData
Issue Type: Improvement
Components: core
Affects Versions: 2.0.1
Reporter: suyash yadav
Fix For: 2.0.1
Hi Team,
We have observed one more issue, We had created one table and a timeseries MV
on it. We had loaded almost 15 hours of data into it and then when we were
loading 16th hour data the loading failed because of some reason but it caused
MV to go empty. Our mv has now zero rows. Could you please let us know if there
is any bug or this is how it is supposed to work. Because our MV did not have
any avg function so ideally the loading to MV should have been incremental ,
and in that case MV should not have got impacted if the subsequent hour loading
to main table failed. Please have a look into this issue. And let us know what
information you need.
scala> spark.sql("insert into Flow_TS_2day_stats_04062021 select
start_time,end_time,source_ip_address,destintion_ip_address,appname,protocol_id,source_tos,src_as,dst_as,source_mask,destination_mask,dst_tos,input_pkt,output_pkt,input_byt,output_byt,source_port,destination_port,in_interface,out_interface
from Flow_TS_1day_stats_24052021 where start_time>='2021-03-04 07:00:00' and
start_time< '2021-03-04 09:00:00'").show()
[1:38|https://carbondataworkspace.slack.com/archives/D01GLHKSAFL/p1623226096008700]
scala> spark.sql("insert into Flow_TS_2day_stats_04062021 select
start_time,end_time,source_ip_address,destintion_ip_address,appname,protocol_id,source_tos,src_as,dst_as,source_mask,destination_mask,dst_tos,input_pkt,output_pkt,input_byt,output_byt,source_port,destination_port,in_interface,out_interface
from Flow_TS_1day_stats_24052021 where start_time>='2021-03-04 15:00:00' and
start_time< '2021-03-04 16:00:00'").show()
21/06/06 14:25:33 AUDIT audit: \{"time":"June 6, 2021 2:25:33 PM
IST","username":"root","opName":"INSERT
INTO","opId":"4069819623887063","opStatus":"START"}
21/06/06 14:44:14 AUDIT audit: \{"time":"June 6, 2021 2:44:14 PM
IST","username":"root","opName":"INSERT
INTO","opId":"4070940294400824","opStatus":"START"}
21/06/06 16:06:05 AUDIT audit: \{"time":"June 6, 2021 4:06:05 PM
IST","username":"root","opName":"INSERT
INTO","opId":"4070940294400824","opStatus":"SUCCESS","opTime":"4911240
ms","table":"default.Interface_Level_Agg_10min_MV_04062021","extraInfo":{"SegmentId":"6","DataSize":"4.52GB","IndexSize":"108.27KB"}}
21/06/06 16:06:09 AUDIT audit: \{"time":"June 6, 2021 4:06:09 PM
IST","username":"root","opName":"INSERT
INTO","opId":"4069819623887063","opStatus":"SUCCESS","opTime":"6036073
ms","table":"default.flow_ts_2day_stats_04062021","extraInfo":{"SegmentId":"6","DataSize":"12.37GB","IndexSize":"262.43KB"}}[^Stack_Trace]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
