[
https://issues.apache.org/jira/browse/HIVE-24884?focusedWorklogId=776095&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-776095
]
ASF GitHub Bot logged work on HIVE-24884:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 31/May/22 04:06
Start Date: 31/May/22 04:06
Worklog Time Spent: 10m
Work Description: hmangla98 commented on code in PR #3293:
URL: https://github.com/apache/hive/pull/3293#discussion_r885185479
##########
ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/DumpMetaData.java:
##########
@@ -35,33 +40,64 @@
import java.util.Arrays;
import java.util.List;
+@JsonIgnoreProperties(ignoreUnknown = true)
public class DumpMetaData {
// wrapper class for reading and writing metadata about a dump
// responsible for _dumpmetadata files
public static final String DUMP_METADATA = "_dumpmetadata";
+
+ // New version of dump metadata file to store top level dumpmetadata content
in JSON format
+ public static final String DUMP_METADATA_V2 = "_dumpmetadata_v2";
private static final Logger LOG =
LoggerFactory.getLogger(DumpMetaData.class);
+ private static ObjectMapper JSON_OBJECT_MAPPER = new ObjectMapper(); //
Thread-safe.
+ @JsonProperty
private DumpType dumpType;
+ @JsonProperty
private Long eventFrom = null;
+ @JsonProperty
private Long eventTo = null;
+ @JsonProperty
private Path cmRoot;
+ @JsonProperty
private String payload = null;
- private ReplScope replScope = null;
-
- private boolean initialized = false;
- private final Path dumpFile;
- private final HiveConf hiveConf;
+ @JsonProperty
private Long dumpExecutionId;
+ @JsonProperty
private boolean replScopeModified = false;
+ @JsonProperty
+ private String replScopeStr = null;
+ //Ignore rest of the properties
+ @JsonIgnore
+ private ReplScope replScope = null;
+ @JsonIgnore
+ private Path dumpFile;
+ @JsonIgnore
+ private final HiveConf hiveConf;
+ @JsonIgnore
+ private boolean isTopLevel;
+ @JsonIgnore
+ private Path dumpRoot;
+ @JsonIgnore
+ private boolean initialized = false;
+
+ public DumpMetaData() {
+ //to be instantiated by JSON ObjectMapper.
+ hiveConf = null;
+ }
public DumpMetaData(Path dumpRoot, HiveConf hiveConf) {
- this.hiveConf = hiveConf;
- dumpFile = new Path(dumpRoot, DUMP_METADATA);
+ this(dumpRoot, hiveConf, false);
Review Comment:
Reading old version is necessary for existing replication policies that
previously used tab separated dump metadata file.
Issue Time Tracking
-------------------
Worklog Id: (was: 776095)
Time Spent: 1h 40m (was: 1.5h)
> Move top level dump metadata content to be in JSON format
> ---------------------------------------------------------
>
> Key: HIVE-24884
> URL: https://issues.apache.org/jira/browse/HIVE-24884
> Project: Hive
> Issue Type: Task
> Reporter: Pravin Sinha
> Assignee: Pravin Sinha
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> {color:#172b4d}The current content for _dumpmetadata file is TAB separated.
> This is not very flexible for extension. A more flexible format like JSON
> based content would be helpful for extending the content.{color}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
