[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2021-12-07 Thread Manoj Govindassamy (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manoj Govindassamy updated HUDI-2954:
-
Priority: Minor  (was: Major)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Task
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
>Priority: Minor
> Fix For: 0.11.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-03-12 Thread Raymond Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raymond Xu updated HUDI-2954:
-
Component/s: metadata

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Task
>  Components: metadata
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
>Priority: Minor
> Fix For: 0.11.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-03-12 Thread Raymond Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raymond Xu updated HUDI-2954:
-
Epic Link: HUDI-1292

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Task
>Reporter: Manoj Govindassamy
>Assignee: Manoj Govindassamy
>Priority: Minor
> Fix For: 0.11.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-03-27 Thread Raymond Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raymond Xu updated HUDI-2954:
-
Component/s: code-quality

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Task
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Minor
> Fix For: 0.11.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-03-27 Thread Raymond Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raymond Xu updated HUDI-2954:
-
Issue Type: Improvement  (was: Task)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Major
> Fix For: 0.12.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-03-27 Thread Raymond Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raymond Xu updated HUDI-2954:
-
Fix Version/s: 0.12.0
   (was: 0.11.0)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Task
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Minor
> Fix For: 0.12.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-03-27 Thread Raymond Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raymond Xu updated HUDI-2954:
-
Priority: Major  (was: Minor)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Task
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Major
> Fix For: 0.12.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-04-18 Thread Ethan Guo (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Guo updated HUDI-2954:

Priority: Blocker  (was: Major)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Blocker
> Fix For: 0.12.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-04-18 Thread Raymond Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raymond Xu updated HUDI-2954:
-
Sprint: Hudi-Sprint-Apr-19

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Blocker
> Fix For: 0.12.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-04-19 Thread Raymond Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raymond Xu updated HUDI-2954:
-
Story Points: 1

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Blocker
> Fix For: 0.12.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-04-19 Thread Ethan Guo (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Guo updated HUDI-2954:

Priority: Minor  (was: Blocker)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Minor
> Fix For: 0.12.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-04-20 Thread Ethan Guo (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Guo updated HUDI-2954:

Sprint:   (was: Hudi-Sprint-Apr-19)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Minor
> Fix For: 0.12.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-04-20 Thread Ethan Guo (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Guo updated HUDI-2954:

Fix Version/s: 0.12.0

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Minor
> Fix For: 0.12.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-04-20 Thread Ethan Guo (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Guo updated HUDI-2954:

Fix Version/s: (was: 0.12.0)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Minor
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2022-08-16 Thread Sagar Sumit (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sagar Sumit updated HUDI-2954:
--
Fix Version/s: 0.12.1
   (was: 0.12.0)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Minor
> Fix For: 0.12.1
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2023-03-09 Thread Raymond Xu (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Raymond Xu updated HUDI-2954:
-
Fix Version/s: 0.14.0
   (was: 0.13.1)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Improvement
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Minor
> Fix For: 0.14.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2023-10-04 Thread Prashant Wason (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prashant Wason updated HUDI-2954:
-
Fix Version/s: 0.14.1
   (was: 1.0.0)
   (was: 0.14.0)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Wish
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Priority: Minor
> Fix For: 0.14.1
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (HUDI-2954) Code cleanup: HFileDataBock - using integer keys is never used

2023-07-05 Thread Vinoth Chandar (Jira) 


 [ 
https://issues.apache.org/jira/browse/HUDI-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinoth Chandar updated HUDI-2954:
-
Epic Link:   (was: HUDI-1292)
Fix Version/s: 1.0.0
   Issue Type: Wish  (was: Improvement)

> Code cleanup: HFileDataBock - using integer keys is never used 
> ---
>
> Key: HUDI-2954
> URL: https://issues.apache.org/jira/browse/HUDI-2954
> Project: Apache Hudi
>  Issue Type: Wish
>  Components: code-quality, metadata
>Reporter: Manoj Govindassamy
>Assignee: Ethan Guo
>Priority: Minor
> Fix For: 0.14.0, 1.0.0
>
>
>  
> KeyField can never be empty for File. If so, there is really no need for 
> falling back to sequential integer keys in the 
> HFileDataBlock::serializeRecords() code path.
>  
> {noformat}
> // Build the record key
> final Field schemaKeyField = 
> records.get(0).getSchema().getField(this.keyField);
> if (schemaKeyField == null) {
>   // Missing key metadata field. Use an integer sequence key instead.
>   useIntegerKey = true;
>   keySize = (int) Math.ceil(Math.log(records.size())) + 1;
> }
> while (itr.hasNext()) {
>   IndexedRecord record = itr.next();
>   String recordKey;
>   if (useIntegerKey) {
> recordKey = String.format("%" + keySize + "s", key++);
>   } else {
> recordKey = record.get(schemaKeyField.pos()).toString();
>   }
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)