[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:29 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" } , { "name": "field2", "type": "int" } ] } Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"} , {"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"} , {"name": "nested1", "type": "string"} , {"name": "nested2", "type": "string"} ] } } ] } What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "nested1", "type": "string"}, {"name": "nested2", "type": "string"} ] } } ] } What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:34 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2", "type": "int" } ] } Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \{"name": "attr.attribute", "type": "string"}, \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \{"name": "attr.attribute", "type": "string"}, \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" } , { "name": "field2", "type": "int" } ] } Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"} , {"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"} , {"name": "nested1", "type": "string"} , {"name": "nested2", "type": "string"} ] } } ] } What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:36 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2", "type": "int" } ] } Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2", "type": "int" } ] } Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \{"name": "attr.attribute", "type": "string"}, \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \{"name": "attr.attribute", "type": "string"}, \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:37 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition ```json { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2", "type": "int" } ] } ``` Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2", "type": "int" } ] } Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:38 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition ```json { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2", "type": "int" } ] } ``` Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:39 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:38 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition content 123 Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ \{ "name": "field1", "type": "string" }, \{ "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:42 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition {code} some text some nested text some other nested text {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ { "name": "attr.attribute", "type": "string" }, { "name": "content_field", "type": "string" } ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ { "name": "attr.attribute", "type": "string" }, { "name": "nested1", "type": "string" }, { "name": "nested2", "type": "string" } ] } } ] } {code} What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition some text some nested text some other nested text Schema definition { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ \\{"name": "attr.attribute", "type": "string"} , \{"name": "nested1", "type": "string"}, \{"name": "nested2", "type": "string"} ] } } ] } What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:43 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition {code} some text some nested text some other nested text {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "nested1", "type": "string"}, {"name": "nested2", "type": "string"} ] } } ] } {code} What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition {code} some text some nested text some other nested text {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ { "name": "attr.attribute", "type": "string" }, { "name": "content_field", "type": "string" } ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ { "name": "attr.attribute", "type": "string" }, { "name": "nested1", "type": "string" }, { "name": "nested2", "type": "string" } ] } } ] } {code} What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- T
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:45 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition {code} some text some nested text some other nested text {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "nested1", "type": "string"}, {"name": "nested2", "type": "string"} ] } } ] } {code} What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition {code} some text some nested text some other nested text {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "nested1", "type": "string"}, {"name": "nested2", "type": "string"} ] } } ] } {code} What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394487#comment-16394487 ] Johannes Peter edited comment on NIFI-4185 at 3/11/18 12:47 PM: [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record, e. g. {code} content ... {code} or an array of records, e. g. {code} content ... ... {code} The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition {code} some text some nested text some other nested text {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "nested1", "type": "string"}, {"name": "nested2", "type": "string"} ] } } ] } {code} What do you say? was (Author: jope): [~alopresto]: Started implementing an XML Record Reader. Shall I create a separate ticket for this? Similar to the JSON readers, the XML reader will expect either a single record (e. g. content ... ) or an array of records (e. g. content ... ... ) The reader will be aligned with common transformators. "Normal" fields (e. g. String, Integer) can be described by simple key-value pairs: XML definition {code} content 123 {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": "string" }, { "name": "field2", "type": "int" } ] } {code} Parsing of attributes or nested fields require the definition of nested records and a field name for the content (optional, a prefix for attributes can be defined): Property: CONTENT_FIELD=content_field Property: ATTRIBUTE_PREFIX=attr. XML definition {code} some text some nested text some other nested text {code} Schema definition {code} { "name": "testschema", "namespace": "nifi", "type": "record", "fields": [ { "name": "field1", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "content_field", "type": "string"} ] } }, { "name": "field2", "type": { "name": "NestedRecord", "type": "record", "fields" : [ {"name": "attr.attribute", "type": "string"}, {"name": "nested1", "type": "string"}, {"name": "nested2", "type": "string"} ] } } ] } {code} What do you say? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402528#comment-16402528 ] Pierre Villard edited comment on NIFI-4185 at 3/16/18 9:03 PM: --- Hi [~jope], I'm really interested by this work and will be happy to have a look when you submit a PR. A quick question: are you going to require an XSD schema in your reader? or will you infer the schema by reading the input XML data? (XSD would be cool because it allows much more specifications than an Avro schema) I'm asking because one of the issue if you don't have an XSD is to make a difference between an array of 1 record (that could be translated into a single record) and an array of multiple records. Example: I want the two below inputs {code:java} content {code} and {code:java} content content {code} to be converted with the same output schema having an array of records. Does it make sense? was (Author: pvillard): Hi [~jope], I'm really interested by this work and will be happy to have a look when you submit a PR. A quick question: are you going to require an XSD schema in your reader? or will you infer the schema by reading the input XML data? (XSD would be cool because it allows much more specifications than an Avro schema) I'm asking because one of the issue if you don't have an XSD is to make a difference between an array of 1 record (that could be translated into a single record) and an array of multiple records. Example: I want the two below inputs content and content content to be converted with the same output schema having an array of records. Does it make sense? > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Assignee: Johannes Peter >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403399#comment-16403399 ] Johannes Peter edited comment on NIFI-4185 at 3/17/18 12:05 PM: Hi [~pvillard], for this reader I have not planned to require an XSD schema. My intention is that it can be configured in the same way like readers of other formats. I therefore translate Avro definitions to XML structures that are expected by the reader. Generally, the reader expects an array containing zero, one or more records. I use StAX as its pulling logic suits well to the record-lookup requirement. BTW: Do you have an idea which XML structure the reader could expect when users define a map in their schema? Maybe something like this? {code} content {content or object} ... {code} was (Author: jope): Hi [~pvillard], for this reader I have not planned to require an XSD schema. My intention is that it can be configured in the same way like readers of other formats. I therefore translate Avro definitions to XML structures that are expected by the reader. Generally, the reader expects an array containing zero, one or more records. I use StAX as its pulling logic suits well to the record-lookup requirement. BTW: Do you have an idea which XML structure the reader could expect when users define a map in their schema? Maybe something like this? {code} content content or object ... {code} > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Assignee: Johannes Peter >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403399#comment-16403399 ] Johannes Peter edited comment on NIFI-4185 at 3/17/18 12:07 PM: Hi [~pvillard], for this reader I have not planned to require an XSD schema. My intention is that it can be configured in the same way like readers of other formats. I therefore translate Avro definitions to XML structures that are expected by the reader. Generally, the reader expects an array containing zero, one or more records. I use StAX as its pulling logic suits well to the record-lookup requirement. BTW: Do you have an idea which XML structure the reader could expect when users define a map in their schema? Maybe something like this? {code} content {content or object} ... ... {code} was (Author: jope): Hi [~pvillard], for this reader I have not planned to require an XSD schema. My intention is that it can be configured in the same way like readers of other formats. I therefore translate Avro definitions to XML structures that are expected by the reader. Generally, the reader expects an array containing zero, one or more records. I use StAX as its pulling logic suits well to the record-lookup requirement. BTW: Do you have an idea which XML structure the reader could expect when users define a map in their schema? Maybe something like this? {code} content {content or object} ... ... {code} > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Assignee: Johannes Peter >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4185) Add XML record reader & writer services
[ https://issues.apache.org/jira/browse/NIFI-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16403399#comment-16403399 ] Johannes Peter edited comment on NIFI-4185 at 3/17/18 12:07 PM: Hi [~pvillard], for this reader I have not planned to require an XSD schema. My intention is that it can be configured in the same way like readers of other formats. I therefore translate Avro definitions to XML structures that are expected by the reader. Generally, the reader expects an array containing zero, one or more records. I use StAX as its pulling logic suits well to the record-lookup requirement. BTW: Do you have an idea which XML structure the reader could expect when users define a map in their schema? Maybe something like this? {code} content {content or object} ... ... {code} was (Author: jope): Hi [~pvillard], for this reader I have not planned to require an XSD schema. My intention is that it can be configured in the same way like readers of other formats. I therefore translate Avro definitions to XML structures that are expected by the reader. Generally, the reader expects an array containing zero, one or more records. I use StAX as its pulling logic suits well to the record-lookup requirement. BTW: Do you have an idea which XML structure the reader could expect when users define a map in their schema? Maybe something like this? {code} content {content or object} ... {code} > Add XML record reader & writer services > --- > > Key: NIFI-4185 > URL: https://issues.apache.org/jira/browse/NIFI-4185 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Assignee: Johannes Peter >Priority: Major > Labels: json, records, xml > > With the addition of the {{RecordReader}} and {{RecordSetWriter}} paradigm, > XML conversion has not yet been targeted. This will replace the previous > ticket for XML to JSON conversion. -- This message was sent by Atlassian JIRA (v7.6.3#76005)