Use an intermediate work table to put json data streaming in there in the first place and then according to the tag store the data in the correct table
HTH Mich Talebzadeh, Dad | Technologist | Solutions Architect | Engineer London United Kingdom view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Wed, 10 Jan 2024 at 18:51, PRASHANT L <prashant...@gmail.com> wrote: > Hi > I have a use case where I need to process json payloads coming from Kafka > using structured streaming , but thing is json can have different formats , > schema is not fixed > and each json will have a @type tag so based on tag , json has to be > parsed and loaded to table with tag name , and if a json has nested sub > tags , those tags shd go to different table > so I need to process each json record individually , and determine > destination tables what would be the best approach > > >> *{* >> * "os": "andriod",* >> * "type": "mobile",* >> * "device": {* >> * "warrenty": "3 years",* >> * "replace": "yes"* >> * },* >> * "zones": [* >> * {* >> * "city": "Bangalore",* >> * "state": "KA",* >> * "pin": "577401"* >> * },* >> * {* >> * "city": "Mumbai",* >> * "state": "MH",* >> * "pin": "576003"* >> * }* >> * ],* >> * "@table": "product"**}* > > > so for the above json , there are 3 tables created > 1. Product (@type) THis is a parent table > 2. poduct_zones and product_devices , child table >
