[GitHub] [hudi] git-raj commented on issue #2284: [SUPPORT] : Is there a option to achieve SCD 2 in Hudi?

2021-01-25 Thread GitBox 


git-raj commented on issue #2284:
URL: https://github.com/apache/hudi/issues/2284#issuecomment-766523668


   using AWS Glue pySpark and Hudi and S3 as data store: i'm trying to do the 
traditional SCD Type 2 where old record gets updated with the insert datetime 
on 'effective to' field, 'isActive' field becomes 'false', and new row is 
inserted with the insert datetime in 'effective from' field with 'isActive' 
becoming 'true'. Any solution post, or pointers to solve that if possible is 
highly appreciated.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [hudi] git-raj commented on issue #2284: [SUPPORT] : Is there a option to achieve SCD 2 in Hudi?

2021-01-24 Thread GitBox 


git-raj commented on issue #2284:
URL: https://github.com/apache/hudi/issues/2284#issuecomment-766523668


   using AWS Glue pySpark and Hudi and S3 as data store: i'm trying to do the 
traditional SCD Type 2 where old record gets updated with the insert datetime 
on 'effective to' field, 'isActive' field becomes 'false', and new row is 
inserted with the insert datetime in 'effective from' field with 'isActive' 
becoming 'true'. Any solution post, or pointers to solve that if possible is 
highly appreciated.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org