Re: apache-spark mongodb dataframe issue

2020-06-26 Thread Mannat Singh
Hi Jeff Thanks for confirming the same. I have also thought about reading every MongoDB document separately along with their schemas and then comparing them to the schemas of all the documents in the collection. For our huge database this is a horrible horrible approach as you have already

Re: apache-spark mongodb dataframe issue

2020-06-23 Thread Jeff Evans
As far as I know, in general, there isn't a way to distinguish explicit null values from missing ones. (Someone please correct me if I'm wrong, since I would love to be able to do this for my own reasons). If you really must do it, and don't care about performance at all (since it will be

apache-spark mongodb dataframe issue

2020-06-23 Thread Harmanat Singh
Hi Please look at my issue from the link below. https://stackoverflow.com/questions/62526118/how-to-differentiate-between-null-and-missing-mongogdb-values-in-a-spark-datafra Kindly Help Best Mannat