I notice JSON objects are all parsed as Map[String,Any] in Jackson but for some reason, the "inferSchema" tools in Spark SQL extracts the schema of nested JSON objects as StructTypes.
This makes it really confusing when trying to rectify the object hierarchy when I have maps because the Catalyst conversion layer underneath is expecting a Row or Product and not a Map. Why wasn't MapType used here? Is there any significant difference between the two of these types that would cause me not to use a MapType when I'm constructing my own schema representing a set of nested Map[String,_]'s?