Hi guys Hopefully someone can help me, or at least explain stuff to me.
I use a tool that required JDBC metadata (tables/columns etc) So using spark 1.3.1 I try stuff like: registerTempTable() or saveAsTable() on my parquet file. The former doesn't show any table metadata for JDBC connections, but you can query the table, which is annoying. The latter shows a table but the column metadata is 1 column type array<string>, again I can query the table. What I found I can do though is create a standard SQL table in beeline with all its columns defined, and then insert into that table the contents of my invisible parquet table, but I assume that removes the data from parquet and stores it in hive, and I'd prefer to stick with parquet. Ideally i'd like to be able to run CREATE TEMPORARY TABLE XYZ USING org.apache.spark.sql.parquet OPTIONS ( path "/user/ubuntu/file_with_id.par" .... define my table columns ) Is something like that possible, does that make any sense? Thanks Tom
