How do you access a cached Spark SQL Table from a JBDC connection?
Hello there, I have a JBDC connection setup to my Spark cluster but I cannot see the tables that I cache in memory. The only tables I can see are those that are in my Hive instance. I use a HiveContext to register a table and cache it in memory. How can I enable my JBDC connection to query this in memory table? Brandon
RE: How do you access a cached Spark SQL Table from a JBDC connection?
Can you describe how did you cache the tables? In another HiveContext? AFAIK, cached table only be visible within the same HiveContext, you probably need to execute the sql query like “cache table mytable as SELECT xxx” in the JDBC connection also. Cheng Hao From: Brandon White [mailto:bwwintheho...@gmail.com] Sent: Wednesday, July 15, 2015 8:26 AM To: user Subject: How do you access a cached Spark SQL Table from a JBDC connection? Hello there, I have a JBDC connection setup to my Spark cluster but I cannot see the tables that I cache in memory. The only tables I can see are those that are in my Hive instance. I use a HiveContext to register a table and cache it in memory. How can I enable my JBDC connection to query this in memory table? Brandon
Re: How do you access a cached Spark SQL Table from a JBDC connection?
I cache the tell with hiveContext.cacheTable("tableName") On Tue, Jul 14, 2015 at 5:43 PM, Cheng, Hao wrote: > Can you describe how did you cache the tables? In another HiveContext? > AFAIK, cached table only be visible within the same HiveContext, you > probably need to execute the sql query like > > “cache table mytable as SELECT xxx” in the JDBC connection also. > > > > Cheng Hao > > > > *From:* Brandon White [mailto:bwwintheho...@gmail.com] > *Sent:* Wednesday, July 15, 2015 8:26 AM > *To:* user > *Subject:* How do you access a cached Spark SQL Table from a JBDC > connection? > > > > Hello there, > > > > I have a JBDC connection setup to my Spark cluster but I cannot see the > tables that I cache in memory. The only tables I can see are those that are > in my Hive instance. I use a HiveContext to register a table and cache it > in memory. How can I enable my JBDC connection to query this in memory > table? > > > > Brandon >
RE: How do you access a cached Spark SQL Table from a JBDC connection?
So you’re with different HiveContext instances for the caching. We are not expected to see the cached tables cached with the other HiveContext instance. From: Brandon White [mailto:bwwintheho...@gmail.com] Sent: Wednesday, July 15, 2015 8:48 AM To: Cheng, Hao Cc: user Subject: Re: How do you access a cached Spark SQL Table from a JBDC connection? I cache the tell with hiveContext.cacheTable("tableName") On Tue, Jul 14, 2015 at 5:43 PM, Cheng, Hao mailto:hao.ch...@intel.com>> wrote: Can you describe how did you cache the tables? In another HiveContext? AFAIK, cached table only be visible within the same HiveContext, you probably need to execute the sql query like “cache table mytable as SELECT xxx” in the JDBC connection also. Cheng Hao From: Brandon White [mailto:bwwintheho...@gmail.com<mailto:bwwintheho...@gmail.com>] Sent: Wednesday, July 15, 2015 8:26 AM To: user Subject: How do you access a cached Spark SQL Table from a JBDC connection? Hello there, I have a JBDC connection setup to my Spark cluster but I cannot see the tables that I cache in memory. The only tables I can see are those that are in my Hive instance. I use a HiveContext to register a table and cache it in memory. How can I enable my JBDC connection to query this in memory table? Brandon