Temporary tables are created in temp file space within the session. Once the session is closed then the temporary table goes
scala> rs.registerTempTable("mytemp") And this is the temporary file created with the above command drwx------ - hduser supergroup 0 2016-03-11 17:09 /tmp/hive/hduser/a1a80c24-213a-41e0-bbcb-75faf38597c3 drwx------ - hduser supergroup 0 2016-03-11 17:09 /tmp/hive/hduser/a1a80c24-213a-41e0-bbcb-75faf38597c3/_tmp_space.db Now if I open a second session, run the same query and create another temporary table as before scala> rs.registerTempTable("mytemp") I would get drwx------ - hduser supergroup 0 2016-03-11 17:22 /tmp/hive/hduser/cfc7f73c-cfc9-4cc0-a09d-823b07051cbd drwx------ - hduser supergroup 0 2016-03-11 17:22 /tmp/hive/hduser/cfc7f73c-cfc9-4cc0-a09d-823b07051cbd/_tmp_space.db These two tables reference different files so they never collide and they are private to the session. If you are familiar with #tables in Sybase or SQL Server, these temporary tables in Spark play the same role. They are valid within that session and private to that session HTH Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 11 March 2016 at 15:42, ram kumar <ramkumarro...@gmail.com> wrote: > No, I am not aware of it. > > Can you provide me with the details regarding this. > > Thanks > > On Fri, Mar 11, 2016 at 8:25 PM, Ted Yu <yuzhih...@gmail.com> wrote: > >> temporary tables are associated with SessionState which is used >> by SQLContext. >> >> Did you keep the session ? >> >> Cheers >> >> On Fri, Mar 11, 2016 at 5:02 AM, ram kumar <ramkumarro...@gmail.com> >> wrote: >> >>> Hi, >>> >>> I registered a dataframe as a table using registerTempTable >>> and I didn't close the Spark context. >>> >>> Will the table be available for longer time? >>> >>> Thanks >>> >> >> >