Thanks for response. I want to figure out is there a way to share data without writing it to disk because of performance issues.
Chenyang Zhang Software Engineering Intern, Platform Redwood City, California [cid:EnterpriseAI_Banner_1200_e6f8b810-93f3-44f1-b795-bb502b7a52ae.png]<https://c3.ai/?utm_source=signature&utm_campaign=enterpriseai> © 2022 C3.ai. Confidential Information. On Jun 24, 2022, at 11:15 AM, Sean Owen <sro...@gmail.com<mailto:sro...@gmail.com>> wrote: EXTERNAL EMAIL: This email originated outside of our organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Spark is decoupled from storage. You can write data to any storage you like. Anything that can read that data, can read that data - Spark or not, different session or not. Temp views are specific to a session and do not store data. I think this is trivial and no problem at all, or else I'm not clear what you're asking. On Fri, Jun 24, 2022 at 1:03 PM Chenyang Zhang <chenyang.zh...@c3.ai<mailto:chenyang.zh...@c3.ai>> wrote: Hi, Thanks so much for the response from https://issues.apache.org/jira/browse/SPARK-39549<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSPARK-39549&data=05%7C01%7CChenyang.Zhang%40c3.ai%7C16869542c92f4af9188708da560d9886%7C53ad779a93e7485cba20ac8290d7252b%7C1%7C0%7C637916913636880755%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=sSW7Pi2vB9NTks0rCvRIZdO4ubJELlprARyXBO60upc%3D&reserved=0>. I am curious what do you mean by write down to a table and read it from a different Spark Application. Do you mean a table in a database or the Spark to_table() api? Could I read the table created in different Spark Session through to_table()? Is there any api I could use to achieve my objective? I noticed there are some apis, e.g. createGlobalTempView. Thanks so much for your help. Bets regards, Chenyang Chenyang Zhang Software Engineering Intern, Platform Redwood City, California <EnterpriseAI_Banner_1200_e6f8b810-93f3-44f1-b795-bb502b7a52ae.png><https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fc3.ai%2F%3Futm_source%3Dsignature%26utm_campaign%3Denterpriseai&data=05%7C01%7CChenyang.Zhang%40c3.ai%7C16869542c92f4af9188708da560d9886%7C53ad779a93e7485cba20ac8290d7252b%7C1%7C0%7C637916913636880755%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=tACD7UJOVFHeyQhjH81dx2nJ5mo%2BdgaT%2F09YoqhCXyE%3D&reserved=0> © 2022 C3.ai. Confidential Information.
