Re: spark - local question

2022-10-31 Thread Sean Owen
Sure, as stable and available as your machine is. If you don't need fault
tolerance or scale beyond one machine, sure.

On Mon, Oct 31, 2022 at 8:43 AM 张健BJ  wrote:

> Dear developers:
> I have a question about  the pyspark local
> mode. Can it be used in production and Will it cause unexpected problems?
> The scenario is as follows:
>
> Our team wants to develop an etl component based on python language. Data can 
> be transferred between various data sources.
>
> If there is no yarn environment, can we read data from Database A and write 
> it to Database B in local mode.Will this function be guaranteed to be stable 
> and available?
>
>
>
> Thanks,
> Look forward to your reply
>


spark - local question

2022-10-31 Thread 张健BJ
Dear developers:
 I have a question about the pyspark local mode. Can it be used in production 
and Will it cause unexpected problems? The scenario is as follows: 
Our team wants to develop an etl component based on python language. Data can 
be transferred between various data sources. 
If there is no yarn environment, can we read data from Database A and write it 
to Database B in local mode.Will this function be guaranteed to be stable and 
available?
Thanks,
Look forward to your reply