You do not need Hadoop. However, you should think about using it. If you use Spark to load data directly from Oracle then your database might have unexpected loads of data once a Spark node may fail. Additionally, the Oracle Database, if it is not based on local disk, may have a storage bottleneck. Furthermore, Spark standalone has no resource management mechanism for supporting different slas, you may need yarn (hadoop) for that. Finally, using the Oracle Database for storing all the data may be an expensive exercise. What I have seen often is that hadoop is used for storing all the data and managing the resources. Spark can be used for machine learning over this data and the Oracle Database (or any relational datastore, Nosql database, in-memory db) is used to serve the data to a lot of users. This is also the basic idea behind the lambda architecture.
Le mar. 22 sept. 2015 à 7:13, Sri <sriesh.subb...@gmail.com> a écrit : > Hi, > > We have a usecase where we get the dated from different systems and > finally > data will be consolidated into Oracle Database. Does spark is a valid > useless for this scenario. Currently we also don't have any big data > component. In case if we go with Spark to ingest data, does it require > hadoop. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Ingestion-into-Relational-DB-tp24761.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
