Re: Accessing Teradata DW data from Spark

2020-06-13 Thread Gourav Sengupta
Hi, Partitioning works in teradata, but your user may have core and memory restrictions. So please do adjust the number of queries hitting parallel to teradata based on partitions used in your query. I am able to extract data to S3 in 3 hours from on premise teradata which from teradata export

Accessing Teradata DW data from Spark

2020-06-10 Thread Mich Talebzadeh
Using JDBC drivers much like accessing Oracle data, one can utilise the power of Spark on Teradata via JDBC drivers. I have seen connections in some articles which indicates this process is pretty mature. My question is if anyone has done this work and how is performance in Spark vis-a-vis