Re: optimize hive query to move a subset of data from one partition table to another table

2018-02-11 Thread Richard Qiao
Would you mind share your code with us to analyze? > On Feb 10, 2018, at 10:18 AM, amit kumar singh wrote: > > Hi Team, > > We have hive external table which has 50 tb of data partitioned on year > month day > > i want to move last 2 month of data into another table >

Re: Apache Spark - Structured Streaming Query Status - field descriptions

2018-02-11 Thread Richard Qiao
Can find a good source for documents, but the source code “org.apache.spark.sql.execution.streaming.ProgressReporter” is helpful to answer some of them. For example: inputRowsPerSecond = numRecords / inputTimeSec, processedRowsPerSecond = numRecords / processingTimeSec This is explaining

Re: Run jobs in parallel in standalone mode

2018-01-16 Thread Richard Qiao
> Do you have any opinion for the solution. I really appreciate > > > > Onur EKİNCİ > Bilgi Yönetimi Yöneticisi > Knowledge Management Manager > > m:+90 553 044 2341 d:+90 212 329 7000 > > İTÜ Ayazağa Kampüsü, Teknokent ARI4 Binası 34469 Maslak İs

Re: Run jobs in parallel in standalone mode

2018-01-16 Thread Richard Qiao
Curious you are using"jdbc:sqlserve" to connect oracle, why? Also kindly reminder scrubbing your user id password. Sent from my iPhone > On Jan 16, 2018, at 03:00, Onur EKİNCİ wrote: > > Hi, > > We are trying to get data from an Oracle database into Kinetica database