That inputformat is a batch one, so there's no state backend. You need to
output the fetched data somewhere AFAIK
On Thu, Dec 6, 2018 at 3:49 PM miki haiat <miko5...@gmail.com> wrote:
> Hi Flavio ,
> That working fine for and im able to pull ~17m rows in 20 seconds.
>
> Im a bit confuse regarding the state backhand ,
> I could find a way to configure it so im guessing the data is in the
> memory ...
>
> thanks,
> Miki
>
>
>
> On Thu, Dec 6, 2018 at 12:06 PM Flavio Pompermaier <pomperma...@okkam.it>
> wrote:
>
>> the constructor of NumericBetweenParametersProvider takes 3 params: long
>> fetchSize, long minVal, long maxVal.
>> If you want parallelism you should use a 1 < fetchSize < maxVal.
>> In your case, if you do new NumericBetweenParametersProvider(50, 3, 300)
>> you will produce 6 parallel tasks:
>>
>> 1. SELECT .... BETWEEN 3 and 50
>> 2. SELECT .... BETWEEN 51 and 100
>> 3. SELECT .... BETWEEN 101 and 150
>> 4. SELECT .... BETWEEN 151 and 200
>> 5. SELECT .... BETWEEN 201 and 250
>> 6. SELECT .... BETWEEN 251 and 300
>>
>>
>> On Thu, Dec 6, 2018 at 10:32 AM miki haiat <miko5...@gmail.com> wrote:
>>
>>> hi Flavio ,
>>>
>>> This is the query that im trying to coordinate
>>>
>>>> .setQuery("SELECT a, b, c, \n" +
>>>> "FROM dbx.dbo.x as tls\n"+
>>>> "WHERE tls.a BETWEEN ? and ?"
>>>>
>>>> And this is the way im trying to parameterized
>>>
>>> ParameterValuesProvider pramProvider = new
>>> NumericBetweenParametersProvider(10000, 3,300);
>>>
>>> I also tried this way
>>>
>>> Serializable[][] queryParameters = new String[1][2];
>>> queryParameters[0] = new String[]{"3","300"};
>>>
>>>
>>> On Wed, Dec 5, 2018 at 6:44 PM Flavio Pompermaier <pomperma...@okkam.it>
>>> wrote:
>>>
>>>> whats your query? Have you used '?' where query should be parameterized?
>>>>
>>>> Give a look at
>>>> https://github.com/apache/flink/blob/master/flink-connectors/flink-jdbc/src/test/java/org/apache/flink/api/java/io/jdbc/JDBCFullTest.java
>>>>
>>>
>>
