Abdullah,
This is the feed configuration that I am using:
drop dataverse experiments if exists;
create dataverse experiments;
use dataverse experiments;
create type TwitterUserType as closed {
screen-name: string,
lang: string,
friends_count: int32,
statuses_count: int32,
name: string,
followers_count: int32
}
create type TweetMessageType as closed {
tweetid: int64,
user: TwitterUserType,
sender-location: point,
send-time: datetime,
referred-topics: {{ string }},
message-text: string
}
create dataset Tweets(TweetMessageType) primary key tweetid;
create feed TweetFeed1 using socket_adapter
(
("sockets"="127.0.0.1:10001"),
("address-type"="IP"),
("type-name"="TweetMessageType"),
("format"="adm"),
("duration"="1200")
);
set wait-for-completion-feed "false";
connect feed TweetFeed1 to dataset Tweets;
After that I start pouring tweet records via socket 127.0.0.1:10001. After the
first batch is ingested (does no matter what the size is) record generator
stops and closes socket from the client side. Then I am launching record
generator again via socket (note feed is still connected) and that’s where
ingestion stops.
You can check the modified Young-Seek’s ingestion clients which I am using
https://github.com/ty1er/asterix-statistics-experiments. I can give you tips
how to start it.
> On Feb 21, 2016, at 20:13, abdullah alamoudi <[email protected]> wrote:
>
> Ildar,
> Can you share an example I can try and work on?
>
> I have created a "test" change over the weekend that allows me to start
> different servers and test the socket/RSS feeds on them. unfortunately
> waiting for other reviews for other fixes to make it before I can add the
> new test cases.
>
> I would fix this issue and add a test case for it but I am not clear on
> what is happening vs. the expected behavior. More details would help us fix
> this.
> Thanks,
> Abdullah.
>
> On Mon, Feb 22, 2016 at 4:02 AM, Ildar Absalyamov <
> [email protected]> wrote:
>
>> Hi all,
>>
>> Whenever the socket-based feed adapter is used, what happens when the
>> socket is closed on the client side?
>> Is it supposed to continue ingestion once the connection is reestablished
>> again?
>> Did anyone tried that? From my observations when the connection is
>> established the second time, on the feed’s side socket read operation
>> blocks and further ingestion does not happen.
>>
>> Best regards,
>> Ildar
>>
>>
Best regards,
Ildar
