Thank you!
Hi,I should be able to look at this one in a few days when I return and resumework (I am currently on a vacation till 5th Nov).However, the exception suggests that building the secondary feed pipelineencountered an exception. Can you share the logs so that I can get a betterunderstanding of the sequence of steps that happened as your statementexecuted.I have intermittent access to net, but should be able to revert with somedelays.On Nov 2, 2015 2:32 AM, "Jianfeng Jia" <[email protected]> wrote:I’ve tried the “apply AQLfunction” idea, but got an NullPointException:
java.lang.NullPointerException
at
org.apache.asterix.aql.translator.AqlTranslator.getFeedJointKey(AqlTranslator.java:2268)
at
org.apache.asterix.aql.translator.AqlTranslator.getFeedConnectionRequest(AqlTranslator.java:2214)
at
org.apache.asterix.aql.translator.AqlTranslator.handleConnectFeedStatement(AqlTranslator.java:2130)
at
org.apache.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:362)
at
org.apache.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:114)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
at org.eclipse.jetty.server.Server.handle(Server.java:347)
at
org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439)
at
org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
at java.lang.Thread.run(Thread.java:745)
The ddl that I was using is here:
https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql
<
https://gist.githubusercontent.com/JavierJia/ca596df82ffdd456001f/raw/ea86f9ad1531a68c3ecf9036ef5b69976893149d/feed-ddl.aql
Anyone has any idea? Thank you!
On Oct 30, 2015, at 12:30 AM, Heri Ramampiaro <[email protected]> wrote:
Although I haven’t tested this (I have mostly used & created java-based
UDFs),
one can execute AQL calls from within a UDF. Feeds allows functions to
execute arbitrary AQL statements (DDL, DMLs etc). I.e., I believe what
you are trying to do is possible.
For example you could do:
(Given that you have a rec. type called Tweet, and a dataset
ProcessedTweets
create feed CoordTwitterFeed if not exists
using “push_twitter" (("type-name"="Tweet”))
apply function find-intersection;
(Here "find-intersection” is an AQL function that does the step
specified under 2)).
To do nr. 3 the easiest way is to have a modified version of the
“hashTag”
(let’s call this “tweetlocator”). Java-based UDF (see my previous
message with the
TweetLib example). You can then connect this as a secondary feed,
connected to
“CoordTwitterFeed”.
E.g.:
create secondary feed ProcessedTwitterFeed from feed CoordTwitterFeed
apply function “tweelib#tweetlocator";
Thereafter you can connect the feeds to appropriate datasets.
(Since the find-intersection is an AQL function, you can also call
“tweelib#tweetlocator”
inside this function).
Best,
-heri
On Oct 30, 2015, at 12:18 AM, Young-Seok Kim <[email protected]> wrote:
I'm not sure whether the following UDF is possible or not, but
hopefully it
is.
What we're trying to do is to have the following UDF.
The UDF
1) accepts an incoming tweet record through a feed job as an input and
then
2) takes a field, more specifically, coordinate field value from the
tweet
record and sends a spatial-intersect query using the coordinate in
order to
find out the corresponding county of the coordinates (we have created
AsterixDB instance which stores US county shapes records into a dataset
and
created R-tree index on the polygon field of it, so the query will
return
the county effectively using the R-tree index)
3) creates a new tweet record consisting of the original record's
fields +
the returned county value
4) ingests to a tweet dataset in the AsterixDB instance.
Can we have such an UDF?
Best,
Young-Seok
On Thu, Oct 29, 2015 at 3:53 PM, Heri Ramampiaro <[email protected]>
wrote:
2. Can we use AQL function in those kind of feed UDFs?
Can you give an example of what you are trying to do?
I.e do you want to run an AQL inside a UDF or use an AQL function as a
UDF connected to a running feed?
-heri
Best,
Jianfeng Jia
PhD Candidate of Computer Science
University of California, Irvine
Best,
Jianfeng Jia
PhD Candidate of Computer Science
University of California, Irvine
|
cc.logs