Its certainly something wrt Env. Can you just copy paste couple of errors ?
On Fri, Jul 24, 2015 at 2:00 PM, Ajay <[email protected]> wrote: > I did the same from command line. > > mvn clean install -DskipTests -DskipCheck > mvn idea:idea > > > Java version is 1.8 > > I tried opening the project directly and selected > /home/ajay/openSource/incubator-lens. Also I tried importing as maven > project. Both the cases I see compilation errors. > > - > Ajay > > > On Fri, Jul 24, 2015 at 1:42 PM, Rajat Khandelwal <[email protected]> > wrote: > > > I'm using idea too. There are no compilation issues. Can you do > > > > mvn clean install -DskipTests -DskipCheck > > mvn idea:idea > > > > from command line. > > > > > > Also, check your java version. Lens is configured for java >=7 > > > > On Fri, Jul 24, 2015 at 1:36 PM Ajay <[email protected]> wrote: > > > > > From console, the build goes thru fine. > > > > > > From Idea IDE, it is giving compilation errors (say one such file > > > QueryExecutionServiceImp.java > > > Lens Server project). > > > > > > Are you guys using Eclipse? > > > > > > > > > On Fri, Jul 24, 2015 at 1:14 PM, Yash Sharma <[email protected]> > wrote: > > > > > > > Could you share the error. Might be env issues. > > > > Also can you build from the console directly ? > > > > > > > > On Fri, Jul 24, 2015 at 12:54 PM, Ajay <[email protected]> wrote: > > > > > > > > > Thanks. > > > > > > > > > > I am using intelliJ Idea IDE. I checked out the source from GIT. > But > > I > > > am > > > > > seeing some compilation issues. Is there any documentation or notes > > on > > > > how > > > > > to set up the development envirnment? > > > > > > > > > > - > > > > > Ajay > > > > > > > > > > On Fri, Jul 24, 2015 at 11:44 AM, amareshwarisr . < > > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > Hello Ajay, > > > > > > > > > > > > Nice to see your interest in exploring Lens. > > > > > > > > > > > > Looking at the requirement posted by you, seems you are only > > > interested > > > > > in > > > > > > REST api on top of cassandra. See the lens REST api for query > > service > > > > > here > > > > > > - > > > http://lens.incubator.apache.org/resource_QueryServiceResource.html > > > > > > > > > > > > Lens provides REST api for query actions (submit, fetch results, > > > > cancel, > > > > > > getstatus), query history and OLAP data model. > > > > > > > > > > > > If you are not interested in OLAP data model, you can simple run > > > native > > > > > > queries on lens which would give REST api and history for you as > > > well. > > > > > For > > > > > > more details on OLAP data model, you can read the doc on website. > > > > > > > > > > > > For new driver to be rewritten, it need not know anything about > > OLAP > > > > data > > > > > > model. But all drivers are given HiveQL query to execute, if > driver > > > > > accepts > > > > > > Hive QL directly you dont have to do much on the query itself. If > > > not, > > > > > the > > > > > > driver has to translate the query into something driver can > > > understand. > > > > > > > > > > > > For example : JDBCDriver converts HiveQL to SQL. - You can have > > look > > > at > > > > > > JDBCDriver for converting into SQL. Hive QL is almost SQL, except > > it > > > > > might > > > > > > have to map some UDFs to SQL udfs. > > > > > > > > > > > > ok. Coming back to your questions, answers are inline. > > > > > > > > > > > > > > > > > > On Fri, Jul 24, 2015 at 9:08 AM, Ajay <[email protected]> > > wrote: > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > Couple of questions (that comes to my mind right now) > > > > > > > > > > > > > > 1) To create a Cluster instance to connect to a Cassandra > cluster > > > > > > > (Cassandra Java driver), we need to know the following: > > > > > > > a) ip address and port no of few instances of the cluster > > > > > > > b) authentication (if enabled) > > > > > > > c) few more configuration (loading balancing policy, ret policy > > and > > > > > > > compression etc..) > > > > > > > > > > > > > > Lens drivers are following xml based configuration for passing > > > > > > information. See > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-lens/blob/master/lens-driver-hive/src/main/resources/hivedriver-default.xml > > > > > > and > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-lens/blob/master/lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml > > > > > > , > > > > > > how driver specific configuration is put. Also read > > > > > > http://lens.incubator.apache.org/admin/config-server.html to > > > > understand > > > > > > configuration in lens. > > > > > > > > > > > > > > > > > > > Do we have any configuration support(yaml file or any other) in > > > Lens. > > > > > If > > > > > > > yes, any pointers (java file names or WIKI) > > > > > > > > > > > > > > 2) Cassandra cluser/session and prepared statements are multi > > > thread > > > > > safe > > > > > > > and recommended to be cached and reused. > > > > > > > > > > > > > > How does Lens support Caching (in memory or distributed)?. Any > > > > pointers > > > > > > > (java file names or WIKI) > > > > > > > > > > > > > > I did not understand the caching part here. But if you are > > talking > > > > > about > > > > > > cassandra sessions to be used across user queries, you can look > at > > > how > > > > > hive > > > > > > sessions and connections are used in HiveDriver - > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/incubator-lens/blob/master/lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 11:16 PM, Yash Sharma < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > That is great. We can probably pick the most recent Java > driver > > > and > > > > > > > > Cassandra version then. Since it addresses couple of old > > issues. > > > > > > > > > > > > > > > > Regarding Spark SQL for querying Cassandra I would let other > > > > > > contributors > > > > > > > > suggest. > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 11:04 PM, Ajay <[email protected] > > > > > > wrote: > > > > > > > > > > > > > > > > > Thanks Yash. > > > > > > > > > > > > > > > > > > In Between, ALLOW FILTERING is supported in Cassandra Java > > > driver > > > > > > [1]. > > > > > > > > What > > > > > > > > > is the Cassandra and Java driver version we plan to > > support?. I > > > > > have > > > > > > > > worked > > > > > > > > > on Cassandra 2.0.x and 2.1.x and Java driver 2.1.x and > ALLOW > > > > > > FILTERING > > > > > > > > > worked. > > > > > > > > > > > > > > > > > > Secondly, as you mentioned I am aware of these limitation > in > > > CQL. > > > > > But > > > > > > > > more > > > > > > > > > features recently added in Cassandra 2.2.x. Also, other > > option > > > to > > > > > > work > > > > > > > > > around this to use Cassandra Spark connector and use Spark > > SQL. > > > > > > > > > > > > > > > > > > 1) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://docs.datastax.com/en/drivers/java/2.0/com/datastax/driver/core/querybuilder/Select.html > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 10:07 PM, Yash Sharma < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Just to add on... > > > > > > > > > > > > > > > > > > > > There will be few challenges to support entire SQL on > > > Cassandra > > > > > > > > however. > > > > > > > > > > Last time I tried translating SQL to CQL I faced couple > of > > > > > > scenarios > > > > > > > - > > > > > > > > > > > > > > > > > > > > 1. Filtering non indexed column in Cassandra > > > > > > > > > > 2. Filtering by subset of primary key > > > > > > > > > > 3. OR condition in where clause > > > > > > > > > > > > > > > > > > > > You can probably start without these being a blocker - In > > > > > parallel > > > > > > we > > > > > > > > can > > > > > > > > > > discuss how this can be implemented on our stack. Others > > can > > > > > > provide > > > > > > > > > their > > > > > > > > > > opinions here. > > > > > > > > > > > > > > > > > > > > Hope its helpful. > > > > > > > > > > > > > > > > > > > > ----- > > > > > > > > > > Examples: > > > > > > > > > > Here 'trending_now' is a dummy table with (id, rank, > > pog_id) > > > > > where > > > > > > > > > > (id,rank) is primary key pair. > > > > > > > > > > > > > > > > > > > > 1. > > > > > > > > > > cqlsh:recsys> select * from trending_now where > > pog_id=10004 ; > > > > > > > > > > Bad Request: No indexed columns present in by-columns > > clause > > > > with > > > > > > > Equal > > > > > > > > > > operator > > > > > > > > > > > > > > > > > > > > 2. > > > > > > > > > > cqlsh:recsys> select * from trending_now where rank=4; > > > > > > > > > > Bad Request: Cannot execute this query as it might > involve > > > data > > > > > > > > filtering > > > > > > > > > > and thus may have unpredictable performance. If you want > to > > > > > execute > > > > > > > > this > > > > > > > > > > query despite the performance unpredictability, use ALLOW > > > > > FILTERING > > > > > > > > > > P.S. ALLOW FILTERING is not permitted in Cassandra java > > > driver > > > > as > > > > > > of > > > > > > > > now. > > > > > > > > > > > > > > > > > > > > 3. > > > > > > > > > > cqlsh:recsys> select * from trending_now where rank=4 or > > > > > > id='id0004'; > > > > > > > > > > Bad Request: line 1:40 missing EOF at 'or' > > > > > > > > > > > > > > > > > > > > 4. Valid Query: > > > > > > > > > > cqlsh:recsys> select * from trending_now where > id='id0004' > > > and > > > > > > > rank=4; > > > > > > > > > > > > > > > > > > > > id | rank | pog_id > > > > > > > > > > --------+------+-------- > > > > > > > > > > id0004 | 4 | 10002 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 9:45 PM, Yash Sharma < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > The basic idea is to translate the Lens Query plan > (Which > > > is > > > > a > > > > > > Hive > > > > > > > > > AST) > > > > > > > > > > > to the DataStore specific Plan/Query. > > > > > > > > > > > > > > > > > > > > > > The best example I can think of right now is the > Elastic > > > > Search > > > > > > > patch > > > > > > > > > by > > > > > > > > > > > Amruth. You can go through the JIRA [1] for the > detailed > > > > > > discussion > > > > > > > > and > > > > > > > > > > the > > > > > > > > > > > Review Board [2] for code reference. > > > > > > > > > > > > > > > > > > > > > > Best regards > > > > > > > > > > > > > > > > > > > > > > 1. https://issues.apache.org/jira/browse/LENS-252 > > > > > > > > > > > 2. https://reviews.apache.org/r/36434 > > > > > > > > > > > > > > > > > > > > > > On Thu, Jul 23, 2015 at 9:30 PM, Ajay < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > >> Thanks Yash. > > > > > > > > > > >> > > > > > > > > > > >> Is there any documentation or WIKI on the Lens > Driver. I > > > am > > > > > > going > > > > > > > > > > through > > > > > > > > > > >> the code as well. > > > > > > > > > > >> > > > > > > > > > > >> Thanks > > > > > > > > > > >> Ajay > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> On Thu, Jul 23, 2015 at 12:01 PM, Yash Sharma < > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > >> > > > > > > > > > > >> > Hi Ajay, > > > > > > > > > > >> > Welcome to the Lens Dev. > > > > > > > > > > >> > We do have plans for a Cassandra driver [1] for Lens > > but > > > > no > > > > > > one > > > > > > > > has > > > > > > > > > > >> picked > > > > > > > > > > >> > it up yet. It would be great if you can pick up the > > task > > > > and > > > > > > > > submit > > > > > > > > > a > > > > > > > > > > >> patch > > > > > > > > > > >> > for review. > > > > > > > > > > >> > > > > > > > > > > > >> > Also drop a note on the list in case you stumble > upon > > > any > > > > > > issue. > > > > > > > > > > Someone > > > > > > > > > > >> > will always be around to help you out. > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > 1. https://issues.apache.org/jira/browse/LENS-654 > > > > > > > > > > >> > > > > > > > > > > > >> > On Thu, Jul 23, 2015 at 11:41 AM, Ajay < > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > >> > > > > > > > > > > > >> > > Hi, > > > > > > > > > > >> > > > > > > > > > > > > >> > > I recently noticed about Apache Lens project. > > > Currently > > > > we > > > > > > are > > > > > > > > > > >> building > > > > > > > > > > >> > > REST APIs for Apache Cassandra (proprietary) as > > there > > > no > > > > > > such > > > > > > > > > Apache > > > > > > > > > > >> open > > > > > > > > > > >> > > source project exists. Now as Lens supports REST > > APIs > > > > for > > > > > > > Hadoop > > > > > > > > > and > > > > > > > > > > >> > JDBC, > > > > > > > > > > >> > > I want to know is there any plan to support for > > Apache > > > > > > > Cassandra > > > > > > > > > as > > > > > > > > > > >> its > > > > > > > > > > >> > > support CQL and Spark SQL (thru Spark connector) > > which > > > > are > > > > > > > more > > > > > > > > > SQL > > > > > > > > > > >> like. > > > > > > > > > > >> > > If yes, I wish to know the details and contribute > as > > > > well. > > > > > > > > > > >> > > > > > > > > > > > > >> > > Thanks > > > > > > > > > > >> > > Ajay > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
