Hi Alfonso, Please take into a consideration that the property 'gora.datastore.kudu.masterAddress' is overridden in the class GoraKuduTestDriver [1]. Because KuduTestHarness generates random ports which need to be configured at runtime. Probably, you should change the property there too.
I will test my code with a docker container, in order to figure out the origin of the issue. Please let me know if someone faces this issue when building the project. [1] https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/src/test/java/org/apache/gora/kudu/GoraKuduTestDriver.java#L41 Cheers, John. El vie., 9 ago. 2019 a las 12:22, Alfonso Nishikawa (< alfonso.nishik...@gmail.com>) escribió: > Hi, > > I installed a standalone Kudu server (compiled from sources) in a virtual > machine and configured test/resources/gora.properties to use "nosql:8051" > master. > > The tests freezes waiting for kudu response and a tons of connections to > the master gets created. > > *To the community:* Can someone please clone > https://github.com/jhnmora000/gora/tree/GORA-485 and build it to see if > the problem can be reproduced? > > I attach snapshots showing how connections stays waiting. > > Thank you! > > Best Regards, > > Alfonso Nishikawa > > > > El jue., 8 ago. 2019 a las 20:18, Alfonso Nishikawa (< > alfonso.nishik...@gmail.com>) escribió: > >> Hi, John. >> >> Tried using Oracle jdk 1.8 and found the same core dump: >> >> [INFO] Running org.apache.gora.kudu.store.TestKuduStore >> [ERROR] Tests run: 44, Failures: 0, Errors: 40, Skipped: 4, Time elapsed: >> 52.466 s <<< FAILURE! - in org.apache.gora.kudu.store.TestKuduStore >> [ERROR] testNewInstance(org.apache.gora.kudu.store.TestKuduStore) Time >> elapsed: 1.834 s <<< ERROR! >> java.io.IOException: failed to start masters: Unable to start Master at >> index 0: >> /tmp/kudu-binary-jar4319751617646651391/kudu-binary-1.9.0-linux-x86_64/bin/kudu-master: >> process exited on signal 6 (core dumped) >> >> (I expected to fail too, since the problem doesn't look like being >> related to the jvm). >> >> Thanks for giving it a look. Don't know what must be the problem :\ >> >> Best Regards, >> >> Alfosno Nishikawa >> >> >> El mar., 6 ago. 2019 a las 4:26, John Mora (<jhnmora...@gmail.com>) >> escribió: >> >>> Hi Alfonso, >>> >>> Unfortunately, I have not been able to reproduce the issue. Maybe it is >>> related with my Java version (Oracle), I will try with OpenJDK. >>> Some details about my development environment: >>> >>> os.detected.name: linux >>> os.detected.arch: x86_64 >>> os.detected.version: 4.10 >>> os.detected.version.major: 4 >>> os.detected.version.minor: 10 >>> os.detected.release: linuxmint >>> os.detected.release.version: 18.3 >>> os.detected.release.like.linuxmint: true >>> os.detected.release.like.ubuntu: true >>> os.detected.classifier: linux-x86_64 >>> >>> Java >>> java version "1.8.0_171" >>> Java(TM) SE Runtime Environment (build 1.8.0_171-b11) >>> Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode) >>> >>> Maven >>> Apache Maven 3.3.9 >>> Maven home: /usr/share/maven >>> Java version: 1.8.0_171, vendor: Oracle Corporation >>> Java home: /usr/lib/jvm/java-8-oracle/jre >>> Default locale: en_US, platform encoding: UTF-8 >>> OS name: "linux", version: "4.10.0-38-generic", arch: "amd64", family: >>> "unix" >>> >>> >>> Best, >>> John. >>> >>> El lun., 5 ago. 2019 a las 16:48, Alfonso Nishikawa (< >>> alfonso.nishik...@gmail.com>) escribió: >>> >>>> Hi, >>>> >>>> I am using now the following pom configuration I got from executing >>>> `mvn dependency:tree`: >>>> >>>> <dependency> >>>> <groupId>org.apache.kudu</groupId> >>>> <artifactId>kudu-binary</artifactId> >>>> <classifier>linux-x86_64</classifier> >>>> <version>1.9.0</version> >>>> <scope>test</scope> >>>> </dependency> >>>> >>>> When I execute `mvn clen package` on gora-kudu I find that it spawns >>>> the following command: >>>> >>>> kudu-master >>>> --fs_wal_dir=/tmp/mini-kudu-cluster8989984398759938222/master-0/wal >>>> --fs_data_dirs=/tmp/mini-kudu-cluster8989984398759938222/master-0/data >>>> --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 >>>> --tsk_num_rsa_bits=512 --rpc_bind_addresses=*127.26.116.190*:39535 >>>> --webserver_interface=*127.26.116.190* --webserver_port=0 >>>> --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false >>>> --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 >>>> --log_dir=/tmp/mini-kudu-cluster8989984398759938222/master-0/logs >>>> --server_dump_info_path=/tmp/mini-kudu-cluster8989984398759938222/master-0/data/info.pb >>>> --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports >>>> --unlock_experimental_flags --unlock_unsafe_flags --rpc_reuseport=true >>>> --master_addresses=*127.26.116.190*:39535,*127.26.116.189*:33913, >>>> *127.26.116.188*:42253 >>>> >>>> >>>> I highlight the IP addresses because they clearly are not my computer, >>>> and I guess that is why the tests can't connect to the the database. >>>> >>>> Any idea on how to solve this? >>>> >>>> Thank you! >>>> >>>> >>>> Best Regards, >>>> >>>> Alfonso Nishikawa >>>> >>>> >>>> >>>> El lun., 5 ago. 2019 a las 8:39, Alfonso Nishikawa (< >>>> alfonso.nishik...@gmail.com>) escribió: >>>> >>>>> Hi, John. >>>>> >>>>> I get a core dump from the binary kudu server when trying to run the >>>>> tests. Didn't find a log file, but will search thoroughly later. Happened >>>>> anytime to you? Does it happens to anyone? >>>>> >>>>> I am using Ubuntu 18.04 >>>>> >>>>> Thank you! >>>>> >>>>> Regards, >>>>> >>>>> Alfonso Nishikawa >>>>> >>>>> El dom., 4 ago. 2019 20:10, Furkan KAMACI <furkankam...@gmail.com> >>>>> escribió: >>>>> >>>>>> Hi John, >>>>>> >>>>>> I've already made my comments at your PR. Please check them carefully >>>>>> and ask me if you need help. >>>>>> >>>>>> For the documentation, I've checked what you've done. On the other >>>>>> hand, I would want to encourage you to write a blog post about your Kudu >>>>>> implementation and demonstrate an example of Kudu integration with Gora >>>>>> as >>>>>> like a tutorial. >>>>>> >>>>>> Kind Regards, >>>>>> Furkan KAMACI >>>>>> >>>>>> On Sun, Aug 4, 2019 at 1:59 AM John Mora <jhnmora...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Hi all. >>>>>>> >>>>>>> I have updated my report in the Wiki[1]. >>>>>>> >>>>>>> Also, I have sent a PR with my last commits for review [2]. Please >>>>>>> give it a look if you have time. >>>>>>> >>>>>>> This week, I will continue working on the documentation of the kudu >>>>>>> datastore. >>>>>>> >>>>>>> Please let me know if you have suggestions. >>>>>>> >>>>>>> [1] >>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>> [2] https://github.com/apache/gora/pull/178 >>>>>>> >>>>>>> Best, >>>>>>> John. >>>>>>> >>>>>>> El mié., 31 jul. 2019 a las 11:17, carlos muñoz (< >>>>>>> carlosr...@gmail.com>) escribió: >>>>>>> >>>>>>>> Hi John, >>>>>>>> >>>>>>>> Thanks for the update. I reviewed your code a little bit, it is >>>>>>>> looking good. I think tha you should send a PR in order to receive >>>>>>>> feedback >>>>>>>> from other community members. >>>>>>>> >>>>>>>> Best, >>>>>>>> Carlos >>>>>>>> >>>>>>>> El dom., 28 jul. 2019 a las 23:20, John Mora (<jhnmora...@gmail.com>) >>>>>>>> escribió: >>>>>>>> >>>>>>>>> Hi all. >>>>>>>>> >>>>>>>>> I updated my report in the Wiki[1]. Also, I pushed my last commits >>>>>>>>> to my branch [2]. Please give it a look if you have time. >>>>>>>>> >>>>>>>>> This week, I will give a look to the documentation of datastores. >>>>>>>>> >>>>>>>>> Please let me know if you have suggestions. >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>>>> [2] https://github.com/jhnmora000/gora/tree/GORA-485 >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> John >>>>>>>>> >>>>>>>>> El mié., 24 jul. 2019 a las 11:34, John Mora (< >>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>> >>>>>>>>>> Hi Alfonso, >>>>>>>>>> >>>>>>>>>> Yes, I was using this class javafx.util.Pair. It is not a problem >>>>>>>>>> I will find an alternative, it is only an utilitary class. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> John >>>>>>>>>> >>>>>>>>>> El mar., 23 jul. 2019 a las 12:36, Alfonso Nishikawa (< >>>>>>>>>> alfonso.nishik...@gmail.com>) escribió: >>>>>>>>>> >>>>>>>>>>> Hi, John. >>>>>>>>>>> >>>>>>>>>>> I checked out your code and it looks good :) >>>>>>>>>>> I found that you use javafx, but that is not present in OpenJDK >>>>>>>>>>> and fails to compile, and since we don't stick to Oracle JVM I would >>>>>>>>>>> suggest to change it. >>>>>>>>>>> >>>>>>>>>>> Good job, keep it going :) >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> >>>>>>>>>>> Alfonso Nishikawa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> El sáb., 20 jul. 2019 a las 22:25, John Mora (< >>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>> >>>>>>>>>>>> Hi. >>>>>>>>>>>> >>>>>>>>>>>> I updated my report in the Wiki[1]. Also, I pushed my last >>>>>>>>>>>> commits to my branch [2]. Please give it a look if you have time. >>>>>>>>>>>> >>>>>>>>>>>> This week, I will give a look to the map reduce tests for >>>>>>>>>>>> DataStores. >>>>>>>>>>>> >>>>>>>>>>>> Please let me know if you have suggestions. >>>>>>>>>>>> >>>>>>>>>>>> [1] >>>>>>>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>>>>>>> [2] https://github.com/jhnmora000/gora/tree/GORA-485 >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> John >>>>>>>>>>>> >>>>>>>>>>>> El sáb., 13 jul. 2019 a las 19:31, John Mora (< >>>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>>> >>>>>>>>>>>>> Hi all >>>>>>>>>>>>> >>>>>>>>>>>>> I updated my report in the Wiki[1]. Also, I pushed my last >>>>>>>>>>>>> commits to my branch [2]. Please give it a look if you have time. >>>>>>>>>>>>> >>>>>>>>>>>>> This week, I will be working in the getPartitions and >>>>>>>>>>>>> deleteByQuery methods and testing the other tests in the >>>>>>>>>>>>> DataStoreTestBase >>>>>>>>>>>>> class. >>>>>>>>>>>>> >>>>>>>>>>>>> Please let me know if you have suggestions. >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>>>>>>>> [2] https://github.com/jhnmora000/gora/tree/GORA-485 >>>>>>>>>>>>> >>>>>>>>>>>>> Best, >>>>>>>>>>>>> John. >>>>>>>>>>>>> >>>>>>>>>>>>> El mié., 10 jul. 2019 a las 16:17, John Mora (< >>>>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Alfonso, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks so much for your time and support for this project. I >>>>>>>>>>>>>> will work on your comments. Responses inline :) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> El mar., 9 jul. 2019 a las 16:38, Alfonso Nishikawa (< >>>>>>>>>>>>>> alfonso.nishik...@gmail.com>) escribió: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi, John. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Sorry for the delay, I am changing work and I have been very >>>>>>>>>>>>>>> busy :( I will try to answer your questions :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> *> In the Employee example there is a field called >>>>>>>>>>>>>>> 'dateOfBirth'. I tried to map that field with the >>>>>>>>>>>>>>> UNIXTIME_MICROS datatype >>>>>>>>>>>>>>> of Kudu (I intuitively assumed this is a date.). However, in >>>>>>>>>>>>>>> the java world >>>>>>>>>>>>>>> the Employee field is a Long value and the kudu datatype is a >>>>>>>>>>>>>>> Timestamp. >>>>>>>>>>>>>>> So, I was wondering whether I should force the usage of the >>>>>>>>>>>>>>> UNIXTIME_MICROS >>>>>>>>>>>>>>> datatype for this field or just use a LONG datatype in Kudu.* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In Avro 1.8 were introduced "Logical Types" so there is a >>>>>>>>>>>>>>> "date" type with an underlying "int" [1]. It's the first time I >>>>>>>>>>>>>>> read about >>>>>>>>>>>>>>> because until the last version upgrade of Avro this weren't >>>>>>>>>>>>>>> there. I would >>>>>>>>>>>>>>> suggest to ignore "dates" and map dateOfBirth as long, since in >>>>>>>>>>>>>>> any case >>>>>>>>>>>>>>> -in avro- the value is the unix epoch. After this first >>>>>>>>>>>>>>> approach, a design >>>>>>>>>>>>>>> improvement would be great, though :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> - Would be good to have in the mapping a "timestamp" type so >>>>>>>>>>>>>>> KuduStore converts between the Entity long field <-> Kudu >>>>>>>>>>>>>>> timestamp storage? >>>>>>>>>>>>>>> - Is there any other approach? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I think that Entity long field <-> Kudu timestamp conversion >>>>>>>>>>>>>> that the best alternative right now. Because, I would add more >>>>>>>>>>>>>> compatible >>>>>>>>>>>>>> datatypes to the mapping parameters which users can use. And this >>>>>>>>>>>>>> conversion should not be dificult to implement in my opinion. >>>>>>>>>>>>>> Also, the new >>>>>>>>>>>>>> Date datatype of avro could be implemented in newer versions >>>>>>>>>>>>>> because it >>>>>>>>>>>>>> would need further analysis in other datastores too. I will work >>>>>>>>>>>>>> on that. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> *> What is the Gora's policy regarding flush()? * >>>>>>>>>>>>>>> *> KuduClient has multiple flushing modes >>>>>>>>>>>>>>> <https://kudu.apache.org/apidocs/org/apache/kudu/client/SessionConfiguration.FlushMode.html>and >>>>>>>>>>>>>>> also can set time interval >>>>>>>>>>>>>>> <https://kudu.apache.org/releases/1.2.0/apidocs/org/apache/kudu/client/KuduSession.html#setFlushInterval-int-> >>>>>>>>>>>>>>> for automatic flush.* >>>>>>>>>>>>>>> *> Should theses behaviors be configurable using >>>>>>>>>>>>>>> gora.properties file? or just use the default configurations.* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> What we do in HBase is configure an autoflush option in >>>>>>>>>>>>>>> gora.properties [2] which is used when instanced the Table, but >>>>>>>>>>>>>>> at the same >>>>>>>>>>>>>>> time we implement the flush() method to force the flush [3]. I >>>>>>>>>>>>>>> would >>>>>>>>>>>>>>> suggest to follow that example, but adding the flushing options >>>>>>>>>>>>>>> of Kudu. >>>>>>>>>>>>>>> What flushing mode (and time interval if it applies) do you >>>>>>>>>>>>>>> suggest? >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Well, IMHO the default flush mode (auto flush sync) will do >>>>>>>>>>>>>> the job for most use cases. But I will add a configuration in >>>>>>>>>>>>>> gora.properties for selecting the other modes and specifying a >>>>>>>>>>>>>> autoflush >>>>>>>>>>>>>> time if needed by the user. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> *> Also, while reviewing the datastore interface I noticed >>>>>>>>>>>>>>> this method 'getPartitions(Query<K, T> query)'. What is the >>>>>>>>>>>>>>> expected >>>>>>>>>>>>>>> behavior of this method?, should I use the partition definition >>>>>>>>>>>>>>> in the xml >>>>>>>>>>>>>>> mapping file for this?.* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The method getPartitions(Query) is related to Hadoop. Apache >>>>>>>>>>>>>>> Gora integrates with Hadoop implementing a custom Map and >>>>>>>>>>>>>>> Reduce that >>>>>>>>>>>>>>> allows to get/write Entities directly. >>>>>>>>>>>>>>> You can take a look at HBase's implementation [4], which >>>>>>>>>>>>>>> relies o.a.h.hbase.mapreduce.TableInputFormatBase [5] to >>>>>>>>>>>>>>> compute the splits (start key---end key) with the location of >>>>>>>>>>>>>>> the split to >>>>>>>>>>>>>>> create a colection of partitions [6]. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> So, if Kudu is allowed to perform computation using local >>>>>>>>>>>>>>> kudu splits, then this method does the needed preparation to >>>>>>>>>>>>>>> allow to "send >>>>>>>>>>>>>>> the computation to where the data is locally". >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In any case, you can see that: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> - MongoDB store implementation does not implement >>>>>>>>>>>>>>> splitting [7] >>>>>>>>>>>>>>> - Cassandra store implementation does not implement >>>>>>>>>>>>>>> splitting [8] >>>>>>>>>>>>>>> - Aerospike store implementation does not implement >>>>>>>>>>>>>>> splitting [9] >>>>>>>>>>>>>>> - Accumulo store implementation* does* implement >>>>>>>>>>>>>>> splitting [10] >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If Kudu has a method to get the different splits for a table >>>>>>>>>>>>>>> and its locations, then you will be able to implement the full >>>>>>>>>>>>>>> feature. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> This is Hadoop related and it is not trivial. I haven't >>>>>>>>>>>>>>> elaborated much, so if you find you need more information let >>>>>>>>>>>>>>> me know :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> I will check whether Kudu has these features in order to >>>>>>>>>>>>>> implement this method. If not I will use the default >>>>>>>>>>>>>> implementation found >>>>>>>>>>>>>> in other backends. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> About Queries, what I can tell is that Hbase only implements >>>>>>>>>>>>>>> "Start key" + "End key" because it has only 2 operations: "get" >>>>>>>>>>>>>>> and "scan", >>>>>>>>>>>>>>> and the querying is for "scan" operation, were you want an >>>>>>>>>>>>>>> interval (or >>>>>>>>>>>>>>> all) of the rows. Does Kudu have more querying functionality? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> Yes, Kudu implements a Scanner for querying data among with >>>>>>>>>>>>>> conditional predicates for filtering. I am using those classes. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> About other topic, I am trying to install Kudu in standalone >>>>>>>>>>>>>>> (all in 1 node). Do you use a Cloudera installation or do you >>>>>>>>>>>>>>> have a >>>>>>>>>>>>>>> standalone installation? How do you do it? I found some >>>>>>>>>>>>>>> instructions, but >>>>>>>>>>>>>>> they talk about compiling Kudu [11]. I was looking for >>>>>>>>>>>>>>> something like >>>>>>>>>>>>>>> HBase, that it is unzip + execute "hbase start". >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> I am using an embedded mini-cluster which comes with compiled >>>>>>>>>>>>>> binaries and can be used with maven[1] for testing my code. Once >>>>>>>>>>>>>> I get it >>>>>>>>>>>>>> mature enough I think I will be testing the datastore with a >>>>>>>>>>>>>> docker >>>>>>>>>>>>>> container [2]. I could not find a unzip+execute bundle either >>>>>>>>>>>>>> and I am >>>>>>>>>>>>>> kinda noob for compiling it myself. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] >>>>>>>>>>>>>> https://kudu.apache.org/docs/developing.html#_jvm_based_integration_testing >>>>>>>>>>>>>> [2] https://hub.docker.com/r/usuresearch/apache-kudu/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Good job and thank you!! :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Alfonso Nishikawa >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1] - >>>>>>>>>>>>>>> https://avro.apache.org/docs/1.8.0/spec.html#Logical+Types >>>>>>>>>>>>>>> [2] - >>>>>>>>>>>>>>> https://github.com/apache/gora/blob/apache-gora-0.9/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L175 >>>>>>>>>>>>>>> [3] - >>>>>>>>>>>>>>> https://github.com/apache/gora/blob/apache-gora-0.9/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L458 >>>>>>>>>>>>>>> [4] - >>>>>>>>>>>>>>> https://github.com/apache/gora/blob/apache-gora-0.9/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L472 >>>>>>>>>>>>>>> [5] - >>>>>>>>>>>>>>> https://github.com/apache/gora/blob/apache-gora-0.9/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L479 >>>>>>>>>>>>>>> [6] - >>>>>>>>>>>>>>> https://github.com/apache/gora/blob/apache-gora-0.9/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L517 >>>>>>>>>>>>>>> [7] - >>>>>>>>>>>>>>> https://github.com/apache/gora/blob/apache-gora-0.9/gora-mongodb/src/main/java/org/apache/gora/mongodb/store/MongoStore.java#L533 >>>>>>>>>>>>>>> [8] - >>>>>>>>>>>>>>> https://github.com/apache/gora/blob/apache-gora-0.9/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraStore.java#L292 >>>>>>>>>>>>>>> [9] - >>>>>>>>>>>>>>> https://github.com/apache/gora/blob/apache-gora-0.9/gora-aerospike/src/main/java/org/apache/gora/aerospike/store/AerospikeStore.java#L369 >>>>>>>>>>>>>>> [10] - >>>>>>>>>>>>>>> https://github.com/apache/gora/blob/apache-gora-0.9/gora-accumulo/src/main/java/org/apache/gora/accumulo/store/AccumuloStore.java#L902 >>>>>>>>>>>>>>> [11] - https://kudu.apache.org/docs/installation.html >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> El lun., 8 jul. 2019 a las 3:42, John Mora (< >>>>>>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi all. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> As every week I updated my report in the Wiki[1]. Also, I >>>>>>>>>>>>>>>> pushed my last commits to my branch [2]. Please give it a look >>>>>>>>>>>>>>>> if you have >>>>>>>>>>>>>>>> time. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> This week, I will be continue working in the Queries >>>>>>>>>>>>>>>> implementation, please reach me out if you have any >>>>>>>>>>>>>>>> suggestions. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Also, while reviewing the datastore interface I noticed >>>>>>>>>>>>>>>> this method 'getPartitions(Query<K, T> query)'. What is the >>>>>>>>>>>>>>>> expected >>>>>>>>>>>>>>>> behavior of this method?, should I use the partition >>>>>>>>>>>>>>>> definition in the xml >>>>>>>>>>>>>>>> mapping file for this?. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>> John. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>>>>>>>>>>> [2] https://github.com/jhnmora000/gora/tree/GORA-485 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> El dom., 30 jun. 2019 a las 16:56, John Mora (< >>>>>>>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi all. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I received my first evaluation from the Google Summer of >>>>>>>>>>>>>>>>> Code program with a positive result. Thanks so much for your >>>>>>>>>>>>>>>>> support and >>>>>>>>>>>>>>>>> confidence to the project and me. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I updated my report of this week in the Wiki[1]. Also, I >>>>>>>>>>>>>>>>> pushed my last commits to my branch [2]. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This week, I will be reviewing my the serialization/ >>>>>>>>>>>>>>>>> deserialization process in order to identify optimizations >>>>>>>>>>>>>>>>> specific for >>>>>>>>>>>>>>>>> Kudu. Because I used a generic methods of other backends >>>>>>>>>>>>>>>>> which probably >>>>>>>>>>>>>>>>> could be better tuned for kudu. Also, I will start working on >>>>>>>>>>>>>>>>> the Queries >>>>>>>>>>>>>>>>> implementation. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> BTW, I added a question to the wiki about Date types. >>>>>>>>>>>>>>>>> Please give it a look if you have time. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>>>>>>>>>>>> [2] https://github.com/jhnmora000/gora/tree/GORA-485 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>> John >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> El jue., 27 jun. 2019 a las 21:02, John Mora (< >>>>>>>>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi Carlos. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks for the reminder. I submitted the form yesterday. >>>>>>>>>>>>>>>>>> :D >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>> John. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> El jue., 27 jun. 2019 a las 17:34, carlos muñoz (< >>>>>>>>>>>>>>>>>> carlosr...@gmail.com>) escribió: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi John >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The first Google Summer of Code evaluation is due on >>>>>>>>>>>>>>>>>>> June 28th. Please make sure you submit your Mentors' >>>>>>>>>>>>>>>>>>> evaluation on time. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>> Carlos >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> El dom., 23 jun. 2019 a las 18:29, John Mora (< >>>>>>>>>>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi all. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> FYI, I updated my report of this week on the Wiki[1]. >>>>>>>>>>>>>>>>>>>> Also, I pushed my last commits to my branch [2]. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> As I mentioned in the reports I would like to know how >>>>>>>>>>>>>>>>>>>> datastores deal with flush(), should it work always >>>>>>>>>>>>>>>>>>>> manually executed?. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Finally, This week I will be implementing object >>>>>>>>>>>>>>>>>>>> serialization/deserialization in the methods put, get, >>>>>>>>>>>>>>>>>>>> delete, exists. Do >>>>>>>>>>>>>>>>>>>> you have any suggestions on how to proceed with this task?. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Footnote: Thanks for the feedback Carlos, I fixed the >>>>>>>>>>>>>>>>>>>> problem. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>>>>>>>>>>>>>>> [2] https://github.com/jhnmora000/gora/tree/GORA-485 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>> John >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> El lun., 17 jun. 2019 a las 22:58, carlos muñoz (< >>>>>>>>>>>>>>>>>>>> carlosr...@gmail.com>) escribió: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi John >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Your last changes look good to me. Keep it up. But, I >>>>>>>>>>>>>>>>>>>>> noticed that you have created an Enumeration for >>>>>>>>>>>>>>>>>>>>> datatypes, which is very >>>>>>>>>>>>>>>>>>>>> similar to the kudu-client's [2]. Probably you should >>>>>>>>>>>>>>>>>>>>> replace [1] for [2] >>>>>>>>>>>>>>>>>>>>> in order to avoid code duplication. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/src/main/java/org/apache/gora/kudu/mapping/Column.java#L76 >>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>> https://kudu.apache.org/apidocs/org/apache/kudu/Type.html >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>> Carlos >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> El sáb., 15 jun. 2019 a las 12:01, John Mora (< >>>>>>>>>>>>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi all. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I updated my report of this week on the Wiki[1]. I >>>>>>>>>>>>>>>>>>>>>> noticed that my code is lacking some javadoc >>>>>>>>>>>>>>>>>>>>>> documentation I think I will >>>>>>>>>>>>>>>>>>>>>> be working on that this week, also I would like to >>>>>>>>>>>>>>>>>>>>>> enable and check schema >>>>>>>>>>>>>>>>>>>>>> management tests (createSchema, existsSchema, etc.). >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Cheers, >>>>>>>>>>>>>>>>>>>>>> John. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> El mar., 11 jun. 2019 a las 0:11, John Mora (< >>>>>>>>>>>>>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Hi Alfonso. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks so much for your feedback. I am working on >>>>>>>>>>>>>>>>>>>>>>> your comments. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>>>> John >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> El lun., 10 jun. 2019 a las 16:11, Alfonso Nishikawa >>>>>>>>>>>>>>>>>>>>>>> (<alfonso.nishik...@gmail.com>) escribió: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi, John. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Regarding your questions at the report [1]: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> - How to represent partitioning configurations >>>>>>>>>>>>>>>>>>>>>>>> on the mapping file. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> This was discussed in other emails, isn't it? :) >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> - KuduTestHarness requires the Maven plugin >>>>>>>>>>>>>>>>>>>>>>>> os-maven-plugin, which needs Maven 3.1.1+, is it a >>>>>>>>>>>>>>>>>>>>>>>> problem for Apache Gora? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> I believe it is not a problem. My Ubuntu comes with >>>>>>>>>>>>>>>>>>>>>>>> 3.6.0, far from 3.1.1, and I assume everyone uses >>>>>>>>>>>>>>>>>>>>>>>> Maven 3 in a quite new >>>>>>>>>>>>>>>>>>>>>>>> version :) >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> [1] - >>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Alfonso Nishikawa >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> El lun., 10 jun. 2019 a las 21:07, Alfonso >>>>>>>>>>>>>>>>>>>>>>>> Nishikawa (<alfonso.nishik...@gmail.com>) escribió: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi, John. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thank you! >>>>>>>>>>>>>>>>>>>>>>>>> Things I have seen: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> - The version of a maven dependency [1] should go >>>>>>>>>>>>>>>>>>>>>>>>> on the Dependency Management of the root pom [2]. >>>>>>>>>>>>>>>>>>>>>>>>> Same for [3] and from >>>>>>>>>>>>>>>>>>>>>>>>> there, should not set the version there. >>>>>>>>>>>>>>>>>>>>>>>>> - Set test dependencies' scope to test, at [4] and >>>>>>>>>>>>>>>>>>>>>>>>> from there. >>>>>>>>>>>>>>>>>>>>>>>>> - Set the indentation to 2 spaces for the pom [5] >>>>>>>>>>>>>>>>>>>>>>>>> - Missing "t" in "localhost" at [6]. >>>>>>>>>>>>>>>>>>>>>>>>> - Port 13 for Kudu? That is "Daytime Protocol" RFC >>>>>>>>>>>>>>>>>>>>>>>>> 867 and you will need root permission to run it. The >>>>>>>>>>>>>>>>>>>>>>>>> default port for kudu >>>>>>>>>>>>>>>>>>>>>>>>> is 7051, isn't it? >>>>>>>>>>>>>>>>>>>>>>>>> - I would ask you to add the same functionality to >>>>>>>>>>>>>>>>>>>>>>>>> load the mapping from configuration as in HBase's >>>>>>>>>>>>>>>>>>>>>>>>> store [7] in you >>>>>>>>>>>>>>>>>>>>>>>>> KuduStore [8]. This will have implications on your >>>>>>>>>>>>>>>>>>>>>>>>> readMapping at [9], so >>>>>>>>>>>>>>>>>>>>>>>>> take a look at the one for HBase at [10] >>>>>>>>>>>>>>>>>>>>>>>>> - I know it is in other backends, but avoid >>>>>>>>>>>>>>>>>>>>>>>>> RuntimeExceptions (at least in Java since we have the >>>>>>>>>>>>>>>>>>>>>>>>> checked ones) like in >>>>>>>>>>>>>>>>>>>>>>>>> [11]. You can wrap them in GoraException. An example >>>>>>>>>>>>>>>>>>>>>>>>> is [12] >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> And nothing more :) >>>>>>>>>>>>>>>>>>>>>>>>> Keep going, good job. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> [1] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/pom.xml#L98 >>>>>>>>>>>>>>>>>>>>>>>>> [2] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/pom.xml#L890 >>>>>>>>>>>>>>>>>>>>>>>>> [3] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/pom.xml#L121 >>>>>>>>>>>>>>>>>>>>>>>>> [4] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/pom.xml#L180 >>>>>>>>>>>>>>>>>>>>>>>>> [5] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/pom.xml >>>>>>>>>>>>>>>>>>>>>>>>> [6] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/src/test/resources/gora.properties#L18 >>>>>>>>>>>>>>>>>>>>>>>>> [7] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L92 >>>>>>>>>>>>>>>>>>>>>>>>> [8] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/src/main/java/org/apache/gora/kudu/store/KuduStore.java#L53 >>>>>>>>>>>>>>>>>>>>>>>>> [9] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/src/main/java/org/apache/gora/kudu/mapping/KuduMappingBuilder.java#L81 >>>>>>>>>>>>>>>>>>>>>>>>> [10] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L822 >>>>>>>>>>>>>>>>>>>>>>>>> [11] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/GORA-485/gora-kudu/src/main/java/org/apache/gora/kudu/mapping/KuduMappingBuilder.java#L141 >>>>>>>>>>>>>>>>>>>>>>>>> [12] - >>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L268 >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Alfonso Nishikawa >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> El sáb., 8 jun. 2019 a las 20:26, John Mora (< >>>>>>>>>>>>>>>>>>>>>>>>> jhnmora...@gmail.com>) escribió: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi all. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> I have just updated my weekly reports on Cwiki >>>>>>>>>>>>>>>>>>>>>>>>>> [1]. This next week I think I should be focusing on >>>>>>>>>>>>>>>>>>>>>>>>>> the create schema >>>>>>>>>>>>>>>>>>>>>>>>>> operation and solving the issue of the partitioning >>>>>>>>>>>>>>>>>>>>>>>>>> configurations in the >>>>>>>>>>>>>>>>>>>>>>>>>> mapping file. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Please let me know if you have suggestions, my >>>>>>>>>>>>>>>>>>>>>>>>>> last commits are available here [2] >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>> https://cwiki.apache.org/confluence/display/GORA/GORA-485+Apache+Kudu+datastore+for+Gora+Reports >>>>>>>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/jhnmora000/gora/tree/GORA-485 >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Best, >>>>>>>>>>>>>>>>>>>>>>>>>> John >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>
