On Fri, Mar 11, 2011 at 10:37 AM, Mattias Persson <matt...@neotechnology.com> wrote:
> And I'm curious about why the neo4j lucene layer adds overhead and how your > code looks like in your own solution. I really don't know, didn't had time to investigate in neo4j code but I'm indexing a SHA1 hash key pointing to a Node just to prove presence of the data within the DB using plain Lucene solutions (using the ChenilleKit.codehaus.org project) and your Index framework to compare them and here are the results doing 200000 subsequent chunks/rows of data With the neo4j Index framework: [DEBUG] 11/56/2011 14:56:08,438 StatistichengModule.PickUpPoller Fatte 200000 righe in 108482 ms [DEBUG] 11/59/2011 14:59:51,061 StatistichengModule.PickUpPoller Fatte 200000 righe in 146664 ms [DEBUG] 11/02/2011 15:02:30,317 StatistichengModule.PickUpPoller Fatte 200000 righe in 159256 ms [DEBUG] 11/05/2011 15:05:08,680 StatistichengModule.PickUpPoller Fatte 200000 righe in 158363 ms [DEBUG] 11/08/2011 15:08:34,501 StatistichengModule.PickUpPoller Fatte 200000 righe in 205821 ms [DEBUG] 11/12/2011 15:12:51,690 StatistichengModule.PickUpPoller Fatte 200000 righe in 257189 ms [DEBUG] 11/17/2011 15:17:11,589 StatistichengModule.PickUpPoller Fatte 200000 righe in 259899 ms [DEBUG] 11/21/2011 15:21:45,109 StatistichengModule.PickUpPoller Fatte 200000 righe in 273520 ms [DEBUG] 11/26/2011 15:26:04,802 StatistichengModule.PickUpPoller Fatte 200000 righe in 259693 ms [DEBUG] 11/30/2011 15:30:32,269 StatistichengModule.PickUpPoller Fatte 200000 righe in 267467 ms [DEBUG] 11/34/2011 15:34:32,804 StatistichengModule.PickUpPoller Fatte 200000 righe in 240535 ms [DEBUG] 11/43/2011 15:43:14,960 StatistichengModule.PickUpPoller Fatte 200000 righe in 355129 ms [DEBUG] 11/50/2011 15:50:22,323 StatistichengModule.PickUpPoller Fatte 200000 righe in 427363 ms [DEBUG] 11/58/2011 15:58:12,846 StatistichengModule.PickUpPoller Fatte 200000 righe in 470523 ms With the plain Lucene solution (external to neo4j db): [DEBUG] 12/55/2011 14:55:03,997 StatistichengModule.PickUpPoller Fatte 200000 righe in 48138 ms [DEBUG] 12/55/2011 14:55:53,533 StatistichengModule.PickUpPoller Fatte 200000 righe in 49537 ms [DEBUG] 12/56/2011 14:56:54,773 StatistichengModule.PickUpPoller Fatte 200000 righe in 61240 ms [DEBUG] 12/57/2011 14:57:54,157 StatistichengModule.PickUpPoller Fatte 200000 righe in 59384 ms [DEBUG] 12/58/2011 14:58:52,667 StatistichengModule.PickUpPoller Fatte 200000 righe in 58510 ms [DEBUG] 12/00/2011 15:00:20,518 StatistichengModule.PickUpPoller Fatte 200000 righe in 87851 ms [DEBUG] 12/02/2011 15:02:29,176 StatistichengModule.PickUpPoller Fatte 200000 righe in 82548 ms [DEBUG] 12/04/2011 15:04:52,302 StatistichengModule.PickUpPoller Fatte 200000 righe in 77700 ms [DEBUG] 12/07/2011 15:07:09,584 StatistichengModule.PickUpPoller Fatte 200000 righe in 77727 ms [DEBUG] 12/08/2011 15:08:26,778 StatistichengModule.PickUpPoller Fatte 200000 righe in 77194 ms [DEBUG] 12/09/2011 15:09:39,495 StatistichengModule.PickUpPoller Fatte 200000 righe in 72717 ms [DEBUG] 12/11/2011 15:11:04,032 StatistichengModule.PickUpPoller Fatte 200000 righe in 84537 ms [DEBUG] 12/12/2011 15:12:35,806 StatistichengModule.PickUpPoller Fatte 200000 righe in 91774 ms [DEBUG] 12/14/2011 15:14:13,789 StatistichengModule.PickUpPoller Fatte 200000 righe in 97983 ms [DEBUG] 12/15/2011 15:15:48,196 StatistichengModule.PickUpPoller Fatte 200000 righe in 94407 ms [DEBUG] 12/17/2011 15:17:16,819 StatistichengModule.PickUpPoller Fatte 200000 righe in 88623 ms [DEBUG] 12/19/2011 15:19:09,743 StatistichengModule.PickUpPoller Fatte 200000 righe in 76255 ms [DEBUG] 12/20/2011 15:20:16,357 StatistichengModule.PickUpPoller Fatte 200000 righe in 66614 ms [DEBUG] 12/21/2011 15:21:16,136 StatistichengModule.PickUpPoller Fatte 200000 righe in 59779 ms [DEBUG] 12/22/2011 15:22:29,535 StatistichengModule.PickUpPoller Fatte 200000 righe in 73399 ms [DEBUG] 12/23/2011 15:23:51,419 StatistichengModule.PickUpPoller Fatte 200000 righe in 81884 ms [DEBUG] 12/25/2011 15:25:14,380 StatistichengModule.PickUpPoller Fatte 200000 righe in 82961 ms [DEBUG] 12/26/2011 15:26:21,857 StatistichengModule.PickUpPoller Fatte 200000 righe in 67477 ms [DEBUG] 12/27/2011 15:27:34,632 StatistichengModule.PickUpPoller Fatte 200000 righe in 72775 ms [DEBUG] 12/28/2011 15:28:45,497 StatistichengModule.PickUpPoller Fatte 200000 righe in 70864 ms [DEBUG] 12/29/2011 15:29:44,987 StatistichengModule.PickUpPoller Fatte 200000 righe in 59491 ms [DEBUG] 12/31/2011 15:31:46,182 StatistichengModule.PickUpPoller Fatte 200000 righe in 87324 ms [DEBUG] 12/33/2011 15:33:16,841 StatistichengModule.PickUpPoller Fatte 200000 righe in 90659 ms [DEBUG] 12/34/2011 15:34:48,457 StatistichengModule.PickUpPoller Fatte 200000 righe in 91616 ms [DEBUG] 12/36/2011 15:36:17,984 StatistichengModule.PickUpPoller Fatte 200000 righe in 89527 ms [DEBUG] 12/38/2011 15:38:02,083 StatistichengModule.PickUpPoller Fatte 200000 righe in 104099 ms [DEBUG] 12/39/2011 15:39:47,166 StatistichengModule.PickUpPoller Fatte 200000 righe in 105083 ms [DEBUG] 12/41/2011 15:41:32,083 StatistichengModule.PickUpPoller Fatte 200000 righe in 104917 ms [DEBUG] 12/43/2011 15:43:06,179 StatistichengModule.PickUpPoller Fatte 200000 righe in 94096 ms [DEBUG] 12/46/2011 15:46:05,766 StatistichengModule.PickUpPoller Fatte 200000 righe in 82731 ms [DEBUG] 12/47/2011 15:47:31,977 StatistichengModule.PickUpPoller Fatte 200000 righe in 86211 ms I stop early with neo4j Index framework cause the trend is always increasing. Cheers -- Massimo http://meridio.blogspot.com _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
