[
https://issues.apache.org/jira/browse/BLUR-18?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491140#comment-13491140
]
Aaron McCurry commented on BLUR-18:
-----------------------------------
-Ashish
At this point Blur is fairly close to HBase in architecture.
Here's the short version of the architecture:
-Shard servers are grouped into clusters, usually 1 cluster per HDFS file
system.
-Controller servers act as query and data routers in front of 1 or more shard
clusters. Basically they make all the data and tables in the shard clusters
look like one large system.
-Tables are contained within a shard cluster, and are broken up into many
shards. The number of shards are defined at table setup. Shard equals Lucene
index.
-The shards are evenly distributed across the shard servers within the cluster.
So if you have 1000 shards and are running a 100 shard servers in a cluster,
then each of the shard servers will have 10 shards.
-In the event of a shard server goes down, the shards that were being served by
the down shard server are automatically brought back online across the
remaining shard servers.
-At this point there aren't any automatic splits in Blur, however Lucene if
very efficient in it's indexes so single indexes (shards) can grow to a very
large size before needing to be split.
> Rework the MapReduce Library to implement Input/OutputFormats
> -------------------------------------------------------------
>
> Key: BLUR-18
> URL: https://issues.apache.org/jira/browse/BLUR-18
> Project: Apache Blur
> Issue Type: Improvement
> Reporter: Aaron McCurry
> Fix For: 0.2.0
>
> Attachments: 0001-BLUR-ID-18-Created-New-Version-of-Files.patch,
> 0001-BLUR-ID-18-New-Writables.patch
>
>
> Currently the only way to implement indexing is to use the BlurReducer. A
> better way to implement this would be to support Hadoop input/outputformats
> in both the new and old api's. This would allow an easier integration with
> other Hadoop projects such as Hive and Pig.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
