I've started by only coping $log.dir from server A to server B. Both
server A and server B ran same version of kafka 0.8 with same
configuration files.
However, after running kafka 0.8 on server B I get the following
exception when I tried to fetch the message:
2013-02-28 05:56:35,851] WARN [KafkaApi-1] Error while responding to
offset request (kafka.server.KafkaApis)
kafka.common.UnknownTopicOrPartitionException: Topic topic_general
partition 0 doesn't exist on 1
at
kafka.server.ReplicaManager.getLeaderReplicaIfLocal(ReplicaManager.scala:163)........
However, the folder topic_general-0 exists and there are files
00000000000000000000.log and 00000000000000000000.index there . There
are also a replication-offset-checkpoint file in this $log.dir folder.
I then copied by $log.dir and also the zookeeper folder from server A
to server B and run it. In the zookeeper folder I have the following
files:
-rw-r--r--. 1 root root 296 Feb 28 06:12 snapshot.0
-rw-r--r--. 1 root root 67108880 Feb 28 06:12 log.1
-rw-r--r--. 1 root root 67108880 Feb 28 06:12 log.4b
-rw-r--r--. 1 root root 4817 Feb 28 06:12 snapshot.4a
With both log data and zookeeper data copied over to server B I am
getting start up errors in zookeeper log
INFO Got user-level KeeperException when processing
sessionid:0x13d20830d3e0000 type:create cxid:0x1
zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a Error
Path:/brokers/ids/1 Error:KeeperErrorCode = NodeExists for
/brokers/ids/1 (org.apache.zookeeper.server.PrepRequestProcessor)
and start up errors in kafka server log:
[2013-02-28 06:15:00,228] ERROR [Partition state machine on Controller
1]: State change for partition
[topic_84784ecc-3803-42eb-bcdd-31dc42b697c6, 0] from OfflinePartition
to OnlinePartition failed (kafka.controller.PartitionStateMachine)
kafka.common.PartitionOfflineException: All replicas for partition
[topic_84784ecc-3803-42eb-bcdd-31dc42b697c6, 0] are dead. Marking this
partition offline
And I am getting the same error when trying to fetch messages:
2013-02-28 06:20:01,516] WARN [KafkaApi-1] Error while responding to
offset request (kafka.server.KafkaApis)
kafka.common.UnknownTopicOrPartitionException: Topic topic_general
partition 0 doesn't exist on 1
at
kafka.server.ReplicaManager.getLeaderReplicaIfLocal(ReplicaManager.scala:163)
I am running both zookeeper and kafka on the same server. I only have
one server so the replication factor is 1.
Looks like something went wrong for me. Any ideas?
thanks!
Jason
On Wed, Feb 27, 2013 at 6:59 PM, Neha Narkhede <neha.narkh...@gmail.com> wrote:
> As Jun described, the purpose of broker.id is to be able to move data from
> one broker to the other without changes. I believe this should work in 0.8
> as well. However, we've never tried it so not sure if there are bugs. Let
> us know how it goes.
>
> Thanks,
> Neha
>
>
> On Wed, Feb 27, 2013 at 1:31 PM, Jason Huang <jason.hu...@icare.com> wrote:
>
>> Thanks for the response.
>>
>> I apologize - I should have mentioned that we are trying out kafka 0.8
>> because of its new exciting features.
>>
>> So for 0.8, what do I need to do with the zookeeper data? copy
>> zookeeper file from $dataDir in zookeeper.properties from server A to
>> server B, and then run server B's kafka instance?
>>
>> Jason
>>
>> On Wed, Feb 27, 2013 at 12:57 PM, Neha Narkhede <neha.narkh...@gmail.com>
>> wrote:
>> > Jason,
>> >
>> > Assuming this question is for 0.7, you can just copy the data in the log
>> > directory and restart the new broker. You don't need to copy the
>> zookeeper
>> > data.
>> >
>> > Thanks,
>> > Neha
>> > On Feb 27, 2013 2:11 AM, "Jason Huang" <jason.hu...@icare.com> wrote:
>> >
>> >> Hello,
>> >>
>> >> Is there a way to migrate kafka data from server A to another server B?
>> >>
>> >> Can I just copy the data from $log.dir in server.properties and
>> >> zookeeper file from $dataDir in zookeeper.properties from server A to
>> >> server B, and then run server B's kafka instance?
>> >>
>> >> thanks,
>> >>
>> >> Jason
>> >>
>>
