Hello, zjk, have you finally solved this problem? I encountered exactly the
same issue recently while training a model with TensorFlow 1.15.
On Tuesday, May 3, 2022 at 1:42:19 AM UTC+8 Deanna Garcia wrote:
> I think you should file this bug with tensorflow since you are only using
> protobufs through them.
>
> On Saturday, April 23, 2022 at 10:49:41 AM UTC-7 zjk wrote:
>
>> Hi, I am not a direct user of protobuf, I was training my model with
>> tensorflow using Python, while attempting to save my checkpoint model, this
>> error related to protobuf occurred:
>>
>> [libprotobuf ERROR google/protobuf/wire_format_lite.cc:577] String field '
>> tensorflow.TensorShapeProto.Dim.name' contains invalid UTF-8 data when
>> parsing a protocol buffer. Use the 'bytes' type if you intend to send raw
>> bytes.
>> Traceback (most recent call last):
>> File "train.py", line 77, in
>> train(model)
>> File "train.py", line 24, in train
>> model.optimize()
>> File "/user-data/HyperBox-main/script/model/box_model.py", line 329, in
>> optimize
>> self.save_model(itr)
>> File "/user-data/HyperBox-main/script/model/box_model.py", line 140, in
>> save_model
>> self.saver.save(self.sess, filename)
>> File
>> "/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
>>
>> line 1208, in save
>> self.export_meta_graph(
>> File
>> "/opt/conda/lib/python3.8/site-packages/tensorflow/python/training/saver.py",
>>
>> line 1254, in export_meta_graph
>> graph_def=ops.get_default_graph().as_graph_def(add_shapes=True),
>> File
>> "/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
>> line 3345, in as_graph_def
>> result, _ = self._as_graph_def(from_version, add_shapes)
>> File
>> "/opt/conda/lib/python3.8/site-packages/tensorflow/python/framework/ops.py",
>> line 3262, in _as_graph_def
>> graph.ParseFromString(compat.as_bytes(data))
>> google.protobuf.message.DecodeError: Error parsing message
>>
>> from the information above, we can see that:
>> 1) some so-called 'invalid UTF-8 data' was sent to protobuf;
>> 2) Use the 'bytes' type if you intend to send raw bytes.
>>
>> the function at the second to last line compat.as_bytes(data) is defined
>> as:
>> def as_bytes(bytes_or_text, encoding='utf-8'):
>> if isinstance(bytes_or_text, bytearray):
>> return bytes(bytes_or_text)
>> elif isinstance(bytes_or_text, _six.text_type):
>> ##_six.text_type = unicode
>> return bytes_or_text.encode(encoding)
>> elif isinstance(bytes_or_text, bytes):
>> return bytes_or_text
>> else:
>> raise TypeError('Expected binary or unicode string, got %r' %
>> (bytes_or_text,))
>>
>> we can see that this function makes sure that what will be passed into
>> graph.ParseFromString(which later passed to protobuf) is definitely utf-8
>> data or bytes, however, the protobuf keeps claiming the data passed to it
>> is not valid utf-8 data. Taking one step back, even if the data was not
>> valid utf-8, it is definitely bytes, which it should take in normally.
>>
>> And I printed some information before the execution of
>> graph.ParseFromString(compat.as_bytes(data)):
>>
>> pprint(type(data))
>> graph.ParseFromString(compat.as_bytes(data))
>>
>> and here's the result:
>>
>> the data has already been bytes even before it is passed into
>> compat.as_bytes!
>>
>> I am sorry for sounding emotional, it's just that I've been stuck with
>> this problem for too long and it really gets me frustrated.
>>
>>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/protobuf/77532789-5439-4a1b-bf21-321d4e9ac398n%40googlegroups.com.
