Hi Cory, line 440 in the 1.2.1 release of Apache CouchDB looks like this:
{ok, <<RawBin:TotalBytes/binary>>} = file:pread(Fd, Pos, TotalBytes),
Your initial mail reported a 'badmatch' error on this line and indicated a
response of the form {ok, Bin}. The match that's failing is therefore the
TotalBytes check; i.e. CouchDB tried to read TotalBytes bytes from Fd starting
at position Pos but failed to do so.
The TotalBytes value is not checksummed, so this is indeed a common way that
disk failure manifests in CouchDB. When we go to read a term from disk we first
read a few bytes from the file which represent the size of the term (i.e.,
TotalBytes). What's probably happened is that the TotalBytes value is junk, and
CouchDB tried to read some crazy large value from disk. The Erlang I/O
subsystem returned all bytes up to EOF, and the process crashes when the sizes
don't match.
If you're inclined to do some surgery and you can figure out the value of "Pos"
from the stacktrace you can try truncating the file just before that number of
bytes; the resulting file is still a valid CouchDB view group, and whatever is
missing will automatically be reindexed. Or you could blow away the view group
file entirely and have it rebuild from scratch.
Adam
> On Dec 23, 2014, at 4:50 PM, Cory Zue <[email protected]> wrote:
>
> Hey Alexander,
>
> Thanks for the note. Upgrading sounds like a good suggestion, although my
> first priority right now is just understanding if I have any reason to be
> concerned about data corruption/loss. Do these stack traces provide any
> more insight?
>
> 1.
>
>
> [{couch_file,
>
> read_raw_iolist_int,
>
> 3,
>
> [{file,
>
> "couch_file.erl"},
>
> {line,
>
> 440}]},
>
> {couch_file,
>
> maybe_read_more_iolist,
>
> 4,
>
> [{file,
>
> "couch_file.erl"},
>
> {line,
>
> 430}]},
>
> {couch_file,
>
> handle_call,
>
> 3,
>
> [{file,
>
> "couch_file.erl"},
>
> {line,
>
> 336}]},
>
> {gen_server,
>
> handle_msg,
>
> 5,
>
> [{file,
>
> "gen_server.erl"},
>
> {line,
>
> 588}]},
>
> {proc_lib,
>
> init_p_do_apply,
>
> 3,
>
> [{file,
>
> "proc_lib.erl"},
>
> {line,
>
> 227}]}]}
>
> 2.
>
> [{couch_file,read_raw_iolist_int,3,[{file,\"couch_file.erl\"},{line,440}]},\n
> {couch_file,maybe_read_more_iolist,4,[{file,\"couch_file.erl\"},{line,430}]},\n
> {couch_file,handle_call,3,[{file,\"couch_file.erl\"},{line,336}]},\n
> {gen_server,handle_msg,5,[{file,\"gen_server.erl\"},{line,588}]},\n
> {proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,
> 227}]}]}","reason":"{gen_server,call,[<0.17085.3>,{pread_iolist,8607618203},
> infinity]}"}
>
> also, any suggestions on {"error":"unknown_error","reason":"function_clause
> "}?
>
> thanks in advance!
> Cory
> On Tue, Dec 23, 2014 at 3:20 AM, Alexander Shorin <[email protected]> wrote:
>
>> Hi,
>>
>> Without complete stacktrace it's hard to say, but
>> badmatch/function_clause commonly is about unhandled case in code.
>> According your CouchDB version the very first advice is to upgrade it
>> up to the latest stable release since alot of things had been fixed
>> since 1.2.1 day. Security ones are too.
>> --
>> ,,,^..^,,,
>>
>>
>> On Tue, Dec 23, 2014 at 6:37 AM, Cory Zue <[email protected]> wrote:
>>> Hi all,
>>>
>>> We recently had an accident and lost our database and had to restore
>> from a
>>> daily backup. Since restoring, couchdb has seemed to work ok for the most
>>> part, but has been giving strange, hard-to-reproduce errors that seem
>>> somewhat random. I've pasted a few examples below.
>>>
>>> We have reason to believe that there may be other issues - possibly
>>> hardware related - on the machine (a postgres instance running on the
>> same
>>> box is also giving some problems). However, I'm wondering if there is
>>> anything that can be easily guessed about what's going on from the
>> category
>>> of couch errors we're seeing. Here are some sample errors:
>>>
>>>
>>> [Tue, 23 Dec 2014 01:04:39 GMT] [error] [<0.29607.3>] function_clause
>> error
>>> in HTTP request
>>> [Tue, 23 Dec 2014 01:04:39 GMT] [error] [<0.29567.3>] Uncaught server
>>> error: function_clause
>>> [Mon, 22 Dec 2014 22:06:21 GMT] [error] [<0.17340.3>]
>>> {error_report,<0.30.0>,
>>>
>>> {<0.17340.3>,crash_report,
>>>
>>>
>>>
>> [[{initial_call,{couch_file,init,['Argument__1']}},
>>>
>>>
>>> {pid,<0.17340.3>},
>>>
>>>
>>> {registered_name,[]},
>>> {error_info,
>>> {exit,
>>> {{badmatch,
>>> {ok,
>>>
>>> <<102,101,49,45,52,48,99,57,45,57,57,97,51,45,
>>>
>>> (followed by a huge string of numbers and what looks like a stack trace).
>>>
>>> [Tue, 23 Dec 2014 02:06:45 GMT] [error] [<0.143.0>] Compaction daemon -
>> an
>>> error ocurred while compacting the view group `viewname` from database
>>> `dbname`: {{badmatch,
>>>
>>> {ok,
>>>
>>> <<98,105,
>>> (another huge string of numbers and stack trace)
>>>
>>> We're running couchdb 1.2.1 in case that helps.
>>>
>>> any thoughts?
>>>
>>> thanks,
>>> Cory
>>
