Re: need to have samba-4.2.11 on amanda server
Hello!
I finally found time to upgrade samba from 4.1.23 to 4.4.14 and try this out
myself. I'm running amanda 3.3.9 on Slackware 14.2 and had no problems with
samba 4.1.23.
I have applied both patches mentioned. Result: I still get 'Can't translate
pathname [...] to UTF-8' errors.
The problem is that Slackware doesn't support UTF-8 out of the box. By default
LANG is set to en_US (apart from that only LC_COLLATE is set to 'C' - no other
LC_... variables set). On my server I have set LANG to de_DE for german
umlauts and samba is configured to use ISO-8859-15 ('unix charset =
ISO-8859-15' in smb.conf).
When I modify amsamba to set LC_CTYPE to 'de_DE' instead of 'en_US.UTF-8'
before executing smbclient, amdump runs without complaining. So far it looks
like it might be a better solution to make it configurable what amsamba sets
LC_CTYPE to.
Now that amanda dumps all files without errors, I get the following when using
amrecover to recover a test-file 'Bäcküp.txt':
amrecover> ls
2017-06-05-16-45-41 "Bäcküp.txt"
amrecover> add *
Added file "/Test/Bäcküp.txt"
When extracting amrecover prints
/bin/tar: ./Test/Bäcküp.txt: Nicht im Archiv gefunden.
which means 'not found in archive'.
When I start amrecover with LC_CTYPE set to en_US.UTF-8 it extracts the file,
but then the saved file is named Bäcküp.txt and not Bäcküp.txt (it's the same
for de_DE).
Because I use a vtape-configuration for testing I can easily look inside the
tar in the vtape-directory:
# dd if=1.soll.__haben_AGENDA_Test.0 bs=32k skip=1 | tar tvzf -
[...]
-rw-r--r-- 0/0 6 2017-06-05 16:10 ./Test/Bäcküp.txt
The tar seems to be okay, but amrecover must be doing something wrong. What's
puzzling me is that another file with umlauts, dumped directly from the linux-
filesystem and not via smbclient, is displayed right by amrecover:
amrecover> ls
2017-06-05-17-31-28 "änöthertest.txt"
Extracting works and the tar itself also looks okay:
# dd if=2.soll._etc.1 bs=32k skip=1 | tar tvzf - | grep test.txt
[...]
-rw-r--r-- root/root 5 2017-06-05 17:30 ./änöthertest.txt
I'm really wondering why there are 2 tar-files with seemingly perfect umlauts,
but one is displayed wrong in amrecover.
Any ideas?
Mike
Am Dienstag, 10. Januar 2017, 09:00:11 schrieb Jean-Louis Martineau:
> Hi All,
>
> The attached patch fix amsamba in 3.4.1 to works with newer version of
> samba.
>
> In 3.3.9 you also need the patch from
> https://github.com/zmanda/amanda/commit/585b2139153f8d4a93ccc796983eedfa5a38
> e6ce
>
> Jean-Louis
>
> [...]
Re: need to have samba-4.2.11 on amanda server
Hi All,
The attached patch fix amsamba in 3.4.1 to works with newer version of
samba.
In 3.3.9 you also need the patch from
https://github.com/zmanda/amanda/commit/585b2139153f8d4a93ccc796983eedfa5a38e6ce
Jean-Louis
On 06/01/17 09:08 AM, Jean-Louis Martineau wrote:
On 06/01/17 07:18 AM, Mike Höhn wrote:
> Am Freitag, 6. Januar 2017, 11:40:44 schrieb Stefan G. Weichinger:
>> Anyone using smbclient for backups with late amanda?
>> If yes, which version of samba?
> Stefan,
>
> I've had the same problems with samba 4.4.5 and amanda 3.3.9 on
Slackware.
> I had found this:
>
> http://www.backupcentral.com/forum/16/267955
>
> It's not amanda related, but describes the problem smbclient has
with a non-
> UTF locale.
>
> IIRC I tried to set LANG like described before starting amdump.
Unfortunately
> it didn't work for me. But perhaps this is where to look for a solution.
>
> Mike
>
If setting an environment variable is required, try setting it in the
amsamba script before all exec call
$ENV{'LANG'} = 'en_US.UTF-8'
Jean-Louis
*Disclaimer*
The information contained in this communication from the sender is
confidential. It is intended solely for use by the recipient and
others authorized to receive it. If you are not the recipient, you are
hereby notified that any disclosure, copying, distribution or taking
action in relation of the contents of this information is strictly
prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been
automatically archived by *Mimecast Ltd*, an innovator in Software as
a Service (SaaS) for business. Providing a *safer* and *more useful*
place for your human generated data. Specializing in; Security,
archiving and compliance. To find out more Click Here
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-06 um 20:16 schrieb Stefan G. Weichinger: > -> the amsamba-DLEs were dumped correctly now, no more bad news in the > report etc Unfortunately it's not fixed yet, I was too early with my posting. I will provide logs off-list.
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-06 um 19:56 schrieb Jean-Louis Martineau: On 06/01/17 01:47 PM, Stefan G. Weichinger wrote: You point me at commit 585b213 ? 3.4.1 should already have that, as far as I understand. Post the amsamba debug file and the complete sendbackup debug files so I can see which argument is used to execute amsamba and smbclient? Can you also post your patched amsamba? and the path of that file? Maybe I found the fix now as you pointed to that commit: I diffed application-src/amsamba.pl between the two branches 3_3 and 3_4, they are *different* Now I patched the installed amsamba script (so that in fact it is the version from branch 3_3) and re-ran my test amdump. -> the amsamba-DLEs were dumped correctly now, no more bad news in the report etc Maybe the patch didn't make it to 3.4.1 really? - minor note: in "amstatus" there is no more progress visible for these DLEs, only "10g dumping" but not any clue how much is dumped already (while the actual amdump). That was nice to have sometimes ... - Thanks a lot so far, I will test these backups asap for their content, especially if german umlauts and special chars are still valid after a restore. And now I run these delayed backups and get into the beginning weekend. Thanks for your patience, regards Stefan
Re: need to have samba-4.2.11 on amanda server
On 06/01/17 01:47 PM, Stefan G. Weichinger wrote: Am 2017-01-06 um 19:42 schrieb Jean-Louis Martineau: 3.3.9 will not work with newer samba, you must use 3.4.1 The amsamba patch is in the 3_3 branch, you can get it from svn or git. This server runs amanda-3.4.1 samba-3.4.5 You point me at commit 585b213 ? 3.4.1 should already have that, as far as I understand. Post the amsamba debug file and the complete sendbackup debug files so I can see which argument is used to execute amsamba and smbclient? Can you also post your patched amsamba? and the path of that file? Jean-Louis Disclaimer The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful. This email has been scanned for viruses and malware, and may have been automatically archived by Mimecast Ltd, an innovator in Software as a Service (SaaS) for business. Providing a safer and more useful place for your human generated data. Specializing in; Security, archiving and compliance. To find out more visit the Mimecast website.
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-06 um 19:42 schrieb Jean-Louis Martineau: 3.3.9 will not work with newer samba, you must use 3.4.1 The amsamba patch is in the 3_3 branch, you can get it from svn or git. This server runs amanda-3.4.1 samba-3.4.5 You point me at commit 585b213 ? 3.4.1 should already have that, as far as I understand.
Re: need to have samba-4.2.11 on amanda server
On 06/01/17 12:29 PM, Stefan G. Weichinger wrote:
Am 2017-01-06 um 15:08 schrieb Jean-Louis Martineau:
If setting an environment variable is required, try setting it in the
amsamba script before all exec call
$ENV{'LANG'} = 'en_US.UTF-8'
I edited amsamba accordingly, test failed again.
To me it seems that the output of smbclient has changed a bit again
and therefore amanda can't interpret its summary line or so:
... dump failed: [no backup size line]
# sendbackup.20170106181516.debug
Fri Jan 06 18:18:36.296112465 2017: pid 30625: thd-0x8060200:
sendbackup: 119: strange(?): tar:712 Total bytes received: 6034735616
Fri Jan 06 18:18:36.298046851 2017: pid 30627: thd-0x8060200:
sendbackup: Index created successfully
Fri Jan 06 18:18:36.479196763 2017: pid 30625: thd-0x8060200:
sendbackup: critical (fatal): error [no backup size line]
3.3.9 will not work with newer samba, you must use 3.4.1
The amsamba patch is in the 3_3 branch, you can get it from svn or git.
Jean-Louis
/usr/lib/amanda/libamanda-3.4.1.so(+0x2de11)[0xb7645e11]
/usr/lib/libglib-2.0.so.0(g_logv+0x1d6)[0xb75479dc]
/usr/lib/libglib-2.0.so.0(g_log+0x33)[0xb7547c29]
/usr/libexec/amanda/sendbackup(parse_backup_messages+0x41a)[0x804f202]
/usr/libexec/amanda/sendbackup(main+0x1360)[0x804bf20]
/lib/libc.so.6(__libc_start_main+0xf4)[0xb735e49c]
/usr/libexec/amanda/sendbackup[0x804d924]
Disclaimer
The information contained in this communication from the sender is
confidential. It is intended solely for use by the recipient and others
authorized to receive it. If you are not the recipient, you are hereby notified
that any disclosure, copying, distribution or taking action in relation of the
contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been
automatically archived by Mimecast Ltd, an innovator in Software as a Service
(SaaS) for business. Providing a safer and more useful place for your human
generated data. Specializing in; Security, archiving and compliance. To find
out more visit the Mimecast website.
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-06 um 18:43 schrieb Jean-Louis Martineau:
Stefan,
Are you using amsamba? or you are using the old hack that allowed the
program "GNUTAR" to backup samba share?
Your dumptype must have the following lines to use amsamba:
program "APPLICATION"
application {
plugin "amsamba"
}
afaik I use amsamba, although configured slightly different:
define application app_amsamba {
plugin "amsamba"
property "SMBCLIENT-PATH" "/usr/bin/smbclient"
property "GNUTAR-PATH" "/bin/tar"
property "AMANDAPASS" "/etc/amandapass"
property "ALLOW-ANONYMOUS" "guest"
}
define dumptype amsamba {
comment "per smbclient"
global
program "APPLICATION"
application "app_amsamba"
dumpcycle 2
}
Re: need to have samba-4.2.11 on amanda server
Stefan,
Are you using amsamba? or you are using the old hack that allowed the
program "GNUTAR" to backup samba share?
Your dumptype must have the following lines to use amsamba:
program "APPLICATION"
application {
plugin "amsamba"
}
Jean-Louis
On 06/01/17 12:29 PM, Stefan G. Weichinger wrote:
Am 2017-01-06 um 15:08 schrieb Jean-Louis Martineau:
If setting an environment variable is required, try setting it in the
amsamba script before all exec call
$ENV{'LANG'} = 'en_US.UTF-8'
I edited amsamba accordingly, test failed again.
To me it seems that the output of smbclient has changed a bit again
and therefore amanda can't interpret its summary line or so:
... dump failed: [no backup size line]
# sendbackup.20170106181516.debug
Fri Jan 06 18:18:36.296112465 2017: pid 30625: thd-0x8060200:
sendbackup: 119: strange(?): tar:712 Total bytes received: 6034735616
Fri Jan 06 18:18:36.298046851 2017: pid 30627: thd-0x8060200:
sendbackup: Index created successfully
Fri Jan 06 18:18:36.479196763 2017: pid 30625: thd-0x8060200:
sendbackup: critical (fatal): error [no backup size line]
/usr/lib/amanda/libamanda-3.4.1.so(+0x2de11)[0xb7645e11]
/usr/lib/libglib-2.0.so.0(g_logv+0x1d6)[0xb75479dc]
/usr/lib/libglib-2.0.so.0(g_log+0x33)[0xb7547c29]
/usr/libexec/amanda/sendbackup(parse_backup_messages+0x41a)[0x804f202]
/usr/libexec/amanda/sendbackup(main+0x1360)[0x804bf20]
/lib/libc.so.6(__libc_start_main+0xf4)[0xb735e49c]
/usr/libexec/amanda/sendbackup[0x804d924]
Disclaimer
The information contained in this communication from the sender is
confidential. It is intended solely for use by the recipient and others
authorized to receive it. If you are not the recipient, you are hereby notified
that any disclosure, copying, distribution or taking action in relation of the
contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been
automatically archived by Mimecast Ltd, an innovator in Software as a Service
(SaaS) for business. Providing a safer and more useful place for your human
generated data. Specializing in; Security, archiving and compliance. To find
out more visit the Mimecast website.
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-06 um 15:08 schrieb Jean-Louis Martineau:
If setting an environment variable is required, try setting it in the
amsamba script before all exec call
$ENV{'LANG'} = 'en_US.UTF-8'
I edited amsamba accordingly, test failed again.
To me it seems that the output of smbclient has changed a bit again and
therefore amanda can't interpret its summary line or so:
... dump failed: [no backup size line]
# sendbackup.20170106181516.debug
Fri Jan 06 18:18:36.296112465 2017: pid 30625: thd-0x8060200:
sendbackup: 119: strange(?): tar:712 Total bytes received: 6034735616
Fri Jan 06 18:18:36.298046851 2017: pid 30627: thd-0x8060200:
sendbackup: Index created successfully
Fri Jan 06 18:18:36.479196763 2017: pid 30625: thd-0x8060200:
sendbackup: critical (fatal): error [no backup size line]
/usr/lib/amanda/libamanda-3.4.1.so(+0x2de11)[0xb7645e11]
/usr/lib/libglib-2.0.so.0(g_logv+0x1d6)[0xb75479dc]
/usr/lib/libglib-2.0.so.0(g_log+0x33)[0xb7547c29]
/usr/libexec/amanda/sendbackup(parse_backup_messages+0x41a)[0x804f202]
/usr/libexec/amanda/sendbackup(main+0x1360)[0x804bf20]
/lib/libc.so.6(__libc_start_main+0xf4)[0xb735e49c]
/usr/libexec/amanda/sendbackup[0x804d924]
Re: need to have samba-4.2.11 on amanda server
On 06/01/17 07:18 AM, Mike Höhn wrote:
> Am Freitag, 6. Januar 2017, 11:40:44 schrieb Stefan G. Weichinger:
>> Anyone using smbclient for backups with late amanda?
>> If yes, which version of samba?
> Stefan,
>
> I've had the same problems with samba 4.4.5 and amanda 3.3.9 on Slackware.
> I had found this:
>
> http://www.backupcentral.com/forum/16/267955
>
> It's not amanda related, but describes the problem smbclient has with a non-
> UTF locale.
>
> IIRC I tried to set LANG like described before starting amdump. Unfortunately
> it didn't work for me. But perhaps this is where to look for a solution.
>
> Mike
>
If setting an environment variable is required, try setting it in the
amsamba script before all exec call
$ENV{'LANG'} = 'en_US.UTF-8'
Jean-Louis
Disclaimer
The information contained in this communication from the sender is
confidential. It is intended solely for use by the recipient and others
authorized to receive it. If you are not the recipient, you are hereby notified
that any disclosure, copying, distribution or taking action in relation of the
contents of this information is strictly prohibited and may be unlawful.
This email has been scanned for viruses and malware, and may have been
automatically archived by Mimecast Ltd, an innovator in Software as a Service
(SaaS) for business. Providing a safer and more useful place for your human
generated data. Specializing in; Security, archiving and compliance. To find
out more visit the Mimecast website.
Re: need to have samba-4.2.11 on amanda server
Am Freitag, 6. Januar 2017, 11:40:44 schrieb Stefan G. Weichinger: > Anyone using smbclient for backups with late amanda? > If yes, which version of samba? Stefan, I've had the same problems with samba 4.4.5 and amanda 3.3.9 on Slackware. I had found this: http://www.backupcentral.com/forum/16/267955 It's not amanda related, but describes the problem smbclient has with a non- UTF locale. IIRC I tried to set LANG like described before starting amdump. Unfortunately it didn't work for me. But perhaps this is where to look for a solution. Mike
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-05 um 22:47 schrieb Stefan G. Weichinger: Am 2017-01-04 um 23:08 schrieb Stefan G. Weichinger: Still problems with samba-4.2.14 and amanda-3.4.1: ? smbclient: tar:943 Fatal: Can't translate pathname './Sanit„r&Heizung/' to UTF-8 ? smbclient: NT_STATUS_UNSUCCESSFUL listing \\* ? smbclient: tar:704 do_list fail NT_STATUS_UNSUCCESSFUL ? smbclient: tar:370 tar_process failed as I am trying to bisect this by installing several samba-releases: do I need to rebuild amanda everytime after rebuilding another release of samba? (gentoo context) Anyone using smbclient for backups with late amanda? If yes, which version of samba? I find bug-reports around smbclient and its tar feature at debian etc https://bugzilla.samba.org/show_bug.cgi?id=11854 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=821801
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-04 um 23:08 schrieb Stefan G. Weichinger: > Still problems with samba-4.2.14 and amanda-3.4.1: > > ? smbclient: tar:943 Fatal: Can't translate pathname > './Sanit„r&Heizung/' to UTF-8 > ? smbclient: NT_STATUS_UNSUCCESSFUL listing \\* > ? smbclient: tar:704 do_list fail NT_STATUS_UNSUCCESSFUL > ? smbclient: tar:370 tar_process failed as I am trying to bisect this by installing several samba-releases: do I need to rebuild amanda everytime after rebuilding another release of samba? (gentoo context) thanks
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-04 um 22:24 schrieb Stefan G. Weichinger: > fyi: samba-4.2.11 : same problems > rebuilding 4.2.14 now > > if problems persist, I contact samba-devs did so already, see samba-ml. Still problems with samba-4.2.14 and amanda-3.4.1: ? smbclient: tar:943 Fatal: Can't translate pathname './Sanit„r&Heizung/' to UTF-8 ? smbclient: NT_STATUS_UNSUCCESSFUL listing \\* ? smbclient: tar:704 do_list fail NT_STATUS_UNSUCCESSFUL ? smbclient: tar:370 tar_process failed
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-04 um 18:43 schrieb Stefan G. Weichinger: Am 2017-01-04 um 18:40 schrieb Stefan G. Weichinger: I have to upgrade to samba-4.2.11 (gentoo linux) on an amanda server running amanda-3.4.1 Does anyone know if this: http://www.backupcentral.com/forum/16/263253 ... is still an issue? seems more a samba-bug, right? maybe I should contact their ml ... sorry for the noise. fyi: samba-4.2.11 : same problems rebuilding 4.2.14 now if problems persist, I contact samba-devs
Re: need to have samba-4.2.11 on amanda server
Am 2017-01-04 um 18:40 schrieb Stefan G. Weichinger: I have to upgrade to samba-4.2.11 (gentoo linux) on an amanda server running amanda-3.4.1 Does anyone know if this: http://www.backupcentral.com/forum/16/263253 ... is still an issue? seems more a samba-bug, right? maybe I should contact their ml ... sorry for the noise.
