Hallo, I am running an RPi4 with latest RasbianOS. I want to do an backup some dirs/files on an Synology NAS via NFS nad rsync. It is working but mode attributes are not correct. User / Group are right.
This is in my fstab:
192.168.2.5:/volume1/hoshi_ext/opt/fhem /mnt/ds9_fhem nfs nfsvers=3,sync,noauto 0 0A cron job is running this script:
-- #!/bin/bash mount /mnt/ds9_fhem && sleep 2 # FHEM Logadteien sichern sichern rsync --delete -avzu /opt/fhem/log/ /mnt/ds9_fhem/ # Backuplaufwerk auswerfen sleep2 && umount /mnt/ds9_fhem
On Tue, 6 Oct 2020 15:38:14 +0200, Hans-Werner Kneitinger declaimed the following:
Is this applicable?
-p, --perms preserve permissions """
-- Wulfraed Dennis Lee Bieber AF6VN wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/
Could it be the file system on the NAS which you don't mention? Something that doesn't have the concept of permissions at all or they aren't enabled? Or permissions that don't map to Unix that well especially when shared over NFS? Just guessing here. Maybe provide more info?
I believe a Synology supports either ext4 or btrfs ...
He has that set - '-a'
from man entry...
-a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)
It a bit baffling as I do almost exactly the same over nfs version 3. But to another linux machine that I setup - not some NAS box.
One thing I have added recently is --numeric-ids option as I found that while I keep all user uids equal across my system, some of the system uids - those less than 1000 e.g. for various demon services etc - change depending on distro and version of distro it seems. So some of my backup had their uids changed to match to same username on the NFS server. When I copied back after a disk break I got a few niggles. I don't know why I thought it should have mapped back - but ... who knows. The --numeric-ids means I dont have to care how I put the files back.
I had exactly the same issue when I tried to recover a Pi from a backup for the first time, and it took a bit of sorting out.
Although most of my Pi's have been cloned from a previous one, the ids vary depending on the order certain services are installed, so they can quickly get out of step.
I did think of harmonising them across all dozen Pi's, but it would be a lot of work to change all files on each to match the new values. Using
--numeric_ids is a much quicker solution.
---druck
Hmm.
I do something superficially similar:
1) attach a USB disk to my house server and mount it. There is one partition on this disk.2) for each of the four machines being backed up (three over my LAN the 4th is the house server
2a) launch rsync, run as root, on the house server to use an sshd connection to read changed files from each machine to update files on the USB disk. The files from each machine are backed up to a separate root directory, named to match each host being backed up.3) When done, unmount the USB disk and store ot offline.
When I've needed to recover files from the backup, I've mounted the USB disk on the host containing the master files and copied the files and directories back off the backup disk. I've never had to correct user or group IDs unless I'm restoring to a freshly formatted disk which hasn't yet had user names restored, and even then, putting users and groups back with adduser etc after restoring the files needs no tweaking of file permissions provided I have a record of the corresponding user and group ids and use them when recreating /etc/passwd and /etc/group. I also do an overnight backup from the house server to another USB disk, but this uses rsnapshot and takes about 8 minutes a night plus the same time again for the weekly snapshot, which just shuffles file images on the backup disk.
It looks like the main difference is that I don't mount the backup disk on the host being backed up, which AFAIK has the side effect of completely decoupling the user and group IDs in the two filing systems.
Maybe you could try that?
The rsync SSHD transfer may be a bit slower - can take between 10-40 minutes per host, depending on the size of the weekly distro update, because I keep two generations of backup disk which means that each weekly backup sees two weeks worth of data that needs to be copied across, but at least I have no user/group id issues.
-- Martin | martin at Gregorie | gregorie dot org
Am 06.10.20 um 18:54 schrieb Jim Jackson:
I thought so.
Can you explain this a little mor how to do, please?
-- cu hawe
Am 06.10.20 um 16:20 schrieb Anssi Saari:
The NAS HDDs are ext4. The NFS export settings are set via GUI. Limited to Hosts, rw previleges, no squash, sync, previlegs ports only, Users have access to dirs.
-- cu hawe
...
My homeserver is the NAS. I want to store some files ther periodictly. The NAS is backuped to USB.
That I haver never got working. I try it for some week, but no success.
-- cu hawe
So I guess the next question is, does this happen only with rsync or other programs too if you copy files over?
Am 07.10.20 um 09:43 schrieb Anssi Saari:
I am logging in as admin-user on the client. I mount te nfs manualy and started the Midnight Commander "MC" on the client. I could not change the mode from 777 to anything else. Same when I am via WEBIN connectet to the client.
mkdir -p /mnt/ds9_vmail/test -> same mode is 0777. Group:user root:root and can be changed via chown to anthing eles.
touch /mnt/ds9_vmail/test/test.txt -> same as above.
-- cu hawe
Is it simply down to different UIDs and GIDs on the two systems?
I always try and maintain the same UID and GID across systems on my LAN for this reason, it makes things work more smoothly.
-- Chris Green
Am 07.10.20 um 09:43 schrieb Anssi Saari:
Thank you for hint, solved the problem.
-- cu hawe
Am 06.10.20 um 15:38 schrieb Hans-Werner Kneitinger:
This did the trick.
--- From NAS FAQ Database Applying default UNIX permissions When Apply default UNIX permissions is checked, default UNIX permissions set in the Linux client are applied instead of Windows ACL permissions when uploading or creating files and folders. Applied permissions are the same as permissions applied by the UNIX command umask. The default umask value is 022. Note? For Windows ACL enabled shared folders (all shares excluding "photo" and shares on external drives), please run the chmod command on your Linux or FTP client to change folder and file permission types from Windows ACL to UNIX. Enabling this option might cause inconsistent permission issues between different protocols. To avoid inconsistencies, we suggest leaving this option disabled.
-- Set the switch solved my problem.
Am 07.10.20 um 10:46 schrieb Chris Green:
It is not so easy.
No access on UIDs on the NAS. Not every user on the Client has an account on the NAS.
-- cu hawe
But you're using NFS which means the NFS drive is mounted on the system you're connecting from as if it's a local drive there has to be some sort of strategy for handling UIDs and GIDs. What do you expect/hope it will do?
-- Chris Green
Am 07.10.20 um 12:43 schrieb Chris Green:
The RPis are datacolectors or a control-systems. Every application has its own user:group settings. System user:group differ much between NAS and different distributions. The applicatin itself has no need to access NFS.
user:group whre alwas OK, but note MODE permissios. Mode are 0x0777. After setting the correct NAS NFS switch for LINUX and not Windows ACL. mode, permissons ale correct too.
There is no need to restore the backup to an other distribution. Only when a RPi (or its SD Card) died, I want to setup a fresh system and restore the colected data to the fresh installation. For that I have a RPi in cushion and an image from the every fresh installation.
It works like this:
The script is executed by admin-user and did a rsync -a* on nfs. Its all local, no internet. protected by firewalls, never accessed by public.
user:group are correct shown by name if I look from this client. user:group are correct shown by number if I look direkt from the NAS. Thats OK for my, it is a backup. This Backup, and all other too including the NAS itself, are backuped via NAS Backup Tool to USB Drive connected to NAS.
I don't know better.
-- cu hawe
For me it things went wrong when backing a Raspberry Pi with rsync to a SD card image file on another machine, where the id's for users and system processes were different.
For example on the Raspberry Pi you have user A with id 1001 and user B with id 1002, but on the other machine users were created with ids 1004 and 1005. When rsync backs up it sees files with ids 1001 and 1002 owned by A and B, and 'helpfully' writes then out with correct ids for users A and B on that machine which are 1004 and 1005. The problem comes when you take that backup image and put it back on the original Pi, as it knows nothing about ids 1004 and 1005.
The solution is to use --numeric-ids argument to rsync, then when backing up it will write the same ids as reads, in the case above 1002 and 1003, even those those ids corresponds to completely different users on that machine. But now when you copy the backup image back to the Pi, it has retained the correct ids.
---druck
Am 07.10.20 um 22:22 schrieb druck:
Thank you for explanation but my use-case seems to be little different. Its for backup/recovery only. No need for file sharing between clients, all local only. The NAS is the central backup storage. No RPi user has an account on the NAS. I think there is no need for and its better to have less accounts as possible.
The RPis are data collectors or controllers. I have a backup-image from every fresh setup and an RPi on cushion.
-- cu hawe
ElectronDepot website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.