Transfer TB of files in the background via rsync and tmux on linux
Problem
Transfering files in the background via rsync
is possible via nohup
command and some other workarounds.
These don't really work when your transfer takes days (>50TB of data) because the background job might get killed by the system.
In order to create a persistent session we need to use tmux
.
Solution
Authentication
Create a ssh key and upload it to the server you want to transfer from.
ssh-keygen -t ed25519
ssh-copy-id -i ~/.ssh/id_rsa.pub [your_old_server.domain.com]
Create a new session and run rsync
tmux new -s [your_session_name]
rsync --avhPW --stats [your_old_server.domain.com]:[/source/path/] [/destination/path]
Detach from session
Press CTRL+B
and then D
. You session will detach and run in background.
Re-attach session
To get back into your session just:
tmux a -t [your_session_name]
That's it.