A comprehensive, practical guide to file synchronization β from rsync and Robocopy to cloud sync, real-time watchers, scheduled syncs, two-way strategies, and secure network transfers.
File synchronization is the process of keeping two or more copies of a set of files identical, no matter where they live. Whether you work across a laptop and a desktop, maintain a fleet of servers, hand off project folders between teammates, or back critical business data up to an off-site location, the ability to synchronize files reliably is one of the most useful skills a modern computer user can build. A good sync workflow prevents data loss, resolves version conflicts predictably, saves bandwidth by transferring only what changed, and automates work that would otherwise consume hours of manual copying.
Unlike a one-time copy, synchronization is designed to run repeatedly. Each run compares the source and destination, determines what has changed since the last run, and reconciles those differences according to clearly defined rules. Depending on the tool you choose, synchronization can be one-way (mirror a source to a destination), two-way (bidirectional, with both sides contributing changes), continuous (real-time, as files change), or scheduled (on a fixed interval such as hourly or nightly).
In this guide you will learn when to use each approach, how to configure the leading tools on Windows, macOS, and Linux, how to automate sync jobs with cron, systemd timers, and Task Scheduler, and how to secure your synchronization over SSH, VPN, or TLS. You will also see concrete command examples, comparison tables, folder-tree diagrams, and troubleshooting recipes for the problems you will inevitably hit in production.
Before you build an automated sync, agree on which side is authoritative. The hardest sync bugs come from unclear ownership β knowing which copy wins a conflict is a design decision, not a tool decision.
These three concepts sound similar and are often confused, yet they solve different problems. A plain copy is a one-shot duplication: run it today, run it again tomorrow, and it rewrites everything regardless of whether anything changed. A sync intelligently transfers only differences, keeps the destination structurally identical to the source, and can delete files on the destination when they are deleted on the source. A backup adds a time dimension β it preserves historical versions so you can recover a file as it existed last week even if it has since been changed or deleted.
| Operation | Transfers | Deletes on target? | Keeps history? | Typical use |
|---|---|---|---|---|
| Copy | All files | No | No | Initial seed, one-shot transfer |
| Mirror sync | Only changes | Yes | No | Match a target to a source exactly |
| Two-way sync | Only changes, both ways | Optional | Rarely | Multiple active editing locations |
| Backup | Only changes | Never (archives) | Yes (versions) | Disaster recovery, legal retention |
The practical implication: if you need protection from accidental deletion or ransomware, do not rely on sync alone. Pair your sync with a versioned backup (restic, Duplicati, Veeam, Time Machine, or a snapshot-capable filesystem like ZFS or Btrfs). Sync keeps the current state consistent; backup lets you travel in time.
A mirror sync will cheerfully propagate deletions and corrupted files to your target. If the source is attacked or the drive fails after the sync runs, both sides are gone. Always have an independent backup.
rsync is the de-facto standard for file synchronization on Unix-like systems and, via WSL, Cygwin, or MSYS2, on Windows as well. Its killer feature is the delta-transfer algorithm: for each file that already exists on the destination, rsync compares block checksums and transfers only the parts that have changed. For a 5 GB virtual machine image where a few megabytes have been modified, rsync may transfer only those modified blocks instead of the entire file.
rsync [options] source destination # Local sync β mirror /data/projects to an external drive rsync -avh --delete /data/projects/ /mnt/backup/projects/ # Remote sync over SSH β push to a server rsync -avzh --delete -e ssh ./site/ user@server:/var/www/site/ # Dry run first β see what would change without touching anything rsync -avh --delete --dry-run /data/projects/ /mnt/backup/projects/
The trailing slash matters. /data/projects/ means the contents of that directory; /data/projects (no slash) means the directory itself. Misusing the slash is the single most common rsync mistake β it ends up nesting folders inside folders.
| Option | Meaning |
|---|---|
-a | Archive mode β preserves permissions, timestamps, symlinks, owners |
-v | Verbose, show each file as it transfers |
-z | Compress data during transfer (great for slow links) |
-h | Human-readable sizes |
-P | Show progress and keep partial files to resume |
--delete | Remove files on destination that no longer exist on source |
--exclude PATTERN | Skip files or folders matching the pattern |
--dry-run | Show what would happen without doing it |
--bwlimit=KBPS | Throttle bandwidth so you don't saturate the network |
--checksum | Compare by checksum rather than size and mtime |
rsync -avzhP --delete \ --exclude='.git/' \ --exclude='node_modules/' \ --exclude='*.log' \ --exclude='.DS_Store' \ -e 'ssh -i ~/.ssh/deploy_key -p 2222' \ ./myapp/ [email protected]:/srv/myapp/
Here -e specifies the exact SSH command, including a custom key and non-standard port β ideal for production deployments. The --exclude patterns prevent development artifacts from being pushed to the server.
Always run --dry-run at least once when you add --delete. One misplaced slash can wipe an entire directory tree. The dry run previews deletions clearly before you commit.
Scheduled syncs are fine for most workloads, but some use cases demand instant propagation β think of a developer editing a file on a laptop and expecting it to appear on a remote test VM within milliseconds, or an operator who wants new photos pushed off a camera memory card as soon as they land. For these cases you need file-system event watchers that trigger a sync as soon as a change is detected.
# Install sudo apt-get install inotify-tools # Watch a folder and rsync on every change while inotifywait -r -e modify,create,delete,move ./src/; do rsync -avz --delete ./src/ user@server:/srv/app/src/ done
brew install fswatch fswatch -o ./notes | while read num; do rsync -avz ./notes/ [email protected]:/volume1/notes/ done
Facebook's Watchman and the elegant Syncthing project both provide cross-platform file watching with higher-level APIs. Syncthing in particular ships as a complete peer-to-peer sync daemon with a web UI, conflict resolution, versioning, and end-to-end encryption; it is often the fastest way to get real-time sync between two or more machines without configuring servers.
Watchers trigger on every save, including auto-saves and editor swap files. Debounce your sync (wait 1β2 seconds of quiet before firing) and exclude temp files like *.swp, .#*, and *~ to avoid storms.
Robocopy ("Robust File Copy") ships with every modern Windows release and is the preferred tool for synchronizing folders on the Windows platform. It understands long paths, retries transient errors, preserves ACLs and timestamps, supports multi-threading, and offers a true mirror mode. Despite the name, Robocopy is a full-featured sync tool, not just a copier.
# Mirror C:\Projects to D:\Backup\Projects (deletes files removed from source) robocopy C:\Projects D:\Backup\Projects /MIR /R:3 /W:5 /MT:16 /LOG:C:\Logs\sync.log /TEE # Sync to a network share robocopy C:\Projects \\fileserver\Share\Projects /MIR /R:3 /W:5 /Z # Preserve NTFS permissions and attributes exactly robocopy C:\Projects D:\Backup\Projects /MIR /COPYALL /DCOPY:DAT
| Switch | Purpose |
|---|---|
/MIR | Mirror β identical to /E /PURGE, matches destination to source exactly |
/E | Copy subfolders including empty ones |
/COPYALL | Copy ALL file info (data, attributes, timestamps, security, owner, auditing) |
/MT:N | Multi-threaded copy with N threads (default 8) |
/R:n /W:s | Retry n times, wait s seconds between retries |
/Z | Resumable mode β restart partial transfers |
/LOG:file | Write output to log file |
/TEE | Show output on console AND write to log |
/XD /XF | Exclude directories or files by pattern |
/L | List only β the Robocopy equivalent of a dry run |
Robocopy's exit codes are bit-flags, not conventional 0-for-success. Values 0, 1, 2, and 3 are all successful outcomes; only 8 and above indicate real failures. If you call Robocopy from a scheduled task or script, check for %ERRORLEVEL% GEQ 8 rather than %ERRORLEVEL% NEQ 0.
Cloud sync services turn file synchronization into a background utility: install an agent, point it at a folder, and the service continuously replicates changes to and from the cloud, which then propagates to every other device on the same account. The big three on Windows and macOS are OneDrive, Google Drive, and Dropbox; all three handle offline edits, conflict copies, selective sync of subfolders, and shared folders between users.
C:\Users\jane\ βββ OneDrive\ β βββ Documents\ β synced to cloud + other devices β βββ Pictures\ β βββ WorkProjects\ βββ Projects\ β local only, rsync to NAS nightly β βββ client-alpha\ β βββ client-beta\ βββ Scratch\ β excluded from all sync
For scripted or server-side cloud sync, rclone supports more than 70 cloud providers (S3, Backblaze B2, Google Drive, OneDrive, Dropbox, Azure Blob, Wasabi, SFTP, WebDAV, and many more) with a single command-line interface. It is to cloud storage what rsync is to filesystems.
# Configure a remote (interactive) rclone config # One-way sync local -> cloud (mirror; deletes on remote) rclone sync ./site s3:my-bucket/site --progress # Copy without deleting rclone copy ./photos gdrive:Backup/Photos --progress # Mount a cloud bucket as a local drive rclone mount s3:my-bucket /mnt/s3 --daemon
Consumer cloud sync is not a backup. If you delete a file locally, it is deleted in the cloud. Enable file versioning on your cloud account (OneDrive, Google Drive, Dropbox all support it) or pair the sync with a separate archival backup.
A manual sync is useful once; an automated sync is useful forever. Every major operating system ships a scheduler that can run your sync command on a cadence of your choosing β every 15 minutes, every night at 2 AM, every Monday at boot, and so on.
# Edit your crontab crontab -e # Every 15 minutes */15 * * * * /usr/bin/rsync -az /data/ user@backup:/srv/backup/ >> /var/log/sync.log 2>&1 # Nightly at 02:30 30 2 * * * /usr/bin/rsync -az --delete /data/ user@backup:/srv/backup/ # Every Monday at 06:00 0 6 * * 1 /usr/local/bin/weekly-sync.sh
# /etc/systemd/system/datasync.service [Unit] Description=Sync data to backup [Service] Type=oneshot ExecStart=/usr/bin/rsync -az --delete /data/ user@backup:/srv/backup/ # /etc/systemd/system/datasync.timer [Unit] Description=Run datasync hourly [Timer] OnCalendar=hourly Persistent=true [Install] WantedBy=timers.target # Enable it sudo systemctl daemon-reload sudo systemctl enable --now datasync.timer
# Create a daily 2 AM robocopy task via schtasks schtasks /Create /SC DAILY /TN "NightlySync" /ST 02:00 /RL HIGHEST ^ /TR "robocopy C:\Projects D:\Backup\Projects /MIR /R:3 /W:5 /LOG:C:\Logs\sync.log" # Or in PowerShell with ScheduledTasks module $action = New-ScheduledTaskAction -Execute 'robocopy.exe' ` -Argument 'C:\Projects D:\Backup\Projects /MIR /R:3 /W:5' $trigger = New-ScheduledTaskTrigger -Daily -At 2am Register-ScheduledTask -TaskName 'NightlySync' -Action $action -Trigger $trigger -RunLevel Highest
One-way sync is easy: the source wins, every time. Two-way sync is harder because both sides can diverge between runs. If Alice edits a file on her laptop while Bob edits the same file on the shared NAS, what should happen? There are only three honest answers: keep Alice's version, keep Bob's version, or keep both as conflict copies and let a human decide.
| Tool | Model | Conflict handling | Best for |
|---|---|---|---|
| Unison | Pairwise, run on demand | Prompts user; keeps both | Laptop β desktop, controlled syncs |
| Syncthing | Continuous peer-to-peer | Keeps both as .sync-conflict-* | Multiple devices, always-on sync |
| Resilio Sync | Continuous P2P (commercial) | Conflict copies | Teams, large files |
| rclone bisync | Scheduled bidirectional | Keeps newer or both | Cloud β local bidirectional |
# ~/.unison/laptop-desktop.prf root = /Users/jane/Documents root = ssh://[email protected]//Users/jane/Documents auto = true batch = true prefer = newer ignore = Name {.DS_Store,*.tmp,.git} # Run it unison laptop-desktop
Never use --delete on both sides of a two-way sync without a safety net. A race between two runs can turn "file deleted on one side" into "file deleted everywhere" β exactly the ransomware behaviour you are trying to defend against.
Syncing across the network multiplies your options: SMB and NFS let you treat a remote folder as if it were local; SSH tunnels let you carry rsync safely over untrusted networks; VPNs give you LAN-like access from anywhere; and dedicated sync protocols (Syncthing, Resilio) handle NAT and discovery for you.
# Mount the share sudo mkdir -p /mnt/winshare sudo mount -t cifs //windowspc/Share /mnt/winshare \ -o username=jane,password=secret,uid=$(id -u),gid=$(id -g),vers=3.0 # Sync to it rsync -av --no-perms --no-owner --no-group /data/ /mnt/winshare/data/
# On server β /etc/exports /srv/data 192.168.10.0/24(rw,sync,no_subtree_check) sudo exportfs -ra sudo systemctl enable --now nfs-server # On client sudo mount -t nfs server:/srv/data /mnt/data rsync -av /local/project/ /mnt/data/project/
rsync -avz -e 'ssh -J jump.example.com' \ ./backups/ [email protected]:/srv/backups/
Laptop (Windows) ββSMBβββ
ββββΊ NAS (Linux)ββrsync+SSHβββΊ Off-site backup server
Desktop (macOS) βSyncthingβ
β
ββββΊ rclone βββΊ S3 bucket (cold archive)
A sync tool happily transfers every secret you own β API keys, databases, customer data, source code, personal photos. Treat your sync pipeline with the same care you would a production application.
authorized_keys command and no shell access.crypt backend, restic, or age/gpg before upload.rsync -c (checksum compare) or shasum/sha256sum on sampled files to catch silent corruption.# On the backup server, in ~/.ssh/authorized_keys command="rrsync -wo /srv/backup",no-pty,no-agent-forwarding,no-port-forwarding ssh-ed25519 AAAA... backup@laptop
This limits the key to running rrsync (restricted rsync) in write-only mode under /srv/backup β if the key is stolen, the attacker still cannot open a shell, read other files, or escape the backup directory.
Pair integrity verification with a canary file: write a file with a known checksum every sync and verify it on the destination. Any mismatch means something between the two endpoints is lying.
Sync problems are usually one of four shapes: permission errors, partial or incomplete transfers, locked or in-use files, and conflicting changes. Here is a diagnostic recipe for each.
sudo if syncing system paths, or use rsync --no-perms --no-owner --no-group when crossing to SMB/FAT targets that cannot store POSIX permissions./COPYALL and when crossing user profiles.Flaky networks, VPN drops, or sleeping laptops leave half-copied files behind. Use rsync --partial --append-verify (bundled as -P) and Robocopy's /Z to resume rather than restart. For scheduled tasks, add a retry wrapper that re-runs the sync on failure.
Databases, mailboxes, and open Office documents are often locked by their owner process. Rather than copying the live file, use the application's own snapshot or export (e.g., mysqldump, VSS on Windows, sqlite3 .backup) and sync the resulting snapshot. On Windows, Robocopy with /B (backup mode) can bypass some locks if you have the Backup Operator privilege.
# rsync β maximum verbosity + stats
rsync -avh --stats --itemize-changes --dry-run SRC/ DST/
# Compare two folders by checksum
diff -r <(cd A && find . -type f -exec sha256sum {} \;) \
<(cd B && find . -type f -exec sha256sum {} \;)
# Robocopy β log and list only
robocopy SRC DST /MIR /L /LOG:preview.log /NFL /NDL
# PowerShell β spot files changed in the last hour
Get-ChildItem C:\Projects -Recurse -File |
Where-Object { $_.LastWriteTime -gt (Get-Date).AddHours(-1) }If timestamps on the destination look wrong, suspect clock skew. rsync's modification-time comparison breaks when the two machines disagree by more than a second. Install NTP/chrony on both sides.
A reliable file synchronization workflow is not about picking the "best" tool β it is about matching the right tool to each job and layering safeguards so a single mistake cannot destroy your data.
--delete or /MIR.With these habits in place, synchronization becomes a quiet, boring part of your infrastructure β exactly what it should be. The files you need are where you expect them to be, the moment you expect them to be there, with no surprises when it matters most.
Start simple: one rsync or Robocopy command, run manually, in dry-run mode. Promote it to a scheduled job only after you have watched it succeed by hand several times. Complexity is earned, not assumed.