How To Synchronize Files And Directories Using Zaloha.sh
Zaloha.sh is a directory synchronizer that stands out by its small size and simplicity. It is actually a BASH script. It consists of only one file, Zaloha.sh, whose size is 125 kB. Approximately half of that file is documentation, the other half is program code. Zaloha.sh works out-of-the-box. However, as a shell script, it is easily adjustable to create eventual non-standard solutions out of it. Using Zaloha, we can easily synchronize files and directories on Linux, Unix and Microsoft Windows.
Please note that Zaloha.sh is not as feature-rich as competing backup solutions, but those competing solutions are also bigger, more complex and black-box-like. Internally, Zaloha.sh uses only standard Unix commands, such as find, sort, awk, mkdir, rmdir, cp and rm, to make it work. Zaloha.sh works well natively under Linux, Unix and Windows (Cygwin is required, of course).
Install Zaloha.sh Directory Synchronizer
Installing Zaloha is trivial. Just download the file Zaloha.sh from the GitHub repository and move it to your $PATH, for example /usr/local/bin/.
Git clone Zaloha repository and move Zaloha.sh file to your $PATH using commands:
$ git clone https://github.com/Fitus/Zaloha.sh.git
$ sudo mv Zaloha.sh/Zaloha.sh /usr/local/bin/zaloha.sh
Finally, make it executable:
$ sudo chmod +x /usr/local/bin/zaloha.sh
On windows, download the script to a suitable local directory. I chose to put it to directory C:\TOOLS.
On Windows, as an additional step, CygWin must be installed. I already have CygWin installed on my Notebook. If you haven’t install CygWin yet, please refer the official installation instructions.
Synchronize Files And Directories On Linux Using Zaloha.sh
The typical usage of Zaloha.sh is as follows.
$ zaloha.sh --sourceDir=<path_to_dir> --backupDir=<path_to_dir>
Let me show you a simple example.
I am going to create two directories namely test1 and test 2.
$ mkdir test1 test2
I have two text files called file1 and file2 on test1 directory.
$ ls test1/ file1 file2
And test2 directory is empty.
Now, I am going to synchronize test1 and test2 directories using Zaloha:
$ zaloha.sh --sourceDir=test1/ --backupDir=test2/ --color
Press “y” to continue.
ANALYZING ./test1/ AND ./test2/ =========================================== Parsing .............................................. done. find ./test2/.Zaloha_metadata/ -path ./test2/.Zaloha_metadata/999_mark_executed -printf '///\tL\t%y\t%s\t%Ts\t%F\t%D\t%i\t%n\t%u\t%g\t%m\t%P\t///\t%l\t///\n' find ./test1/ -path ./test1/.Zaloha_metadata -prune -o -ipath './test1/$RECYCLE.BIN' -prune -o -path './test1/.Trash-[0-9]*' -prune -o -path ./test1/lost+found -prune -o -printf '///\tS\t%y\t%s\t%Ts\t%F\t%D\t%i\t%n\t%u\t%g\t%m\t%P\t///\t%l\t///\n' find ./test2/ -path ./test2/.Zaloha_metadata -prune -o -ipath './test2/$RECYCLE.BIN' -prune -o -path './test2/.Trash-[0-9]*' -prune -o -path ./test2/lost+found -prune -o -printf '///\tB\t%y\t%s\t%Ts\t%F\t%D\t%i\t%n\t%u\t%g\t%m\t%P\t///\t%l\t///\n' Cleaning ............................................. done. Checking ............................................. done. Sorting (2) .......................................... done. Differences processing ............................... Zaloha AWK: Warning: No last run of Zaloha found (this is OK if this is the first run) done. Sorting (3) .......................................... done. Post-processing and splitting off Exec1 and Exec4 .... done. Sorting (4) and selecting Exec2 ...................... done. Preparing shellscript for Exec1 ...................... done. Preparing shellscript for Exec2 ...................... done. Preparing shellscript for Exec4 ...................... done. Preparing shellscript to touch file 999 .............. done. Preparing shellscripts for case of restore ........... done. TO BE COPIED TO ./test2/ =========================================== NEW file1 NEW file2 Execute above listed copies to ./test2/ ? [Y/y=Yes, other=do nothing and abort]: y cp --preserve=timestamps ./test1/file1 ./test2/file1 cp --preserve=timestamps ./test1/file2 ./test2/file2 TO BE REMOVED FROM ./test2/ ===========================================
As you can see in the above output, Zaloha uses the standard Unix commands – find, awk, sort and cp.
Let us check if the files have been synchronized or not.
$ ls test1 file1 file2
$ ls test2 file1 file2
Well, it works!!
Synchronization to USB flash drive
The following steps have been tested on a Windows 10 machine, however the procedure is same for all operating systems.
I have all my work files on the notebook concentrated in a directory structure under C:\WORK_DATA. Currently it contains several thousands files of diverse types.
I regularly backup that directory to a (pretty old) 2 GB USB flash drive. When I plug that USB flash drive into my notebook, it appears as drive G. The backup directory on the USB flash drive is G:\WORK_BACKUP.
So, now the actual test:
Recently I have been on a meeting with a customer and created a Word document Meeting_Notes_0109.docx from it. I have also amended an offer to that customer, which is Excel spreadsheet offer_01.xlsx.
So the expectation is that Zaloha.sh performs the backup of these two files, leaving the other files untouched.
Let’s run Zaloha.sh to see:
$ /c/TOOLS/Zaloha.sh --sourceDir="/c/WORK_DATA" --backupDir="/cygdrive/g/WORK_BACKUP" --color
It can be seen that Zaloha.sh runs Unix find on both directories C:\WORK_DATA (under CygWin /c/WORK_DATA) and G:\WORK_BACKUP (under CygWin /cygdrive/g/WORK_BACKUP), then it performs several processing steps and finally it presents what it is going to do, asking for confirmation.
The file Meeting_Notes_0109.docx is tagged as NEW (in black color), and the file offer_01.xlsx is tagged as UPDATE (in red color). The red color of the UPDATE tag alerts the user that this operation needs more attention, as old data in the backup directory will be overwritten.
After confirming the prompt with a “y”, Zaloha.sh performs the presented operations.
If we now, out of curiosity, run the above command again, we can see that Zaloha.sh does the analysis steps, and then does nothing, as the directories are already synchronized.
Let’s now test another feature of Zaloha.sh, a “reverse synchronization”.
Assume I had, for some reason, to work directly on the USB flash drive and I want my work to get reverse synchronized back to the notebook.
This feature must be explicitly enabled by the –revNew and –revUp options.
The –revNew option instructs Zaloha.sh to copy to source directory (the notebook) files on backup directory (the USB drive) which exist only on the backup directory and are newer than the last run of Zaloha.sh.
The –revUp option instructs Zaloha.sh to copy to source directory (the notebook) files on backup directory (the USB drive) which exist in both locations and the newer files are in the backup directory.
Let’s now test it. I go to the USB flash drive (i.e G:\WORK_BACKUP\IKT\2020) and create a new Word document named Meeting_Notes_0113.docx in there. Next, I open Excel on the file offer_01.xlsx, do some editing and save it.
Now let’s run Zaloha.sh using the following command line:
$ /c/TOOLS/Zaloha.sh --sourceDir="/c/WORK_DATA" --backupDir="/cygdrive/g/WORK_BACKUP" --color --revNew --revUp
It can be seen that Zaloha.sh indeed reverse synchronized Meeting_Notes_0113.docx and offer_01.xlsx back to the notebook. Again the REV.UP tag of offer_01.xlsx is displayed in red color, alerting the user that data is going to be overwritten.
What should be noted further: The detection of files that need synchronization is based on file modification times and file sizes, which is a usual method used by many synchronization tools. However, Zaloha.sh is able to perform byte by byte comparison as well. This can be activated by a command line option.
On file systems that support hardlinks, Zaloha.sh is able to de-duplicate hardlinks and copy the hardlinked file only once to the backup directory. Again, this can be activated by a command line option.
Zaloha.sh contains several other options to modify its function. All are described in the documentation. The documentation also devotes a decent amount of text to description of the internal workings of Zaloha.sh. One important item to be mentioned here is that Zaloha.sh creates a working directory .Zaloha_metadata under the backup directory, where it keeps its temporary and metadata files.
Getting Zaloha.sh to work is indeed easy, especially on Linux. Together with its easy adjustability due to being a shell script, Zaloha.sh might deliver added value in several imaginable situations. For more details, refer Zaloha GitHub repository given below. The repository contains, besides Zaloha.sh itself, a simple demo test case, a screenshot, and online documentation. The online documentation is the same as the documentation within Zaloha.sh, but the online version is in HTML format and is nicer.
Hope this helps.