Moving to GitHub, slowly

The software on this pages will slowly be moved to GitHub https://github.com/hilbix/. The CVS repository will be migrated to GIT as well, so the history will be preserved, a bit. See FAQ.

Scylla and Charybdis, cmpfast - Tools

The tools are developed under Linux with ESR's paradigm release early, release often in mind.
So you can consider this beta software, or alpha, or pre-alpha, or even worse ;)

Have a look in the download directory for all downloads.
As always here, all you get is the source. No binaries here.

cmpfast 0.2.3-20100330-214523

Fast compare two files binary

⇒ ⇒ ⇒ The development shifted to another location. All future versions will be published at https://github.com/hilbix/cmpfast ⇐ ⇐ ⇐

This is just a binary compare for two files.

Why another program as cmp does the job?

cmp is not very fast when you read two files from the same hard drive. The problem is called "thrashing", as the read heads of the hard drive are moving wildly on the surface and effective read speed drops very low.

cmpfast reads files in bigger blocks (by default 1 MB), thus reducing thrashing a lot. It also switches sides to double the continuously read blocks, thus speeding up fastcmp again. On modern fast hard drives you still hear a thrashing sound, this is because you still have 30 to 40 head movements per second to compare 30 MB or more per second. It would be not easy to go below this number, as it then needs a highly filesystem optimized low level access which is unlikely to be possible. So even cmpfast is very simple it's not easy to speed up more, even not if you raise the compare buffer size. 1 MB should be a good size for all possibilities.

Cmpfast has following "standard" return values now:
* 0 (true) if everything ok (compares equal)
* 1 for usage etc.
* 10 if files differ in contents
* 101 if EOF on first file
* 102 if EOF on second file
* other return values (probably 255) on misc errors (like missing files)

History:

version 0.2.3-20100330-214523

download (254028 bytes) sig

Bugfix:

Sometimes EOF was not detected correctly due to rounding problems. Usually instead of EOF a byte diff was detected. (If not the compared file probably was constructable from the previous reads somehow.)

Read: Previous versions DID NOT WORK correctly!

version 0.2.2-20081020-030200

(227107 bytes archive)

O_DIRECT now skipped on filesystems lacking O_DIRECT support.

If open() fails with O_DIRECT, it is retried without. Some filesystems (like FUSE) apparently do not support O_DIRECT.

version 0.2.1-20081020-012928

(227007 bytes archive)

O_DIRECT for open() lessens buffer cache impact

Another speedup fix: With O_DIRECT the kernel hopefully uses no buffer cache for the data, which lessens the memory pressure of cmpfast. On some systems it might be that this slows down the performance of cmpfast itself, however then it reduces overall system load, which is desireable.

version 0.2.0-20080513-175041

(211105 bytes archive)

Better scriptable return values

version 0.1.0-20080423-055959

(210292 bytes archive)

Option -v added to show progress

version 0.0.1-20070926-051223

(190608 bytes archive)

Minor bug removed: Now cmpfast uses less memory in most cases.

Also version of tinolib was updated.

version 0.0.0-20070421-020201

(170630 bytes archive)

First rough version.

In this version I started to provide error tags. However the output format will change in future when error tags are completely supported by tinolib.

License and Disclaimer

All you can see here is free software according to the GNU GPL.
Copyright (C)2000-2011 by Valentin Hilbig
Note that the software comes with absolutely no warranty of any kind.
You use the software at your own risk.
Valentin Hilbig cannot be hold responsible for any unintended damage,
lost data or malfunction of the software you can find here.

Last modified: 2011-09-12 by Valentin Hilbig [ Imprint / Impressum ]