The software on this pages will slowly be moved to GitHub https://github.com/hilbix/. The CVS repository will be migrated to GIT as well, so the history will be preserved, a bit. See FAQ.
The tools are developed under Linux with
release early, release often
So you can consider this beta software, or alpha, or pre-alpha, or even worse ;)
Have a look in the download directory for all downloads.
As always here, all you get is the source. No binaries here.
Create md5sums for easy shell usage
⇒ ⇒ ⇒ The development shifted to another location. All future versions will be published at https://github.com/hilbix/md5chk ⇐ ⇐ ⇐
Sometimes you want to pipe a list of files to md5sum. This then looks like
find . -type f -print0 | xargs --null md5sum -b -- | while read a b do ...
The problem is that the $b then starts with a * and you don't know what happens if the filename contains nonstarndard characters like LF. Some md5sum dialects just create garbage, others escape the MD5 sum.
md5chk doesn't do this and was designed from ground up for bash usage. It guarantees that the md5 sum always is "clean" and the filename always is properly escaped for shell reads. Note that there is no way for shell reads to express the newline character nor spaces at the end of an argument. So an unescape via ANSI strings must be done, sorry.
find . -type f -print0 | md5chk -n | while read -r md5 name do eval name="\$'$name'" ...
Note that the "eval" line does the escape entirely runs in the shell and thus is very fast.
Another common usage is "create md5 string from a file":
md5="`md5chk -iq "$name"`"; echo $?
Now you can create an md5sum from argument directly, too:
md5="`md5chk -dq "$string"`"
Bugfix release and options -k added
Now large files (more than 2 GB) can be processed, one of my machines exposed that bug to me.
Checksumming a large file with options -l and -m I observed the problem, that long parts of NUL blocks always have an equal sum. So the interleaving still does not protect against wrong ordering.
The complete sum will detect this, however this leaves a big puzzle on how to assemble all the pieces such that they are ok again. This now is solved with the new option -k.
By mixing in the block number into the MD5 hash, the hashes become unique, such that - with interleaving - you can sort all block such that they fit where they belong. (It's still a nontrivial task, but can be done.)
Option -k is followed by Option -p.
Options -c, -l and -m added, install path now /usr/local/bin/
If you have a very big fiile the MD5 sum can be broken up into parts, say each 10K one MD5 sum is taken. If set to 0 means all the full file.
Output is then like FIRST+SECOND+LAST=COMPLETE
With option -l the -m is overlapping, that is each output MD5 covers two blocks instead of one.
Output then is like FIRST-FIRSTSECOND-SECONDLAST+LAST=COMPLETE
Also new is option -c, with this you can do an MD5 of data which goes throgh a pipe.
Added option -p (and --shit)
Option -p initalizes the MD5 algorithm with some arbitrary string. This is to modify the md5 algorithm, to change calculated values. For example 'md5chk -p a -d b c' gives the same as 'md5chk -d ab ac', but the prefix is not shown in the output.
The new integration option --shit is prepared, but not yet operational. (Sorry for the name, but I needed a silly option name which will not interfere with another option which already is in use.) Before you ask: SHIT will provide a built in co-processing method like shell pipes. However it will skip many fork() calls which are very slow platforms like CygWin. It also will work over network sockets.
Bugfix, DEL now is an escaped character, too.
Options in help output sorted, too
(210153 bytes archive)
Minor bug removed and option -z added
char can be unsigned on some platforms. However I doubt this code will it ever make to such platforms ever ;)
Option z is to write NUL terminated lines (companion to reading NUL terminated lines).
|[view more history]
|[view complete history]
License and Disclaimer
All you can see here is free software according to the GNU GPL.