Interface description for a privat magic pattern file for ext4magic.0.3.2
SYMPTOMS
ext4magic has strong dependencies to the libraray libmagic from the FILE command.
With this library, the data blocks of deleted regions on the file system are searched for possible file beginnings.
These functions often called many thousand times during an attempt to recover a file system with the magic-function.
But not only for finding file beginnings is this function important, indirectly this is also helpful for finding the end of some files and the accuracy of revocered files.
Primary, the magic-function has been developed and optimised for file-5.04.
The FILE project is a very active project and many users submitted many changes and enhancements in the meantime.
Thereby, in the meantime are 15 additional FILE versions published.
The file descriptions are more precisely from FILE version to FILE version, it will be added more and more new and often rare and not typical Linux files, and the file signature database is getting bigger. This is good for the user who uses the command file and thus always achieved better results there. But that's bad for ext4magic because the functions must be applied to very many raw data blocks of the file system.
While browsing the deleted data bocks many file beginnings are reported, they are often wrong and come from poorly-defined file signatures which are unknown for ext4magic and therefore can not catch. Therefore it comes from time to time to the demolition of a recover of a known file. Thus increased the cut off and erroneous restored files and thus to more data blocks need to be investigated by libmagic. The result of the magic function is associated with each FILE different version, and the number of erroneously recoverter files is often increased in new versions and the magic-function will always be slower.
The development of ext4magic does not have enough power and can not follow the rapid development of file signatures in the FILE versions and ext4magic require not all signatures on a typical user desktop Linux. A possible solution is a own small signature database for ext4magic.
The interface to this is added in version 0.3.2
INTERFACE
The interface is a magic pattern file named /usr/share/misc/ext4magic or /usr/local/share/misc/ext4magic (search in this order)
The format of this file is the same as used by the file(1) command and is described in magic(5).
A good idea is to generate additional a precompiled magic file named of this pattern file. (not accepted is only the precompiled file)
When starting ext4magic and the option "-m" or "-M", then this pattern file or the precompiled file is used as a data base for the Magic-function of ext4magic. You can see a notice then in the first lines of output from ext4magic
use magic-db on "/usr/share/misc/ext4magic"
If the file is corrupted, or is not accepted for another reason, then aborts ext4magic the processing when starts the Magic3 function.
"ERROR: can't find libmagic"
This ext4magic private magic pattern file should match the syntax of the FILE version used by the system. The precompiled pattern file should also be created with the same version of the FILE command.
Generating your own pattern file
It is planned to create an optimized pattern file with a syntax that works in all versions FILE and maintain these in ext4magic project. For this would be some optimization and fine tuning to exactly this pattern file in the next version ext4magic. At the moment you can experiment with a own made pattern file, proceed as follows.
# check your file version
file -v
# downloade the source of this version from ftp://ftp.astron.com/pub/file/
# unpack the source
# cd to the magic directory at the source
cd file-5.??/magic/
# cat the pattern files and write this to /usr/local/share/misc/ext4magic
mkdir -p /usr/local/share/misc/
cat Header Localstuff Magdir/* > /usr/local/share/misc/ext4magic
# precompile
cd /usr/local/share/misc/
file -C -m ext4magic
you should now see there 2 files "ext4magic" and "ext4magic.mgc" and ext4magic whis option -m should use these.
A thus generated pattern file for ext4magic will not bring benefits. They will typically include the same as those of the system.
ext4magic not need all pattern of magic/Magdir/ contained in FILE source. The development of ext4magic has tested a file with about half of the files there.
The result was similar as the complete list used. But the speed of magic-function was significantly better.
Only the following file from the magic/Magdir/ directory used in first tests:
acorn, allegro, amanda, animation, archive, audio, blender, btsnoop, c-lang, cafebabe, cddb, citrus, claris, commands, compress, console, database, diamond,
diff, dolby, dump, editors, efi, elf, epoc, filesystems, flash, fonts, fortran, frame, freebsd, games, gcc, gimp, gnome-keyring, gnu, gnumeric, grace,
graphviz, iff, images, intel, island, java, jpeg, kde, kml, lex, linux, lisp, llvm, lua, luks, mach, magic, mail.news, mathcad, mathematica, matroska,
mcrypt, mime, mirage, misctools, mozilla, msdos, msvc, netbsd, netscape, ole2compounddocs, pbm, pdf, perl, pgp, pkgadd, printer, python, revision, riff, rpm,
rtf, ruby, sc, securitycerts, sgml, sniffer, sql, tex, tgif, timezone, troff, unicode, uuencode, virtual, vmware, vorbis, windows, wordprocessors, xwindows
Caution: If the source code is compiled, there is also a file named magic.mgc.
This is a precompiled file from all the files there and should not be included.
This you can take as a starting point for your own experiments.
The pattern there are still partly much too large in these orginal pattern files, and can still be shortened and thus probably much better results for ext4magic be achieved.
Disable the function
Delete the files "ext4magic" and "ext4magic.mgc" in directory /usr/share/misc/ or /usr/local/share/misc/
ext4magic then use the normal system magic pattern configuration of the file command.