It’s all about the Objects!

In 2024 Q3, we will be completing our Data Carving Engine, with native support for 100’s of file types, as well as object edge detection support for thousands more.  The end goal is to increase our file types to 5,000+, all with file identification, objects extraction, best in class data carving and malicious objects detection.  Watch out virus scanners, malware detectors, metadata extractors, steganography and data recovery tools.  We’re heading to your domains.

Carving Objects Tab
Data Carving Objects

In 2024 Q1 & Q2, we laid the groundwork for our new Data Carving Engine (name not yet released), and introduced Object Carving. We start by searching for Object Signatures, then index all of the full and partial objects identified. Next, we piece the objects together into full and partial files to be carved. The screen shot (right) shows the Objects tab, as an example of what you would see when carving a Graphics Interchange Format (GIF) image. The Files tab includes the resulting GIF file. The Matches tab shows all signature matches, used and ignored. One challenge in carving at the object level is filtering out the false positives.

We recently wrote the blog post, “The 10 Common Data Carving Approaches“, to provide a base of understanding for us to build upon when we introduce our new methods. While many of the techniques listed are important to include, the current industry solutions implement them like using a sledge hammer to drive in a screw. Just searching for file header and footer signatures, then using trial and error to test the questionable results with interpreters and/or external applications, are blunt instruments. Our methods are more like using surgical instruments to operate around delicate organs. Your data is vital to you, and should be treated delicately as highly valuable assets. Another blog post, detailing Object Carving, will be provided soon.

Objects Viewer
Objects Viewer

During 2023 Q3, we created the ObjectsEngine and connected it to the D3 Objects viewer.  This new engine facilitates the addition of file structures by our engineers as well as Dark Data Detective Advanced Research version owners, and lays the ground work for the upcoming Data Carving, Advanced Metadata Extraction and Malicious Objects Detection.  In future releases, additional file structure database entries will be included internally.  Advanced users may add their own structures, and bypass included structures, by adding their own XML based structure files.  We are also continuing to polish the Dark Data Detective user interface.   Objects are the building blocks of every file. The simplest text file consists of one Text Body object, but files with more structure contain a Header Object, at least one Body Object and perhaps a Footer Object.  More complex files may contain hundreds of objects, of many different types.  For years, we have been empowering our larger customers, with the ability to extract file objects from within their own products.  Now, we are upgrading our Objects Viewer (in the Advanced Researcher tier) to better present the objects in the currently selected file.  There will also be an Object Types Viewer, that acts as a window into our Objects Database.  With improvements to our objects extraction, our Metadata Viewer will also receive an upgrade to extract far more metadata than it does today.