AIMS@SAA Part One: CREW Workshop

CREW: Collecting Repositories and E-Records Workshop
SAA 2011
Chicago, IL 8/23/2011

The AIMS partners hosted a workshop in the run-up to the 2011 SAA Annual Meeting in August. 45 participants from the US and Canada joined us in exploring the challenges, opportunities and strategies for managing born-digital records in collecting repositories.

The workshop was organized around the 4 main functions of stewardship that the AIMS project has focused on: Collection Development, Accessioning, Arrangement and Description, and Discovery and Access. In addition to the AIMS crew (no pun intended) presenting on the research done through the AIMS project, several guest presenters showcased case studies from their own hands-on approaches to managing born-digital materials. Seth Shaw, from Duke University discussed the evolution of electronic record accessioning at Duke University and his development of the Duke Data Accessioner. Gabriela Redwine discussed work done in arrangement and description at the Harry Ransom Center at the University of Texas at Austin. Finally, Erin O’Meara showcased work done at the University of the North Carolina at Chapel Hill to facilitate access to born-digital records through finding aid interfaces.

In between presentations the participants engaged in lively discussions around provocative questions and hypothetical scenarios. At the end of the event, the AIMS partners felt they had gained just as much from the day’s activities as they hoped the participants had. Ideas that were discussed and case study examples will help strengthen the findings of the white paper due out this fall.

See the workshop presentations after the jump! 

Forensic Workstation pt3

A guest posting from Nicola Herbert, Digital Project Preservation Assistant at Hull University Archives

Once we had the forensic workstation up and running (see part 1 and part 2 in this on-going series) we installed MS Office and Mozilla Thunderbird (for working with Outlook .pst files). We also installed FTK Imager, Karen’s Directory Printer, DROID and the MUSE e-mail visualisation tool (in beta, but provides a very interesting perspective on the data). We are also planning to purchase Quickview Plus, a piece of software that enables viewing a range of file formats without requiring the original software on your PC.

We had already played around with these tools on our normal PCs and had run them on files copied from digital media prior to setting up the workstation.

Having received our two Tableau write-blockers we were eager to combine the separate processes we had developed into an integrated workflow. We have two write-blockers, one for USB devices (T8-R2) and one for internal hard drives from PCs and laptops (T35es). Simon’s visit to Jeremy John at the British Library had whetted our appetite for getting our mini digital forensics lab in operation.

USB devices
After a thorough read-through of the instructions we tested out the USB write-blocker first. Setting it up is relatively simple; the vital thing is to make the connections between device and write-blocker, write-blocker and forensic PC before switching on power to the write-blocker. The forensic workstation recognises the USB device as normal, and off you go.

We then run FTK Imager to create a logical image of the device. We tested the various formats and settings available and eventually decided that creating true forensic images would raise too many trust issues with potential depositors with regard to us being able to restore deleted files. For this reason we will create ‘Folder contents only’ forensic images which recreate the device as it would appear in normal use. From here we are exploring our options for exporting the files from the disk image, but we have found that the exported files display an altered Accessed date – any comments/suggestions on this issue would be gratefully received.

We also create directory listings of the contents with MD5 and SHA-1 checksums. From the disk image and directory listing we can start to consider the arrangement for the collection, using Quickview Plus to preview file contents.

Our second write-blocker can be used with IDE and SATA hard drives...but more of this in part 4!

Forensic Workstation pt2

When we moved from the University campus to our new joint facilities with Hull City Archives and the Local Studies Library we took the opportunity to upgrade many of our PCs – leaving a few older specimens “just in case” anybody was so desperate that they were willing to accept a machine that was reluctant to start-up!

Recently the library has been re-organising its stock and space-utilisation ahead of a major refurbishment. Our old PC was discovered in the basement and ear-marked for disposal (well recycling really but disposal is less ambiguous). It was at this point, and with a new-found digital archives perspective, that I realised the potential of this machine to become our first digital forensics workstation. With an internal 3.5” floppy drive, CD drive and 2 USB ports this was a combination that seemed to promise possibilities for dealing with a range of media but also the chance to transfer the files once they had been extracted. The PC with slightly grubby keyboard and monitor were shipped to their new home at the History Centre.

I had by this time, started to identify requirements for a new PC to act as a workstation for the capture of hard-drives and other large volume of material. This request intrigued a colleague Tom in ICT and a visit was duly arranged, Tom was really interested in our work and offered to help. Tom took our PC and returned it a few days later - with a clean version of the Windows XP image installed aswell as an internal zip drive added.

Tom has also promised to put aside a couple of internal 3.5” floppy drives as an insurance policy for the drives failing as Jeremy Leighton John at the British Library had reported mixed results when using the external USB floppy drives. Having two workstations, one old and one new, will give us an option for dealing with some media formats; a USB drive for 3.5" floppy drives and an external 250MB zip drive. The latter was found when clearing-out an old cupboard and came with all cables and even its original installation CD proving that assembling a forensic workstation does not have to cost a fortune and I have heard several tales of kit assembled via ebay purchases.

AIMS at SAA

Today's post is just a brief announcement...The AIMS team will be taking part in two events at next week's Society of American Archivists Annual Meeting. The first is a workshop we've developed to provide an opportunity for archivists and technologists to discuss issues related to collection development, accessioning, appraisal, arrangement and description, and discovery and access of these materials. Unfortunately, space issues have required us to limit registration and it is now full. However, we promise to post a longer recap to this blog after the event.

No such limitations exist for our other SAA event, a presentation entitled Born-Digital Archives in Collecting Repositories: Turning Challenges into Byte-Size Opportunities, which will be given August 27th at 8 a.m. At this presentation the AIMS Digital Archivists will describe a bit of the high-level framework being developed by the AIMS project to characterize archival workflows for born-digital materials in archival repositories.

We hope to see you there!

Digital Forensics for Digital Archivists

I’ve been very fortunate here at UVa to have at my disposal some wonderful resources for getting up to speed with born-digital theory and practice. First and foremost, UVa is home to Rare Book School which has offered a course on Born Digital Materials for the past two years (and I’ve just learned will offer it again in 2012). I was able to take this course in July along with 11 fellow classmates from around the country. A week and a half later I was then off to the headquarters of Digital Intelligence, Inc. makers of our Forensic Recovery of Evidence Device (FRED) for Computer Forensics with FRED. This was a two day course covering basic digital forensic skills as well as the FRED system.

Mulder and Scully are concerned about the viability of this forensic evidence gathered next to UVa's FRED...

Given my great bounty, and my belief in professional karma, I’ve decided to give a brief overview of both of these classes here on the blog followed by my thoughts on a potential Digital Forensics for Archivists class/workshop that I’d really like to see developed, by myself or whomever! Two major classes out there that I have not taken are the DigCCurr Professional Institute and SAA’s electronic records workshop. Anyone with experiences in those classes, please add your comparisons in the comments.

RBS L95 — Born Digital Materials: Theory and Practice

Overall, I’d say this class has the perfect name: there’s an almost equal amount of theory and practice. That may sound like faint praise, but it’s really not. It’s something that too few workshops or classes get right. Instructors Naomi Nelson and Matt Kirschenbaum deserve much credit for a well constructed week that built practice on top of theory.

For someone new to the field of the born-digital it’s a great foundation. Concepts like metadata, preservation, “the cloud,” essential characteristics, physicality/materiality and digital humanities are combined with real-life examples from libraries, archives, and the university. This overview allowed us to attack the fundamental question of the class: what should we be trying to accomplish when we attempt to “save” (or steward, curate, safeguard, preserve, “archive”) born-digital materials.

On the practical side of things, digital forensics is covered and students get the opportunity to do a few lab exercises with emulators, floppy drives, and older models of equipment. The syllabus and reading list provide an excellent bibliography for further research.

It’s a relatively high-level class and therefore a great way to get started or a great way to get administrators thinking intelligently about the issues they need to face. I think that a more practitioner-focused and through digital forensics curriculum in the archives or cultural heritage setting could complement the course very nicely.

Computer Forensics with FRED training

University of Virginia decided to invest in the FRED technology last year and has not regretted it. While the FRED can do lots of neat things, I feel it is important to note that many or all of the same things can be done with other hardware and software, it just takes a bit more persistence. Similarly, despite the name a lot of this course dealt with basic data and file system concepts, as well as a little bit about some of the specific hardware most commonly found. In the future, DI is going to be splitting this up into two classes: Digital Forensic Essentials and Digital Forensics with FRED. The first part is a two day course and covers the hardware, data, and system stuff. The second is a one day class that covers the specifics of FRED. Although the first class will be more expensive than the current combined class is, it would be of more interest to those in the archival world.

As it is geared for law enforcement, a lot of time was spent on detected deleted, fraudulent, or hidden material. While all the cops in the room thought that this would be of no use to me, I disagreed. I need to know what I am collecting (whether inadvertent or not), whether it is authentic, and how to communicate with donors to decide how to deal with it. In addition, if we can get donors to agree to let us transfer backup or deleted versions of manuscripts, we’ll gain a wealth of information about how the final version evolved. Knowing that such recovery is possible is one of the more glamorous promises of digital forensics.

We also learned how to create and navigate disk images. While some of this stuff was fairly easy for me to pick up beforehand from Peter Chan’s tutorials, the extra practice and insight was very useful.

Digital Forensics for Archivists

Based on my experiences in these two classes, I would propose a Digital Forensics for Archivists workshop geared specifically for those interested in incorporating forensic techniques into the capture and processing of digital materials. The outline of topics I would expect to see on the syllabus below is probably a bit ambitious for a one-day workshop and would certainly have some hurdles to overcome related to provisioning hardware for all. However, these are the areas I’ve come to think of as necessary for an archive to be prepared for the variety of media that we will be collecting for the continuing future.

Digital Forensics for Archivists

• Hardware basics


• IDE, SCSI, SATA, USB, Firewire
• Floppy drives
• Optical disks
• Hard drives
• Internal basics (motherboard, pci, power, etc.)


• Operating Systems


• DOS
• Windows
• MAC OS
• Linux


• File system basics


• FAT


• NTFS


• HPFS


• Forensic vs. logical copying


• What happens to deleted data


• How it can be recovered


• Why you need to know…


• Write blocking


• How to achieve it


• Image files


• Types


• Software


• Uses


• Emulation and Migration


• Cost/benefit of each


• Possible use cases for each

So what do you think? Pipe dream? Useful? Impractical? Let me know in the comments…

Forensic workstation pt 1

A key part of dealing with born-digital archives is the ability to receive and process material without making changes to the underlying metadata including date created, date accessed etc – data that researchers will be looking to use and rely on. As archivists we place considerable emphasis on our roles as custodians and with digital material it is important that we treat the material carefully and appropriately. Fortunately there are tools that help us with the authenticity of born-digital files the most obvious of which is the checksum.

An important legacy of the AIMS project for us at Hull is working towards our ability to take born-digital material from depositors as a normal part of our work. A key component of this is a forensic workstation – by which I mean a PC (or two) through which material can be safely captured following a clear process, in-effect replicating the isolation room for receiving paper material. This will allow us to undertake a forensic examination – to check the material is what we expected or agreed to take, including the ability to generate a manifest of the material to send to the depositor, and that it does not include viruses etc.

There seem to be two main routes – to purchase FRED which stands for Forensic Recovery Evidence Device (other digital forensic workstation solutions are available). A second and more organic solution, and the one we intend to adopt at Hull, is to start with a new PC and to add appropriate hardware and software to this to provide the equivalent functionality. At the moment we are pondering a name for this with current suggestions including:
- Hal - Hull Archives Laboratory
- Harold – Hull Archives Recovery Of Legacy Data
- Hilary - Hull Investigator for Library and Archives RecoverY
- Dawn – Digital Archives WorkstatioN
but we are open to other suggestions until the machine is installed and formally named!

We don’t want to become a computer museum with an extensive range of hardware, software and operating systems environments for any possible eventuality. We do want a core ability to handle material we reasonable expect to receive – including material on 3.5” floppy disks, zip disks, hard drives etc. We intend to develop and extend our capacity as need dictates – if we get material in a format we will consider whether we need to support this ourselves or whether a suitable 3rd party is more appropriate.

Central to this is the need for write blockers which prevent you from writing or updating the files. Having read countless websites I felt I knew what they were supposed to do but had a nagging doubt that my knowledge was incomplete.

A tour of the British Library eMss Labs courtesy of Jeremy Leighton John (as featured on the BBC Radio 4 programme 'Tales from the Digital Archives' broadcast in May but still available online) confirmed the simplicity of theory and the fragility of the media – just having the hardware isn’t enough – you also need some luck that you have the correct drivers to read the specific version of the media. In the next few weeks I hope to place our order for the various bits and pieces and will update you on this exciting journey!!

Curators Workbench workshop

I was fortunate enough to attend the Curator’s Workbench workshop at the British Library last week. It was a chance to see, have a play and discuss the tool with its developers Greg Jansen and Erin O’Meara from University of North Carolina. The tool is designed to aid with the accession, arrangement, description and staging of material prior to ingest into a digital repository. Essentially the tool has an interface designed for archivists can use.

The session featured a walk-through and chance to have a play with experts on-hand if you had a problem – only necessary as we had latest ‘unstable’ release including the latest enhancements to functionality and GUI. Stable versions are available for download via GitHub. I am especially smitten with the crosswalk feature providing a drag’n’drop interface for mapping the metadata with METS. There is also the date recogniser which allows you to map the date format to the ISO standard, though there could be issues if the data is in a variety of formats, ie 1984 would be transformed to 01/01/1984.

It has a different take to where arrangement and description occurs in the workflow to that intended for Hypatia in the AIMS workflow, but it does raise some interesting questions that I hope to explore in more detail over the next few months.

It was also interesting to hear features and functionality on their wish-list including disc images, multiple users, recording processing notes, PREMIS and so the list goes on!
The discussion that followed was really enlightening as it highlighted the different approaches that archives are currently adopting to the preservation of born-digital archives.

I picked-up some useful pointers to software and tools I haven’t used before – Bulk extractor, Google Refine, and came away determined to throw more stuff at Curators Workbench, to join the users discussion list (done) and to figure out some of the aspects we have avoided so far things like PREMIS and METS etc !