Practical First Steps

Last week I helped organise a training day on born-digital archives for the East of England Regional Archive Council. I was joined by Chris Hilton from the Wellcome Library, Ellie Robinson from LSE and Grant Young from Cambridge University Library. The day followed a similar pattern to an event hosted in Hull last November. There were four main elements to the day:

Institutional Overview
The four of us gave a brief overview of the development of digital preservation in our respective institutions and included Chris’s now legendary simplification of OAIS to "Get Stuff - Put stuff somewhere - Keep stuff safe & Show stuff to people".  Ellie talked through the development at LSE from a risk analysis perspective to get institutional backing to then moving on to actually doing it - the latter sentiment being one of the mantras for the day. Grant talked about his work with digital content - much of it digitised rather than born-digital but now occupying an eye-watering 67TB (both LSE and Hull have about 120GB of born-digital material).

Practical First Steps
The four of us then gave a short presentation offering some practical tips; I looked at conducting a survey to identify material already held in the archives and how this often meant the media had been accessioned but not the contents! Chris shared the experiences at Wellcome of 'Dealing with depositors', Ellie looked at 'Handling born-digital material' including accessioning, virus check and other stages at LSE and Grant talked about 'Issues around File Formats' highlighting a number of challenges and suggesting strategies that could be adopted.

Questions and Answers

The day also included two question and answer sessions designed to get delegates talking about the particular aspects and issues of concern to them. Questions touched on a range of topics including depositors, DRAMBORA, how to approach hybrid collections and depositor agreements. We also heard of work being conducted in a number of local authority archives and hopefully they will share their work and experiences with colleagues in the near future.


Demonstrations
Delegates were split into four groups and given demonstrations on using Karen's Directory Printer, DROID and also using FTK Imager with a write-blocker to read a PC hard drive (from my garage) the fourth diversion was a look at two different born-digital scenarios for delegates to consider how they might respond.

There was common agreement on the need to do something, and widespread acknowledgement that there wasn't a single solution or approach. Wellcome, LSE and Hull were all looking at the issue of bulk-ingest into repositories whilst retaining the relationships between files as represented through an often complex series of folders. It so happens that at Hull one of our developers is looking at this very issue so I hope to have an update on this in the next few weeks.

A key theme of the day was collaborating and helpline colleagues and in this spirit all of the presentations are now available on the Hull History Centre born-digital archive pages - thanks to all of the speakers for making this an interesting and informative day.

Not a typical week

At the end of the AIMS project I returned to my post as Senior Archivist with digital archives added to my todo list alongside public searchroom duty, working with paper collections, responsibilities for maintaining our website and online catalogue, managing staff and volunteers etc etc.

This week has not been typical.

Monday
Accession two recent deposits including a small set of floppy disks created between 1995-1999 using a Psion (I think judging by some of the data visible using FTKImager).  The other item was a CD with minutes created in the last couple of years by a charity – so nothing to worry about in terms of formatting but it did highlight issues around filename consistency. I contacted the depositor and they were happy to receive suggestions about future naming conventions which will be a great help. I was also able to ask about material that reflected the complete range of activities of the charity and hope that further material will be forthcoming.

Tuesday

One of the outcomes following the publication of the AIMS White Paper has been to share experiences with colleagues in other institutions. On Tuesday our guests were Nancy McGovern and Kari Smith from MIT and it was a great opportunity to share experiences and discuss aspects surrounding processes, workflows and tools. As always I came away with a list of other tools to try and research papers to look out for! We were joined by my colleague Chris Awre who talked about the work at Hull using Fedora for our institutional repository and in particular Hydra and the opportunity this offered for sharing development work.

Wednesday

Spent some of Wednesday preparing for a one day workshop at Cambridge about born-digital archives next week. The day is designed to encourage colleagues to take the first steps and will include colleagues from LSE and the Wellcome Library and will feature demonstrations of write-blocker hardware and tools including Karen’s Directory Printer and DROID.

Thursday

Received an email out of the blue from a colleague working in Vancouver, which was really nice – they had been following the AIMS Blog and wanted to ask some questions and I was happy to clarify a few aspects that had been mentioned. In replying I also sought more information about their own experiences and whether we had tackled email. Whilst we haven’t tackled this explicitly (yet) I have had a play with the MUSE tool which gives a unique perspective on the stuff with-in an 'mbox' file and offers a sentiment graph that instantly grabs you.

Friday

What better for a Friday afternoon than a quick spell of taking photographs of the floppy disks I accessioned on Monday. It took longer than it should have done due to lack of practice and the need to find something to prop up the disk so we could capture the information written on the edge of the disk – our conservator Christine found a small clear display stand that is ideal and this has been requisitioned for future photographic needs.

This hasn't been a typical week – I have probably done more in the last five days than the preceding two months - but then things rarely are in archives – and for many working in the profession the range and variety is one of the best parts of the job.

Brith Gof

Last week I was fortunate to be invited to a two day workshop hosted by the National Library of Wales (NLW) at Aberystwyth. The workshop brought together archivists, academics, library and ICT staff and students to discuss the “challenges” surrounding the Brith Gof and Clifford McLucas collections. The primary focus was on aspects of the digital preservation – with a range of media and carriers including floppy disks, zip drives, SyQuest cartridges (a new one to me) and Mac Book G4 amongst these two collections.

Discussions touched a wide range of topics and issues including digital forensics, cataloguing hybrid collections, digital curation, emulation and access. There was also a reminder of the complexity of intellectual property rights for performance material where different rights might be held for the set design, the score etc etc.

There was also a public event where a number of items from the collections were used as prompts by Professor Mike Pearson, co-founder of Brith Gof, to stimulate his recollections of Brith Gof. The range of items selected highlighted the rich nature of the collections that includes photographs, videos, set designs, huge banners (aswell as the more traditional paper archives). One aspect that is also being considered is how to capture/record the impact that Brith Gof had on those watching the performance.

I talked about the AIMS project and highlighted a number of the questions and issues that had arisen from our shared experiences that we have captured in our White Paper. I really enjoyed the discussions, the informal nature and the reminder about making a deliberate effort to engage and attract a range of audiences.

Today is International Archives Day (twitter #archday12) to see what everybody has been saying.

Forensic workstation pt 4

Earlier parts of this series had touched on identifying our needs and requirements from a workstation (see part 1), re-purposing an old PC into our first workstation (see part 2) and our early experiences of write-blockers and FTK Imager (see part 3)

Our experiments with our write-blockers have been limited, but each time we get them out of their boxes they seem a little less scary. The recent deposit of some born-digital audio and video material totalling over 200GB has thrown-up a number of new issues for us to consider.

The files came to us on an external hard drive formatted for a Mac that we needed to return to the depositor at which point we knew the files would be deleted - placing greater emphasis on the need to get the capture process correct as we wouldn't be able to return to the depositor and try again!

We were unable to browse the files in Windows Explorer, but were able to see the files using FTK Imager and our USB write-blocker. The sheer size of the files is something we are going to have to get used to with a 45 minute QuickTime film is 10.1GB and a 43 minute wav file is 671MB.

The workstation already has PaintShop Photo Pro for viewing and converting image files but only the standard viewers for audio and video content. So we started to look for open source software for viewing and converting the audio and film files, I wanted something that had a graphical and not a command line interface, as I was keen for other staff to develop skills and experience in handling this type of content.

As with our earlier use of tools like Karen's Directory Printer and DROID once we have become familiar with software we then document our use by creating a simple 'Idiots Guide' - this allows us to record both issues and solutions that we have encountered.

A bit of browsing and a few recommendations later and we have now installed Audacity v2 and WinFF but we will also take a look at others including Handbrake and FFmpeg before making a final decision.

We are keenly awaiting the forthcoming release of the DPC Technology Watch Report on 'Preserving Moving Pictures and Sound' and revisiting the FutureArch blog entries on media formats.

UKAD Archives Discovery Forum 2012

A few reflections on the UKAD Archives Discovery Forum 2012 that I attended yesterday

The day started with a really interesting keynote piece from Bill Thompson (BBC) about his role in using the archives to forge partnerships with a range of organisations and he highlighted the range and diversity by talking about several projects including a project this summer with the Arts Council, the centenary of the First World War and an exciting initiative called Digital Public Space (diagram on collectionslink website)

Joy Palmer (MIMAS) gave a talk about the JISC Discovery programme and the ongoing work to demystify aspects like APIs, persistent URIs, user interfaces and measuring both impact and value.

Teresa Doherty (The Women's Library) then spoke about name authority records and how you can help make your collections more discoverable by adding links to the archives from relevant biographical pages on Wikipedia - something we started to do at Hull several years ago but this was a useful reminder to revisit this simple approach that can have a huge impact on the visibility of your collections.

After a great networking opportunity called lunch Lindsay Ould (Kings College London) talked us through the JISC funded FIDO project (Forensic Information in Digital Objects) and their experiences, highlighting a range of technical, skills-based and ethical issues and also their use of OS Forensics software.

I then gave a presentation about born-digital archives, but took a different approach - instead of focussing on the work we have undertaken at Hull I presented a very brief SWOT analysis to highlight many of the issues we have experienced.

There was then a series of short presentations including Sam Velumyl (The National Archives) who gave an overview of the TNA Finding Archives project , Teresa Dixon (West Yorkshire Archives Service) spoke about the History to Herstory website which features over 80,000 images including the Amy Johnson letters held at the Hull History Centre.

Kimberly Kowal (British Library) spoke about a crowd-sourcing map project which saw 725 maps geo-referenced in a week (see a blog entry about this project) and Alison Cullingford spoke about the Research Libraries UK Unique and Distinctive Collections project.

The final session was from Bill Stockting (British Library) about the completion of the Integrating Archives and Manuscripts System - bringing a vast number of legacy data sources and systems and over 1.5m records together and this now sits behind the http://searcharchives.bl.uk/ site.

Despite all of this there were a host of other sessions I would like to have attended including linked data, the National Archives new catalogue and the Old Maps online project.

Update 2nd April - slides from the sessions have been added to The National Archives website, see the Documenting Collections page

Archives and Society

Two weeks ago I spoke at the Archives and Society series of tasks held at the Institute of Historical Research about the progress and work at Hull as a result of the AIMS project. Whilst highlighting the AIMS White Paper the bulk of the talk was about the practical steps we had taken at Hull with born-digital archives; starting with a simple survey of collections and then followed by photography of media and creating a forensic workstation (a tale told in multiple parts see - part 1, part 2 and part 3).

I sought to encourage those present to download software like Karen's Directory Printer and DROID and to have a go - using a few test files will help increase your familiarity with many of the issues associated with digital preservation.

I managed to stop in time for questions - and these included aspects relating to the fact that the issues I raised were not "new" and whether we would still be making the same case in 5 years time (I hope not) and the need for automated tools to help us cope with the sheer volume of material (an obvious need) and the associated risk of releasing material that you haven't explicitly checked because of the sheer volume of files..

A PDF version of the slides is available - the talk was also recorded and I will add a link to the podcast when it is available.

AIMS White Paper now available

After a huge amount of effort the AIMS White Paper has finally been finished and is now available online.

The White Paper is intended as a framework for guide good practice in terms of archival tasks and objectives necessary for success. It builds upon the experiences of the four partner institutions - the universities of Hull, Stanford, Virginia and Yale - to process a range of collections with an array of format and media issues and using different software – we were keen to make this software agnostic and have gone back to the archival principles at the heart of the processes.

In many areas we found many similarities with existing practice with paper records and for some aspects we found there were multiple ways of achieving certain goals and we didn't want to be prescriptive in any way.  So instead it highlights key decision points and aspects of policy that may be determined at an institutional level and is intended to help people making the same journey that we have made – finding out about projects, tools & case studies and starting to build knowledge, skills & infrastructure.  

Although the publication of the White Paper officially marks the end of the AIMS project the institutions intend to continue collaborating and sharing their experiences on this blog.

We welcome your comments and feedback to the White Paper on this blog – whether you have implemented the framework or just found the guidance useful.

Day of Digital Archives – some personal reflections

To mark the Day of Digital Archives I thought I would add a personal note about the “journey” I have made in the last two years. It was about this time in 2009 that it was announced that the AIMS Project was being funded by the Andrew W. Mellon Foundation and that I would be seconded from my post as Senior Archivist to that of Digital Archivist for the project.

At the time I had considerable experience of digitisation but very little about digital archives. So I began reading a few texts and following references and links to other sources of information until I had a pile of paper several inches high of things to read. At first there was a huge amount to take-in – new acronyms, especially from the frightening OAIS, and plenty of projects like the EU funded Planets initiative. It seemed that the learning would never stop – there was always another link to follow another article to read and it was really difficult to determine how much was making sense.

Talking to colleagues who were already active in this field also revealed how little digital media we actually had at the University of Hull Archives – just over two years ago we literally had a handful of digital media whilst others were already talking about terabytes of stuff. Fortunately the AIMS project sought to breakdown the workflow into four distinct sub-functions and placed emphasis on understanding the process compared to ‘traditional’ paper archives which reduced the sense of being overwhelmed by it all.

Since then I feel I have come along way – I have attended a large number of events and spoken at a fair few and quickly become both familiar and comfortable with the language. I do appreciate the time I have been able to dedicate solely to the issue of digital archives and that many colleagues are embracing this “challenge” without this luxury.

The biggest recommendation I can make is to start having a play with the software – many of the tools that we use at Hull University are free – Karen’s Directory Printer for creating a manifest of records including checksums that have been received; FTK Imager for disc images etc etc. Nor do you have to wait for digital archives or risk changing key metadata whilst you are experimenting – you can use any series of digital files or old media that are lurking in the way of a drawer. We have also created a forensic workstation and shared our experiences via this blog.

Once we had started to experiment, we created draft workflows and documentation and refined this as we experimented further – all tasks from photography of media to using write-blockers do become less daunting the more frequently you do them. Having learnt from many colleagues we have started to add content to the born-digital archives section of the History Centre website. I have also used some of my own email to play with the MUSE visualisation tool to understand how it might allow us to provide significantly enhanced access to this material in the future.

Although the project funding has now finished and I have returned to my “normal” job I do think that digital archives has now become part of my normal work and each depositor is now specifically asked about digital archives and in public tours of the building we explicitly mention the challenges and the opportunities of digital archives. We don’t have all of the answers yet – archiving e-mail in particular still scares me, but don’t feel as daunted as I did two years ago.

A Tale of Two conferences

Last week I was fortunate to be part of the AIMS team presenting our work at the SAA Conference in Chicago. Despite the Saturday 8am start of our session and the impending threat of Hurricane Irene well over 150 delegates turned-out to hear our presentation which included both an introduction to the AIMS framework and reporting our practical experiences through case studies. If you missed it or want to relive it the presentations are available online via Slideshare.

On Friday I spoke at the ARA conference in Edinburgh – the theme of which was advocacy and as part of a Data Standards Group I spoke about the skill set that I had acquired during my change of role from archivist to digital archivist as a result of the AIMS project.

Although the two presentations were different in content and context they both included the same message – an attempt to breakdown the perceptions and myths surrounding born digital archives. In talking about skills in Edinburgh I sought to highlight the relevance of the traditional archive skills in the digital age and to encourage more individuals to do something.

It also raised the question – something that arose in the AIMS unconference in Charlottesville and the UK workshop in London, of when will digital archives become “the norm”. We don’t know the exact answer to this, but I do know it is necessary if we are to successfully manage the challenges of born-digital archives and strive to meet the increasing expectations of our users.

Friday also marked the end of a six month contract during which Nicola Herbert has helped us with the practical elements of digital preservation at Hull. I would like to thank Nicola for her hard work and direct users to her guest blogs on photography of media and write-blockers.

AIMS@SAA Part Two: SAA Session 502

SESSION 502 - Born-Digital Archives in Collecting Repositories: Turning Challenges into Byte-Size Opportunities
SAA 2011
Chicago, IL 
Aug 27, 2011

As the endnote to their foray into the SAA 2011 Annual Meeting, the AIMS Digital Archivists delivered a presentation on the AIMS project on Saturday morning. Although we were competing with Hurricane Irene’s effect on travel schedules, an 8 a.m. Saturday timeslot, and presentations from our colleagues Michelle Light, Dawn Schmitz and John Novak’s on delivering born-digital materials online as well as the Grateful Dead Archivist and a member of the band Phish, attendance was pretty darn good! We were pleased to be able to speak with some colleagues after the session and facilitate a few discussions during the question and answer portion of the session.

The presentation itself gave a brief overview of the project and then focused on the AIMS framework, or the four areas we’ve identified as key functions of stewardship for born-digital materials: Collection Development, Accessioning, Arrangement and Description, and Discovery and Access.

We’re very happy to share our slides here through slideshare. Remember, this is just a taste of what’s to come in the white paper this fall, so keep checking the blog for updates!

Slide are posted here after the jump! 

AIMS@SAA Part One: CREW Workshop

CREW: Collecting Repositories and E-Records Workshop
SAA 2011
Chicago, IL 8/23/2011

The AIMS partners hosted a workshop in the run-up to the 2011 SAA Annual Meeting in August. 45 participants from the US and Canada joined us in exploring the challenges, opportunities and strategies for managing born-digital records in collecting repositories.

The workshop was organized around the 4 main functions of stewardship that the AIMS project has focused on: Collection Development, Accessioning, Arrangement and Description, and Discovery and Access. In addition to the AIMS crew (no pun intended) presenting on the research done through the AIMS project, several guest presenters showcased case studies from their own hands-on approaches to managing born-digital materials. Seth Shaw, from Duke University discussed the evolution of electronic record accessioning at Duke University and his development of the Duke Data Accessioner. Gabriela Redwine discussed work done in arrangement and description at the Harry Ransom Center at the University of Texas at Austin. Finally, Erin O’Meara showcased work done at the University of the North Carolina at Chapel Hill to facilitate access to born-digital records through finding aid interfaces.

In between presentations the participants engaged in lively discussions around provocative questions and hypothetical scenarios. At the end of the event, the AIMS partners felt they had gained just as much from the day’s activities as they hoped the participants had. Ideas that were discussed and case study examples will help strengthen the findings of the white paper due out this fall.

See the workshop presentations after the jump! 

Forensic Workstation pt3

A guest posting from Nicola Herbert, Digital Project Preservation Assistant at Hull University Archives

Once we had the forensic workstation up and running (see part 1 and part 2 in this on-going series) we installed MS Office and Mozilla Thunderbird (for working with Outlook .pst files). We also installed FTK Imager, Karen’s Directory Printer, DROID and the MUSE e-mail visualisation tool (in beta, but provides a very interesting perspective on the data). We are also planning to purchase Quickview Plus, a piece of software that enables viewing a range of file formats without requiring the original software on your PC.

We had already played around with these tools on our normal PCs and had run them on files copied from digital media prior to setting up the workstation.

Having received our two Tableau write-blockers we were eager to combine the separate processes we had developed into an integrated workflow. We have two write-blockers, one for USB devices (T8-R2) and one for internal hard drives from PCs and laptops (T35es). Simon’s visit to Jeremy John at the British Library had whetted our appetite for getting our mini digital forensics lab in operation.

USB devices
After a thorough read-through of the instructions we tested out the USB write-blocker first. Setting it up is relatively simple; the vital thing is to make the connections between device and write-blocker, write-blocker and forensic PC before switching on power to the write-blocker. The forensic workstation recognises the USB device as normal, and off you go.

We then run FTK Imager to create a logical image of the device. We tested the various formats and settings available and eventually decided that creating true forensic images would raise too many trust issues with potential depositors with regard to us being able to restore deleted files. For this reason we will create ‘Folder contents only’ forensic images which recreate the device as it would appear in normal use. From here we are exploring our options for exporting the files from the disk image, but we have found that the exported files display an altered Accessed date – any comments/suggestions on this issue would be gratefully received.

We also create directory listings of the contents with MD5 and SHA-1 checksums. From the disk image and directory listing we can start to consider the arrangement for the collection, using Quickview Plus to preview file contents.

Our second write-blocker can be used with IDE and SATA hard drives...but more of this in part 4!

Forensic Workstation pt2

When we moved from the University campus to our new joint facilities with Hull City Archives and the Local Studies Library we took the opportunity to upgrade many of our PCs – leaving a few older specimens “just in case” anybody was so desperate that they were willing to accept a machine that was reluctant to start-up!

Recently the library has been re-organising its stock and space-utilisation ahead of a major refurbishment. Our old PC was discovered in the basement and ear-marked for disposal (well recycling really but disposal is less ambiguous). It was at this point, and with a new-found digital archives perspective, that I realised the potential of this machine to become our first digital forensics workstation. With an internal 3.5” floppy drive, CD drive and 2 USB ports this was a combination that seemed to promise possibilities for dealing with a range of media but also the chance to transfer the files once they had been extracted. The PC with slightly grubby keyboard and monitor were shipped to their new home at the History Centre.

I had by this time, started to identify requirements for a new PC to act as a workstation for the capture of hard-drives and other large volume of material. This request intrigued a colleague Tom in ICT and a visit was duly arranged, Tom was really interested in our work and offered to help. Tom took our PC and returned it a few days later - with a clean version of the Windows XP image installed aswell as an internal zip drive added.

Tom has also promised to put aside a couple of internal 3.5” floppy drives as an insurance policy for the drives failing as Jeremy Leighton John at the British Library had reported mixed results when using the external USB floppy drives. Having two workstations, one old and one new, will give us an option for dealing with some media formats; a USB drive for 3.5" floppy drives and an external 250MB zip drive. The latter was found when clearing-out an old cupboard and came with all cables and even its original installation CD proving that assembling a forensic workstation does not have to cost a fortune and I have heard several tales of kit assembled via ebay purchases.

AIMS at SAA

Today's post is just a brief announcement...The AIMS team will be taking part in two events at next week's Society of American Archivists Annual Meeting. The first is a workshop we've developed to provide an opportunity for archivists and technologists to discuss issues related to collection development, accessioning, appraisal, arrangement and description, and discovery and access of these materials. Unfortunately, space issues have required us to limit registration and it is now full. However, we promise to post a longer recap to this blog after the event.

No such limitations exist for our other SAA event, a presentation entitled Born-Digital Archives in Collecting Repositories: Turning Challenges into Byte-Size Opportunities, which will be given August 27th at 8 a.m. At this presentation the AIMS Digital Archivists will describe a bit of the high-level framework being developed by the AIMS project to characterize archival workflows for born-digital materials in archival repositories.

We hope to see you there!

Digital Forensics for Digital Archivists

I’ve been very fortunate here at UVa to have at my disposal some wonderful resources for getting up to speed with born-digital theory and practice. First and foremost, UVa is home to Rare Book School which has offered a course on Born Digital Materials for the past two years (and I’ve just learned will offer it again in 2012). I was able to take this course in July along with 11 fellow classmates from around the country. A week and a half later I was then off to the headquarters of Digital Intelligence, Inc. makers of our Forensic Recovery of Evidence Device (FRED) for Computer Forensics with FRED. This was a two day course covering basic digital forensic skills as well as the FRED system.

Mulder and Scully are concerned about the viability of this forensic evidence gathered next to UVa's FRED...

Given my great bounty, and my belief in professional karma, I’ve decided to give a brief overview of both of these classes here on the blog followed by my thoughts on a potential Digital Forensics for Archivists class/workshop that I’d really like to see developed, by myself or whomever! Two major classes out there that I have not taken are the DigCCurr Professional Institute and SAA’s electronic records workshop. Anyone with experiences in those classes, please add your comparisons in the comments.

RBS L95 — Born Digital Materials: Theory and Practice

Overall, I’d say this class has the perfect name: there’s an almost equal amount of theory and practice. That may sound like faint praise, but it’s really not. It’s something that too few workshops or classes get right. Instructors Naomi Nelson and Matt Kirschenbaum deserve much credit for a well constructed week that built practice on top of theory.

For someone new to the field of the born-digital it’s a great foundation. Concepts like metadata, preservation, “the cloud,” essential characteristics, physicality/materiality and digital humanities are combined with real-life examples from libraries, archives, and the university. This overview allowed us to attack the fundamental question of the class: what should we be trying to accomplish when we attempt to “save” (or steward, curate, safeguard, preserve, “archive”) born-digital materials.

On the practical side of things, digital forensics is covered and students get the opportunity to do a few lab exercises with emulators, floppy drives, and older models of equipment. The syllabus and reading list provide an excellent bibliography for further research.

It’s a relatively high-level class and therefore a great way to get started or a great way to get administrators thinking intelligently about the issues they need to face. I think that a more practitioner-focused and through digital forensics curriculum in the archives or cultural heritage setting could complement the course very nicely.

Computer Forensics with FRED training

University of Virginia decided to invest in the FRED technology last year and has not regretted it. While the FRED can do lots of neat things, I feel it is important to note that many or all of the same things can be done with other hardware and software, it just takes a bit more persistence. Similarly, despite the name a lot of this course dealt with basic data and file system concepts, as well as a little bit about some of the specific hardware most commonly found. In the future, DI is going to be splitting this up into two classes: Digital Forensic Essentials and Digital Forensics with FRED. The first part is a two day course and covers the hardware, data, and system stuff. The second is a one day class that covers the specifics of FRED. Although the first class will be more expensive than the current combined class is, it would be of more interest to those in the archival world.

As it is geared for law enforcement, a lot of time was spent on detected deleted, fraudulent, or hidden material. While all the cops in the room thought that this would be of no use to me, I disagreed. I need to know what I am collecting (whether inadvertent or not), whether it is authentic, and how to communicate with donors to decide how to deal with it. In addition, if we can get donors to agree to let us transfer backup or deleted versions of manuscripts, we’ll gain a wealth of information about how the final version evolved. Knowing that such recovery is possible is one of the more glamorous promises of digital forensics.

We also learned how to create and navigate disk images. While some of this stuff was fairly easy for me to pick up beforehand from Peter Chan’s tutorials, the extra practice and insight was very useful.

Digital Forensics for Archivists

Based on my experiences in these two classes, I would propose a Digital Forensics for Archivists workshop geared specifically for those interested in incorporating forensic techniques into the capture and processing of digital materials. The outline of topics I would expect to see on the syllabus below is probably a bit ambitious for a one-day workshop and would certainly have some hurdles to overcome related to provisioning hardware for all. However, these are the areas I’ve come to think of as necessary for an archive to be prepared for the variety of media that we will be collecting for the continuing future.

Digital Forensics for Archivists

• Hardware basics


• IDE, SCSI, SATA, USB, Firewire
• Floppy drives
• Optical disks
• Hard drives
• Internal basics (motherboard, pci, power, etc.)


• Operating Systems


• DOS
• Windows
• MAC OS
• Linux


• File system basics


• FAT


• NTFS


• HPFS


• Forensic vs. logical copying


• What happens to deleted data


• How it can be recovered


• Why you need to know…


• Write blocking


• How to achieve it


• Image files


• Types


• Software


• Uses


• Emulation and Migration


• Cost/benefit of each


• Possible use cases for each

So what do you think? Pipe dream? Useful? Impractical? Let me know in the comments…

Forensic workstation pt 1

A key part of dealing with born-digital archives is the ability to receive and process material without making changes to the underlying metadata including date created, date accessed etc – data that researchers will be looking to use and rely on. As archivists we place considerable emphasis on our roles as custodians and with digital material it is important that we treat the material carefully and appropriately. Fortunately there are tools that help us with the authenticity of born-digital files the most obvious of which is the checksum.

An important legacy of the AIMS project for us at Hull is working towards our ability to take born-digital material from depositors as a normal part of our work. A key component of this is a forensic workstation – by which I mean a PC (or two) through which material can be safely captured following a clear process, in-effect replicating the isolation room for receiving paper material. This will allow us to undertake a forensic examination – to check the material is what we expected or agreed to take, including the ability to generate a manifest of the material to send to the depositor, and that it does not include viruses etc.

There seem to be two main routes – to purchase FRED which stands for Forensic Recovery Evidence Device (other digital forensic workstation solutions are available). A second and more organic solution, and the one we intend to adopt at Hull, is to start with a new PC and to add appropriate hardware and software to this to provide the equivalent functionality. At the moment we are pondering a name for this with current suggestions including:
- Hal - Hull Archives Laboratory
- Harold – Hull Archives Recovery Of Legacy Data
- Hilary - Hull Investigator for Library and Archives RecoverY
- Dawn – Digital Archives WorkstatioN
but we are open to other suggestions until the machine is installed and formally named!

We don’t want to become a computer museum with an extensive range of hardware, software and operating systems environments for any possible eventuality. We do want a core ability to handle material we reasonable expect to receive – including material on 3.5” floppy disks, zip disks, hard drives etc. We intend to develop and extend our capacity as need dictates – if we get material in a format we will consider whether we need to support this ourselves or whether a suitable 3rd party is more appropriate.

Central to this is the need for write blockers which prevent you from writing or updating the files. Having read countless websites I felt I knew what they were supposed to do but had a nagging doubt that my knowledge was incomplete.

A tour of the British Library eMss Labs courtesy of Jeremy Leighton John (as featured on the BBC Radio 4 programme 'Tales from the Digital Archives' broadcast in May but still available online) confirmed the simplicity of theory and the fragility of the media – just having the hardware isn’t enough – you also need some luck that you have the correct drivers to read the specific version of the media. In the next few weeks I hope to place our order for the various bits and pieces and will update you on this exciting journey!!

Curators Workbench workshop

I was fortunate enough to attend the Curator’s Workbench workshop at the British Library last week. It was a chance to see, have a play and discuss the tool with its developers Greg Jansen and Erin O’Meara from University of North Carolina. The tool is designed to aid with the accession, arrangement, description and staging of material prior to ingest into a digital repository. Essentially the tool has an interface designed for archivists can use.

The session featured a walk-through and chance to have a play with experts on-hand if you had a problem – only necessary as we had latest ‘unstable’ release including the latest enhancements to functionality and GUI. Stable versions are available for download via GitHub. I am especially smitten with the crosswalk feature providing a drag’n’drop interface for mapping the metadata with METS. There is also the date recogniser which allows you to map the date format to the ISO standard, though there could be issues if the data is in a variety of formats, ie 1984 would be transformed to 01/01/1984.

It has a different take to where arrangement and description occurs in the workflow to that intended for Hypatia in the AIMS workflow, but it does raise some interesting questions that I hope to explore in more detail over the next few months.

It was also interesting to hear features and functionality on their wish-list including disc images, multiple users, recording processing notes, PREMIS and so the list goes on!
The discussion that followed was really enlightening as it highlighted the different approaches that archives are currently adopting to the preservation of born-digital archives.

I picked-up some useful pointers to software and tools I haven’t used before – Bulk extractor, Google Refine, and came away determined to throw more stuff at Curators Workbench, to join the users discussion list (done) and to figure out some of the aspects we have avoided so far things like PREMIS and METS etc !

Photographing the digital: creating images of Hull University Archives’ digital media

A guest posting from Nicola Herbert, Digital Project Preservation Assistant at Hull University Archives

Over the last few months I have been working with the AIMS team at Hull University. My role entails getting stuck into some practical processing of the born-digital collections in the Hull University Archives as well as planning aspects of digital preservation. A lot of our work so far has been to discover and document the material that we already hold in what we thought were purely paper collections and I have written a workflow for the discovery of these items and their preparation for ingest into Fedora. As part of this workflow we decided to photograph all of the removable media we currently have and create a process for photography of new deposits when they arrive.

Why bother?
By retaining photographs of the original media alongside content we will be able to provide an image of the appearance of the original media to researchers if they request it. For the foreseeable future we are storing the image files on a shared drive, but they will eventually be stored as an element of metadata with the digital files in our Fedora Repository. We will be dealing with large numbers of media items so need to ensure consistency in the way the media is photographed and information recorded from those images.

Process
Having not previously numbered the discs, we decided on a simple running number within each accession. Despite our familiarity with labelling paper material, it seemed more complicated with digital. Our conservator advised against sticking labels (even conservation grade) onto the plastic casing of a floppy or Amstrad disc. Though a specialist CD marker can be used to label CDs, we were reluctant to permanently mark the items! After a worryingly long thought process we decided to stick to the old faithful method of writing in pencil on the existing label or case.

I then started planning the process. Despite trying to anticipate the different elements of information to include for each media type, it was only trial runs photographing actual media that gave the full picture - i.e. that Amstrad discs have three aspects to photograph (Side A, Side B and the edge). Lots of seemingly trivial questions arose - like whether to photograph the case or whether to photograph a label if blank. Getting the process right from the start will save time in the long run.

We decided to create a ‘clapperboard’ to photograph with the items for a failsafe way to ensure easy identification. I decided on a reusable form printed on a transparency which we can label with a drywipe marker. Putting theory into practice needed several trial runs; after each one I adapted the form and the procedure.

In addition I wrote up detailed notes describing the procedure for each type of media we anticipate encountering. We worked out a sensible image quality – so to ensure legibility of the labels without clogging up our servers with unnecessarily large images. Once the photographs have been taken they are renamed and filed. We also maintain an inventory of the items and record the media and label information alongside it. This ensures that if we send items (like our Amstrad discs) away to a third party we can match them to our records when they return.

This process has been satisfying to complete and enables us to tick at least one thing off our to-do list. Anyone can get this part of the process completed – even for material which is stored on a shared drive, photography of the original media is a useful process.

AIMS: the UnConference

Not two full weeks into my new job as Digital Archivist at UVa on the AIMS grant, I rolled up my sleeves to facilitate and host an unconference with my fellow Digital Archivists. Our unconference would be two full days of discussions, demonstrations, lightning talks, and networking with digital archivists from around the globe. At first the thought was a little terrifying – I’m not even fully sure I know what this job is yet, how could I actually lead discussions on the salient topics? But my fears were baseless: all the unconference attendees were thoughtful, articulate, and lively participants. I learned much more from them than they probably did from me.

The unconference was held on the 13th and 14th of May at the Omni Hotel in Charlottesville. The 27 participants represented libraries, archives, museums, and digital humanities centers across the US, Canada, and the United Kingdom. Despite the differences in our institutions, backgrounds, and training, we learned that we not only shared similar challenges, but also the same hopes for collaboration and innovation.

The first day started off with a round of lightning talks. Each participant had 5 minutes to present a topic, project, problem or idea that they were interested in talking about. The variety in the talks was remarkable to me, traversing the breadth and depth of all that can be thought of as “born-digital” and the many processes involved in managing it. The lightning talks were also great way to get an introduction to each participant as well as their perspective or the particular issues they were dealing with in their institution. A brief outline of each of the talks is available on the AIMS Unconference Wiki.

Workshop on "Using FTK Imager and AccessData FTK to Capture and Process Born Digital Materials"

On April 22, I conducted a 2-hour workshop on "Using FTK Imager and AccessData FTK to Capture and Process Born Digital Materials.” The purpose of the workshop was to give staff a hands-on experience in using FTK Imager and AccessData FTK. Eight colleagues from the Stanford University Libraries attended the workshop – primarily from Special Collections and University Archives and the Humanities and Social Sciences Group.

The workshop covered the following:

FTK Imager – how to:
1. Download and install the software (free software - http://accessdata.com/support/adownloads).
2. Create a forensic image of an USB flash drive.
3. Create a logical image of the same flash drive.

AccessData FTK – how to:
1. Load an image – for this workshop we used a sampling from the Stephen Jay Gould papers.
2. View technical metadata generated by the software.
3. Arrange column settings to see specific file attribute (e.g. duplicate files).
4. Search for social security numbers using pattern search.
5. Test the full-text search function.
6. Flag files with sensitive information with "privileged" tag (such as those with social security numbers, etc.)
7. Use the bookmark feature for hierarchical information and apply it to groups of files (e.g. series, subseries, etc.)
8. Label groups of files with user defined labels (controlled vocabulary for computer storage media, document type suggested in the workshop, subject headings or access rights, etc.)
9. View files with specific bookmarks and labels.

Many incoming collections are hybrid collections – containing both analog and digital material. The digital component will become even greater as we move forward. Empowering all archivists to use a tool such as AccessData FTK to process the digital materials would be very useful.