top of page

 Week 1:

The first week at the Celtic Arts Foundation (CAF) was one of exploration and discovery. I took a road trip up to Mt. Vernon to meet Elena Bianco, our sponsor for the fieldwork. Elena is a librarian for Skagit Valley College and is a longtime volunteer for CAF.

            The building itself is impressive, with an accessible wheelchair ramp at the front entrance of this building in this quaint, rural town tucked along the I-5 corridor connecting Seattle and Vancouver, BC. Famous for its tulip growing, the town quickly turns to farm country within a mile of the CAF facility.

            Inside, visitors enter a lobby covered with photographs of bagpipe players and smiling board members. The large performance space to the left hosts a stage and an open, multi-use floor with vaulted ceilings. Across the lobby, a board room with a long table contained a handful of volunteers who looked to be cataloging and identifying physical documents. Elena showed me a collection of books as well that they had been working on.

            One of the volunteers was Corey Cherrington, a previous UW MLIS student who has created a through and completely necessary Policies & Procedures document for CAF which will guide both Riley and I through our DFW.

            The majority of this week was spent reading the document and watching youtube videos that Corey created. The videos walk us through, step by step, using the software programs CAF uses, including Bagit, Sharepoint, and Preservica. I spent time mid-week creating the Student Learning Outcomes as I thought best.

            We met with Elena via zoom on Thursday morning and went through the document together, finally deciding to combine the best elements of Riley’s and mine so that we were both utilizing the same guiding outcomes. We also agreed that we will create a policy in the coming week (with Riley and I coming to next week’s meeting with rough drafts) that will guide us in identifying and tagging photographs that adhere to CAF’s goals of DEI.

            Our goal for the coming week is to pull as many photographs from Sharepoint as possible, downloading into Bagit and creating a spreadsheet that we can share with Elena so that she can verify we are identifying the correct photographs. I will concentrate on the Mastery of Scottish Arts photos, which are mainly of a series of classes and concerts, while Riley will handle Highland Games.

            We have a lot of work ahead of us and I hope we will be able to make some serious progress this semester, helping to organize CAF’s archives.

Week 1
Week 2

Week 2:

           The second week of my DFW was one of confusion, flexibility, and the realization that there was going to be a very steep learning curve on this project. I kept a list of questions for Elena as I went along. I realized, as the week went on, that one of the overarching questions I had was about exactly where the information about the files we were working on needed to be recorded. We had been given directions (through Corey’s videos) on bagging, refining, and trimming the data on files, but where exactly was I to put this information?

           I had started the process by trying to install bagit as per the instructional videos, but despite being able to download python, download bagit and find the command center as instructed, I just could not get bagit to run on my PC. I went back to Github (where the download was for bagit) and saw that there was an option for bagger. I remembered Corey had mentioned the program briefly in one of the videos. That did the trick. I was able to open bagger and quickly read through the help files to bag a file. Riley reached out with the same question—it seemed we were both having the issue, so we were happy to have this resolved. Elena confirmed that we should try to use bagger.

          I followed the instructions for downloading from Sharepoint, bagging the files and cleaning them up with OpenRefine. Corey’s videos were crucial for these processes and very thorough. Just when I thought we were going to have smooth sailing and be able to upload files to Preservica, I ran into an issue. I just could not manage to get the files uploaded with the metadata file attached. I could upload the photos okay, but the metadata just would not transfer.

          At our weekly meeting, we explained the issues to Elena. It seemed that Riley and I had many of the same questions and issues. Elena was able to tell us the steps we would take regarding pasting our data into the inventory template. We found that the link in the Policies & Procedures document was incorrect, so Riley was able to update that during the meeting. We agreed that we should try to set up a meeting with Corey to figure out what was going wrong with the uploads.

           I thought I should work on uploading in different ways. First, I thought I  should change the title in the first column of the metadata file: Corey mentioned labeling it “filename” but in the Dublin Core Template we downloaded from Preservica they have the first column labeled “file or folder name”. I also discovered that you cannot change the template file name, that it must stay “Dublin_Core-template”. So, I had some success there in that it would upload, though not all of the information transferred.

           I decided, too, that I should next try bagging just a single file. The first one I tried had embedded files and this may not be the ideal situation. Because we were having trouble with the upload stage, and until we talk to Corey, we decided that next week we could spend just bagging and refining more files and hold off on the uploading.

           We noted, also, that later in the semester is when we will start adding further descriptions and metadata to the individual photos we have uploaded (info like names of people in the photos.) We also need to make some naming rules for the project before we get too far—for example, right now, some files that are already uploaded all have the same title of “2007 Highland Games”. We want to distinguish between them, but be able to have a rule that we will be able to easily duplicate for other categories. We agreed to work on that over the next few weeks.

           Though technology can be extremely frustrating, there is much in this project to stretch our thinking and many opportunities to learn. Elena and Riley are a joy to work with, so it’s a great environment to learn in.  

Week 3:
This was a week of solidifying procedures. For the first time, through trial and error, I was able to upload a set of photographs to Preservica together with a metadata file, which tracked and attched to the appropriate photos. There were several steps I had to take in order to make this work:
1) I removed the .xmp and the .TIF files from both the folder and the metadata sheet so that I was only dealing with .jpegs
2) I did not attempt any nested files 
3) The metadata sheet needed to be saved as a .csv file and put into the folder with the photos (not outside the file)
4) As noted last week, the .csv file MUST be named Dublin_Core-template.csv every time you upload a file. For those who might be doing this after me, make sure it does not save as an excel file!
5) After the photos upload successfully in Preservica, there are a series of questions you are asked: 
a) Select a delimiter for the .csv file: Choose Comma-it has worked every time. (See Fig 1 in Repository)
b) There is an error for specific files (See Fig 2 in Repository). That is okay. The files should still ingest.

     Elena, Riley, and I had our weekly meeting and then the next day met with Corey as well, as we had questions. We asked her about the naming conventions that we were seeing in Preservica (which made no sense to us), and I asked about how to batch edit and if I made the right call to pull out the .xmp and .TIF files. Basically, she confirmed we were on the right track and that the best way to figure out both edits and uploads is to keep doing what we are doing, trial and error. Or we also have the option to watch help videos from Preservica, which Elena agreed were quite good.
     We found we are going to have to update the Policies & Procedures, as there are more details that could be shared about this process. I am keeping a running list of issues to include.
     My goal this next week is to finish uploading all of the Mastery of Scottish Arts 2008 folders. I will also have to figure out how to upload the .xmp and .TIF files that I omitted from the first upload. Elena thinks it's possible there is some metadata within the .xmp files that could be included at some point. 
     The good news is that on subsequent tries, I was able to upload a nested folder, as long as it was under 1.5 GB (and did not include .xmp files). There are still a few questions we have as a group around naming conventions-- more about that next week as we try to come up with solutions. 
 

Week 3

Week 4:
      We did not hold our weekly meeting because of scheduling conflicts, so we asked questions via email. My questions (and Elena's answers) were:
 

1) There are a bunch of .PREL and .PRV files that I cannot open, though I've started uploading them to Preservica anyway. I'm wonder if I should keep doing that or if I should just wait on those files. They are Adobe Premier Elements files, but I don't have that program. They are uploading fine, they just show as blank files for me (no thumbnail.)

 

I would actually prefer not transferring these over to Preservica because I am wondering if they are duplicates of the jpgs that are already over there. If not, they would need to be converted into a file format that can be viewed in Preservica. I'd prefer not to use up the limited space in Preservica with those.

 

2) There are also some very large video files that are too big. I'm wondering if I should try sending to myself via WeTransfer? Do you think that could work for uploading to Preservica?

 

I don't know. Let's hold off on videos. Again, we have to pay for additional space on Preservica, so if it's too large to transfer, I'd prefer to deal with it later - maybe the videos can be edited for length or split up or something like that.

 

3) It's my inclination to just keep uploading what I can--I'm almost done with the 2008 file, keep notes on what I can't upload, and move on. Since we are almost halfway through the semester, I would love to be able to upload as many years as possible.

 

That sounds like a good plan - just keep track of what doesn't get sent over so we have a record of it.

 

4) There's a question I have about a finicky matter: On the Dublin Core metadata sheet we have the  as the Sharepoint file relationship, for example: MSA/MSA2008/Slideshow. But on the Archival Inventory Sheet, we input it under "Original Folder Title" with ">" rather than "/", i.e. MSA>MSA2008>Slideshow. Should we keep up with this format even though it's different, or should we change one of them now? It's not a huge deal that they are different, it just makes cutting and pasting more laborious.

 

Oh, I didn't notice that! I think we should go with what's on the Dublin Core metadata sheet and keep the two consistent.

     This work is very precise and takes a lot of focus, but I feel like this was the week that I got the basics down. For the next few weeks it will be just a matter of getting it done. It would be nice if we had enough time to go back and work on cleaning up/adding more metadata towards the end of the semester.
I was able to complete uploading the whole 2008 file as planned and have started an excel sheet that tracks all of the folders and files that I did not upload to Preservica.
     One of the things I learned is that to do this work remotely, you need a fast computer with a lot of free space as well as very fast internet. I don't have the ideal situation, so I had to make room on my hard drive in order to keep files until I can upload.
     I haven't had time to look into opening the .xmp files, but I hope to before the semester is over. We are already nearly halfway through and I have only uploaded 1 file so far. Onward and upward!

Week 4

Week 5: 
      The fifth week with CAF was more fruitful than the last. I had my rhythm down of downloading from Sharpoint, bagging, refining the metadata with Open Refine and Excel, and uploading to Preservica. 
      At our weekly meeting, we filled out our mid-term evaluations and Elena seemed pleased with the progress we'd made. At that point (Friday, April 26th) I had uploaded nearly a thousand photos with metadata. We made a few decisions regarding formatting and other questions I had. 
      There had been some stumbling blocks along the way, the first being that I had to pull and separate all of the files we chose NOT to upload to Preservica. I have created another file (Figure 3) wherein I list the file names and filepath for those files I did not preserve. Given the lofty goal of trying to get as much uploaded as possible in the next few weeks, we decided I should press on with the uploads.
      We did choose, however, to revise some of my outcomes, mainly, that I will continue the uploads for another two weeks, aiming to get through 2012, and then, in the last few weeks of the semester, I will aim to organize policies for any incoming interns as well as adding further metadata to what I have uploaded (i.e. information under the "description" metadata field and perhaps changing file names). 
     I've discovered in the days since our meeting that there are many more nested files and different file types that may need further weeding this week. I was relieved to see that the Preservation copy of the CR2s that I uploaded did indeed save as Jpegs, as I had indicated in the Format field (it was just a guess at that point, as I could not find any information on how they might upload in the help files). That meant I did not have to go back into the files and edit. I've tried again to "Bulk edit" metadata with no luck thus far. I am hoping at the end of this week to try again, perhaps digging through the help section to try to learn. This would be a very useful tool in the coming weeks, rather than having to edit fields one by one. Even Corey didn't seem to know how to do it successfully. I'm determined to figure it out!
I've included examples of the finished files and their current metadata in Figure 4. 

Week 5

Week 6:
      This was the most productive week of the semester, as I was on a bit of a roll, downloading, bagging, refining, and uploading. And uploading. And uploading. I managed to process over a thousand photos this week, with a total upload of 2,025 photos in the years 2007, 2008, and 2009. I continued to track the files that we did not upload (video and .xmp files) to Preservica. 
      At our weekly meeting, we discussed a few issues (one being that I am not allowed to format dates in the master Archive Inventory file (they are showing as 2/7/2009 rather than the 2009-02-07 format that the column is supposed to require). The format cells function seems to be locked for me. Elena will look into that. 
      I tried again in vain to bulk edit, but I am not deterred. I will find out this week why this function says "something went wrong" (See Figure 5) whenever I follow the prompts. I will continue to blog about my process here as I solve the mystery.
      Elena and I decided that I should stop at 2009 (though that feels a bit like a defeat, not making it to 2012), because we want to make sure that I can get those 2000 photos in good shape, adding more metada, and updating the policies. There were definitely more documents than either one of us expected within the files we uploaded. 
      I imagine the upcoming weeks will provide further learning opportunities in the metadata, taxonomy and archiving realms. 
 

Week 6
Week 7

 Week 7:
      This week I spent a good amount of time researching how to do bulk edits. I watched Preservica videos, read community help posts, and tried to bulk edit myself again according to the directions in these posts, all to no avail. Finally, I ran across a blog post wherein someone mentioned that bulk edit is only available if you have the Preservica Pro subscription, which CAF does not have. Elena looked up the price, which we thought might be worth it, but it seems cost prohibitive without a grant or some other kind of funding. But, if they are to be consistent throughout their archives, the bulk edit will be essential. Without it, archivists will need to edit the photos one at a time, and there are thousands. 
      I will be saving the Dublin Core templates for each of the files for the next interns, as it may be less time-consuming to delete the files and re-upload with new metadata, vs. having to change fields one by one. Either way, without the bulk edit option, many many hours will be spent making sure that all of the files are uniform.
      In our weekly meeting, we realized, as we only have 3 weeks left, that we have to come to some decisions regarding structure of the files and naming conventions. Corey had named her Robert Burns documents all the same name as the file name with sequential numbering, for example, 2009 MSA-1, 2009 MSA-2. Originally, Elena, Riley and I did not like this idea, but we are coming around on it as the filenames are all over the place currently. We did make the decision to move "original file title:" from the Description field to the end of the file folder location stream in the Relation field. For example: Scottish/Images/Events/MSA/MSA2009/_G0x7345.jpg  Therefore, we will have a record of what the original file was named. We also have that record in the Archive Inventory sheet. 
      This means, of course, that in the MSA files 2007, 2008, and 2009, which I've already uploaded, the original file title will still be in the Description field, and without bulk editing, will have to be changed by hand. That's 2,025 files. I intend to created instructional documents to be added to the Policies and Procedures from these blog posts, which will help future interns to know what has been done and what hasn't. Currently, the log for documents pulled out and left in Sharepoint is over 500. What they will do with those videos and .xmp files is unclear. 
      All of this decision-making means, however, that we have shifted back to wanting me to upload as many photos as possible from the MSA files in the next 3 weeks. I will likely get through 2012 as was the goal a few weeks ago. That means, though, that I likely won't do any adding of descriptors or changing of titles. Though I'm not actually doing the work of identifying and naming people in the photographs, I am following Riley's progress in doing so. Our projects are similar, though he has less photos per file, so that gives him the opportunity to slow down the ingest and spend more time discovering what the photos are about and adding to the Description field. 

     

Week 8

Week 8:
At this week's meeting Riley and I lamented about the frustrations of when the uploads did not work. We shared tips for getting it to work. Elena had contacted Preservica to see if she could get further answers about the bulk edit, but did not have anything more than a contact at that point. 
Elena agreed that we should keep our DC files and made a new depository in Preservica to upload those files. Riley is going to include his files with the photos because there are not many (in case the next interns wish to re-upload vs. editing one by one. I, however, will only upload the DC template (renamed) because there are so many files in the MSA folders. I've included a sample template filled out with a folder's metadata here in Figure 6.
Though my main goal remains continuing to upload as many photos as I can, this next week Riley and I will also input comments into the Policies and Procedures so that Elena can take a look at that before we make any changes to the documents. We will use the comment function in the Word Doc until changes are approved. 
We did not make any further decisions on taxonomy and decided, as Elena said, to "kick it down the road". There really isn't any sense in taking the time to rename files if they aren't sure how they would like them to be in perpetuity. 

Week 9

Week 9:
This penultimate week of the DFW was all about trying to get as much uploaded as possible. I wanted to make sure to get through the 2012 folder, which I eventually did, though a couple of the nested folders with large amounts of photos gave me trouble--mainly that I would have to pay attention and keep touching the screen every 30 seconds or so for nearly 2 hours so the screen wouldn't time out. It's not ideal, as sometimes I would get involved with downloading the next folder in another screen and I'd be forced to start over. Well, I managed it eventually.
      The other project this week was going through and making notes in the CAF Policies and Procedures. After addressing the notes I'd made at tour weekly meeting (See Fig 7), we decided that we should make separate PDF packets that describe the possible pitfalls and solutions we learned and put a link in the Policies and Procedures document, rather than adding a large amount of material.
      So, since I was able to get through 2012 (totaling 3,791 photos uploaded and added metadata), that will allow me to spend this last week preparing documents to make it easier for the next interns.
      Elena was kind enough to offer Riley and I letters of recommendation should we need them. We will meet next week for our final wrap-up. 

Week 10
      This last week was a real push to get all of the instructional documentation together in order to help the next group of interns. I started by revising and clarifying a few of the lines in the Policies and Procedures regarding using Bagger instead of Bagit, as well as separating out some of the crammed together instructions. I then put together a separate document (see Fig 8) that includes much of my learnings from this blog, as well as the artifacts I uploaded (at least the relevant figures.) I also included instructions for anything that gave me trouble. Since I still had a few hours left, I decided to make a goal of uploading the MSA file from 2013. It was a big one, so it took a while, but I managed to do it. I ended with a total upload for the semester of 4,070 photos, with 610 photos and files extracted and documented as such.

At our last meeting, Riley, Elena, and I looked over the document and Riley added a detail to the instructions before we said our final goodbyes. Elena invited us to continue volunteering if we would like to!

Week 10
bottom of page