AI and IIIF Ideas

I am following the Fast AI course and one of the suggestions was to blog as you follow the course. This blog will document some ideas for the second session. In the second session they look at creating an Image classifier. In the example they use Bing to get examples of three different types of Bears; Grizzly, Black and Teddy. The build a machine learning tool that when supplied with an image of a bear will tell them the type of bear.

I would like to use IIIF images, but the issue is getting labeled data that I can easily use. Having worked at the National Library of Wales I know their data best. A few options I’ve been thinking about:

LCTGM Classifier

LCTGM stands for Library of Congress Thesaurus for Graphic Materials and is a way to add subjects to graphical materials. It has the advantage of splitting out the subject heading into discreet parts namely Topic, Geographic and Temporal. I wonder if it would be possible to give the predictor an image and it will return suitable topics for that image.

Data

While working at the NLW there were a number of digitised image collections that were catalogued using LCTGM and these include:

• the John Thomas collection, which is a photographic collection but contains many pictures of people.
• the P B Abery collection which contains a good mix of people and places.
• the Geoff Charles collection which is a very large collection. Geoff Charles was a news photographer so there is a good mix of pictures but unfortunately this was catalogued at story level so any metadata applies to numerous images rather than a 1 to 1 relationship.

All of the above have IIIF images associated with them. Unfortunately the metadata isn’t directly downloadable but a number have been shared with Wikidata. For example to find all of the images from P B Abery collection you can run the following query:

SELECT ?item ?itemLabel ?manifest ?subjectLabel
WHERE 
{
  ?item wdt:P195 wd:Q74836239.
  ?item wdt:P6108 ?manifest
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}

Once you have the manifest e.g:

https://damsssl.llgc.org.uk/iiif/2.0/1293458/manifest.json

you can work out the ID and get to the metadata by going to:

http://dams.llgc.org.uk/behaviour/llgc-id:1294464/fedora-bdef:mets/mets

The LCTGM headings are in the MODS section of the METS documents.

Issues and thoughts

• P B Abery collection seems a good match for machine learning as it has a good range of subjects and is individually catalogued.
• But it will be very time consuming to download 1,746 manifests plus images.
• Some subjects will be more heavily populated than others… For the purposes of the demo I could restrict it to the top 3 most popular subjects
• Test images will need to be restricted to black and white photographs probably taken in a similar era (1911 - 1948)
• If it works it could be used to assign sensible topics to historical photographs.

I think on balance the effort to download the metadata and parse the METS is putting me off this idea.

Tribunal records form identification

While at the NLW I was involved with a crowd sourcing project to transcribe the Cardiganshire War Tribunal records. This is an amazing archive of records where people applied for exemption from enlisting into the Army during WW1. One of the issues with the crowdsourcing was that we found there were different types of forms in the archive, some were the same form but a different version as new versions were released during the war. Others were appeals after an initial application. Each different form required different fields to be entered. To solve this users were asked to identify which form they were transcribing before they started filling in the fields. Now this data has been entered would it be possible to create a predictor that could tell you which type of form you were looking at?

Data

This data is available as IIIF Annotations on GitHub:

https://github.com/NLW-paulm/Welsh-Tribunal-annotations

There is one JSON file per district and 11 districts in total. A district is made up of all of the forms and letters that the tribunal reviewed. They are ordered so that 1 persons application, appeal and any letters are located together. The JSON files are IIIF Annotation Lists and it looks like there is one annotation per page. The text of the annotation is split into fields e.g:

"chars": "Name of Local Tribunal: Aberaeron Borough/Urban District<p>Number of Case: 1<p>Name: Dewi Trefor Jones <p>Address: Castle House Aberaeron <p>Occupation, profession or 
business: Draper's Assistant <p>Attested or not attested: Attested <p>Grounds: I am 73 years of age, in ill health and can't stand any work. <p>Signature of appellant: John Hugh Jones <p>Addres
s of appellant: Castle House, Aberaeron <p>Occupation, profession or business: Draper <p>Why appellant acts for the man: Employer <p>Date: 1916-03-04T10:41:02.000Z<p>Tag: Pink R43/R44: Page 1"

Could be ordered as:

• Name of Local Tribunal: Aberaeron Borough/Urban District
• Number of Case: 1
• Name: Dewi Trefor Jones
• Address: Castle House Aberaeron
• Occupation, profession or business: Draper’s Assistant
• Attested or not attested: Attested
• Grounds: I am 73 years of age, in ill health and can’t stand any work.
• Signature of appellant: John Hugh Jones
• Address of appellant: Castle House, Aberaeron
• Occupation, profession or business: Draper
• Why appellant acts for the man: Employer
• Date: 1916-03-04T10:41:02.000Z
• Tag: Pink R43/R44: Page 1

The bit I am interested in is the Tag which says what type of page/document this is. Possible options are:

• Tag: Beige R186/187, page 1
• Tag: Beige R186/187, page 2
• Tag: Beige R41/42: Page 1
• Tag: Beige R41/42: Page 2
• Tag: Blue R52/53, page 1
• Tag: Blue R52/53, page 2
• Tag: Pink R43/R44: Page 1
• Tag: Pink R43/R44: Page 2
• Tag: Unknown document type

The Unknown document types are usually letters of support. A full annotation looks as follows:

        {
            "@type": "oa:Annotation",
            "motivation": "sc:painting",
            "on": "https://damsssl.llgc.org.uk/iiif/2.0/4001851/canvas/4001854.json#xywh=0,0,3497,4413",
            "resource": {
                "@type": "cnt:ContentAsText",
                "chars": "Description: Letter from Fred Burris & Sons from Horse Shoes and Mule Shoes for war department of British Government.<p>Transcription: We understand from Mr John Rees, 16 Albert St, that exemption has been refused him.   We however, suggest that there must be some error here, particularly as this man is badged No. L 16876, and consequently can only be taken with the permission of the Ministry of Munitions, \nWe think we have previously explained the matter and are at a loss to understand what has happened. The only thing, therefore, is for us to communicate with the War Office and inform them of the details,  \n<p>Name of Tribunal: Aberaeron Borough/Urban District<p>Number of Case: Appeal Form No 4<p>Tag: Unknown document type",
                "format": "text/plain"
            }
        },

The on field gives the required IDs for the IIIF Image and manifest the annotation applies to.

Issues and thoughts

• This data has now been captured for all of the digitised tribunal records, so a predictor may not be that useful
• Could potentially be applied to other collections. Most tribunal records were destroyed although it seems there are copies in Manchester Archive and some from the National Archives