Respect my authorities

Spurred by the International Institute of Social History, which needs to act as a source of authorities to the rest of the world, we hope to implement the following features by the end of August 2010, with Evergreen 2.0 as the integration target:

Ability to have predictable control numbers for the records (both authority and bibliographic), suitable for use at an institution that needs to act as a source of authorities.

• Mike added the maintain_901() database function to ensure that the 901$c always contains the record ID from (authority|biblio|serial).record_entry.id, and added triggers correspondingly.
• Dan and Ole added the maintain_control_numbers() database function and added triggers correspondingly to the authority, bibliographic, and serial MARC record tables. This feature is controlled by the cat.maintain_control_numbers ( Cat: Maintain 001/003/035 according to the MARC21 specification ) global flag, which currently defaults to disabled. If enabled, the following happens on record creation or update:
  • Any existing 001 (Control Number) / 003 (Control Number Identifier) fields are moved into a new 035 (System Control Number) (if that combination does not yet exist); if there is no 001, then this step is ignored.
  • A new 001 is created using the raw numeric record ID.
  • A new 003 is created using a specified OU control Number Identifier (CNI). If the record has a value in the owner column, then the OU setting cat.marc_control_number_identifier is checked; if there is a value that applies to that OU, then that CNI is used. Otherwise, we fall back to the OU shortname.

Link controlled fields in bibliographic records to the authority record for that field via the $0 subfield, per the MARC specification (authority record 035 $a == bib controlled subfield $0). We need to track links so that changes to authority records will automatically generate the corresponding changes in any bibliographic records that are linked to a given authority record (the Syncs functionality, below).

• Mike committed r16816 to implements optional (controlled by a global flag) bib-authority linking via repeatable $0 subfields. The set of controlled fields will likely need to be expanded and adjusted, but this commit implements the same set that is used by the Validate button in the MARC Editor. The agency code in the $0 of the bib and the 035$a of the authority record are both ignored, and only the numeric code (the authority.record_entry.id, forced into the authority record) is used. For example, given the subfield $0(CONS)1234 in bibliographic record ID 9876, 1234 will be inserted into the authority column of the authority.bib_linking table and 9876 will be inserted into the bib column to track the authority-to-bib relationship.
• Dan committed server-side support for ensuring that when bibliographic records merge, the bib value in the authority-to-bib linking table will be updated accordingly r16935.

• Requirement: Ensure any updates to authority records automatically update the controlled fields in the bibliographic records. Should be relatively easy as long as it only affects $0-controlled fields.
  • Mike committed r16817 for an optional (controlled by global flag ingest.disable_authority_auto_update, enabled by default) authority update propagation function, attached to authority.record_entry via a trigger.
  • TODO One possible bug is that the old subfields are deleted and new subfields are added to the end of the field; this could play havoc with the proper order of subfields.
• Requirement: An existing set of records without controlled fields will need to be taught how to be controlled.
  • Dan committed authority_control_fields.pl that runs through a subset or all bibliographic records in the database and checks for matching authority records (based on exact matches of field/subfield(s)/values), then adds the $0 subfield accordingly to any controlled fields it finds. This is a slow update process, but in limited testing it seems to work - and it should only need to be run once for a given set of bibliographic records.

Enable appropriate use of tracing fields (the use of $w 0=g/h for broader/narrower terms). The idea is to build on the authority.tracing_links view already in Evergreen and expose its functionality at appropriate points in the staff client and user interface.

Make it easy to manually add an authority during bibliographic cataloguing, without interrupting the workflow of the bibliographic record the cataloguer was working on.

• Thought: Add Add this as an authority as another contextual option in the MARC editor - which would:
  • Create the new authority record based on the currently highlighted field
  • Grab the record ID for the new authority record
  • Add the appropriate $0 subfield using the are.id to the currently highlighted field
• Also want an Add this as an authority and edit option that would invoke the authority record MARC editor to allow fine-tuning of the authority MARC.

The current MARC editor does not know about any kind of MARC other than bibliographic records. Teach it to properly handle authority records so that the fixed fields don't look insane.

• Dan committed r16934 which taught the MARC editor about fixed fields for authorities.

Highlight in the user interface when an uncontrolled field is validated as a controlled field and linked to a specific authority record (this presupposes that we teach the "Validate" button to add the appropriate $0 to the field when a matching authority is found)

Improve authority selection interface during bibliographic cataloguing (such as the ability to invoke a browse list, perhaps; also to prevent cataloguers from choosing a See From tracing, etc).

• Part 1: in the current contextual display of the selected authority record, the cataloguer is able to pick as many subfields as they like from the entire record. This is too much information. Subfields outside of the 1* and 5* fields should be disabled, possibly even hidden, in this view.
• Part 2: enable the cataloguer to generate a browse list based on the authority browse list support that Mike added (via URLs like http://localhost/opac/extras/browse/marcxml/authority.author/CONS/Mulder http://localhost/opac/extras/startwith/marcxml/authority.author/CONS/Mulder). Ideally the browse list axis (authority.author, authority.subject, authority.title, authority.topic) and search term could be populated from the chosen uncontrolled term. The cataloguer should then be able to select the entry they want to apply to the previously focused field in the MARC editor. The display could be generated from the MARCXML browse list using something like:

  dojo.query('record'); // to grab the list of records, then for each record:
  dojo.query('datafield[tag^="1"]'); // to grab the controlled list of subfields from each record
  dojo.query('datafield[tag="901"] subfield[code="c"]'); // to grab the authority record ID

  The same sort of basic interface could be used to generate a list of authority records upon which the MARC editor could be invoked. Looks like BibTemplate to me … ;)

Make it possible to delete an authority record.

Make it possible to merge authority records, so that if you discover after a period of time that two records describe the same thing, you can automatically shift all affected bibliographic records to point to the merged authority.

Expose authority records for search and retrieval from the outside world via SRU and Z39.50. As we would need a basic set of indices to support SRU/Z39.50, we should also be able to build a fairly simple search interface into the client that would enable cataloguers to retrieve individual records for editing.

Implementation thoughts: largely cut and paste from the existing metabib indexing framework.