Kwalk: A Simple Program to Crosswalk Metadata for Repository Uploads

Abstract

University of Chicago's Center for Digital Scholarship has been utilizing this program to better edit metadata for batch upload to Knowledge@UChicago. There are plans to share this software in the future as it is platform agnostic and has a potential wide range of use cases. Suppose you need to upload 1,000 items to TIND from a source like Lens.org or PLOS journals. You obtain informal metadata for the items by you or another person creating the spreadsheet from scratch, exporting the data, or web scraping each individual record. You might need to do the following after obtaining the data: Rename all the fields in the from the invented field names to TIND's field names; Add some fields that are missing; Leave out some fields you don't want; Combine several fields into one field; Modify the values of date formats or author names in a programmatic way; Generate syntactically correct upload URLs from a simple filename field. Kwalk is a program that lets us write a simple crosswalk that we can apply to each batch of metadata as we receive it and have multiple crosswalks for multiple projects as we work on them in an intermixed fashion. The program allows us to apply special functions to modify date formats, combine literal and field name text, generate uniform upload URLs, and much more.