Zowie

Adds Zotero "select" links to attachment files in a Zotero database on macOS, so that outside of Zotero, you can find the bibliographic entry to which a file belongs. (Only works for local storage, not linked attachments.)

This project is maintained by mhucka

Zowie (“Zotero link writer”) is a command-line program for macOS that writes Zotero select links into the file attachments contained in a Zotero database, so that outside of Zotero, you can find the bibliographic entry to which a file belongs. Only works for local storage, not linked attachments.

License Python DOI GitHub stars Latest release PyPI

Table of contents

Introduction

When using Zotero, you may on occasion want to work with PDF files and other attachment files outside of Zotero. For example, if you’re a DEVONthink user, you will at some point discover the power of indexing your local Zotero database from DEVONthink. However, when viewing or manipulating the attachments from outside of Zotero, you may run into the following problem: when looking at a given file, how do you find out which Zotero entry it belongs to?

Enter Zowie (a loose acronym for Zotero link writer”, and pronounced like the interjection). Zowie scans through the files on your disk in a local Zotero database, looks up the Zotero bibliographic record corresponding to each file found, and writes a Zotero select link into the file and/or certain macOS Finder/Spotlight metadata fields (depending on the user’s choice). A Zotero select link has the form zotero://select/... and when opened on macOS, causes the Zotero desktop application to open that item in your database. Zowie thus makes it possible to go from a file opened in an application other than Zotero (e.g., DEVONthink, Adobe Acrobat), to the Zotero record corresponding to that file.

Regretfully, Zowie can only work with Zotero libraries that use normal/local data storage; it cannot work when Zotero is configured to use linked attachments.

Installation

There are multiple ways of installing Zowie, ranging from downloading a self-contained, single-file, ready-to-run application, to installing it as a typical Python program using pip. Please choose the alternative that suits you.

Alternative 1: downloading the ready-to-run application

Depending on the version of macOS you are using, please see the relevant section below.

macOS version 10.15 (Catalina) or later

First, you need a Python interpreter version 3.8 or higher on your computer. Happily, that’s the version provided by macOS 10.15 and later, but if you have never run python3 on your computer, then the first time you do, macOS will ask if you want to install certain additional software components. (Let it do so.) Check the version of the program python3 that you get by running the following command in a terminal and inspecting the results:

python3 --version

Next,

  1. Go to the latest release on GitHub and find the Assets section
  2. Click on the ZIP file whose name contains your version of Python
  3. Unzip the downloaded file (if your browser didn’t unzip it)
  4. Open the folder that gets created (it will have a name like zowie-1.2.0-macos-python3.8)
  5. Look inside for zowie and move it to a location where you put other command-line programs (such as /usr/local/bin).

If you want to put it in /usr/local/bin but that folder does not exist on your computer yet, you can create it by opening a terminal window and running the following command (prior to moving zowie into /usr/local/bin):

sudo mkdir /usr/local/bin

The following is an example command that you can type in a terminal to move Zowie there:

sudo mv zowie /usr/local/bin

macOS 10.14 (Mojave) and earlier

A runnable version of Zowie created using a different method is available for macOS before 10.15, for which Apple did not provide Python version 3. This copy of Zowie works like any normal command-line program and does not require Python. (However, it does not run on macOS Catalina or later due to Apple security issues.)

  1. Go to the latest release on GitHub and find the Assets section
  2. Click on zowie.zip to download it
  3. Unzip the file; this will leave you with a file named zowie, which is the program itself
  4. Move zowie to a folder where you put other command-line programs (e.g. /usr/local/bin).

If you want to put it in /usr/local/bin but that folder does not exist on your computer yet, you can create it by opening a terminal window and running the following command (prior to moving zowie into /usr/local/bin):

sudo mkdir /usr/local/bin

The following is an example command that you can type in a terminal to move Zowie there:

sudo mv zowie /usr/local/bin

Alternative 2: installing Zowie using pipx

You can use pipx to install Zowie. Pipx will install it into a separate Python environment that isolates the dependencies needed by Zowie from other Python programs on your system, and yet the resulting zowie command wil be executable from any shell – like any normal application on your computer. If you do not already have pipx on your system, it can be installed in a variety of easy ways and it is best to consult Pipx’s installation guide for instructions. Once you have pipx on your system, you can install Zowie with the following command:

pipx install zowie

Pipx can also let you run Zowie directly using pipx run zowie, although in that case, you must always prefix every Zowie command with pipx run. Consult the documentation for pipx run for more information.

Alternative 3: installing Zowie using pip

The instructions below assume you have a Python 3 interpreter installed on your computer. Note that the default on macOS at least through 10.14 (Mojave) is Python 2 – please first install Python version 3 and familiarize yourself with running Python programs on your system before proceeding further.

You should be able to install zowie with pip for Python 3. To install zowie from the Python package repository (PyPI), run the following command:

python3 -m pip install zowie

As an alternative to getting it from PyPI, you can use pip to install zowie directly from GitHub:

python3 -m pip install git+https://github.com/mhucka/zowie.git

If you already installed Zowie once before, and want to update to the latest version, add --upgrade to the end of either command line above.

Alternative 4: installing Zowie from sources

If you prefer to install Zowie directly from the source code, you can do that too. To get a copy of the files, you can clone the GitHub repository:

git clone https://github.com/mhucka/zowie

Alternatively, you can download the files as a ZIP archive using this link directly from your browser using this link: https://github.com/mhucka/zowie/archive/refs/heads/main.zip

Next, after getting a copy of the files, run setup.py inside the code directory:

cd zowie
python3 setup.py install

Usage

For help with usage at any time, run zowie with the option -h.

The zowie command-line program should end up installed in a location where software is normally installed on your computer, if the installation steps described in the previous section proceeded successfully. Running Zowie from a terminal shell then should be as simple as running any other shell command on your system:

zowie -h

If you installed it as a Python package, then an alternative method is available to run Zowie from anywhere, namely to use the normal approach for running Python modules:

python3 -m zowie -h

Credentials for Zotero access

Zowie relies on the Zotero sync API to get information about your references. This allows it to look up Zotero item URIs for files regardless of whether they belong to your personal library or shared libraries, and from there, construct the appropriate Zotero select link for the files. If you do not already have a Zotero sync account, it will be necessary to create one before going any further.

To use Zowie, you will also need both an API user identifier (also known as the userID) and an API key. To find out your Zotero userID and create a new API key, log in to your Zotero account at Zotero.org and visit the Feeds/API tab of the your Settings page. On that page you can find your userID and create a new API key for Zowie.

The first time you run Zowie, it will ask for this information and (unless the -K option is given) store it in your macOS keychain so that it does not have to ask for it again on future occasions. It is also possible to supply the identifier and API key directly on the command line using the -i and -a options, respectively; the given values will then override any values stored in the keychain and (unless the -K option is also given) will be used to update the keychain for the next time.

Basic usage

Zowie can operate on a folder, or one or more individual files, or a mix of both. Suppose your local Zotero database is located in ~/Zotero/. Perhaps the simplest way to run Zowie is the following command:

zowie ~/Zotero

If this is your first run of Zowie, it will ask you for your userID and API key, then search for files recursively under ~/Zotero/. For each file found, Zowie will contact the Zotero servers over the network and determine the Zotero select link for the bibliographic entry containing that file. Finally, it will use the default method of recording the link, which is to write it into the macOS Finder comments for the file. It will also store your Zotero userID and API key into the system keychain so that it does not have to ask for them in the future.

If you are a user of DEVONthink, you will probably want to add the -s option (see the explanation below for the details):

zowie -s ~/Zotero

Instead of a folder, you can also invoke Zowie on one or more individual files (but be careful to put quotes around pathnames with spaces in them, such as in this example):

zowie -s "~/Zotero/storage/26GS7CZL/Smith 2020 Paper.pdf"

Zowie supports multiple methods of writing the Zotero select link. The option -l will cause Zowie to print a list of all the methods available, then exit.

The option -m can be used to select one or more methods when running Zowie. Write the method names separated with commas without spaces. For example, the following command will make Zowie write the Zotero select link into the Finder comments as well as the PDF metadata attribute Subject:

zowie -m findercomment,pdfsubject ~/Zotero/storage

At this time, the following methods are available:

Note that, depending on the attribute, it is possible that a file has an attribute value that is not visible in the Finder or other applications. This is especially true for “Where from” values and Finder comments. The implication is that it may not be apparent when a file has a value for a given attribute, which can lead to confusion if Zowie thinks there is a value and refuses to change it without the -o option.

Filtering by file type

By default, Zowie acts on all files it finds on the command line, except for certain files that it always ignores: hidden files and files with extensions .sqlite, .bak, .csl, .css, .js, .json, .pl, and a few others. If the -m option is used to select methods that only apply to specific file types, Zowie will examine each file it finds in turn and only apply the methods that match that particular file’s type, but it will still consider every file it finds in the directories it scans and apply the methods that are not limited to specific types.

You can use the option -f to make Zowie filter the files it finds based on file name extensions. This is useful if you want it to concentrate only on particular file types and ignore other files it might find while scanning folders. Here is an example (this also using the -s option for reasons given below):

zowie -s -f pdf,mp4,mov ~/Zotero

will cause it to only work on PDF, MP4, and QuickTime format files. You can provide multiple file extensions separated by commas, without spaces and without the leading periods.

Filtering by date

If the -d option is given, the files will be filtered to use only those whose last-modified date/time stamp is no older than the given date/time description. Valid descriptors are those accepted by the Python dateparser library. Make sure to enclose descriptions within single or double quotes. Examples:

zowie -d "2 weeks ago" ....
zowie -d "2014-08-29" ....
zowie -d "12 Dec 2014" ....
zowie -d "July 4, 2013" ....

Special-case behavior

Although Zowie is not aimed solely at DEVONthink users, its development was motivated by the author’s desire to use Zotero with that software. A complication arose due to an undocumented feature in DEVONthink: it ignores a Finder comment if it is identical to the value of the “URL” attribute (which is the name it gives to the com.apple.metadata:kMDItemWhereFroms attribute discussed above). In practical terms, if you do something like write the Zotero select link into the Finder comment of a file and then have a DEVONthink smart rule copy the value to the URL field, the Finder comment will subsequently appear blank in DEVONthink (even though it exists on the actual file). This can be unexpected and confusing, and has caught people (including the author of Zowie) unaware. To compensate, Zowie 1.2 introduced a new option: it can add a trailing space character to the end of the value it writes into the Finder comment when using the findercomment method. Since approaches to copy the Zotero link from the Finder comment to the URL field in DEVONthink will typically strip whitespace around the URL value, the net effect is to make the value in the Finder comment just different enough from the URL field value to prevent DEVONthink from ignoring the Finder comment. Use the option -s to make Zowie to add the trailing space character.

Additional command-line arguments

To make Zowie only print what it would do without actually doing it, use the -n “dry run” option.

If given the -q option, Zowie will not print its usual informational messages while it is working. It will only print messages for warnings or errors. By default, messages printed by Zowie are also color-coded. If given the option -C, Zowie will not color the text of messages it prints. (This latter option is useful when running Zowie within subshells inside other environments such as Emacs.)

If given the -V option, this program will print the version and other information, and exit without doing anything else.

If given the -@ argument, this program will output a detailed trace of what it is doing. The debug trace will be sent to the given destination, which can be - to indicate console output, or a file path to send the output to a file.

When -@ has been given, Zowie also installs a signal handler on signal SIGUSR1 that will drop Zowie into the pdb debugger if the signal is sent to the running process.

Summary of command-line options

The following table summarizes all the command line options available.

Short      Long form opt     Meaning Default  
-a A --api-key A API key to access the Zotero API service    
-C --no-color Don’t color-code the output Use colors in the terminal  
-d --after-date D Only act on files modified after date “D” Act on all files found  
-f --file-ext F Only act on files with extensions in “F” Act on all files found
-h --help Display help text and exit    
-i --identifier I Zotero user ID for API calls    
-K --no-keyring Don’t use a keyring/keychain Store login info in keyring  
-l --list Display known services and exit    
-m --method M Select how Zotero select links are written findercomment  
-n --dry-run Say what would be done, but don’t do it Do it  
-o --overwrite Overwrite previous metadata content Don’t write if already present  
-q --quiet Don’t print messages while working Be chatty while working  
-s --space Append trailing space to Finder comments Don’t add a space
-V --version Display program version info and exit    
-@ OUT --debug OUT Debugging mode; write trace to OUT Normal mode

⚑   Certain files are always ignored: hidden files, macOS aliases, and files with extensions .sqlite, .sqlite-journal, .bak, .csl, .css, .js, .json, .pl, and .config_resp.
⬥   To write to the console, use the character - as the value of OUT; otherwise, OUT must be the name of a file where the output should be written.
★   See the explanation in the section on special-case behavior.

Return values

This program exits with a return code of 0 if no problems are encountered. It returns a nonzero value otherwise. The following table lists the possible return values:

Code Meaning
0 success – program completed normally
1 the user interrupted the program’s execution
2 encountered a bad or missing value for an option
3 no network detected – cannot proceed
4 file error – encountered a problem with a file
5 server error – encountered a problem with a server
6 an exception or fatal error occurred

Known issues and limitations

The following is a list of currently-known issues and limitations:

Additional tips

In the wiki associated with the Zowie project in GitHub, I have started writing some notes about how I personally use Zowie to combine Zotero with DEVONthink.

Getting help

If you find an issue, please submit it in the GitHub issue tracker for this repository.

Contributing

I would be happy to receive your help and participation if you are interested. Everyone is asked to read and respect the code of conduct when participating in this project. Development generally takes place on the development branch.

License

This software is Copyright (C) 2020-2021, by Michael Hucka and the California Institute of Technology (Pasadena, California, USA). This software is freely distributed under a 3-clause BSD type license. Please see the LICENSE file for more information.

Acknowledgments

This work is a personal project developed by the author, using computing facilities and other resources of the California Institute of Technology Library.

The vector artwork of an exclamation point circled by a zigzag, used as the icon for this repository, was created by Alfredo @ IconsAlfredo.com from the Noun Project. It is licensed under the Creative Commons CC-BY 3.0 license.

Zowie makes use of numerous open-source packages, without which Zowie could not have been developed. I want to acknowledge this debt. In alphabetical order, the packages are:

The developers of DEVONthink, especially Jim Neumann and Christian Grunenberg, quickly and consistently replied to my many questions on the DEVONtechnologies forums.