Parsing and manipulating .bib databases
This module contains functions
.bib) and examine the results;The core function is parse-bibtex-file, alongside its companion parse-bibtex-string. This reads a .bib file, and produces a list of entry? structures. These structures can be examined and processed using the various entry-* functions described below.
This produces a list of entry? structures, which might be usefully filtered with filter-entries.
Entries can have ‘local’ variables attached to them during processing. These are set and retrieved using entry-set-local-variable! and entry-get-local-variable respectively.
The three write-bibtex/FMT! functions take an entry list and write it out in one or other suitable formats.
For example, the following program will read an aux file, and write out a new .bib file containing only the entries cited.
(define auxfile-name "path/to/foo.aux")
;; ...includes \bibdata{...}
(module 'bibtex 'aux)
(let ((aux-info (parse-aux-file auxfile-name)))
;; aux-info is ((key key ...) (bibfile bibfile...) stylefile)
(let ((citations (car aux-info))
(bibfile (caadr aux-info))) ;first bibfile
;(printf "citations: ~s~%bibfile=~s~%" citations bibfile)
(let ((entries
(filter-entries (parse-bibtex-file bibfile)
citations)))
(write-bibtex/bib! entries))))See examples/extract-bib.scm for a more elaborate version of the same thing.
The overall structure of a beastie BibTeX parser is to parse a .bib file using parse-bibtex-file, and then extract from it the entries required (possibly using parse-aux-file and filter-entries as above). This gives a list of entry? objects.
It is frdequently convenient to then map these to a list of citation? objects before writing out the results. The object created by make-citation conveniently bundles together information about the citation, accessible via citation-entry, citation-html, citation-key, citation-reference. This is the approach taken by the examples/plain.scm style file in the distribution, but if it's not useful to use that style, then this bit of structure can be omitted.
When formatting an entry into an output, it's useful to use the with-fields-from-entry function to extract and validate entry contents, and the maybe-list, maybe-list/qq, maybe-sprintf functions to conditionally create output content, and the sentence macro to add or omit full stops as necessary.
For example, plain.scm contains a function format-entry-article, which looks like:
(define (format-entry-article e)
(with-fields-from-entry e
(author title journal year)
(volume number pages doi note url)
(let ((al (parse-author-list author)))
(make-citation
e
(sprintf "~a, ~a" (format-authorlist/ref al) (or year "n.d."))
`(li ,@(sentence
(a ((name ,(symbol->string (entry-key e))))
,(format-authorlist/text al))
,(sprintf " (~a), " year)
"‘" ,(titlecase-string/bst title) "’, "
(em ,journal)
,(or (maybe-sprintf " (~a)" number) ", ")
,(maybe-list 'strong (stringify/true volume) ", ")
;alternatively ,@(maybe-list `(strong ,(stringify/true volume)))
,@(maybe-list/qq " pp." nbsp ,(en-dashify pages)))
,@(format-links doi url)
,@(sentence ,note))))))Within the with-fields-from-entry form, the fields for author, title, journal and year are filled with the corresponding data, and if they are absent, then beastie prints a warning and defines them to have some dummy value (ie, they are guaranteed to be non-empty); and the other fields are filled with entry data, or are set to #f.
The function returns a citation? object containing the entry, a citation reference, and an xexpr for a bibliography entry (see the xexpr module). This (li...) list, corresponding to a <li> element, is written using standard Scheme quasiquotation, with the non-fixed parts of the list filled in as usual.
Here, the (maybe-sprintf " (~a)" number) will format the journal number containing the article. If number is present, then this will act as the function sprintf, but number is #f then this will expand to #f without error.
The function stringify/true (from the utils module) will stringify its argument, unless the argument is #f, when it will expand without error to #f. The form (maybe-list 'strong (stringify/true volume) ", ") will act list (list ...), unless any of its arguments are #f, when it will expand to a empty list (maybe-list/qq does the same, but quasi-quotes its arguments).
Finally, the (sentence ...) macro will expand to its contents as a list, except with a full stop appended. If this would be an empty sentence, then it expands to ().
Index:
TO GO: do not use
(bib-string-table k) : look up k in the .bib string table; the value may be a string? or ustring? object (case-sensitive).
(bib-string-table-set! k v) : set k to the value v in the .bib string table; both key and value may be string? or ustring? objects (case-sensitive).
Calling (bib-string-table-set! "jan" "January") is equivalent to finding a @string{jan="January"} in a .bib file, or MACRO{jan}{"January"} in a .bst file.
citation-entry : citation? -> entry? : return the entry? associated with the citation
citation-html c) : citation? -> xexpr? : returns a <li> element containing the bibliographic information for the citation.
citation-key : citation? -> symbol? : return the entry's citation key
citation-reference : citation? -> string? : returns the in-text reference for the citation, such as (Jones 1999)
citation? : any -> boolean? : true if X is a citation object.
A citation? object contains:
<li> element for inclusion in the bibliography (xexpr)entry-crossref : entry? -> (or entry? #f) : If the entry has a crossref field, then this returns that entry (as opposed to the crossref key)
entry-field : entry? symbol? -> (or ustring? #f) : Return one field from the entry, keyed by a symbol representing the field name.
Return #f if the field is not present.
(entry-field/authorlist entry? symbol?) : like entry-field, except that the value is assumed to be an author list, and is parsed to give a list of author? objects, or #f if the field is not present.
entry-field/crossref : entry? symbol? -> (or string? #f) : As ENTRY-FIELD, except that, if the entry has a crossref, then we follow the chain of such references.
entry-fields : entry-> (listof symbol?) : Return the field names of the entry, as a list of symbols.
entry-fields/alist : entry-> (listof (cons symbol? value)) : Return the fields of the entry, as an alist: (listof (cons symbol? value))
entry-get-local-variable : entry? symbol? -> (or any #f) : in entry E, get the entry-local value of a variable K which has been set with entry-set-local-variable! Return #f if the field is not present. We do not (at present) constrain the types of what can be stored here.
entry-key : entry? -> symbol? : Return the citation key of the entry, as a symbol
entry-missing-fields : entry? -> (or (listof (or symbol? (listof symbol?))) #f) : Return a list of required fields missing from the entry, or #f if there are none.
If alternate fields are required -- eg, book requires author or editor -- then the list entry will be a list of symbols, rather than a symbol.
The list of known entry types, and their required fields, is below. The list of optional fields is included here, for completeness.
article : An article from a journal or magazine. Required: author, title, journal, year. Optional: volume, number, pages, month, note.book : A book with an explicit publisher. Required: (author or editor), title, publisher, year. Optional: (volume or number), series, address, edition, month, note.booklet : A work that is printed and bound, but without a named publisher or sponsoring institution. Required: title. Optional: author, howpublished, address, month, year, note.conference : The same as INPROCEEDINGS, included for Scribe compatibility. Required: author, title, booktitle, year. Optional: editor, (volume or number), series, pages, address, month, organization, publisher, note.inbook : A part of a book, which may be a chapter (or section or whatever) and/or a range of pages. Required: (author or editor), title, (chapter or pages), publisher, year. Optional: (volume or number), series, type, address, edition, month, note.incollection : A part of a book having its own title. Required: author, title, booktitle, publisher, year. Optional: editor, volume, or, number, series, type, chapter, pages, address, edition, month, note.inproceedings : An article in a conference proceedings. Required: author, title, booktitle, year. Optional: editor, (volume or number), series, pages, address, month, organization, publisher, note.manual : Technical documentation. Required: title. Optional: author, organization, address, edition, month, year, note.mastersthesis : A Master’s thesis. Required: author, title, school, year. Optional: type, address, month, note.misc : Use this type when nothing else fits. Required: [None]. Optional: author, title, howpublished, month, year, note.phdthesis : A PhD thesis. Required: author, title, school, year. Optional: type, address, month, note.proceedings : The proceedings of a conference. Required: title, year. Optional: editor, (volume or number), series, address, month, organization, publisher, note.techreport : A report published by a school or other institution, usually numbered within a series. Required: author, title, institution, year. Optional: type, number, address, month, note.unpublished : A document having an author and title but not formally published. Required: author, title, note. Optional: month, year.If the entry type is not a known one, then nothing is required, so return #f.
entry-print! : entry? [output-port?] -> unspecified : print the given entry, in BibTeX format, to the optional port (stdout if missing). There is some normalisation in the output: required fields are placed before optional ones, which are in turn before unrecognised ones.
Note that any string abbreviations (ie, BibTeX fields like month=jan) will have been expanded before this point, so won't round-trip into this output.
entry-set-local-variable! : entry? symbol? any -> string? : In (entry-set-local-variable! E K V) in entry E, set the variable 'K to value V; also returns V. This provides a bit of local storage while processing. We do not (at present) constrain the types of what can be stored here.
entry-type : entry? -> symbol? : Return the type of the entry (ie, article, book, ...) as a symbol
entry<? : entry? entry? -> boolean? : Provide a sort order on entries, returning #t if the first argument should be ordered before the second. This is at present based on the entry-key, but may change in future.
entry? : any -> boolean? : Return true if the argument is a BibTeX entry. See PARSE-BIBTEX-FILE.
(falsy x) : true if the argument is falsy. An object is falsy if it is #f, '(), "", or #"".
filter-entries : (listof entry?) (listof symbol?) : In (filter-entries entry-list citation-list), return the entries in the given ENTRY-LIST which have keys which appear in the list of symbols in CITATION-LIST.
TO GO: do not use
`(list/true ...) : Evalates to its arguments if all of those arguments, and their sublists, are non-#f. Evaluates to '() otherwise. The contents are evaluated inside quasiquote.
DEPRECATED: use maybe-list instead.
make-citation : entry? string? xexpr? -> citation? : Use (make-citation entry reference li) to create a citation? object. The ENTRY is the parse BibTeX entry, the REFERENCE is a label such as (Jones 1999), and the LI is the bibliographic information formatted as a li xexpr?. This sort of object is created by programs which do the main work of formatting bibliographic information to HTML.
(maybe-list ...) : Like (list ...), except that if any of the items in the list, or in any sublists, are falsy`, then the whole evaluates to '().
(maybe-list/qq ...) : like MAYBE-LIST, except that the arguments are expanded inside quasiqote.
(maybe-sprintf "fmt" ...) : Like (sprintf fmt ...), except that if any of the arguments are falsy, then the whole evaluates to #f.
(parse-bibtex-file fn) : parses the contents of the indicated file, and returns a (key -> entry) hash-table, where the key is a symbol indicating an entry key, and the entry is an object of type entry?.
If any entries are mangled, and fail to parse, then they are discarded with a warning.
(parse-bibtex-string str) : parse the contents of the string, taken to be the contents of a .bib file. See parse-bibtex-file.
(sentence item ...) : expands to (list item ... ". "), wrapping a sequence of items in a sentence, typically in an xexpr bibliography entry.
The items are expanded inside quasiquote. Any items which are #f, '() or "" are removed.
If the result is an empty list, then this form evaluates to '().
If the final item is a string, then any trailing punctuation is stripped before the full-stop is added.
(sprintf/true "fmt" ...) : Like (sprintf fmt ...), except that if any of the arguments are #f, then the whole evaluates to "".
DEPRECATED: use maybe-sprintf instead.
(sprintf/true/default def "fmt" arg...) : Like (sprintf fmt ...), except that if any of the arguments are #f, then the whole evaluates to DEF.
DEPRECATED: use maybe-sprintf instead.
(truthy x) : true if x is truthy. Equivalent to (not (falsy x)).
(with-fields-from-entry entry (required-field ...) (optional-field ...) body ...) : Evaluate the body in a context where the symbols in the required-field and optional-field arguments are defined to have the values of the corresponding fields in the entry, or #f if absent, before being checked as below.
Each of the required-field elements can be a symbol or a list of symbols. In the former case, each of the fields must be present; if such a required field is absent, then a warning is issued, and it is defined locally to have a non-empty ‘flag value’ (currently the field name uppercased). In the latter case, at least one field must be present; fields that are absent are defined as #f; if none are present, then a warning is issued and one of the values is defined with the flag value.
Thus:
(with-fields-from-entry e
((author editor) title)
(volume number)
(let ((al (parse-author-list (or author editor))))
(if volume ... ...)
... ))If entry e has author, title and volume fields, then in the body, author, title and volume will have the corresponding field value, and editor and number will be #f; there will be no warnings. If, on the other hand, e has no title field, then a warning will be issued, and title set to the flag value. If both the author and editor fields are absent, then a warning will be issued, and one symbol will evaluate to the flag value.
write-bibtex/bib! : (listof entry?) -> unspecified : Write out a parsed BibTeX database (in the form of a list of entries), to the current output port, in .bib syntax.
Write out a parsed BibTeX database, to the current-output-port, in JSON syntax.
The translation to JSON should be fairly obvious, but (a) isn't really specified here so may be subject to change, and (b) can't necessarily be round-tripped back to BibTeX. Thus this should be regarded as an export format, rather than a transport one (the transport format is of course .bib format).
write-bibtex/sexp! : (listof entry?) -> unspecified : Write out a parsed BibTeX database to the current-output-port, as a sexp. The precise format is currently unspecified, and potentially subject to change, but should be fairly obvious.