Export

Converts the raw predictions.jsonl of a run into the released format.

retexo.export

Predicted edit scripts in the released format.

A run of the driver writes one predictions.jsonl per test fold, with the model's raw scores, the gold it was scored against, and the code's internal tags (NOP for a copy, MORPH for an inflection). This module turns such a file into one self-contained record per pair, with the labels of the paper and no gold, so the predictions can be shared and read without the code that produced them.

One record, in SCHEMA_VERSION 1::

{
  "id": "p0001", "benchmark_id": 1, "fold": 4, "reference_type": "cf.",
  "source": {"author": "verg", "work": "verg. aen.", "citation": "<verg. aen. 6.847.1>", "tokens": [...]},
  "reuse":  {"author": "hier", "work": "hier. epist.", "citation": "<hier. epist. 117.7.1.1>", "tokens": [...]},
  "links": [{"reuse": 5, "source": 2, "label": "COPY", "relation": null, "confidence": 0.9994}, ...],
  "frame": [{"start": 0, "end": 6}],
  "insertions": [0, 1, 2, ...],
  "deletions": [0, 1, 3, ...]
}

Indices are word positions in tokens. links has one entry per linked reuse word; label is one of COPY, INFLECT, SUBST, SPLIT and MERGE, and a SUBST link may name its relation (SYN, POS, ...). frame lists the citing formulas as half-open ranges of reuse words, insertions the unlinked reuse words that are not part of one, and deletions the source words no link claims.

convert_record

convert_record(raw: Dict[str, Any], *, fold: Optional[int] = None) -> Dict[str, Any]

One raw prediction record as a released record.

Raises ValueError when the record contradicts itself (a link without a tag, a deletion that a link claims), so a corrupt file fails here and not in somebody's analysis.

read_predictions

read_predictions(path: Path | str, *, fold: Optional[int] = None) -> Iterator[Dict[str, Any]]

The released records of one raw predictions.jsonl.

export_folds

export_folds(sources: Sequence[Path | str], out_dir: Path | str, *, fmt: str = 'jsonl') -> Dict[int, int]

Convert one raw file per test fold into out_dir/fold_<k>.<fmt>; returns pairs per fold.

The fold is read from the records themselves, so the order of sources does not matter.

label_shares

label_shares(records: Iterable[Dict[str, Any]]) -> Dict[str, float]

The share of each label among all links, as a sanity check against a paper table.

label_of

label_of(tag: str) -> tuple[str, Optional[str]]

The label and the relation a code tag stands for; ("SUBST", "SYN") for SYN.