Can a translation model reverse its source and target languages without retraining?

What one-direction translation training learns

An encoder-decoder translation model receives source tokens, builds contextual representations in the encoder, and generates target tokens in the decoder one at a time. The original Transformer is a sequence-to-sequence encoder-decoder model with a decoder that attends to the encoder output. Attention Is All You Need

For a Latin-to-English checkpoint, training adjusts parameters so that the probability of an English target sequence is high when the input is its Latin source. In compact notation, it learns a conditional distribution such as p_theta(English | Latin). Generation repeatedly predicts the next English token given the Latin encoding and the English tokens already generated.

A hypothetical Latin-to-English checkpoint

Suppose training contains the pair Latin "puella aquam portat" and English "the girl carries water." The encoder sees Latin tokens. During training, the decoder is guided toward English tokens. At inference, it has learned a procedure for reading Latin-like inputs and extending an English output. Giving it "the girl carries water" does not tell the model to use the decoder as an encoder or to change the English output distribution into a Latin one. It merely gives an English-shaped sequence to a component trained for Latin-shaped inputs.

The model might still produce text if the tokenizer can assign IDs to the English input. That is not evidence that it has learned English-to-Latin translation. A useful test is whether it succeeds on a held-out set of English sentences and trusted Latin references, not whether a single familiar sentence produces something plausible.

Why compatible shapes do not create an inverse

The same hidden size in the encoder and decoder is an interface choice. It allows the decoder's cross-attention layers to use the encoder's representations. It does not mean that the encoder's computation can be undone by running decoder layers in reverse order, or that their learned matrices represent a reversible change of basis.

Transformers contain attention, nonlinear feed-forward layers, normalization, residual paths, masking, and a final probability distribution over output tokens. These components are not trained as an invertible transformation. Translation itself also is not one-to-one: several Latin phrasings can map to the same English sentence, while one English sentence can have multiple valid Latin renderings depending on register and interpretation. Information about a particular source choice may be absent from the English output.

Knowing p_theta(English | Latin) therefore does not give the model p_theta(Latin | English). A mathematical relation such as Bayes' rule would also require a usable model of Latin sentence probabilities and the appropriate joint distribution. A standard translation checkpoint does not expose a reverse translator by rearranging its tensors.

BERT's use of bidirectional context does not change this conclusion. In that setting, bidirectional describes how token representations can attend to left and right context during its own masked-language-model training objective. It does not mean an encoder-decoder translation model can invert its source-to-target task. BERT paper

Tokenizers allow symbols, not translation directions

A tokenizer converts a string into token IDs. It may be shared across languages, or a model may keep source and target vocabularies or SentencePiece models. The Marian tokenizer documentation explicitly has source and target SentencePiece inputs and source and target language fields. Marian tokenizer parameters

This is a separate question from the model's learned translation direction. A shared subword tokenizer can represent both English and Latin characters and word pieces, yet the model may have seen English only on the decoder side and Latin only on the encoder side. Conversely, a model with separate vocabularies may encode an unsupported language mostly as unfamiliar units. SentencePiece is designed as a language-independent subword tokenizer for neural text processing, but its ability to segment raw text is not a claim that a particular translation checkpoint learned every language pair. SentencePiece paper

Language tags in multilingual systems solve a related control problem. They tell a checkpoint which trained source and target language behavior to use. For M2M-100, the documentation says a language-ID token prefixes source and target text, and target generation begins with the chosen target-language ID. That control signal has value because the model was trained as a many-to-many translator, not because the token itself reverses a bilingual model. M2M100 documentation

Bilingual and multilingual checkpoints answer different questions

The final row deserves caution. Multilingual training can share useful representations across languages, and the M2M-100 research describes a model trained over many language directions. M2M-100 research It does not follow that every imaginable pair, especially a lower-resource or historical language pair, will be usable or accurate. Support and quality are properties of the particular checkpoint, training data, and evaluation, not of the word "multilingual."

How to choose the reverse direction

1. Read the checkpoint's model card and tokenizer documentation. For many Marian checkpoints, the documented naming convention is opus-mt-{src}-{tgt}. A card marked source-language to target-language should be treated as a direction-specific model. MarianMT notes
2. Look first for an existing checkpoint explicitly trained or documented for English to Latin. This is the most direct replacement for a Latin-to-English-only model.
3. If using a multilingual checkpoint, confirm that both languages and the desired direction are supported. Set its source and target language identifiers exactly as its documentation requires. Do not infer the identifiers from a different model family.
4. If no suitable checkpoint exists, train or fine-tune a reverse-direction model on appropriately licensed English-Latin parallel text. This is new training, even if it begins from the original checkpoint's weights.
5. Evaluate the chosen model on sentences not used in training or tuning. Include morphology, word order, names, ambiguous phrases, and the genre you care about. For scholarly or publication use, have a qualified Latin reader review the output.

The appropriate choice depends on the intended use. A learner exploring architecture can try a multilingual model and inspect its language controls. A translation used for research, teaching, or publication needs direction-specific evaluation because plausibility is a weak substitute for correctness.