detokenize

From Wiktionary, the free dictionary
Jump to navigation Jump to search

English

[edit]

Etymology

[edit]

From de- +‎ tokenize.

Verb

[edit]

detokenize (third-person singular simple present detokenizes, present participle detokenizing, simple past and past participle detokenized)

  1. (transitive, computing) To convert (a tokenized representation) back to the original form.