tokenize

Definition from Wiktionary, the free dictionary
Jump to navigation Jump to search

English[edit]

English Wikipedia has an article on:
Wikipedia

Etymology[edit]

token +‎ -ize

Pronunciation[edit]

Verb[edit]

tokenize (third-person singular simple present tokenizes, present participle tokenizing, simple past and past participle tokenized)

  1. (transitive, computing) To reduce to a set of tokens by lexical analysis.
  2. (transitive, computing) To substitute sensitive data with meaningless placeholders.
  3. (transitive) To treat as a token minority.

Derived terms[edit]

Related terms[edit]