R/text_core.R
parallel_tokenize.Rd
Calls optional `setup` on `tok` before launching `TokenizeWithRules` using `parallel_gen
parallel_tokenize(items, tok = NULL, rules = NULL, n_workers = 6)
items
tokenizer
rules
n_workers
None