Tokenize texts in `df[text_cols]` in parallel using `n_workers`
tokenize_df(
df,
text_cols,
n_workers = 6,
rules = NULL,
mark_fields = NULL,
tok = NULL,
tok_text_col = "text"
)data frame
text columns
number of workers
rules
mark_fields
tokenizer
tok_text_col
None