TextBlock_from_folder — TextBlock_from

Build a `TextBlock` from a `path`

TextBlock_from_folder(
  path,
  vocab = NULL,
  is_lm = FALSE,
  seq_len = 72,
  backwards = FALSE,
  min_freq = 3,
  max_vocab = 60000,
  tok = NULL,
  rules = NULL,
  extensions = NULL,
  folders = NULL,
  output_dir = NULL,
  skip_if_exists = TRUE,
  output_names = NULL,
  n_workers = 6,
  encoding = "utf8"
)

Arguments

path: path
vocab: vocabualry
is_lm: is_lm
seq_len: sequence length
backwards: backwards
min_freq: minimum frequency
max_vocab: max vocabulary
tok: tokenizer
rules: rules
extensions: extensions
folders: folders
output_dir: output_dir
skip_if_exists: skip_if_exists
output_names: output_names
n_workers: number of workers
encoding: encoding

Value

None