data-processing-nemo-curator

GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.

Install

KitScout test notes

Not tested by KitScout yet. When our harness runs it, pros, cons and the tasks we tried appear here, with the date and evidence.

Similar tools

• docx 193,970 ★
  Use this skill whenever the user wants to create, read, edit, or manipulate Word documents (.docx files) or Wo
• pdf 193,970 ★
  Use this skill whenever the user wants to do anything with PDF files. This includes reading or extracting text
• pptx 193,970 ★
  Use this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This include
• xlsx 193,970 ★
  Use this skill any time a spreadsheet file is the primary input or output. This means any task where the user
• backup-setup 116,728 ★
  Set up, check, or restore local claude-mem database snapshots. Use when the user says "set up backups", "back