nlp-pretraining
Best practices for language model pretraining and fine-tuning.
Skill by researchclaw
Chalk Labs
Need help deploying nlp-pretraining?
Chalk Labs sets up agents like this inside real teams: connected to your tools, tested on your data, with a human in the loop where it matters.
Install
Works with Claude Code, Claude apps, Codex, Cursor, Gemini CLI, GitHub Copilot, VS Code, OpenCode, Goose, Windsurf. Read the repository before you give any tool access to your data.
npx agentmarket install skill/raja21068-nlp-pretraining
Asks which of your tools to add it to (Claude Code, Codex, Cursor, VS Code, Windsurf, Gemini CLI, Claude Desktop) and shows the change first. See https://agentmart.dev/cli.
Get the weekly diff
The weekly diff: one email a week with new agent tools, notable updates and grade changes, plus security alerts for tools you follow. Double opt-in, one-click unsubscribe.
About
Reviews
Write a reviewNo reviews yet. Installed it? Reviews come from verified installers only. How reviews work
Permissions
What this skill can do and what it asks for, from its SKILL.md. Checked Oct 25, 2026.
- Tools
- No tools found
- Secrets
- None asked for
- Install scripts
- None found
Network hosts it talks to 1
codeload.github.com· found in install command
Files it touches
~/.claude/skills/nlp-pretraining
What's inside
- Skill name
nlp-pretraining- Description
- Best practices for language model pretraining and fine-tuning. Use when generating or reviewing NLP training code.
- License
- MIT
- Metadata
- author: researchclaw · version: 1.0 · category: domain · priority: 3 · references: Devlin et al., BERT, NAACL 2019; Hu et al., LoRA, ICLR 2022 · trigger-keywords: language model,pretraining,fine-tuning,bert,gpt,llm,transformer,nlp,text · applicable-stages: 9,10
- Instructions
- 20 lines in SKILL.md · view on GitHub
Files (1)
- SKILL.md
SKILL.md (first lines)
## NLP Pretraining/Fine-tuning Best Practice Fine-tuning recipe: - Use pre-trained checkpoints (HuggingFace hub) - AdamW optimizer, lr=2e-5 to 5e-5 - Linear warmup (6% of total steps) + linear decay - Batch size: 16-32 (use gradient accumulation for larger effective batch) - 3-5 epochs for classification, 1-2 for generation - Weight decay: 0.01 Parameter-efficient methods: - LoRA: r=8-64, alpha=16-128, apply to q/v projections - Prefix tuning: 10-20 prefix tokens - Adapters: bottleneck dimension 64-256 Evaluation: - Classification: accuracy, F1 (macro for imbalanced) - Generation: perplexity, BLEU/ROUGE, human evaluation - Use multiple seeds and report mean +/- std
From raja21068/AutoScholar, MIT.
More from raja21068/AutoScholar
- 0.409score
statistical-reporting
researchclaw
Statistical test selection, assumption checking, and APA-formatted reporting.
SkillDFreshness: DClaude Code +9 - 0.409score
chemistry-rdkit
researchclaw
Computational chemistry with RDKit for molecular analysis, descriptors, fingerprints, and substructure search.
SkillDFreshness: DClaude Code +9 - 0.409score
biology-biopython
researchclaw
Bioinformatics with Biopython for sequence manipulation, file parsing, BLAST, and phylogenetics.
SkillDFreshness: DClaude Code +9 - 0.409score
nlp-alignment
researchclaw
Best practices for LLM alignment techniques including RLHF, DPO, and instruction tuning.
SkillDFreshness: DClaude Code +9 - 0.409score
hypothesis-formulation
researchclaw
Structured scientific hypothesis generation from observations.
SkillDFreshness: DClaude Code +9 - 0.409score
literature-search
researchclaw
Systematic literature review methodology including search strategy, screening, and synthesis.
SkillDFreshness: DClaude Code +9 - 0.409score
rl-policy-optimization
researchclaw
Best practices for reinforcement learning policy optimization.
SkillDFreshness: DClaude Code +9 - 0.409score
scientific-visualization
researchclaw
Publication-ready scientific figure design with matplotlib and seaborn.
SkillDFreshness: DClaude Code +9 - 0.409score
scientific-writing
researchclaw
Academic manuscript writing with IMRAD structure, citation formatting, and reporting guidelines.
SkillDFreshness: DClaude Code +9 - 0.405score
cv-detection
researchclaw
Best practices for object detection tasks.
SkillDFreshness: DClaude Code +9 - 0.405score
experimental-design
researchclaw
Best practices for designing reproducible ML experiments.
SkillDFreshness: DClaude Code +9 - 0.405score
meta-analysis
researchclaw
Statistical methods for combining results across multiple studies.
SkillDFreshness: DClaude Code +9
Signals
- Stars
- 0
- Forks
- 0
- Open issues
- 0
- Last commit (this folder)
- Apr 12, 2026 (6 months ago)
From GitHub (fetched Oct 24, 2026).
Details
- Version
- 1.0
- License
- MIT
- Repository
- github.com/raja21068/AutoScholar
- First seen
- Oct 24, 2026
- Last checked
- Oct 24, 2026
- Source
- GitHubgithub-graphql
More from researchclaw
- 0.409score
scientific-visualization
researchclaw
Publication-ready scientific figure design with matplotlib and seaborn.
SkillDFreshness: DClaude Code +9 - 0.409score
rl-policy-optimization
researchclaw
Best practices for reinforcement learning policy optimization.
SkillDFreshness: DClaude Code +9 - 0.409score
nlp-alignment
researchclaw
Best practices for LLM alignment techniques including RLHF, DPO, and instruction tuning.
SkillDFreshness: DClaude Code +9 - 0.409score
hypothesis-formulation
researchclaw
Structured scientific hypothesis generation from observations.
SkillDFreshness: DClaude Code +9 - 0.409score
literature-search
researchclaw
Systematic literature review methodology including search strategy, screening, and synthesis.
SkillDFreshness: DClaude Code +9 - 0.409score
scientific-writing
researchclaw
Academic manuscript writing with IMRAD structure, citation formatting, and reporting guidelines.
SkillDFreshness: DClaude Code +9
Done for you
Rather not set it up yourself?
Chalk Labs scopes, builds and runs agents for your team: the task audit, the eval harness, the integrations and the weekly ops report. Fixed scope, priced before we start.
See how we build AI agents →Read the case studies
Chalk Labs is a two-founder AI and growth studio. We build AI agents, CRMs and MVPs for Web3 and AI companies, and have shipped 50+ product and token launches.
- launches shipped
- 50+
- launches shipped
- media placements
- 500+
- media placements
- founders on every account
- 2
- founders on every account
Press we've placed: Forbes · CoinDesk · Decrypt · Cointelegraph · The Block
Rahil Jain, Co-founder, Chalk Labs. Ex-Goldman Sachs engineer. You talk to the person who builds it.
Get nlp-pretraining running in your business
Tell us what you want to automate. A founder replies within one working day with a plan, not a pitch deck.
Cite this page
nlp-pretraining skill. Agent Mart. https://agentmart.dev/skills/raja21068-nlp-pretraining (last updated 2026-10-24)