nemotron-environment-setup

Files (1)

• SKILL.md

SKILL.md (first lines)

# Nemotron Environment Setup

Use this skill when setting up a fresh dfw environment, auditing dependency paths, or debugging why a run is using the wrong NeMo-RL, Megatron-Bridge, Megatron-LM, or SGLang code.

## New User Variables

The concrete paths in this skill are parameterized through the variables below.  Use this mapping for a fresh setup:

```bash
export USER_NAME=<your_dfw_user>
export GITHUB_USER=<your_github_user>

export HOME_DIR=/home/${USER_NAME}
export REPO_DIR=${HOME_DIR}/diffusion_RL/RL

export LUSTRE_USER_ROOT=/lustre/fsw/portfolios/coreai/users/${USER_NAME}
export DRIVER_ENV_ROOT=${LUSTRE_USER_ROOT}/nemorl_uv_driver_envs
export WORKER_ENV_ROOT=${LUSTRE_USER_ROOT}/nemo_rl_worker_venvs
export UV_CACHE_ROOT=${LUSTRE_USER_ROOT}/uv_cache
```

Then create the base directories:

```bash
mkdir -p "${HOME_DIR}/diffusion_RL" "${LUSTRE_USER_ROOT}"
```

If your workspace uses a different Lustre portfolio or project path, set `LUSTRE_USER_ROOT` to that location and keep the driver envs, worker envs, and UV cache underneath it.

## Canonical Working Tree

For the current reference deployment, the main repo is:

```bash
cd "${REPO_DIR}"
```

For a new user, clone the NeMo-RL fork used by this workflow into `REPO_DIR` and check out `dllm_clean`:

```bash
git clone --branch dllm_clean git@github.com:sajadn/RL.git "${REPO_DIR}"
cd "${REPO_DIR}"
```

Check the current branch and commit before changing dependencies:

```bash
git branch --show-current
git rev-parse HEAD
git status --short
```

## Forks Submodules (Megatron-Bridge and Megatron-LM)

The repo uses local editable proxy packages in `pyproject.toml`, backed by Git submodules:

```toml
megatron-bridge = { path = "3rdparty/Megatron-Bridge-workspace", editable = true }
megatron-core = { workspace = true }
```

The source repos live one level deeper:

```text
3rdparty/Megatron-Bridge-workspace/Megatron-Bridge
3rdparty/Megatron-LM-workspace/Megatron-LM
```

Initialize or repair submodules with:

```bash
git submodule sync --recursive
git submodule update --init --recursive
```

Expected `.gitmodules` intent:

```text
Megatron-Bridge: git@github.com:sajadn/Megatron-Bridge.git, branch nemotron-diffusion
Megatron-LM:     https://github.com/NVIDIA/Megatron-LM.git
```

Verify the actual checked-out sources:

```bash
git submodule status --recursive

git -C 3rdparty/Megatron-Bridge-workspace/Megatron-Bridge remote -v
git -C 3rdparty/Megatron-Bridge-workspace/Megatron-Bridge branch --show-current
git -C 3rdparty/Megatron-Bridge-workspace/Megatron-Bridge rev-parse HEAD

git -C 3rdparty/Megatron-LM-workspace/Megatron-LM remote -v
git -C 3rdparty/Megatron-LM-workspace/Megatron-LM rev-parse --abbrev-ref HEAD
git -C 3rdparty/Megatron-LM-workspace/Megatron-LM rev-parse HEAD
```

If committing submodule changes in NeMo-RL, first commit/push the nested repo change in the submodule repo, then commit the updated gitlink in the NeMo-RL repo. Git records only the submodule commit hash, not the whole nested repository contents.

## SGLang Fork

Current setup uses a local editable SGLang checkout in `pyproject.toml`:

```toml
sglang = { path = "${HOME_DIR}/diffusion_RL/sglang-nemotron-dllm-a652eb48/python", editable = true }
```

That local checkout should track the fork:

```text
git@github.com:sajadn/slgang-nemotron-diffusion.git
```

Set up or refresh it with:

```bash
git clone git@github.com:sajadn/slgang-nemotron-diffusion.git \
  "${HOME_DIR}/diffusion_RL/sglang-nemotron-dllm-a652eb48"
cd "${HOME_DIR}/diffusion_RL/sglang-nemotron-dllm-a652eb48"
git checkout restore/a652eb48-good-run

From sajadn/diffusion_RL, Apache-2.0.

More from sajadn/diffusion_RL