build-feature-store

Files (1)

• SKILL.md

SKILL.md (first lines)

# Build Feature Store


> See [Extended Examples](references/EXAMPLES.md) for complete configuration files and templates.

Implement centralized feature management with Feast for consistent feature serving across training and inference.

## When to Use

- Managing features for multiple ML models across teams
- Ensuring training-serving consistency for features
- Implementing point-in-time correct historical features
- Serving low-latency features for real-time inference
- Reusing feature definitions across projects
- Versioning feature transformations
- Building feature catalog for discovery and governance
- Preventing feature leakage in training pipelines

## Inputs

- **Required**: Raw data sources (databases, data lakes, data warehouses)
- **Required**: Python environment with Feast installed
- **Required**: Offline store backend (BigQuery, Snowflake, Redshift, or Parquet files)
- **Required**: Online store backend (Redis, DynamoDB, Cassandra, or SQLite for dev)
- **Optional**: Feature transformation logic (Python, SQL, Spark)
- **Optional**: Entity key definitions (user_id, product_id, etc.)
- **Optional**: Kubernetes cluster for Feast server deployment

## Procedure

### Step 1: Initialize Feast Feature Repository

Set up Feast project structure and configure storage backends.

```bash
# Install Feast with required extras
pip install 'feast[redis,postgres]'  # Add backends as needed

# Initialize new feature repository
feast init my_feature_repo
cd my_feature_repo

# Directory structure created:
# my_feature_repo/
# ├── feature_store.yaml       # Configuration
# ├── features.py              # Feature definitions
# └── data/                    # Sample data (dev only)
```

Configure `feature_store.yaml`:

```yaml
# feature_store.yaml
project: customer_analytics
registry: data/registry.db  # SQLite for dev, use S3/GCS for prod
provider: local

# Offline store for training data
offline_store:
  type: postgres
# ... (see EXAMPLES.md for complete implementation)
```

Production configuration with cloud backends:

```yaml
# feature_store.prod.yaml
project: customer_analytics
registry: s3://feast-registry/prod/registry.db
provider: aws

offline_store:
  type: bigquery
  project_id: my-gcp-project
# ... (see EXAMPLES.md for complete implementation)
```

**Got:** Feast repository initialized with config file, sample feature definitions created, offline and online stores configured, registry path accessible.

**If fail:** Verify database/Redis credentials (`psql -U feast_user -h localhost`), check connection strings format, ensure databases exist (`CREATE DATABASE feature_store`), verify cloud permissions for S3/BigQuery/DynamoDB, test connectivity to storage backends, check Feast version compatibility with backends (`feast version`).

### Step 2: Define Entities and Data Sources

Create entity definitions and connect to raw data sources.

```python
# entities.py
from feast import Entity, ValueType

# Define entities (primary keys for features)
customer = Entity(
    name="customer",
    description="Customer entity",
    value_type=ValueType.INT64,
# ... (see EXAMPLES.md for complete implementation)
```

Define data sources:

```python
# data_sources.py
from feast import FileSource, BigQuerySource, RedshiftSource
from feast.data_format import ParquetFormat
from datetime import timedelta

# Development: File-based source
customer_transactions_source = FileSource(
    path="data/customer_transactions.parquet",
# ... (see EXAMPLES.md for complete implementation)
```

**Got:** Entity definitions reference correct ID columns, data sources connect to raw data successfully, event_timestamp_column exists in source data, created_timestamp_column allows point-in-time queries.

**If fail:** Verify source data files exist and are readable, check BigQuery/Redshift credentials and table access, ensure timestamp columns have correct format (Unix timestamp or ISO8601), verify Kafka connectivity and topic existence, check schema compatibility between sources and entities.

### Step 3: Define Feature Views with Transformations

Create feature views that define how raw data becomes ML-ready features.

From pjt222/agent-almanac, MIT.

More from pjt222/agent-almanac