Performance Optimization Guide¶

This document explains the performance optimizations implemented in PoMa to accelerate submodule cloning and setup times.

The Problem¶

The original setup process was slow due to: - Nokia submodule: 1.5GB repository with full git history - OpenConfig submodule: 4.4MB but still slow with full history
- Traditional git clone --recursive: Downloads complete git history (~5+ minutes)

Optimization Strategies¶

1. Shallow Clone Configuration¶

.gitmodules optimization:

[submodule "nokia"]
    path = nokia
    url = https://github.com/nokia/7x50_YangModels.git
    shallow = true
    branch = master

Benefits: - Downloads only latest commit instead of full history - Reduces Nokia clone from 1.5GB to ~100MB - 90%+ reduction in download time

2. Fast Setup Script¶

scripts/setup-submodules-fast.sh provides multiple optimization levels:

Mode	Command	Speed	Use Case
CI	`./setup-submodules-fast.sh ci`	~10s	Automated builds
Shallow	`./setup-submodules-fast.sh shallow`	~20s	Development (default)
Full	`./setup-submodules-fast.sh full`	~3-5min	Complete git history

CI Mode Optimizations:

# Ultra-aggressive optimizations for CI
git config submodule.recurse false
git submodule update --init --depth 1 --single-branch --no-fetch

3. Makefile Automation¶

make setup provides one-command setup:

setup: setup-env setup-fast setup-bgp
    @echo "✅ Complete PoMa setup finished!"

Benefits: - Automated workflow - Error handling - Progress feedback - Consistent results

4. GitHub Actions Optimization¶

CI pipeline optimization:

- name: Fast submodule setup for CI
  run: |
    chmod +x scripts/setup-submodules-fast.sh
    ./scripts/setup-submodules-fast.sh ci

Results: - CI build time reduced from 8+ minutes to under 2 minutes - More reliable builds with targeted dependency management

Performance Comparison¶

Setup Time Benchmarks¶

Method	Time	Size Downloaded	Description
`make setup`	~30s	~120MB	Complete automated setup
`setup-submodules-fast.sh ci`	~10s	~100MB	Ultra-fast for CI
`setup-submodules-fast.sh shallow`	~20s	~120MB	Fast with shallow clones
`git submodule update --recursive`	3-5min	~1.5GB	Traditional full clone

Real-World Impact¶

Before optimization:

$ time git clone --recursive https://github.com/user/poma.git
real    4m 32s
user    0m 28s  
sys     0m 15s

After optimization:

$ time make setup
real    0m 28s
user    0m 8s
sys     0m 3s

🎯 Result: 90% reduction in setup time

Implementation Details¶

Shallow Clone Benefits¶

Traditional clone:

# Downloads entire git history
git submodule update --init --recursive
# Nokia: 1.5GB (hundreds of commits)
# OpenConfig: 15MB+ (full history)

Optimized clone:

# Downloads only latest commit
git submodule update --init --depth 1 --recursive
# Nokia: ~100MB (single commit)
# OpenConfig: ~4MB (single commit)

Git Configuration Optimizations¶

Submodule configuration:

# Disable automatic recursive operations
git config submodule.recurse false

# Use single branch for faster clones
git config submodule.nokia.branch master
git config submodule.open-config.branch master

Network optimizations:

# Parallel submodule operations
git config submodule.fetchJobs 4

# Shallow clone by default
git config submodule.shallow true

When to Use Each Method¶

Development Workflow¶

# Initial setup (one time)
make setup

# Daily development
source venv/bin/activate
make validate

CI/CD Pipeline¶

# Fastest for automated builds
./scripts/setup-submodules-fast.sh ci

Full History Needed¶

# When you need complete git history
./scripts/setup-submodules-fast.sh full

Troubleshooting¶

# Check submodule status
make status

# Clean and retry
make clean
make setup

Advanced Optimizations¶

Custom Git Configuration¶

For ultra-fast clones:

# Global git optimizations
git config --global submodule.shallow true
git config --global submodule.fetchJobs 8
git config --global fetch.parallel 8

Network-Specific Optimizations¶

For slow networks:

# Use CI mode with minimal data transfer
./scripts/setup-submodules-fast.sh ci

For fast networks:

# Use shallow mode for balance of speed and completeness
./scripts/setup-submodules-fast.sh shallow

Monitoring Performance¶

Built-in Benchmarking¶

# Test setup performance
make benchmark

# Monitor submodule sizes  
make status

Custom Timing¶

# Time any setup method
time ./scripts/setup-submodules-fast.sh ci
time make setup

Troubleshooting Performance Issues¶

Slow Network¶

Use ci mode for minimal data transfer
Consider caching submodules in CI systems

Large Repository Growth¶

Monitor with make status
Use git gc in submodules for cleanup
Consider switching to shallow mode permanently

CI Timeouts¶

Use ./scripts/setup-submodules-fast.sh ci
Enable git credential caching
Consider repository mirrors for enterprise use

Future Optimizations¶

Planned Improvements¶

Git LFS integration for large binary assets
CDN caching for popular model versions
Incremental updates for model changes
Compressed model archives for ultra-fast distribution

Experimental Features¶

Git partial-clone for even smaller downloads
Model version pinning for reproducible builds
Local model caches for offline development

The performance optimizations in PoMa demonstrate that with proper configuration and tooling, even large YANG model repositories can achieve fast, reliable setup times suitable for both development and CI/CD workflows.