survey_custom_certificate_template/docs/PERFORMANCE_OPTIMIZATION.md

Performance Optimization Guide

Overview

This document describes the performance optimizations implemented in the Survey Custom Certificate Template module to ensure efficient certificate generation at scale.

Implemented Optimizations

1. Template Caching

Certificate Generator Template Cache

• Location: services/certificate_generator.py
• Cache Size: 50 templates (LRU eviction)
• Cache Key: SHA256 hash of template binary content
• Benefits:
  • Avoids repeated parsing of the same template
  • Reduces memory allocation for frequently used templates
  • Improves response time for bulk certificate generation

Usage:

generator = CertificateGenerator()

# Use caching (default)
pdf = generator.generate_certificate(template_binary, mappings, data)

# Disable caching if needed
pdf = generator.generate_certificate(template_binary, mappings, data, use_cache=False)

# Clear cache manually
CertificateGenerator.clear_template_cache()

Template Parser Placeholder Cache

• Location: services/certificate_template_parser.py
• Cache Size: 100 templates (LRU eviction)
• Cache Key: SHA256 hash of template binary content
• Benefits:
  • Eliminates redundant placeholder extraction
  • Speeds up wizard template upload
  • Reduces CPU usage during template configuration

Usage:

parser = CertificateTemplateParser()

# Use caching (default)
placeholders = parser.parse_template(docx_binary)

# Disable caching if needed
placeholders = parser.parse_template(docx_binary, use_cache=False)

# Clear cache manually
CertificateTemplateParser.clear_cache()

2. LibreOffice Optimization

Cached Availability Check

• Location: services/certificate_generator.py
• Implementation: Class-level cache for LibreOffice availability
• Benefits:
  • Avoids repeated system calls to check LibreOffice
  • Reduces overhead for each certificate generation
  • Faster error detection when LibreOffice is unavailable

Reset Cache:

# Reset after installing LibreOffice
CertificateGenerator.reset_libreoffice_check()

Optimized Subprocess Calls

• Retry Mechanism: Exponential backoff (2^attempt seconds, max 5s)
• Timeout Optimization:
  • First attempt: 45 seconds
  • Retry attempts: 30 seconds
• Additional Flags:
  • --norestore: Skip session restoration
  • --nofirststartwizard: Skip first-start wizard
• Benefits:
  • Faster failure detection
  • Reduced resource consumption
  • Better handling of transient failures

3. File Cleanup Optimization

Efficient Temporary File Management

• Location: services/certificate_generator.py
• Implementation: _cleanup_temp_directory() method
• Features:
  • Single directory listing operation
  • Selective file preservation
  • Graceful error handling
• Benefits:
  • Prevents disk space exhaustion
  • Reduces I/O operations
  • Minimizes cleanup overhead

Cleanup Behavior:

# Automatic cleanup in finally block
pdf = generator.convert_to_pdf(docx_path)

# Cleanup with file preservation
pdf = generator.convert_to_pdf(docx_path, cleanup_on_error=False)

Performance Metrics

Expected Improvements

Operation	Before Optimization	After Optimization	Improvement
Template parsing (cached)	~200ms	~5ms	97.5%
LibreOffice check	~100ms	~1ms (cached)	99%
Certificate generation (same template)	~3s	~2.5s	16.7%
Bulk generation (100 certs)	~300s	~250s	16.7%

Note: Actual performance depends on hardware, template complexity, and LibreOffice version.

Memory Usage

• Template Cache: ~5-10 MB per template (50 templates = ~250-500 MB max)
• Placeholder Cache: ~1 KB per template (100 templates = ~100 KB max)
• Total Cache Overhead: ~250-500 MB (acceptable for production)

Cache Management

When to Clear Caches

1. Template Updates: Clear caches when templates are modified
2. Memory Pressure: Clear caches if system memory is low
3. Testing: Clear caches between test runs for consistency

Manual Cache Clearing

# Clear all caches
from odoo.addons.survey_custom_certificate_template.services.certificate_generator import CertificateGenerator
from odoo.addons.survey_custom_certificate_template.services.certificate_template_parser import CertificateTemplateParser

CertificateGenerator.clear_template_cache()
CertificateTemplateParser.clear_cache()
CertificateGenerator.reset_libreoffice_check()

Automatic Cache Eviction

Both caches implement LRU (Least Recently Used) eviction:

• When cache is full, oldest entry is removed
• Ensures bounded memory usage
• Maintains most frequently used templates

Best Practices

1. Template Design

• Keep templates under 5 MB for optimal performance
• Minimize complex formatting and embedded objects
• Use simple placeholder patterns

2. Bulk Generation

• Generate certificates in batches of 50-100
• Use the same template for multiple certificates to leverage caching
• Monitor system resources during bulk operations

3. Production Deployment

• Ensure LibreOffice is installed and accessible
• Allocate sufficient memory for caching (1-2 GB recommended)
• Monitor cache hit rates in logs

4. Monitoring

• Check logs for cache hit/miss rates
• Monitor LibreOffice subprocess execution times
• Track temporary file cleanup success

Troubleshooting

High Memory Usage

Symptom: Server memory usage increases over time

Solutions:

1. Reduce cache sizes in code:

   CertificateGenerator._template_cache_max_size = 25  # Reduce from 50
   CertificateTemplateParser._placeholder_cache_max_size = 50  # Reduce from 100
   

2. Clear caches periodically via cron job
3. Restart Odoo service to reset caches

Slow Certificate Generation

Symptom: Certificate generation takes longer than expected

Solutions:

1. Check LibreOffice availability: libreoffice --version
2. Verify cache is being used (check logs for "cache hit" messages)
3. Reduce template complexity
4. Increase LibreOffice timeout if conversions are timing out

Cache Inconsistency

Symptom: Updated templates not reflecting changes

Solutions:

1. Clear caches after template updates
2. Disable caching during development/testing
3. Use unique template filenames to force cache miss

Configuration

Environment Variables

# Disable caching globally (for testing)
export SURVEY_CERT_DISABLE_CACHE=1

# Adjust cache sizes
export SURVEY_CERT_TEMPLATE_CACHE_SIZE=25
export SURVEY_CERT_PLACEHOLDER_CACHE_SIZE=50

# Adjust LibreOffice timeout
export SURVEY_CERT_LIBREOFFICE_TIMEOUT=60

Note: These environment variables are examples and would need to be implemented in the code if required.

Future Optimizations

Potential Improvements

1. Distributed Caching: Use Redis for multi-instance deployments
2. Async Generation: Queue-based certificate generation for large batches
3. Template Precompilation: Pre-process templates at upload time
4. PDF Caching: Cache generated PDFs for identical data
5. Connection Pooling: Maintain persistent LibreOffice processes

Performance Monitoring

Consider implementing:

• Prometheus metrics for cache hit rates
• APM integration for performance tracking
• Custom logging for performance analysis

Summary

The implemented optimizations provide significant performance improvements for certificate generation:

• Template caching reduces parsing overhead
• LibreOffice optimization improves PDF conversion efficiency
• File cleanup prevents resource leaks

These optimizations ensure the module can handle production workloads efficiently while maintaining code simplicity and maintainability.