Skip to main content

Conversion Funnel Metrics System

Conversion Funnel Analytics View

Introduction​

The Wrkbelt Conversion Funnel Metrics System provides comprehensive analytics for understanding customer journey optimization in service booking flows. Unlike simple funnel visualizations that only show aggregate conversion rates, our system delivers actionable insights through three integrated metric categories: Executive Metrics (high-level performance indicators), Step Conversion Rates (detailed funnel progression analysis), and Step Dropoff Analysis (dimensional breakdown of abandonment patterns).

The system transforms raw booking session data into precise business intelligence by analyzing user step progression, engagement patterns, and dimensional performance breakdowns. Each metric includes period-over-period trend analysis with statistical validation to ensure meaningful insights surface only when there's sufficient data for reliable conclusions.

Architecture Overview​

Statistical Foundation​

Our conversion funnel metrics system builds on the same statistical rigor as our insights system:

  • Data Threshold Validation: Minimum 10 sessions per period for reliable trend calculations
  • Period-over-Period Comparison: Equivalent timeframe comparison for trend analysis
  • Engagement Time Outlier Filtering: Percentile-based filtering (2.5th-97.5th for n>100, IQR for smaller datasets)
  • ObjectId Normalization: Consistent string conversion for step ID matching across MongoDB operations

Multi-Layered Architecture​

Core Components​

1. ConversionFunnelService​

The primary orchestrator for all conversion funnel analytics, implementing business logic for step analysis and trend calculations:

@Injectable()
export class ConversionFunnelService extends BaseAnalyticsService {
  /**
   * Business Logic Principles:
   * - Repository layer: Pure data access, no business logic
   * - Service layer: All business logic, statistical analysis, and trend calculations
   * - Step completion rules: Applied consistently across all metrics
   * - Engagement time: Calculated from session timestamps with outlier filtering
   */
  
  async getExecutiveMetrics(
    organizationId: string,
    queryDto: ConversionFunnelExecutiveMetricsQueryDto
  ): Promise<ConversionFunnelExecutiveMetricsResultDto>
  
  async getStepConversionRates(
    organizationId: string,
    queryDto: StepConversionRatesQueryDto
  ): Promise<StepConversionRatesResultDto>
  
  async getStepDropoffTable(
    organizationId: string,
    queryDto: StepDropoffTableQueryDto
  ): Promise<StepDropoffTableResultDto>
}

2. ConversionFunnelRepository​

Extends BaseAnalyticsRepository and integrates with CommonAnalyticsRepository for consistent session counting:

@Injectable()
export class ConversionFunnelRepository extends BaseAnalyticsRepository {
  /**
   * Data Access Principles:
   * - No business logic - pure data retrieval
   * - Uses centralized unique tab session pipeline for consistency
   * - Ensures session counts match executive metrics
   * - Prioritizes sessions with booking_id within each tab_session_id group
   * - Returns raw data for service layer processing
   * - Handles ObjectId/string conversions at MongoDB level
   */
  
  constructor(
    @InjectConnection() connection: Connection,
    protected override readonly marketingSourcePipelineService: MarketingSourcePipelineService,
    private readonly commonAnalyticsRepository: CommonAnalyticsRepository
  )
  
  async getSessionAndStepRawData(
    organizationId: string,
    dateRange: DateRange,
    filters?: AnalyticsFilterDto
  ): Promise<{ bookingSessions: BookingSessionRawData[]; steps: StepDefinition[]; }>
  
  async getEngagementTimeRawData(
    organizationId: string,
    dateRange: DateRange,
    filters?: AnalyticsFilterDto
  ): Promise<number[]>
}

Session Counting Consistency​

Centralized Unique Tab Session Logic​

To ensure consistent session counts between executive metrics and conversion funnel data, both systems use a centralized unique session pipeline:

/**
 * CommonAnalyticsRepository.createUniqueTabSessionsPipeline()
 * 
 * Purpose: Ensures consistent session counting across all analytics modules
 * 
 * Logic:
 * 1. Groups sessions by tab_session_id to ensure uniqueness
 * 2. Within each group, prioritizes sessions with booking_id
 * 3. Falls back to most recent session if no booking_id exists
 * 4. Returns one representative session per unique tab session
 */

// Pipeline stages:
[
  ...standardFilterPipeline,
  {
    $sort: {
      tab_session_id: 1,
      booking_id: -1,        // Sessions with booking_id first
      createdAt: -1          // Then by most recent
    }
  },
  {
    $group: {
      _id: '$tab_session_id',
      session: { $first: '$$ROOT' }  // Take prioritized session
    }
  },
  {
    $replaceRoot: {
      newRoot: '$session'
    }
  }
]

Integration Points:

  • Executive Metrics: Uses CommonAnalyticsService.calculateTotalVisitsMetric()
  • Conversion Funnel: Uses CommonAnalyticsRepository.createUniqueTabSessionsPipeline() in ConversionFunnelRepository
  • Result: Identical session counts across both analytics views

Executive Metrics Analysis​

1. Total Visits​

Source: Reuses CommonAnalyticsService.calculateTotalVisitsMetric() Calculation: Count of unique booking sessions within date range and filters using centralized unique session logic Trend Analysis: Period-over-period comparison with data threshold validation

2. Booking Rate​

Source: Reuses CommonAnalyticsService.calculateBookingRateMetric() Calculation: (Completed Bookings / Total Visits) × 100 Business Logic:

  • Leverages already-computed total visits and completed bookings data
  • Avoids redundant database calls through data reuse
  • Handles zero-baseline scenarios with appropriate messaging

3. Average Engagement Time​

Purpose: Measures how long users spend in the booking flow before completing or abandoning

Calculation Method:

// Repository: Calculate engagement time from session timestamps
engagement_time_ms = {
  $subtract: [
    {
      $ifNull: [
        '$timestamps.booking_session_completed',
        '$timestamps.booking_session_abandoned'
      ]
    },
    '$createdAt'
  ]
}

// Service: Apply outlier filtering and convert to seconds
const filteredTimes = this.applyOutlierFiltering(rawEngagementTimes);
const avgEngagementTimeSeconds = filteredTimes.reduce((sum, time) => sum + time, 0) / filteredTimes.length / 1000;

Key Assumptions:

  • Engagement Start: Session createdAt timestamp
  • Engagement End: Either booking_session_completed OR booking_session_abandoned (one will exist, one won't)
  • Data Quality: Only sessions with valid end timestamps included
  • Outlier Handling: Percentile-based filtering for statistical reliability

Outlier Filtering Strategy:

  • Large datasets (n > 100): Remove top/bottom 2.5% (2.5th-97.5th percentile)
  • Small datasets (n ≤ 100): IQR-based filtering (Q1 - 1.5×IQR to Q3 + 1.5×IQR)
  • Rationale: Eliminates sessions left open for hours/days that skew average calculations

4. Step with Biggest Dropoff​

Calculation: Analyzes all steps in booking flow to identify highest dropoff rate Business Logic:

  • Processes step conversion rates for all steps
  • Identifies step with maximum (sessions_reached - sessions_completed) / sessions_reached × 100
  • Includes trend analysis comparing current vs. previous period performance

Step Conversion Rates Analysis​

Core Business Logic​

Step conversion analysis follows specific business rules for determining completion:

/**
 * Step Completion Determination Rules:
 * 1. Sessions Reached: step_id appears in session_step_state.step_path
 * 2. Sessions Completed: 
 *    - If step is LAST in all_steps_in_flow → booking_id must exist
 *    - If step is NOT last → must have subsequent steps in step_path
 * 3. Dropoff Rate: (reached - completed) / reached × 100
 */

private isStepCompleted(
  session: BookingSessionRawData,
  stepId: string,
  stepPositionMap: Map<string, number>
): boolean {
  const stepPosition = stepPositionMap.get(stepId);
  const allStepsInFlow = session.session_step_state.all_steps_in_flow;
  const stepPath = session.session_step_state.step_path;

  // If step is the last step in flow → check booking_id exists
  if (stepPosition === allStepsInFlow.length - 1) {
    return Boolean(session.booking_id);
  }

  // If step is not last → check if any subsequent steps are in step_path
  const subsequentSteps = allStepsInFlow.slice(stepPosition + 1);
  return subsequentSteps.some(subsequentStepId => 
    stepPath.includes(subsequentStepId)
  );
}

Step Position Mapping​

Purpose: Determines step order within booking flow for completion analysis Calculation: Uses all_steps_in_flow array to find step positions Methodology: Median position across sessions for robustness against outliers

private buildStepPositionMap(
  bookingSessions: BookingSessionRawData[],
  steps: StepDefinition[]
): Map<string, number> {
  // For each step, collect all positions across sessions
  // Use median position for statistical robustness
  // Handles cases where steps appear at different positions
}

Data Processing Flow​

  1. Raw Data Retrieval: Get booking sessions and step definitions
  2. Step Position Mapping: Build position map from all_steps_in_flow
  3. Session Analysis: For each step, calculate reached and completed sessions
  4. Metric Calculation: Compute dropoff rates and average positions
  5. Sorting: Order steps by average position for proper funnel visualization

Key Assumptions​

  • User Journey Tracking: step_path contains actual user progression
  • Flow Definition: all_steps_in_flow defines complete booking flow structure
  • Completion Criteria: Different rules for last step (booking required) vs. intermediate steps (subsequent progression)
  • ObjectId Handling: Consistent string conversion for step ID matching

Step Dropoff Analysis​

Dimensional Breakdown Methodology​

The step dropoff analysis extends basic conversion rates with dimensional analysis to identify patterns:

/**
 * Dimensional Analysis Components:
 * 1. Worst Source: Traffic source with highest dropoff rate for this step
 * 2. Worst Device: Device type with highest dropoff rate for this step  
 * 3. Worst Time: Time period with highest dropoff rate for this step
 * 4. Period Trend: Current vs. previous period dropoff rate comparison
 */

private calculateWorstPerformersForStep(
  bookingSessions: BookingSessionRawData[],
  stepId: string
): {
  source: { name: string; rate: number };
  device: { name: string; rate: number };
  time: { from_datetime: string; to_datetime: string; rate: number };
}

Dimensional Grouping Strategy​

Traffic Source Dimension:

  • Field: attributed_source from session data
  • Grouping: Sessions grouped by source value
  • Dropoff Calculation: Within each source group, calculate completion rate for the step

Device Type Dimension:

  • Field: initialization_context.user_agent.device_type
  • Grouping: Sessions grouped by device type (mobile, desktop, tablet)
  • Dropoff Calculation: Device-specific completion rates

Time Dimension:

  • Field: time_block from session data
  • Grouping: Sessions grouped by hour blocks (e.g., "09:00-09:59")
  • Output Format: Converted to datetime ranges for UI consumption

Trend Analysis Implementation​

Period-over-Period Comparison:

// Calculate trend using centralized method
const metricComparison = this.calculateMetricWithComparison(
  currentDropoffRate,
  previousDropoffRate,
  currentSessionsReached,
  previousSessionsReached
);

// Result includes:
// - trend: UP/DOWN/STABLE based on statistical significance
// - delta_percentage: Magnitude of change
// - Automatic "Not enough data" handling for insufficient sample sizes

Data Threshold Validation:

  • Minimum Sample Size: 10 sessions per period for reliable trend calculation
  • Graceful Degradation: Shows "Not enough data" when thresholds not met
  • Statistical Reliability: Prevents misleading trends from small sample sizes

Business Context Integration​

The analysis provides actionable context for each step:

interface StepDropoffTableRow {
  step_id: string;
  step_name: string;
  step_type: BookingFlowStepType;
  dropoff_count: number;        // Absolute number of dropoffs
  dropoff_rate: number;         // Percentage dropoff rate
  worst_source: { name: string; rate: number; };
  worst_device: { name: string; rate: number; };
  worst_time: { from_datetime: string; to_datetime: string; rate: number; };
  vs_last_period_trend: {
    trend: Trend;
    rate: number;
  };
}

Data Quality and Validation​

ObjectId Normalization​

Challenge: MongoDB ObjectIds vs. string comparisons in business logic Solution: Consistent conversion to strings before analysis

convertObjectIdArrayToStringArray(objectIds: (ObjectId | string)[]): string[] {
  return objectIds.map((objectId) => objectId.toString());
}

// Applied to:
// - session_step_state.step_path (user journey tracking)
// - session_step_state.all_steps_in_flow (flow definition)
// - Step ID comparisons throughout analysis

Engagement Time Data Quality​

Filtering Strategy:

  • Valid Sessions Only: Must have either completion or abandonment timestamp
  • Non-negative Values: Engagement time ≥ 0 (filters invalid timestamp calculations)
  • Outlier Removal: Statistical filtering to remove extreme values that skew averages

MongoDB Aggregation:

// Calculate engagement time in repository
{
  $addFields: {
    engagement_time_ms: {
      $subtract: [
        {
          $ifNull: [
            '$timestamps.booking_session_completed',
            '$timestamps.booking_session_abandoned'
          ]
        },
        '$createdAt'
      ]
    }
  }
}

// Filter valid calculations
{
  $match: {
    engagement_time_ms: { $exists: true, $gte: 0 }
  }
}

Step Data Consistency​

Step Definition Matching:

  • Extract unique step IDs from all sessions' all_steps_in_flow
  • Fetch step definitions only for steps actually used in sessions
  • Handles cases where flow definitions change over time

Position Calculation Robustness:

  • Uses median position across sessions for each step
  • Handles steps that may appear at different positions in different flows
  • Provides stable position mapping for completion analysis

Performance Optimizations​

Parallel Execution Strategy​

Executive Metrics:

// Parallel calculation of independent metrics
const [totalVisits, completedBookings, avgEngagementTime] = await Promise.all([
  this.commonAnalyticsService.calculateTotalVisitsMetric(...),
  this.commonAnalyticsService.calculateCompletedBookingsMetric(...),
  this.calculateAvgEngagementTimeMetric(...)
]);

// Compose dependent metrics from already-computed data
const bookingRate = this.commonAnalyticsService.calculateBookingRateMetric({
  totalVisits,
  completedBookings
});

Step Analysis:

// Parallel current and previous period analysis
const [currentStepMetrics, previousStepMetrics] = await Promise.all([
  this.processStepConversionRates(organizationId, dateRange, filters),
  this.processStepConversionRates(organizationId, comparisonDateRange, filters)
]);

Data Reuse Strategy​

Repository Efficiency:

  • Single getSessionAndStepRawData() call provides data for multiple analyses
  • Step conversion rates and dropoff analysis share same base dataset
  • Avoids redundant database queries through intelligent data sharing

Service Layer Optimization:

  • Executive metrics reuse CommonAnalyticsService calculations
  • Booking rate leverages already-computed visits and bookings data
  • Step position mapping calculated once and reused across analyses

Caching Implementation​

Analytics Query Caching:

@CachedAnalyticsQuery({ ttlSeconds: 600 }) // 10 minutes
async getExecutiveMetrics(organizationId: string, queryDto: ConversionFunnelExecutiveMetricsQueryDto)

@CachedAnalyticsQuery({ ttlSeconds: 1200 }) // 20 minutes  
async getStepConversionRates(organizationId: string, queryDto: StepConversionRatesQueryDto)

@CachedAnalyticsQuery({ ttlSeconds: 1200 }) // 20 minutes
async getStepDropoffTable(organizationId: string, queryDto: StepDropoffTableQueryDto)

Cache Strategy Rationale:

  • Executive metrics: Higher refresh rate for dashboard overview
  • Detailed analyses: Longer cache for complex calculations
  • Redis integration: Automatic cache key generation and graceful fallback

Business Benefits​

1. Comprehensive Funnel Intelligence​

Traditional Approach: Simple conversion rate percentages

  • Limited insight into where and why users drop off
  • No dimensional context for optimization decisions
  • Basic before/after comparisons without statistical validation

Our Approach: Multi-dimensional funnel analysis

  • Precise identification of problematic steps and user segments
  • Statistical trend validation prevents false positive alerts
  • Actionable context for targeted optimization efforts

2. Engagement Time Insights​

Traditional Approach: Time on page or session duration

  • Often inflated by users leaving browser tabs open
  • No filtering for unrealistic engagement times
  • Difficult to benchmark against meaningful standards

Our Approach: Statistically filtered engagement analysis

  • Outlier removal provides realistic engagement time averages
  • Period-over-period trending reveals engagement pattern changes
  • Booking flow specific timing for optimization insights

3. Dimensional Troubleshooting​

Traditional Approach: "Step X has a 40% dropoff rate"

  • No context about which users or conditions contribute to problems
  • Optimization efforts lack direction and focus
  • Difficult to prioritize improvement initiatives

Our Approach: "Step X has 40% dropoff, worst on mobile devices during evening hours from social media traffic"

  • Specific user segments and conditions identified for targeted fixes
  • Clear prioritization framework based on data volume and impact
  • Contextual insights enable precise optimization strategies

Implementation Examples​

Complete Executive Metrics Flow​

async getExecutiveMetrics(
  organizationId: string,
  queryDto: ConversionFunnelExecutiveMetricsQueryDto
): Promise<ConversionFunnelExecutiveMetricsResultDto> {
  
  // 1. Validate inputs and derive date ranges
  this.validateOrganizationId(organizationId, 'conversion funnel executive metrics');
  const dateRange = queryDto.filters.date_range;
  const comparisonDateRange = this.derivePrecedingDateRange(dateRange);

  // 2. Parallel calculation of independent metrics
  const [totalVisits, completedBookings, avgEngagementTime] = await Promise.all([
    this.commonAnalyticsService.calculateTotalVisitsMetric(organizationId, dateRange, comparisonDateRange, queryDto.filters),
    this.commonAnalyticsService.calculateCompletedBookingsMetric(organizationId, dateRange, comparisonDateRange, queryDto.filters),
    this.calculateAvgEngagementTimeMetric(organizationId, dateRange, comparisonDateRange, queryDto.filters)
  ]);

  // 3. Compose dependent metrics from already-computed data
  const bookingRate = this.commonAnalyticsService.calculateBookingRateMetric({ totalVisits, completedBookings });
  
  // 4. Calculate step-specific metrics
  const stepWithBiggestDropoff = await this.calculateStepWithBiggestDropoff(organizationId, dateRange, comparisonDateRange, queryDto.filters);

  return {
    total_visits: totalVisits,
    booking_rate: bookingRate,
    avg_engagement_time_in_seconds: avgEngagementTime,
    step_with_biggest_dropoff: stepWithBiggestDropoff
  };
}

Step Conversion Analysis Flow​

private async processStepConversionRates(
  organizationId: string,
  dateRange: DateRange,
  filters?: AnalyticsFilterDto
): Promise<StepConversionRatesData[]> {
  
  // 1. Get raw data from repository (pure data access)
  const { bookingSessions, steps } = await this.conversionFunnelRepository.getSessionAndStepRawData(organizationId, dateRange, filters);

  // 2. Build step position map for completion logic
  const stepPositionMap = this.buildStepPositionMap(bookingSessions, steps);

  // 3. Process each step with business logic
  const stepMetrics: StepConversionRatesData[] = steps.map((step) => {
    const stepId = step._id;

    // Calculate sessions reached: sessions where step is in step_path
    const sessionsReached = bookingSessions.filter((session) =>
      this.convertObjectIdArrayToStringArray(session.session_step_state.step_path).includes(stepId)
    );

    // Calculate sessions completed using business logic
    const sessionsCompleted = sessionsReached.filter((session) =>
      this.isStepCompleted(session, stepId, stepPositionMap)
    );

    // Calculate dropoff rate
    const dropoffRate = sessionsReached.length > 0
      ? ((sessionsReached.length - sessionsCompleted.length) / sessionsReached.length) * 100
      : 0;

    return {
      step_id: stepId,
      step_name: step.step_name,
      step_type: step.step_type,
      sessions_reached: sessionsReached.length,
      sessions_completed: sessionsCompleted.length,
      dropoff_rate: dropoffRate,
      avg_step_position: stepPositionMap.get(stepId) || 0,
    };
  });

  // 4. Sort by step position for proper funnel order
  return stepMetrics.sort((a, b) => a.avg_step_position - b.avg_step_position);
}

Future Enhancements​

1. Advanced Completion Analytics​

Planned Features:

  • Conversion Path Analysis: Track most common successful user journeys
  • Abandonment Pattern Detection: Identify recurring dropout sequences
  • Session Recovery Tracking: Analyze users who return to complete bookings

2. Enhanced Dimensional Analysis​

Planned Features:

  • Geographic Breakdown: Regional performance analysis for location-based services
  • Temporal Patterns: Day-of-week and seasonal trend analysis
  • Marketing Attribution: Channel-specific funnel performance tracking

3. Predictive Analytics Integration​

Planned Features:

  • Dropoff Risk Scoring: Real-time user likelihood to abandon analysis
  • Completion Time Prediction: Expected booking flow duration estimates
  • Optimization Recommendations: AI-driven improvement suggestions

4. A/B Testing Integration​

Planned Features:

  • Flow Variant Analysis: Compare different booking flow configurations
  • Step Order Optimization: Test different step sequences for conversion impact
  • Statistical Significance Testing: Automated experiment result validation

Conclusion​

The Wrkbelt Conversion Funnel Metrics System provides service businesses with unprecedented visibility into their booking flow performance. By combining precise step-by-step analysis with dimensional breakdowns and statistical trend validation, the system transforms raw user behavior data into actionable optimization intelligence.

The architecture ensures scalability and maintainability while the business logic accurately reflects real user journey patterns. Through careful attention to data quality, statistical rigor, and performance optimization, the system delivers reliable insights that businesses can trust for strategic decision-making.

As booking flows become more sophisticated and user expectations evolve, this system provides the analytical foundation necessary to continuously optimize conversion performance and enhance customer experience.

References​

Statistical Methods​

  • Central Limit Theorem: Sample size validation for trend reliability
  • Percentile-based Outlier Detection: Engagement time filtering methodology
  • Period-over-Period Analysis: Trend calculation with data adequacy validation

Implementation Resources​

Business Intelligence Research​

  • Funnel Analysis Best Practices: Multi-dimensional conversion optimization
  • User Journey Analytics: Session-based behavior tracking methodologies
  • Service Industry Benchmarks: Booking flow optimization standards