fluxly Search skills… ⌘K 中文 Sign in Publish

ab-testing

Engineering Community

Repo stars 0
Author repo skills-registry

Interpretation is structured for decision-making; original keeps the upstream SKILL.md unchanged.

Decide Fit First

Core job: >- Use when this capability is needed. // middleware.ts - Server-side assignment (no flicker) import { NextResponse, type NextRe…
Best fit: Use it when the task has reusable inputs, steps, and validation criteria rather than a one-off answer.
Avoid forcing it: If the source lacks commands, platform support, or external-service evidence, keep those fields unknown instead of guessing.

Design Intent

Structure: The skill is organized around “Quick Start”, “Architecture”, “Workflow”, “Statistical Methods (Choose One)”, showing how the author expects the agent to judge fit, collect context, and produce verifiable output.
Trigger evidence: Prioritize the author’s wording around when to use it, what context to collect, and what output shape to produce.
Evidence boundary: Author text states facts, repository files prove commands and paths, and Fluxly only adds fit, limits, and usage judgment.

How To Use It

Inputs: Provide target material, scope, expected result, forbidden changes, and validation method.
Invocation: Name ab-testing directly; if the source includes slash commands, start with the command and then add task context.
Validation: Start small and check whether the result follows “Quick Start / Architecture / Workflow” before expanding.

Boundaries And Review

Dependencies: It usually needs no extra API key, so start with a small validation task.
Permissions: Declared permissions include read / write; ask the agent to state file, command, and rollback boundaries before acting.
Quality bar: A useful result names the deliverable, evidence, and next action. Generic prose means the task needs tighter context.

A/B Testing Platform for SaaS

Stack: Next.js 16 + Edge Middleware + GA4/GTM + Supabase + Rust-WASM (for stats) Why DIY?: Google Optimize sunset Sept 2023. GA4 has no native A/B testing.

Quick Start

// middleware.ts - Server-side assignment (no flicker)
import { NextResponse, type NextRequest } from 'next/server'

export function middleware(request: NextRequest) {
  const response = NextResponse.next()

  if (!request.cookies.get('exp_hero')) {
    const variant = Math.random() < 0.5 ? 'A' : 'B'
    response.cookies.set('exp_hero', variant, { maxAge: 60*60*24*30, path: '/' })
  }

  return response
}

// Track with GA4
window.dataLayer?.push({
  event: 'experiment_view',
  experiment_name: 'hero_test',
  experiment_variant: variant
})

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                         A/B TESTING FLOW                            │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│   1. ASSIGN (Edge Middleware)                                       │
│   ══════════════════════════                                        │
│   Request → Check cookie → Random assign → Set cookie → Response    │
│   ✓ No flicker (server-side)  ✓ Consistent (cookie-based)          │
│                                                                     │
│   2. RENDER                                                         │
│   ═════════                                                         │
│   Server/Client Component → Read cookie → Show variant              │
│                                                                     │
│   3. TRACK (GTM + GA4)                                              │
│   ════════════════════                                              │
│   dataLayer.push → GTM triggers → GA4 events with variant param     │
│                                                                     │
│   4. ANALYZE                                                        │
│   ═════════                                                         │
│   GA4 Explorations OR Supabase + Rust-WASM Bayesian analysis        │
│                                                                     │
│   5. PERSONALIZE (Advanced)                                         │
│   ═════════════════════════                                         │
│   Contextual bandit → Best variant per user segment                 │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

Workflow

1. Define experiment config (name, variants, weight, segment rules)
1. Implement middleware assignment (see VARIANT-ASSIGNMENT.md)
1. Wire up GTM tracking (experiment_view, conversions with variant param)
1. Register experiment_variant as Custom Dimension in GA4
1. Collect data (minimum 2 weeks for weekly patterns)
1. Analyze significance (see STATISTICAL-ANALYSIS.md)
1. Roll out winner or iterate

Statistical Methods (Choose One)

Method	Best For	Decision Output
Frequentist	Fixed sample, strict control	p-value < 0.05 → significant
Bayesian	Continuous monitoring, intuitive	P(B > A) = 96% → B likely better
Multi-Armed Bandit	Optimize during test	Auto-shift traffic to winner
Contextual Bandit	Personalization	Best variant per user segment

Quick Bayesian (Beta-Binomial):

# A: 50/1000 conversions, B: 72/1000
import scipy.stats as stats
a_samples = stats.beta(51, 951).rvs(100000)  # Beta(1+50, 1+950)
b_samples = stats.beta(73, 929).rvs(100000)  # Beta(1+72, 1+928)
p_b_wins = (b_samples > a_samples).mean()     # → ~0.96 (96%)

Full analysis guide: STATISTICAL-ANALYSIS.md

Key Patterns

Multiple Concurrent Experiments

// middleware.ts
const EXPERIMENTS = {
  hero_cta: { weight: 0.5 },
  pricing_layout: { weight: 0.5 },
  signup_flow: { weight: 0.2 },  // 20% on new variant
}

for (const [name, config] of Object.entries(EXPERIMENTS)) {
  if (!request.cookies.get(`exp_${name}`)) {
    const variant = Math.random() < config.weight ? 'B' : 'A'
    response.cookies.set(`exp_${name}`, variant, { maxAge: 2592000, path: '/' })
  }
}

Phased Rollout

// Config stored in Supabase or Vercel Edge Config
const rolloutPhases = {
  early_access: 0.1,   // 10% new
  public_beta: 0.5,    // 50% new
  general: 1.0         // 100% new (winner)
}

Tracking Conversions

// Include variant in ALL relevant events
window.dataLayer?.push({
  event: 'sign_up',
  method: 'google',
  experiment_name: 'hero_cta',
  experiment_variant: getCookie('exp_hero_cta'),
  eventId: crypto.randomUUID()  // De-duplication
})

When to Use Rust-WASM

Use Case	Why Rust
Monte Carlo simulation (100k+ draws)	10-100x faster than JS
Bayesian posterior computation	Numerical precision
Contextual bandit inference	Real-time ML at edge
Cross-platform consistency	Same logic in browser + server

WASM is NOT needed for: Simple random assignment, cookie handling, event tracking

See: RUST-WASM.md

Anti-Patterns

Don't	Why
Client-side variant assignment	Causes flicker, inconsistent
End test early ("B winning after 2 days!")	Random noise, not signal
Multiple changes in one variant	Can't isolate what worked
Overlapping tests on same element	Interaction effects confound
Skip sample size calculation	Under-powered = false negatives
Ignore segments	Winner overall may lose for key segment

Validation Checklist

No flicker on page load (verify in slow 3G)
Cookie persists across sessions (check 30-day expiry)
GTM Preview shows correct variant in dataLayer
GA4 DebugView receives events with variant param
Custom dimension experiment_variant registered in GA4
~50/50 split verified (check GA4 Realtime)
Conversion events include variant attribution

Reference Index

By Task

I need to...	Read
Implement variant assignment	VARIANT-ASSIGNMENT.md
Choose a statistical method	STATISTICAL-ANALYSIS.md
Set up GTM/GA4 tracking	GA4-GTM-TRACKING.md
Build admin dashboard	ADMIN-DASHBOARD.md
Add personalization/bandits	PERSONALIZATION.md
Optimize with Rust-WASM	RUST-WASM.md
Quick lookup (tools, formulas)	QUICK-REFERENCE.md

By Topic

Topic	Reference
Server-side assignment, multiple experiments, weighted splits	VARIANT-ASSIGNMENT.md
Frequentist vs Bayesian vs Bandits, sample size, pitfalls	STATISTICAL-ANALYSIS.md
GTM variables, GA4 events, BigQuery queries, debugging	GA4-GTM-TRACKING.md
Database schema, API routes, UI components, real-time updates	ADMIN-DASHBOARD.md
Segments, rules-based, Thompson sampling, contextual bandits	PERSONALIZATION.md
Rust setup, beta sampling, bandit implementation, Next.js integration	RUST-WASM.md
Tool recommendations, decision guide, formulas, cheat sheet	QUICK-REFERENCE.md

Source: danzam98/claude-skills-toolkit — distributed by TomeVault.

Fluxly profile Author and license come from source; runtime, permissions, and network are Fluxly detections or estimates

Fluxly category: Engineering
Author-declared agents: No explicit declaration found; this is not inferred or tested compatibility
Static check: 88 / 100 · heuristic scan, not runtime safety proof
Author / version / license: @tomevault-io · no license declared
Fluxly token estimate: Lean
Fluxly setup estimate: Plug-and-play
External API key: No requirement detected
Detected OS requirements: Unspecified
Runtime requirements: Python
Detected file/system behavior: Read-only
Write / modify
Detected network behavior: Local-only
Install commands: None (reference only)

Profile is derived at build time from SKILL.md and install vectors. Subject to drift from author intent.

Heads up: 未限定 allowed-tools，默认拥有全部工具权限。

Output preview ab-testing.preview

The current SKILL.md does not define a fixed output example.

SKILL.md SKILL.md

# A/B Testing Platform for SaaS

> **Stack**: Next.js 16 + Edge Middleware + GA4/GTM + Supabase + Rust-WASM (for stats)
> **Why DIY?**: Google Optimize sunset Sept 2023. GA4 has no native A/B testing.

## Quick Start

```tsx
// middleware.ts - Server-side assignment (no flicker)
import { NextResponse, type NextRequest } from 'next/server'

export function middleware(request: NextRequest) {
  const response = NextResponse.next()

  if (!request.cookies.get('exp_hero')) {
    const variant = Math.random() < 0.5 ? 'A' : 'B'
    response.cookies.set('exp_hero', variant, { maxAge: 60*60*24*30, path: '/' })
  }

  return response
}
```

```tsx
// Track with GA4
window.dataLayer?.push({
  event: 'experiment_view',
  experiment_name: 'hero_test',
  experiment_variant: variant
})
```

---

## Architecture

```
┌─────────────────────────────────────────────────────────────────────┐
│                         A/B TESTING FLOW                            │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│   1. ASSIGN (Edge Middleware)                                       │
│   ══════════════════════════                                        │
│   Request → Check cookie → Random assign → Set cookie → Response    │
│   ✓ No flicker (server-side)  ✓ Consistent (cookie-based)          │
│                                                                     │
│   2. RENDER                                                         │
│   ═════════                                                         │
│   Server/Client Component → Read cookie → Show variant              │
│                                                                     │

…

Evidence boundary and execution chain

Author text anchors workflow facts; Fluxly only indexes current sections, terms, files, and commands.

sections -> Quick Start → Architecture → Workflow → Statistical Methods (Choose One) → Key Patterns → Multiple Concurrent Experiments
terms    -> Stack · Why DIY? · Frequentist · Bayesian · Multi-Armed Bandit · Contextual Bandit · Quick Bayesian (Beta-Binomial) · Full analysis guide
files/cmd -> experimentvariant · exp${name} · A/B · Next.js · GA4/GTM · middleware.ts · next/server · Server/Client
body sha256 -> 7512d8f6e928