fitaiProto/docs/WEBHOOK_TESTING_GUIDE.md

# Webhook Testing Guide

Complete guide for testing the Clerk webhook integration that syncs users to your database.

## Table of Contents

1. [Prerequisites](#prerequisites)
2. [Local Development Testing](#local-development-testing)
3. [Production Testing](#production-testing)
4. [Test Scenarios](#test-scenarios)
5. [Verification Steps](#verification-steps)
6. [Common Issues](#common-issues)

---

## Prerequisites

Before testing webhooks, ensure you have:

- ✅ Clerk account and application set up
- ✅ Admin app running locally or deployed
- ✅ Database schema updated (password field optional)
- ✅ `CLERK_WEBHOOK_SECRET` environment variable set
- ✅ `svix` package installed in admin app

### Quick Setup Check

```bash
# 1. Check environment variables
cat apps/admin/.env.local | grep CLERK_WEBHOOK_SECRET

# 2. Verify svix is installed
cd apps/admin
npm list svix

# 3. Check database schema
cd ../../packages/database
npm run db:push
```

---

## Local Development Testing

### Option 1: Using Clerk CLI (Recommended)

The Clerk CLI forwards webhook events from Clerk to your local server.

#### Step 1: Install Clerk CLI

```bash
npm install -g @clerk/clerk-cli
```

#### Step 2: Login to Clerk

```bash
clerk login
```

Follow the prompts to authenticate with your Clerk account.

#### Step 3: Start Your Dev Server

```bash
cd apps/admin
npm run dev
```

Your server should be running at `http://localhost:3000`

#### Step 4: Start Webhook Forwarding

In a **new terminal window**:

```bash
clerk listen --forward-url http://localhost:3000/api/webhooks
```

You'll see output like:
```
✓ Webhook forwarding is now active!
✓ Webhook secret: whsec_xxxxxxxxxxxxxx

Forwarding webhooks to: http://localhost:3000/api/webhooks
```

#### Step 5: Copy the Webhook Secret

Copy the webhook secret from the output and add it to your `.env.local`:

```env
CLERK_WEBHOOK_SECRET=whsec_xxxxxxxxxxxxxx
```

Restart your dev server after adding the secret.

#### Step 6: Test with a Real Sign-Up

1. Go to your app (mobile or admin)
2. Sign up a new user
3. Watch both terminals:
   - Clerk CLI terminal shows the webhook being forwarded
   - Dev server terminal shows the processing logs

Expected output:
```
Received webhook with ID user_abc123 and type user.created
✅ User user_abc123 created in database
```

### Option 2: Using Webhook Testing Tools

#### Using Svix Play (Web UI)

1. Go to [Svix Play](https://www.svix.com/play/)
2. Enter your webhook URL: `http://localhost:3000/api/webhooks`
3. Select "Clerk" as the provider
4. Choose event type: `user.created`
5. Modify the payload if needed
6. Click "Send"

**Note:** You'll need to expose your local server (use ngrok) for this to work.

#### Using ngrok for Local Testing

If you need a public URL:

```bash
# Install ngrok
npm install -g ngrok

# Start ngrok
ngrok http 3000
```

Use the ngrok URL (e.g., `https://abc123.ngrok.io/api/webhooks`) in Clerk Dashboard or Svix Play.

---

## Production Testing

### Step 1: Configure Webhook Endpoint in Clerk Dashboard

1. Go to [Clerk Dashboard](https://dashboard.clerk.com)
2. Select your application
3. Navigate to **Webhooks** in sidebar
4. Click **Add Endpoint**

#### Configure Endpoint:

- **Endpoint URL:** `https://yourdomain.com/api/webhooks`
- **Events to subscribe to:**
  - ✅ `user.created`
  - ✅ `user.updated`
  - ✅ `user.deleted`

#### Copy Signing Secret:

After creating the endpoint, copy the **Signing Secret** and add it to your production environment variables:

```env
CLERK_WEBHOOK_SECRET=whsec_prod_xxxxxxxxxxxxxx
```

### Step 2: Send Test Event

In Clerk Dashboard webhook settings:

1. Click on your endpoint
2. Click **Send Test Event**
3. Select `user.created`
4. Click **Send**

### Step 3: Verify in Dashboard

Check the webhook attempt logs:

- **Status:** Should be `200 OK`
- **Response:** `"Webhook processed successfully"`
- **Response Time:** < 2 seconds typically

### Step 4: Verify in Database

Query your database to verify the test user was created:

```sql
SELECT * FROM users ORDER BY created_at DESC LIMIT 1;
```

---

## Test Scenarios

### Scenario 1: User Creation

**Objective:** Verify new users are synced to database

**Steps:**

1. Create a new user via sign-up flow
2. Check database for new user record
3. Verify all fields are populated correctly

**Expected Result:**

```sql
-- User should exist in database
SELECT * FROM users WHERE id = 'user_abc123';

-- Result:
-- id: user_abc123
-- email: test@example.com
-- firstName: John
-- lastName: Doe
-- role: client (default)
-- password: NULL or empty
-- createdAt: [timestamp]
-- updatedAt: [timestamp]
```

**Success Criteria:**
- ✅ User record exists
- ✅ Email matches Clerk
- ✅ Name fields populated
- ✅ Default role is 'client'
- ✅ Timestamps are set

### Scenario 2: User Update

**Objective:** Verify user updates sync to database

**Steps:**

1. Update user profile in Clerk (name, email, or role)
2. Check database for updated values
3. Verify `updatedAt` timestamp changed

**Update Name via Clerk Dashboard:**
1. Go to Users → Select user
2. Update First Name to "Jane"
3. Click Save

**Expected Result:**

```sql
SELECT * FROM users WHERE id = 'user_abc123';

-- firstName should now be: Jane
-- updatedAt should be newer than createdAt
```

**Success Criteria:**
- ✅ User fields updated
- ✅ `updatedAt` timestamp changed
- ✅ `createdAt` timestamp unchanged

### Scenario 3: Role Assignment

**Objective:** Verify role metadata syncs correctly

**Steps:**

1. Set user role via Clerk Dashboard:
   - Users → Select user
   - Public Metadata: `{"role": "trainer"}`
   - Save
2. Check database for role update

**Expected Result:**

```sql
SELECT id, email, role FROM users WHERE id = 'user_abc123';

-- role should be: trainer
```

**Success Criteria:**
- ✅ Role updated to 'trainer'
- ✅ Role persisted in database
- ✅ Webhook processed successfully

### Scenario 4: User Deletion

**Objective:** Verify user deletion cascades correctly

**Steps:**

1. Note user ID: `user_abc123`
2. Delete user from Clerk Dashboard
3. Check database - user should be deleted

**Expected Result:**

```sql
-- User should not exist
SELECT * FROM users WHERE id = 'user_abc123';
-- Returns: 0 rows

-- Related records should also be deleted (cascade)
SELECT * FROM notifications WHERE user_id = 'user_abc123';
-- Returns: 0 rows
```

**Success Criteria:**
- ✅ User record deleted
- ✅ Related records deleted (notifications, etc.)
- ✅ No orphaned data

### Scenario 5: Bulk User Creation

**Objective:** Test webhook handling under load

**Steps:**

1. Import multiple users via Clerk Dashboard or API
2. Verify all users created in database
3. Check for any failed webhooks

**Expected Result:**

```sql
-- All users should exist
SELECT COUNT(*) FROM users WHERE created_at > NOW() - INTERVAL 5 MINUTE;
-- Should match number of users created
```

**Success Criteria:**
- ✅ All users synced
- ✅ No duplicate entries
- ✅ All webhook attempts successful

---

## Verification Steps

### 1. Check Webhook Logs (Clerk Dashboard)

**Location:** Clerk Dashboard → Webhooks → Your Endpoint

**What to Check:**
- **Status Code:** Should be `200`
- **Response Time:** Typically < 2 seconds
- **Response Body:** `"Webhook processed successfully"`
- **Attempts:** Should be 1 (no retries needed)

**Red Flags:**
- ❌ Status 400: Verification failed (wrong secret)
- ❌ Status 500: Server error (check logs)
- ❌ Multiple attempts: Server issues or slow response

### 2. Check Server Logs

**Look for these log messages:**

```bash
# Success logs
✅ User user_abc123 created in database
✅ User user_abc123 updated in database
✅ User user_abc123 deleted from database

# Info logs
Received webhook with ID user_abc123 and type user.created

# Error logs (should not appear)
❌ Error verifying webhook: [details]
❌ Error processing webhook: [details]
❌ No primary email found for user: [id]
```

### 3. Verify Database State

**Query Templates:**

```sql
-- Check recent users
SELECT * FROM users ORDER BY created_at DESC LIMIT 10;

-- Check user count by role
SELECT role, COUNT(*) as count FROM users GROUP BY role;

-- Check for users without required fields
SELECT * FROM users WHERE email IS NULL OR firstName = '' OR lastName = '';

-- Check for orphaned records (shouldn't exist after cascade delete)
SELECT n.* FROM notifications n 
LEFT JOIN users u ON n.user_id = u.id 
WHERE u.id IS NULL;
```

### 4. Test Role Assignment API

**Using curl:**

```bash
# Get your auth token from Clerk
TOKEN="your_clerk_session_token"

# Set user role
curl -X POST http://localhost:3000/api/admin/set-role \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "targetUserId": "user_abc123",
    "role": "trainer"
  }'

# Expected response:
# {
#   "success": true,
#   "message": "User role updated to trainer",
#   "user": { ... }
# }
```

---

## Common Issues

### Issue 1: Webhook Returns 400 - Verification Failed

**Symptoms:**
```
Error verifying webhook: [Svix error details]
Status: 400
```

**Causes:**
- Wrong webhook secret
- Missing Svix headers
- Replay attack (old timestamp)

**Solutions:**

1. **Verify webhook secret:**
   ```bash
   echo $CLERK_WEBHOOK_SECRET
   # Should start with whsec_
   ```

2. **Regenerate secret:**
   - Go to Clerk Dashboard → Webhooks
   - Click your endpoint → Rotate Secret
   - Update `.env.local` with new secret
   - Restart server

3. **Check headers:**
   ```typescript
   // Add debug logging to webhook handler
   console.log('Headers:', {
     svix_id: headerPayload.get('svix-id'),
     svix_timestamp: headerPayload.get('svix-timestamp'),
     svix_signature: headerPayload.get('svix-signature'),
   });
   ```

### Issue 2: User Not Created in Database

**Symptoms:**
- Webhook returns 200
- Server logs show success
- User not in database

**Solutions:**

1. **Check database connection:**
   ```bash
   cd packages/database
   npm run db:studio
   # Verify you can see the database
   ```

2. **Check for unique constraint violations:**
   ```sql
   -- Check if user with email already exists
   SELECT * FROM users WHERE email = 'test@example.com';
   ```

3. **Verify schema migration ran:**
   ```bash
   cd packages/database
   npm run db:push
   ```

4. **Check server logs for SQL errors:**
   ```
   Error processing webhook: UNIQUE constraint failed: users.email
   ```

### Issue 3: Role Not Updating

**Symptoms:**
- Webhook processed successfully
- Role in database doesn't change

**Solutions:**

1. **Verify metadata format in Clerk:**
   ```json
   {
     "role": "admin"
   }
   ```
   (Should be in **Public Metadata**, not Private or Unsafe)

2. **Check role value:**
   - Must be exactly: `admin`, `trainer`, or `client`
   - Case-sensitive
   - No extra spaces

3. **Force webhook retry:**
   - Update any field on the user
   - This triggers `user.updated` webhook

### Issue 4: Webhook Not Received

**Symptoms:**
- Event in Clerk
- No webhook attempt logged
- Server never hit

**Solutions:**

1. **Verify endpoint URL in Clerk Dashboard:**
   ```
   Production: https://yourdomain.com/api/webhooks
   Local (with ngrok): https://abc123.ngrok.io/api/webhooks
   ```

2. **Check subscribed events:**
   - Ensure `user.created`, `user.updated`, `user.deleted` are checked

3. **Verify server is accessible:**
   ```bash
   # Test endpoint manually
   curl -X POST https://yourdomain.com/api/webhooks \
     -H "Content-Type: application/json" \
     -d '{"test": true}'
   
   # Should return 400 (missing headers), not 404
   ```

4. **Check firewall/security rules:**
   - Ensure Clerk IPs not blocked
   - HTTPS must be valid (not self-signed cert)

### Issue 5: Slow Webhook Processing

**Symptoms:**
- Webhook takes > 5 seconds
- Clerk retries multiple times
- Timeouts

**Solutions:**

1. **Optimize database queries:**
   ```typescript
   // Add indexes
   // Run migrations separately
   // Use transactions for bulk operations
   ```

2. **Add timeout handling:**
   ```typescript
   // Process webhook asynchronously
   // Return 200 immediately
   // Handle sync in background
   ```

3. **Check database connection pool:**
   ```typescript
   // Ensure connection pool is sized correctly
   // Close connections properly
   ```

---

## Automated Testing

### Unit Test Example

```typescript
// apps/admin/__tests__/webhook.test.ts
import { POST } from '@/app/api/webhooks/route';

describe('Webhook Handler', () => {
  it('should create user on user.created event', async () => {
    const mockRequest = new Request('http://localhost/api/webhooks', {
      method: 'POST',
      headers: {
        'svix-id': 'msg_test',
        'svix-timestamp': Date.now().toString(),
        'svix-signature': 'test_signature',
      },
      body: JSON.stringify({
        type: 'user.created',
        data: {
          id: 'user_test123',
          email_addresses: [
            {
              id: 'email_test',
              email_address: 'test@example.com',
            },
          ],
          primary_email_address_id: 'email_test',
          first_name: 'Test',
          last_name: 'User',
          public_metadata: { role: 'client' },
        },
      }),
    });

    const response = await POST(mockRequest);
    expect(response.status).toBe(200);
  });
});
```

---

## Monitoring Checklist

Use this checklist for ongoing webhook monitoring:

- [ ] Check webhook success rate in Clerk Dashboard (should be > 99%)
- [ ] Monitor webhook response times (should be < 2s)
- [ ] Review failed webhook attempts weekly
- [ ] Verify user count in Clerk matches database
- [ ] Check for orphaned records monthly
- [ ] Test role assignment flow quarterly
- [ ] Review webhook logs for errors
- [ ] Validate cascade deletes are working

---

## Next Steps After Testing

Once webhooks are tested and working:

1. ✅ **Document for your team** - Share this guide
2. ✅ **Set up monitoring** - Use Clerk Dashboard or custom monitoring
3. ✅ **Plan for scale** - Consider async processing for high volume
4. ✅ **Implement role UI** - Build admin interface for role management
5. ✅ **Add audit logging** - Track who changes what roles
6. ✅ **Test failure scenarios** - What happens if database is down?

---

## Support Resources

- **Clerk Webhooks Docs:** https://clerk.com/docs/integrations/webhooks
- **Svix Documentation:** https://docs.svix.com
- **Project Documentation:** See `CLERK_WEBHOOK_SETUP.md`
- **Troubleshooting:** See `TROUBLESHOOTING.md`

---

**Last Updated:** 2024
**Webhook Handler:** `apps/admin/src/app/api/webhooks/route.ts`