Video KYC Agent

The Video KYC Agent provides automated identity verification by processing video files containing Aadhaar card information. It extracts and validates identity details from both audio (spoken information) and visual (Aadhaar card image) components of the video, performing cross-validation to generate a verification score.

Base URL

/api/agents/video_kyc

Authentication

All endpoints require authentication. Sign up to the https://cloud.nextneural.ai to get your API key.

How It Works

The Video KYC Agent performs comprehensive identity verification:

Audio Processing: Extracts audio from video, transcribes speech, and extracts Aadhaar details
Visual Processing: Detects Aadhaar card in video frames and extracts text using OCR
Cross-Validation: Compares audio and visual data to calculate a verification score (0-100)
Quality Assessment: Analyzes multiple frames to find the best quality image for processing

Endpoints

1. Health Check

Check if the Video KYC agent service is running.

Endpoint: GET /health

Authentication: None required

Response:

{
  "status": "healthy",
  "service": "VIDEO KYC Agent"
}

2. Process KYC Video

Process a video file to extract and verify Aadhaar card information from both audio and visual components. The file is fetched securely using the document UUID - no file paths or URLs are exposed.

Endpoint: POST /kyc_process

Authentication: Required (scope: agent:video_kyc)

Request Body:

{
  "document_id": "550e8400-e29b-41d4-a716-446655440000",
  "force_reprocess": false
}

Parameters:

document_id (required, string): UUID of the knowledge base document containing the video file
force_reprocess (optional, default: false): Re-process video even if already analyzed

Request Example:

curl -X POST "https://nextneural-api.superteams.ai/api/agents/video_kyc/kyc_process" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "550e8400-e29b-41d4-a716-446655440000",
    "force_reprocess": false
  }'

Response (Successful Processing):

{
  "kbDocumentId": "550e8400-e29b-41d4-a716-446655440000",
  "audioTranscript": "My name is Rajesh Kumar Singh. My date of birth is 15th August 1990. My Aadhaar number is 1234 5678 9012.",
  "parsedAudioData": {
    "name": "Rajesh Kumar Singh",
    "dob": "15/08/1990",
    "aadharId": "1234 5678 9012"
  },
  "parsedVisualData": {
    "name": "RAJESH KUMAR SINGH",
    "dob": "15/08/1990",
    "aadharId": "1234 5678 9012",
    "gender": "Male",
    "confidence": 92.0
  },
  "verificationScore": 95.5,
  "detectionConfidence": 92.0,
  "warning": null,
  "alreadyEvaluated": false,
  "evaluatedAt": "2025-01-14T10:30:00",
  "isReevaluation": false,
  "createdAt": "2025-01-14T10:30:00",
  "updatedAt": "2025-01-14T10:30:00"
}

Response (Cached Result):

{
  "kbDocumentId": "550e8400-e29b-41d4-a716-446655440000",
  "audioTranscript": "...",
  "parsedAudioData": { ... },
  "parsedVisualData": { ... },
  "verificationScore": 95.5,
  "detectionConfidence": 92.0,
  "warning": null,
  "alreadyEvaluated": true,
  "evaluatedAt": "2025-01-14T10:30:00",
  "isReevaluation": false,
  "createdAt": "2025-01-14T10:30:00",
  "updatedAt": "2025-01-14T10:30:00"
}

Response (With Warnings):

{
  "kbDocumentId": "660e8400-e29b-41d4-a716-446655440001",
  "verificationScore": 45.0,
  "detectionConfidence": 65.0,
  "warning": "Incomplete data extracted from video. Please upload a clearer video",
  "parsedVisualData": {
    "name": "RAJESH KUMAR",
    "dob": "",
    "aadharId": "",
    "gender": "",
    "confidence": 65.0
  },
  "parsedAudioData": { ... },
  "alreadyEvaluated": false,
  "isReevaluation": false,
  "createdAt": "2025-01-14T10:30:00",
  "updatedAt": "2025-01-14T10:30:00"
}

Notes:

The file path is automatically fetched from the knowledge base document using the document_id
The document_id must belong to the authenticated user (ownership is verified)
If force_reprocess=true, existing KYC data will be re-processed and updated
The system automatically selects the best frame from the video for OCR processing
No file paths or URLs are exposed in API requests or responses
Verification score is calculated by comparing audio and visual data:
- Name match: 40 points (exact) or 20 points (partial)
- DOB match: 20 points
- Aadhaar number match: 40 points
Maximum verification score is 100

3. Get My KYC Records

Retrieve all KYC records for the authenticated user.

Endpoint: GET /my_kyc_records

Authentication: Required (scope: agent:video_kyc)

Query Parameters:

limit (optional, default: 100): Maximum number of records to return

Request Example:

curl -X GET "https://nextneural-api.superteams.ai/api/agents/video_kyc/my_kyc_records?limit=50" \
  -H "Authorization: Bearer YOUR_TOKEN"

Response:

{
  "success": true,
  "userId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "count": 3,
  "records": [
    {
      "id": 789,
      "analyzedAt": "2025-01-14T10:30:00",
      "verificationScore": 95.5,
      "name": "Rajesh Kumar Singh"
    },
    {
      "id": 788,
      "analyzedAt": "2025-01-13T15:20:00",
      "verificationScore": 88.0,
      "name": "Priya Sharma"
    },
    {
      "id": 787,
      "analyzedAt": "2025-01-12T09:45:00",
      "verificationScore": 72.5,
      "name": "Unknown"
    }
  ]
}

Notes:

Returns records in reverse chronological order (newest first)
Only returns records belonging to the authenticated user
name field shows the name from audio data, falling back to visual data or "Unknown"

4. Get Specific KYC Record

Retrieve detailed information for a specific KYC record.

Endpoint: GET /kyc_record/{record_id}

Authentication: Required (scope: agent:video_kyc)

Path Parameters:

record_id (required): The ID of the KYC record to retrieve

Request Example:

curl -X GET "https://nextneural-api.superteams.ai/api/agents/video_kyc/kyc_record/789" \
  -H "Authorization: Bearer YOUR_TOKEN"

Response:

{
  "success": true,
  "record": {
    "kbDocumentId": "550e8400-e29b-41d4-a716-446655440000",
    "audioTranscript": "My name is Rajesh Kumar Singh. My date of birth is 15th August 1990. My Aadhaar number is 1234 5678 9012.",
    "parsedAudioData": {
      "name": "Rajesh Kumar Singh",
      "dob": "15/08/1990",
      "aadharId": "1234 5678 9012"
    },
    "parsedVisualData": {
      "name": "RAJESH KUMAR SINGH",
      "dob": "15/08/1990",
      "aadharId": "1234 5678 9012",
      "gender": "Male",
      "confidence": 92.0
    },
    "verificationScore": 95.5,
    "detectionConfidence": 92.0,
    "warning": null,
    "alreadyEvaluated": false,
    "evaluatedAt": "2025-01-14T10:30:00",
    "isReevaluation": false,
    "createdAt": "2025-01-14T10:30:00",
    "updatedAt": "2025-01-14T10:30:00"
  }
}

Notes:

Only the record owner can access their KYC records
Returns 404 if record doesn't exist or user doesn't have access

Data Models

Parsed Audio Data Structure

{
  "name": "Full Name",
  "dob": "DD/MM/YYYY",
  "aadharId": "XXXX XXXX XXXX"
}

Parsed Visual Data Structure

{
  "name": "FULL NAME",
  "dob": "DD/MM/YYYY",
  "aadharId": "XXXX XXXX XXXX",
  "gender": "Male/Female",
  "confidence": 92.0
}

Full Record Structure

{
  "kbDocumentId": "550e8400-e29b-41d4-a716-446655440000",
  "audioTranscript": "Transcribed and translated speech...",
  "parsedAudioData": {
    "name": "Full Name",
    "dob": "DD/MM/YYYY",
    "aadharId": "XXXX XXXX XXXX"
  },
  "parsedVisualData": {
    "name": "FULL NAME",
    "dob": "DD/MM/YYYY",
    "aadharId": "XXXX XXXX XXXX",
    "gender": "Male/Female",
    "confidence": 92.0
  },
  "verificationScore": 95.5,
  "detectionConfidence": 92.0,
  "warning": null,
  "alreadyEvaluated": false,
  "evaluatedAt": "2025-01-14T10:30:00",
  "isReevaluation": false,
  "createdAt": "2025-01-14T10:30:00",
  "updatedAt": "2025-01-14T10:30:00"
}

Field Descriptions:

Top-Level Fields:

kbDocumentId: UUID of the knowledge base document
audioTranscript: Full transcribed and translated speech from video
verificationScore: Cross-validation score (0-100)
detectionConfidence: Aadhaar card detection confidence (0-100)
warning: Warning message if any issues detected
alreadyEvaluated: Whether this was a cached result
evaluatedAt: Timestamp when analysis was performed
isReevaluation: Whether this is a re-processed result
createdAt: Record creation timestamp
updatedAt: Last update timestamp

Parsed Audio Data:

name: Name extracted from speech transcription
dob: Date of birth in DD/MM/YYYY format
aadharId: 12-digit Aadhaar number with spaces

Parsed Visual Data:

name: Name extracted from Aadhaar card OCR (usually uppercase)
dob: Date of birth extracted from card
aadharId: Aadhaar number from card OCR
gender: Gender information from card
confidence: Detection confidence score (0-100)

Verification Score Calculation

The verification score (0-100) is calculated by comparing audio and visual data:

Comparison	Points	Condition
Name - Exact Match	40	Audio name exactly matches visual name
Name - Partial Match	20	Some name parts match between audio and visual
DOB - Exact Match	20	Date of birth matches exactly
Aadhaar - Exact Match	40	Aadhaar number matches exactly

Score Interpretation:

90-100: Excellent match, high confidence verification
70-89: Good match, acceptable verification
50-69: Moderate match, manual review recommended
0-49: Poor match, verification failed

Error Responses

All endpoints may return the following error responses:

400 Bad Request:

{
  "detail": "document_id is required for security validation"
}

403 Forbidden (Document Access):

{
  "detail": "Access denied. Document 550e8400-e29b-41d4-a716-446655440000 does not belong to user a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}

403 Forbidden (Invalid Document):

{
  "detail": "Document with UUID 550e8400-e29b-41d4-a716-446655440000 does not exist in knowledgebase"
}

404 Not Found:

{
  "detail": "File path not found for KB document."
}

404 Not Found (Record):

{
  "detail": "KYC record not found or you don't have access to it"
}

500 Internal Server Error:

{
  "detail": "Error processing video: [error message]"
}

Warning Messages

The system may return warning messages in certain scenarios:

"No Aadhaar card detected in video": Vision model failed to detect any Aadhaar card in the analyzed frames
"Incomplete data extracted from video. Please upload a clearer video": Some required fields (name, DOB, or Aadhaar number) are missing from the extracted data

Best Practices

Video Quality Requirements

Resolution: Minimum 720p recommended
Duration: 10-30 seconds optimal
Lighting: Good, even lighting on the Aadhaar card
Focus: Card should be clearly visible and in focus
Audio: Clear speech, minimal background noise
Content: User should speak name, DOB, and Aadhaar number clearly

Recording Guidelines

Hold the Aadhaar card steady in frame for at least 3-5 seconds
Ensure card is flat and not tilted
Avoid glare or reflections on the card
Speak clearly and at moderate pace
Pronounce Aadhaar number digit by digit
Use proper date format (day, month, year)

Integration Workflow

# 1. Upload video to knowledge base (via your file upload system)
# This creates a KB document with UUID: 550e8400-e29b-41d4-a716-446655440000

# 2. Process the video for KYC verification (file is fetched via document_id)
curl -X POST "https://nextneural-api.superteams.ai/api/agents/video_kyc/kyc_process" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "550e8400-e29b-41d4-a716-446655440000"
  }'

# 3. Check verification score and validate
# If score >= 70: Approved
# If score < 70: Manual review or re-capture

# 4. Retrieve record later if needed
curl -X GET "https://nextneural-api.superteams.ai/api/agents/video_kyc/kyc_record/789" \
  -H "Authorization: Bearer YOUR_TOKEN"

# 5. List all KYC records for auditing
curl -X GET "https://nextneural-api.superteams.ai/api/agents/video_kyc/my_kyc_records" \
  -H "Authorization: Bearer YOUR_TOKEN"

# 6. Re-process if needed (e.g., improved AI model)
curl -X POST "https://nextneural-api.superteams.ai/api/agents/video_kyc/kyc_process" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "document_id": "550e8400-e29b-41d4-a716-446655440000",
    "force_reprocess": true
  }'

Performance Considerations

Processing Time: Typically 15-30 seconds per video
Frame Analysis: Default analyzes up to 30 frames (configurable)
Caching: Results are cached; re-processing requires force_reprocess=true
Concurrent Processing: Multiple videos can be processed simultaneously
Storage: Cropped Aadhaar frames are saved for audit purposes

Security Features

User Isolation: All queries are private to your account
Document Ownership: Only you can access your documents
Authentication: All endpoints require valid API tokens with agent:video_kyc scope
UUID-based Identification: Secure, non-sequential identifiers for documents
No File Path Exposure: File paths are never exposed in API requests or responses
Frame Storage: Extracted frames stored with unique timestamps for audit trails
Data Validation: Cross-validation between audio and visual data prevents fraud

Troubleshooting

Low Verification Scores

Possible Causes:

Poor video quality or lighting
Unclear audio or background noise
Card not fully visible in frame
Incorrect pronunciation of details
Translation errors for non-English speech

Solutions:

Re-record video with better quality
Improve lighting conditions
Speak more clearly and slowly
Hold card steady in frame
Use force_reprocess=true to retry

No Aadhaar Card Detected

Possible Causes:

Card not visible in any frame
Card too small or far from camera
Poor lighting or focus
Card obstructed or partially hidden

Solutions:

Ensure card is clearly visible
Move card closer to camera
Improve lighting
Hold card steady for longer duration

Missing Data Fields

Possible Causes:

OCR failed to extract text
Text on card is blurry or unclear
Partial card visibility
Damaged or worn card

Solutions:

Use clearer video with better resolution
Ensure entire card is visible
Clean card surface before recording
Adjust camera angle to avoid glare

Base URL​

Authentication​

How It Works​

Endpoints​

1. Health Check​

2. Process KYC Video​

3. Get My KYC Records​

4. Get Specific KYC Record​

Data Models​

Parsed Audio Data Structure​

Parsed Visual Data Structure​

Full Record Structure​

Verification Score Calculation​

Error Responses​

Warning Messages​

Best Practices​

Video Quality Requirements​

Recording Guidelines​

Integration Workflow​

Performance Considerations​

Security Features​

Troubleshooting​

Low Verification Scores​

No Aadhaar Card Detected​

Missing Data Fields​

Base URL

Authentication

How It Works

Endpoints

1. Health Check

2. Process KYC Video

3. Get My KYC Records

4. Get Specific KYC Record

Data Models

Parsed Audio Data Structure

Parsed Visual Data Structure

Full Record Structure

Verification Score Calculation

Error Responses

Warning Messages

Best Practices

Video Quality Requirements

Recording Guidelines

Integration Workflow

Performance Considerations

Security Features

Troubleshooting

Low Verification Scores

No Aadhaar Card Detected

Missing Data Fields