Dataset Methodology

From Brief to Structured Delivery

A rigorous, technical approach to capturing Indian conversational voice. We combine managed sourcing, multi-layered annotation, and high-standard QA to support speech AI systems built for real-world use.

1

Requirement Scoping

2

Script Design

3

Speaker Sourcing

4

Managed Capture

5

Multi-pass Annotation

6

Technical QA

7

Secure Delivery

Managed Data Collection

Unlike scraped or synthetic data, our collection is human-in-the-loop. We record across real-world acoustic environments, such as vehicles, busy markets, and quiet offices, scoped to your use case.

Speaker Recruitment

We work with screened contributors across supported Indian language regions. Speaker profile requirements are agreed per project and collected only where relevant, consented, and appropriate for the dataset scope.

Recording Protocol

Standard recording specification is 48kHz / 16-bit linear PCM, confirmed per project. We capture spontaneous speech via scenario-based prompting, supporting natural pauses, fillers, and code-mixing (Hinglish) that scripted data misses.

Transcript Annotation

Each audio file undergoes three passes: (1) Verbatim transcription, (2) Language ID tagging for code-mixed turns, and (3) Speaker diarisation with timestamp alignment, reviewed by a human annotator. Project-scoped annotation fields can be added where agreed, such as intent labels, QA notes, speaker labels, or other delivery fields.

QA Process

Approved datasets pass human review according to the QA level agreed for the project. For gold-standard datasets, we can work to specific quality targets, including WER/CER where relevant and share measured QA metrics during project scoping. Metadata is validated using automated schema check scripts.

Delivery Format

Data is delivered in structured delivery layouts (JSON, XML, or CSV). We provide pre-split Train/Dev/Test subsets and detailed documentation on speaker metadata and environment profiles.

Standard Schema

Our datasets follow a strict file structure and metadata schema, structured for easier use in common ML workflows, with delivery formats agreed during scoping.

File Hierarchy
root/
├── audio/
│   ├── session_001_mic_01.wav
│   ├── session_001_mic_02.wav
│   └── ...
├── transcripts/
│   ├── session_001.json
│   └── ...
├── metadata.csv
└── stats.json
Transcript Object (JSON)
{
  "session_id": "SNX_HI_042",
  "utterances": [
    {
      "speaker": "SPK_1",
      "start": "0.420",
      "end": "3.150",
      "text": "kal office band rahega?",
      "lang": "hinglish"
    }
  ],
  "environment": "indoor_ambient"
}

QA Performance Standards

We measure quality across four technical vectors before any data is greenlit for delivery.

QA-Reviewed Delivery

Human-audited verbatim accuracy review for every speaker turn.

Audio-Grounded Validation

Audio-grounded checks reduce the risk of 'ghost' text or synthetic insertions in transcripts.

Speaker Diarisation

Speaker boundaries are aligned to audio onset and offset, with human review.

Schema Validation

Automated JSON/CSV structural checks for consistent metadata structure.

Governance &
Ethics Protocol

Data integrity isn't just technical; it's legal. Consent-linked collection workflows keep contributor permission, usage scope, and dataset records connected to the project brief.

Explicit Opt-in

Contributors complete task-specific consent flows before eligible recordings enter delivery review.

PII Scrubbing

We review transcripts and metadata to remove personal identifiers such as names, numbers, and addresses, with manual review flagging sensitive audio segments.

Fair Compensation

Transparent payment pipelines designed to support fair and timely contributor payments for approved recordings.

Usage scope agreed per project

Dataset usage rights, AI training or evaluation permissions, retention, client sharing, and delivery terms are defined by the applicable agreement and consent scope.

Ready to Benchmark Our Data?

Request a technical specification sheet or a preview snippet for your specific language pair and domain.