Back to blog

Python · Tutorial · Transcripts

Research a Topic on YouTube: Transcripts, Stats and Top Comments for Every Video in a Search

Search YouTube for a topic, then save each video's transcript, view count and top comments as Markdown. A Python script that builds the folder in under three minutes, a real run on 10 videos, and what it costs.

· 5 min read · YTAPI

Watching ten videos to learn what YouTube creators and viewers think about a topic takes two to three hours. Reading them takes about twenty minutes, and an LLM can summarise them in seconds—provided the text is already collected in one clean place.

This guide builds that folder. It searches YouTube for any query, then downloads each video's full transcript, basic view metrics, and top-voted comments into an organized Markdown folder with an index.md catalog.

The output works directly with whatever tools you use to digest text: NotebookLM, Claude Projects, Cursor, Obsidian, or local grep. The comments are the piece people usually skip. The video shows what the creator wanted to showcase; the top comments show what broke, what viewers pushed back on, and what the video omitted.

What you need

  • Python 3.9 or later with requests (pip install requests).
  • A YTAPI key in the YTAPI_KEY environment variable. New accounts get 200 free credits at ytapi.dev. The 10-video run below used 21 credits.

The script

Save this as topic_research.py and run it with python topic_research.py "self-hosted password manager" 10.

import os
import pathlib
import re
import sys
import time

import requests

API = "https://api.ytapi.dev"
HEADERS = {
    "Authorization": f"Bearer {os.environ['YTAPI_KEY']}",
    "User-Agent": os.environ.get("UA", "topic-research/1.0"),
}
OUT = pathlib.Path("research")


def get(path, **params):
    # Retry briefly on 429/503 rate limits; return any other response directly.
    for attempt in range(4):
        res = requests.get(f"{API}{path}", headers=HEADERS, params=params, timeout=90)
        if res.status_code not in (429, 503):
            return res
        time.sleep(int(res.headers.get("Retry-After", 2 ** attempt)))
    return res


def search(query, limit):
    res = get("/v1/search", q=query, type="video", limit=limit)
    res.raise_for_status()
    return [item for item in res.json()["items"] if item.get("type") == "video"]


def slug(text):
    return re.sub(r"[^a-z0-9]+", "-", text.lower()).strip("-")[:60] or "video"


def save(video):
    vid = video["id"]
    transcript = get("/v1/transcripts", video_id=vid, format="text")
    comments = get(f"/v1/videos/{vid}/comments", sort="top")
    top = comments.json().get("comments", []) if comments.ok else []

    lines = [
        f"# {video['title']}",
        "",
        f"- Channel: {video['author']}",
        f"- Length: {video.get('length_text', '?')}",
        f"- Views: {video.get('view_count_text', '?')}, published {video.get('published_text', '?')}",
        f"- URL: https://www.youtube.com/watch?v={vid}",
        "",
        "## Transcript",
        "",
        transcript.text if transcript.ok else f"(no transcript: HTTP {transcript.status_code})",
        "",
        "## Top comments",
        "",
    ]
    for c in top:
        text = " ".join(c.get("text", "").split())
        lines.append(f"- {c.get('like_count', 0):,} likes: {text}")

    path = OUT / f"{slug(video['title'])}.md"
    path.write_text("\n".join(lines) + "\n", encoding="utf-8")
    return transcript.ok, len(top), path


def main(query, limit=10):
    OUT.mkdir(exist_ok=True)
    index = [
        f"# YouTube research: {query}",
        "",
        "| Video | Channel | Views | Transcript | Comments |",
        "|---|---|---|:---:|---:|",
    ]
    for video in search(query, limit):
        has_transcript, n_comments, path = save(video)
        index.append(
            f"| [{video['title']}]({path.name}) | {video['author']} | "
            f"{video.get('view_count_text', '')} | {'yes' if has_transcript else 'no'} | {n_comments} |"
        )
        print(f"saved {path.name}")

    (OUT / "index.md").write_text("\n".join(index) + "\n", encoding="utf-8")
    print(f"wrote {OUT / 'index.md'}")


if __name__ == "__main__":
    main(sys.argv[1], int(sys.argv[2]) if len(sys.argv) > 2 else 10)

A real run

On 13 November 2026, we ran this against the query "self-hosted password manager" with limit=10. The entire process completed in 146 seconds without hitting any 429 rate limit. It generated 11 files: 10 video markdown dossiers and one index.md summary table.

Together, the files contain 32,349 words of spoken transcripts and 182 top comments.

The generated index.md ranked by YouTube search relevance:

Each markdown dossier looks like this:

# Complete Guide: KeePassXC | Free, Open-Source, Android Sync

- Channel: TroubleChute
- Length: 29:46
- Views: 124,802 views, published 5 years ago
- URL: https://www.youtube.com/watch?v=jL7gfM27EUA

## Transcript

what's up my name is tech number here for troubleshoot and welcome back to another video so if you're like me you're tired of all of these saas programs...

## Top comments

- 19 likes: i honestly believe syncthing may have just made my file syncing between pc and phone like a billion times easier. Thanks A LOT!
- 27 likes: i was searching for a decentralized password manager which works well on mac and mobile devices and found your video. Its really helpful. Thank you

What happens when an LLM reads the files

We passed the research/ folder to Claude Code and asked: "What do the videos agree on, and what do the comments push back on?"

Here is what the synthesis uncovered:

  1. What the videos agree on:

    • Open-source, self-hosted solutions (Vaultwarden, Passbolt, KeePassXC) eliminate recurring SaaS subscriptions and vendor lock-in.
    • Docker containerization is the standard deployment pattern for server-based solutions.
    • Setting up proper reverse proxies with valid SSL/TLS certificates and mandatory 2FA is non-negotiable.
  2. What the comments reveal (the real-world friction):

    • The VPS paradox: Multiple top comments call out creators who preach "own your data, don't trust third parties" while deploying directly onto Linode, DigitalOcean, or Hetzner VPS instances without disk encryption:

      "Don't give your passwords to third parties? Instead, give them to a cloud provider. Got it. :p don't forget full disk encryption!" (554 likes)

    • Backups are the actual product: Commenters note that setting up a password manager is easy; reliable automated backups are why people pay for 1Password or Bitwarden cloud:

      "For a password manager access is only part of the problem. Backup up is the larger value add for me when it comes to paid services." (52 likes) "My raspberry pi died and took all my passwords with it. I lost a lot of accounts. So if you plan on doing this, I'd recommend using actual server grade equipment..." (Viewer response)

    • Threat modeling realities:

      "My threat model is mostly me forgetting my own master password to the password manager. Everything else is a lesser threat." (1,900 likes)

These practical caveats rarely make the main video script, but they appear immediately in the top-voted comments.

What it costs

YTAPI charges per successful response; failed lookups are not billed:

CallUnit CostCount for 10 VideosCredits Spent
Search query (/v1/search)1 credit / page1 page1 credit
Transcripts (/v1/transcripts)1 credit / video10 videos10 credits
Top comments (/v1/videos/{id}/comments)1 credit / video10 videos10 credits

Total: 21 credits for 10 videos (roughly 2 credits per video).

With the 200 free sign-up credits, you can run this research workflow across roughly 90 videos before needing a paid credit pack. If you want detailed video metadata (like subscriber counts, chapters, and hashtags), you can add a call to full video info for 1 additional credit per video.

Variations

  • Filter by upload date: Add "upload_date": "year" or "today" to search() parameters to catch only recent perspectives.
  • Single document format: If you prefer one monolithic document for a single file upload, write all video sections sequentially into one combined_research.md file separated by ---.
  • Full channel archives: If you need to pull every video from a specific creator rather than a search query, use the channel video listing workflow or the one-click web export.

FAQ

Can I run this using an AI agent directly? Yes. In Claude Code, Cursor, or Claude Desktop, you can configure the YTAPI MCP server and prompt: "Search YouTube for 'self-hosted password manager', fetch the transcripts and top comments for the top 5 videos, and synthesize their main disagreements." The agent will call the exact same endpoints.

What happens if a video has no captions? If a video has disabled captions or YouTube has not generated an ASR track, /v1/transcripts returns HTTP 404 (0 credits deducted). The script records (no transcript: HTTP 404) in the dossier and continues without failing. See what to do when a YouTube video has no captions for fallback strategies.

Why not scrape comments with web scraping libraries? YouTube heavily rate-limits and blocks data center IP ranges with HTTP 429 and bot-verification challenges. YTAPI handles proxy rotation, request headers, and retries so your scripts don't stall after the third video.