I’m a tech person—not the type who grinds out code all day, but not the type who never touches it either. For me, coding is pure interest; I’m not looking to make a full living out of it, just looking to find some joy in the process of exploration. As my handle suggests, my writing style will be a "raw log"—because I believe the exploration of technology is always a journey. If anyone cares about the process, a slightly refined "exploration log" is often more valuable than a polished summary.

I’m Hurting—So What’s the Move?

After getting burned by versioning issues in that interview and the struggles I mentioned before, I knew I had to change something. Having built a few small AI apps, I knew the solution: I needed to feed the AI as much information as it was missing—specifically, the latest documentation and code samples. But how do I present that to the AI? My first thought was MCP (Model Context Protocol). If the AI knows that it doesn't know something during the Vibe Coding process, it should be able to go out, find the relevant new features or code samples, and then code based on those examples. MCP is essentially the interface that allows the AI to call external tools during runtime.

That led to two questions:

1. Where do I find these features and code samples?

2. Once I have this info, how do I serve it via MCP?

I consulted Gemini, and here is the verdict: currently, there isn't a one-stop shop, but there are ways to do it if you split it into two steps. For the first question—latest features and samples—they usually exist in the library’s docs or auto-generated documentation. But scraping them is the real hurdle. Let’s talk about that.

Scraping the Documentation

Gemini suggested two main paths: SaaS-based scrapers or local scraping libraries, both of which can convert pages into Markdown. I tested two:

1. The SaaS Route: Firecrawl

• The free tier exists, but it’s limited to about 500 pages a month.

• It supports real-time AI crawling, but that’s an extra cost.

• It supports "MCP + Real-time AI Crawl," but doesn't seem to support "MCP + Pre-scraped content" out of the box.

• Firecrawl is open-source and can be self-hosted via Docker, but the build process for Docker Compose is painfully slow and full of errors (I’m on an aging Mac that can't be updated anymore—I'm not going to torture myself). I gave up on this after one try.

2. The Local Route: Crawl4ai

• This is a Python library that supports various crawling methods and filtering. It works great and, most importantly, it’s free. We’ll look at the results in a bit.

• It’s very easy to use: I just had the AI write a call script, gave it a root directory for the docs and a prefix filter, and I was good to go.

AI told me that Cursor's built-in scraping is likely powered by Firecrawl, so the SaaS version is probably top-tier. I tried using it to crawl some LangChain docs, and the quality was solid. But the credits vanish fast—without paying, it’s basically a non-starter for large docs.

When I moved to Crawl4ai, I had to decide where to run it. Documentation is usually hosted globally, and scraping requires a lot of back-and-forth communication. Plus, network latency (and the Great Firewall if you're in certain regions) can be a pain. After some thought, I found the perfect spot: Google Colab.

Some people might ask why I’m so obsessed with Google products. The truth is, Google provides so many low-cost (often free) and open solutions for developers to play with. At its core, Google Colab is just an online Jupyter Notebook running on a temporary VM. It’s meant for ML/AI and data science, but you can absolutely use it as a standard, cloud-based Jupyter environment. A pro-tip: Colab gives you free GPU credits, so you can even use it for light LLM fine-tuning or small-scale BERT pre-training. Here, by mounting Google Drive to the Jupyter Notebook, I could have Colab scrape the pages for me. By setting a reasonable sleep interval, I scraped thousands of pages without a single IP ban.

Here is the Jupyter script I used to scrape the Kubernetes JavaScript SDK documentation (I’m hosting a mirror of it on a github.io page):

# Install crawl4ai
!pip install crawl4ai 
# Initialize crawl4ai - note: you might need to restart the Colab runtime after first run
!crawl4ai-setup 

from google.colab import drive 
drive.mount('/content/drive') # Mount Google Drive

import asyncio
import os
import re
from urllib.parse import urlparse
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, CacheMode

# ==================== Config ====================
# 1. Target & Scope
BASE_URL = "https://zhibinyang.github.io/kubernetes-client-node-docs-v1.4-unofficial/modules/index.html"
PREFIX = "https://zhibinyang.github.io/kubernetes-client-node-docs-v1.4-unofficial/"

# 2. Extension Filter (Set to None or [] to crawl all pages under the prefix)
ALLOWED_EXTENSIONS = ['.html', '.htm']

# 3. Storage Settings
OUTPUT_DIR = "/content/drive/MyDrive/Docs/Kubernetes-Node"
TRACKER_FILE = os.path.join(OUTPUT_DIR, "crawled_urls.txt")

# 4. Crawling Preferences
SLEEP_TIME = 1.0  # Seconds between requests
# ===============================================

os.makedirs(OUTPUT_DIR, exist_ok=True)

def clean_url(url):
    """Remove anchors and query params to ensure uniqueness"""
    return url.split('#')[0].split('?')[0].rstrip('/')

def should_crawl(url, prefix, processed_set, allowed_exts):
    """Check if a URL meets the crawling criteria"""
    cleaned = clean_url(url)

    # Basic check: Not processed and matches prefix
    if cleaned in processed_set or not cleaned.startswith(prefix):
        return False

    # Extension check: Must match if a list is provided
    if allowed_exts:
        path = urlparse(cleaned).path
        # Handle index cases: paths ending in / usually correspond to index.html
        if path.endswith('/') or not os.path.basename(path):
            return True
        return any(path.lower().endswith(ext.lower()) for ext in allowed_exts)

    return True

def get_file_name(url):
    """Naming logic: Use the last two parts of the path"""
    path = urlparse(clean_url(url)).path.strip('/')
    # Strip known extensions to save as .md
    if ALLOWED_EXTENSIONS:
        for ext in ALLOWED_EXTENSIONS:
            path = re.sub(re.escape(ext) + r'$', '', path, flags=re.IGNORECASE)

    parts = [p for p in path.split('/') if p]
    if len(parts) >= 2:
        name = f"{parts[-2]}-{parts[-1]}"
    elif len(parts) == 1:
        name = parts[0]
    else:
        name = "index"

    return re.sub(r'[^\w\-]', '_', name) + ".md"

async def universal_crawler():
    # 1. Load progress
    processed_urls = set()
    if os.path.exists(TRACKER_FILE):
        with open(TRACKER_FILE, 'r') as f:
            processed_urls = set(line.strip() for line in f if line.strip())

    # 2. Crawler Config
    browser_config = BrowserConfig(headless=True)
    run_config = CrawlerRunConfig(
        cache_mode=CacheMode.BYPASS,
        word_count_threshold=100,
        remove_overlay_elements=True
    )

    async with AsyncWebCrawler(config=browser_config) as crawler:
        queue = [BASE_URL]

        while queue:
            current_raw_url = queue.pop(0)
            current_url = clean_url(current_raw_url)

            # Re-check (prevent duplicates in queue)
            if current_url in processed_urls:
                continue

            print(f"🚀 Processing: {current_url}")

            result = await crawler.arun(url=current_url, config=run_config)

            if result.success:
                # A. Save file
                file_name = get_file_name(current_url)
                file_path = os.path.join(OUTPUT_DIR, file_name)
                with open(file_path, "w", encoding="utf-8") as f:
                    f.write(result.markdown)

                # B. Update tracker
                processed_urls.add(current_url)
                with open(TRACKER_FILE, "a") as f:
                    f.write(current_url + "\n")

                # C. Find new links
                for link in result.links.get("internal", []):
                    link_url = clean_url(link['href'])
                    if should_crawl(link_url, PREFIX, processed_urls, ALLOWED_EXTENSIONS):
                        queue.append(link_url)
            else:
                print(f"❌ Failed: {current_url} - {result.error_message}")

            await asyncio.sleep(SLEEP_TIME)

# Execute
await universal_crawler()

One limitation of Colab is that it requires your browser to stay open and your connection to be active to keep the VM alive. For a task like this, keeping it open all day isn't a huge deal. But if your network drops for more than 30 minutes, the resources are released and your script stops. My script accounts for this by tracking progress, so you can pick up where you left off.

The Kubernetes docs were my "Phase 2" scrape. "Phase 1" was the entire documentation for LangChain.js and LangGraph.js—about 400 pages. While the Kubernetes script was running, I was already testing the search effectiveness of the LangChain data. I didn’t even wait for the K8s scrape to finish; I was too excited to see how it worked with MCP.

Here is a look at one of the scraped files:

# Tools
Tools extend what agents can do—letting them fetch real-time data, execute code...
## Create tools
### Basic tool definition
The simplest way to create a tool is by importing the `tool` function...

Now that I've scraped it, how do I use it?

I spent a lot of effort getting this data, so I needed an MCP service to serve it up immediately. Following the principle of "Don't reinvent the wheel," I looked for the simplest implementation. That's when I turned back to a piece of software I've been using for a while: Obsidian.

Obsidian is a free Markdown note-taking app with a massive plugin ecosystem. I previously used a plugin called "Copilot" which was great—it uses a built-in vector database combined with Embedding Model APIs (like OpenAI) to provide a full RAG (Retrieval-Augmented Generation) experience for your local notes. Since it's a note-taking app, Markdown rendering is built-in.

If there were a plugin that could link a vector database and an Embedding model while providing an MCP interface, I’d have the best of both worlds: managing docs in Obsidian (reviewing and editing them) while letting my AI IDE query them. I found a plugin called MCP Server (not in the official store; you have to clone it from GitHub and build it manually).

GitHub: Minhao-Zhang/obsidian-mcp-server

This plugin has all the hallmarks of a proper RAG system: configurable Embedding models, chunking, overlap settings, similarity thresholds, and top-k retrieval. It seemed perfect. However, I later realized a major flaw: it doesn't support incremental updates. If you add one new document, you have to rebuild the entire vector database. If you have a lot of docs, your API costs are going to climb fast.

After getting it running, I tested it using the official MCP Inspector:

npx @modelcontextprotocol/inspector

Using the SSE mode, I plugged in the SSE address from the Obsidian plugin and connected. I could then list the tools and use simple_vector_search to query my docs.

Does it actually work?

We scraped the docs with Crawl4ai and built the vector DB in Obsidian. We were dying to see how this improved the Vibe Coding experience. But the Inspector tests revealed a problem immediately.

Looking at the LangChain Markdown example from earlier, most documentation pages start with a huge introduction/navigation section and end with a long list of related links. These sections are packed with keywords. If you use standard chunking (say, 1000 or 2000 characters), these headers and footers become their own chunks.

From a vector DB perspective, these "noise" chunks actually have a higher semantic density than the actual code blocks. Plus, the intros often mention features from other pages. As a result, when you search via the MCP interface, the top results—sorted by similarity—are often just the intro/navigation chunks from five different pages, rather than the code implementation you actually need.

You can see it yourself if you look at the raw documentation pages: https://docs.langchain.com/oss/javascript/langchain/models

From this angle, after all this work to get the pipeline running, the results were... underwhelming. So, is there anything we can do to save our Vibe Coding setup?

There is, though whether it works is a story for next time. Stay tuned.

I’m a tech person—not the type who grinds out code all day, but not the type who never touches it either. For me, coding is pure interest; I’m not looking to make a full living out of it, just looking to find some joy in the process of exploration. As my handle suggests, my writing style will be a "raw log"—because I believe the exploration of technology is always a journey. If anyone cares about the process, a slightly refined "exploration log" is often more valuable than a polished summary.

How I Started "Vibe Coding"

My journey into Vibe Coding started with an interview two months ago—or rather, not the preparation for it, but the post-interview reflection. It was a live test where I was told in advance I’d have one hour to build an Agent. I mistakenly assumed it would be a low-code task (since it was a Solutions Architect role), so I spent a day brushing up on Coze. When the interview started and they said it was a "full-code" test, I was totally blindsided. Since I was already there, I tried to wing it using my old code snippets and ChatGPT/Gemini to force out a LangGraph.js implementation. The problem? I hadn't touched it in a month, and LangGraph.js had jumped from v0.3 to v1.0. Other dependencies had changed too, and most of my snippets were broken.

In a desperate "just make it work" attempt, I started compromising. I ditched the PDF parsing (dependency issues), gave up on running the Graph with State (I was in such a rush I missed a parameter and couldn't debug it), and just used LangChain.js with basic context (which forced me to add an intent classifier at the start). By the time I managed to paste the PDF text in manually and get the intent branches working, my hour was up—and I hadn't even written the core logic yet.

The interviewer told me that with my approach, unless I had the questions in advance, it was almost impossible to finish in an hour. But, if I had used something like Cursor with a proper setup, an hour is usually plenty.

That moment felt exactly like 2015, seeing how smooth the iPhone was for the first time. Suddenly, the Nokia in my hand just wasn't "it" anymore.

What Tools Do I Use for Vibe Coding?

Looking at the landscape today, Vibe Coding tools are everywhere, from IDEs to CLIs. I checked out some CLI tools and tried the Gemini CLI, but it felt a bit too "high-end" for my needs. I think the key point is: when the pros are Vibe Coding, they are mostly designing and reviewing code. For me, besides building a POC, I still need to learn. Features like "Jump to Definition" and seeing code diffs in an IDE are essential for me, so I decided to stick with an IDE.

On the other hand, to be honest, I tried Cursor four or five months ago. But as a JetBrains Toolbox user of over 10 years, I just couldn't deal with the UI (even with VS Code, I only use it for config files or as a temporary clipboard). Plus, I have this obsession with always having the root directory file tree expanded in the top left. The first time I opened Cursor and saw that empty, "where-do-I-even-start" interface, I gave up in less than an hour. When I reinstalled it two months ago, the new version felt much cleaner and at least acceptable, so I won't nag about that anymore.

I tried building some POCs with Cursor. As a free user, it was great until the credits ran out. The "stamina" wasn't quite there. Back then, I tried switching to Gemini 2.5 Flash—it’s much cheaper and almost impossible to run out of credits, but the performance definitely took a hit.

I also tried Google's Firebase Studio. It’s a web-based IDE with a VS Code-style interface. The "everything-in-browser" approach is fresh, but considering the usual "connectivity issues" with Google products here, even with tools, the experience was a bit flaky and prone to disconnects.

Lately, I've been experimenting with JetBrains + Google Code Assist. Even though Code Assist is just a plugin, it has Vibe Coding capabilities. However, since JetBrains isn't as "open" as others, there are many functional limitations. For instance, you can't just click to run a script; you have to manually copy the command it gives you into the terminal. Support for peripheral tools is almost non-existent. I started with WebStorm 2025.1.1 (released mid-2025), which only supported Prompt Templates and MCP. And the MCP didn't even support HTTP requests; I found all sorts of issues during stdio testing. You couldn't even pick the model—it seemed to be locked to Gemini 2.5.

Since MCP was acting up, I tried upgrading WebStorm to the latest 2025.3.1. Then, a "miracle" happened: my Gemini connection died. After researching with AI for a while, I concluded that the 2025.3 major release had a massive change—JavaScript now has its own Runtime. As a result, my previous global proxies or environment variables stopped working. I couldn't find a fix with my current proxy setup, and the AI suggested moving to a TUN-mode proxy, so I put that aside for a bit.

I looked at other candidates and spent an hour on the Gemini CLI before giving up again. I went back to Cursor, but my quota vanished instantly. Finally, I saw Google's new Antigravity. This IDE looks very ambitious. I wanted to try it, but got stuck at the login screen. A quick search revealed—you guessed it—the same proxy issues. At that point, I realized I really had to spend some time figuring out this TUN thing. So, I spent the next day on that.

Today, once I got the TUN setup working, I finally logged into Antigravity and could actually start experimenting.

Now, let's get back to our main topic: the nightmare of versioning.

How Painful is Versioning in Vibe Coding?

If the libraries you use were designed with forward compatibility in mind, and the ecosystem maintained perfect dependency harmony, you might never feel this pain.

While I was messing with Google Code Assist over the past few days, a core theme emerged. I wanted to untie the knot I encountered during that interview: modern software libraries move too fast. Many updates happen after the training cutoff for LLMs. New changes are often incompatible with old ones, meaning the AI gives you old code that breaks when you install the latest version. But if you don't use latest, the AI often suggests random versions and gives you code that simply won't run.

Here are a few glaring examples:

OpenAI API Model Calls in LangChain.js

Letting an AI write LangChain/LangGraph code is a giant trap. From what I’ve seen in LangChain.js and LangGraph.js, the syntax changes between v0.1, v0.2, v0.3, and v1.0 are massive. AI models often fail to match the correct version in their answers, leading to parameter errors or deprecated syntax.

For example, in LangChain.js v0.3, calling an OpenAI-compatible model (like Doubao API) like this was fine:

const model = new ChatOpenAI({
    apiKey: process.env.OPENAI_API_KEY,
    configuration: {
        baseURL: process.env.OPENAI_BASE_URL,
    },
    model: "doubao-seed-1-6-flash-250828", 
    temperature: 0.7,
});

But in v1.0, this throws an error:

'Incorrect API key provided: ************************. You can find your API key at https://platform.openai.com/account/api-keys.'

You guessed it—v1.0 reserved the model parameter exclusively for OpenAI and uses it as the "flag" to identify a native OpenAI call. Other compatible models must now use the modelName parameter.

I've even seen AI suggest this for the same code:

const model = new ChatOpenAI({
    apiKey: process.env.OPENAI_API_KEY,
    baseURL: process.env.OPENAI_BASE_URL,
    model: "doubao-seed-1-6-flash-250828", 
    temperature: 0.7,
});

In reality, LangChain.js never had this syntax. The AI likely "hallucinated" this from the LangChain Python version, which uses a base_url parameter in that position.

Kubernetes JavaScript SDK Version Changes

Logic dictates that K8s is a massive project, but maybe not enough people use the K8s + JS/TS combo, which leads to issues.

I ran into this with WebStorm + Google Code Assist. When I asked the AI to generate code for K8s resources, it always gave me something like this:

await coreV1Api.readNamespacedPod("my-pod", "default");

But if you check the signature for readNamespacedPod in the actual library, the first argument is now a params object containing name, namespace, etc. It should be:

await coreV1Api.readNamespacedPod({ 
    name: "my-pod", 
    namespace: "default" 
});

This was a breaking change across the entire SDK. In the current v1.4, all resource calls have shifted to this new format.

Once you fall into this trap during Vibe Coding, it’s a nightmare. You chat about something else, ask it to refactor some code, and it "helpfully" reverts every call in the file back to the old format. You tell it not to, you show it the correct way, but it forgets after a few rounds. Sigh...

What Do I Do Now?

I've been writing for too long today. I'll pick this up again tomorrow.