DEV Community: Spicrawl

Google Was Autocorrecting Our Brand Name, here's What I Did

Ellie — Sat, 31 Oct 2026 10:03:18 +0000

How I Helped Google Recognise Our Brand Name: A Month of Learning SEO for Spicrawl

When we launched Spicrawl on September 25, 2026, I noticed something that felt a little strange.

If I searched for our brand name, Spicrawl, Google kept correcting it to Sprawl. Our website wasn't showing up properly, even when I searched for the exact brand name.

At first, I thought this was just normal for a new website. And in a way, it was. Google didn't really know our brand yet.

I honestly didn't know much about SEO at that point. I wasn't an SEO expert (and I'm still learning), and I had no proper roadmap. I started researching, reading, trying things, making mistakes, and slowly understanding how search engines discover and recognise a new website.

After around a month of work, Google finally started recognising our brand name correctly.

Is this a huge SEO achievement? Probably not. There is still a lot of work left. But for me, it was a small milestone worth sharing.

Here are some things I worked on and learned along the way.

1. Started with the technical SEO basics

I first focused on making sure search engines could properly discover and understand our website.

Some of the things I worked on:

Page titles and meta descriptions.
Sitemap and robots.txt.
Canonical URLs and duplicate-page issues.
Making important pages discoverable.
Fixing indexing issues in Google Search Console.
Making the website accessible to AI crawlers and tools through resources like llms.txt and machine-readable discovery files.

The goal was simple: make it easier for search engines and AI systems to understand what Spicrawl is.

I also learned that submitting a page for indexing doesn't guarantee Google will index it. Sometimes Google discovers a page but chooses not to index it.

2. Listed Spicrawl on MCP registries

Since Spicrawl has an MCP server, getting it listed in relevant MCP directories was another important part of the work.

These are the platforms I worked on:

Official MCP Registry — my first priority and the most important listing.
Glama
Smithery
LobeHub MCP Marketplace
cursor.directory

Some submissions are still pending.

One thing I learned the hard way: Submit to the Official MCP Registry first, then wait at least a week or two before manually submitting to other directories.

Many MCP directories automatically fetch listings from the official registry. Check the others after a week or two; you may find that most listings are already there.

I made the mistake of submitting manually first and adding the official listing later. This resulted in duplicate listings on some platforms.

So, save yourself that little headache: official registry first, then check the others.

3. Made our brand information consistent

I realised that Google needs consistent information to understand that different profiles and websites belong to the same brand.

I worked on:

Adding social media links to our website.
Updating our About page.
Adding our YouTube channel to structured brand information.
Keeping our brand name and description consistent across platforms.

Small details, but they help build a clearer picture of a brand online.

4. Worked on product and startup directories

I also explored product, startup, and software directories to build Spicrawl's presence across the web.

Some platforms include:

G2
SaaS Hub
SourceForge
AlternativeTo
Make.rs
StackShare
GoodFirms
Siteefy
PeerPush
Website Hunt
Startup Inspire
OpenHunts

Most of the marketplace listings were handled by my colleague, while I also explored and worked on several directories.

There are many more to explore. My suggestion is to focus on platforms relevant to your product and audience rather than submitting everywhere just for backlinks.

5. Created social media profiles and started posting

I also worked on creating and maintaining Spicrawl's social media presence.

Social profiles can appear in Google search results quite quickly. Having consistent brand names, profile URLs, descriptions, and regular posts gives people more ways to discover your brand.

You don't need to post every day. Just start building your presence.

6. Researched keywords and planned content

I started exploring what people were actually searching for around web scraping and AI agents.

Some topics I explored:

Web scraping for RAG and LLMs.
Scraping dynamic websites.
Web scraping 403 errors.
Scraping websites with login.
HTML vs Markdown for LLMs.
Web scraping vs APIs.

This helped me understand that writing content isn't just about publishing more blogs. It's about answering questions people are already searching for.

7. Monitored everything

I used Google Search Console and Bing Webmaster Tools to understand what was happening with our website.

I checked:

Which pages were indexed or not indexed.
Search impressions and clicks.
Indexing and canonical issues.

I also explored IndexNow to notify search engines about updated URLs.

These tools helped me understand what was happening instead of simply guessing.

What I learned from this experience

A few things I'd suggest to anyone launching a new brand:

Set up Google Search Console and Bing Webmaster Tools early.
Get your technical SEO basics right.
Keep your brand information consistent across platforms.
Submit to the Official MCP Registry first if you have an MCP server.
Explore relevant product and startup directories.
Create social profiles and keep them active.
Research search queries before planning blog content.
Be patient. Indexing and brand recognition take time.

A small milestone, not the finish line

Today, when I search for Spicrawl, Google still shows a “Did you mean…” suggestion, but it no longer autocorrects the brand name. Instead, it shows information about Spicrawl, including our website, GitHub, and listings across other platforms.

That's a small change, but it means a lot to me because I know how much research and effort went into getting here.

I don't think one particular SEO task fixed everything. It was a combination of small improvements over time.

And honestly, this is just the beginning. There is still so much more to learn and work on.

I'm sharing this because I know there are other people launching their first product or website who might be facing the same issue.

If you're building a new brand, don't get discouraged if Google doesn't recognise your name immediately. Start with the basics, keep building your presence, and give it time.

I'm still learning too. :)

If you've worked on SEO for a new brand or have any suggestions to improve our visibility further, I'd love to hear your thoughts. I'm still learning, and every suggestion helps!

We built our 20-second launch video entirely in code (React, CSS and no 3D library)

Ellie — Thu, 22 Oct 2026 11:47:16 +0000

We built our 20-second launch video entirely in code (React, CSS and no 3D library)

We needed a launch video for Spicrawl, our web scraping API for AI agents. We didn't have a motion designer or After Effects, so we built the whole thing in code. It's React, CSS and a bit of SVG, rendered to MP4 with Remotion. Even the music is generated in JavaScript.

Here's the result 👇

Below is how each part works, in case you want to try it.

The setup: Remotion

Remotion lets you write a video as React components. Each frame is a render of your components at a given frame number. You animate by reading the current frame and computing styles from it:

const frame = useCurrentFrame()
const opacity = interpolate(frame, [0, 18], [0, 1], { extrapolateRight: 'clamp' })
return <h1 style={{ opacity }}>Scrape.</h1>

If you're building apps or AI agents that need live web data, check out Spicrawl. It provides a web scraping API and MCP to help you extract data from websites and use it in your projects.

Spicrawl is currently free to try during beta.

I Tested 16 Job Sources With a Scraping API, here's What Worked!

Mike Alex — Fri, 09 Oct 2026 12:14:05 +0000

I Tested 16 Job Sources With a Scraping API. Here's What Worked

Job hunting can get messy quickly.

You find a role on LinkedIn, another on a startup's career page, and a few more on remote job boards. Soon, you have dozens of tabs open and no single place to track everything 😅

I started thinking about building a small tool that collects job listings from different sources and organizes them in one place.

As a product manager working with Spicrawl, I wanted to explore how its web scraping API could help.

In our October 2026 tests, 15 out of 16 job sources returned usable listings. Here's what we learned.

1. Not every website needs a browser

Some job sources provide data through public APIs or RSS feeds. Others rely on JavaScript to display listings.

Here's a snapshot of our results:

Source	What worked	Credits
Remote OK API	100 job items	1
Remotive API	Structured job data	1
LinkedIn single posting	Full job description	1
LinkedIn job search	Rendered listings	3
Y Combinator jobs	Rendered listings	3
Glassdoor jobs	Rendered listings	3
Indeed search	Listings after scrolling	8

These are results from our tests, not guaranteed results for every request.

The takeaway? Start with the simplest request and only add browser rendering when you need it.

2. Collect job listings with Spicrawl

You'll need a Spicrawl API key, which you can get through Spicrawl.

For a JavaScript-rendered job page, you can use:

curl https://api.spicrawl.com/v1/scrape \
  -H "Authorization: Bearer $SPICRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.ycombinator.com/jobs",
    "js_render": true,
    "cache": false
  }'

This loads the page in a browser so its JavaScript can build the listings.

In our October tests, this returned around 28 KB of content for three credits.

For a source that already provides JSON, you can use a plain fetch instead. For example, Remote OK returned 100 items for one credit.

3. What about pages that load more jobs while scrolling?

This was one of the more interesting results from our tests.

Indeed returned only a tiny, 60-byte page when we tried JavaScript rendering alone. The request technically succeeded, but the job listings weren't there.

We added scrolling actions:

{
  "url": "https://www.indeed.com/jobs?q=data+engineer&l=Remote",
  "js_render": true,
  "wait": 2000,
  "actions": [
    {"scroll": {"to_bottom": true}},
    {"scroll": {"to_bottom": true}}
  ]
}

The response grew to around 68 KB of listings. This request cost eight credits because browser actions use the full-browser rendering tier.

A successful HTTP response doesn't always mean you received useful data. Check the returned content and target status before saving results.

4. Turn scraped pages into a job tracker

Once you've collected the page content, you can extract fields such as:

Job title and company
Location and employment type
Salary, when available
Required skills
Application URL

Spicrawl's autoparse: true option can return structured data embedded in a page, including schema.org JobPosting data when available.

You can store the results in a database or spreadsheet and build features such as job filters, application tracking, and AI-powered comparisons against your skills.

Just remember that not every page contains every field, and some job listings may be outdated.

5. What we learned

A few lessons stood out:

Use APIs and feeds when available. They're often cheaper and easier to process.
Render JavaScript when needed. Some listing pages don't contain their data in the initial HTML.
Check the actual response. Empty pages can still return HTTP 200.
Respect website rules. Check terms and robots.txt, follow rate limits, and collect only the data you need.

What would you build?

A job collector is just one use case. The same idea could help organize internships, scholarships, research opportunities, or listings for a job aggregator.

If you were building a job-hunting assistant, what would you want it to do first?

Find relevant jobs, compare opportunities, match jobs to your skills, or track applications?

I'm exploring these use cases while working on Spicrawl, and I'd genuinely love to hear your ideas.

Want to explore more job boards? Check out the Spicrawl job-postings scraping guide for tested API requests, credit costs, JavaScript rendering, and tips for scraping different job sources.

Explore Spicrawl currently free during beta, with no credit card required.