DEV Community: Kabir Hossain

How to validate JSON responses without forcing heavy dependencies on your users

Kabir Hossain — Sat, 31 Oct 2026 08:10:37 +0000

When you build an API testing framework, checking status codes is the easy part.

assert response.status_code == 200

The hard part is validating the JSON payload. You need to check that the fields are correct, that nested data exists, and eventually, that the schema matches what the API documentation promised.

I've been building Praman, an open-source QA engine in Python. I just finished the API automation layer, and the test suite is now at 663 passing tests. Here is the architectural decision I had to make when adding JSON Schema validation.

If you want to validate a JSON payload against a schema in Python, the standard tool is the jsonschema library. It is excellent, it is powerful, and it supports Draft 2020-12.

But here is the problem: I don't want to force every user of Praman to install it.

Some teams just want to make a GET request and check a few fields. They don't need a full schema validator. If I make jsonschema a required dependency in my pyproject.toml, I am bloating their environment for a feature they might never use.

So I used an optional dependency architecture.

I put the schema validation logic in a dedicated module, but I didn't add jsonschema to the required dependencies. Instead, it lives in an optional extra. If a user tries to call the schema validation method without having the library installed, Praman raises a clear error message:

"To use JSON Schema validation, please install the optional dependencies: pip install praman[schema]"

This is how professional libraries are built. The core package stays lean, and the advanced features are opt-in. It respects the client's environment.

I separated the API assertions into three levels of complexity.

First, top-level field validation. If I just want to check that the response contains a user_id and a status field, I can do that without any external libraries. It's just a dictionary lookup.

Second, nested path validation. If I need to check response["data"]["attributes"]["created_at"], I wrote a simple path resolver. Again, zero dependencies.

Third, full JSON Schema validation. This is where the optional dependency kicks in. If the client actually needs to enforce a Draft 2020-12 schema, they install the extra.

This layered approach means the framework is useful immediately for simple tests, but it scales up when the project gets serious.

When you test APIs, you need to know if the request actually succeeded.

The second half of the API layer was about handling authentication and headers cleanly. I added a Praman-owned ApiClient. It handles Bearer authentication, configurable headers, and the standard HTTP methods: GET, POST, PUT, PATCH, and DELETE.

More importantly, it validates HTTP status codes automatically. If an API returns a 500 when you expected a 200, the framework fails the test with a clear message, rather than forcing the test author to write assert response.status_code == 200 on every single test.

The test suite is now at 663 passing tests. The full build passed on a clean wheel installation, meaning a client can install Praman, run an external API test, and everything works.

If you want to look at the code, the repository is public on GitHub: https://github.com/pranromumu/praman

Next up is the remaining Phase 2 work: UI automation and Page Objects. The API foundation is done. The framework is growing.

Making a Python QA framework installable: what Phase 1 actually requires

Kabir Hossain — Sun, 25 Oct 2026 06:55:27 +0000

When you build a testing framework, the actual test execution logic is only half the work. The other half is making it installable, configurable, and ready for someone else's repository.

I just published the Phase 1 milestone for Praman, an open-source QA engine for Python. The repo is live on GitHub. Here is what it took to get the core framework packaged and running.

The first challenge wasn't writing more browser-automation code. It was building the wheel.

Building a source distribution and a wheel forces you to look at your code from the perspective of a client. I had to verify the package metadata, ensure that Pytest was declared correctly, and make sure the plugin auto-registration worked. Praman uses a pytest11 entry point, which means when a client installs the wheel, the praman_page fixture is available immediately. They don't have to copy any conftest.py files.

The only way to trust this is to test it in isolation. I installed the wheel in a fresh Python environment and ran a smoke test against it. It passed. That single external test proves more than a hundred local tests, because it confirms the packaging actually works.

The test suite is currently at 476 tests passing. One of the tests covers the configuration precedence we talked about in the last article. Another covers the pytest plugin discovery. They run in seconds, which means they can go into a CI pipeline on every commit without slowing things down. The GitHub Actions workflow is already set up and passing.

A QA framework is only as good as its documentation.

I spent time writing a CLIENT_SETUP.md guide and a full README. The README needed to explain the exact difference between the standard configuration pipeline (YAML first, then environment variables, then CLI arguments) and the pytest fixture configuration (which uses two specific environment variables to load the YAML file). Those are two separate concerns, and if the documentation blurs them, the client will guess wrong.

I also added the MIT license and cleaned up the repository structure. The framework folder holds the core code, the tests folder holds the verification suite, and the pyproject.toml declares the build dependencies.

This is not a finished platform. It is a foundation. The roadmap includes API and UI automation, CI/CD integrations, and AI-assisted test analysis. But none of those matter if the base package doesn't install cleanly.

If you want to look at the code or try the wheel, the repository is public. The feedback loop on packaging is what makes the next phase possible.

Config precedence in Python: why your resolver shouldn't know what "CLI" means

Kabir Hossain — Wed, 14 Oct 2026 03:43:48 +0000

Most config systems I've written have the same shape. A function that looks at the environment, looks at the command line, looks at a YAML file, and decides which one wins. Something like this:

def load_config():
    if cli_args.timeout:
        return cli_args.timeout
    if os.getenv("TIMEOUT"):
        return os.getenv("TIMEOUT")
    return yaml_config.get("timeout", 30000)

Multiply that by every setting and you get a function that's 200 lines long, impossible to test in isolation, and impossible to change precedence on without editing every branch.

The version I've been working on separates two things the if/else version fuses together. What order sources apply in. And how they merge. Once those are separate, precedence stops being code and starts being structure.

The resolver doesn't know what "CLI" means. Here's the merge function in full:

def resolve_config(sources: Iterable[SourceSpec]) -> dict:
    merged_config = {}

    for spec in sources:
        try:
            config = spec.source.get_config()
        except Exception as error:
            if spec.failure_policy == FailurePolicy.REQUIRED:
                raise
            logger.warning(
                "Optional configuration source %s failed: %s",
                spec.source.__class__.__name__,
                error,
            )
            continue

        merged_config = {**merged_config, **config}

    return merged_config

Read that and notice what it doesn't do. It doesn't check whether a source is a CLI source, an env source, or a YAML source. It doesn't sort anything. It doesn't decide that one kind of source should win over another kind. It takes a list of sources, merges them left to right, and returns the result.

The last source in the list wins. That's the entire precedence rule.

That's not a limitation. It's the design. The moment the resolver knows what "CLI" means, it can start making decisions it shouldn't. It can start reordering. It can start special-casing. And once it does that, you can't test precedence without also testing the resolver's opinions, and you can't change precedence without reading the resolver's code.

Precedence is a list. That list comes from a builder:

def build_standard_pipeline(file_path: str, env: Environment, cli_args=None) -> list[SourceSpec]:
    yaml_source = YAMLSource(file_path, env)
    env_source = EnvSource()
    cli_source = CLISource(args=cli_args)
    return [
        SourceSpec(source=yaml_source, failure_policy=FailurePolicy.REQUIRED),
        SourceSpec(source=env_source, failure_policy=FailurePolicy.OPTIONAL),
        SourceSpec(source=cli_source, failure_policy=FailurePolicy.OPTIONAL),
    ]

YAML first. Env second. CLI last. Since the resolver merges left-to-right and later sources override earlier ones, CLI wins over env wins over YAML. That's the whole precedence system, and it's expressed as three lines of a list.

If you want to change precedence, say you decide env should beat CLI, you change the order of that list. Nothing else changes. No if branches to hunt down. No settings that read precedence from a config file. Just the order of the list.

The tests are where the pattern earns its keep, because they lock in behaviors, not implementation details.

First test. Later source wins.

def test_1_later_source_wins():
    source_a = FakeConfigSource({"browser": "chromium"})
    source_b = FakeConfigSource({"browser": "firefox"})
    source_c = FakeConfigSource({"browser": "webkit"})

    specs = [
        SourceSpec(source=source_a, failure_policy=FailurePolicy.REQUIRED),
        SourceSpec(source=source_b, failure_policy=FailurePolicy.REQUIRED),
        SourceSpec(source=source_c, failure_policy=FailurePolicy.REQUIRED)
    ]

    result = resolve_config(specs)
    assert result["browser"] == "webkit"

That's the base contract. Merge is left-to-right, last write wins. It's obvious in isolation and it's the thing every other behavior depends on.

Second test. Reordering changes result.

def test_2_reordering_changes_result():
    source_a = FakeConfigSource({"browser": "chromium"})
    source_b = FakeConfigSource({"browser": "firefox"})
    source_c = FakeConfigSource({"browser": "webkit"})

    specs = [
        SourceSpec(source=source_c, failure_policy=FailurePolicy.REQUIRED),
        SourceSpec(source=source_a, failure_policy=FailurePolicy.REQUIRED),
        SourceSpec(source=source_b, failure_policy=FailurePolicy.REQUIRED)
    ]

    result = resolve_config(specs)
    assert result["browser"] == "firefox"

This one looks trivial and isn't. It proves the resolver has no hidden opinion about which source should win. If someone ever helpfully adds logic to the resolver that reorders sources, say putting CLI sources last regardless of where they appear, this test fails. That's the category of mistake it prevents. Someone trying to make precedence smarter inside the merge.

Third test. The resolver doesn't sort.

def test_3_resolver_does_not_sort():
    # Simulate a logically reversed order: CLI-like -> YAML-like -> ENV-like
    # The resolver should NOT know these names mean anything.
    cli_like = FakeConfigSource({"browser": "webkit"})
    yaml_like = FakeConfigSource({"browser": "chromium"})
    env_like = FakeConfigSource({"browser": "firefox"})

    specs = [
        SourceSpec(source=cli_like, failure_policy=FailurePolicy.REQUIRED),
        SourceSpec(source=yaml_like, failure_policy=FailurePolicy.REQUIRED),
        SourceSpec(source=env_like, failure_policy=FailurePolicy.REQUIRED)
    ]

    result = resolve_config(specs)
    assert result["browser"] == "firefox"

This is the one I'd defend hardest. It uses three sources named after the standard precedence chain, CLI, YAML, ENV, but passes them in a deliberately wrong order. The resolver's job is to merge them in the order given, not to fix it.

If you ever find yourself wanting to add precedence knowledge to the resolver, this test is the one that will stop you. That's the point. Tests aren't just assertions about current behavior. They're arguments against future versions of yourself who will have worse ideas.

There's a subtle thing in the resolver that's easy to miss.

try:
    config = spec.source.get_config()
except Exception as error:
    if spec.failure_policy == FailurePolicy.REQUIRED:
        raise
    logger.warning(...)
    continue

Each source carries its own failure policy. YAML is required. If the file is missing or malformed, the pipeline fails. Env and CLI are optional. If they fail to load, the pipeline continues with what it has.

This is deliberately a separate concern from precedence. A source can be last in precedence, highest priority, and optional, failing silently. A source can be first in precedence, lowest priority, and required. The two axes are independent.

def test_5_failure_policy_remains_independent(caplog):
    valid_yaml = FakeConfigSource({"browser": "chromium", "timeout": 30000})
    failing_env = FailingSource()
    valid_cli = FakeConfigSource({"browser": "webkit"})

    specs = [
        SourceSpec(source=valid_yaml, failure_policy=FailurePolicy.REQUIRED),
        SourceSpec(source=failing_env, failure_policy=FailurePolicy.OPTIONAL),
        SourceSpec(source=valid_cli, failure_policy=FailurePolicy.REQUIRED)
    ]

    with caplog.at_level("WARNING"):
        result = resolve_config(specs)

    assert result["browser"] == "webkit"
    assert result["timeout"] == 30000
    assert "Optional configuration source" in caplog.text

Env fails. The pipeline continues. CLI still overrides YAML. Timeout from YAML still applies because nothing else set it. Precedence and failure handling do their jobs without interacting.

Fusing these two concerns is the most common mistake I see in config systems. It looks natural. If the CLI arg is missing, fall back to env, which falls back to YAML. But it collapses two orthogonal decisions into one tangled function. Keeping them separate makes each one testable in isolation and changeable without touching the other.

One more test, because it's the one that guards the whole design:

def test_7_resolver_remains_unchanged():
    sig = inspect.signature(resolve_config)
    params = list(sig.parameters.values())
    assert len(params) == 1
    assert params[0].name == "sources"

This asserts that resolve_config takes exactly one parameter. Nothing else. No pipeline name, no environment, no flags. Just the list of sources.

It sounds pedantic, and it is, deliberately. Six months from now, someone (probably me) will want to pass an Environment into the resolver so it can adjust precedence based on the environment. This test makes that change fail loudly, which forces a conversation about whether the resolver should know anything about environments. The answer is almost always no. But without the test, the answer quietly becomes yes, one small change at a time.

What this pattern buys you.

Precedence is a list. Changing it means reordering a list, not editing logic.

The resolver does one thing. It merges dictionaries left to right. That's it. It's 20 lines and it doesn't know what a CLI is.

Failure handling is orthogonal. Each source declares whether it's required or optional. The resolver respects that declaration without conflating it with priority.

Tests lock behaviors, not implementations. Each test asserts something category-level about what the system should never do, not just what it currently does.

None of this is novel. It's the same pattern behind a dozen config libraries. But the version where precedence is structure, rather than a decision buried in a conditional, is the version I've found easiest to live with. Because the day you need to change precedence is the day you least want to be reading the resolver's code.

How I scrape any website in under 5 minutes (Python + Playwright)

Kabir Hossain — Wed, 07 Oct 2026 04:48:09 +0000

python, webdev, tutorial, automation
Every scraping tutorial I've read online ends the same way.

"Here's how to grab a title with BeautifulSoup." Twenty lines of Python. Works on the demo page. Fails on a real website.

That bothered me for a long time. Because real websites are messy. They use JavaScript. They block bots. They hide data behind logins. They paginate. They rename their HTML classes every few months and break every script you've written.

So instead of writing another "20-line scraper" tutorial, I want to show you what actual production scraping looks like. Not the theory — the workflow I use for real client jobs.

What most tutorials skip
When someone asks me to scrape a site, I don't open a fresh Python file and start writing selectors. I follow a process.

Step one is an audit. I check the site first. Is it static HTML or JavaScript-rendered? Does it have clear repeating elements like product cards? Does it paginate cleanly? Does robots.txt allow scraping?

If I skip this step, I waste hours trying to scrape a site that was never going to work. Five minutes of auditing saves five hours of frustration.

Building the audit as a tool
Since I audit every site the same way, I automated the audit itself.

Here's what my tool does when I run it:

bash
tendem-audit
It asks for a URL. Then it:

Fetches the page using the same fetch strategy I use for scraping (static HTTP first, Playwright second)

Checks robots.txt

Auto-detects the repeating card block using heuristics

Fingerprints the CMS (Shopify, WooCommerce, WordPress, Magento)

Prints a verdict: green, yellow, orange, or red

Green means I can quote the job today. Red means I politely decline.

That one step — automating the audit — saved me more time than any scraper I've written.

The actual scraping
Once I know the site is scrapeable, the scraping itself is boring. That's how it should be.

bash
tendem-scrape https://shop.example.com/products --preset shopify --max-pages 50 --format csv,json,sqlite
That one command:

Fetches all 50 pages automatically by following the "Next" link

Uses the WooCommerce preset to find the right fields (title, price, image, link)

Handles the pagination URL changes

Merges items into one dataset

Writes CSV, JSON, and SQLite files

Total time: two minutes. For a 1,000-product catalog.

What production scraping actually means
The scraping part is 20% of the job. The other 80% is what makes a client pay.

Here's what I deliver for every scraping job — no exceptions:

Clean CSV, ready to open in Excel

Not a raw dump. Columns with the right names. No hidden characters. UTF-8 BOM so Excel doesn't mangle it.

Interactive HTML dashboard

A single file the client opens in their browser. Sortable columns. Filterable search box. Works offline. No dependency on my server.

Data quality report

A file that lists exactly what problems I found: missing titles, empty prices, duplicate rows, broken image links. Clients trust the data because they can see what was checked.

Audit trail

What was scraped, when, from what URL, by what method. If the client asks "how did you get this?", the answer is one JSON file away.

The technical stack

For those who want to know how it works under the hood:

Python 3.10 or newer

Playwright for anything JavaScript-rendered

BeautifulSoup with lxml for parsing

Pydantic v2 for validated data models

SQLite or PostgreSQL for storage

GitHub Actions for the CI pipeline

Pytest and Allure for tests and reports

The framework has over 70 automated tests. There's a public test dashboard on GitHub Pages that shows them passing.

What I learned doing this for real clients
Three things stand out.

First, sessions and logins matter more than anything. Half the sites I scrape require a login. If you don't learn how to save cookies and reuse them, half the job disappears.

Second, quality control is what clients actually pay for. A raw CSV is worth ten dollars. A CSV plus a quality report plus a dashboard is worth a hundred. Same data, different packaging.

Third, when a site changes, your tests catch it before the client does. That's not a nice-to-have — it's the difference between a one-off job and a monthly subscription.

Try it yourself
bash
git clone https://github.com/pranromumu/tendem-scraper
cd tendem-scraper
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
playwright install chromium

tendem-scrape https://books.toscrape.com/ --max-pages 3 --open
The last command scrapes three pages and opens the dashboard in your browser.

What's next
I'm building this out as a service for e-commerce sellers and marketing agencies who need competitor pricing, product catalogs, and lead lists on a schedule. If you have a scraping project in mind, or you just want to talk about Python automation, feel free to reach out.

The GitHub repo is open source. Fork it, break it, submit a pull request.

About me: I'm a web scraping and automation engineer based in Malaysia. I build Python tools that extract data from websites and deliver clean, structured files with quality guarantees.

Live demo: https://pranromumu.github.io/tendem-scraper/

GitHub repo: https://github.com/pranromumu/tendem-scraper

Fiverr gig: https://www.fiverr.com/s/3AAA3yL

If you found this post useful, connect with me. I write about Python, scraping, and automation for real client work.

From Script to Secure Pipeline: How I Created a "Trust Boundary" in CI/CD

Kabir Hossain — Fri, 25 Sep 2026 06:33:41 +0000

A few months ago, I thought CI/CD simply meant writing a GitHub Actions YAML file that runs pytest.
If tests pass, the result is 'green'; if they fail, it's 'red'pretty straightforward, right?
However, while progressing through my QA automation roadmap (projects P141–P170), my mentor introduced me to a concept that completely shifted my perspective on testing: the 'Trust Boundary'.
When a test fails in CI, developers need evidence (such as screenshots, traces, or logs) to debug the issue. But how can they be certain that the evidence stored hasn't been corrupted? How do they know the pipeline is making the correct decision?
Here is how I built a secure CI pipeline to address this challenge.

Preserving Evidence of Failure (The if: always()Strategy) Typically, if a GitHub Actions job fails, subsequent steps do not execute. This means that if your Playwright test crashes, the step to upload artifacts won't run-causing the failure screenshot to be lost forever as soon as the virtual machine shuts down. To solve this, I used the if: always() condition for the upload step: -name: Upload Failure Evidence uses: actions/upload-artifact@v4 if: always() with: name: test-evidence path: artifacts/ This ensures that even if the test crashes, the evidence is safely uploaded to GitHub's cloud storage.
Independent Auditor/Verifier (Decoupled from the Test Job) If the test job itself generates the evidence and also verifies it, the situation becomes akin to grading one's own homework. Even if a bug corrupts the file, the test job might still report a successful verification (verification = true) based on incorrect data. To resolve this, I split my pipeline into separate jobs: Test Job: Runs Pytest, generates evidence, calculates the SHA-256 hash, and uploads it to the cloud. Verify Job: Downloads the artifact, recalculates the hash from the raw bytes, and compares the two. This establishes a 'trust boundary.' Here, the Verify job acts as an independent auditor. If anyone (or a system bug) alters or tampers with the evidence while it is stored in the cloud, the hashes will no longer match, and the auditor will mark the pipeline as 'failed.'
The Final Quality Gate Finally, I added a third job: 'The Gate.' This job depends on the 'Verify' job (though the if: always() condition was not used). Its sole purpose is to display the final decision: ================================= FINAL EVIDENCE GATE ================================= Test execution: PASS Artifact upload: PASS SHA-256 verification: PASS FINAL GATE: PASS If verification fails, the 'Gate' step is skipped, and the overall workflow is marked as 'Red' (failed). What's next?"Praman"(Proof) By completing this stage of CI/CD, I realized that QA isn't just about finding bugs; it is about building an engineering framework based on trust and evidence. Stemming from this insight, I’ve named my future architecture'Praman'-a word that translates to 'Proof' or 'Evidence'. The next steps in my roadmap involve transforming this pipeline into a modular, "drop-in" Python QA engine that freelance clients can easily integrate into their own repositories. It will be a tool designed to execute tests, gather irrefutable evidence, and (eventually) leverage AI to analyze the root causes of failures. If you are building CI/CD pipelines, I highly recommend exploring independent artifact verification. It elevates your pipeline from a mere "script runner" into a professional-grade engineering system. What aspects of CI/CD are you currently learning? Let’s connect!

My QA Automation Journey: From Zero to CI/CD 🚀

Kabir Hossain — Mon, 21 Sep 2026 22:43:34 +0000

Hello,
I am embarking on a long journey into QA automation—and documenting my experiences along the way.
Just a few months ago, while I could run pytest, I lacked a clear understanding of what a professional automation framework actually looks like.
I was familiar with some basic concepts of Python and automation. However, there was a significant gap between knowing isolated tools and understanding how everything integrates within a real-world QA engineering workflow.
So, I decided to bridge that gap through hands-on practice.
I am following a roadmap comprising over 200 practical, hands-on projects. It begins with Python and Playwright, progressing through CI/CD, observability, and AI-assisted testing, ultimately leading to the creation of an AI-driven QA platform.
Here is what I have worked on so far:
✅ Python Fundamentals
✅ Playwright
✅ Pytest
✅ Page Object Model
✅ API Testing
✅ Authentication and Session Handling
✅ Network Interception
✅ Screenshots, Traces, Videos, and HTML Reports
✅ GitHub Actions and CI/CD

I am currently approaching Project 152 (P152).
Through this process, I have realized something important: learning QA automation isn't just about learning how to write a test.
It is equally crucial to understand what happens when a test fails. For instance, I recently set up a CI 'quality gate' for a project where the process went like this:

Tests fail
↓
CI pipeline fails
↓
Evidence or data regarding the failure is collected
↓
Artifacts are uploaded

If a test fails, the pipeline should fail too—but I didn't want the evidence or data concerning that failure to be lost along with the pipeline itself.
It was a small project, certainly, but it taught me a valuable lesson about how a real-world CI pipeline should function.
Throughout this journey, I will continue to document my project, mistakes, problem-solving approaches, and everything new I learn.