DEV Community: 4sapi

GitHub Weekly: Agent Skills Sweep Top 3, “De-AI” Design Leads

4sapi — Mon, 02 Nov 2026 07:17:23 +0000

Abstract

GitHub Trending's weekly ranking, read on November 2, 2026, lists 16 projects. Thirteen of them are Skills, plugins or companion tools built for coding agents such as Claude Code, Codex and Cursor. All three top entries are Skills. Impeccable gained about 12,000 stars this week as a design-vocabulary and "AI-flavor" detection rule set. Hallmark from Together AI added about 11,600 stars with a "pages should not look AI-generated" stance. Cloudflare's open-source security-audit-skill added about 10,000 stars, turning the seed of its internal vulnerability-hunting system into a six-stage security-audit Skill. The week's main line is clear: the community is no longer comparing only which model is stronger. It is comparing who can give agents better craftsmanship: aesthetics, security auditing, SEO, and cross-session memory.

1. The Weekly Signal: Skills over Raw Model Power

The GitHub weekly hotlist is the GitHub Trending ranking sorted by stars added in the current week. The edition read on November 2, 2026 contains 16 projects. Of those, 13 serve the coding-agent ecosystem.

The distribution itself is the story. Skill-based extensions now outnumber raw model releases in community attention. Most of these Skills install with a single npx skills add command, but at runtime they consume the model quota of the agent's own configuration. Some single runs can burn hundreds of thousands of tokens. The popularity of Skills is therefore also a statement about where the ecosystem believes the leverage sits: not in the model alone, but in the working method loaded on top of it.

2. The Top Twelve by Weekly Stars

Sorted by stars added this week, the top twelve are:

pbakaus/impeccable: +12,017
nutlope/hallmark: +11,566
cloudflare/security-audit-skill: +10,069
pingdotgg/3code: +6,926
aftan-m/ECC: +6,299
obra/superpowers: +5,896
thedotstack/claude-mem: +5,852
JetBrains/thinkrail: +5,760
TauricResearch/TradingAgents: +5,432
Leonxlnx/taste-skill: +5,336
AndrejDaniel/claude-seo: +5,034
multica-ai/andre-karpathy-skills: +4,459

The set spans five categories: design de-AI, security auditing, agent working methods, memory and IDE, and vertical domains. The design category dominates the top. Impeccable and Hallmark together account for the two largest weekly gains, and taste-skill adds a third design entry further down the list.

3. The Weekly List at a Glance

By weekly stars, the 16 projects fall into five categories. The leading entries show both weekly gains and total star counts:

Design "de-AI": pbakaus/impeccable +12,017 (total 99,417); Nutlope/hallmark +11,566 (44,950); Leonxlnx/taste-skill +5,336 (107,848); jakubkrehel/skills +1,607 (10,106)
Security auditing: cloudflare/security-audit-skill +10,069 (56,442)

4. The Three Head Projects Take Different Approaches

The top three design Skills each solve the same problem differently.

4.1 Impeccable: A Design Vocabulary

Impeccable (Apache-2.0) is a set of "design words". It provides 24 commands such as bolder, quieter, polish, critique and audit, letting the user direct one kind of change with a single word. It also ships 62 deterministic rules for detecting AI flavor. Those rules do not call a model and do not require an API key. Impeccable is better at modifying existing pages. The latest version, cli-v4.8.0, was released on October 26.

The model-free rules are the notable part. A design tool that detects "AI-ness" without invoking a model means the detection is reproducible and cheap to run at scale. There is no hidden prompt, no vendor dependency, and no per-check token cost. That design choice is what makes the 62 rules a reusable asset rather than a wrapper around another model.

4.2 Hallmark: A Generator with Hard Bottom Lines

Hallmark (MIT, from Together AI) is a generator with hard constraints. It bundles 50 theme material sets plus 755 openly licensed photos. After building, it runs a soplint check across 8 widths. Contrast, focus states, mobile layout and fabricated facts are treated as bottom lines. If a page does not meet them, it is not delivered.

4.3 taste-skill: Direction First, Then Build

taste-skill (MIT) is based on a study of 247 hand-picked websites. It establishes a direction first, then looks for "an idea that belongs only to this client", and finally screenshots itself and fixes the first three problems, with at most three iterations. The README is explicit that one build takes roughly 30 to 45 minutes and consumes hundreds of thousands of tokens.

4.4 The Shared Discipline

The common thread across all three is discipline rather than model capability. They all require real material and refuse to fabricate numbers. They all check the finished product in a real browser, rather than letting the model claim it "looks good".

5. Cloudflare's Security-Audit Skill: Six Stages of Vulnerability Hunting

cloudflare/security-audit-skill is the original seed of Cloudflare's vulnerability-hunting system. The README states that the system later expanded into a multi-stage system covering the whole company, described in the Cloudflare blog post "Build your own vulnerability harness". This Skill is the single-repository starting point of that system.

Its core design is that every stage is re-checked by a fresh, independent agent:

Reconnaissance: map the architecture, trust boundaries and input surface, and write architecture.md and a coverage ledger.
Coverage-guided hunting: dispatch mutually isolated hunter agents according to the ledger, and let a "coverage review" find places that were missed.
Candidate validation: hand each suspected vulnerability to a new agent whose task is to disprove it.
Structured output: split results into confirmed, pending and rejected, and validate against a JSON Schema.
Record review: dispatch another new agent to verify the source code referenced by the final conclusion.
Report: generate the report from the reviewed records.

Runs are cumulative. A second run works against the previous ledger to fill gaps. The Skill ships 11 hunting prompts classified by attack surface, covering web protocols, supply chains, cloud deployment and LLM prompt injection. It can be installed with one line: npx skills add ... --skill security-audit.

The "every stage gets a fresh agent" rule is the architectural point. The finder and the verifier are deliberately separated. A new agent that inherits none of the finder's context is less likely to confirm a plausible-looking hypothesis just because it was framed convincingly. Falsification is an explicit assignment, not an afterthought. This mirrors the separation of duties that security teams impose on human reviewers, and it is enforced by process rather than by trust.

6. Other Projects Worth Watching

Beyond the top three, several projects carry strong signals.

JetBrains ThinkRail is an incubation project, created only in July, that gained 5,760 stars this week. It wraps the open-source coding agent pi into a lightweight IDE. Each task gets a git worktree workspace, so multiple agents run in parallel without interfering. Project intent is written into a set of SPEC.md files and preserved across sessions.

claude-mem has crossed 109,000 total stars and stays on the list. It compresses each agent tool call into an observation record and injects those records by index into the next session.

claude-seo contains 26 sub-Skills and 19 dedicated agents. A full SEO audit takes about 10 to 15 minutes. One dedicated check evaluates how likely a page is to be cited by AI search products such as Google AI Overviews, ChatGPT and Perplexity.

andrej-karpathy-skills contains only a single CLAUDE.md file, yet holds 228,000 total stars. A well-written behavior specification is itself in high demand.

7. Hands-On: Install This Week's Skills with One Command

Most Skills on this week's list support Vercel's npx skills command-line tool, which works across Claude Code, Codex, Cursor and OpenCode. Node 18 or higher is required.

Design Impeccable (run in the project root; after install, reload tools and run /impeccable init): npx impeccable install
Design Hallmark (-g means global install): npx skills add npm:@hallmark -g
Design taste-skill (requires Node 22+ and a local Chrome): npx skills add https://github.com/Leonxlnx/taste-skill --skill "design-taste-frontend"
Security audit Cloudflare security-audit-skill: npx skills add https://github.com/cloudflare/security-audit-skill --skill security-audit --global

Interactive commands after install:

/impeccable critique the pricing cards
/hallmark audit ./index.html

The security-audit Skill does not need a special command. In the repository to be audited, start the agent and say "run a security audit on this repository".

8. Where the Tokens Go: Runtime Cost Is Real

These Skills run on the model already configured for the agent. Security audits and multi-round design iterations spawn multiple sub-agents, so token consumption jumps sharply. A heavy run can reach tens of thousands of tokens.

The billing model matters here. Skills are not free computation; they are instruction sets that multiply the agent's model usage. A single design iteration reported at hundreds of thousands of tokens means the cost of "de-AI design" is dominated by model spend, not by the Skill file itself.

When such workloads need to point at different model backends, or when several Skills and agents share one budget, a consistent routing and metering layer becomes necessary. An API gateway such as 4sapi lets a team set a custom base URL with an OpenAI-compatible endpoint, so agents and Skills call the same interface regardless of which provider sits behind it, while usage and spend are tracked in one place.

9. Two Things to Check Before Installing

Skills execute scripts with your permissions. Impeccable's live mode runs the validation script specified in package.json; the README warns to inspect it before use in an unfamiliar repository. Hallmark's browser check needs puppeteer-core and Chrome. For a Skill of unknown origin, read SKILL.md and the scripts directory first.

Do not stack same-kind Skills. The three design Skills have conflicting rules, and enabling them together can leave the agent torn between contradictory guidance. Pick one, run it on a few pages, compare the effect, and then decide.

10. Frequently Asked Questions

Q: Where is the GitHub hotlist, and what is the difference between weekly and monthly rankings?
A: The weekly ranking sorts by stars added this week and better reflects sustained attention. This article uses the weekly ranking.

Q: What is the difference between a Skill and an Agent service?
A: A Skill is a small functional module that does not need its own process. Its core is a single SKILL.md, which provides tools to the agent through a protocol. For work like design and auditing that does not need extra capability, a Skill is lighter. An Agent service is used when the task must connect to an external system independently.

Q: Why do these Skills cost so many tokens?
A: The Skill itself is text. The cost comes from the model runs it triggers. Design iteration that screenshots and fixes three problems, or a security audit that dispatches fresh agents per stage, multiplies inference calls. A heavy workflow can reach hundreds of thousands of tokens in one build, so the practical budget is set by the model pricing, not by the Skill install.

11. Summary

In the first week of November 2026, the GitHub weekly ranking put coding-agent Skills in the top three spots, and 13 of the 16 listed projects serve the agent ecosystem. "De-AI" design is the hottest direction. Impeccable, Hallmark and taste-skill represent three different ideas: a design vocabulary, a hard-bottom-line generator, and a "decide direction before building" flow. Cloudflare turned the seed of its internal vulnerability-hunting system into a six-stage audit Skill, which is the project most worth attention from engineering teams this week.

The ranking has a deeper implication. The spread of Skills is a shift in where value is created in the agent stack. Model quality still matters, but the marginal leverage is moving to the working methods, checklists and safety flows that surround the model. A design Skill that refuses to fabricate numbers, an audit Skill that re-checks every claim with a fresh agent, and a memory Skill that compresses tool calls into indexed observations are all examples of the same idea: the product is not the model, it is the disciplined process around the model.

Star counts were read on November 2, 2026. The hotlist changes daily, and install commands should follow the latest instructions in each repository README.

The Ultimate Tech Stack for Building an AI Agent in 2026

4sapi — Fri, 30 Oct 2026 08:34:45 +0000

Building an AI Agent is like cooking. You can have the best ingredients (LLMs), but if your kitchen (tech stack) is a mess, the final dish will be bad.

I've been iterating on AI Agents for months. Today, I'm sharing my "ultimate stack"—the tools I use for development, testing, deployment, and monitoring.

1. The Brain: LLM APIs

For Complex Reasoning: OpenAI's gpt-4o or Anthropic's claude-3.5-sonnet. (Don't use 3.5-turbo for agents, it fails on tool calling).
For Fast & Cheap Tasks: Llama 3 8B via Groq. The token generation speed is insane and perfect for simple routing tasks.

2. The Framework: LangGraph

As I mentioned in a previous article, I prefer LangGraph over standard LangChain. The state-machine model gives me the deterministic control I need so my agents don't spiral into infinite loops.

3. The Memory: Supabase + pgvector

Instead of setting up a whole new vector database, I use Supabase. It provides a Postgres database with vector extensions built-in. It handles both my user authentication and my RAG memory.

4. The Observability: Langfuse

You cannot debug an Agent using print() statements. Langfuse is an open-source tool that traces every LLM call, tool call, and prompt. It shows you exactly where the Agent failed and how many tokens you burned.

5. The API Router: OpenRouter / LiteLLM

If you are like me and constantly switch between GPT, Claude, and open-source models, don't write new code for each API. Use OpenRouter. One API key, hundreds of models. (Bonus: It handles rate limits gracefully).

6. The Evaluation: Ragas

How do you know if your Agent is getting better or worse? Ragas is an evaluation framework that tests your RAG pipelines and agents on metrics like "faithfulness" and "answer relevance".

The Stack Summarized:
User Input -> Next.js -> LangGraph (Orchestrator) -> Groq/OpenAI -> Supabase (Memory) -> Langfuse (Monitoring)

What does your stack look like? Am I missing any game-changing tools? Let me know below!

LangChain vs. CrewAI vs. AutoGen: Which AI Agent Framework Actually Works for Production?

4sapi — Thu, 29 Oct 2026 09:43:02 +0000

The AI Agent ecosystem is exploding. Every week there's a new framework claiming to be the "ultimate solution" for building autonomous agents.

As a developer, it's overwhelming. To cut through the noise, I spent the last two weeks building the exact same multi-agent workflow (a research assistant that searches the web, summarizes data, and writes a report) using the three most popular frameworks: LangChain, CrewAI, and AutoGen.

Here is my honest, unsponsored breakdown of what actually works and what falls apart when you try to push them to production.

The Test Case

Goal: Create a 3-agent workflow (Researcher -> Analyst -> Writer).
Tools needed: Web Search (Tavily), File Reader (Pandas).
Constraint: Must run locally using Llama 3 8B first, then fallback to GPT-4o.

1. LangChain (LangGraph)

LangChain has transitioned heavily into LangGraph for agents.

The Good: Complete control. You can define the exact state graph, loops, and conditional edges. If you want to build a complex, deterministic workflow, LangGraph is king.
The Bad: Steep learning curve. The documentation moves fast, and you'll often find yourself diving into the source code to understand why a node isn't firing.
Verdict: Best for complex, production-grade applications where you need to control every state transition.

2. CrewAI

CrewAI takes a role-playing approach (Manager, Researcher, Writer).

The Good: The fastest to get started. You define roles, goals, and backstories in pure Python, and it works out of the box. It's incredibly intuitive.
The Bad: Abstraction leakage. When agents fail to communicate, it's very hard to debug because the underlying prompts are dynamically generated and hidden from you.
Verdict: Great for rapid prototyping and simple sequential tasks. Harder to scale for highly complex, non-linear workflows.

3. AutoGen (Microsoft)

AutoGen focuses on conversational multi-agent systems.

The Good: Brilliant for tasks requiring continuous back-and-forth reasoning. The concept of GroupChat is powerful for simulating team dynamics.
The Bad: Prone to infinite loops. You have to strictly enforce termination conditions (like max_consecutive_auto_reply) or you'll burn through your API credits in minutes.
Verdict: Best for research and reasoning-heavy tasks, but requires careful guardrails.

The Verdict

If I had to ship a product tomorrow, I would build my base logic with LangGraph for its reliability, but use CrewAI for quick internal proof-of-concept tests.

What about you? Have you tried building with these frameworks? Which one do you trust for production? Let me know in the comments!

Building Reliable AI Agents: How I Fixed JSON Hallucinations in Tool Calling

4sapi — Wed, 28 Oct 2026 10:12:24 +0000

If you've built an AI Agent that uses external tools, you know the pain. You define a strict JSON schema for your tool, send the prompt, and the LLM returns something like this:

Here is the data you requested:

json
{
"action": "search",
"query": "OpenAI API limits"
}

I hope this helps!

markdown

And your Python json.loads() function just crashed.

LLMs are text predictors, not schema validators. When you ask them to output JSON, they love to add Markdown syntax, conversational filler, or even hallucinate extra parameters.

In this post, I’ll share the 3-layer approach I used to bring the Tool Calling accuracy of my Agent from ~70% to near 99%.

The Test: OpenAI vs. Claude vs. Llama 3

I ran a simple stress test with 50 complex nested parameters using raw prompts. Here's the failure rate of raw JSON output:

GPT-4o: 12% failure (mostly added Markdown backticks)
Claude 3.5 Sonnet: 15% failure (tended to add conversational text)
Llama 3 70B (via Groq): 45% failure (hallucinated keys, missing quotes)

To fix this, relying on a simple prompt like "Please output in JSON" is no longer enough. We need a structured engineering approach.

Layer 1: The System Prompt & Schema Validation

First, stop parsing with standard json.loads(). Use a library that can repair broken JSON or enforce schemas, like json_repair and pydantic.

Always pair your model with a strict Pydantic model:

from pydantic import BaseModel, Field
from typing import Literal

class AgentAction(BaseModel):
    tool_name: Literal["web_search", "database_query"] = Field(description="The name of the tool to use")
    parameters: dict = Field(description="The exact arguments for the tool")

# This ensures we strictly validate the output before feeding it to the next node.

Layer 2: Native Function Calling (Stop Parsing Text)

If you are still prompting models to "output JSON", you are doing it wrong. Use the native tool calling APIs provided by OpenAI, Anthropic, or Google.

Here is how it works with the OpenAI Python SDK:

import openai

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string"},
                "unit": {"type": "string", "enum": ["c", "f"]}
            },
            "required": ["location"]
        }
    }
}]

response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
    tool_choice="auto" 
)
# The API guarantees valid JSON structure in response.choices[0].message.tool_calls

Note: Even with native tool calling, models occasionally generate invalid enum values. Always catch the exception and pass it back to the model in a retry loop.

Layer 3: The Retry & "Self-Correction" Loop

When parsing fails, don't crash. Feed the error back to the LLM. This is the most powerful pattern in Agent engineering.

def execute_tool_with_retry(llm_response, max_retries=3):
    for attempt in range(max_retries):
        try:
            # Attempt to parse and validate
            parsed_action = AgentAction.model_validate_json(llm_response)
            return run_tool(parsed_action.tool_name, parsed_action.parameters)

        except Exception as e:
            error_msg = f"Your previous output caused an error: {str(e)}. Please fix the JSON format and try again."
            # Send the error back to the model
            llm_response = get_new_completion(error_msg) 

    raise Exception("Agent failed to generate valid tool calls after multiple attempts.")

The Result

By combining strict Pydantic validation, native Tool Calling, and a feedback retry loop, the Agent's execution reliability jumped from a frustrating 70% to virtually 99%. The key is to never trust the LLM blindly.

Building a reliable AI Agent requires treating the LLM not as a magic brain, but as an unreliable API that needs strict input/output sanitization.

Have you encountered JSON parsing errors in your Agent workflows? How do you handle model hallucinations when calling external tools? Let me know in the comments!

P.S. I'm currently documenting real-world performance and stress tests of various AI Agents on my personal blog. If you're building with LLMs, I'd love to connect and hear what metrics you care about most.

Your LLM Returned Valid JSON. Can Your App Trust It?

4sapi — Tue, 27 Oct 2026 02:28:44 +0000

Getting an LLM to return valid JSON feels like a milestone. The response parses, the fields exist, and the application can finally do something with the output.

But there’s another question: are those fields actually correct?

Consider a support assistant that turns a customer’s message into a structured request:

{
  "order_id": "ORD-1042",
  "action": "refund",
  "reason": "damaged_item"
}

Everything looks reasonable. Yet the customer might have written: “My item arrived damaged. Can I get a replacement? I don’t want a refund.”

The JSON is valid. The action is wrong.

Three checks worth separating

1. Can the application parse it?

This is the syntax check. Is the response valid JSON, or did it include extra text, broken quotes, or an incomplete object?

2. Does it match the expected structure?

This is the schema check. Are the required fields present? Are values the right types? Is action one of the allowed options?

A schema can reject an unknown action. It cannot, by itself, determine whether an allowed action matches the customer’s intent.

3. Is the result supported and permitted?

This is where application logic matters. Does the order belong to the authenticated customer? Did the customer request this action? Is the order eligible? Does the operation require confirmation?

These checks need information beyond the shape of the model’s response.

Separate interpretation from execution

For a workflow like this, I’d treat the model’s output as a proposed interpretation. The backend would check the order, apply the relevant rules, and ask for clarification when the request is ambiguous.

It also helps to give the model a way to express missing information. A required field with no “unknown” option creates pressure to fill in something, even when the source doesn’t support a confident answer.

For example, an incomplete request could go into a needs_clarification state instead of being forced into refund or replacement.

That state should have a real application path: ask a follow-up question, preserve the existing information, and wait before taking action.

A small test that reveals a lot

Try evaluating the same workflow with:

A clear request.
A request that explicitly rejects an action.
A message containing two different order numbers.
A request with a missing identifier.
A customer changing their mind halfway through the conversation.

Check both the extracted fields and what the application does next. A response can pass every formatting check and still lead to the wrong operation.

Structured output makes model responses easier to process. Deciding when to trust those responses remains part of the application’s design.

Have you encountered an LLM response that passed validation but was still wrong in a way that mattered? How did you catch it?