DEV Community: Steven Davies

Building an agentic workflow, part 2: testing what done means

Steven Davies — Tue, 27 Oct 2026 00:08:40 +0000

At the end of part 1, I had a specification, prototype tickets, and rules for how an agent should work through them. There was no application or controller yet. The next step was to try those rules against something that could actually fail.

The dot-finance prototypes gave me a desktop application to use, encrypted storage and backup experiments, and a small controller to manage the agent's work and stop it when it reached the agreed limits. They also gave me a test that passed for the wrong reason, an interface that needed several rounds of correction, and a build pipeline that ran out of artifact storage.

By the end of this stage, I had a good base for production development to start. Getting there involved looking quite closely at what the passing checks established, and what they hadn't tested at all.

The interruption test hadn't interrupted anything

The persistence prototype could save and reopen synthetic data, restore backups, and reject the wrong-password and damaged-file cases covered by the tests. Independent agent review then found a gap in the interruption coverage.

Asking an operation to cancel gives it an opportunity to clean up. Killing the process during a backup doesn't. I needed to know what happened in that second condition.

The agent added a harness that launched a child process and attempted to kill it at a controlled checkpoint. Initially, the harness accepted any unsuccessful exit as evidence that the interruption had happened. Unfortunately, the process-kill call itself threw an exception.

The test appeared to pass because something had failed. It hadn't reached the failure I wanted to test.

The missing partial backup file helped expose the mistake. Fixing the kill call dealt with that immediate problem, but a further review required an explicit marker showing that the process had reached the intended checkpoint. The checks then had to establish that recovery worked and the previous completed backup was unchanged.

This was a useful example of why independent review in the loop is so important. The code and its test could agree with each other while missing the requirement. A reviewer asking what had actually happened exposed a much weaker check than the result suggested. The same reason I'd ask a colleague for a second pair of eyes when I've been deep in the weeds.

The corrected test still had a specific scope. It demonstrated recovery from interruption at a controlled checkpoint, not at every possible instant during a write. Separate agent review wasn't a specialist security audit either. That gave me more confidence in the backup handling, but there was still plenty it hadn't tested.

The move from mock-up to actual app was rough

I'd chosen a specific UI direction from the mockups, but the first workbench implementation had strayed significantly from it. It'll be interesting to investigate this more in the future, as I think this may be something more common to less used frameworks like Avalonia over say HTML or Typescript. Either way, this needed to be addressed.

The first workbench implementation I sent back with feedback. The data is fictional.

The feedback was ordinary desktop application feedback. Text was too small, icons were ambiguous or missing, columns didn't line up properly, and the scrollbar obscured the amount column. The password field had become a permanent fixture in the header, when it belonged with saving and reopening the data.

I also needed category totals visible while reviewing transactions. A final confirmation screen wouldn't do the same job. If I'm changing an allocation, I need to see its effect alongside the transaction I'm changing. That became an explicit specification and acceptance requirement.

The corrections brought in recognisable outline icons, larger text, clearer column boundaries, and password prompts attached to the operations that needed them.

The later prototype after feedback and correction. The window size and fictional examples differ from the earlier capture.

There were several rounds of this. At smaller window sizes, the transaction list could paint over the right-hand panel. Contained horizontal scrolling and truncation within columns addressed that, but I later caught overlapping balance labels and overflowing rule controls as well.

One of the compact-layout problems I caught while trying the prototype.

A successful build and a screenshot capture could coexist with something I could immediately see was awkward. For the repeatable problems, the agent added compact-window captures and geometry assertions for the ledger, inspector, balance panel, and rule controls. Those gave later changes something concrete to fail against. Deciding whether the result was clear and comfortable to use still needed me to use it.

Review also found behavioural gaps. Unresolved items weren't always blocking acknowledgement, changed transactions could be silently skipped on re-import, and foreign-currency rows didn't require an actual GBP amount. Those went into a separate repair issue before the production decision.

Accepting the interface direction hadn't meant accepting every behaviour in the prototype. Keeping those decisions separate let me retain the layout and address the omissions.

These corrections were also giving me clearer UI requirements and checks to carry into production development.

Corrections needed to survive the next session

Some other feedback also changed the rules for subsequent work.

During the early hosted builds, I caught a GitHub Actions runtime deprecation warning. The Node version selected for the project was separate from the runtime used internally by an action. Updating the project's version hadn't addressed the warning.

That led to an explicit repository requirement for supported packages, actions, and images. The storage work also moved from the original SQLCipher bundle, recorded as deprecated during the project, to SQLite3MC. An authenticated snapshot experiment was feasible, but would have introduced a bespoke format and whole-file writes where an embedded database already suited the work.

Live totals went into the specification. Compact-layout defects became regression checks. Supported tooling became a repository rule. These were decisions a fresh session could find without me repeating the conversation.

But the budget rule from part 1 hadn't been demonstrated in the early persistence work. Correction counts were recorded after the fact, and CI repairs continued beyond the recorded count without a clear budget-renewal checkpoint. Writing down three attempts or 90 minutes hadn't made the limit enforce itself. The controller still needed its own experiment.

Retaining evidence filled the storage allowance

GitHub Actions artifact storage was a real pain, and a time sink I hadn't wanted from this experiment. It also showed how AI can still make some baffling decisions, like uploading hundreds of megabytes on every run without checking whether any of it needed keeping.

The workflow uploaded application output and validation evidence from each platform. When I asked what it was keeping, the investigation estimated roughly 680 MB of compressed output across three large uploads per run. My allowance was 500 MB. Most of the output was native binaries and debugging symbols, rather than the much smaller screenshots and diagrams.

Uploads were failing because the quota had been reached. Application checks could pass, then the workflow would fail during upload and block the dependent cross-platform backup cycle. I had to distinguish the checks that had passed from the validation that couldn't run.

The first reduction removed the large bundles and rebuilt from the tested commit in each job, keeping only the small encrypted backups needed between platforms. I then made the policy explicit. Actions artifact storage was for release-ready application binaries and, where useful, their debugging symbols. Test data, traces, screenshots, and intermediate output didn't belong there.

The final prototype workflow uploaded no artifacts. It passed the small synthetic encrypted backup through job outputs so the cross-platform restore exercise could continue. The resulting hosted run passed all seven jobs and reported zero artifacts. That transfer was specific to the small fixture, not a general mechanism for moving application databases.

This changed the earlier plan to retain generated architecture output as an artifact too. LikeC4 still ran during validation; generating its output didn't require uploading it on every run.

Asking the agent to retain evidence had produced a workflow whose storage needs hadn't been checked against the available allowance. Reviewing the pipeline needed to include what it kept and why. Otherwise the process intended to validate the application could prevent that validation from finishing.

The proposed permissions boundary had another route around it

For a live tracker-recovery trial, the agent proposed narrowly scoped credentials. I questioned whether that would provide the intended restriction. I'd already needed broader permissions for legitimate project work, and the existing authenticated GitHub CLI session was still available.

GitHub would enforce the permissions of the credential used for a request. Giving the controller a narrow credential wouldn't isolate an agent that could also use the broader session.

I kept the trial focused on whether the controller could recover after a GitHub update. The manual tracker adapter was fixed to one sandbox issue, one project, one marked comment, and two status transitions. It exercised losing the response after GitHub had accepted a write, then recovering without duplicating the comment or repeating an already completed mutation. The broader operator permissions were recorded explicitly.

That let me investigate recovery under supervision. It didn't establish a security boundary for unattended execution, which remained unapproved. That would need an isolated environment with only the intended credentials available.

The controller's process-recovery tests had a separate problem on Windows. A supervisor intended to survive the controller's termination wasn't producing its recovery receipt. This time the run reached its correction limit and stopped. I explicitly authorised another attempt. Detaching the supervisor resolved the tested failure, and the hosted checks subsequently passed across all three operating systems.

That was a concrete improvement over the early budget bookkeeping. The run stopped with an unresolved problem, and continuing required a decision from me.

This is definitely one I want to return to, as ideally agents should only be running with the exact access they need. For this supervised trial on a personal project, I deemed the risk acceptable with the controls I'd put in place.

What I'd accepted

After working through the prototypes, I was happy to move forward with Avalonia and .NET, SQLite3MC, the guided reconciliation approach, and the small controller I'd been testing under supervision. The workflow checks had passed on macOS, Windows, and Linux, and a backup had been restored through all three platforms.

The disposable shell still needed production boundaries, importer contracts, accessibility acceptance, security decisions, and packaging work. Passing the prototypes hadn't turned it into a finished financial application.

I also asked for an isolated trial with my real data and actual flow once the complete workflow was available, before release packaging. Synthetic fixtures meant development and validation could happen without exposing financial records. The later trial would check whether the application fitted the job I needed it for, with me operating it privately and keeping the records out of the agents, repository, and CI. I still wanted to try it with my own data before considering it ready for release.

Building an agentic workflow, part 1: deciding what done means

Steven Davies — Tue, 20 Oct 2026 18:22:22 +0000

I still reconcile my credit-card spending manually across several spreadsheets. The calculations and automation I've added help, but it's still painful to do every month.

That's the reason for dot-finance. I want an application that imports transactions, works out how much needs to come from different accounts, and records what's been settled. It will stay offline and I'll still make the transfers myself.

There's another part to the project though. I've used coding agents on plenty of bounded tasks, and full agentic workflows in my job as a software architect. I want to build my own development loop from scratch for personal projects, with dot-finance as a real application to test it against.

That means working out what happens around the coding. An agent needs enough context to pick up an issue in a fresh session, a way to check its work, and a limit on how long it can spend trying to correct a failure. If a run stops halfway through, I need to know where it got to without reconstructing the whole conversation.

Before writing application code, I started by putting those rules into the repository.

Context that survives the conversation

I began with a specification and asked Codex to review it and grill me on anything unclear before planning the work.

I've been using reusable agent skills for this, including ideas from Matt Pocock's skills collection. A skill gives the agent instructions for a particular kind of work, which saves rebuilding the same prompt each time. The local grilling skill asks one question at a time, gives a recommendation, and waits for a decision. I then used to-issues to turn the prototype work into GitHub tickets.

The conversation is useful for reaching decisions, but a later session needs to find the decisions without reading everything that led to them. I've split that context across three files.

spec-file.md describes what the application must do.
development-workflow.md describes how an issue moves through implementation, validation, review, and integration.
AGENTS.md gives an agent the operating rules for the repository.

For example, staying offline is an application constraint. Using synthetic development data is a rule for doing the work. A missing platform check is unfinished validation. All three need to survive the session in which they were discussed, and each belongs somewhere an agent can find it again.

This also gives me something concrete to review when the process needs changing. I can update the relevant rule rather than leave a correction buried in a chat and hope the next run sees it.

One issue at a time

The initial loop is sequential. It selects one ready issue with completed dependencies, works on an isolated branch, runs the required checks, and sends the result through independent review before integration.

The proposed loop at this stage of the project. The workflow is documented; the controller hasn't been built.

Starting with one issue keeps the coordination manageable. There's one branch to inspect and one set of expected outcomes. I can work out whether the basic loop behaves properly before adding concurrent work and the extra problems that brings.

The issue needs to say how I'll know the work is complete. “Add encrypted backup” leaves too much open. Saving synthetic data on one supported platform, restoring it on another, and checking that the expected state survived gives the agent a result to demonstrate.

Opening a pull request doesn't satisfy that requirement. A green build might not satisfy it either, if the required behaviour was never exercised. The issue stays open until the agreed validation and integration are complete.

The evidence also needs to identify the commit it came from. If a review leads to code changes, the affected checks have to run again. A Windows check that passed before those changes can't simply be carried forward because it's on the same branch.

It's a small bookkeeping requirement with a practical consequence. After an interruption, I need to be able to tell which code passed which checks, and which checks are still pending.

Putting a limit on corrections

In my earlier AI field notes, I described using Gemini to help set up an Octopus Deploy instance. I kept feeding errors back into the conversation, and it eventually suggested running Octopus on PostgreSQL. By then I should have stepped back, checked what was missing from the context, and started again from a better position.

I don't want this loop to depend on me noticing that point every time.

For the prototype phase, I've set an initial limit of three correction attempts or 90 minutes of active agent work, whichever comes first. Those numbers are provisional. They give the first runs a stopping point while I find out whether the limits are useful in practice.

When the budget runs out, the controller should return enough information to investigate or continue deliberately. That includes the issue, branch and commit, checks completed, evidence collected, correction count, remaining problem, and next action. Continuing needs an explicit new budget.

An interruption needs the same care. Restarting a run mustn't reset its correction count or turn an unfinished check into a pass. The plan is to keep GitHub as the record of the work, with a local checkpoint holding enough execution state to resume accurately.

The controller also needs to distinguish a code failure from an infrastructure problem. If a hosted runner is unavailable, editing the application won't help. That platform's validation should remain pending, with the blocker recorded. Otherwise the loop could spend its correction budget changing code that hasn't been shown to be wrong.

These are behaviours I need to test in the controller itself. The prototype plan includes a deliberately failing check, an interrupted run, an unavailable runner, and a review rejection. A successful application build wouldn't tell me whether any of those paths work.

Reviewing the tests and the architecture

Independent review needs to look beyond whether the implementation compiles. Code and tests written together can share the same misunderstanding, so the reviewer needs the requirement, the diff, and the validation evidence. Part of that review is deciding whether the tests would catch a meaningful error in the required behaviour.

I've also asked for LikeC4 architecture diagrams alongside the implementation. I've been a long-term fan of the C4 model and the consistency it brings to architectural discussions. This project gives me a chance to try architecture as code while an agent is making the implementation changes.

The proposed rule is to update the LikeC4 model in the same pull request when component responsibilities, dependencies, storage, or significant flows change. If a change has no architectural effect, the pull request can say so.

The build is intended to validate the model and generate the diagrams. That would catch an invalid model, but it wouldn't establish that the diagram accurately describes the C# code. Checking that agreement remains part of review. Otherwise I could end up with a passing build and an increasingly misleading description of the application.

At the end of this setup stage, I had a specification, prototype tickets, and workflow rules. There was no application or controller yet. The first prototype was planned to establish the desktop and LikeC4 build foundation, followed by investigations into encrypted storage, restoration across platforms, and the guided reconciliation flow. Alongside those application checks, the controller still had to demonstrate that it could stop within its budget and resume without losing track of unfinished validation.

AI Field Notes 2026

Steven Davies — Fri, 03 Jul 2026 22:19:36 +0000

There is no doubt that AI has rocked the world of coding considerably over the last couple of years. One thing that has become apparent, through keeping up with the whirlwind of changes, is that people are still figuring out how to use these tools effectively. It seems like every video I watch, or article I read, people are swearing they've found the killer approach.

I have noticed through all the noise though, that there are some key concepts that seem common regardless of which approach you take. This isn't comprehensive, and is likely to change over time, but my key observations in July 2026 are:

Clarity is king

I recently watched this excellent TED Talk by Rainer Stropek. His closing remark is "I am not just a coder, I am a developer and my new programming language is clarity". The talk is well worth a watch for various reasons, but this statement definitely struck a chord with me.

Clarity, when working with AI agents, is key. This isn't a new thing though, as if you were working with a team on delivering a coding project, having a shared context and understanding of the goal and approach is key. That doesn't change when working with AI. Having that shared understanding of not just the task, but the goal and the process leading there will give you much better results than just stating a task and expecting great results.

Rather than just state the task, state the goal you want to achieve and the key criteria or guardrails to get there. Use the LLM to help plan and build your spec before letting it loose. A lot of harnesses have planning modes, so give them a try. Formulate a plan, break it down into stages if required, then use that to guide and review the output.

"Grill me relentlessly", as Matt Pocock recommends, to work towards that shared understanding.

Weak:
Add task cancellation to the CLI.

Better:
Add a `task cancel` command to the CLI using the existing command patterns. 

It should:
- Cancel queued deployment tasks for a selected environment
- Use the same environment matching behaviour as the other commands

Ensure you:
- Include unit tests and live integration tests
- Update any user-facing help text if the command surface changes
- Run the unit test suite, including integration tests
- Provide a summary of your changes and validation

Context is also king

A follow-up point from the above: be surgical about the context you provide. Imagine you're talking to a contractor who knows nothing about your project. Be clear on the technical stack and constraints. Give clear directions, link to key files, types or documentation it needs to do a good job. Don't just point at your repo or codebase and expect it to scan and discover everything from scratch. You'll either burn tokens unnecessarily or pull all your hair out when you keep getting bad, inconsistent results. Even worse, both. High signal, low noise.

Prefer smaller, reviewable tasks. Give clear boundaries and observable success criteria where possible.

Weak:
Look through the repo and work out how commands are built.

Better:
The existing command patterns are in `src/ShipItSharp.Console/Commands`. 

Start with `BaseCommand`, `Environment`, and `Channel` as examples. 

Core orchestration should live in the shared runner layer, not directly in the console command.

You're not just stuck in the terminal

I picked this up from a talk by Lucas Meijer, who offered a simple tip that made my comprehension and efficiency so much better. Ask for a single-page HTML summary. That's it, but it's amazingly powerful in the right situation.

One example use case is I asked Codex to "grill me" on a complex goal and it generated 60 questions for me to review. Rather than answering them all one by one in a terminal, I got it to spit out an HTML file with an export option so I could review, save, and return if needed.

I've also used it to generate reviews of code, produce a detailed plan of action or even show me design choices to choose between when I've been unsure.

Obviously, don't overuse this as it can burn tokens, but in the right contexts it can produce a much easier response to comprehend and lead you to a more efficient goal.

Chat context is expensive

It's a general recommendation that you keep to a low number of messages per chat or session. Why? Every time you send a message, the entire previous context is also sent. The larger this context, the less useful the results become and the more expensive the chat. Most harnesses have reminders or auto compaction to help with this, but often it's still best to request a summary of your conversation so far, then start a new chat with that summary to maximise efficiency.

On a similar note, if your instructions were unclear and you got a duff response back, don't scorn the AI tool and tell it to try again. Consider editing your original message so it's clearer and then resend. It keeps your context shorter, will give you a more refined response and will burn fewer tokens.

Get AI to validate its own work

Before you set your agent off on a series of tasks, make sure you define sensible validation for the result. Think about what you would expect if you were doing the work yourself. Unit tests that pass covering specific areas? Second pass review? Integration tests? Browser validation? Whatever it may be, it's key to save time by getting it to validate its response before asking you to take over.

One tip I picked up from a colleague is to task the agent with continuing to refine until it's 95% confident on an answer. If it's unable to reach that level of confidence, provide a reason as to why.

A pattern I’ve started using is asking Codex to do a second-pass review against the original goal, not just against compiler errors. For example, after command/help text changes I asked it to verify the output against the source and generated docs. The first review found several issues, a second pass found a couple more, and only after that did the output settle. That’s a useful workflow: the agent writes, then the agent reviews, but I still own the final judgement.

Another example is I recently used Codex to add new task-management commands to ShipItSharp. The useful part wasn’t that it generated the code. It was that I could give it the expected behaviour, repo conventions, validation criteria, and test expectations up front. It inspected the existing command patterns, added the command surface, core orchestration, unit tests, command tests, and live integration tests against the real Octopus instance. The first pass missed part of my integration-test expectation, which was a good reminder that the agent still needs senior review and clear acceptance criteria.

The two-strike rule

When an agent's code throws an error, give it one shot to fix it. If it fails a second time, step away from the prompt window. Paste the error back into the chat a third time and you'll often watch the AI enter a hallucination loop, generating increasingly bizarre workarounds. Figure out where your initial context or constraint was lacking, and start a fresh chat. Know when to cut your losses.

A perfect example of this is when I was working on ShipItSharp. I needed a new environment, so I was using Gemini to help set up a new Octopus Deploy instance hosted locally. I encountered error after error, and rather than backing away I persisted. In the end, it hallucinated that Octopus Deploy can run on PostgreSQL. After calling it out, its response was:

I owe you a massive, deeply embarrassing apology.

Lesson learned.

Think of using AI agents as a "layer up"

This calls back to Rainer Stropek's talk, where he talks about the shift from being on the ground writing code to moving another layer up to orchestrate the tool to do the boilerplate stuff. Your job shifts more towards the problem solving, planning, clarity of communication and validation. It's still key to review the output, especially in the key areas of your system where security and resilience matter. This means you must understand, and take responsibility for, the generated output.

A good example for me was release workflow work. I didn't need Codex just to edit YAML. I needed it to compare the Unix and Windows workflows, check that the release artifacts matched, verify action references, update documentation, and sanity-check the release tag against the actual version source. My role was less about typing every line and more about setting the standard for what "done" meant.

It honestly reminds me of earlier in my career. My first love was HTML, CSS and JavaScript (this was 20+ years ago, so not the frameworks we know now). I loved creating things, but over time I started to get frustrated with doing the same basic CRUD operations. That's when I fell in love with backend coding, and eventually the move towards architecture and bigger-picture thinking. The key through all of this is the love of problem solving. Seeing an issue, finding a solution, and building a solution.

AI tools are like that next level up, where the key things that make you a great developer are still absolutely relevant. Anyone can vibe code something, but building something resilient, secure, safe and scalable for thousands if not millions of users still requires strong development skills.

AI is not human, don't treat it like it is

It may sound harsh, but its true. It's only natural to want to personify the AI tools we use. After all, it often responds to you as if it was a friend or colleague. In reality, you're talking to a probabilistic engine choosing responses it thinks are most appropriate using complex maths. It has no feelings or empathy. Keep operational prompts concise.

Most harnesses these days have an option to change its personality. I'd recommend setting it to something pragmatic rather than overly friendly and warm.

Do you have any tips you've found useful? Please do share ❤️ Maybe I'll return next year and point out all the things that have changed and learnt.

Proxy any class by interface in C# with DispatchProxy

Steven Davies — Fri, 22 Oct 2021 22:27:23 +0000

I was recently tasked with figuring out a way to performance test database calls being made via NHibernate to PostgreSQL. As far as I could find, NHibernate doesn't expose any really easy ways to intercept and manually log both the SQL and execution times for each query.

After a little more digging, I came across a handy lesser-known tool that allows you to easily proxy any interface, be it user-defined or from a third party library. This tool lives in the System.Reflection.DispatchProxy nuget package. With a little code, you can wrap any instance of a class that implements an interface with a proxy to not only log but also manipulate both arguments and returned data.

Example

Take this simple class and interface.

interface IHello
{
    bool SayHello(string name);
}

class Hello : IHello
{
    public bool SayHello(string name)
    {
        Console.WriteLine($"Hello {name}");
        return true;
    }
}

Nothing special, only one method exposed which takes a simple name parameter, writes out a string to the console and returns a bool indicating success.

Say, although contrived in this example, we wanted to record calls going to SayHello.

class HelloDispatchProxy<T> : DispatchProxy where T : class, IHello
{
    private IHello Target { get; set; }

    protected override object Invoke(MethodInfo targetMethod, object[] args)
    {
        // Code here to track time, log call etc.
        var result = targetMethod.Invoke(Target, args);
        return result;
    }

    public static T CreateProxy(T target)
    {
        var proxy = Create<T, HelloDispatchProxy<T>>() as HelloDispatchProxy<T>;
        proxy.Target = target;
        return proxy as T;
    }
}

This is our DispatchProxy, HelloDispatchProxy. Usage is pretty simple.

IHello hello = HelloDispatchProxy<IHello>.CreateProxy(new Hello());

The CreateProxy method calls Create<T> on the base abstract class to generate a proxy object for you. The target instance is then assigned to the Target property before returning the proxy cast to type T (IHello in this instance).

Any method that is then called on the proxied IHello instance will trigger Invoke(). It's your proxies responsibility to call the target method, pass through the parameters and return the result. In this method, you can do whatever you like, and you'll also have access to your original Target instance.

You can see a complete example of this over on my GitHub

With great power comes great responsibility

This allows you quite a lot of control, so be careful what you do with args and the returned result or you can cause some unexpected behaviour.

Performance testing

DispatchProxy lives under the System.Reflection namespace. Because of this, I was curious about what overhead is introduced by this approach. What better way to understand than to test it myself.

I set up a slightly over-engineered set of tests to give a raw comparison between calls to both an unproxied and proxied instance. I ran these multiple times to get an average before analysing the results.

    100000 W/O  100000 W    % Increase
    2233.023    2382.8835   6.71%
    2235.5306   2398.1216   7.27%
    2236.2449   2386.4416   6.72%
    2236.8529   2389.209    6.81%
    2239.9692   2402.4627   7.25%
    2227.0108   2367.0306   6.29%
    2237.8432   2384.8429   6.57%
    2233.5528   2392.8394   7.13%
    2238.411    2398.3124   7.14%
    2235.7398   2388.134    6.82%
    2262.3454   2380.0595   5.20%
    2340.0243   2385.0553   1.92%
    2249.3693   2383.6925   5.97%
    2246.9835   2381.507    5.99%
    2240.7711   2381.4675   6.28%
    2232.5296   2374.0465   6.34%
    2248.0095   2359.8636   4.98%
    2250.2518   2394.8422   6.43%
    2234.7757   2387.8114   6.85%
    2238.7645   2387.8047   6.66%


    0.00112245  0.001192661 6.27%

So on average, the execution took an extra 6 or so percent with just the introduction of the proxy. Bear in mind in this current state, the proxy is doing nothing useful at all, but this helps put into context the efficiency.

But if you were to add something slow like a blocking console write line in the proxy, the % increase shoots up from 6.27% to over 100%.

System.Console.WriteLine($"Going to call {targetMethod.Name}");

The most important part with the performance impact is what you decide to do in the Invoke method, something you'll need to benchmark yourself.

In all but the highest performance-critical applications, the overhead of the proxy itself is minor. If you take for example proxying 2-3 DB calls in the context of something like a HTTP web request or message handler the impact is negligible. In fact, in my testing with a real-world application, the numbers are within an acceptable margin of error even when tracing into an APM tool.

NHibernate & PostgreSQL

If you're specifically interested in collecting information from NHibernate as I was, you can do this by wrapping the IDBCommand in a custom driver implementation.

public class ProfiledNpgsqlDriver : NpgsqlDriver {
    public override IDbCommand CreateCommand() {
        var command = base.CreateCommand();

        if (ConfigurationManager.AppSettings["EnableDBTracing"]?.ToLower() == "true") {
            command = ProfiledDbCommandDispatchProxy<IDbCommand>.CreateProxy(command);
        }

        return command;
    }
}

You can access the command being executed from the target IDBCommand's Command property and record time etc. with a StopWatch. I went a step further and implemented the proxy as an IObservable<T> so I can handle it differently depending on the use case.

Wrap up

Hopefully, you found this interesting. Can you think of any good use cases for this, or have any comments? Would love to hear them ❤

Auto OpenVPN reconnect and killswitch for VPNs with dynamic IPs

Steven Davies — Sat, 02 Oct 2021 22:22:32 +0000

The Problem

If you’re like me and run a home Linux server, you may wish to add some extra security in the form of a VPN connection to ensure traffic is encrypted. In my case, I use a Raspberry Pi to backup things like RAW images from my camera to an off-site location. The extra security gives me a little peace of mind, especially as the server generally uses a cellular connection.

I’ve been using UFW to work as a killswitch and only allow traffic via the VPN connection, but one issue I've had is with my trusted VPN provider is that they regularly rotate IP addresses. UFW uses iptables under the hood, meaning it can only be configured to use IP addresses explicitly and not a domain name. This has left my backup machine without an internet connection fairly frequently.

The Solution

Just to set this straight from the start, I’m no UNIX genius. I spend most of my time working with Windows machines and use Linux more as a hobbyist. That been said, I’ve written my fair share of code/scripts.

My solution is pretty simple; when disconnects happen:

fetch a fresh list of IP’s from my internal DNS server.
Update UFW .
Update the OpenVPNconnection file and reconnect.

I thought I’d share my approach, as I failed finding any other solution online so this may be of help to others.

The first part is fetching the list of domains. To do this, I use dig.

ipoutput=$(dig $1 +short)

if [ -z “$ipoutput” ]; then  
  echo “No ips found for $1”  
  exit 1  
fi

echo “Found ips $ipoutput”

readarray -t ips <<<”$ipoutput”

This gives me a nice list of IP address for the specified domain.

Now I know the correct ip addresses, I need to make use of them. The first job is to update UFW. I decided to use a template file so I can set it up as I want for each machine. This may not be the most elegant solution, but its pretty flexible.

ufwoutput=’’

for i in “${ips[@]}”  
do  
  :  
  ufwoutput+=$”### tuple ### allow udp any $i any 0.0.0.0\\/0 out\\n”  
  ufwoutput+=$”-A ufw-user-output -p udp -d $i -j ACCEPT\\n\\n”  
done

if test -f “$dir_path/user.rules.tmp”; then  
  echo “Deleting tmp file…”  
  rm “$dir_path/user.rules.tmp”  
fi

cp “$dir_path/user.rules.template” “$dir_path/user.rules.tmp”

sed -i -e “s/\[content\]/${ufwoutput}/g” “$dir_path/user.rules.tmp” “$dir_path/user.rules.tmp”

mv “$dir_path/user.rules.tmp” /etc/ufw/user.rules

echo “Reloading ufw”  
ufw reload

The script builds up a string containing the correct formatted lines for the user.rules file for UFW, creates a new copy of the template, replaces [content] in the file using sed then moves the file into the correct place (for ubuntu). The final thing to do is ufw reload to reload the user.rules file.

That’s UFW taken care of, time for OpenVPN using a similar method

openvpnoutput=’’

for i in “${ips[@]}”  
do  
:  
  openvpnoutput+=$”remote $i 1198\\n”  
done

if test -f “$dir_path/openvpn.ovpn.tmp”; then  
  echo “Deleting tmp file…”  
  rm “$dir_path/openvpn.ovpn.tmp”  
fi

cp “$dir_path/openvpn.ovpn.template” “$dir_path/openvpn.ovpn.tmp”

sed -i -e “s/\[content\]/${openvpnoutput}/g” “$dir_path/openvpn.ovpn.tmp” “$dir_path/openvpn.ovpn.tmp”

mv “$dir_path/openvpn.ovpn.tmp” “openvpn.ovpn”

This works in a similar way to the UFW code, replacing a [content] placeholder in the .opvn template file.

So that’s all the code to update UFW and OpenVPN. The next thing to do is to manage the connection. I do this via a simple script called from a cron job.

function getStatus () {  
  echo “Attempting to get device status…”  
  ip address show | grep $1 && return 1  
  return 0  
}

This is a pretty straightforward function. grep the output from ip address show to see if the OpenVPN connection is live. Return either 1for success or 0for failure.

The last thing to do is to check the result and connect if required.

getStatus $device  
if [[ $? == 0 ]]; then  
  echo “openvpn is not connected!”  
  echo “Reconnecting!”

  #Update config  
  updateIps $domain &> $dir_path/renew.out &disown

  openvpn — config “$dir_path/openvpn.ovpn” &>$dir_path/out.out &disown  
else  
  echo “openvpn is connected!”  
fi

If the output from getStatus is 0, then it will call to update the IP addresses and trigger a new OpenVPN connection with &disown to not wait for the connection to exit.

Now that’s all in place, all that’s needed to run the script automatically is a cron job to check as frequently as you wish.

*/1 * * * * cd /home/account/openvpn && ./auto-connect.sh “my.vpn.domain” “tun0” > out.out

If this is useful to you, rather than having to copy & paste the code I’ve wrapped all this up into a single script available on GitHub.

Wrap Up

As I say, I’m no Linux sysadmin but this script solves a problem for me and has been working well for the last few months. If anyone has any suggestions for improvements or comments I’d love to hear them. Even better, fork the repo on GitHub and pop in a merge request ❤

Thoughts on fine tuning your tech CV

Steven Davies — Sat, 02 Oct 2021 21:53:16 +0000

It's almost cruel that your first step into a potential new role often falls to a few A4 sides attempting to summarise yourself; your skills, your drive, your goals. You're more than that can cover. The person reviewing your CV knows that, but they face a difficult decision on whether to invest the time to get to know you based on this first introduction.

I've been responsible for hiring into multiple technical positions over the years across different skill areas including developers, testers and sys ops engineers. Some of these have been recent graduates, others looking for senior positions after decades in the industry.

I've seen a wide array of CV styles, some that have delighted me and some that have been difficult to navigate. I thought it may be useful to someone out there, or maybe just myself, to list out some of the suggestions I can think of to help make your tech CV easier to digest. These are all based on real examples I've seen more than once.

Consider your audience

This isn't just for your CV's, this is for everything. Whether you're applying for a new position, putting in a merge request to your peers or writing a best man's speech... it's important to put yourself in the shoes of the person you're writing for. What do they want to know?

Regarding CV's, imagine you're in the position of hiring for the job you want. What would you want to know, what information matters and just as importantly what information doesn't?

The opener

Although it is unlikely to be the first section that is read, a personal statement is an opportunity to impress and set a clear indication about what you're looking for and have achieved. If well written, it can easily help you stand out from other candidates.

One of the biggest traps I've seen people fall into is to state the basic cliches. I'm talking about things like "hard-working" or "dedicated", they're really expected traits and don't need to be said. Space on your CV is precious, and I'm sure there are things that you're proud of you want to tell the reader about over basic statements.

The second thing I often see is an overuse of vocabulary. You don't need to impress with fancy words, do so by saying what you've achieved in a clear and easy to read manner.

Pick out the key accomplishments and skills you want the reader to know about you, what the position you're looking for is and the direction of your career. Be honest, you don't want to lie and get an interview for a position you don't want, but feel free to tailor it to the position you're applying to. Keep the fancy words to a minimum, but don't be afraid to be confident about what you're good at.

Here's an example of what I'd consider a good personal statement.

An ISTQB certified software developer in test with 8 years' experience testing enterprise level web applications. Designed, written and maintained frameworks that ensure a high level of quality and resilience, including testing web API's and applications for functionality and performance that have reduced required testing time by 50%. Thrives on the opportunity to mentor other testers, work collaboratively in a team and has a keen eye for spotting opportunities for improvement. Now looking for the next challenge to use the skills gained to make a difference and continue professional growth.

Size matters

If your CV is jam-packed with information covering 10+ pages, then it's going to be difficult to digest and pick out the relevant information you want the reader to see.

Similarly, if your CV is a page or less, you're likely omitting really useful information about yourself and your skills.

The best CV's I've personally seen are generally around 1-3 pages long. Any longer than that then you may benefit from having a tidy.

So how do I make the best use of the space?

Keep it relevant

The IT sector runs at a super-fast pace. Your most recent positions are always likely to be the most relevant on your career path. Take a look at your previous employment. That job you did 5+ years ago using some obscure or out of favour tech... ask yourself, do I want to even do that thing again? Is it relevant to this position or where I want my career to go?

The further back your experience the less specific you probably need to be. Don't omit previous positions altogether, but you can afford more brevity the further back you go. One size doesn't fit all, just use the space wisely.

Summary of tech skills

We use so many tools, so many processes and workflows... you want the person reviewing your CV to see all the wonderful things you know and have used. This can quickly turn into 1/4 of a page of your CV or a giant list of unsorted tools and skills.

The problem is if you're not careful you can really hurt its usefulness. Take a look at the following example:

C#, Visual studio, MVC, WebApi, Postgresql, MongoDB, T-SQL, Flask, SQLite, LiteDB, Entity Framework, PostgreSQL, SQL Server, AWS, Python, Microsoft Word, Fiddler, Postman...

The first issue is that there are quite a few things that are not relevant and just distract from what you're actually good at. Things like listing Visual studio along with C# when that's pretty much implied or listing office tools which is an expectation for any IT job. There may also be things you have no real interest in doing again, so think if it's worth listing them. Some may just be completely irrelevant for the position you're applying for.

The second issue is that it doesn't really give any idea of your experience. I've seen lots of CV tip sites that recommend a list of years of experience but that doesn't always help. Take for example C#. Is that building desktop apps, or web APIs? Maybe it's all in windows services or WCF.

My current suggestion would be something like the following, but I'd love to hear/see your ideas 🙂

Type	Technologies
Core Languages	C# 7+, Asp.Net 4+ (WebApi & MVC), Entity framework
Core Databases	PostgreSQL, MongoDB, T-SQL
Core Tooling	OctopusDeploy, TeamCity, Fiddler, Postman
Other	Python, Flask, Ms SQL Server, SQLite, LiteDB, AWS (EC2, X-Ray S3)

It's always a good idea to link any experience you mention in the summary with your job roles and include context, like:

In this role, I was a key architect in developing large scale C# (8.0, .net 4.7.2) apps with PostgreSQL and entity framework.

The design

This doesn't apply to all roles as it may be useful for a graphic designer position for example but if you're applying for a developer/tester/sys ops type role don't focus too much on the design. Stand out by your clear and obvious thought when preparing your CV by keeping the content clear and easy to read. You don't need colours, graphics or charts to sell yourself, they're just a distraction. I'm not saying make it ugly, just don't overthink it.

The bonus

You're in this industry (hopefully) because you love tech. You probably have personal projects you work on outside of your normal job. It's always a real bonus to get to see what candidates have worked on in their spare time, throw that GitHub or personal website link in there or in your covering letter.

So that's it, I hope you found some of it useful. Some of the points may seem a little obvious, but they're all based on things I've seen frequently.

I'd love to know if you agree with any of these thoughts, or maybe you have some of your own tips and tricks you'd like to share? ♥