I think we’re facing a pivotal moment. We’ve all been enjoying frontier model access at prices that don’t reflect the underlying cost structure, because VC money was paying the difference. That’s about to change1. The capex is real, the providers are burning money to acquire us as users, and that can’t continue indefinitely.

Because of that, I think we are all going to be learning how to use LLMs more effectively. I can see three levers:

1. Routing. We are starting to see this, but still, as of right now, model selection is more or less a config option. The big-model-for-everything pattern is wasteful: a subtask that parses a git commit message and pipes it back to an orchestrator does not need a frontier model. Smaller models aren't just cheaper either; they're faster and have lower latency variance, which compounds in agentic loops where you're chaining many calls. I expect high-granularity routing, between model sizes, between providers, and between hosted and local, to become a natural part of any coding tool.

2. Caching. My intuition right now is that by far the most tokens are wasted by re-scanning the same parts of the code base when starting new tasks. I know that Claude Code does some caching internally, but I am thinking along the lines of “manual caching” by adding English prose in the form of comments and markdown files everywhere. I go into some details about this in The Robot is not a Junior Developer. It’s a Senior Developer caught in Groundhog Day. You can see how I do this in practice in the root AGENTS.md file. It’s also why I recently upgraded Kanel to support generating markdown documentation of a PostgreSQL database, so the canonical description of the schema exists once, in a stable form.

3. Trimming. Tools like rtk help optimize how many tokens are spent on output from tool calls. Speaking like a caveman can apparently help, though I haven’t tried that myself. And this is the lever txtfold pulls: reducing large bodies of text to something a model can actually work with.

When I started creating txtfold, I had a vision of compression in mind. I'd hit two situations that motivated it: a pile of large log files, and a 40MB JSON file. I knew that they would contain mostly repetitive data, with a few interesting outliers. I also knew that they were too large for the context window of any LLM, so even though I could just tell one to look at them, I didn’t trust what they would actually do.

That led me to thinking about something like RLE or LZW, but producing text. Such algorithms are, after all, designed exactly for retaining important information while contracting repetition. This turned out to be a dead end because the output was still very verbose, and the losslessness wasn't buying anything useful.

So I went the other way: lossy but deterministic reduction. The important bit here is that there is no AI inside txtfold trying to assess what is important and what is not. It’s all purely algorithmic. Instead, you can let your LLM play with it and configuration options until you get a useful summary out. Once you have such a config (“recipe”), you can apply it repeatedly to new text from the same source and get consistent results.

That determinism matters. An LLM-based summarizer is a black box you have to re-trust on every input. A txtfold recipe is a thing you can read, version, and reason about.

txtfold is a small bet on one of three levers. The other two need their own tools, many of which already exist. If you're building in this space, I'd love to hear about it.

https://kristiandupont.github.io/txtfold/

It comes with a new built-in generator for Markdown. This can be used to document your data model for your fellow team members, but the primary use case that I picture is LLM consumption. I’ve used an MCP server for my developer database which does work, but it feels slightly clumsy to me, and including a markdown with with an @-tag when pointing something out feels very natural.

I’ve created an example for the dvd-rental database. You can see the result here.

Even if you are using an ORM that mirrors the database schema in some way, those files often miss some of the finer nuances that are possible with Postgres. With Kanel, you can construct documentation that goes into just about as much detail as you could want. The engine has access to everything that extract-pg-schema exposes, which allows you to document indices, procedures, ranges, domains and most other Postgres-specific features.

Setting it up requires a bit of configuration work, as I don’t think I would be able to write a general-purpose template that would suit all teams. It works with Handlebars templates which will be used to generate the final result.

I don’t think that is a great analogy any more. At this point, Claude (my preferred provider at the moment) feels much more like a talented senior developer. But crucially, a senior developer who is experiencing a perpetual day 1 on the team.

There are two ways of getting better and more specific output from an LLM: fine-tuning and retrieval-augmented generation (RAG).

Fine-tuning is like making a student study a curriculum for a long time. Over time, they internalize a worldview. Certain assumptions simply become “how things are.”

RAG is more like allowing the student to bring notes during an exam.

In one sense, the former is much more powerful. But in many real situations, the latter is more effective. If the question is narrow, and the relevant piece of knowledge is very specific, it often matters more that the right note is present in front of the student than that the student has spent years internalizing the entire curriculum.

Scanning a codebase, reading READMEs, and opening individual files is exactly this kind of exam-time context injection. When an AI does it, that is RAG, but obviously humans do it as well.

When an experienced engineer is introduced to a code base, their first task is not writing code. It is reconstructing local reality. They scan the architecture and attempt to infer conventions. But even after doing that, their first PR’s will need course-correcting by the other team members. They will have assumed that standards are followed and that visible patterns are intentional, both of which are frequently wrong.

Standards will not be followed because the developer who started something didn’t know about them or maybe they did but they decided to be clever, for good or bad reasons. And even clear patterns in the code base may reflect past decisions that were revised just last week.

The real rules live in commit history, Slack or email threads, half-remembered discussions, and the team’s collective memory. A human absorbs this over time. An AI doesn’t. Every session, it wakes up on day one. Now, in theory, fine-tuning would be the equivalent here, but at least with our current tooling, this isn’t an option.

AI-Assisted Programming Is Not Delegation. It’s Context Construction.

Given this perspective, how can we best work with LLM’s as software developers?

Well, we need to give the robot as much help as possible which means, somewhat paradoxically, adding more English to our code: comments, READMEs (I’m considering putting one in every folder), agent files with detailed descriptions.1 This creates an “exoskeleton” for the LLM to work with. Ask it to keep this up to date.

However, as tempting as it might be to document every idiosyncrasy of the codebase in an agent file, this will result in large prompts full of information, most of which will be irrelevant to most tasks, with a real risk of confusing the LLM. Even if the context window is large enough, you still want to be conscious of what you put in there. In an uncanny, human-like fashion, the primacy and recency effects where most attention is paid to the first and last bits of information presented have been observed in LLM’s!

Thus, the primary job of the human engineer in an AI-assisted workflow is not to write code. It is to compile the world the code will be written inside.

For every non-trivial change, you have to decide:

• What assumptions about this part of the system is the LLM likely to make

• Which of those assumptions are true and matter for this change

• Which assumptions the model is likely to get wrong

• And which slice of local reality must be injected to prevent that

In other words: what would you need to tell a talented developer with zero institutional memory so that they can operate effectively?

You are not “using AI.” You are compiling context for non-persistent intelligence.

And that is now the real job.

I am big fan of linter rules in general. They offer a nice way of establishing rules for your code base. This is handy both for aligning with team mates as well as your future self. With Schemalint, you can apply this to the database schema which is often a fundamental part of a software systems architecture.

The newest version adds support for triggers. I have been wanting this for a long time, because triggers are one of the more subtle parts of a schema and therefor arguable one of the more important aspects for a tool like this. A trigger (or the lack thereof) is not very visible in most database inspection tools, so it’s easy to miss when something is wrong.

For instance, I greatly prefer having a dedicated table and a trigger for soft deletes over having an archived_at column, as outlined in this blog post. I like this because the archived_at approach requires an extra is null condition in every single query you make to this table. And what’s worse: forgetting to do so often doesn’t look like a bug when developing. However, the dedicated table approach requires that you have the trigger in place and forgetting to set that is also easy to miss (though only needs to happen once per table). But now, Schemalint can catch those for you!

There is a custom example rule for enforcing setting a last_update column in the examples folder. There is a triggers field on tables and views in the input to the rules (see the details here).

In my experience, this lasts something like a month or two, and then it starts to die off. Think of this initial adjustment phase as the ‘pre-contagion window,’ where a new hire hasn’t yet been infected by the habits — good or bad — of their new team.

This period of fresh perspective is fleeting, and if you’re not careful, it’s easy to squander.

If you’re a manager or colleague to the new hire

Make an effort to be extra patient with these complaints. First of all, they tend to go away. Yes, they may be things you’ve already considered, attempted, or found impractical for various reasons. It might also be the case that you’ve been doing some silly exercise so many times that you don’t see it anymore, even if it costs you minutes every day. If you find yourself dismissing the complaint due to competing priorities, consider whether addressing it might actually save time and effort in the long run. Are you too busy cutting down trees to sharpen the saw? Conversely, and this goes without saying, don’t tolerate toxic behavior from anyone, including new hires.

Consider asking new hires to document any issues they notice during their onboarding period.

If you’re the new hire

Perhaps show this blog post to your colleagues, as a sort of preemptive move to make them more patient with you.

Remember, you’re uniquely positioned to make impactful arguments. There is a lot more potency in suggestions that are backed up with real experience. If you suggest switching to tool X in order to solve problem Y and you can add that you did exactly this at your last job and the tradeoffs were worth it, that is a lot more convincing than someone just suggesting to try it and see what happens.

By recognizing the value of the pre-contagion window, you will continually improve your team’s processes.

In this context, a bug is a piece of code that doesn’t do what the programmer intended, and an error is an undesired outcome or behavior.

Now, I am deliberately being vague about what distance refers to, as it’s a bit of a combination. It can mean “in the code”, measured in lines, files, folders, or services. It can also mean “in time”, where the visible error might appear significantly later than when the bug in question was triggered.

For syntax errors, this distance is practically zero. A squiggly line appears right underneath the bug. The error is both co-located and immediate, at least if you are working in a syntax-highlighting editor.

On the other hand, if you have a service that accidentally inserts garbage into your database, you might not discover it for a long time. The distance can be severe, both in terms of code location and time. This sort of bug is significantly harder to track down and fix.

So, it seems valuable to assess code not only on how potentially error-prone a pattern or piece of code is, but also on what the potential bug-to-error distance is, and try to minimize it.

How does one do that in practical terms? For instance, it has affected my view on type inference. For a while, my perspective was: infer all the things! Type safety without the plumbing code, best of both worlds — surely, that’s the way to go, always? Well, specifying types is like creating little valves in the code. “At this point, this is what I expect things to look like”. If you don’t do this, you will have less plumbing code but the price you pay is an increased bug-to-error distance because a piece of data may have a different shape than you expected, inferred from something “far away”.
I now tend to enable the explicit-module-boundary-types linter rule when writing Typescript, which forces explicitly typed signatures for exported functions. I am sure there are plenty of other similar considerations that I haven’t thought of yet.

If you haven’t worked with LLM’s as a developer, you might mistakenly think something similar to what I did: Hey, OpenAI has an API, it’s just ChatGPT without the UI! Chat is basically solved!

Well, it turns out that that is not quite the case.

Making an API call for completions feels more like using Mechanical Turk. You talk to it using natural language which is amazing, but it’s like every request is handled by someone new who doesn’t know anything, so you need to explain the whole thing from scratch every time. And the thing is: there is a hard limit to how long your description can be. So you need to summarize everything for it and figure out what context it needs to know for this particular message.

Imagine you received a letter out of the blue that said: “Well, at least I did 50 pushups today” and were expected to respond something. You don’t know who it’s from or even why they sent you this. What do you respond? Obviously, you can’t really say anything meaningful. There isn’t even a question in there! The job of the bot developer is to turn such a message into a piece of text that would enable you to not only respond something meaningful but also make it feel to the recipient as if you are just continuing a conversation that they were already having with you.

Establishing this context is very challenging. One common solution, at the moment, is retrieval augmented generation, or RAG, where you have a template that looks something like this:

You are a helpful coach chat bot. Your purpose is to assist the user with health, wealth and well-being.

Here are some messages from the conversation that may or may not be related:
[[related-messages]]

Here are the ten most recent messages:
[[recent-history]]

User says:
[[message]]

This provides a closer approximation to meaningful context, especially if one of those related messages tells you something else about pushups, like say, that the user has a goal of 50 per day for a month, or that they couldn’t do them because of an injury. Also, the most recent messages will make it clearer to you what you were talking about specifically, and what style of communication the two of you were using — was it a formal conversation, an inspirational pep-talk or perhaps more of a friendly bantering situation?

While fetching the most recent messages is straightforward, identifying the most relevant ones is anything but trivial. The first place to start might be to create vector embeddings out of every message in the history. Then you can find, say, the 10 messages with the highest cosine similarity to the incoming message. It will mean that previous messages that are similar will be inserted into the template, which is a place to start. Of course, similar is not the same as related.

I was using something close to this for a while and it did work but my bot felt quite distracted which was frustrating. Now, debugging this is really hard, but I suspect it’s because finding similar messages simply isn’t good enough. It wouldn’t remember old, related conversations if they weren’t sufficiently similar so it felt like I had to remind it of things constantly. It would apologize profusely and seem to recall when reminded, but that almost made it more annoying.

To address this, I implemented a strategy of tagging messages to create and utilize categories. That helped a bit but now my bot had developed dementia instead. It would often repeat points it had made in the past. That is also a seriously weird experience!

Another interesting thing is that since my bot communicates with the user at random times via the phone, I needed to tell it how long it was since the last interaction, what date, weekday and time of day it currently is, because otherwise it might say “good morning” in the evening, and it would carry every conversation as if there had been no delay. One thing I found here which seems quite intuitive when you think about it is that it was much better to tell it “last interaction was 3 days ago” instead of a specific date and then today’s date afterwards. It’s not great at math, so help it when you can!

I wanted the bot to not only react when the user initiates a chat. It should also reach out now and then. This isn’t something LLM’s will do automatically but a solution that seems to work well is quite simple: after each message, I ask the LLM for when it would like to follow up if it hasn’t heard from the user, and if so, what the reminder message should be. I then set a timer for the follow-up and re-initiate the chat with the reminder message.

But the hardest part, which is probably going to be the next uncanny valley for us to cross, is to convincingly “simulate” empathy. In order to make it feel like the bot cares about the user, it needs to be interested in them, learn about them and have a theory of mind. The latter is basically a way of saying that it should try to picture what the user is thinking and what their mood/mental state is like.

One thing I did which felt like a step in the right direction to me was this bit of text in the prompt:

You should decide for each message if you are in “empathy” mode or “problem solving” mode. Don’t mix the two in one message.
In “empathy” mode, you are looking to understand and possibly to help the user understand. Ask questions. If ${member.name} seems incongruent or you are confused, ask for clarification.
In “problem solving” mode, you should offer solutions and suggestions. You can still ask questions but those will probably be of practical nature.

Secondly, I have given the bot a “note pad”. It can add a note to this after each message. I then run two types of “dream cycles” where it reorganizes its thoughts. The server runs these asynchronously to the conversations. The simpler one runs daily. This one will make the bot read its notes, conversation and other inputs from today and update its note pad. Currently, the entire note pad is injected into every prompt which doesn’t scale that well, so I might look into a tagging system there as well.

The second dream cycle is weekly and more resource intensive. It will analyze various facets of the ongoing conversation, access its data, and perform multiple interpretation runs. For instance, it will try to spot what the users particular vocabulary is. If they say they “went for a run”, does that mean a 10 minute sprint or a 2 hour jog?

I studied neuroscience at university and while I always found it fascinating, it never felt as tangible and, well, basic as it does to me now. This is probably what is most exciting to me in all of this.

So the above changes have been about making it closer to human. But I want my bot to act as a coach. What makes a good coach? Well, one thing I intend to do is to give it a library. I might for example use my old collection on Procrastotherapy which has several good resources that I keep forgetting about. I imagine I will create little descriptions for each book/essay/video and then put those in my database as vector embeddings as well, so it can look those up. It would also be fun for it to keep up with new research by simply reading papers or articles posted to some subreddit or something. These are all vague ideas so far.

Bottom line is that programming LLM’s is fun! It’s a weird hybrid between programming, psychology, neuroscience and library science. It feels super empowering and humbling at the same time. Interestingly, I think it forces you to think in terms of empathy yourself: what does the bot know at this point? How can I articulate things so it will have the necessary basis for answering? When you think about it, this is a lot like what you do (or should do) when having a conversation with a fellow human being. That is just super fascinating to me.

<a href="https://medium.com/media/29d020d306156aa817f80f5d7766ba71/href">https://medium.com/media/29d020d306156aa817f80f5d7766ba71/href</a>

Here are some of the relevant links:

• Meme generator

• Maker’s Schedule, Manager’s Schedule

• Why you shouldn’t interrupt a programmer (comic)

• The joke and its relation to the unconscious

• Culture comparison tool

Using types to remember and enforce the shape of your data model is extremely valuable.

It doesn’t stop there, though. Type systems can be used to enforce a meta-architecture of sorts, which in many ways is even more powerful. Languages like Rust and Haskell excel at this. A common phrase among Haskellites is “when it compiles, it works”.

Let’s look at a trivially simple example. The most famous design pattern from the nineties was the mighty singleton. I don’t personally see it much these days but it probably sneaks in many places still. The idea is that there are some things you want one and only one instance of, like the connection to the database. And while you are perfectly capable of remembering this, your experience tells you that the developers you work with simply cannot be trusted not to create new instances, the little rascals, so you can nudge them in the right direction with this pattern.

What happens when you try to instantiate a singletonized class:

Ah-HA! The type system saves the day, and your junior developer will realize that they are doing something they are not supposed to. The error message isn’t too helpful — it’s sort of like your mechanic telling you that there is insufficient electrical current flow between the car’s alternator, voltage regulator and starter motor, when all you need to know is that your battery is dead. But, with the singleton being such a common pattern, many developers will intuitively look for a static getInstance method or similar.

What we’re really doing with our complicated types is building a fixture for building our app. Or, if we’re making a library, it might be at an even higher level of abstraction. With higher-kinded types we might be enabling creating types for creating a certain architecture. Let me be completely clear that I am not arguing against typing, I just think that that it can and should be complemented with linting. In fact, I think that any project of significant size, private or public, should come with a set of custom linter rules.

The React Rules of Hooks is a nice example. The linter rules are extremely helpful when using hooks, and I cannot imagine the kind of trickery they would have had to pull if they wanted to achieve the same using types.

I am working on a library that synchronizes state from the database to the frontend in endpoints. Basically, if any mutation is made, the response should contain an array that describes those changes. For simple CRUD operations, this can be done automatically. But for instance, when a table has a trigger attached, there can be undetected updates which require “manual” updates. That is quite easy to forget, so I wanted to try and create a guard rail for this.

My first intuition was to use the type system. Maybe such mutations could return a state value that was marked as unresolved, and then a function that creates the necessary updates would take this state value and mark it as resolved. Endpoints would then have to return such a state value, if it was unresolved, it would trigger a compiler error. In Rust, I think this could be done quite elegantly because of its sophisticated ownership tracking. But I failed to come up with a nice solution in Typescript.

Instead, I wrote an ESLint rule. This is what the first version looked like, with some helper functions omitted:

module.exports = {
  create: function (context) {
    return {
      CallExpression(node) {
        if (isCallExpression(node, "router", ["get", "post", "put", "patch", "delete"])) {
          if (node.arguments.length !== 2 || node.arguments[1].type !== "ArrowFunctionExpression") {
            return;
          }
          const knexCalls = countMethodCalls(node.arguments[1], "db", "knex");
          const resolveCalls = countMethodCalls(node.arguments[1], "db", "resolve");

          if (knexCalls !== resolveCalls) {
            context.report({
              node,
              message: `There must be a corresponding db.resolve call for each db.knex call.`,
            });
          }
        }
      },
    };
  },
};

Basically, it checks that any db.knex call in an endpoint is complemented by a db.resolve call by making sure they are called the same number of times. I have since changed it quite significantly, but I wanted to show this version because it’s so trivial. If you are refraining from writing linter rules because it seems daunting, consider giving it a try. (Pro tip: ChatGPT or Co-Pilot are great at it!).

While this rule does not cover everything, it has the very nice feature that the error message tells you what the solution is. If this had been a type error, you would almost certainly be facing a cryptic message that only makes sense if you already know what’s wrong. With this solution, our architecture is both more self-validating and self-documenting. It sure beats looking in that eternally outdated wiki that we all edited after the big planning meeting a year and a half ago!

But, you might say, this doesn’t sound very sound! Indeed, it‘s not. And I know that this is off-putting to some more than others. I personally don’t mind the lacking soundness in the Typescript type system, but to some it’s a complete deal-breaker. To me, the beauty is that linter rules are easy to disable in special conditions, so you really only do need to get them 80% right for them to be useful.

In my mind, the triad of typing, testing and linting should be the gold standard of writing solid code.

My late grandmother was uncomfortable in this world in her old age. Even though she never experienced captivity, I think she might well have quoted Brooks in The Shawshank Redemption:

“The world went and got itself in a big damn hurry”.

<a href="https://medium.com/media/c261bd401b5f20a78f49d2e22cb05645/href">https://medium.com/media/c261bd401b5f20a78f49d2e22cb05645/href</a>

My grandmother thought the world of computers and the internet was crazy, not just silly but completely out of control. This was almost a decade ago. Although I empathized with her perspective, I saw the globalized world as a vast playground, brimming with opportunities and excitement. For me, technology and cheap flights (despite my climate-related concerns) were things to love. I was a digital nomad at the time and my life felt like one big adventure. Yes, things were changing fast but that change was almost always good!

Now, I am certainly getting older, but with everything that has happened in 2023 already (at the time of writing, Jarvis and Auto-GPT are the most recent, mind-blowing news that I am aware of), I am starting to feel a bit like Brooks or my grandmother. I have an instinctive desire to let this whole AI thing “settle”. I tell myself that it’s a shift, like the internet or smartphones, and like those times, things will be crazy for a bit but then we’ll adapt to the new reality and it becomes normal.

Tim Urban creates a great metaphor for human history: a book. If all of human history was compressed into 1000 pages, the amount of progress that has happened on the very last page makes the entire rest of the book pale in comparison. You should read the blog post, it’s great like all of his writing. It really nails the level of exponential that we’re going through at this point in time.

The thing about exponential growth is that it’s hard to grasp even when you understand it. The Wheat and chessboard problem points out the simple fact that if you start with one grain of wheat and double it 64 times, you end up with 2^64−1 grains, an unspeakable amount of wheat. I know this, intellectually. And yet, if I just try to picture a chess board where each square has double the amount of the previous one, and the first square has merely one single grain of wheat, my mind insists that you would end up with just a pile of wheat. Something maybe in the range of thousands of grains.

If we are indeed experiencing the ignition of an “intelligence explosion”, we aren’t (well I certainly am not) capable of grasping it. Tim Urban predicts that the world will either enter a state of utopia where just about every problem is solved, or it will result in our extinction. I think he might be right, but I also think that until one of those things happen, we are going to feel like we’ve been strapped to a rocket, just trying to hold on.

Software development can be fickle like the fashion industry and right now, utility classes are on fire. I suspect that in time, they will fade a bit, the way micro services or NoSQL databases have. We have a tendency as a community to become infatuated with new principles, believing that they are a universal solution that must be applied everywhere, to everything.

So, while I can attest to the many benefits of utility classes, I find that there are a few downsides worth pointing out.

When a designer hands me a Figma file, utility classes are great because I can easily identify the ones that match the margin, padding etc. that I am looking at. It’s a lot easier than scanning through CSS files, looking for the class that matches everything, if a such even exists. However, it is also easy to overlook subtle details like letter spacing and line height. Those are things my non-designer eye is more likely to overlook and since there isn’t a rule set once and for all, I need to actively remember them.

The same applies to classes for pseudo-states like hover and disabled. The fact that you always have to specify everything makes this particularly easy to miss as you can quickly look at the result and not notice that anything is missing, unless you explicitly try out those states.

Tailwind recommends extracting components in React or whatever your framework is to encapsulate styles, rather than CSS classes. This generally seems to me to be a sound principle. Nevertheless, there are situations where it can be frustrating to have to split code into atoms, especially if you have a strict one-component-per-file policy (which I don’t recommend, though!)

Finally, there are a number of things that I really just want to set and forget. For instance, I generally want all my links to look the same. Having to add a bunch of classes to each one feels unnecessary and silly, as does creating a component.

Overall, Tailwind is a fantastic framework that I will continue to use. I have just reached the personal conclusion that there is absolutely nothing wrong with creating a .css file or two to complement it (and to be clear, I don’t think the Tailwind people would say otherwise!) You can use apply or you can just make plain old styles, it’s all good. Just don’t be dogmatic for the sake of dogma.

My dad is a man of fierce integrity. If you have a principle you should stand by it, also when it’s not convenient for you. He also has a very big heart. He holds himself to high standards but never lectures others. He is in many ways my hero, even though he would probably consider such a sentiment to be “noget amerikansk pjat” (American babble).

He is 80 years old and my son is 1, so there is a risk that the two won’t get to know each other well. This gave me the idea to set up an “interview” with him. Very informal and casual, I wanted it to feel like a regular conversation but where I ask him about his childhood, life and perspectives in general.
This turned out not just to be a very enjoyable experience but also, I learned a lot. There was so much that I didn’t know, about growing up in the Danish country side just after WWII, about his friends and extended family and, in turn, about my own childhood.
I now have a few hours of material and we’re just reaching his teens. I am really excited about this and I am hitting myself for not doing it sooner, and with more people. What I wouldn’t give to have such footage with my mom who passed away two years ago! Or even close friends, or my brother, from 20 years ago! I need to make a habit out of this.

I will never share these videos with anyone other than a few agreed-upon exceptions. I want to make sure it feels intimate and real and I don’t want anyone to be watching their words. The purpose is not to make “content” or anything, it’s to preserve a snapshot of something precious that will some day disappear.

I decided to write this post because I honestly think it’s something many people should consider doing in some form with their loved ones. Maybe in a similar setup, or maybe in a variation. I would love to hear alternative or complementary ideas.

I struggled to pinpoint what exactly was lacking. Yes, generated content was not “human”, whatever that means, but neither is the real world — nature and the shape of our planet is the result of a bunch of random events (unless you believe in intelligent design I guess, in which case this whole discussion must take on a new level of semantics).

One distinct problem was that even though the generated content was unpredictable in the specifics, you would always detect patterns. The way looking in a kaleidoscope theoretically offers unlimited visuals but still gets boring quickly. But even when such patterns were hard to detect, the mere fact that there was no intent behind anything just made it feel pointless. Which I find interesting philosophically, because why would a made-up but equally unreal world be any less “pointless”?

I think what has me so stunned by the advances in AI that we saw in 2022 is that this limitation seems to have been overcome. The output from various AI generators doesn’t seem pointless to me. ChatGPT writes fiction where I want to follow, I am curious about what it comes up with. In spite of it not being sentient, not knowing anything about Australia, math or why these texts are funny, I find them to be genuinely entertaining:

Elad Richardson on Twitter: "So impressive 🦘 pic.twitter.com/ng61A0ch02 / Twitter"

So impressive 🦘 pic.twitter.com/ng61A0ch02

(Incidentally, John Cleese gave a great talk on creativity which it seems ChatGPT has watched)

Similarly, the image generators create stuff that triggers emotions in me the way looking at (human-made) art does. I have yet to experience this from a music generator but I suspect I will before the end of 2023. One of the strongest indicators of the shift, to me, is that we now seem to be discussing what art is and isn’t. It reminds me of the discussions that appeared when Bitcoin was created about what money is and isn’t.

The thing is, all of this simply illustrates Moravec’s Paradox which was well known years ago. For some reason, I still believed that the progression of automation would go: blue collar → white collar → executive → creative, with the last step being a possible “never”. Turns out that was where it started. While I don’t believe we’re at the singularity yet (the tools that generate code are mind blowing but still quite helpless and certainly not capable of creating more advanced AI themselves), it does feel like the world has been turned upside down.

Basically, index files serve one of two purposes: as encapsulation or as a namespace. The corollary of this is that index files act as guard rails for imports — I avoid importing something from a subfolder of a folder containing an index file.

Encapsulation

If an index file is used for encapsulation, it should contain a default export for the primary entity (class, component, function, etc.) in the folder. It may also contain a number of named exports for related, “secondary” items. This is useful when you want to hide the implementation details of a module and expose only a single interface. For example, consider the following folder structure:

src
└── TopBar
    ├── index.js
    ├── TopBar.js
    ├── DropDown.js
    └── SearchBox.js

In this case, the TopBar folder contains the implementation for a top bar component, including a few sub-components. The index file re-exports the component in TopBar.js:

// index.js
export { default } from './TopBar';

To use the component, you would import the default export from the TopBar folder:

import TopBar from './TopBar';

Namespace

If an index file is used as a namespace, it should contain a bunch of named exports for everything in the folder. I used to do this a lot but I have actually stopped doing it in favor of just having the folder name in the import. However, it can still be useful to group related entities under a single identifier if they feel particularly strongly related.

For example, consider the following folder structure:

src
└── shared-components
    ├── LoadingButton.jsx
    ├── Spinner.jsx
    └── ModalDialog
        ├── ModalDialog.jsx
        ├── index.js
        └── Overlay.jsx
    ├── Chevron.jsx
    └── index.js

In this case, the shared-components folder contains four components. The index.js file in the

export { default as LoadingButton } from './LoadingButton';
export { default as Spinner } from './Spinner';
export { default as ModalDialog } from './ModalDialog';
export { default as Chevron } from './Chevron';

This makes shared-components work like a sort of package that you can import the components from. The win over importing the components from their respective sub-directories is small but might be to your preference.

Importing from Sub-folders

If a folder has an index file, I consider that a guard rail which should prevent me from importing anything from sub-folders. Now, files that are in the folder or a sub-folder themselves are allowed to import stuff, you just shouldn’t cross the boundary, so to speak. This is something I want to create a linter rule for because it could be enforced automatically, but for now I keep track of it manually.

Emergent Design

Both use cases are well suited for emergent design.

In the case of encapsulation, you can start by writing something in a single file. When that file becomes too large or complex, you replace it with a folder of the same name (sans extension), and from the perspective of the rest of the code, everything remains the same.

When used for namespaces, this is a nice first step for grouping related code before moving it into a separate package, either in your mono-repo using workspaces or all the way to NPM, depending on how general-purpose it becomes.

..default exports??

Some people argue that the concept of default exports was a mistake and I have some sympathy for that. At this point I’ve used them so much that it’s basically muscle memory and I’ve found that VSCode has sufficient refactoring capabilities for handling them, to the point where the downsides are negligible.

Sorry, this is a bit of a rant. My preferred routing library for React has been Raviger for a while but recently I needed to figure out how the most recent version of React Router works. It turns out that there are plenty of answers on StackOverflow and elsewhere that explain how to do things, many for outdated versions of the API.

I know that the React Router developers have taken flak for changing things throughout the years and I don’t mean to single them out. They have created great value for the community which I have capitalized on for free. I am grateful and they owe me absolutely nothing.

The small point I want to make, though, is this: if you are changing your API so much that it’s not just a “major breaking change”, but rather a complete redesign, consider changing the name of your library rather than just the version. For ecosystem related reasons as well as pedantic ones.

15 years ago I felt like the world was improving at record speeds because of technological advances. And I was a part of it. Not that I did anything particularly influential, but I identified with the tech sector and it felt like “my people” were freeing the world of the old, evil corporations that were rife with nepotism and corruption, helping it turn more meritocratic and equal. I felt genuinely optimistic about where the world was headed and I was always excited about products launches, software or hardware.

Today they make me anxious.

Not only because the concept of “startups” no longer feels like the breath of fresh air it used to, but more like a parody of itself. As Paul Graham became wealthy it seems he also became convinced that the only thing that motivates anyone is wealth, sounding more and more like those corporatists of yesteryear that were happy with the status quo. But while I find that disappointing, it’s not what this post is about. I am more uneasy about the technology itself, given the speed we are advancing at.

Two products have affected me in particular: DALL-E-2 and GitHub Copilot. They are both mind blowing to me and have given me this sinking feeling of technology moving not only really fast, but faster than I thought possible. And while still fun, I am starting to feel that the tech industry that I used to be proud of belonging to is like having a pet tiger that was cute (and a bit cool) as a cub but is suddenly becoming hard to control and I can’t really do anything about it.

I guess I am experiencing a threat similar to what so many professions before mine must have felt as technological advances made them irrelevant. Maybe I simply belong on /r/leopardsatemyface. Now, software development is not going to disappear any time soon. Maybe not in 15 years or maybe even in my lifetime. But just like the two rocks in the short film Das Rad (embedded above — if you haven’t, I highly recommend watching it, it’s just 8 minutes long), it feels like we’re past a sort of tipping point of hyperbolic growth. The only thing I am certain of now is that I have no idea what will happen.

I am not particularly worried about my own financial well being. I think I will be fine. I do worry slightly about my sense of self. Creativity is a big part of what I feel defines me, and if that loses all value then it will surely affect me. Then, there are obviously a number of Black Mirror-esque dystopias that don’t seem all that unrealistic any more. But mostly I struggle with how to guide my son when he grows up. In that regard I feel truly lost. What will be a viable career in 20 years? Will the concept of a “career” even make sense by then? I have no idea, and I am uncomfortable with that.

I gave a talk at Copenhagen React about the tRPC framework and how we use it instead of REST or GraphQL at Mymee:

https://www.youtube.com/watch?v=k1TCueEhhJo

It may just exist in my little bubble, but it seems that there is some conventional wisdom that says that in negotiations, you should try to get the other party make the first offer.

It’s not that I have researched this a whole lot. I listened to a podcast once with some guy who had negotiated deals for lots of big companies and was considered a heavy weight in the field. I remember being a bit annoyed with this because when you represent IBM in some deal, you almost certainly have what everyone knows is the real, deciding factor: leverage. At that point, I don’t really care about your theories on body language and whatever. I am sure he made some good points but none that left a lasting impression on me.

Being the one who makes the first offer, however, is something I have practiced for a long time and I feel it serves me well.

I can see two risks with it. One, you end up like Michael Jr. in Road to Perdition (sadly I couldn’t find this clip anywhere):

Michael Sullivan, Jr.:
> So when do I get my share of the money? 
Michael Sullivan:
> Well… how much do you want? 
Michael Sullivan, Jr.:
> Two hundred dollars. 
Michael Sullivan:
> Okay. Deal.

…

Michael Sullivan, Jr.:
> Could I have had more? 
Michael Sullivan:
> You’ll never know.

The second risk is that you suggest something so outrageous that you get an Al Swearengen-style response:

Now, regarding situation #1, how likely do you think the situation is where your new employer thinks to themself “Great! We were going to offer twice the salary — but now we can get better drinks at the next company party instead”? I imagine it’s probably quite unlikely. If you are completely unaware of the value of the product or service you are buying or selling, negotiation is not the time where you are going to discover it.

As for #2, well even if this is indeed how they respond, do you think you would then have been interested at the price point (or whatever is being negotiated) that they had in mind? It’s not a pleasant situation for sure, but it’s probably good to get it out of the way and you can all be on your way.

If you want the other party to offer first because you think of a negotiation as a sort of poker game and you think that them revealing their hand gives you an advantage, I just have to say that I don’t see what that would be. State what you would like and work from there. If the other party wants a different number, it’s now on them to convince and compel you to move and not the other way around.

I was interviewed by Nick Janetakis for the Running In Production Podcast:

Submotion Helps You Manage Access Control for Your SAAS Subscriptions
In this episode of Running in Production, Kristian Dupont goes over building a SAAS app to manage access control to…runninginproduction.com

We get into quite a few details about the architectural choices I’ve made for Submotion and how it all works together.

I’ve written before about the way I let the Postgres schema act as the source of truth by generating types with Kanel (assisted by Schemalint). In addition to this, I mention a homemade framework that sort of resembles a server-side Redux. This gives me typesafe database queries that are automatically reflected all the way to the frontend. I am frequently asked to make it open source and that is still the plan. I just need to untangle it from the Submotion source code which just never seems to be high enough priority. I promise I will get around to it :-)

..ensuring sound mocks and supplying Typescript types.

Submotion offers a central place to manage SaaS accounts and subscriptions. This is made possible by connecting to a bunch of third party API’s. It’s an amazing fact that so many companies expose API’s that allow you to integrate with their service, to mutual benefit. Using such API’s is fairly trivial and even though things like OAuth implementations quite often diverge from the standard, it’s usually not too hard to set up.

However, testing such integrations is painful.

Proper end-to-end testing is not feasible in a CI setup. It’s not only slow and relies on access to outside servers that could be down, it’s also next to impossible to set up useful fixtures.

Initially I tried to set up a record/playback system to alleviate this. That works well in some situations, but Submotion almost always connects via OAuth which is difficult to automate (by design), and even then, I still had the issue with fixtures. I would want to test that Submotion detects, say, that an account has been deleted. In order to do that, I had to actually create and subsequently delete an account in Slack or whatever, a very involved task. 
So I dropped that approach and for a while I just accepted that mocks were the best I could do. And in fact, I am still writing manual mocks, but I did manage to improve the situation.

I was discussing this with my friend Lars who has put a lot of thought into these and similar issues. He suggested that I move the assertions from the test into production code. Now, I remember doing this sort of thing a lot in C++ where assertions are common in the debug build and then removed in production. However, this didn’t feel like a Node-native approach and it rubbed me the wrong way somehow. But, as he pointed out, such assertions could just log warnings in production, and when running locally or via tests, they could throw. That way, I would continuously run validation on real responses meaning that said validations are kept up to date and good enough to keep my mock data sound. Although this is strictly validating, it still feels to me like a variation on the parse, don’t validate philosophy coined by Alexis King. And while it’s still work in progress, so far it’s a big step forward.

Here is what I do in practice:

1. Make a request or two to the 3rd party API in question, and save the JSON responses in a file. This is the sketch.

2. Convert these into JSON Schemas using a tool like https://transform.tools/json-to-json-schema

3. Clean up the schemas. They can rarely be inferred correctly from a single response, so some tweaking is required. Notably, I add format fields for things like emails, URLs or IP addresses where applicable.

4. Then, use AJV (with ajv-formats) to validate the responses right after making a request. If validation fails, log an error or, if running tests, throw an exception

5. Finally, use as-typed with the schema to get a typed response. It’s not perfect but much nicer than an untyped one and I can be confident that my runtime validation is aligned with the type check.

Now, I could replace JSON Schema validation with something like io-ts or Zod which I am playing with, but this is what I am doing for now. Here is an example and a simplified version of the approach:

Of course, schema validation/type checking only looks at the shape of the responses which means there is still lots of room for semantic errors. There is nothing stopping me from expanding this setup with more semantic assertions which would further improve the quality but I haven’t really found my sweet spot for that yet. Perhaps in a future post.