Markwhen is a tremendously underrated piece of software. It’s a good syntax for creating timelines with a great interactive UI bolted onto it. What mermaid did for charts, markwhen does for timelines. Plus you can use it as a calendar if you want.

The markwhen editor I use is Meridiem, which you can also download as a desktop app and it’ll edit text files on your computer. The editor is pretty amazing. It syncs what you click on in the timeline to where it is defined in the text document.

The syntax is really good: it supports a bunch of date formats, shorthands like just writing 2009-2012, relative dates, like writing 5 years and that automatically coming 5 years after the last event.

Understanding time is one of my biggest cognitive challenges. It has always been hard: keeping the months in order, remembering how long something has been happening, remembering what happened first. There might be a name for this inability to conceive of time intuitively: whatever it is, I have that. So I rely heavily on calendars and tools. Every event and birthday is in Fantastical for me.

But I didn’t have a tool for seeing longer-term events until Markwhen. Now I can look at things like where I’ve lived, my career trajectory, vesting, QSBS timelines, vacation planning, and more, all on in a single interface. It’s been a huge boost.

Pretty often in Val Town’s vitest test suite we have some scenario where we want to set up some data in the database and then use the test.each method to test different operations on the database. We finally did the work of figuring out how to represent this nicely today and came up with something like this:

import { describe, it as _it, expect } from "vitest";

describe("tests", () => {
  const it = _it.extend<{
    time: number;
  }>({
    // biome-ignore lint/correctness/noEmptyPattern: vitest is strict about this param
    time: async ({}, use) => {
      use(await Promise.resolve(Date.now()));
    },
  });

  c.for([[1, 2]])("should work", ([a, b], { time }) => {
    expect(a).toEqual(1);
    expect(b).toEqual(2);
    expect(time).toBeTypeOf("number");
  });
});

This required a few different bits of understanding:

• it.for is a better it.each. We were using it.each, but it doesn’t support text context.
• Test context is a pretty decent way of storing fixtures. Before we were using beforeEach and beforeAll and assigning to variables that we’d refer to in test cases. it.extend is a bit smoother than that.
• Still, manually repeating the types of the fixtures in it.extend is an annoyance, probably cause by the fact that you call use(value) instead of return value to assign fixtures. There’s probably a good reason why they did that, but nevertheless I’d prefer to have these fixture values inferred.
• Vitest is pretty wild in that it does static analysis checks, which is why I had to disable a biome rule here for that empty object pattern: vitest insists in destructuring, and biome does not like empty object destructuring.

Well, I deleted all my tweets and unfollowed everyone on Twitter, using Cyd. I migrated my old tweets to a Bluesky account, and saved the exported backups from Twitter for posterity. I am keeping the account and its handle so that they don’t get reassigned and squatted.

Twitter was a big deal for me. As of this writing, my defunct account has 14.7k followers. That number, and the ability to spread messages there, and the network of great people I met there, and the silly flamewars, had a mostly-positive impact on my career and life. I loved Twitter. Leaving it probably reduces my reach by a bit. I’m pretty active on Bluesky and Mastodon, but the tech community has been slow to migrate.

But it has obviously and absolutely outlived its purpose. It’s run by a Nazi who is causing my country and my friends and the world harm. It is festering with bigots and misinformation. The purpose of Twitter is to spread hatred.

This is not ‘virtue signaling’ or whatever. Twitter’s owner has made his views crystal clear, and the product has been molded into something that serves his political goals. What used to be a social network is now a tool in his destructive conquest. Why would I help him, even a little?

Obviously I’m getting out of there and you should too.

Extremely niche stuff here, this probably isn’t relevant to 99% of people, but nevertheless, for someone this will eventually be useful.

Fastify with the TypeBox type provider is a pretty great experience. It lets you define types of route parameters, query string parameters, and responses. It gives you both runtime and static (TypeScript) type safety.

Fastify also has a system for Decorators, in which you can enhance its objects with extra methods. So, for example, you could add a new method to their reply object – the object usually called response in Express and some other systems.

We’ve also been using neverthrow, a TypeScript library that provides a Result type, just like in fancy languages like Rust. A Result can contain a successful value, or an error, and it preserves good types for that error, totally unlike JavaScript’s chaotic try/catch exception system. neverthrow has been mostly great for us, with some minor nits.

Neverthrow has helped with using shared application logic between Remix, tRPC, and Fastify: we can have some shared error types and ‘unwrap’ them in ways that each of the systems prefers. I wanted to do the same for Fastify.

So I wanted to add a method like reply.sendResult(result: Result<T, E>), and for the ‘ok’ value in the result to have the nice types that I would get from the reply.send(value: T) method. This is not super obvious, because all of the examples of using decorateReply don’t include any of its generic parameters. Took a while but I figured it out. The incantation is:

declare module "fastify" {
  interface FastifyReply<
    RouteGeneric,
    RawServer,
    RawRequest,
    RawReply,
    ContextConfig,
    SchemaCompiler,
    TypeProvider,
    ReplyType,
  > {
    sendResult<T extends ReplyType, E extends MappableErrors>(
      result: Result<T, E>
    ): FastifyReply;
  }
}

It’s verbose because we have to repeat all of the existing type parameters for the generic FastifyReply type in order to access that last ReplyType parameter, but after we’ve done that, it merges into existing types.

And here, MappableErrors is the base class for our known set of Error subclasses. You probably have different ones. And then the functional part of the plugin looks like this:

export const fastifySendResultPlugin = fp(async (fastify: Fastify) => {
  fastify.decorateReply("sendResult", function (result) {
    return result.match(
      (value) => {
        return this.send(value);
      },
      (error) => {
        // Map your errors to fastify responses…
      }
    );
  });
});

Not too bad! Lets us have both good Result types and strict types inferred from the schemas in Fastify routes. Roughly the same trick would apply for any other kind of reply decorator for a wrapped type, or the same idea for another Result library, like true-myth or Effect.

And herein lies the problem with the sudden surge and interest in artificial intelligence. AI-generated creativity isn’t creativity. It is all hat, no cowboy: all idea, no execution. It in fact relies on the obsession with, and fetishization of, THE IDEA. It’s the core of every get-rich-quick scheme, the basis of every lazy entrepreneur who thinks he has the Next Big Thing, the core of every author or artist or creator who is a “visionary” who has all the vision but none of the ability to execute upon that vision.

From AI and the fetishization of ideas.

Of all the critiques of AI art, this is the one that sticks in my head the most. It rhymes with my takeaways from The World Beyond Your Head, which argues forcefully for a new monism, rejecting mind-body dualism and arguing for an acceptance of people as embodied, living in a world with physical limitations and inconveniences. And skewering the notion that the mind should be thought of as some independent force that is inconvenienced by its inability to directly affect change.

We don’t just think about something, come up with the idea, and then enact the idea with our hands, bodies, and tools. The idea and its result are shaped by the tools, what’s possible, what we’re capable of.

I am reminded of this constantly because of the milieu in which I operate, one that is very psyched about AI. The most recent newsworthy incident was this quote from someone whose business promises to translate your musical ideas directly into music:

It’s not really enjoyable to make music now… it takes a lot of time, it takes a lot of practice, you have to get really good at an instrument or really good at a piece of production software. I think the majority of people don’t enjoy the majority of time they spend making music.

But I encounter a lot of different spins on this now. Becoming a good programmer takes time, so does becoming an artist. What if all the people with ideas but no time or skills or persistence or real interest could participate and turn their ideas into the thing? Surely non-musicians have great ideas for songs that they could turn into great songs if it weren’t for the inconvenience of musical instruments.

One way to look at this – not a charitable way, but a view that feels true to me – is that managers view all need for human labor as an inconvenience. In part because they rarely get to experience what it’s like to be closer to a creative process, but also because they constantly experience the inconvenience of checking on deadlines and paying invoices. They would simply rather manage a robot than a human, so the only other people they have to interact with are other executives. Peak economic efficiency.

when you don’t create things, you become defined by your tastes rather than ability. your tastes only narrow & exclude people. so create. - Why the lucky stiff

A more charitable way to discuss it is to say that creativity, perception, values are all just one big unified thing. I recently completed my first sewing project - a bag for my bicycling trips - and have spent the last week in awe of the quality of my backpack and clothing. I always assumed that constructing clothing was complicated, but now I’m on the subway looking at my ILE bag and lost in thought trying to understand which part was sewed in what order to make it all so neat and durable.

Creating things makes you look at the existing world differently. It makes you more impressed with existing art, music, or whatever to try doing it yourself. It makes you appreciate the effort that has gone into creating the world as it exists right now.

I just went through the painful process of updating an old NPM module to build ESM & CJS dists, with DTS files, and a CLI. It was bad. In summary:

I ended up using TSUP. It was a frustrating experience. I think the real root of the frustration is:

1. Most modules like this need .dts files, so that people who import from them have TypeScript types.
2. The TypeScript team has bizarrely refused to implement dts tooling to TypeScript.
3. All the third-party plugins are, as a result, kind of grumpy that they’re reimplementing things with TypeScript internals that TypeScript should be doing itself, so they’re all pretty burned out, it seems.

So, using esbuild is great, but if you’re distributing a library, you probably need esbuild-plugin-d.ts, which is not very adopted and even the maintainer warns you away from using it. So it’s kind of all stuck.

I’ve been working on editors again: tweaking CodeMirror to make it work better in Val Town. It’s really hard work, unfortunately the kind of hard work that seems like it should be easy. It has me thinking a bit about why front-end engineering is difficult.

In fancy terms, it’s multivariate and extended over time. Frontends, backends, browsers, and users are all variable, unpredictable actors.

There are no single-player web applications: the simplest model of frontend software is a user interacting with a webpage, but the user and the webpage have similar capabilities. Your frontend application can respond to and produce events. It can modify the webpage.

So can the user: they can modify webpages at any point. Maybe they load every page in a new session, breaking your assumptions about how a website will persist state. Or maybe they never create a new session - they have 80+ tabs open that never get closed, so they will continue using versions of your frontend application that you released months ago, which now break when talking to an updated backend server.

But those aren’t the only players. Browsers are active participants in the game: a browser might translate a webpage and modify the DOM in a way that makes React crash. It might disable an API that the application expects, like localStorage, in a way that makes its behavior unpredictable. Ad-blockers might prevent some of the application from loading.

Distribution is probably the single most compelling attribute of the web over native applications. There’s no app store, no mandated release process, no installation step. But just as you as the application developer have more power, so do browsers and users.

The interplay between these actors is what makes things difficult, and difficult in a way that’s underemphasized in both Computer Science dogma and hacker culture. It isn’t some algorithmic challenge or hard performance problem, or anything that can be solved by switching programming languages. It’s an understanding of uncertainty in a deployment target that is constantly shifting and has a lot of unknowns.

After working in this realm, it’s a treat to work on ‘hard’ problems that at least give you more certainty about the environment. Optimizing SQL queries, for example – it can be challenging, but at least I know that I’m writing SQL for Postgres 16. Frontend engineering can feel like optimizing a query and not knowing whether it’ll run in DuckDB, Postgres, or MySQL, and even whether you can trust that the database schema will still be the same when it runs.

It’s hard work. It’s not cool. And I am nowhere near mastering it. I totally get it when people have an aversion to it, or think that the methods used today are wild. But mostly the methods are wild because the problem is, too. A straight-line solution to the problems of the front-end would be lovely, but those problems don’t really permit one.

I got the chance to chat with Jeremy Jung on the Software Sessions podcast. It was a fun, casual, candid conversation about Placemark and my time spent bootstrapping. Give it a listen!

See the last ast-grep rule I posted for context here… this rule is targeted at cases in vitest in which we had tests like

it('loads', () => {
  expect(loader()).resolves.toEqual([])
})

This introduced a subtle bug in tests because there’s no await keyword, so we’re not waiting for the loader() method to finish before moving on with other test cases. The proper way to write that test is:

it('loads', async () => {
  await expect(loader()).resolves.toEqual([])
})

Vitest 3 autodetects this bug, but Val Town haven’t upgraded yet. So in the meantime, this ast-grep rule creates a nice guardrail.

id: require-expect-await-rejects
language: tsx
severity: error
rule:
  pattern: expect($$$ANY).rejects
  not:
    inside:
      kind: await_expression
      stopBy: end

Yes, typescript-eslint has a rule called require-await that does just this. But we dropped Eslint (and Prettier) for Biome last year and I’m not looking back. The old eslint setup was both slow and fragile – it would break every time we bumped TypeScript versions. The Biome team is planning to implement type inference in 2.0, so maybe we’ll get all the power of something like typescript-eslint eventually!

Today I fixed a bug in Val Town in which we had some bad nesting. This happens if you have something like this:

function Component() {
  return <a href="https://google.com">
    Google
    <a href="https://yahoo.com/">Yahoo</a>
  </a>;
}

This is not allowed in HTML and React will produce a warning when it happens. It’s pretty easy to catch in a one-off case, but in a large codebase, this kind of bug will sneak in.

We didn’t have any way to automatically flag this kind of bug. I’ve been using ast-grep to build custom rules against bugs like this. The learning curve is steep, and the documentation could be a lot better, but the power of ast-grep is significant: it’s both extremely flexible and lightning-fast, so you can run your ast-grep rules in git hooks without ever noticing a slowdown.

I cooked up a rule to catch this violation:

id: no-nested-links
language: tsx
severity: error
rule:
  pattern: <a $$$>$$$A</a>
  has:
    pattern: <a $$$>$$$</a>
    stopBy: end

And can be pretty confident that this kind of bug won’t crop up again.

In the most naive version of course! If you were to have a component that rendered an <a> element and included {children}, then you could easily trigger this bug. But at least the most common, simple case is covered.

I’ve been doing a lot of refactoring into neverthrow recently and hit this surprising case today:

import { ok, err, Result } from "neverthrow";

// Have one function that returns a Result<string, Error>
// value. In this case, the 'ok' type shouldn't matter 
// because we never refer to it and only use its
// error type.
function getPerson(): Result<string, Error> {
    return err(new Error('Oh no!'))
}

// And then a method that returns a Result<number, Error>
// type, which we will only use the number part from.
function getLength(name: string): Result<number, Error> {
    return ok(name.length)
}

// Combine them, and get a surprising result!
function combined() {
    const p = getPerson();
    if (p.isErr()) return p;
    return getLength(p.value);
}

// Even though we only took the error side from
// getPerson, we still have its return type (string)
// lurking in the return type from combined!
const r = combined().map(val => {
     val
  // ^? (string | number)
})

See this code in the TS playground! Basically this is somewhat surprising because I might guess that the .error part of a Result is narrowed to a Result<never, Error>, but I guess it isn’t!

The solution was to reconstruct an error type with no ok side:

if (p.isErr()) return err(p.err);

I know, this code isn’t as functional-programming aesthetic as it could be, but like most things that have users, the Val Town codebase is pragmatic and non-pure.

Overall my feelings on neverthrow so far:

• It’s pretty practical and has yielded a big improvement for us.
• The learning curve is a lot less steep than Effect, and the documentation is a lot better, but it’s still another thing to learn, and it’s easy to in ways that eat a lot of the complexity without capturing the benefit. It is something I’ve needed to advocate for and help out with quite a bit.
• We use something akin to my proposal of a resultFromAsync function in lots and lots of places. I think that should be part of neverthrow and find it pretty difficult to image living without it.
• We also have a bunch of other utility functions around neverthrow to translate errors to their formats in tRPC, Remix, and Fastify.
• I think that JavaScript/TypeScript‘s poor ergonomics around error handling are hard to fix with an add-on library and I hope a language-standard method arrives eventually.

Today on Val Town I made this: Surprising embeddings!

This is inspired by this other blog post which critiqued the common-sense value of embeddings. There’s a whole essay on the surprising embeddings page unpacking what the demo is showing and what it means. Check it out!

We had a bunch of code like this in our vitest test suite:

expect(createdUser).toBeTruthy();
expect(createdUser?.email).toBe(mockEmail);
expect(createdUser?.profileImageUrl).toBe(mockProfileImageUrl);

This needs the pesky ? optional chaining operator because createdUser is maybe an undefined value. But vitest has that expect() call that makes sure it’s truthy, right!

Expectations in vitest aren’t also test assertions - maybe for some hard TypeScript reasons or because they can be loose assertions that keep running code, or whatever. Anyway:

vitest provides an assert function that you can use like this:

assert(createdUser, 'User exists');
expect(createdUser.email).toBe(mockEmail);
expect(createdUser.profileImageUrl).toBe(mockProfileImageUrl);

Unlike an expect, assert does tell TypeScript that the code after the assert won’t run if the assertion fails, so you can get rid of any optional chaining after that point. Nifty!

Google has been adding hard-to-disable AI features to Gmail. I stopped enjoying the Gmail UI years ago, and really using Fastmail instead for my personal email. But, work email is usually Google.

I’ve been using Mimestream for my work Gmail account and highly recommend it. It’s somewhat expensive, but worth it for me: fast, slick, and has just the features I want. If you’re frustrated with Gmail’s UI, check it out.

neovim is arguably the most important application I have on my computer. It’s the text editor I’ve been using for well over a decade (if you count vim as well), and it is different enough from every other text editor that it’s really unlikely that I’ll switch to anything else anytime soon.

Right now I use a configuration of neovim based on LazyVIM. Neovim’s built-in features are not enough for me or, I suspect, the average coder: most people want things like LSP-driven autocompletion and integration with code linters. LazyVIM is maintained by Folke Lemaitre, an incredibly productive programmer who is leading the way on plugin development.

I sponsor neovim’s development but really I want to pay something like $12/month for a tool like neovim. There’s some risk of ruining the open source ethos, but on the other hand, a well-functioning editor is easily worth that amount of money to me. Neovim’s competitors are healthily funded VC-backed startups and Microsoft-backed behemoths.

As great as it is, neovim suffers from a burnout problem. Someone will write a great plugin and attract a lot of users who hit bugs and report them, and the maintainer doesn’t have the capacity or motivation to handle it all. Your average neovim user, including me, doesn’t especially want to learn all the ins and outs of plugin development in Lua: they want a working editor. I want to pay for that. Why not make a neovim company?

I published this on Observable a while ago but in the interest of keeping all my content here for the long term, I’m syndicating it. Filecoin has neither replaced S3 nor died off in the time since publishing, so I consider it still fresh.

How is Filecoin supposed to work? Really, how is it supposed to work?

So as far as I understand it, FileCoin is trying to supplant Amazon S3 for file storage. It is extremely hard to see how this would work.

How a company might work

If we assume that the basic principles of economics stay the same, the basic economics of selling storage consist of:

1. The cost of providing the service: hard disks, networking, paying people to keep the hard disks spinning, buying new hard disks when they break, paying for electricity and air conditioning.
2. The income of people buying the service.

The margin between the two is profit.

File sharing is a competitive business

Traditionally, we understand file storage as a competitive market. Some players, like Amazon, have higher margins than others, but at the end of the day a company can compare S3, B2, Wasabi, R2, and any number of similar services with prices, and if those services are a better deal, then they’ll buy those instead.

How FileCoin is different

FileCoin intermediates customer and provider in a way that makes providers “more” interchangeable and homogenous. You put FileCoins in, you get storage out, and in the middle you might choose a provider. But you aren’t signing a contract with a business, and probably not paying taxes on anything.

In theory, FileCoin would create an efficient market through competition. This probably wouldn’t be true immediately, might not be true ever, but let’s accept the long-run theory.

Thought experiment one:

Okay, so how does this work? Here’s a thought experiment:

I, Tom, want to compete with Amazon S3. I set up an LLC or C Corp, buy some hard disks, put up a website, accept payments. Does this work? Is this a common business plan? Generally, no, right? I don’t have any scale advantages. I don’t have a contract with a fast internet provider for discounted connectivity. I don’t have a relationship with multiple hard disk providers like Backblaze does, or network colocation like Cloudflare or Amazon. I’d lose, right?

So conversely, I, Tom, become a node on the FileCoin/IPFS network. Does this work, and why? IPFS, as a distributed system, as far as I can tell, is strictly less efficient than a “trusted, centralized” system.

How are the coins supposed to be useful?

FileCoin uses its own unit of currency, the FIL. How does this work? The price changes through supply and demand, as well as, like other cryptocurrencies, extreme volatility, manipulation, and so on. The SEC is not there to catch the bad traders.

So, let’s say the price of a FIL changes from $1 to $100. If I paid 1 FIL ($1) for 1 gigabyte of storage yesterday, would I pay 1 FIL ($100) today, and why? If I’m a buyer, is this price volatility a benefit in any way?

It’s extremely unlikely that my business is charging for its widgets in units of Filecoins, but my costs are priced in Filecoins. What’s the value of having a different currency for different things, with its own volatility? Isn’t this mostly downside? Companies that have to deal with multiple currencies tend to spend a lot of time and money hedging that risk away. The risk isn’t seen as a good thing: if your public company starts making exchange-rate bets, that’s generally a sign that you’ve strayed from the thing your company was supposed to be doing in the first place.

The math

Here’s my mental math with FileCoin:

• File storage is a low-margin, not-very-fun business with structural bars to new entrants
• FileCoin makes file storage cool because it adds an element of speculation and technological pizazz, but it doesn’t change any of economics or structural problems
• This doesn’t seem like it’ll work.

Non-answers

Some non-answers here are:

1. FileCoin is not the end goal, it’s just a jumping-off point, and the real prize is services built on top of the Protocol Labs stack, like the virtual machine. Sure, fine, not really an answer. And if the economics at the base layer don’t work, how are we supposed to build on top of them? The foundational layer of economies tends to be things that are stable.
2. The tokens go up. As I wrote before, the tokens go up is fun for people who have tokens, but as far as I can tell, price volatility is purely a negative for the central thesis of buying and selling storage. Would token appreciation cause people to buy more storage, or less, or do we probably convert FIL to USD and spend the equivalent of USD? Even in a supposed crypto future in which Ethereum and Bitcoin are reserve currencies, we’d still convert from FIL to that unit and vice-versa, right?

Non-statements

• IPFS is neat, in some ways! I think it’s not very useful yet and it’s pretty over-architected and overpromised, but it’s cool and cool people have worked on it and want it to be good. FileCoin is different from IPFS.

The slightly-expanded argument

Take a utility project - like HiveMapper, FileCoin, or Helium. Think about what it is without the crypto element: just one person selling wifi to another, or streetview images, or file storage. Is that thing high-margin?

We have comparables for this. We know the margin of big, scaled storage providers. Smaller ones, without magical differentiation, will have lower margins because they will have structural disadvantages (e.g. they’ll pay retail for bandwidth). We know how much people get paid to drive for streetview competitors: it’s not much.

Add in the crypto coin element, and figure out why it’s a goldrush opportunity or even just a sustainable business, and figure out why. What’s the thing that turns these low-margin grinds into such great opportunities?

Implementation-defined limits

• The length of a URL: can be as low as 2,083 characters in Internet Explorer, but Firefox displays URLs up to 65,536 characters.
• HTTP Headers: Apache sets a limit of 8k and Nginx limits to 8k as well.
• Emails: are limited to 20-25MB.

Terence Eden wrote a good article about how most people don’t care about quality, in response to a genre of article about how quality is declining. It’s a rich, long-running vein of discourse, the spiritual successor of discussions about how “pop music” is rotting our brains. Or see Neil Postman’s warnings about a vast descent into triviality caused by TV culture.

Thinking about how companies want to produce AI slop and trash and people want to consume AI slop and trash leads me directly into the void, so I’ll note some of the other stuff involved:

1. All art has both an interior language and an exterior language. In the interior, there are in-jokes and a passion for tiny details. There’s “music for musicians,” for people who will notice odd time signatures, and films that only make sense to film buffs. There are architectural details that are universally noticed by architects and fans that fly unnoticed by everyone else. Arts have their ‘practical purposes’ of raw entertainment, and then they have this underlying detail that delights the snob.
2. Companies want to create slop, but most humans hate creating slop. Journalists don’t like writing listicles or quizzes. Software engineers don’t like implementing dark patterns, unless they’re exceptionally cash-motivated. People who create quality stuff are driven to do so by non-economic factors, by a love of their craft.
3. “Appreciating good art and design” is a privilege: it chiefly requires time and money. Netflix is famously competing with sleep, not HBO. Its shows are explicitly designed for people who want something to fall asleep to, or have playing while they cook dinner. People don’t have lifestyles where they are going to theaters for two hours of focused cinema. The cheapening of things that used to be quality is in many ways a side effect of the skyrocketing costs of housing and healthcare, and our continued failure to create more leisure time despite increasing GDP.

There’s plenty of reason to despair. But, as Terence notes, there always has been. There’s always been a plethora of bad movies and books and everything.

But, some people will continue to create things of quality, and others will continue to appreciate that. For both economic and non-economic reasons.

I just wrapped up some international travel - four cities in India, more about that later maybe - and this was the first time I invested a little time in getting better cell / internet coverage.

Saily worked really well: you give them money, they give you an eSIM that worked instantly on my iPhone, which is unlocked. There surely might be cheaper options, but paying about $20 for a trip’s worth of internet was great, and a lot better than AT&T’s default. The catch is that you don’t get a phone number.

Getting geofenced to India was creepy, so I used Mullvad VPN, which also worked well. VPNs tend toward scammy business models, but Mullvad I liked, and I especially liked their simple, one-time-purchase way of buying access.

Next time around, I’ll remember to install WhatsApp beforehand because it’s so commonly used, and is hard to bootstrap without a phone number.

I’ve been revisiting the vintage 37signals blog post A Rant Against Maximization. I am by nature, a bit of a maximizer: I put a lot of time into research about everything and have grown to be unfortunately picky. On the positive side, I don’t regret purchases very frequently and can be a good source of recommendations.

But this year I decided to starting figuring out how to spend money more effectively – to figure out what it’s useful for that gives me joy and long-term satisfaction. Partly inspired by Die With Zero and Ramit Sethi’s philosophies.

It has been a tough transition. I’m used to finding some price/quality local maximum, and nice stuff is always past that point. Leicas, luxury cars, fancy clothes, etc are usually 80% more expensive and 20% technically-better than the value-optimizing alternative.

To be clear, I bought a fancy bicycle, no BMWs for this guy. But it followed the same diminishing-marginal-utility arc as other fancy stuff. I’ve spent a lot of time thinking about whether I would have been better off choosing a different point on the cost/value curve.

But really: for most things there are a wide range of acceptable deals. We were not born to optimize cost/value ratios, and it’s not obvious that getting that optimization right will really bring joy, or getting it wrong (in minor ways) should make anyone that sad. And it’s a tragedy to keep caring about inconsequential costs just because of psychology. I’m trying to avoid that tragedy.

Anyway, the bike is awesome.

Also, with the exception of sports cars and houses, people in the tech industry and my demographic in the US likes to keep its consumerism understated. For example, an average tech worker in San Francisco tends to look clean-cut middle class, but going up the stairs from the subway you’ll see a lot of $500 sneakers. There are acceptable categories of consumerism – you can buy a tremendously oversized house and get very little flack for it (maybe the home is a good investment, though I have my doubts), and a big car (bigger the better, to protect your family in crashes with other big cars, apologies to the pedestrians). Uncoincidentally, I guess, these are also the two purchases that in the US are usually financed, or in other words, leveraged.

I think a weird thing that’s happening with capitalism right now is that I want brand consistency more than ever. It’s come up in conversations with friends, too, how brands like OXO, IKEA, Costco or Muji are starting to feel like an escape, rather than a luxury. Even Target, in-store, is kind of a place where you can, mostly, buy a thing that is fine.

It’s mostly because of the Amazon effect. Amazon went from a store to a ‘marketplace’ of drop-shipped nameless junk that takes too long to wade through. Alibaba was this kind of experience before Amazon even thought about it. Walmart is the same sort of experience now. It’s an overwhelming and complicated experience: I’ll look at t-shirts, and for the same ‘brand’, there are multiple ‘sellers’ from whom Amazon dynamically picks to serve my sale. The color blue is two cents cheaper than green. For anything more complicated than a t-shirt, the odds of it being a counterfeit are fairly high.

I don’t want to engage in the supply chain this much, and I don’t think most people do either, even if it enables incredible new levels of cheapness.

To some extent, Amazon has to be a reflection of consumer preference: buying more stuff, faster, for the cheapest price possible. And e-commerce is a hard industry.

But I strongly believe that there’s an opportunity for a brand like Costco’s ‘Kirkland’ or OXO to become the standard place for middle-class people to buy stuff. Paying 5-10% more for something with better odds of being genuine and high-quality, and for a less overwhelming junk-pile buying experience… there’s something there.

Update: Folks at GitHub wrote a fast BPE implementation in Rust, which is fast, relatively simple, and algorithmically clever.

Today in stories from software running in production.

At Val Town, we implemented Semantic Search a little while ago. It uses an OpenAI vector embedding model to turn some text into a vector that can be used to find similar bits of content. It’s nifty!

Like the rest of OpenAI’s endpoints, the vector embedding model endpoint costs money, and has limits. The limits and pricing are expressed in terms of tokens. So we truncated our input text by tokens. The recommended way to do this is to use tiktoken. Here’s a bit of Python and some output:

import tiktoken
import timeit

encoding = tiktoken.get_encoding("cl100k_base")

for len in [100, 1000, 10000, 100000]:
    print(
        len,
        timeit.timeit(lambda: encoding.encode("a" * len), number=10))

Output:

➜ python hello.py
100 0.00041395798325538635
1000 0.003758916980586946
10000 0.3689383330056444
100000 39.23603595799068

You might have noticed that, while the len increasing by one decimal point, the time it takes to compute the encoding is increasing more. It’s superlinear. Bad news!

This is shared behavior between the TypeScript and Python parts of the module, and has been reported and people are trying to fix it, but at the core the problem is that Tiktoken implements Byte Pair Encoding in a simple way that is not performant, and gets scary in this pathological “repeated character” case.

Maybe this’ll get fixed upstream. In the meantime, you might want to truncate text by characters before truncating it with a tiktoken-based encoder. This problem is similar to regex backtracking attacks in my opinion: it can be both triggered accidentally as well as intentionally, and cause a scary pinned main thread in production.

Thorsten Ball wrote a really thoughtful piece about LLM-based coding assistants. A lot of accomplished, productive engineers use them. Adaptability is important for software engineers. Why not dive in?

I’m not passionately against coding assistants, but neither do I use them on a daily basis in my code editor. I use Claude occasionally to write some shell script or tell me how awk works. A lot of folks I respect are using these coding assistants all day. There’s obviously something there.

Without laying out the full cheesy analogy to the recent political events, needless to say we should all be vividly aware that people are bound to make decisions based on a cloud of emotions, prejudices, assumptions, cultural narratives, and reckons.

So, taking Thorsten’s well-argued piece as true, which I think it is - LLMs are pretty good at a lot of things and are a legitimate labor-saving device used by professionals - let me poke around at some of the narratives.

LLMs as acceptance and lubricant for the crushing weight of complexity

One of the primary vibes of modern-day engineering is the absolutely overwhelming complexity of everything. If you work on servers, it’s the sprawling mess of Kubernetes. Or you work on client-side applications and it’s React. Or you swear off React and embrace the web platform, which relies on the gigantic Chromium codebase. Did you know that V8 has three different optimizing compliers within it? If you’ve held on to the simplicity of CSS, well, it’s now a much more complex language. CSS has nesting and variables and it’s getting the masonry layout thing soon, too. Even if you’re working on low-level compiler and language design, you might have to use LLVM, which is gigantic and would take a lifetime to fully understand.

I mean, are we skidding toward a Butlerian Jihad or is this more of a ‘late capitalism’ feature? I don’t know. A topic for a different micro-blog post.

But LLMs are of a piece with all of this. They write both English and code with no shame or attempt at concision. So much of it, too much! You might want to use another LLM to understand the code that you’ve written. And another to refactor it. Say what you will about the mountains of technical debt at your average startup, but that technical debt was hard-won, written by hand over the course of years. With modern tools, we can greatly accelerate that process. Do we want to accelerate that process?

And not only do LLMs make it easy to write complicated software, putting LLMs into the software lets you write fantastically more complicated user-facing products as well. “The computer telling users to commit self-harm” is a bug that we had not even considered to until recently, but now we have to. Traditional software had bounded abilities and limitations, whereas LLM-integrated software is truly unlimited in both directions.

LLMs as another reason to wonder about where productivity gains go

Some people who are very hyped up about LLMs are excited about how they are such a force multiplier. They are excited to automate their jobs and finish everything 5x faster.

This raises a lot of questions:

• If you are able to work a lot faster, can you work less?
• For the few folks whose wages are based on their output rather than the time they spend at a job, will those wages go up when their output goes up? What about after everyone else adopts the same labor-saving device?

The answers seem iffy here. There’s lots of interesting research and a lot of headlines about how automation might cut labor demand. At the very least, once a tool like LLMs becomes ubiquitous, it’s not a unique advantage, by definition.

Remember how John Maynard Keynes thought that we’d work 15 hours a week because productivity would make long hours unnecessary?

LLMs as GPS units of the future

My sense of direction, already weak, is atrophied because I constantly use GPS. I use it when I’m walking around my own neighborhood. It’s silly.

What else is like this? Freeform writing: thanks to this blog, I write for the “public” (hello ten people who see this in RSS) pretty often. If I didn’t have this, I probably wouldn’t be nearly as comfortable with writing. The skill would atrophy.

What would I learn if I used LLMs often? Probably something. What would I forget? Probably a bunch, as well. The kind of refactoring that I do by hand on a daily basis requires some mental exercise that I’d probably forget about if I was coding-by-instruction.

LLMs make you think like a manager

Programming is antisocial. Not entirely, but it’s sort of a solitary exercise. It’s nice to have focus time, to face the blank sheet of paper and figure out solutions.

LLM assistants are chat interfaces. You beg the computer to do something, it does something wrong, you ask it to fix the problem. You’re managing. You’re acting as a manager, and chatting.

Sometimes that’s a useful arrangement. Heck, ELIZA was occasionally useful even though it was a tiny, scripted experience. Talk therapy can be good, and chat interfaces can be productive. But at least for some of us, the need to keep asking for something, trying to define the outputs we want and clarify and re-clarify: this is managing, and sometimes it’s nicer to write in solitude than to beg the computer like an micromanager.

So, in summary: maybe people shy away from copilots because they’re tired of complexity, they’re tired of accelerating productivity without improving hours, they’re afraid of forgetting rote skills and basic knowledge, and they want to feel like writers, not managers.

Maybe some or none of these things are true - they’re emotional responses and gut feelings based on predictions - but they matter nonetheless.

It doesn’t matter what kind of database you use, eventually you will learn to fear it.

With Val Town we’ve been using Postgres, and a lot of our foreign keys have cascade constraints set. So, for example, a user in the users table might have some settings in the settings table, and because the settings table has a user_id foreign key and a cascade relationship, if that user is deleted then the settings will be too, automatically.

We were deleting some ancient data from a table like this, and the deletes took forever. Deleting a single row would hang for minutes and take down the prod database. Any guesses?

ON DELETE CASCADE is kind of like an implicit select, on a different table, that runs every time that you delete a row. So, if you have a foreign key relationship that has a huge or poorly-indexed table on one side of it, guess what: each and every deletion will have huge unexpected overhead.

I’m not a big fan of CASCADE relationships for this reason, but the foreign key-ness might be enough to trigger this performance bottleneck.

It is a good reminder that a lot of the very good, helpful magic in relational database is also implicit behavior. Not only can triggers and constraints affect the behavior of operations on a table, the triggers and constraints on another table’s foreign keys can affect the performance of a table.

I found myself browsing through flamework, a Flickr-style framework developed by some of the developers who developed Flickr, including the legendary Aaron Straup Cope and Cal Henderson, who went on to co-found Slack and presumably make a billion dollars. And I was reading Mu-An’s thing about JavaScript. She is legendary as one of the brains behind GitHub and tasteful and clever uses of HTML, JavaScript, and Web Components. And following along with Alex Russell critiquing Bluesky’s frontend.

I’ve been overall bad, because I, like you am living through a bad era and throwing yet another take onto the pile is cringe for both of us - who am I to speak, who are you to listen? Anyway:

• React, on a daily basis, is livable but annoying. The level of complexity is sky-high, even when I spend a lot of energy trying to limit that complexity.
• On the other hand, the level of complexity of web applications is pretty high. User expectations are different, I keep saying. Flickr was fantastic, but it was not a realtime-updating website that optimized for the browsing habits of the youth, who spend a half-second on most content. I love GitHub, but there is a reason why people are using Linear more and more: Linear feels like a realtime desktop application while GitHub feels like a website.
• I just can’t summon the clarity or oomph required to critique this stuff anymore. Everything is, like, a trickle-down consequence of requirements and culture and history, man! Pointing fingers at some software developer or whatever, is neither all that accurate nor that effective. What’s the point? To make people feel bad? Most people are trapped in their technical decisions by several layers of management anyway. And people already feel bad!
• Man, the web platform is not that great. I keep wanting it to be great, but half the time when I think that knowing about some HTML element will save me from having to use a React thingamabob that adds 50kb to my bundle… that HTML element just isn’t it, man! I need to style those select elements, or lazy-load that details element, or implement some implementation-wise horrible but essential-for-the-product scroll or focus or style experience which is just a little too much to implement with just CSS hacks.
• Honestly, the parts of GitHub that have moved from Ruby on Rails to React are mostly, in my experience, worse now. GitHub issues might be slightly fancier with a few extra features, but there are noticeable loading flickers and plenty of new bugs, like hovercards that don’t go away.
• That said, and I have to keep repeating this, user expectations are changing. People are used to apps, not websites. They are surprised if every view that they see is not realtime-updating. Linear and Pierre see this and are making modern-style alternatives with realtime subscriptions and local-first stuff and heavy client apps.
• I don’t think everything should be a React app! I want more things to be like Flickr used to be, and GitHub used to be. But at the same time, I don’t see an obvious way out of the current dynamics. Yelling is popular but the track record isn’t very good. Being quietly annoyed about the web’s descent into complexity, my preferred approach, doesn’t work very well either. A few organizations are bucking the trend - Kagi, for example, has good JavaScript-lite frontends. Reddit has gone web components and it seems like an improvement.

I haven’t found the perfect setup yet, but here’s what’s been working recently:

• PragmataPro Mono Liga is the fancy, sort-of-expensive programming font that I’ve been using for the last four years. It’s working great. Sometimes there are exotic nerd font icons it lacks, but that is an acceptable issue: mostly nerd fonts are a thorn in my side and I try to turn those icons off in all the software I can.
• I switch between vim-paper and catpuccin as my Neovim themes. Mostly vim-paper lately, which is a light-background theme with minimal color variation. With how fancy neovim is getting, it’s hard to find themes that support all the highlight groups.
• I’ve been using LazyVim but with some features turned off: alpha, headlines, bufferline, render-markdown, and pairs I turn off. I add oil for a super simple file browser, noneckpain to center my buffers, and gitlinker to get permalinks to line ranges of code.
• The main struggles right now are about toggling code formatting on & off, and when vstls starts getting out of sync with the TypeScript language server. As I mentioned in my Zed review, the difficulty of developing TypeScript code outside of VS Code is mostly Microsoft’s fault, not the fault of open source maintainers.

Today I noticed some dead code in the application I work on, Val Town. We have a bit of technical debt: a medium amount. Building technology in startups is all about having the right level of tech debt. If you have none, you’re probably going too slow and not prioritizing product-market fit and the important business stuff. If you get too much, everything grinds to a halt. Plus, tech debt is a “know it when you see it” kind of thing, and I know that my definition of “a bunch of tech debt” is, to other people, “very little tech debt.”

Anyway, I wanted to find unused code, and found Knip. And, wow - it worked on the first try. Sure, in our complex monorepo it flagged some false problems, but most of what it uncovered was valid, setting it up was very easy, and it’s very fast. I am totally impressed.

I was able to prune more than 5 unused dependencies and a few hundred lines of code in 20 minutes by using knip’s suggestions. Bravo!

I am not a regular user of the Arc browser, as I’ve mentioned before. But that’s been for personal-preference reasons: I close my tabs very often, so the UI that was oriented toward lots of tabs, open for a long time and neatly organized, wasn’t the tradeoff that I wanted. I definitely wrote a tweet or two about how Arc, despite being a very new browser, was not a new browser engine but another Chromium project - something that’s technically true and annoying but irrelevant for most people. See also, all the projects that say that they’re an “operating system” but end up being Android skins or layers on Linux.

Anyway, The Browser Company is no longer going to focus on Arc and they’re going to build something new. There’s a good YouTube video about this from their CEO. I have some thoughts.

1. The user-base size that consumer-facing startups need to meet expectations is just unfathomable because their average revenue per user rounds to zero. Arc is wildly successful but it needs to be just outstandingly successful to succeed. Right now it’s very popular in tech circles, but consumer software tech startups need to be popular like Amazon or Walmart.
2. One of the most interesting take-aways from the video is the idea that “Arc is a power-user tool” with a learning curve. This is true in the mildest sense, but that is enough. The median user’s appetite for complexity is near-zero. Apps should have one or two main actions per screen. Swiping is fine. Forms shouldn’t have more than one input. AI is perfect for this: AI interfaces are just a button and a textfield, but the level of gratification and sense of power they give is infinite. See The World Beyond Your Head for why I’m so obsessed with this idea.
3. The Browser Company has been doing an industry-leading job of being human. Their YouTube channel is great. Josh has ‘looked and talked like a human’ since way before Mark Zuckerberg got his gold chain and personal stylist. The product has “done well on social.” This had enormous upside, and I think that it has really affected the reaction to this news about Arc: when Google discontinues products, people don’t tweet at Sundar Pichai, but people feel connected to this company, for better or worse. The Browser Company feels like a collection of normal, nice people, which is part of what makes the news tricky: VC-backed startups are not reasonable, normal, or happy with good results. They need to shoot the moon, not create a fairly successful, well-loved macOS app. This applies to all of them, including the ones that I’ve worked at, and it’s just part of the deal. But it’s a set of expectations that no ordinary person would arrive at if they were building a company from scratch without these incentives.
4. I had a call with Josh before The Browser Company had started development, and there was just a general direction that things could be better, not any particular features or specific ideas in mind. That was a reason why I didn’t engage that much further at that point, but I’m astounded by how many good ideas did come out of The Browser Company for Arc. You can’t manufacture inspiration but they did a really good job of being both creative and methodical and creating something compelling. This makes me pretty optimistic for their next product, though I’m not really the target market for AI-centric computing, just as I’m not the target market for tab-hoarding user interfaces. I am the target market for high-end bicycles, what else, I’m not sure.
5. There are a bunch of ‘alt browsers’ like Horse Browser, Zen, and surely others that I’m forgetting. Horse is mostly Pascal, which is cool. Really great macOS software has, for decades, been created by companies shaped a lot different from modern venture-backed startups. Panic, iA, Alfred and Omni, for example, are very unique companies with traditional business models that produce software comparable to that made by startups aiming for the multi-billion valuations. There are different ways to win the race.

Earlier this year I wrote about how, despite Web Components having so many public advocates, there were few success stories of the technology in the wild. And even some incidents of companies moving away from Web Components to React, like GitHub has been doing.

Well: Reddit is the success story I can point to now. When I wrote the blog post in January, they were feature-flagging their Web Components frontend only for some users or sessions. But now it’s the default. As a casual Reddit user, I think the WC version of Reddit has been an improvement in my experience: the site loads fast, has good accessibility properties, and seems stable to me. I’d love to hear from folks who work at Reddit how the transition has gone, from an engineering culture & DX perspective!

I’ve bought too many domain names in my time. I think that now, more than, ever, that’s a mistake.

Established domains are heavily favored by search engines - Google says that they aren’t, but Google was lying about that. The result is that, if you use a domain for a while and it gets linked from other websites, it will be a target for domain squatters, who will grab it instantly if you ever stop using that domain name and put some scam or gambling content on it. You’ll feel bad, people who link to your site will feel bad, etc. This happened to me today: I had been using a full domain for the Simple Statistics documentation site, but moved it to GitHub Pages to simplify the wide array of domains I have to keep track of. And immediately that domain was squatted.

I don’t have a solution to this problem. It’s how the web works. But if I were to do it again, I’d use subdomains of macwright.com more often, rather than having a lot of individual domains.

Shop notes from working on my project about bikeshare data:

• The ‘cool kid’ formats are Zarr, DuckDB, and Parquet. Working with them has been kind of a struggle: you can use DuckDB to query its own DuckDB database format, or to query data in Parquet files. The two options are sort of in contention - there are reasons to use Parquet with DuckDB and reasons not to.
• Annoyingly, DuckDB really tailors its data ingestion to a situation in which you already have CSV files or Parquet files as input. In this case, I have neither - I have to do some transformation from a bunch of nested JSON files into a tabular format. CSV isn’t the worst format, but I fear what happens to my data types as they travel between places: it’s a bit annoying. “Creating a file of data” really feels like a corner case for these things… the documentation always includes some example of setting up a DataFrame with some hard-coded array of 10 numbers: what if I have some Python code that’s transforming 100 million rows - just… a big array? So far the answer has been yes, but it feels very wrong, given the one weird trick of all Python libraries is to do all the internal hard parts in Rust, C++, or Fortran.
• The tools for working with this stuff are pretty overlapping: you can write a Parquet file with Polars, or pyarrow, or Pandas. I’ve been using Polars because it’s the cool new thing. That could be a mistake but it hasn’t bitten me yet.
• Right in the middle of rebuilding on parquet, the folks at Earthmover released Icechunk, which is their multi-dimensional array storage. I’ve been tinkering with Xarr, too, as an option, and Icechunk seems like a perfect option.
• Overall, I find it kind of frustrating how this datascience tooling stuff doesn’t give that much detail about how access patterns work. Like, when I use Clickhouse or Postgres, there’s pretty good documentation and third-party writing about how fast it is to query on one column or another, how indexes work, etc. It’s harder to find that kind of documentation for this stack: how do I save a Parquet file that’s easy to partially read based on a range of one column or another? I know that some of my queries are slow in DuckDB, but how do I make them faster? It’s a bit of a mystery - still learning there.

A year ago, Ink & Switch dropped a blog post about Embark, a prototype for a trip planner, in which places and times could be embedded in text and the planner could easily compare different transit mode options. There were a lot of interesting ideas in there – some of which are more relevant to Ink & Switch’s long-running research areas than to my personal interests.

I got the chance to play with Embark, and it was inspirational. I’ve been wanting to build something like it ever since. I did come away with the conclusion that I wanted a plain-text Markdown representation, not the structured representation in Embark. The structured text editor was too magical and felt kind of hard to pick up. The multi-view system also was a little unintuitive, for me.

I’ve written two abortive prototypes so far. I haven’t gotten far, mostly because of a few basic stumbling blocks. Here are some of them:

Pasting Google Links

As much as I love OpenStreetMap, Mapbox, and so on, I do most of my POI searching in Google, and I use Google’s system for saving places I want to go to. Ideally, this tool would work well with copy & pasting a URL from Google Maps and putting it on the map. This is a good example of something that seems so easy, and is nearly impossible. Google Maps URLs are extremely opaque. Scraping Google Maps HTML for those pages is virtually impossible. There are no identifiers anywhere in the whole thing. Google has Place IDs for their Geocoding API, but these aren’t exposed to normal users anywhere.

So, most likely I’ll have to just integrate the Google Geocoder into the project and store the place_ids, and maybe create some single-use domain that is like https://geospatiallinkredirector.com/google/place_id that points to Google Maps.

Directions

I think that using the Transit API is the clear answer here. Mostly I care about city trips anyway, and it handles that super well.

Editing

I think maybe this should live as an Obsidian plugin, so that I don’t have to build storage, or editing fundamentals. There’s some precedent of enhanced links in Obsidian, the github-link plugin that I already use.

But there are some sharp edges for very advanced Obsidian plugins: most of the cool plugins use fenced code blocks for their custom content. Interacting with the rest of the Markdown document is a bunch harder because Obsidian has its own ‘hypermarkdown’ document parser, with an undocumented structure. I’d possibly have to re-parse the document using a different Markdown parser for this project, which makes me a little nervous.

POIs

The assumption that most POIs will be from Google will probably make this project more straightforward, but I’ve been getting nerdsniped by the other alternatives. Like, is the Overture GERS ID system useful here, and should I use its nicely-licensed open source dataset? What about SafeGraph IDs? I don’t know. I should probably just use Google Maps and call it a day.

Realistically, the biggest problem with getting this side project done is my lack of time allocated to side projects recently. I’m spending my 9-5 focused on Val Town, and my weekends and afternoons are dedicated more to reading, running, social activities, and generally touching grass, rather than hacking on web projects. I miss the web projects, but not that much.

Updated to include links to documentation that does exist on the react.dev site. I do think that it’s hard to discover and this concept is underemphasized relative to how important it is for working with React hooks, but as the team correctly points out, there are some docs for it.

React hooks… I have mixed feelings about them. But they are a part of the system I work with, so here are some additional notes that it’s weird that some of the React docs don’t cover. Mostly this is about useEffect.

The “rule of useEffect dependencies” is, in shorthand, that the dependency array should contain all of the variables referenced within the callback. So if you have a useEffect like

const [x, setX] = useState(0);
useEffect(() => {
	console.log(x);
}, []);

It is bad and wrong! You are referencing x, and you need to include it in the deps array:

const [x, setX] = useState(0);
useEffect(() => {
	console.log(x);
}, [x]);

But, this is not a universal rule. Some values are “known to be stable” and don’t need to be included in your dependencies array. You can see some of these in the eslint rule implementation:

const [state, setState] = useState() / React.useState()
              // ^^^ true for this reference
const [state, dispatch] = useReducer() / React.useReducer()
              // ^^^ true for this reference
const [state, dispatch] = useActionState() / React.useActionState()
              // ^^^ true for this reference
const ref = useRef()
      // ^^^ true for this reference
const onStuff = useEffectEvent(() => {})
      // ^^^ true for this reference
False for everything else.

So, state setters, reducer dispatchers, action state dispatchers, refs, and the return value of useEffectEvent: these are all things you shouldn’t put in dependencies arrays, because they have stable values. Plus the startTransition method you get out of a useTransition hook - that’s also stable, just not included in that source comment.

Honestly, this is one of the things that annoys me most about hooks in React, which I touched on in my note about Remix: the useEffect and useMemo hooks rely heavily on the idea of object identity and stability. The difference between a function that changes per-render versus one whose identity stays the same is vast: if you plug an ever-changing method into useEffect dependencies, the effect runs every render. But React’s documentation is underwhelming when it comes to documenting this - it isn’t mentioned in the docs for useEffect, useState, or any of the other hook API pages.

As Ricky Hanlon notes, there is a note about the set method from useState being stable in the ‘Synchronizing with Effects’ documentation, and a dedicated guide to useEffect behavior. This is partly a discovery problem: I, and I think others, expected the stability (or instability) of return values of built-in hooks to be something documented alongside the hooks in their API docs, instead of in a topical guide.

And what about third-party hooks? I use and enjoy Jotai, which provides a hook that looks a lot like useState called useAtom. Is the state-setter I get from Jotai stable, like the one I get from useState? I think so, but I’m not sure. What about the return values of useMutation in tanstack-query, another great library. That is analogous to the setter function and, maybe it’s stable? I don’t know!

It’s both a pretty critical part of understanding React’s “dependencies” system, but it’s hard to know what’s stable and what’s not. On the bright side, if you include some stable value in a dependencies array, it’s fine - it’ll stay the same and it won’t trigger the effect to run or memo to recompute. But I wish that the stability or instability of the return values of hooks was more clearly part of the API guarantee of both React and third-party hook frameworks, and we had better tools for understanding what will, or won’t change.

Updated 2024-11-19

There’s a lot of bike infrastructure in NYC which is usable for getting around, but which parts are so good you can really enjoy yourself on a bike? Most of the spots are obvious, but here they are, so far:

Brooklyn

• The path along Belt Parkway, and how it continues on Jamaica Bay Greenway, all time MVP bike route. Smooth, totally separated from the road, and really well-maintained. Kinda reminds me of the Anacostia Riverwalk Trail from DC, but much longer.
• Prospect Park. There are always road cyclists doing hot laps, but as long as they don’t give you any trouble, it’s just a really damn nice park, and constantly filled with life.
• Eastern Parkway and Ocean Parkway, though both require a lot of stopping.
• 4th Ave – as a way to access the Belt Parkway, mostly. Far enough south, the tire shops start to take over the bike lane, so it’s not that great, but still, it’s a long, useful, marked bike lane. But why are there so many tire shops? How many tires does a car go through every year?
• Flushing Ave / Kent Ave, which takes you up to Williamsburg. It’s a bike lane and feels like “the city,” but it’s relatively safe and there’s good people-watching.

Queens

• Forest Park: nice smooth, car-free roads.

Manhattan

• The Hudson Greenway / Empire State Trail. Often flooded with tourists and people who have not ridden a bicycle in years re-learning how to ride a bicycle. But once you get north of the Intrepid Museum, it starts to get much nicer and smoother.
• Central Park. I’ve honestly only ridden it a few times, but… it’s Central Park.

Long Island

• The trail alongside Wantagh State Parkway.
• The Massapequa Trail.

New Jersey

• Henry Hudson Drive in Palisades Park, accessible by biking up the Hudson Greenway and crossing the George Washington Bridge. Blissful smooth roads with minimal car traffic. Super popular with cyclists.

What am I missing? I want to ride all the (nice) paths!

More people writing about this!

• See Brian Cobb’s route from Downtown Brooklyn to Jackson Heights.

I think that even the most overhyped technology usually delivers some benefit to the world. And often succeeds quietly, long after the hype has died. Recent examples include 3D printing, which has found massive success in prototyping, medical applications - a friend had a filling 3D-printed right in his doctor’s office - and niche consumer items. Etsy is awash with 3D printed lamps, some even that I own. Or drones, which are now used all the time in news coverage, on job sites, and by people filming themselves hiking.

I suspect that even if Augmented Reality doesn’t take off, it’ll leave in its wake big advances in miniaturized projectors, improved optics, and scene understanding algorithms in computer vision and ML. The internet of things didn’t really work and most people’s Alexa speakers are only used for setting alarms, but the hype-wave did justify the deployment of much-needed technologies like IPv6, Zigbee, and BLE.

So, the thought is: none of this applies to crypto. It didn’t work, and it also didn’t fund the development of any lasting technological advance. There’s no legacy. The crypto industry’s research didn’t create new foundations for building decentralized databases. Next-generation cryptography kept rolling on, and, as far as I know, none of it owes much to the cryptocurrency industry. Nothing new has been discovered about economics: as Matt Levine says, “One thing that I say frequently around here is that crypto keeps learning the lessons of traditional finance at high speed.” It’s hard to name anything of value that came out of this hype wave. We incinerated all that investment, and randomly redistributed some wealth, and… what else?

The best I can come up with is the popularization of zero-knowledge proofs, which play some role in Zerocash and Ethereum but are a fundamental advance in security and have other applications.

Maybe there’s something I’m missing? But it reminds me of the end of Burn After Reading: “What did we learn? We learned not to do it again.”

My workspace gets pretty bright during the day, and I find it easier to work with a light color scheme neovim config then. But syncing everything up has been a hassle. Notes on how I get this to work, because I’m far from a neovim & lua master:

This is all using LazyVim, but may differ for you.

iTerm

I have an iTerm profile called “Light” that has a light background theme.

Neovim background

I want Neovim to match iTerm’s light or dark setting. So I pull the $ITERM_PROFILE environment variable that iTerm sets and use it to switch neovim’s background option:

In ~/.config/neovim/lua/config/options.lua

opt.background = vim.fn.getenv("ITERM_PROFILE") == "Light" and "light" or "dark"

Neovim color scheme

My default color scheme - aurora - doesn’t look that great in light mode, so I switch to papercolor. So, in ~/.config/neovim/lua/plugins/color.lua, another switch based on the iTerm environment variable:

return {
  {
    "LazyVim/LazyVim",
    opts = {
      colorscheme = vim.fn.getenv("ITERM_PROFILE") == "Light" and "papercolor" or "aurora",
    },
  }
}

I’ve been interested in taxes for a while - The Deficit Myth, Triumph of Injustice, and Capital have been my favorite reads so far. A tax on unrealized capital gains has been in the media cycle recently, and I asked about it on Twitter, because people seem to oppose it for surprising reasons. I think one of the core questions is what are taxes for? The obvious answer is to fund government services, but there’s some useful detail under that:

To fund specific things

A lot of taxes that supposedly fund something, don’t. For example, a lot of people think that tolls, gas taxes, and generally “user fees” paid by drivers are the funding source for roads. They’re usually a funding source, but in a lot of states, they fund less than half of the budget for roads. This leads to a lot of darky humorous scenarios in which people complain about how cyclists aren’t funding something because they aren’t paying gas taxes or tolls, but the thing they’re talking about isn’t funded by those taxes anyway.

Real estate taxes paying for schools is a classic case, but even that is not entirely true, with local schools being funded by state, federal, and varies dramatically by state.

To encourage consuming some things and discourage others

Sin taxes are controversial but they’re definitely part of the picture: high taxes on cigarettes seem justified to me because that product causes untold deaths and doesn’t have much in the ‘pros’ column.

Or take New York’s long-delayed congestion pricing plan, a pigouvian tax on people casually driving into and around Manhattan. I’ve been at tables of people discussing whether the fee would just go into the MTA’s slush fund, or whether it would fund specific programs that they wanted it to - valid questions, but the other thing, the thing that is perhaps most important, is that it would discourage excess traffic in Manhattan. Fewer cars means fewer kids getting asthma from pollution, fewer drivers killing cyclists and pedestrians. It means faster buses and quieter streets. The revenue is great, but if all of the revenue went into a money-burning machine, there would still be reasons to implement the plan.

To shape the wealth curve

The approach from the MMT & Piketty corner, clumsily summarized, is that taxes are a way to redistribute wealth, as much as they are a way to generate tax revenue. MMT claims that the national deficit is fundamentally a value in accounting that the government can affect through issuing more currency: the only limiting factor is how much inflation is caused by that process. Money is created and destroyed by the government, and the demand for money is created by taxes. I think this is a very compelling theory.

One direction you can take this is that high taxes on the ultra-wealthy are essentially an end in themselves: it’s nice to use that money to build infrastructure or fund the national parks, but the decrease in the country’s gini coefficient enough to justify them.

This also generally makes sense arithmetically: the enormous amount of wealth held by the top 1% in America is staggering, but nothing in comparison to the national budget. The federal budget is measured in trillions, while our top billionaires are in the low hundreds of billions. It would be good, in my opinion, to tax those billionaires: inequality is bad in every way. But taxing them to pay for stuff is a different, and sort of less compelling justification, than taxing them to take some money from them, money that they could never spend in a hundred lifetimes and, despite all the media hype, are not donating fast enough.

I wanted to have a one-shot command to boot up and log in to a Linux container on my Mac. After some trial and error, the command is:

docker run -it --rm ubuntu

Maybe there are even quicker ways to do it, but that is pretty instant. I needed this today because I ran across a bot network on GitHub which was trying to get people to install a malicious .exe file. I wanted to poke around with the payload but limit the blast radius.

Note that [[Docker]] is not a great security sandbox, but it’s better than nothing. And poking around with a Windows payload in a Linux container on a macOS host is probably safe-ish.

Thanks Justin for telling me about the --rm option that gets rid of the machine after you’re done!

A long time ago I read Playing to Win, a book about winning arcade games. One of its key takeaways was the idea of a “scrub.” A summary definition:

A scrub is not just a bad player. Everyone needs time to learn a game and get to a point where they know what they’re doing. The scrub mentality is to be so shackled by self-imposed handicaps as to never have any hope of being truly good at a game. You can practice forever, but if you can’t get over these common hangups, in a sense you’ve lost before you even started. You’ve lost before you even picked which game to play. You aren’t playing to win.

The backstory is that the author of the book wins arcade games in boring ways sometimes, by exploiting basic advantages like doing the same movie a lot of times in a row. Other players think this is unsportsmanlike, but it doesn’t matter: the author wins.

Another story I think of is from a 2010 speech at Princeton by Jeff Bezos, which I’ll summarize…

Bezos is with his grandfather, who is smoking. He does some math, which he thinks will impress his grandpa, and figures out how much time each cigarette is taking off his life. His grandpa hears this and says “Jeff, one day you’ll understand that it’s harder to be kind than clever.”

This happened when Jeff Bezos was 10. Whether his legacy has been defined by kindness or cleverness I’ll leave up to the reader.

Ehrenberg suggests the dichotomy of the forbidden and the allowed has been replaced with an axis of the possible and the impossible. The question that hovers over your character is no longer that of how good you are, but of how capable you are, where capacity is measured in something like kilowatt hours—the raw capacity to make things happen. With this shift comes a new pathology. The affliction of guilt has given way to weariness—weariness with the vague and unending project of having to become one’s fullest self. We call this depression.

This is from The World Beyond Your Head, which blew my mind recently.

The AI had also written some copy. “Are you tired of starting your day with a bland and boring breakfast? Look no further than PureCrunch Naturals.” With a few more clicks, the man showed how the right combination of generative AI tools can, like a fungus, spawn infinite assets: long-form articles, video spots, and social media posts, all translated into multiple languages. “We are moving,” he said, “into a hyperabundance decade.”

This is from The Age of Hyperabundance from Laura Preston in n+1, which is extremely worth reading.

Do we focus on productivity and output more than we do on the ethical valence of what we’re doing? Is the abundant output of LLMs so attractive because it is the theoretical limit of maximum productivity / minimal meaning?

Some issues with neovim that I’ve solved recently that took me way too much time to solve so I’m writing here in the hope that others will encounter these suggestions…

Linting a whole project

Let’s say you add some eslint rule or switch linters in a project: you want to lint the whole project. This took me a long time to figure out how to do, because I expected full-project linting to connect to nvim-lint or the LSP system in Neovim. It turns out that the Language Server Protocol doesn’t really have a concept of full-project linting: VS Code does it by a weird hack. workspace-diagnostics.nvim tries to replicate that hack for Neovim, but I couldn’t get that to work.

The solution was overseer.nvim, which I installed via LazyVim and lets me run just npm run lint and it parses the results into diagnostics that I can browse via trouble.nvim. Works great. TSC.nvim does the same thing, but specialized for checking a project with TypeScript.

Searching for text in files in directories

I run neovim with no file tree - to toggle into the file view, I use oil.nvim, which shows a single level at a time. Nearly all of my navigation is via telescope.nvim, whether searching for filenames, or live-grepping the codebase. But in large projects I sometimes want a more precise search experience - for example, only searching for a string if it appears in a file in a directory.

The solution is the live-grep-args plugin to telescope:

return {
  {
    "nvim-telescope/telescope.nvim",
    dependencies = {
      {
        "nvim-telescope/telescope-live-grep-args.nvim",
        -- This will not install any breaking changes.
        -- For major updates, this must be adjusted manually.
        version = "^1.0.0",
      },
    },
    config = function()
      require("telescope").load_extension("live_grep_args")
    end
  }
}

It’s pretty easy to use: if you search for an unquoted string like const e, then it uses the default mode. If you quote that string, like "const e" then it starts treating it as a filtered query, so you can write "const e" app to only search for that string within the app directory.

Jim Nielsen riffs on iA’s writing about AI in Impressionist Blogging.

It will no longer be enough to blog in order to merely “put out content”. The key will be what has always been key to great blogging: expressing your unique, inimitable impression.

Blogging is expressing your impression. It’s deriving action from thought. Regardless of how much the AI sphere may not be giving thought to its actions, continued blogging in the face of that reality is deliberate action with thought — something only us humans can do. Blog. Blog against the dying of the light.

I find some comfort in this perspective. Though, through the years, I’ve made a conscious effort to make my writing kind of impersonal: for a long time, avoiding the first-person, avoiding any obviously attempts at having a writing style. But the humanity comes through whether I wanted it to or not.

I find it hard to avoid looking at the AI wave as kind of a battle: those using AI to pump out text are only able to do so because of the reams of handcrafted writing that is now reduced to ‘training data.’ The more AI-generated content that’s published, the worse the training data, because of that disintegrating-tape-loop self-annihilation sort of cycle. Despite my earnest efforts to ban the robots from this website, they’re definitely still using it as training data. As I wrote back in 2021): When you share something for free, the first thing that happens is that someone else will sell it.

But Jim - and iA - are finding some peace with the scenario, and I’m trying to as well. After all, my little corner on the internet isn’t going to wrestle these giant corporations to the ground. I’m going to keep writing and making things because it brings me joy, and I might as well find some way to do so without grumpiness.

Am I writing too much about AI? Tell me in the comments on Mastodon, I guess. I’ve found it pretty hard to figure out what kind of ‘vibes’ I’m putting out in the world or what I give the impression of focusing on the most.

Today I was working with part of Val Town where we had a form element, but we were using it with tRPC instead of hooking it up to Remix’s form magic. Remix cuts with the grain of HTML and unlocks the power of the web platform, letting you use the form element with native validation and smooth fallbacks to zero-JavaScript workflows. tRPC doesn’t. You can implement good UIs with tRPC, but it is pretty agnostic to how you submit a form, and most people are going to write JavaScript Abominations™️ for their form submit flows.

Anyway, I learned a new DOM API: requestSubmit. Here’s an example:

<form
  onSubmit={(e) => {
    e.preventDefault();
    // JavaScript…
  }}
>
  <textarea
    value={value}
    onChange={(e) => setValue(e.target.value)}
    onKeyDown={async (e) => {
      if (
        e.key === "Enter" &&
        !e.shiftKey &&
        e.target instanceof HTMLTextAreaElement
      ) {
        e.preventDefault();
        e.target.form?.requestSubmit();
      }
    }}
  />
</form>

I want this form to submit when someone presses Enter while in the textarea, and to run some JavaScript in the onSubmit handler. Maybe there are other ways to submit the form too, like a button element. All of those should just submit the form through the onSubmit handler.

My first guess was e.target.form.submit(), but that doesn’t go through the submit listener! Instead, requestSubmit has an interesting bit of documentation on MDN:

The obvious question is: Why does this method exist, when we’ve had the submit() method since the dawn of time?

The answer is simple. submit() submits the form, but that’s all it does. requestSubmit(), on the other hand, acts as if a submit button were clicked. The form’s content is validated, and the form is submitted only if validation succeeds. Once the form has been submitted, the submit event is sent back to the form object.

Turns out, requestSubmit lets you submit forms programmatically but as if they had been submitted by clicking a button. Nifty!

Today we noticed an issue in Val Town’s TypeScript support.

// The 'bug' was that source code like:
const x = <X>(arg: X) => {};

// Was getting formatted to this:
const x = <X,>(arg: X) => {};

This is an arrow function with a generic argument called X.

Interesting! Why would dprint want to insert a trailing comma in a generic? I discovered this fascinating issue.

Let’s say that the input was this instead:

export const SelectRow = <T>(row: Row<T>) => false;

Some people have probably guessed the issue already: this is colliding with TSX syntax. If that line of code were like

export const SelectRow = <T>something</T>;

Then you’d expect this to be an element with a tag name T, rather than T as a generic. The syntaxes collide! How is TypeScript to know as it is parsing the code whether it’s looking at JSX or a generic argument?

The comma lets TypeScript know that it’s a generic argument! If you check out some behavior:

• prettier adds the comma in a generic
• biome’s parser does not allow the generic without a comma
• dprint adds the comma
• typescript doesn’t parse it

So, if you’re using generics with arrow functions in TypeScript and your file is a TSX file, that comma is pretty important, and intentional!

For us, the issue was that we were relying on CodeMirror’s syntax tree to check for the presence of JSX syntax, and it was incorrectly parsing arrow functions with generic arguments and the trailing comma as JSX. One of CodeMirror’s dependencies, @lezer/javascript, was out of date: they had fixed this bug late last year. Fixing that squashed the false warning in the UI and let the <T,> pattern work as intended.

Looking at this and the history of the lezer package is a reminder of how TypeScript is quite a complicated language and it adds features at a sort of alarming rate. It’s fantastic to use as a developer, but building ‘meta’ tools around it is pretty challenging, because there are a lot of tricky details like this one.

What makes this even a bit trickier is that if you are writing a .ts file for TypeScript, with no JSX support, then generic without the extra comma is valid syntax:

const x = <X>(y: X) => true;

For what it’s worth, I don’t use arrow functions that often - I like function declarations more because of hoisting, support for generators, and their kludgy-but-explicit syntax. For example, arrow functions also have a little syntax collision even if you don’t consider TypeScript: you can generally omit the {} brackets around the arrow function return value, like

const x = () => 1;

But what if it’s an object that you’re trying to return? If you write

const x = () => { y: 1 };

Then you may be surprised to learn that you aren’t returning an object like { y: 1 }, but you’ve defined a labeled statement and the function x will, in fact, return undefined. You can return an object from an arrow function, but you have to parenthesize it:

const x = () => ({ y: 1 });

So I kind of like function declarations because I just don’t have to think about these things that often.

Thanks David Siegel for reporting this issue!

For many years now, I’ve considered “democratizing programming” as a major personal goal of mine. This has taken many forms - a long era of idolizing Bret Victor and a bit of following Alan Kay. The goal of making computing accessible was a big part of why I joined Observable. Years before joining Observable, I was making similar tools attempting to achieve similar goals. I taught a class on making games in computers. At Mapbox, we tried to keep a fluid boundary between engineers & non-engineers, as well as many resources to learn how to code, and many people successfully made the jump from non-coding jobs to coding jobs, and vice-versa.

But what does it really mean to “democratize programming?” It’s a narrowly technical goal that its adopters imbue with their own values. Most commonly, I think the intents are:

• Programming is a beautiful experience, a way to learn, an end in itself, something that everyone should have as a way to think.
• We need more programs and we don’t have enough programmers to write them. The supply/demand imbalance between the number of skilled programmers versus companies who need them has been persistent. Maybe more bootcamps or more college grads is not the solution and better tools is.
• Software should be more like programming. Modern software is inflexible and simple programming-like interfaces are the way to give users more power. For example, Photoshop macros or HyperCard-like interfaces are the way forward.
• Programming jobs are nice and it would be good if more people could access them.

Those are the intents that I can think of impromptu – maybe there are others; let me know. For me, I came to believe most strongly in the last one: it’s useful to make programming jobs accessible to more people.

Basically I think that material circumstances matter more than ever: the rent is too damn high, salaries are too damn low. Despite the bootcamps churning out new programmers and every college grad choosing a CS/AI major, salaries in the tech industry are still pretty decent. It’s much harder to land a job right now, but that is probably, partially, an interest-rate thing.

The salaries for societally-useful things are pretty bad! Cities pay their public defenders and teachers poverty wages. Nonprofit wages at the non-executive levels are tough. And these aren’t easy jobs, and don’t even earn the kind of respect they used to.

So, a big part of my hope that more people have the ability to become programmers is just that my friends can afford their rent. Maybe they can buy houses eventually, and we can all live in the same neighborhood, and maybe some folks can afford to have kids. Achieving that kind of basic financial security right now is, somehow, a novelty, a rarity, a stretch goal.

Which brings me to the… ugh… LLM thing. Everyone is so jazzed about using Cursor or Claude Workspaces and never writing code, only writing prompts. Maybe it’s good. Maybe it’s unavoidable. Old man yells at cloud.

But it does make me wonder whether the adoption of these tools will lead to a form of de-skilling. Not even that programmers will be less skilled, but that the job will drift from the perception and dynamics of a skilled trade to an unskilled trade, with the attendant change - decrease - in pay. Instead of hiring a team of engineers who try to write something of quality and try to load the mental model of what they’re building into their heads, companies will just hire a lot of prompt engineers and, who knows, generate 5 versions of the application and A/B test them all across their users.

But one of the things you learn in studying the history of politics is that power is consolidated by eliminating intermediate structures of authority, often under the banner of liberation from those authorities. In his book The Ancien Régime and the Revolution, Tocqueville gives an account of this process in the case of France in the century preceding the Revolution. He shows that the idea of “absolute sovereignty” was not an ancient concept, but an invention of the eighteenth century that was made possible by the monarch’s weakening of the “independent orders” of society—self-governing bodies such as professional guilds and universities. - The World Beyond Your Head

Will the adoption of LLMs weaken the semi-independent power of the skilled craftspeople who call themselves “programmers”? And in exchange for making it easy for anyone to become a programmer, we drastically reduce the rewards associated with that title?

Some creative jobs maybe will go away, but maybe they shouldn’t have been there in the first place. - Mira Murati

Is the blasé attitude of AI executives toward the destruction of skilled jobs anything other than an attempt at weakening skilled labor and leaving roughly two levels of society intact, only executives and interchangeable laborers? Of course, you can ask what do executives do to keep their jobs, but at some point that is kind of like asking what do kings do to keep their jobs in the ancien régime: the king is the king because he’s the king, not because he’s the best at being the king.

Anyway, I don’t want my friends to move to the suburbs because the rent is too high and the pay is too low. Who is working on this?

In 2018, I wanted to create a book identification code translator to help me manage my self-hosted reading log. I built it at bookish.tech and it would translate between ISBN, OCLC, LCCN, Goodreads, and OLID identifiers.

It broke recently because OCLC’s Worldcat redesigned their website, and bookish relied on scraping their website for well-formed microdata in order to pull information from it.

The new Worldcat website is a Next.js-powered piece of junk. It’s a good example of a website that completely flunks basic HTML semantics. Things that should be <details> elements are instead inaccessible React components. It doesn’t work at all without JavaScript. Bad job, folks!

I don’t like to link to Goodreads because of its epic moderation problems and quiet Amazon ownership. Linking to OpenLibrary is nice but they rarely have the books that I read, and it’s easy to look up ISBN codes on that website.

So, from now on, sadly: my books will probably just be referenced by ISBN codes. bookish.tech is offline and I abandoned an effort to turn it into a CLI. OCLC sucks.

I continue to hop between browsers. Chrome is undoubtedly spying on me and nags me to log in - why do I want to log in to a browser!? Arc is neat but vertical tabs waste space and tab sync is weird and I always end up with 3 Arc windows without ever intending to open a new one. Plus, it wants me to log in to the browser - why would I want to log in to a browser!? Orion has great vibes but the devtools are Safari’s, which might be fine, but I am very locked in to Chromium-style devtools. Firefox… I want to believe, but… I think it’s just dead, man. Safari is fine but again, the devtools lockin.

For now, I just want Chrome without the spying, so I’m trying out ungoogled-chromium. Seems fine so far.

🎉 Programming pro tip I guess 🎉

If you’re writing JavaScript, Rust, or a bunch of other languages, then you’ll typically have dependencies, tracked by something like package.json or Cargo.toml. When you install those dependencies, they require sub-dependencies, which are then frozen in place using another file, which is package-lock.json, or Cargo.lock, or yarn.lock, or pnpm-lock.yaml, depending on nitpicky details of your development environment.

Here’s the tip: read that file. Maybe don’t read it top-to-bottom, but when you install a new module, look at it. When you’re figuring out what dependency to add to your project for some new problem you have, look at the file, because that dependency might already be in there! And if you’re using modules from people you respect in package.json, reading the lockfile gives you a sense of what dependencies and people those people respect. You can build a little social graph in your head.

Now, dependencies, I know: it’s better to have fewer of them. Managing them is grunt-work. But few projects can do without them, and learning good habits around managing and prioritizing them is pretty beneficial. They’re also a reflection of a real social element of programming that you are part of if you’re a programmer.

Here’s a conundrum that I’ve been running into:

APIs

On one hand, a company like Val Town really needs well-maintained REST APIs and SDKs. Users want to extend the product, and APIs are the way to do it.

The tools for building APIs are things like Fastify, with an OpenAPI specification that it generates from typed routes, with SDKs generated by things like Stainless or so.

APIs are versioned - that’s basically what makes them APIs, that they provide guarantees and they don’t change overnight without warning.

Applications

On the other hand, we have frameworks like Remix and Next that let us iterate fast, and are increasingly using traditional forms and FormData to build interfaces. They have server actions and are increasingly abstracting-away things like the URLs that you hit to submit or receive data.

Though we have the skew problem (new frontends talking to old versions of backends), there is still a greater degree of freedom that we feel when building these “application endpoints” - we’re more comfortable updating them without keeping backwards-compatibility for other API consumers.

Other contraints

• dogfooding is essential: if your company doesn’t use your own APIs, it probably won’t catch bugs or notice areas for improvement in time.
• Performance is essential: we don’t want to make a lot of network hops when dealing with vital functionality.

Solve for X

So how do you solve for this? How do you end up with a REST API with nice versions and predictable updates, and a nimble way to build application features that touch the database? Is it always going to be a REST API for REST API consumers, and an application API for the application?

• At Val Town, we currently have a Remix server that powers the application, and a Fastify server that powers an API interface to the content. There are many places of duplication: you have specifically-optimized routes in Remix to show filtered views of vals or to create new resources, that are designed to fit into the application and its error handling. And you have routes in Fastify that expose the same sorts of resources, but are designed with more general patterns in mind. Sometimes they share database queries under the hood, but they are still separate.
• Companies like Stripe and (ha, to a lesser level) Mapbox build applications on the same APIs that they expose to the public, I think. For Mapbox, this meant that those API endpoints were pretty slow to update and held some applications back. Stripe apparently discovered the secret to fast API evolution while maintaining backwards-compatibility, but that kind of magic is not commonly available.
• We could hit the Fastify server from the Remix server, the “backend for frontend” pattern, but this would introduce some network hops to each request, and some adaptation from one to the other’s request shapes - forms in Val Town use FormData, mostly, and the Fastify server uses JSON. This seems a little complex, and if there was a new API feature in the Fastify server, we’d have to deploy it, generate a new TypeScript SDK with Stainless, update the SDK in Remix, and then implement the new request: I do not think this is a nimble enough pattern for an early-stage startup to move fast enough.
• We could support Remix’s API as the REST API. But this seems deeply weird: does anyone generate OpenAPI specs from Remix routes? And the URL structure of our website is meant to be user-friendly, whereas the URL structure of the REST API is meant to be explicit, computer-friendly, and to use nesting structure in a pedantic way.

In short, I’m not sure! We’re taking a some-of-each approach to this problem for now.

I wrote a little bit about how I published this microblog with Obsidian, and I recently published an Obsidian plugin. I’m a fan: I’ve used a lot of note-taking systems, but the only ones that really stuck were Notional Velocity and The Archive. And now, Obsidian.

Before Obsidian

Versus Notational Velocity / The Archive:

• I miss the native mac experience of those apps. Obsidian’s performance is pretty great, but every once in a while I’ll notice some behavior that reminds me that it’s an embedded web view.
• Obsidian has pretty good momentum and I think the odds of it sticking around for the long term are good. Versus Notational Velocity, which stalled and was restarted twice by different development teams, and The Archive, which I think has a very small dev team.
• Obsidian also succeeds in a solid iPhone app. I don’t write notes on my iPhone, but I like having them accessible. You can technically make this work with The Archive, but it didn’t work very well.

Anyway, to my Obsidian setup. It’s not fancy, because I have never found a structured system that makes sense to me.

Obsidian features that I don’t use

There are a lot of features that I don’t use:

• I don’t use the ‘graph view’ at all. It’s fun to look at it, but I’ve never found it, or any other graph view, useful, ever? I’ve written about this before, how some of these ‘second brain’ representations seem like a fantasy.
• I don’t use the Canvas feature at all. Mind mapping doesn’t work for me.
• I don’t use tags at all. I haven’t found anything that tags provide for me that I can’t get with search.
• Very rarely I’ll add an image to a note in Obsidian. I don’t use any other kinds of attachments. I use DevonThink for PDFs and documents and it is incredible.

I’ve turned off the graph view and tab view using combinations of the Hider and Minimal Theme settings plugins.

Theme & aesthetic

This is what my Obsidian setup looks like right now, as I’m writing this. I followed a lot of advice in Everyday Obsidian to get it there, so I don’t want to share everything that got me there, but anyway: it’s the Minimal theme, and the iA Quattro font. Like I said, the graph view, canvas, and tags aren’t useful to me, so I hide them. I usually hide the right bar too. I’ve been trying out vertical tabs, and am on the fence as to whether I’ll keep them around.

The Minimal Theme is a godsend. Obsidian by default looks okay-ish - much better than Roam Research or LogSeq, but with the Minimal Theme it looks really nice, on par with some native apps.

Organization

I use pretty rough folders with broad categories - notes on meetings, trips, finance, and so on. There are some limited subfolders, but I’m not trying to get that precise. It is impossible to mirror the brain’s structure in this kind of organization, and…

I think, importantly, that simple organizational structures are good for thinking. I see all of these companies pitching like hierarchical nested colored tags that are also folders and spatially arranged, and I think it’s essentially bad. Forcing notes into a simple set of folders is useful. Having to write linear text and choose what to prioritize is useful.

Think back to writing essays in high school: that forced us to produce structure from our thoughts. That is a good skill to have, it helps us think. It is strictly better than an English class in which all of the term papers are word clouds of vibes with arrows pointing everywhere.

The day note

Every day I’m computering, I’ll create a day note in Obsidian and use it as the anchor for everything else I write or am working on. Basically there are three shortcuts I rely on all the time that I’ve customized:

• Command-D: Daily Notes: Open Today’s Daily Note
• Command-0: Templates: Insert Current Date
• Command-. (Command-Period): Tasks: Toggle task done

The daily note template has nothing in it. I tried inserting prev and next links into it, but it was more annoying than helpful. I have the “Open Daily Note on Startup” option toggled on.

The daily note is usually a bulleted list with links to pages of what I’m working on or writing that day. The Command-. keybinding is wonderful, because it doesn’t just toggle a task to done or not - if you have plain text on a line, it will toggle between a list bullet, a new todo item, and a finished todo item. I use that keybinding all the time.

The other thing is Command-0 for today’s date. This is something I learned from the Zettelkasten ideology: this is a lazy form of an ID for me. A lot of my ongoing notes will consist of multiple sections where I type ## for a heading and then Command-0 to insert the day’s date, to add updates. Automatic date management is extremely useful to me: I’ve always had a lot of trouble calculating, understanding, and remembering dates. Could probably diagnose or pathologize that, but let’s leave that for another day: thankfully calendars exist.

Views & properties

I use properties a lot. Even if they don’t help that much with ‘retrieval’ or aren’t useful in views, I just like the idea of regularized structures. So many notes have a url property for the URL of that technology, person, place, etc.

For some categories, like “stuff I own,” I use properties for the order number, color, price, and other trivia. Every once in a while this comes in handy when something needs a new battery and I can easily look up what kind it takes. I’m also constantly trying to buy more durable things, and it has been really useful to have a record of how long a pair of jeans last (not long!).

Plugins

All together, I use:

• Freeform, of course, for data visualizations.
• Auto link title for pasting links and getting decent titles. This is mostly great, but the average quality of page titles on the internet is poor, because of SEO, and it also can’t handle private URLs.
• Minimal Theme Settings for tweaking little theme settings.
• Hider for tweaking even more theme settings.
• Dataview for making automatic tables of notes for which I’ve added properties.
• Tasks for fancier todo lists. I am on the fence about this one and might not use it for much longer. I still use Things for tasks, too, and it’s better in most ways.
• And I use Obsidian Sync, in part to get a good mobile editing experience. Also I want to pay these folks some money. I want Obsidian to stick around, and like that they’re user-supported.

We use Remix at Val Town and I’ve been a pretty big fan of it: it has felt like a very responsible project that is mostly out of the way for us. That said, the way it works with forms is still a real annoyance. I’ve griped about this in an indirect, vague manner, and this document is an attempt to make those gripes more actionable and concrete.

Gripes with FormData

Remix embraces traditional forms, which means that it also embraces the FormData object in JavaScript and the application/x-www-form-urlencoded encoding, which is the default for <form> elements which use POST. This is great for backwards-compatibility but, like we’ve known about for years, it just sucks, for well-established reasons:

• It is an encoding for text data only. If you have a number input, its value is sent to the server as a string.
• Checkbox and radio inputs are only included with the form data if they’re checked. There is no undefined in form encoding.

These limitations trickle into good libraries that try to help, like conform, which says:

Conform already preprocesses empty values to undefined. Add .default() to your schema to define a default value that will be returned instead.

So let’s say you have a form in your application for writing a README. Someone writes their readme, tries to edit it and deletes everything from the textarea, hits submit, and now the value of the input, on the server side, is undefined instead of a string. Not great! You can see why conform would do this, because of how limited FormData is, but still, it requires us to write workarounds and Zod schemas that are anticipating Conform, FormData, and our form elements all working in this awkward way.

Gripes with form submissions and useEffect

This is kind of gripes with “reacting to form submission in Remix” but it often distills down to useEffect. When you submit a form or a fetcher in Remix, how do you do something after the form is submitted, like closing a modal, showing a toast, resetting some input, or so on?

Now, one answer is that you should ask “how would you do this without JavaScript,” and sometimes there’s a pleasant answer to that thought experiment. But often there isn’t such a nice answer out there, and you really do kind of just want Remix to just tell you that the form has been submitted.

So, the semi-canonical way to react to a form being submitted is something like this code fragment, cribbed from nico.fyi:

function Example() {
	let fetcher = useFetcher<typeof action>();
	
	useEffect(() => {
		if (fetcher.state === 'idle' && fetcher.data?.ok) {
			// The form has been submitted and has returned
			// successfully
		}
	}, [fetcher]);
	
	return <fetcher.Form method="post">
	   // …

The truth is, I will do absolutely anything to avoid using useEffect. Keep it away from me. React now documents its usage as saying to “synchronize a component with an external system”, which is almost explicitly a hint to try not to use it ever. Is closing a modal a form of synchronizing with another system, probably not?

The problems with useEffect are many:

• A lot of times you’ll have something that you’re reacting to, like in the example above, fetcher: when fetcher changes, you want the effect to run. It is a trigger. But you also have things that you want to use in the function, like other functions or bits of state - thing that you aren’t reacting to, but just using. But you have to have both categories of things in the useEffect dependency array, or your eslint rule will yell at you.
• The effect runs whenever the things in its dependency array changes, but knowing when they’ll change is like a whole extra layer of programming-cognitive-overload that is not reflected in type systems and is hard to debug and document. Does the identity of a setState function stay the same? How about one that you get from a third-party library like Jotai? Does the useUtils hook of tRPC guarantee stable identities? Do the parts of a fetcher change? The common answer is “maybe,” and it’s constantly under-documented.
• Effects require you to think about some state machine of fetcher.state and fetcher.data?.ok, or whatever you’re depending on, and figure out where exactly is the state that you’re looking for, and hope that there aren’t unusual ways to enter that state. Like let’s say that you’re showing a toast whenever fetcher.state === 'idle' && fetcher.data?.ok. Is there a way to show a toast twice? The docs for fetcher.data say “Once the data is set, it persists on the fetcher even through reloads and resubmissions (like calling fetcher.load() again after having already read the data),” so is it possible for fetcher.state to go from idle to loading to idle again and show another toast despite not performing the action again? It sure seems so!

The “ergonomics” of this API are rough: it’s too easy to write a useEffect hook that fires too few or too many times.

I have a lot of JavaScript experience and I understand object mutability and most of the equality semantics, but when I’m thinking about stable object references in a big system, with many of my variables coming from third-party libraries that don’t explicitly document their stability promises, it’s rough.

Some new APIs I’ve seen floating around:

Popover API - this has gotten a lot of buzz because JavaScript popover libraries are seen as overkill. I have mixed feelings about it: a lot of the code in the JavaScript popover implementations has to do with positioning logic - see floating-ui as a pretty authoritative implementation of that. The DOM API doesn’t help at all with positioning. Using CSS relative or absolute for positioning isn’t enough: flipping popovers around an anchor is really table stakes, and that still will require JavaScript. Plus, just like the <details> element, pretty much every time I use <details> for collapsing or popover for showing things, I eventually want to lazy-render the stuff that’s initially hidden, which brings us back to the scenario of having some JavaScript. It does look like the API will help with restoring focus and showing a backdrop. So: my guess is that this will be possibly useful, but it won’t eliminate the need for JavaScript in all but the most basic cases.

I’m kind of broadly skeptical of these new browser elements - for example, color inputs, date inputs, and progress elements seem to all fall just short of being useful for advanced applications, whether because they don’t support date ranges, or aren’t style-able enough. They just aren’t good enough to displace all of the JavaScript solutions.

As Chris points out, the popover API does provide a better starting point for accessibility. There are aria attributes for popups, but a lot of DIY implementations neglect to use them.

As Chase points out, there’s a separate proposed anchor positioning API that might solve the positioning problem! Very early on that one, though - no Firefox or Safari support.

CSS Custom Highlight API - now this one I can get excited about, for two reasons:

1. “Browser search” is really hard to do in userspace. When you hit cmd+f in a browser, a lot of very subtle behaviors are invoked that are specific to your browser and hard to replicate in JavaScript - highlighting across elements, searching for words with locale replacements (searching for “naive” matches “naïve”). The custom highlight API makes this somewhat more possible to replicate in userspace - something that we really wanted to do at Observable and never could pull off.
2. “Code highlighting” is a nearly worst-case scenario for excessive DOM sizes - every little syntax token needs to be wrapped with a <span> or some other element. This really hurts the performance of pages that show source code. Using custom highlighting for code syntax highlighting would be amazing. The only downside is that it would necessarily be client-side only, so no server rendering and probably a flash of unhighlighted code.

Array.with is super cool. Over the long term, basically everything that was introduced in underscore (which was itself heavily inspired by Ruby’s standard library) is becoming part of core JavaScript, and I’m into it. I think one of the takeaways of the npm tiny modules discourse is that if you have a too-small standard library, then you get a too-big third-party ecosystem with modules for silly things like detecting whether something is a number. There are just too many modules like that, and too many implementations of the same well-defined basic problems. The world is better now that we have String.padStart instead of several left-pad modules, including mine, sadly.

I am still using Obsidian pretty heavily. I use the Daily note plugin to create a daily note which anchors all the links to stuff I’m working on and doing in a particular day. I wanted better navigation between days without having to keep the calendar plugin open, and decided on this little snippet -

[[<% tp.date.now("YYYY-MM-DD", -1) %>|← Yesterday]] - [[<% tp.date.now("YYYY-MM-DD", +1) %>|Tomorrow →]]

This requires the Templater plugin installed and the Trigger templater on new file creation option enabled, and the daily note date syntax be set to YYYY-MM-DD.

I still find Templater a bit intimidating. The only other place where I use it is for this, the microblog, which has this:

<% await tp.file.move("/Microblog/" + tp.file.creation_date("YYYY[-]MM[-]DD")) %>

Since this microblog is published using Jekyll, its filenames need the YYYY-MM-DD date prefix. This command renames a new microblog file according to that syntax, and then I add the slug and title for it manually.

Something I’ve idly wondered about in regards to the geospatial industry is the impact of Esri staying private. Esri is big and important, but it’s hard to say precisely how much, because it’s private - maybe it does $1.3B in revenue, it has somewhere around 40% or 43% market share. I don’t know what market cap that would give it as a public company, that’s a question for some Wall Street intern.

But, given that Esri isn’t public, what public geospatial companies are there? It’s hard to say.

There are a few public satellite imaging companies like Planet Labs, which has had a hard time since going public via SPAC, and Blacksky and Terran (same story). All of which trade at well under $1B, with Blacksky and Terran closer to $200M, which is kind of a mid-lifecycle private startup valuation. Maxar used to be a public company and has huge revenue and reach, but went private under a private equity firm in 2023 for $6.4B.

What about software? There are far fewer examples. The best example is TomTom, listed in Amsterdam as TOM2. HERE used to be a subsidiary of Nokia, but was bought by a consortium of automakers and is now owned by a lot of minority stakes from Intel, Mitsubishi, and others. Foursquare is still private, and so are smaller players like Mapbox and CARTO.

Of course Google, Apple, Microsoft, and Meta have their enormous geospatial departments, but they are heavily diversified oligopolies and none of them offer GIS-as-a-service or GIS-as-software-product in any real form - in fact when they do tinker in geo it tends to be as open source organizations rather than productization, those efforts probably cross-funded by their advertising and data-mining cashflows.

Trimble seems like the closest I can find to a “pure-play” public GIS company - they at least list geospatial as one of their top products on the front page. Architecture/CAD focused companies Autodesk, Bentley, and Hexagon are also publicly-traded - maybe they count, too.

Thanks Martijn for noting that TomTom is public.

With sensors, microphones, and cameras, cars collect way more data than needed to operate the vehicle. They also share and sell that information to third parties, something many Americans don’t realize they’re opting into when they buy these cars.

The irony of privacy with cars is that private industry has an incredibly well-tuned privacy-invading, tracking machine, but license plates as a means of identification have completely failed, with the endorsement of the police.

Cars without license plates are everywhere in New York City, but the Adams administration has been slow to address the problem, allowing tens of thousands of drivers to skirt accountability on the road each year. The city received more than 51,000 complaints about cars parked illegally on the street with no license plates in 2023, according to 311 data analyzed by Streetsblog. From those complaints, the Department of Sanitation removed only 1,821 cars, or just 3.5 percent.

In major cities, it’s a routine occurrence to see ‘ghost cars’ with missing or fake identification. Government interventions into running red lights are opposed by the ACLU until they can sort out the privacy impact. Cars also grant people additional privacy rights against search and seizure that they wouldn’t have riding a bike or pushing a cart.

I am, for the record, pretty infuriated by drivers who obscure the identity of their cars. Cars in cities are dangerous weapons that disproportionately hurt the disadvantaged. We don’t allow people to carry guns with filed-off serial numbers, we shouldn’t allow people to drive cars with no license plates.

There are tradeoffs: increasing surveillance of cars via red lights and license plate enforcement will undoubtedly save pedestrians and also it will definitely push people into further precarity and cost them their jobs when they lose their licenses or can’t pay the ticket. In a perfect world, we’d have Finnish-style income-based ticket pricing and street infrastructure that prevented bad behavior from occurring in the first place. But in the meantime there are some hard tradeoffs between reducing traffic deaths and further surveilling and criminalizing drivers.

So: cars are a surveillance-laden panopticon for private industry, but governments don’t have enough data, or enough motivation, to prosecute even the most brazen cases of people driving illegally and anonymously. And private industry isn’t doing much to combat road deaths.

The good

• We manage a WebWorker, which previously required that we create a separate entry point and build it with esbuild and handle the “fingerprinting” of the file (to manage its CDN caching) ourselves. Vite just bundles it based on the import, like it would with code-splitting. Much more robust, much simpler.
• Vite has a sentryVitePlugin that works much better than Sentry’s Remix integration and is reassuringly general-purpose.
• We already build around a lot of patterns in epic-stack, so following Kent’s lead with his Vite PR made everything a lot easier.
• Vite’s HMR is magical and fast and I am mostly on board with its approach of barely bundling packages in development.

The bad

• We hit an issue that has been reported in Vite, Jotai, and TanStack Query repos around source map handling that required a manual workaround.
• After fixing that, our build with sourcemap: true now runs out of memory on Render, consistently. The servers that build Val Town from source are plenty powerful and this is pretty surprising.

In summary: I love the idea of Vite, but the fairly extensive pull request I wrote for the switch is going on the backburner. I’ll revisit when Remix makes Vite the default compiler, at which point hopefully Vite is also a little lighter on its memory usage and they’ve fixed the bug around source map error locations.

This is all a somewhat recent realization for me, so there may be angles I don’t see yet.

So I was reading Bailout Nation, Barry Ritholtz’s book about the 2008 financial crisis, and one of its points was that charitable trusts were one of the reasons why CDOs and other dangerous and exotic financial products were adopted with such vigor: the trusts were legally required to pay out a percentage of assets per year, like 5%, and previously they were able to do so using Treasury Bills or other forms of safe debt because interest rates were higher. But interest rates were dropped to encourage growth, which made it harder to find reliable medium-return instruments, which put the attention on things like CDOs that produced a similar ‘fixed’ return, but were backed by much scarier things.

And then I was sitting in a park with a friend talking about how this was pretty crazy, and that these trusts are required to pay at least that percentage. He said (he knows a lot more about this than I do) that the 5% number is an IRS stipulation, but it is often also a cap: so the trust can’t pay more than that. Which - he’s not very online, and this is paraphrasing - is an incredible example of control from the grave.

All in all, why do charitable trusts exist? Given the outcomes of:

• A billionaire donating all of their money to a charity at their death, or even before
• Or, a billionaire putting their money in a charitable trust that pays 5% a year

The outcomes are simply worse when you use a trust, right? Using trusts gives charities less money, because they have to accept a trickle of yearly donations instead of just receiving the money. In exchange for withholding the full amount from charities, heirs receive the power to direct the trust to one cause or another, and the prestige of the foundation’s name and continuing wealth. This isn’t better for charities, right?

It seems thoroughly wrong that our legal system allows people to exert control long after they’re dead, and to specifically exert the control to withhold money from active uses. Charitable trusts are better understood as ways to mete out little bits of charity in a way designed to benefit wealthy families.

As far as I remember it - Against Charity, a whole anti-charity book that I read, didn’t lay out this argument, even though it is definitely a thing that people are thinking.

I am a pretty faithful neovim user. I’ve been using vim/neovim since around 2011, and tolerate all of its quirks and bugs because the core idea of modal editing is so magical, and it has staying power. VIM is 32 years old and will probably be around in 30 years: compare that to all of the editors that were in vogue before VS Code came around - TextMate, Atom, Brackets, Sublime Text. They’ve all faded in popularity, though most of them are still around and used by some people, though Atom is officially done.

But using VIM to develop TypeScript is painful at times: I’m relying on a bunch of Lua plugins that individuals have developed, and because Microsoft somehow has developed LSP, the standard for editors communicating with language tools, and also developed TypeScript, a language tool, and somehow, has still not made the “TypeScript server” speak LSP, these tools have to work around little inconsistencies about how TypeScript works, and instead of using Neovim’s built-in LSP support, they have TypeScript-specific tooling. Because Microsoft can’t get their wildly successful product to speak the wildly-successful specification that they also created.

So, I dabble with Zed, which is a new editor from the same team as Atom, but this time written in Rust and run as a standalone startup instead of a unit inside of GitHub.

Now Zed is pretty good. The latency of VS Code, Atom, and other web-technology-based editors is very annoying to me. Zed is extremely fast and its TypeScript integration is pretty good. Plus, it’s now open source. I think its chances of being around in 30 years are probably lower than VS Code’s: Zed is built by a pre-revenue startup, and releasing complex code as open source is absolutely no guarantee of longevity: there is no skilled community that will swoop in and maintain the work of a defunct startup, if they were to ever go defunct.

So, on days when neovim is annoying me, I use Zed. Today I’m using it and I’ll take some notes.

VIM support is overall pretty good

Zed’s vim mode is not half-assed. It’s pretty darn good. It’s not actually vim - it’s an emulation layer, and not a complete one, but enough that I’m pretty comfortable with navigation and editing right off the bat.

That said, it needs a little extra to be usable for me. I used the little snippet of extra keybindings in their docs to add a way to switch between panes in a more vim-like fashion:

[
  {
    "context": "Dock",
    "bindings": {
      "ctrl-w h": ["workspace::ActivatePaneInDirection", "Left"],
      "ctrl-w l": ["workspace::ActivatePaneInDirection", "Right"],
      "ctrl-w k": ["workspace::ActivatePaneInDirection", "Up"],
      "ctrl-w j": ["workspace::ActivatePaneInDirection", "Down"]
    }
  }
]

Still looking for a way to bind ctrl-j and ctrl-k to “previous tab” and “next tab”. The docs for keybindings are okay, but it’s hard to tell what’s possible: I don’t get autocomplete hints when I type workspace:: into a new entry in the keybindings file.

Shortcut hints in the UI are the default shortcuts, not the vim versions

• Another mild annoyance with using Zed as a vim-replacement is that if I right-click something like a variable, the UI shows potential actions and the shortcuts to trigger those actions, but the shortcuts are the defaults, not the vim bindings. I don’t want to hit F2 to rename a symbol, I want to type cd.
• Next thing I miss is my ripgrep-based search in neovim which I used constantly (through telescope). Unless you have a perfect file-per-React-component system or have a great memory for which files contain what, it’s pretty important to have great search. Zed’s search is fine, but it’s a traditional search mode, not geared for quick navigation.
• Creating new files: in vim I almost always do :tabnew %, hit enter, which expands % into the current filename, and then edit the filename of the current file to produce a new name. This workflow doesn’t work in Zed: % doesn’t expand to the current filename. The only way I can find to save a new file is through the file dialog, which is super tedious. Without this tool in general, I’m kind of left with less efficient ways to do things: to delete the currently-open file, I’d usually do ! rm %, but I can’t.

Overall, Zed’s TypeScript integration is great, and it’s a very fast, pretty well-polished experience. When I tested it a few months ago, I was encountering pretty frequent crashes, but I didn’t encounter any crashes or bugs today. Eventually I might switch, but this trial made me realize how custom-fit my neovim setup is and how an editor that makes me reach for a mouse occasionally is a non-starter. I’ll need to find Zed-equivalent replacements for some of my habits.

For the last few years, I’ve had Aranet 4 and AirGradient sensors in my apartment. They’re fairly expensive gadgets that I have no regrets purchasing – I love a little more awareness of things like temperature, humidity, and air quality, it’s ‘grounding’ in a cyberpunk way. But most people shouldn’t purchase them: the insights are not worth that much.

So here’s the main insight for free: use your stove’s exhaust fan. Gas stoves throw a ton of carbon dioxide into your living space and destroy the air quality.

This is assuming you’re using gas: get an electric range if you can, but if you live in a luxury New York apartment(1) like me, you don’t have a choice.

I used to run the exhaust fan only when there was smoke or smell from cooking. This was a foolish decision in hindsight: run the exhaust fan whenever you’re cooking anything. Your lungs and brain will thank you.

1. luxury here means basic, run-down apartments with no amenities that are now expensive because of the housing shortage caused by rampant obstructionism of housing in dense cities

There are a lot of companies pitching new kinds of wallets, with lots of ads - Ridge is one of the most famous. An option that never seems to come up in internet listicles but I’ve sworn by for years is the Hawbuck wallet.

My personal preferences for this kind of thing is:

1. Lightweight
2. Durable
3. Ideally, vegan

Hawbuck checks all the boxes. I used mighty wallets before this and got a year or two out of them, but Hawbuck wallets wear much, much slower than that. Dyneema is a pretty magical material.

I’m happy it’s also not leather. I still buy and use leather things, despite eating vegan, simply because the vegan alternatives for things like shoes and belts tend to be harder to get and they don’t last as long.

(this is not “sponsored” or anything)

We’ve been using Linear for a month or two at Val Town, and I think it has ‘stuck’ and we’ll keep using it. Here are some notes about it:

• The keyboard shortcuts are as good as people say they are: you can do things like hover your mouse over a row in a list, hit a keyboard shortcut, and it’ll apply to the hovered target item. This is really impressive stuff.
• I do quite like the desktop app - it feels pretty polished, for an Electron (or Tauri, not sure) application. I’ve been using Vimium pretty heavily again - a Chrome extension that adds VIM-style keybindings to Chrome. It is a godsend for my RSI, which is triggered when I use a mouse, but Vimium totally botches most website built-in keybindings. Keybindings are a really hard problem in general but using a standalone wrapped-web app seems like a good way to make them more reliable, by insulating them from whatever funky Chrome extensions you’re using at the moment.
• I dearly miss the ability to permalink a section of code, paste it into a comment, and GitHub to turn that into an inline code snippet. It was so nice for discussing things like regressions, because you could point right to the ground-truth.
• I have really mixed feelings about Linear’s editor, which has some Markdown abilities - typing _ around a string will make it italicized - but not others - Markdown links don’t become links, unlike Notion’s Markdown-ish editor. I get that it’s built to be friendly for both managers and engineers, thus creating an interface between the people who know Markdown and the ones who don’t. But it’s an awkward middle ground that forces me to use a mouse more than I’d like
• Man, there are so many ways to organize stuff. I have in the past had mixed experiences with Linear for just this reason - it allows people to create labyrinthine systems of organization, and then try to apply their tags and milestones and projects to the real world and say “make it happen,” and, alas, the map doesn’t turn into the terrain. But on the other hand, the “cycles” system, which is kind of like a time-constrained milestone that restarts every week or two - I like that. It injected some good energy into the organization.
• It really is extremely pretty - it’s up there with Notion in terms of applications that just look expensive, like a top-of-the-line Volvo (this is not a dig, I think Volvos, and Polestars, look great).
• The realtime sync is usually great, but sometimes two people are editing the same ticket at the same time, and it’s just weird. Realtime sync for the “state of the world” seems good, realtime sync which feels like stepping on everyones toes or peeping over someone’s shoulder, not that great.
• It’s pretty similar on mobile to GitHub’s experience, for now. They’re teasing a native app, which I hope is great.

My last big project at Mapbox was working on Mapbox Studio. We launched it in 2015.

For the web stack, we considered a few other options - we had used d3 to build iD, which worked out great but we were practically the only people in the internet using d3 to build HTML UIs - I wrote about this, in “D3 for HTML” in 2013. The predecessor to Mapbox Studio, TileMill, was written with Backbone, which was cool but is almost never used today. So, anyway, we decided on React when we started it, which was around 2014.

So it’s been a decade. Mapbox Studio is still around, still on React, albeit many refactors later. If I were to build something new like it, I’d be tempted to use Svelte or Solid, but React would definitely be in the running.

Which is to say: wow, this is not churn. What is the opposite of churn? Starting a codebase and still having the same tech be dominant ten years later? Stability? For all of words spilled about trend-chasing and the people talking about how one of these days, React will be out of style, well… it’s been in style for longer than a two-term president.

When I make tech decisions, the rubric is usually that if it lasts three years, then it’s a roaring success. I know, I’d love for the world to be different, but this is what the industry is and a lot of decisions don’t last a year. A decision that seems reasonable ten years later? Pretty good.

Anyway, maybe next year or the year after there’ll be a real React successor. I welcome it. React has had a good, very long, run.

At Val Town, we recently introduced a command-k menu, that “omni” menu that sites have. It’s pretty neat. One thing that I thought would be cool to include in it would be search results from our documentation site, which is authored using Astro Starlight. Our main application is React, so how do we wire these things together?

It’s totally undocumented and this is basically a big hack but it works great:

Starlight uses Pagefind for its built-in search engine, which is a separate, very impressive, mildly documented open source project. So we just load the pagefind.js file that Starlight bakes, using an ES import, across domains, and then just use the pagefind API. So we’re loading both the search algorithms and the content straight from the documentation website.

Here’s an illustrative component, lightly adapted from our codebase. This assumes that you’ve got your search query passed to the component as search.

import { Command } from "cmdk";
import { useDebounce } from "app/hooks/useDebounce";
import { useQuery } from "@tanstack/react-query";

function DocsSearch({ search }: { search: string }) {
  const debouncedSearch = useDebounce(search, 100);
  
  // Use react-query to dynamically and lazily load the module
  // from the different host
  const pf = useQuery(
    ["page-search-module"],
    // @ts-expect-error
    () => import("https://docs.val.town/pagefind/pagefind.js")
  );

  // Use react-query again to actually run a search query
  const results = useQuery(
    ["page-search", debouncedSearch],
    async () => {
      const { results }: { results: Result[] } =
        await pf.data.search(debouncedSearch);
      return Promise.all(
        results.slice(0, 5).map((r) => {
          return r.data();
        })
      );
    },
    {
      enabled: !!(pf.isSuccess && pf.data),
    }
  );

  if (!pf.isSuccess || !results.isSuccess || !results.data.length) {
    return null;
  }

  return results.data.map((res) => {
    return (
      <Command.Item
        forceMount
        key={res.url}
        value={res.url}
        onSelect={() => {
          window.open(res.url);
        }}
      >
        <a
          href={res.url}
          onClick={(e) => {
            e.preventDefault();
          }}
        >
          <div
            dangerouslySetInnerHTML={{ __html: res.excerpt }}
          />
        </a>
      </Command.Item>
    );
  });
}

Pretty neat, right? This isn’t documented anywhere for Astro Starlight, because it’s definitely relying on some implementation details. But the same technique would presumably work just as well in any other web framework, not just React.

I see the S&P 500 referenced pretty frequently as an vanilla index for people investing. This isn’t totally wrong, which is why this post is short. But, if you have the goal of just “investing in the market,” there’s a better option for doing that: a total market index. For Vanguard, instead of VOO, it’d be VTI. For Schwab, it’s SCHB. Virtually every provider has an option. For background, here’s Jack Bogle discussing the topic.

The S&P 500 is not a quantitative index of the top 500 companies: it has both selection criteria and a committee that takes a role in selection. In contrast, total market indices are typically fully passive and quantitative, and they own more than 500 companies.

So if you want to “own the market,” you can just do that. Not investment advice.

An evergreen topic is something like “why are websites so big and slow and hard and video games are so amazing and fast?” I’ve thought about it more than I’d like. Anyway, here are some reasons:

• Web pages are just-in-time delivered, with no installation required. Modern video games typically require both a long install process, downloading tens of gigabytes onto the console. Even after that, they take minutes to boot up: when I play Cyberpunk on my XBox S, the loading screen takes at least a minute. It’s fast after that, but it’s fast because of that loading phase.
• Video game development cycles are long and extraordinarily expensive. A recent failed game that didn’t make a serious media stir cost over 140 million dollars to make. There is no startup that will pour over $100 million on a website before even launching it. And AAA video games, which are often the ones that people have in mind, take years to develop.
• Video games are generally single-tasked: you don’t have 10 of them open at a time, ready to switch tabs at any moment. They’re usually full-screen too, so they don’t even need to be composited with other graphics in a window manager, you can just shoot pixels straight to the screen.
• The web is an aggressively heterogenous platform, moreso than nearly anything else. Webpages by default support any screen size, input method, light and dark mode, and pixel density. Large websites are expected to support multiple languages, and scale down to cheap feature-phones. So websites, when they do express native-like code, need to do so through WASM rather than some platform-specific binary. This is a lot of the power, and the struggle, of the web: you are writing code for an unimaginably wide range of devices.

Someone asked over email about why I stopped building Placemark as a SaaS and made it an open source project. I have no qualms with sharing the answers publicly, so here they are:

I stopped Placemark because it didn’t generate enough revenue and I lost faith that there was a product that could be implemented simply enough, for enough people, who wanted to pay enough money, for it to be worthwhile. There are many mapping applications out there and I think that collectively there are some big problems in that field:

1. The high end is captured by Esri, whose customers dislike the tool but tolerate it and are locked into it, and Esri is actually very good at what it does.
2. The low end is captured by free tools that are subsidized by big companies like Google, or run by open source communities like QGIS, which causes users to generally expect similar software to be super cheap. VC-funded startups are able to underprice their software for a few years and spend tens of millions building it. Placemark was fully bootstrapped, self-funded, and built just by me.
3. There are vast differences in user expectations that make it very hard to make a software product in the middle, between the complexity of QGIS and the simplicity of Google Maps - people want some combination of analytics, editing, social features, etc that are all hard to combine into anything simple.
4. It is very hard to build a general-purpose piece of software like Placemark. If I were to do it again, I’d do something in a niche, targeting one specific kind of customer in one specific industry.

I do want to emphasize that I knew most of this stuff going into it, and it’s, like good that geo has a lot of open source projects, and I don’t have any ill feelings toward really any of the players in the field. It’s a hard field!

As I’ve said a bunch of times, the biggest problem with competition in the world of geospatial companies is that there aren’t many big winners. We would all have a way different perspective on geospatial startups if even one of them had a successful IPO in the last decade or two, or even if a geospatial startup entered the mainstream in the same way as a startup like Notion or Figma did. Esri being a private company is definitely part of this - they’re enormous, but nobody outside of the industry talks about them because there’s no stock and little transparency into their business.

Also, frankly, it was a nerve-wracking experience building a bootstrapped startup, in an expensive city, with no real in-person community of people doing similar kinds of stuff. The emotional ups and downs were really, really hard: every time that someone signed up and cancelled, or found a bug, or the servers went down as I was on vacation and I had to plug back in.

You have to be really, really motivated to run a startup, and you have to have a very specific attitude. I’ve learned a lot about that attitude - about trying to get positivity and resilience, after ending Placemark. It comes naturally to some people, who are just inherently optimistic, but not all of us are like that.

Val Town switched to Remix as a web framework a little over a year ago. Here are some reflections:

• The Remix versioning scheme is a joy. They gradually roll out features under feature flags, so you have lots of time to upgrade.
• Compared to what seems like chaos over in Next.js-land, Remix hasn’t had many big breaking changes or controversies. It’ll eventually adopt RSC, but I am glad that it is not the ‘first mover’ in that regard.
• We’ve hit a few bugs, in types and utf-8 support, and docs, but Remix is rarely the culprit when there’s an outage or some quirk in the application.
• We haven’t switched to Vite but are pretty excited to. Initially the fact that Remix is obviously a “collection of parts,” like react-router and esbuild, rather than a “monolithic framework,” was a source of uncertainty, but I now see it as a strength. It is cool that Vite will allow Remix to have a narrower role.
• The actions/loaders/Forms paradigm is pretty great, but I am still not happy about FormData and not excited about how actions are typed in TypeScript and how much serialization gunk there is. God, that Twitter thread was irritating. I sometimes think conform might help here. We use tRPC a bunch because its DX is superior to Remix’s: the types “just work”, there’s no FormData inference & de-inference required, it’s easy to just call a method.
• The Remix community is pretty good, there are some good guides and documentation sites to be had, especially epic stack. I’m not especially worried about the project losing steam - any concerns I had right after Shopify bought Remix are gone. I also don’t think that any of the React-alternative frameworks are that tempting yet, though I’m keeping an eye on SolidStart and SvelteKit. Please, please do not @ me about how much you like Vue, I do not care at all and you do not need my approval to keep liking Vue.
• Most products have a Remix integration - instrumentation with OpenTelemetry just works, Sentry’s tracing integration just works. Clerk has an integration but using Clerk is one of my top regrets for this application: we’ve encountered so many bugs, and so little momentum on support and fixes.
• We barely use nested layouts, one of Remix’s main features. There just aren’t that many opportunities to. We also don’t really use loaders for the list of vals, either: I think that full stack components are the answer here, but we haven’t implemented that yet.

In summary: Remix mostly gets out of the way and lets us do our work. It’s not a silver bullet, and I wish that it was more obvious how to do complex actions and it had a better solution to the FormData-boilerplate-typed-actions quandry. But overall I’m really happy with the decision.

Things that have worked to get me back on a running regimen and might work for you:

1. Try to run all the streets in my neighborhood. I use CityStrides, there are many similar apps.
2. Run the same exact route, every time, at any speed: focus on consistency-only. Repetition legitimizes.
3. Focus on just one Strava Segment and try to either become the “Local Legend” or get a good time.

My friend Forest has been making some good thoughts about open source and incentives. Coincidentally, this month saw a new wave of open source spam because of the tea.xyz project, which encouraged people to try and claim ‘ownership’ of existing open source projects, to get crypto tokens.

The creator of tea.xyz, Max Howell, originally created Homebrew, the package manager for macOS which I use every day. He has put in the hours and days, and been on the other side of the most entitled users around. So I give him a lot of leeway with tea.xyz’s stumbles, even though they’re big stumbles.

Anyway, I think my idea is that murky incentives are kind of good. The incentives for contributing to open source right now, as I do often, are so hard to pin down. Sure, it’s improving the ecosystem, which satisfies my deep sense of duty. It’s maintaining my reputation and network, which is both social and career value. Contributing to open source is a way to learn, too: I’ve had one mentor early in my career, but besides that I’ve learned the most from people I barely know.

The fact that the incentives behind open source are so convoluted is what makes them sustainable and so hard to exploit. The web is an adversarial medium, is what I tell myself pretty often: every reward structure and application architecture is eventually abused, and that abuse will destroy everything if unchecked: whether it’s SEO spam, or trolling, or disinformation, no system maintains its own steady state without intentional intervention and design.

To bring it back around: tea.xyz created a simple, automatic incentive structure where there was previously a complex, intermediated one. And, like every crypto project that has tried that before, it appealed to scammers and produced the opposite of a community benefit.

If I got paid $5 for every upstream contribution to an open source project, I’d make a little money. It would be an additional benefit. But I’m afraid that the simplicity of that deal - the expectations that it would create, the new community participants that it would invite - would make me less likely to contribute, not more.

• Update: there’s even more chaos from tea.xyz - now people are submitting spam packages to NPM to try to get tokens. It’s all so stupid.
• Update: read this twitter thread about a maintainer of modules on npm who adds unnecessary self-dependencies in order to boost his download numbers, because those counts are tied to payouts from tidelift.

This came up for Val Town - we implemented code folding in our default editor which uses CodeMirror, but wanted it to work with JSX elements, not just functions and control flow statements. It’s not enough to justify a module of its own because CodeMirror’s API is unbelievably well-designed:

import {
  tsxLanguage,
} from "@codemirror/lang-javascript";
import {
  foldInside,
  foldNodeProp,
} from "@codemirror/language";

/** tsxLanguage, with code folding for jsx elements */
tsxLanguage.configure({
  props: [
    foldNodeProp.add({
      JSXElement: foldInside,
    }),
  ],
})

Then you can plug that into a LanguageSupport instance and use it. Amazing.

I’ve been writing some CSS. My satisfaction with CSS ebbs and flows: sometimes I’m happy with its new features like :has, but on the other hand, CSS is one of the areas where you really often get bitten by browser incompatibilities. I remember the old days of JavaScript in which a stray trailing comma in an array would break Internet Explorer: we’re mostly past that now. But in CSS, chaos still reigns: mostly in Safari. Anyway, some notes as I go:

Safari and <details> elements

I’ve been using details more instead of Radix Collapsible for performance reasons. Using the platform! Feels nice, except for CSS. That silly caret icon shows up in Safari and not in Chrome, and breaks layout there. I thought the solution would involve list-style-type: none or appearance, but no, it’s something worse:

For Tailwind:

[&::-webkit-details-marker]:hidden

In CSS:

details::-webkit-details-marker {
  display: hidden;
}

flex min-content size

I’ve hit this bug so many times in Val Town. Anything that could be user-supplied and might bust out of a flex layout should absolutely have min-width: 0, so that it can shrink.

There’s a variant of this issue in grids and I’ve been defaulting to using minmax(0, 1fr) instead of 1fr to make sure that grid columns will shrink when they should.

I’ve been using just for a lot of my projects. It helps a bunch with the context-switching: I can open most project directories and run just dev, and it’ll boot up the server that I need. For example, the blog’s justfile has:

dev:
  bundle exec jekyll serve --watch --live --future

I used to use Makefiles a bit, but there’s a ton of tricky complexity in them, and they really weren’t made as cheat-sheets to run commands - Makefiles and make are really intended to build (make) programs, like run the C compiler. Just scripts are a lot easier to write.

A brief and silly life-hack: headlamps are better flashlights. Most of the time when you are using a flashlight, you need to use your hands too. Headlamps solve that problem. They’re bright enough for most purposes and are usually smaller than flashlights too. There are very few reasons to get a flashlight. Just get a headlamp.

With all love to the maintainers, who are good people and are to some extent bound by their obligation to maintain compatibility, I just have to put it out there: if you have a new JavaScript/TypeScript project and you need to parse or render Markdown, why are you using marked?

In my mind, there are a few high priorities for Markdown parsers:

• Security: marked isn’t secure by default. Yes, you can absolutely run DOMPurify on its output, but will you forget? Sure!
• Standards: it’s nice to follow Commonmark! The original Markdown specification was famously permissive and imprecise. If you want to be able to switch Markdown renderers in the future, it’s going to be a lot nicer if you have a tight standard to rely on, to guarantee that you’ll get the same output.
• Performance: Markdown rendering probably isn’t a bottleneck for your application, but it shouldn’t be.

So, yeah. Marked is pretty performant, but it’s not secure, it’s doesn’t follow a standard - we can do better!

Use instead:

• micromark: the “micro” Markdown parser primarily by wooorm, which is tiny, follows Commonmark. It’s great. Solid default.
• remark: the most extensible Markdown parser you could ever imagine, also by wooorm.
• markdown-it: don’t like wooorm’s style? markdown-it is pretty good too, secure by default, and commonmark-supporting.

marked is really popular. It used to be the best option. But there are better options, use them!

I’ve been trying to preserve as much of Placemark now that it’s open-source. This has been a mixed experience: some products were really easy to move away from, like Northwest and Earth Class Mail. Webflow was harder to quit. But replay.web came to the rescue, thanks to Henry Wilkinson at web.recorder.

Now placemark.io is archived, but nearly complete and at feature-parity, but costs next to nothing to maintain. The magic is the wacz format, which is a specific flavor of ZIP file that is readable with range requests. From the geospatial world, I’ve been thinking about range requests for a long time: they’re the special sauce in Protomaps and Cloud Optimized GeoTIFFs. They let you use big files, stored cheaply on object storage like Amazon S3 or Cloudflare R2, but lets browsers read those files incrementally, saving the browser time & memory and saving you transfer bandwidth & money.

So, the placemark.io web archive is on R2, the website is now on Cloudflare Pages, and the archive is little more than this one custom element:

<replay-web-page source="https://archive.placemark.io/placemark%202024-01-19.wacz" url="https://www.placemark.io/"></replay-web-page>|

This is embedding replayweb.page. Cool stuff!

God, it’s another post about Web Components and stuff, who am I to write this, who are you to read it

Carlana Johnson’s “Alternate Futures for Web Components” had me nodding all the way. There’s just this assumption that now that React is potentially on its way out (after a decade-long reign! not bad), the natural answer is Web Components. And I just don’t get it. I don’t get it. I think I’m a pretty open-minded guy, and I’m more than happy to test everything out, from Rails to Svelte to htmx to Elm. It’s all cool and good.

But “the problems I want to solve” and “the problems that Web Components solve” are like two distinct circles. What do they do for me? Binding JavaScript-driven behavior to elements automatically thanks to customElement? Sure - but that isn’t rocket science: you can get nice declarative behavior with htmx or hyperscript or alpine or stimulus. Isolating styles with Shadow DOM is super useful for embed-like components, but not for parts of an application where you want to have shared style elements. I shouldn’t sloppily restate the article: just read Carlana.

Anyway, I just don’t get it. And I find it so suspicious that everyone points to Web Components as a clear path forward, to create beautiful, fast applications, and yet… where are those applications? Sure, there’s “Photoshop on the Web”, but that’s surely a small slice of even Photoshop’s market, which is niche in itself. GitHub used to use Web Components but their new UIs are using React.

So where is it? Why hasn’t Netflix rebuilt itself on Web Components and boosted their user numbers by dumping the heavy framework? Why are interactive visualizations on the New York Times built with Svelte and not Web Components? Where’s the juice? If you have been using Web Components and winning, day after day, why not write about that and spread the word?

People don’t just use Rails because dhh is a convincing writer: they use it because Basecamp was a spectacular web application, and so was Ta-Da List, and so are Instacart, GitHub, and shopify. They don’t just use React because it’s from Facebook and some brain-virus took them over, they use it because they’ve used Twitter and GitHub and Reddit and Mastodon and countless other sites that use React to create amazing interfaces.

Of course there’s hype and bullying and all the other social dynamics. React fans have had some Spectacularly Bad takes, and, boy, the Web Components crowd have as well. When I write a tender post about complexity and it gets summed up as “going to bat for React” and characterized in bizarre overstatement, I feel like the advocates are working hard to alienate their potential allies. We are supposed to get people to believe in our ideas, not just try to get them to lose faith in their own ideas!

I don’t know. I want to believe. I always want to believe. I want to use an application that genuinely rocks, and to find out that it’s WC all the way, and to look at the GitHub repo and think this is it, this is the right way to build applications. I want to be excited to use this technology because I see what’s possible using it. When is that going to happen?

“If you want to build a ship, don’t drum up people to collect wood and don’t assign them tasks and work, but rather teach them to long for the endless immensity of the sea.” - Antoine de Saint Exupéry

Apps that use CodeMirror

• Obsidian
• Repl.it
• Val Town
• …more on the CodeMirror site

Apps that use Monaco

• StackBlitz
• CodeSandbox
• Deno Deploy
• Cloudflare Workers

How do you set a Cache-Control header on an object in R2 when you’re using rclone to upload?

I burned a lot of time figuring this out. There are a lot of options that look like they’ll do it, but here it is:

--header-upload='Cache-Control: max-age=604800,public,immutable'

That’s the flag you want to use with rclone copy to set a Cache-Control header with Cloudflare R2. Whew.

Reason: sure, you can set cache rules at like 5 levels of the Cloudflare stack - Cache Rules, etc. But it’s really hard to get the right caching behavior for static JavaScript bundles, which is:

• 404s aren’t cached
• Non-404s are cached heavily

This does it. Phew.

This is a Devtools feature that you will only need once in a while, but it is a life-saver.

Some frontend libraries, like CodeMirror, have UIs like autocompletion, tools, or popovers, that are triggered by typing text or hovering your mouse cursor, and disappear when that interaction stops. This can make them extremely hard to debug: if you’re trying to design the UI of the CodeMirror autocomplete widget, every time that you right-click on the menu to click “Inspect”, or you click away from the page to use the Chrome Devtools, it disappears.

Learn to love Emulate a focused page. It’s under the Rendering tab in the second row of tabs in Devtools - next to things like Console, Issues, Quick source, Animation.

Click the Rendering tab, find the Emulate a focused page checkbox, and check it. This will keep Chrome from firing blur events and letting CodeMirror or your given library from knowing that you’ve clicked out of the page. And now you can debug! Phew.

I’ve been thinking about this. Placemark is going open source in 10 days and I’m probably not founding another geo startup anytime soon. I’d love to found another bootstrapped startup eventually, but geospatial is hard.

Anyway, geospatial data is big, which does not combine well with real-time collaboration. Products end up either sacrificing some data-scalability (like Placemark) or sacrificing some edibility by making some layers read-only “base layers” and focusing more on visualization instead. So web tools end up being more data-consumers and most of the big work like buffering huge polygons or processing raster GeoTIFFs stays in QGIS, Esri, or Python scripts.

All of the new realtime-web-application stuff and the CRDT stuff is amazing - but I really cannot emphasize enough how geospatial data is a harder problem than text editing or drawing programs. The default assumption of GIS users is that it should be possible to upload and edit a 2 gigabyte file containing vector information. And unlike spreadsheets or lists or many other UIs, it’s also expected that we should be able to see all the data at once by zooming out: you can’t just window a subset of the data. GIS users are accustomed to seeing progress bars - progress bars are fine. But if you throw GIS data into most realtime systems, the system breaks.

One way of slicing this problem is to pre-process the data into a tiled format. Then you can map-reduce, or only do data transformation or editing on a subset of the data as a ‘preview’. However, this doesn’t work with all datasets and it tends to make assumptions about your data model.

I was thinking, if I were to do it again, and I won’t, but if I did:

I’d probably use driftingin.space or similar to run a session backend and use SQLite with litestream to load the dataset into the backend and stream out changes. So, when you click on a “map” to open it, we boot up a server and download database files from S3 or Cloudflare R2. That server runs for as long as you’re editing the map, it makes changes to its local in-memory database, and then streams those out to S3 using litestream. When you close the tab, the server shuts down.

The editing UI - the map - would be fully server-rendered and I’d build just enough client-side interaction to make interactions like point-dragging feel native. But the client, in general, would never download the full dataset. So, ideally the server runs WebGL or perhaps everything involved in WebGL except for the final rendering step - it would quickly generate tiles, even triangulate them, apply styles and remove data, so that it can send as few bytes as possible.

This would have the tradeoff that loading a map would take a while - maybe it’d take 10 seconds or more to load a map. But once you had, you could do geospatial operations really quickly because they’re in memory on the server. It’s pretty similar to Figma’s system, but with the exception that the client would be a lot lighter and the server would be heavier.

It would also have the tradeoff of not working offline, even temporarily. I unfortunately don’t see ‘offline-first’ becoming a real thing for a lot of use-cases for a long time: it’s too niche a requirement, and it is incredibly difficult to implement in a way that is fast, consistent, and not too complex.

Wrote and released codemirror-continue today. When you’re writing a block comment in TypeScript and you hit “Enter”, this intelligently adds a * on the next line.

Most likely, your good editor (Neovim, VS Code) already has this behavior and you miss it in CodeMirror. So I wrote an extension that adds that behavior. Hooray!

Every database ID scheme that I’ve used has had pretty serious downsides, and I wish there was a better option.

The perfect ID would:

• Be friendly to distributed systems - multiple servers should be able to generate non-overlapping IDs at the same time. Even clients should be able to generate IDs.
• Have good index locality. IDs should be semi-ordered so that new ones land in a particular shard or end up near the end of your btree index.
• Have efficient database storage: if it’s a number, it’s stored as a number. If it’s binary, it should be stored as binary. Storing hexadecimal IDs as strings is a waste of space: Base16 takes up twice as much space as binary.
• Be roughly standardized and future-proof. Cleverness is great, but IDs and data schemas tend to last a long time, and if they don’t last that long, need to survive migrations. A rare boutique ID scheme is a risk.
• Obscure order and addresses - in other words, not be an auto-incrementing number. It is bad to reveal how many things are in a database, and also bad to give people a way to enumerate and find things by tweaking a number in a URL.

Almost nothing checks all these boxes:

• Auto-incrementing bigints are almost perfect, but they aren’t friendly to distributed systems because only one computer knows what the next number is. They also reveal how many things are in a database. You can use Sqids to fix that, though - a surprisingly rare approach.
• All of the versions of UUIDs that are fully standardized have pretty bad index behavior, and cause poor index locality - even v1. But they’re very distributed-systems friendly, and they definitely obscure numbering.
• Orderable new schemes like ulid are cool, but there isn’t a straightforward way to store them as binary, in Postgres. UUIDs are stored as binary, and they’re relatively niche - there’s no postgres implementation of ulids, for example. ULID can be stored in UUID columns, but isn’t valid as a UUID.
• UUID v7 looks like it checks every box, but it’s not fully standardized or broadly available yet. The JavaScript implementations are great but have very little uptake, and Postgres, both by default and in the uuid-ossp module, doesn’t support it.

So for the time being, what are we to do? I don’t have a good answer. Cross our fingers and wait for uuid v7.

• Daniel writes in that the best of both worlds might be achieved by using a numeric ID internally and a UUID for URL-based lookups.
• Update as of 2024-06-21: the popular uuid JavaScript module now supports UUID v7.

I am, relative to many, a sort of React apologist. Even though I’ve written at length about how it’s not good for every problem, I think React is a good solution to many problems. I think the React team has good intentions. Even though React is not a solution to everything, it a solution to some things. Even though React has been overused and has flaws, I don’t think the team is evil. Bask in my equanimity, etc.

However,

The state of React releases right now is bad. There are two major competing React frameworks: Remix, funded by Shopify and Next.js, funded by Vercel. Vercel hired many members of the React team.

It has been one and a half years since the last React release, far longer than any previous release took.

Next.js is heavily using and promoting features that are in the next release. They vendor a version of the next release of React and use some trickery to make it seem like you’re using React 18.2.0 when in fact you’re using a canary release. These “canary releases” are used for incremental upgrades at Meta, too, where other React core developers work.

On the other hand, the non-Vercel and non-Facebook ecosystems don’t have these advantages. Remix suffers from an error in React that is fixed, but not released. People trying to use React 18.3.0 canary releases will have to use npm install --force or overrides in their package.json files to tie it all together.

This strategy, of using Canary releases for a year and a half and then doing some big-bang upgrade to React 19.0.0: I don’t like it. Sure, there are workarounds to use “current” Canary React. But they’re hacks, and the Canary releases are not stable and can quietly include breaking changes. All in all, it has the outward appearance of Vercel having bought an unfair year headstart by bringing part of the React team in-house.

An evergreen blog topic is “writing my own blogging engine because the ones out there are too complicated.” With the risk of stating the obvious:

Writing a blog engine, with one customer, yourself, is the most luxuriously simple web application possible. Complexity lies in:

• Diversity of use-cases: applications that need to work on multiple devices, different languages, work with screen readers, might need to work offline, or on a particular network.
• The real world. Everything about the real world is complicated: time, names, geography, everything. Governments can’t simplify this very much, so they build extremely complicated technology so they can serve every citizen (in theory). Companies can “define their customers” and simplify this a bit. Individuals can simplify this a lot: just me, my timezone, my language.
• The problem area. Something like how Microsoft Word decides cursor position, or Excel calculates formulas - there are actually hard problems out there, with real complexity to solving them, with no simple solution.

Which is all to say, when I read some rant about how React or Svelte or XYZ is complicated and then I see the author builds marketing websites or blogs or is a Java programmer who tinkers with websites but hates it – it all stinks of narrow-mindedness. Or saying that even Jekyll is complicated, so they want to build their own thing. And, go ahead - build your own static site generator, do your own thing. But the obvious reason why things are complicated isn’t because people like complexity - it’s because things like Jekyll have users with different needs.

Yes: JavaScript frameworks are overkill for many shopping websites. It’s definitely overkill for blogs and marketing sites. It’s misused, just like every technology is misused. But not being exposed to the problems that it solves does not mean that those problems don’t exist.

HTML-maximalists arguing that it’s the best way probably haven’t worked on a hard enough problem to notice how insufficient SELECT boxes are. Or how the dialog element just doesn’t help much. Complaining about ARIA accessibility based on out-of-date notions when the accessibility of modern UI libraries is nothing short of fantastic. And what about dealing with complex state? Keybindings with different behaviors based on UI state. Actions that re-render parts of the page - if you update “units” from miles to meters and you want the map scale, the title element, and the measuring tools to all update seamlessly. HTML has no native solution for client-side state management, and some applications genuinely require it.

And my blog is an example of the luxury of simplicity – it’s incredibly simple! I designed the problem to make it simple so that the solution could be simple. If I needed to edit blog posts on my phone, it’d be more complicated. Or if there was a site search function. Those are normal things I’d need to do if I had a customer. And if I had big enough requirements, I’d probably use more advanced technology, because the problem would be different: I wouldn’t indignantly insist on still using some particular technology.

Not everyone, or every project, has the luxury of simplicity. If you think that it’s possible to solve complicated problems with simpler tools, go for it: maybe there’s incidental complexity to solve. If you can solve it in a convincing way, the web developers will love you and you might hit the jackpot and be able to live off of GitHub sponsors alone.

See also a new start for the web, where I wrote about “the document web” versus “the application web.”

I’ve been working on oldfashioned.tech, which is sort of a testbed to learn about htmx and the other paths: vanilla CSS instead of Tailwind, server-rendering for as much as possible.

How are tooltips and modals supposed to work outside of the framework world? What the Web Platform provides is insufficient:

• The title attribute is unstyled and only shows up after hovering for a long time over an element.
• The dialog element is bizarrely unusable without JavaScript and basically doesn’t give you much: great libraries like Radix don’t use it and use role="dialog" instead.

So, what to do? There’s:

• The pure CSS option. Seems like balloon.css is the main example. Unmaintained for three years, but maybe that works? Wouldn’t have the right placement for tooltips if they’re on an edge of the screen. Tooltips also can’t contain HTML or styling.
• Or maybe I should use floating-ui and write a little extension. The DOM-only version of floating-ui is tiny, and the library is very high quality and used everywhere - it’s what Radix uses.

I think it’s kind of a bummer that there just aren’t clear options for this kind of thing.

I wrote about this pattern years ago, and wrote an update, and then Classes became broadly available in JavaScript. I was kind of skeptical of class syntax when it came out, but now there really isn’t any reason to use any other kind of “class” style than the ES6 syntax. The module pattern used to have a few advantages:

• You didn’t need to keep remembering what this was referring to - before arrow functions this was a really confusing question.
• You could have private variables.

Well, now classes can use arrow functions to simplify the meaning of this , and private properties are supported everywhere, we can basically declare the practice of using closures as psuedo-classes to be officially legacy.

Let’s say you’re running some web application and suddenly you hit a bug in one of your dependencies. It’s all deployed, lots of people are seeing the downtime, but you can’t just push an update because the bug is in something you’ve installed from npm.

Remember patch-package. It’s an npm module that you can install in which you:

• Edit the dependency source code directly in node_modules
• Run npx patch-package some-package
• Add "postinstall": "patch-package" to your scripts

And from now on when npm install runs, it tweaks and fixes the package with a bug. Obviously submit a pull request and fix the problem at its source later, but in times of desperation, this is a way to fix the problem in a few minutes rather than an hour. This is from experience… experience from earlier today.

I’ve been moving things for Placemark’s shutdown as a company and noting some of the exit experiences:

• Loom is surprisingly hard to exit from. There’s no bulk export option, no way to export metadata.
• Webflow doesn’t support exporting sites with CMS collections (blogs, docs, etc). It supports exporting the CMS content, and the templates, but not the two together.
• Earth Class Mail has a pretty respectable offboarding flow that does a good job warning you of the ramifications of closing the virtual address.
• Legalinc’s service to close down the LLC was fast and cost about $600. Maybe there are cheaper options, but I’m satisfied with the speed & ease of use.
• Northwest registered agent was also super clear and easy to close down. I had a great experience with them from start to finish.

The hot new thing in CSS is :has() and Firefox finally supports it, starting today - so the compatibility table is pretty decent (89% at this writing). I already used has() in a previous post - that Strava CSS hack, but I’m finding it useful in so many places.

For example, in Val Town we have some UI that shows up on hover and disappears when you hover out - but we also want it to stay visible if you’ve opened a menu within the UI. The previous solution required using React state and passing it through components. The new solution is so much simpler - just takes advantage of Radix’s excellent attention to accessibility - so if something in the UI has aria-expanded=true, we show the parent element:

.valtown-pin-visible:has([aria-expanded="true"]) {
  opacity: 1;
}

• User preferences should be columns in the users table. Don’t get clever with a json column or hstore. When you introduce new preferences, the power of types and default values is worth the headache of managing columns.
• Emails should probably be citext, case-insensitive text. But don’t count on that to prevent people from signing up multiple times - there are many ways to do that.
• Most text columns should be TEXT. The char-limited versions like varchar aren’t any faster or better on Postgres.
• Just try not to use json or jsonp, ever. Having a schema is so useful. I have regretted every time that I used these things.
• Make as many things NOT NULL as possible. Basically the same as “don’t use json” - if you don’t enforce null checks at the database level, null values will probably sneak in eventually.
• Most of the time choose an enum instead of a boolean. There is usually a third value beyond true & false that you’ll realize you need.
• Generally store times and dates without timezones. There are very, very few cases where you want to store the original timezone rather than store everything in UTC and format it to the user’s TZ at display time.
• Most tables should have a createdAt column that defaults to NOW(). Chances are, you’ll need it eventually.

I love Strava, and a lot of my friends do too. And some of them do most of their workouts with Peloton, Swift, and other “integrations.” It’s great for them, but the activities just look like ads for Peloton and don’t have any of the things that I like about Strava’s community.

Strava doesn’t provide the option to hide these, so I wrote a user style that I use with Stylus - also published to userstyles.org. This hides Peloton workouts.

@-moz-document url-prefix("https://www.strava.com/dashboard") {
    .feed-ui > div:has([data-testid="partner_tag"]) {
        display: none;
    }
}

This microblog, by the way… I felt like real blog posts on macwright.com were becoming too “official” feeling to post little notes-to-self and tech tricks and whatnot.

The setup is intentionally pretty boring. I have been using Obsidian for notetaking, and I store micro blog posts in a folder in Obsidian called Microblog. The blog posts have YAML frontmatter that’s compatible with Jekyll, so I can just show them in my existing, boring site, and deploy them the same way as I do the site - with Netlify.

I use the Templater plugin, which is powerful but unintuitive, to create new Microblog posts: key line is

<% await tp.file.move("/Microblog/" + tp.file.creation_date("YYYY[-]MM[-]DD")) %>

This moves a newly-created Template file to the Microblog directory with a Jekyll-friendly date prefix. Then I just have a command in the main macwright.com repo that copies over the folder:

microblog:
  rm -f _posts/micro/*.md
  cp ~/obsidian/Documents/Microblog/* _posts/micro

This is using Just, which I use as a simpler alternative to Makefiles, but… it’s just, you know, a cp command. Could be done with anything.

So, anyway - I considered Obsidian Publish but I don’t want to build a digital garden. I have indulged in some of the fun linking-stuff-to-stuff patterns that Obsidian-heads love, but ultimately I think it’s usually pointless for me.

I started another “awesome” GitHub repo (a list of resources), for CodeMirror, called awesome-codemirror. CodeMirror has a community page but I wanted a freewheeling easy-to-contribute-to alternative. Who knows if it’ll grow to the size of awesome-geojson - 2.1k stars as of this writing!

ViewPlugin.fromClass only allows the class constructor to take a single argument with the CodeMirror view.

You use a Facet. Great example in JupyterLab. Like everything in CodeMirror, this lets you be super flexible with how configuration works - it is designed with multiple reconfigurations in mind.

Example defining the facet:

export const suggestionConfigFacet = Facet.define<
  { acceptOnClick: boolean },
  { acceptOnClick: boolean }
>({
  combine(value) {
    return { acceptOnClick: !!value.at(-1)?.acceptOnClick };
  },
});

Initializing the facet:

suggestionConfigFacet.of({ acceptOnClick: true });

Reading the facet:

const config = view.state.facet(suggestionConfigFacet);

I heavily use the ~/tmp directory of my computer and have the habit of moving to it, creating a new temporary directory, moving into that, and creating a short-lived project. Finally I automated that and have been actually using the automation:

I wrote this tiny zsh function called tt that creates a new directory in ~/tmp/ and cd’s to it:

tt() {
    RANDOM_DIR=$(date +%Y%m%d%H%M%S)-$RANDOM
    mkdir -p ~/tmp/"$RANDOM_DIR"
    cd ~/tmp/"$RANDOM_DIR" || return
}

This goes in my .zshrc.

This took way too long to figure out.

The File polyfill in Remix has the fresh new .stream() and .arrayBuffer() methods, which aren’t mentioned on MDN. So, assuming you’re in an action and the argument is args, you can get the body like:

const body = await unstable_parseMultipartFormData(
  args.request,
  unstable_createMemoryUploadHandler()
);

Then, get the file and get its text with the .text() method. The useful methods are the ones inherited from Blob.

const file = body.get("envfile");

if (file instanceof File) {
   const text = await file.text();
   console.log(text);
}

And you’re done! I wish this didn’t take me so long.