It's a serious handicap to trust people you can't have a dialogue with. And yet, we're basically forced to. While it's nice to have sources you can trust, it's a good start to just learn from a lot of sources with different viewpoints. Better still, sources that directly contradict one another. No, reading two news sites about the same event doesn't count. At the very least, read up on the history of everything. Nothing in society suddenly happens. We're all subject to a very long history that heavily influences where we're at and where we're going. Go read materials over two hundred years old. They'll help you escape the current context and examine the world through fresh eyes.
I mentioned in my post on How to Practice that Finding good tests is honestly the hardest part. Well, here's such a test. For every point you don't immediately understand, you've got studying to do.
This is one of those pieces I add to the sadly poorly studied practice of software rewrites. I think rewrites are both too common and not common enough. They're far too common when a new batch of hires who don't understand the system come into contact with it and the institution doesn't have enough pressure to prevent them blindly rewriting it without first understanding the inherit complexity of the thing.
At the other end of the spectrum, they're too infrequently done by developers after they've built a system but repeatedly patched it in response to the real customer needs until it's full of things that only make sense in the context of some future ideal state and two other legacy designs they still haven't moved off of.
The two key variables in play seem to me to be experience of the developers with the codebase and size of the change required of the codebase. Sadly, I only know of a few fairly old and limited studies on the empirical evidence associated with rewrites. Don't get me wrong, there's a lot on refactoring, but rewriting as a concept has become a taboo word we should reclaim. There are absolutely times where both the fastest and highest quality outcome lies in rewriting the software based on the understanding of what the customer seems to need now they've used the existing software.
Please stop trying to make self documenting code a thing. It's never going to be a thing. Naming your functions after the opening to War and Peace doesn't make it any easier to understand. Just write documentation.
I still remember when someone told me they honestly couldn't understand how you could create a web app without React. Sometimes I wonder how anyone manages to build anything while contently undermining the foundation. Maybe they're secretly engaged in a pursuit to generate perpetual employment?
The key to scripting is having a fairly extensive library of routine operations, easy to use lists and hashmaps, and a runtime that can ideally fit in a shebang. There's no reason you can't do this with C. You just have to learn the first hard truth of C, abandon the standard library. To that end, stb is kind of that, Sean Barrett's standard library.
Always look ahead to the extraction phase of the capital cycle. How are they going to get their money out. The two they'll likely reach for given the last few decades of software business models will likely be selling your inputs and selling space in the outputs. It'll be interesting to watch how this capability evolves over the next few years and who will pay the most for each.
New attack pathology. There's been a lot of physical device vulnerabilities over the last decade. Things we've mostly been discounting because they required wireless proximity to exploit. We have just figured out how to weaponize this. Imagine first collecting a router based botnet, then using that to spray exploits on routers based on IP geolocation to compromise a bunch of people in a rough geographic area.
Fairly timely piece given the current lecture series. I'm actually kind of excited by what lies beyond our digital tech obsession. I wonder what will hold our collective imagination once we see tech for the self-licking ice cream cone it tends to be.
From my understanding when this goes into effect it means that little clause in every software license that reads something like, THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND stops working legally. You sell it, you're legally responsible for it.
I've been saying for so long we need something like this. How it'll ideally play out is that companies that sell software will be legally on the hook for that software. Those companies will need liability insurance. The insurance industry will gather lots of data on what leads to payouts. That in turn will form lists of practices you need to follow to lower your premiums. Software will be forced to improve kicking and screaming.
There's an absolutely huge number of what ifs and uncertainties around this. That it's possibly happening at all is fantastic to hear. Half a century late, but better now than never.
Starting a series of Friday lectures about education. It's always important to understand what the purpose of a thing is. You only understand that by seeing what it does. Interrogating the name of a thing is completely counter productive. Thinking of what it should do from first principals also doesn't help.
People I talk to are always talking about education and the education industry as though they're interchangeable. Like schools are where the learning is. Seeing past the institution is important to start to understand it has interests and those interests are only marginally if at all interested in education.
You must constantly be on guard for sophistry. It's so easy to uncritically accept things that sound completely rational and logical when you already agree with it. Oh how we love viewing novices as incurable dunces. If you think novices over estimate their abilities, you'd be wrong. They just don't know what they don't know. Sure there are lairs out there. Seemingly a lot of them. But when asked to evaluate how skilled someone thinks they are, it's not that novices over estimate and experts under estimate. It's that novices have low precision while experts improve their precision.
This doesn't help explain why there are so many people engaged in bullshitting. Seek truth elsewhere for that one. But hopefully it helps people stop referring to the Dunning-Kruger effect along with many of the other studies being debunked in the replication crisis.
I don't use DragonflyBSD, but keeping up to date on the latest developments in modern filesystems is always a joy.
I've never used the patches, but it looks like someone's maintaining an OpenBSD patchset for it. I'm not sure if OpenBSD has interest in mainlining support for it. It's not really built in the spirit of increasing security, so I have doubts. Time will tell.
Not sure what the future has in store, but I'd really like to do more systems programming. Sadly making money doing this seems particularly difficult in Toronto. On the other hand, there's always building your own business based on Teller's wisdom that, Sometimes magic is just someone spending more time on something than anyone else might reasonably expect.
My career ambitions aside, this is a fantastic talk about building web application SaaS type things like a systems programmer. Namely that fewer dependencies means fewer problems long term.
Why is it almost nobody writing backend software seems to understand isolation levels? I think you should basically be required to present an oral argument about all the levels before being allowed to even connect to the database.
It was funny watching people freak out about the Timeless Timing Attacks which now allows attackers to precisely exploit timing attacks against HTTP2 services. It's only a problem because developers have been blissfully unaware of what Read Committed even means and why shards aren't the secret ingredient in the web scale sauce.
Not a lecture Friday. Just a praise for the Jepsen testing framework. If you're not familiar and work on a system that isn't just a monolith, you really need to read up on it. Ideally, start using it.
I recently brought up this model in a conversation when I was talking about product adoption and market fit. I think it's a great way to understand how people adopt technology. It's super important to understand that your late market fundamentally thinks differently about your product compared to your early market. The late market wants something different and if you don't understand the right time to pivot, you'll either end up passed by a competitor or alienating your early adopters too early and sink.
There's a great lecture by Grace Hopper I watched years ago. Well, the NSA has released a much improved version in two parts.
It's well worth the time. She's a pioneer who's understanding of systems both digital and physical are things we still get wrong. So many of the things she raises are only now reemerging as problems we're beginning to talk about. I always come away with something new just about every time I rewatch these.
That's so cool! I've obviously not reproduced it, but if it's true, that's such an amazing insight. I wonder what's clicking and what impact those clicks have on things in their vicinity.
If there's one field that works hard on understanding computer performance concerns but also has insane opsec to keep a near iron clad grip on the things they learn, it's high frequency trading. I'm not competent to understand how you can make more money if you're faster, but some people are apparently doing it. Here's at least a paper to get some insights into what they're doing. Some interesting insights in there too.
Too many developers are taught computer architecture like it's 1975. Here is why, reason by reason, hardware just isn't going to get exponentially faster ever again. Why programming sequentially is fundamentally dead, And why the future is in efficiently managing memory, not compute.
An exceptional primer on the physics and theory underpinning quantum computing. It's not magic. It's not a faster digital computer. It's something completely different. If we can get a real machine to execute the architecture from theory, at the scale we need, we should be able to compute some functions in fewer operations than required by a Von Neumann machine (what every existing digital computer is). Nothing says how fast these operations would be on a real machine.
I hadn't heard of The Young Lady’s Illustrated Primer, but I've lost count of how many times I've heard this idea regurgitated and recycled. Glad someone helped lay out where the idea came from and why it doesn't work outside fiction.
Being one of the earliest systems in AWS, S3 has a bunch of quirks unlike many other systems. Now that it's become a de-facto internet protocol for better or worse, it's worth really understanding it deeply, because changing it isn't something anyone can really do at this point. It also means the knowledge is now somewhat broadly applicable. That, or at least by understanding it deeply, you can begin to understand how and why other implementations have the quirks they do.
Great lecture by the creator of STP. I love the thought that maybe, just maybe, it's worth teaching students networking concepts other than just how to use TCP/IP. I'd argue we've stooped to just teaching HTTP and left the rest as trivia. Either way, a great lecture cutting through all the dogma of networking, trying to get you to understand how networking works as a concept.
Great primer on the PNG file format. It made understanding a bunch of the spec much simpler and helped me optimize my favicon spritesheet maker.
It also covers the QOI format. Alternative, arguably better, widely unsupported image formats abound. They're still cool to learn about. They're mostly great if you're writing your own desktop application and thus not beholden to Web Browsers Chrome.
A good, more rounded look at old software. It's not like old software was always insanely fast. Some of it was painfully slow in its day. The problem is there's so many things today that should for absolutely no reason be slow, and yet they are. If you're transcoding video, that's going to be slow. That software has probably been heavily optimized. You're just working with potentially terabytes of raw uncompressed data between formats. That's going to take a while. If you're typing in a text box, why for the love of computation is that slow? What could possibly be that hard about converting keyboard input into glyphs on screen? Nothing. It's just garbage software.
I know a bunch of these and yet there's still more. Most systems have Python installed by default. Turns out, that brings along a bunch of cool utilities for free. The one I probably use the most is python -m http.server which lets you serve any directory on your system as a simple HTTP server. Great for transferring files in a pinch and some basic web development.
You can fight entropy, but it's embrace is all but inevitable in large enough teams (measured by total having worked on a codebase, not just those currently working on it). It's just entropy and software's lack of real world constraints played out. You can keep cleaning but you're fighting a loosing battle against physics.
Not only that, but the second law actually implies that not only is cleaning a waste of time long term, but letting complexity spiral out of control also counter intuitively imbues the software with a competitive advantage. Namely that trying to EEE or clone a convoluted messy system is much harder and therefor more expensive.
I'm not really sure what to think about this. Obviously I don't like it. I'll probably keep tilting at windmills for a while.
I love playing out these situations and constantly seeing just how badly globalism has left whole countries hollow. I quipped a few years ago to a coworker when discussing economies that you can't eat data. That an economy built entirely on service jobs might have some structural issues. Based on their reaction I worry things are probably going to get a whole lot worse before they get better.
Really great stuff in there for anyone building linting tools. One of my favourite things about linters is that they force you to flag individual lines of code as knowingly problematic. Rust's unsafe block is a fantastic idea. For example, any time I want a global variable, I'm forced to add a little comment to acknowledge that this is generally a bad idea. It communicates clearly to others reading that I'm doing something that usually causes problems. Then we can have a discussion about whether or not my choice is a good one.
I'd love if we could get better data into linters that help clarify what we actually know causes problems and what things are just somebody's opinion without substance. Like if each entry had an empirically measured effect size. Then I could trivially tune the linter to pick the level of complexity reduction that seems appropriate to my use case. Honestly, just any empiricism in these tools at all would be nice. For now at least, rules can be turned on and off in a way that improves discussion among practitioners.
This so neatly summarizes years of experience in building and maintaining software in an essay that's likely straight forward enough for non-experts to grasp. It's fantastic, worth a read, and likely worth sharing.
Great deep dive on the resource management systems in the aviation industry. It's always fascinating to me learning about all that's going on behind the scenes in systems that make huge amounts of that complexity their problem and not yours.
You can either get involved in your polity to help shape the future you want or accept what other people without your point of view decide for your future without asking you. Politics is just the methodology by which groups of people decide who gets what when. If you want things, maybe get involved, even if you just want to be left alone. It would be great if computers could go back to being a niche hobby for the socially distant. Too bad we let them eat the world.
This one definitely needs an introduction to help read it critically. This is part of the declassified Office of Strategic Services' Simple Sabotage Field Manual. It's goal was to outline the, innumerable simple acts which the ordinary Individual citizen-saboteur can perform to help the WWII war effort within axis aligned institutions while trying to hide behind plausible deniability. Essentially, weaponize incompetence. When reading this, stop and think about the point. Not only about what it says to do, but what the opposite of that involves.
Great bit of philosophy to rethink your metric for action. Doing nothing is usually better than being busy doing nothing of value or being fully engaged in doing the wrong thing.
I really wish I had something like Jai's use block. When you dangle a function out there that only one location's supposed to call, you just know someone will add it to their code to spite you.
Great rundown on common places to get better JavaScript performance. I really hope it goes without saying that you have to profile your actual code and not just rely on these micro-benchmarks. That said, micro-benchmark write-ups are a great place for inspiration for things to try in your own code.
I still don't think you should, but if you're going to write programs in bash, you should at least try and know what you're doing to prevent shooting yourself in the foot.
On a simplicity kick. Seriously, stop complicating things when they're too simple for you. A program really can just be a series of nested function calls and no more. Resist the temptation to "simplify" an already simple problem.
The amount of time it takes to remove code you didn't write in most code bases is way too high. Too many people think the source code is the valuable part of the company when it is quite clearly the mental model in the heads of the programmers that provides at least ten times the value.
The trick to reading code is to stop trying to play computer in your head. It's also how you debug code, modify other's code, etc.. You're sitting in front of a computer. Let it compute things for you. Said another way, don't think when you should compute and don't compute when you should think.
Design for Conway's Law, not against it. Seriously, what this shows is that software that has multiple people working on it who's lowest common manager is high up in the org (i.e. they don't share direct manager but only a director or worse vice president) have the highest number of bugs in them.
Getting performance right is important. Deciding to change everything and rewrite it all because you can't be bothered to measure and try and fix things is an all too common hubris.
Very simple concept, if your writing isn't obviously valuable to the reader it's worthless. Acting on this is much harder, but at least with the right direction you can aim to improve.
Great talk about the history of many languages. I do like that we don't have to commit to purely functional languages. You really should be writing more pure functions though. I get side effects are necessary and mutable state is faster. That doesn't mean you can't collect the mutability and side effects into isolated portions of the whole.
I always forget to do this in Postgres. Though arguably it's usually not an issue since most of the performance issues I track down are pretty obvious when you see the query planner trying to run a nested loop join.
