EmpowerApps Show 216 with Graham Lee

---
From Hot Dog Detectors to LLMs
---

[00:00:00] Leo Dion: Welcome to another episode of Empower Apps. I'm your host, Leo Dion. Today I'm joined with Graham Lee. Graham, Thank you so much for joining us again.

[00:00:10] Graham Lee: it's great to be back. Thanks for having me back.

[00:00:12] Leo Dion: yeah, how are you doing?

[00:00:13] Graham Lee: Good, good, good.

[00:00:15] Leo Dion: it was good to catch up iOSDevUK. and I really liked your presentation and I think, you know, we've done a lot of episodes already on AI, but I think there's there's a I think you bring an interesting perspective I wanted to cover today. but before we get into that, let's go ahead and I'll let you go ahead and introduce yourself.

[00:00:35] Graham Lee: Sure, thank you. so I'm Graham. I live in the UK. I it's pretty hard for me to keep track of what I do 'cause it keeps changing a lot. But at the moment I am running a kind of one person company called Chiron Codex, that is about understanding what the kind of shift towards LLM-based tools and AI augmented software engineering means for us as software engineers and means for our industry. So I'm writing a book that's about interaction patterns in AI augmented software development. I've got an app called Axminster. I'm sure we'll talk about that later. That's in review at the moment, so it might be out. Hoping it's be gonna be out by the time that this episode airs, but you can you never tell with the App Store review process. do some like consultancy for engineering organizations helping them to like make sure that their processes are going to be things that they actually want to improve by using AI. Yeah, the way I think of it is that like AI can be an accelerator, and acceleration has both an amount and a direction, right? So you've got to like make sure you're pointing in the right direction before you kind of you know push the pedal down. otherwise you're just gonna do the wrong thing much faster.

[00:01:51] Leo Dion: Yeah, yeah, that's a good point. What's your his how has your perspective changed on AI since the beginning of the decade, I guess? And were you surprised by its rise or were you like or what what has surprised you what has shocked you? What do you think people get wrong? That kind of stuff.

[00:02:14] Graham Lee: No, that's a really interesting topic to explore because it has changed quite a lot. I mean I started doing what we was then called machine learning back in about twenty sixteen I think. and I was working for a financial technology startup in the the realm of giving financial advice. So it was like software mediated financial advice. to like individual people. And in the UK, like I'm sure it's regulated in many countries. I'll just quickly explain that like in the UK, financial advice is regulated. You have to be a professional financial advisor to do it. And you have to basically sign your name to the advice you give. And it's like your reputation and your career that's on the line if that advice turns out to be like wrong or fraudulent or anything like that. and that has meant there's quite a bit of in some ways risk aversion towards adopting new software, but it also means that there's a lot of opportunity because someone who is willing to like try out the software can basically do whatever they want as long as they're willing to read the thing that comes out at the end of it and sign their name on it. so yeah, about ten years ago, we kind of built a machine learning model. That was like back when you like literally took a bunch of CSV files and a Python script running a module called pandas or something like that and like made your own neural network and then trained it from scratch. And we trained it on the case history of like a particular financial advisor so that we could categorize what advice they would give in like you know in another situation. Or we could train it on another financial advisor and compare the two and see if they were consistent. see if a particular document was like an outlier for compliance purposes, all this kind of stuff. It was a load of work and like the results were kind of you know, they they were good. It was interesting that they it could do it, but it like a huge risk to kind of you know, base your entire financial advice business on this. It was a a useful kind of side tool, like a a kind of cross check. And then when generative AI and large language models came out, firstly I'll hands up, I did not see that coming. Yeah, we were in the realm of an AI tool could tell you whether a picture had a hot dog in it, eighty five percent of the time, and they'd be wrong the completely wrong the other fifteen percent. And then we suddenly went to you can like do some kind of cool demo, so you can get it to give it give you a summary of a Shakespeare play as a limerick, or you can get it to write a Valentine's card with the name and some interesting attributes about like someone. but at no point did I think yeah this is going to be kind of useful for generating working software until the point where it was. yeah and even when I was kind of like evaluating these tools kind of on software teams as part of like a software engineering experience, I was going, Yeah, that yeah, it can generate code, but the code doesn't compile, it doesn't do the right thing, this is never gonna be useful. wait, hold on a second. It suddenly became useful and there was a kind of shift that was imperceptible when you're in it, but looking back on it was very quick. sometime around twenty twenty three, they went from being interesting demos to actually useful, valuable tools.

[00:05:34] Leo Dion: Right, right. Everybody was adding it to their website and their app. and I so one of the things too I've noticed is a lot of people have a negative and not I'm not disagreeing with this, but a lot of how people have a negative opinion of AI because they're kind of force-fed it and it's usually like prose, which it's getting better at, but it's still not great. Image generation, video generation, music gener right, that kind of stuff. And it's like That stuff is okay to garbage. And those of us who are like software developers, I think part of it is just the people who make this stuff are software developers. So it's they understand what they're training, right? But like for those of us, it's like wow, like the unless you're philosophically against AI, like it is extremely useful. For a lot of people getting a lot of code out. a and there is there is like, as we'll talk about today, there is a right way to do it. so like yeah, it seems like that to us it's like, yeah, we're on the cutting edge of where AI will be, whereas everybody else is kind of experiencing like dregs of AI training, I guess.

[00:06:53] Graham Lee: Yeah, and I I think that there's there's also a part of that reaction and it's like you know, it's totally like valid reaction and yeah, I don't want to kind of like sound like kind of sort you know AI boosters saying, right, this is gonna like change the way that we do the entire society and yeah. you know, there are contexts where it makes more sense. And like what I want to kind of come back to is like the the idea of creativity. the reason AI music does suck. And the reason that AI music sucks is that it is generic. you say, I want, a a three and a half minute hip hop track. It will give you like what is the most middle of the road three and a half minutes of hip hop like you can possibly imagine. and there is yeah, there are kind of different s kind of schools of thought on whether software is like you know an an art a kind of creative mindset or whether it's a tool, whether it's like a constructive thing. And the fact is that both are true. Like you know, software is engineering. That is the combination of you know art and design and science to creating artifacts that people use and want to use. A lot of software is entirely what you might call like blue-collar code, right? You know, if if what you're doing is taking some JSON from a web service, maybe paginating it, caching it, like and then kind of ordering it by date so that you can show it in some kind of timeline feed. The bit where you design the timeline feed and what the user can do with that is you know creative and artistic. And the bit where you just like load some JSON from a web service is entirely generic. And it doesn't matter whether that software is generic in fact putting too much creativity in there is probably the wrong thing to do because we we know how to solve that problem. We've solved it three thousand million times in the last, tw six months or whatever.

[00:08:50] Leo Dion: Right, right, right. isn't the joke that iOS development is basically you have fancy formatter for JSON,

[00:08:57] Graham Lee: Ha ha.

[00:08:58] Leo Dion: That's what I've heard like non I guess non mobile developers kinda say that. So like yeah.

[00:09:04] Graham Lee: yeah, been through parts of my career where all I'm doing is putting JSON in a table view over and over again.

[00:09:10] Leo Dion: Yeah. So let's talk let's so let's talk about that and I I like this term. I think it's still valid the software centaur. I'll let you go ahead and explain that and kind of where we are as far as that's concerned.

[00:09:26] Graham Lee: Sure. So yeah, Garry Kasparov, who was the Russian chess grandmaster who was beaten by an AI system and not a large language model, a different like approach to AI, in the nineteen eighties, which was an IBM call computer called Deep Blue. He created this term the centaur to refer to humans whose capabilities were augmented by AI. And what he was referring to, or like he was kind of extrapolating from his experiences in chess. Like every chess grandmaster who's playing today has much higher scores than Garry Kasparov ever achieved, like, yeah, when i in his professional career and all of his contemporaries. Like they are better and the the score is kind of you know like weighted by based on Who you win against and what this their score is and everything. So it's the people playing chess today are objectively better at playing chess than people who were playing chess in the 1980s. Why is that? It's because they now use the AI tools that defeated Kasparov in the 80s to play chess, to learn how to play chess, to develop new strategies and to get to understand positions and to get better at chess. So a centaur in the chess world. realm is a better chess player than any human who doesn't use AI because they're actually expanding their craft, they're expanding their tool set and their understanding using chess. So my question is what's a software centaur look like? Yeah. What what is what is the way to become a better software engineer through using AI tools? Because the alternative, which is a fear that a lot of people have and it's justified, is

---
The Software Centaur
---

[00:11:13] Graham Lee: that we become worse software engineers because every time we get a problem we just paste the problem into the you know into like Claude Code or whatever tool we're using and then like hit yes every time it asks a permission question and then commit the result or like ship the result to the App Store without understanding what's happened in the intervening time.

[00:11:35] Leo Dion: isn't the answer yes? Isn't the answer yes, software will get worse and yes, software will get better because depending on the company and w the willingness.

[00:11:46] Graham Lee: Yeah, absolutely. So there will be some places that want to kind of get better at craft. There'll be some places that want to like just ship more and ship faster. Yeah. there'll be some places that just want to fire a bunch of people. yeah, so that's another point is you know, in my opinion yeah, i it in in my opinion, like technology has never stolen a job, right? You know, employers have stolen jobs and they might use technology to do it. But anyway, like Yes, there will be a vast variety of experiences. and so it depends on to some extent and always has done. Whenever people have said like, TDD is the way that you need to do software engineering or you need to use functional programming or like whatever this week's intervention is, you know, you need to use like memory safe languages or type safe languages or whatever, well you have to demonstrate that that's better and like you have to demonstrate that you create better products as a result. So if you know, if we want to say there's a better way of doing software using AI. We kind of have to qualify what it means to be better and like demonstrate that qualification.

[00:12:54] Leo Dion: so yeah, let's talk about that. We'll jump right in and say what what are some keys to making better software with AI?

[00:13:06] Graham Lee: So it gives us the opportunity to use all of our understanding of AI, also to sorry, all of our understanding of software, but also in some senses to use all if everybody else's understanding of software. Like, yeah, these things are trained on like you know, huge amounts of knowledge. And that means that like if we understand sort of how our software development process works as a system, but we maybe don't understand one particular aspect of it, we can use these tools to do a passable job at that. So like, you know, as I said, when you ask it to generate music, you get super gener generic music out. but like if I don't understand a particular part of software engineering, I'm not that good at operations. Like, you know, I've written a couple of like apps that have Docker containers and Kubernetes clusters and all of the buzzwords in and I've either like well in fact every time I've done that, professionally, I've been working in a company where there are ops people who can help me get it right or can like you know give me a platform on which I do that right. But like if I if I'm working on it and I want to just like kind of produce a prototype or maybe I'm working in a team where everybody has like iOS experience and nobody has the kind of depth of back end skills to get that operations part, that kind of deployment part correct. We can do an average job of that now. Yeah, we can do a generic Kubernetes back end without actually understanding what it is. I'm also like really bad at yeah, user experience and design. I have no graphical skills at all, but I can do an average job of it now. And I can also like see when the model I know, I can generate something that looks bad and go, okay, that looks bad. I know that I actually need to go and find a pre professional to do this. So I can go and collaborate with an ops person if my like generic backend stuff doesn't work, or I can go and collaborate with like a designer if my like generic approach to design doesn't work. But I have the opportunity to at least like explore that space and understand my problem in much more detail. Rather than just treating a designer as a completely alien person who speaks a different language and going, can you just make this look good? I can say, right, hi here's what I've tried and I can see that these things are bad. yeah, let's now have the conversation where we combine our expertise to make a great thing.

[00:15:46] Leo Dion: So it sounds like you're saying like the floor has been raised, but like how and speed is faster, but how can we get better quality? Is it just basically what you're saying is everybody has more I'm trying to like put this all together, but everybody has more experience accumulated because AI brings that to the table? Is that is that correct?

[00:16:15] Graham Lee: It's great. everybody has access to I mean, so like you know, t taking the internet, pre AI, everyone had access to tons of information about writing software, right? Yeah, there was Stack Overflow, there's Experts Exchange, there were all sorts of like forums on particular technologies, there were discussion groups, mailing lists, like Apple's developer forums. and you needed to like know what you were searching for, know where to search it. like know how to kind of make sense of what he was saying. Or, and this is relevant to the kind of LLM story, like take something that looked roughly correct from Stack Overflow and paste it into your code and not understand how it worked. Like that was that that has been considered.

[00:16:57] Leo Dion: I well I I obviously don't know firsthand.

[00:17:01] Graham Lee: but yeah but like it was quite a lot of work to do all of that research. Now doing like a kind of first pass of that research is really quick. You know, that that, if anything, is a more impactful use of AI than even generating code. It's just like generating answers to questions based on a corpus of training data. And we know that there's like hallucinations and we know that there's risks that the answers aren't valid, so we still need to kind of apply our critical skills, but we get to the point where we have something that we can critically analyse way earlier. so yeah, we have more access to a load of different skills. We also have the opportunity to use that to augment our own learning. Like a specific example, I do know something about software testing. Like I wrote a book on test driven development like a number of years ago. and so I know how to write automated tests. I know how to write UI tests and integration tests, unit tests, performance tests, blah blah blah. There's a technique that I used to not use very much. I understood that it's value, but I just didn't really understand how to apply it called mutation testing. And what mutation testing is, is you take your existing test suite and you take the software that you're testing and you just change something in the test. So maybe like You've got sorry, you change something in the code. So maybe you've got some code that adds two numbers together and you subtract them instead. Or you always return zero from that and you like don't actually do the addition. And you run your tests again. If your tests still pass, then your tests aren't very good because like you have some like behavior that you depend on in your app that doesn't actually get tested. and so what you then do is you refine your your tests so that they would catch that problem and then you know that they're working as like regression checks. And I didn't really do that before, but now, like maybe sort of two years ago, 18 months ago or something, I just kind of prompted a model like, hey, how how would mutation testing work in this code base? And it like came up with a Python script that would like make changes to the code and then it ran the tests and it said, okay, so these six things didn't get caught, so we're going to delete the tests that test them and redesign them. And I was like, okay, now I understand this thing. And now I use mutation testing all the time on pro on my projects. It's given me the ability to apply a new technique to my code. Not because I've told, told told a model, hey, go off and do this, but because it's helped me understand how to do it from how I get the value out of it.

[00:19:46] Leo Dion: Right. It's it's almost like teaching by example, I guess.

[00:19:50] Graham Lee: Yeah, totally. at that kind of teaching by example. They're also good at Socratic teaching. So you can get them to like just generate a bunch of questions about your code or about your product or about you know, like a problem statement you've got from a customer or something, and just you know, check that you actually understand everything that that is written down there.

[00:20:11] Leo Dion: Yeah, when we did the episode with Donny, he was talking about this grill me skill that I've now used that basically asks you a ton of questions to confirm stuff. So yeah.

---
Cocoa Thinking & Store-Bought Software
---

[00:20:21] Leo Dion: I do I do want to cover one more thing about how and I this is the part I thought was interesting about your talk. How the history of computing and your experience with Cocoa and Objective-C affects how you look at AI and how LLMs have affected software development.

[00:20:44] Graham Lee: Yeah, that's really interesting. I was reflecting on that this morning. What Objective-C or like object oriented programming in general is about modelling a software system as a collection of like little independent computer programs that communicate by sending messages to get a big thing done. And a lot of the techniques that you use are effectively like role playing bits of computer. Right? Yeah, you you like i if you're using what are called CRC cards, which is class responsibility and collaborator, you're designing an object by saying, I've got this type of thing that's in my code, what does it do? And what other objects are going to talk to it? And within that context, like if you think of each kind of object as an independent actor that's just receiving messages and then doing stuff and then responding. It's very easy to fit the idea of an LLM into that model because an LLM is you know effectively an independent actor that receives stuff and then like does something with it, in this case, generating text or tool uses. so you can you it doesn't kind of make it this is about like writing code in a different way. It makes it about this is using a computer in a different way, or even this is using a computer In the same way I was already doing it, but with like different possibilities. and then, yeah, as we'll come on to and I mentioned in my introduction, like my book is about patterns and the patterns and pattern languages of programming came from the object oriented programming world and it was a new approach to design that was based on understanding how certain shapes of problem have shared solutions and how you can identify that those problems are the same and apply the same solution to them.

[00:22:42] Leo Dion: Yeah, yeah. Well, I like I like the part about how the Apple I, I don't know how many people know that, but the Apple I is basically a kit, and it's like, yeah, you have to put your software all together and how that's rapidly changing. So yeah, like segueing from that, there's the concept of like having to build it from scratch, and like one of the things you talk about is how The job was never about typing, it was never about putting the thing together so much as delivering quality software. how should like let's just start with indie developers, how should they think about that as far as building building their apps in particular?

[00:23:28] Graham Lee: Yeah, so I like at the high level, one thing that I want to sort of I want to caution against a particular interpretation, and this ends up being like good news for indie developers. I want to like caution against the interpretation that like all software engineering jobs are gonna go away because we can now automate it. because like for how or rather like, you know, everyone can write their own software, so why would they buy any software from you? for how long has it been possible like to grow your own vegetables? Like something like twenty thousand years, right? Yeah. Do you grow all of your own vegetables or do you buy vegetables from the store?

[00:24:09] Leo Dion: Yeah, you well, depending.

[00:24:11] Graham Lee: Some people garden, some people buy yeah, exactly. even people who garden, for the most part, buy some stuff from the store. Either because it requires resources they don't have like particular soil types or you know, s more space than they need, or because they want carrots and they've only planted onions.

[00:24:29] Leo Dion: Right, right.

[00:24:30] Graham Lee: so yeah, but if twenty thousand years after we invented like agriculture, people are still buying vegetables from a store, or like yeah coming down a bit more r relevant to software, like we've been teaching literacy to the entire of you know, certainly Western societies for like a century or more now. people still buy books from other people. Like you could just write all of your own books, but people still read other people's books.

[00:25:01] Leo Dion: kitchens now and people still go out and eat yeah.

[00:25:04] Graham Lee: exactly. yeah, you can put you can cook at home, people don't. They go out to restaurants. That's a another great example. the idea that because an individual can write software, they will write all of the software that they ever need to use is like false. There will still be a market for selling things that are you know, i i where it's quicker like supermarket example is quicker to buy it off a shelf than it is to build it yourself. the kind of artisan example, you will get a nicer thing if you buy it from someone else than you would do if you made it yourself. You might get a more reliable thing. And so those attributes are still going to be valued. and that's what we're gonna have to provide. taking a particular h you know, I don't know how many to do apps there are on the store, but that seems like one where you might think and I certainly did like think when I s started considering how AI would affect software engineering. I used to think there's no more market for to-do apps because everyone wants them to work a slightly different way. So everyone can just get their own bespoke to do app that does exactly what they want. The thing is most people won't do that. They either won't want to do it or they won't feel like they have time to do it or they won't be motivated to do it. or and it'll be cheaper or better for them to buy someone's notes app from the store. So even though that seems like like a kind of low hanging fruit for everybody gets their own version that does exactly what they want, in practice that won't happen. And people's well, some people will use the reminders app that's built into iOS or the Notes app or whatever, but some people will still buy other ones because it's better for them than trying to do it themselves.

[00:26:49] Leo Dion: So my question for you then is there a middle ground opportunity there for I don't know go into the cooking analogy but like you know meal kits for instance or like like You here's a s seed kit or something like that for your garden or something like that. 'Cause I I'm just thinking it was a Ben Thompson article a few weeks ago that I've been dwelling on about like dynamic the idea of a dynamic app per se. do you think there's a place for that? And how would that how how do you think that would look?

[00:27:27] Graham Lee: So yeah, the the kind of analogy yeah, but we were talking about Apple's history and how we went from the Apple I kit computer to effectively, better fully integrated Apple II, Apple III and then completely changed the world with the Macintosh and then the graphical user interface and everything. let are there example from Apple's history, what is the HyperCard or the AppleScript of the AI process? world. is there a a world where you have bits of software components that are you know written by software engineers and bits where you kind of you know either glue those together or add your own components that are created by LLMs or where there's just like gaps in your architecture and you just say to an LLM, right, this thing needs you know, this type of information, this thing needs it y this type of information. Your job is to like you know, is to adapt those two things together and just kind of play computer every time that question comes up. from a yeah, from a technological perspective, that absolutely makes a lot of sense. in some ways, that is what the new version of Siri is supposed to do with like App Intents. You know, I come up with a c a question, and the and the contextual Siri but understands all your personal kind of context and information and what you're doing on your phone. Yeah, I come up with a question like I'm I've got a meeting with Leo in forty five minutes, work out which of these tasks on my app I can actually start writing before like the call starts. And and then, you know, it uses all that information to to do that. In practice, like the the the risks are are are the same as they've always been, but with a lot more cognizant of them now and what makes it difficult is the security model because you're basically saying, right, this little bit of arbitrary information over here, which might be confidential, it might you know, it might need to not change outside of certain situations, is now like interfaced with this model that will take some random like string of text and just interpret it as a set of instructions and then do it. Which is much more you know it that's more like going back to the world where y your computer just had a basic prompt and you could type in any program you wanted, even though the program itself is now free form text. You've got to think about what are the implications, what are the the impacts, what are the risks of doing that. And I honestly think that a lot of you know, because of the attitude to security particularly for s a company like Apple and their approach to privacy, the risk of allowing you to do anything with that kind of tool is such that we won't really get anything more than sandboxes, even though there's like huge usability opportunities there.

[00:30:22] Leo Dion: Do you think that's the right thing to do?

[00:30:26] Graham Lee: no. I you know, I I think that so I I come from a background like before I was in the Apple world, the reason I came into the Apple world was that I was like a huge free software person and basically like back in two thousand four, Mac OS X was like Linux, but it carried on working when you plugged in a monitor and you didn't have to like r edit a device driver file. So I come from that world where I think of it as my computer and I'm free to do what I want. And so I even on devices like the iPhone and the iPad, I you know, don't like the restrictions on what I'm allowed to do as a user and on like the software that I'm allowed to use. So but then I also acknowledge that I'm a particular type of person who like has put a load of effort into understanding this stuff in depth and so has much better understanding of the risks and everything there and like a lot of us are in a position of making choices on behalf of people who you know while they certainly could understand those choices, you know we're not kind of making any comments on their abilities or intelligence, we're making comments on what they choose to focus their time on and it ain't computers. So we're we're being trusted to make decisions on their behalf and so we have to make sure we're making the good ones.

[00:31:45] Leo Dion: Great. Yep. Yeah, I think some people just that's not their thing and they don't want to customize and that's that's the benefit of Apple to a lot of people is like it works outside the box. Or it works out of the box, excuse me.

[00:31:59] Graham Lee: In theory.

[00:32:00] Leo Dion: In theory anyways.

---
Meat Proxies, Jobs & Jevons Paradox
---

[00:32:01] Leo Dion: so yeah, let's talk about like having control as a software engineer. Because you know, there's the whole idea of like meat proxies and stuff. that term's been thrown around. But like where how do you make sure that you're in control and you're using the LLM to be actually productive? And not just letting it tell you what to do.

[00:32:27] Graham Lee: No, absolutely. And I c the first time I heard this meat proxies phrase, I I thought of times in my life when like I have been employed as a software engineer to take like text that you know, a problem report from Jira that a customer has like written or a you know, product manager has written and then an analyst has interpreted and then my job is to turn that into something that a tester is going to test and thinking, Well, I have been the meat proxy already, just like the this was well before AI even existed. right, the so this comes down to responsibility, right? The yeah, you have to understand what your responsibility is and to whom that responsibility lies. And for me, yeah, if you like thinking about kind of the Agile Manifesto, like the first principle is that like your your primary responsibility is the delivery of valuable software to your customers, or working software. that is kind of true. I would then like you know add some kind of ethical dimension and say that I also have a responsibility to society and to like the other people who are impacted by like by my software existing and saying that I have to do things that are beneficial to them as well as to the the customer, it can't just be about making more money for someone. within that so there's my fundamental framework. I am trying to deliver something that is of value to some people and that is generally beneficial to the the world in which those people exist or to the society in those in which those people exist. How am I going to do that? And if I can't answer that question, how am I going to do that in the way that I am writing my software then I am not writing my software in the right way. So if i i i if the customer wants f wants their software to do a particular thing and that's what they find valuable, I have to make sure that the software does that particular thing, which means I have to like A deliver that functionality and B like validate that that actually exists and that it does it in a way that the customer can use and is happy to use. Can I just make so firstly, c could I just like pay someone else to do all of that and then not inspect their work and just like, you know, outsource it and just give whatever the outsource agent delivers to my customer without like looking at it, without evaluating it, without asking any questions. Yes, but I probably wouldn't want to. What's the difference here? The the only difference is that rather than giving it to someone else, I'm prompting an LLM. So if I don't feel comfortable hiring a contractor and then making the contractor talk to the customer directly, I shouldn't feel comfortable about using an LLM and making the LLM talk to the customer directly.

[00:35:25] Leo Dion: at this. Do you think that's the perspective of a lot of the leading AI companies out there, or do you think they're trying to figure out a way not to have to deal with that?

[00:35:36] Graham Lee: There's a bunch of different takes. I think like a and they've also different people have put this in different ways based on how it lands, really. so there there is there has been a take and I think like Dario Amodei, the CEO of Anthropic, used to say this a lot and has pivoted away from it. That like, yeah, we'll be able to make kind of a bunch of different knowledge workers redundant. We'll be able to eliminate a bunch of jobs because you can just get computers to do it.

[00:36:07] Leo Dion: that would be taken negatively?

[00:36:09] Graham Lee: Well, so in in some ways that should be that could be a positive thing, right? You know, if you think about like Star Trek, nobody has a job in Star Trek because you know, the everything can be automated so people do the things that they want to do. Like, you know, no one's being paid to like be an officer in Starfleet, for example, or nobody is being paid to grow the wine on Picard's Vineyard. They do it because because that's what they want to do. The difficulty with saying we're going to like make everybody unemployed is a transition where we still have a society where you need to like have money to do stuff and that money comes from doing work. and that's why like making people unemployed is bad, is because we haven't got any like safety net for though for people who are unemployed. We haven't got any

[00:37:03] Leo Dion: We're unemployable.

[00:37:05] Graham Lee: or unemployable, right. so within that framework, like and particularly a lot of software engineers, I am one of them, enjoy writing code. You know, it's something that we like doing. Now just because we like doing it, it doesn't that doesn't mean that there is a requirement for somebody else to pay us to do it. Like, yeah, we I have no entitlement to be hired as a a a as a code writer because I like writing code. but if someone's gonna threaten the entire basis of like not only code writing but all of no knowledge work. And so not only can I no longer do that for money anymore, but I also can't do any of the other things I could possibly do without skill anymore, then yes, I'm gonna be threatened and I'm gonna feel bad about that. But a different way to look at it, the way that I have used on my blog, and I know you've got a link to this so it'll be in the show notes. and the and I'm not unique in this is there's a an idea in economics called Jevons paradox which Jevons was industrial age like mid nineteenth century he observed that as it became more efficient to to extract coal demand for coal went up basically like if you make it more efficient to use the resource you increase demand for resource. You can also see that if you've ever driven around Los Angeles. Los Angeles has super bad traffic. some people can like not get to work in a day because they get in a traffic jam for eight hours and they have to turn around to go home. and the way that LA has tried to fix this is by building wider and wider roads. And you know, you feel wider roads means more cars. so we should have less traffic. But people go, right, the road's better now. I can drive to this like destination, so they get on the road and then there's more traffic. so increasing the d the efficiency of using the resource, which in this case is the roads, inc increases demand so you get more traffic. Well, that should apply to knowledge work as well. If you make it more if you make it easier to employ human intelligence, there should be an increase in demand for human intelligence. There are way more knowledge workers in the internet era than there were in the kind of pen and paper era. And there were way more knowledge workers in the pen and paper era than there were in the era when everyone except religious people was illiterate. So yeah, we another way of looking at is that we're going to unlock a whole new collection of jobs in knowledge work and we don't even know what those are yet.

[00:39:36] Leo Dion: Yeah, I agree with that completely. We don't know what those are yet. despite the fact that we've created fields like prompt engineering and all that. Like yeah, I don't or context engineering, whatever. It's like I think we're still in the baby stages w with regard to that.

[00:39:51] Graham Lee: those might be kind of tasks that become part of the new work, even if they don't become whole careers and jobs in their own right. It might be that prompt engineering is something that everyone has to learn at grade school before they can, go to university. Or it might be that a whole bunch of different knowledge work jobs have prompt engineering as a skill that you need.

[00:40:12] Leo Dion: Yeah, yeah, exactly.

---
Interaction Patterns for AI
---

[00:40:13] Leo Dion: So let's get into patterns. well that was probably one of my favorite parts because a lot of those were like is it's and I think we I told you this after the talk, but like when we talk about like design patterns, you know, visitor pattern, etc., it's almost like, you implement that pattern. Like you have to learn the pattern so you implement it. But it's in fact more like what I found is I I implement the pattern and then I realize it's a pattern and that's what I'm finding with a lot of what you talked about in your talk about like these certain patterns that we follow and I'm like, yeah that that makes sense. That's exactly what I do. But yeah what like is that it's that kind of your perspective on it and then what what have you found is the the biggest lesson you've learned with these

[00:41:05] Graham Lee: So I presented an early part of so talking about my interaction patterns in AI augmented software development and I presented an early one of these to a workshop at conference, the XP 2026 conference in São Paulo, back in April. and a load of the feedback I had was people saying I have that pattern. I the thing that I have described in the in this pattern is something that they were already doing. And that is great news when it comes to patterns because what you're trying to do is as I said earlier, you're trying to like define the like universal characteristics of some category of problem and then say, so if you have a problem that fits in this category, here's a solution that you can apply. And here's the opportunities and the costs of using that solution, here's here's the other things it unlocks. and so what the AI patterns are about is me reflecting on my work with AI tools, like my colleagues, other people in the community who going, right, you know, I've or like yeah, people post on LinkedIn and say, Hey, I've discovered this thing I can do with an AI tool and you know, and I go, yeah, yeah, you're not the first person. There's a common yeah, there there's a common thread through this. Or people saying, I'm stuck on this problem, and then I can say, Well, yeah, my yeah, the people I was working with like a couple of months ago were stuck on that problem. We came up with this solution, you can try that. so really like the the fundamental thing that I've learned is that like we're still in a very exploratory phase. with using AI tools. We're still in the everyone is like either paying the whatever it is eighteen dollars a month to Claude and using that, or they're like, you know, downloading Gemma 4 or Qwen or something onto their computer and they're trying that. and everyone is kind of going through this like this same set of both like technical and also emotional changes as they learn how to how to work with these tools. and that that will settle down over time. but also a lot of what people need help with is or the a lot of the benefit that people can get is the going from type in a problem statement and get some source code out to type in a problem statement and references to your company's way of working or your project documentation or your coding style or whatever it is so that the thing you get out at the end, A works in terms of being able to like integrate it into your app or your project, and B is more likely to be something that you would want to like use and want to rely on. Going back to that point about responsibility, you know, the reason that we have coding standards and testing practices and code review and all these things is because we want to give our customers the things that we're comfortable with maintaining and that we're like proud to put our names behind. So a lot of it is about kind of getting there and not just having some random Stack Overflow style like function that just sits in space and like doesn't match the way that we work, which comes from our understanding of our responsibilities to the people that we work with.

[00:44:30] Leo Dion: Yeah, so let's get into some of these patterns. first of all, prompt management basically the idea is is having something either like create a prompt for you or record a prompt or give examples of a prompt, right? And let me know if I'm off base on any of these.

[00:44:48] Graham Lee: no, that's yeah, that's fair. So yeah, there there is so we talked about prompt engineering as like something that was very briefly a job that people could have and these days is more like a skill that a lot of people need. And so the prompt management patterns are essentially they aren't a guide to prompt engineering because that changes every time there's a new model or like new capabilities in the tools or whatever. What they are is a framework for how you think about prompt engineering. So the core of the prompt management tool set is Bake Off. And Bake Off is like it it is the scientific method applied to prompts. So you know it's very like we know that models are non deterministic, or at least like if you use them with the like the ways that you typically use them are non deterministic and they'll randomly choose like likely tokens to complete your stream. so Bake Off says like if you've got something that kind of that works like to some certain extent and then you think you want to like make a change, for example, editing your prompt to like add a a different rule or to change the wording or to make something explicit that wasn't before, try your thing out a bunch of times before you make the change. Make only that change, try your thing out again a bunch of times. and then compare the results. Now, yeah, that has a a like a word that people use, which is an eval. You what you're doing there is performing an eval on your entire LLM setup with like two different prompts and seeing how that changes. But what's important is the process, it's the identifying what problem you have or what like improvement you want to make. hypothesising that a particular change will make that improvement and then making only that change and testing whether it does or not. so like within that you then have a continuous improvement loop that applies to prompts but it also applies to you know evaluating different models or to changing like the parameters of a model or to like switching your harness tools or whatever.

[00:47:01] Leo Dion: Okay. Yeah, I like that. so next we got I think this is a quick one, memory management, just the idea of like somehow exporting part of context into something that can then be read out anywhere else.

[00:47:16] Graham Lee: so yeah, the a model itself has like all of the weights that were created by feeding it tons and tons of training data and then those are tweaked and biases added by a process called fine tuning, where basically humans kind of set goals and then evaluate the output and then like nudge the model so that it goes more towards particular types of output than others. And then what happens is that like all of that is saved to a a really big file and that is the model and that is the thing that you use in inference. And it never changes again. So that model would always work the same way given the same input information. so the memory is about choosing information in your project, in your context, in your organization, that is relevant to the tasks you're trying to get the model to do that you attach to the prompt so that it does it in ways that are appropriate for for your tasks. And and here I have to admit, like, it is very hard to talk about AI without pretending in some way that it's intelligent. and so I'm going to use words very loosely That sound like human cognition words and really are just analogies. But like what you're trying to do with memory management is to get the model to learn, and I'm using my scare quote fingers as I say that. learn what the way is that you want it to work. It's not learning at all. You're just feeding it like extra rules or whatever,

[00:48:54] Leo Dion: Right.

[00:48:55] Graham Lee: every time you ask it a question or every time you tell it to do something.

[00:48:58] Leo Dion: Yep. Discover Opportunities. I really like this one. I feel like I used that a lot early on when I was trying to get ChatGPT to do prose. And then and then it was like once I started l using prose less, I've been yeah, using this less, but then I think it's really good for basically the kind of stuff not so much code you're gonna disagree, but I I think it is good for code, but I don't think you get as much bang for your buck compared with images or text or video, et cetera.

[00:49:35] Graham Lee: So yeah, so I w and I think that the patterns in the discover opportunity section are some of the best things for people who are like maybe kind of coming to generative AI tools for the first time. Like you know, maybe you've been kind of sceptical about how well they'd work, but yeah, you know, your employer is saying you have to use this now and so you're kind of coming on board or you know, you're just kind of ready to take a go. These are the patterns that most that are a easiest to adopt because they don't actually make any changes to your code, and also help to help you to understand how you can integrate these tools into your workflow. So like the first one is Just Ask. that's what we were talking about earlier with the like the Socratic method for doing like reviews or for doing education. Just Ask the model questions about your project. Ask the model so I need to add this feature. Where is the view that has these controls on it? where is the view model that shows this data? How does this like field get into the data that goes to the back end? What would happen to performance if I changed this list of lists? into dictionary or if I used SwiftData instead of a flat file for this data.

[00:50:59] Leo Dion: Right.

[00:50:59] Graham Lee: and the others it the other patterns in that collection are kind of similar to that. They're all about like finding out things. The most interesting one is Disclose Ambiguity because that's about finding out where the gaps in your prompt are. So you know if you you want the LLM to generate some software, don't just like type your prompt in and see what it does. Type your prompt in and then ask what are the hidden assumptions in this prompt? And then you might f you might find that you've like missed some stuff out and you can answer it. You might find that your customer doesn't know what they want and you have to go back to them and say like hey so what do we actually do here? and that's that yeah that is where we start to get into the kind of centaur superpower thing because then we're we're out of the kind of turnkey do what our customers tell us to into do what our customers want f us to do and collaborate with them on designing great software that solves their problems.

---
Quality, Tests & Anti-Patterns
---

[00:52:00] Leo Dion: I wanna hop ahead to quality improvement because that's the part I'm really interested in. A lot of the gains people see in AI are typically speed. Where where can AI improve the quality of your best code, let's say?

[00:52:18] Graham Lee: Yeah, so and then let's like let's combine the two. So the first thing and a very common pattern is the You Review pattern, right? You know, you you tell your LLM your prompts, you generate a bunch of code, and then you sit there and you try and read all the code. and A, these things can generate code much faster than like any of us can type. and so there's way more code to review. and B, that's really dull. we none of us got into software engineering because we wanted to read other people's code all day. We would in it because we wanted to write code and make the computer do interesting things. So so all we've done is like made software engineering slower and more boring by like adopting this tool. That doesn't sound great. and then people have the great idea, well, why don't we just get the model to review the code? Because if the LLM can like write the code really fast and the LLM can read the code really fast, then We'll just get it to do both bits, but then it's effectively marking its own homework and any problems that you had prompting it are going to remain in the output because the your your instructions for make sure this does the thing that I wanted are based on the same wrong instructions. So then we have to work out, okay, how do we get this thing to do things better and faster because we want both. And part of my answer turns out to be to not always use an LLM. there's a pattern called Replace Vibes with Tools, where I say if you've got some deterministic part of your process, like say you want to v verify that some software does a particular thing, don't ask a model, does it do this thing? Write a test. Because that test is objective. It is deterministic. Every time you run that test, you will get a pass if the software actually does that thing you'll get a fail if it doesn't do that thing. or if I'm not entirely sure what my customer meant and I can't get my model to like fill in the gaps because they don't know what the customer means. You know, it doesn't know what the customer means either. It's just like reading this text. Why don't I just build a prototype? So like yes, these things are great at generating code. We don't need that code to go into production though. We could build a prototype. show that to the customer, go, is this what you mean? They could go, well, no, actually I meant this. And then we could go and build the software and throw away what the model wrote and just build whatever it is we've learned from that experience.

[00:54:37] Leo Dion: Yep. Yeah, I like that. Yeah, a good point about prototypes too, because like I mean I'm just thinking of like a boss who's just like, just go do this and then you do it and That's not what I wanted. Yeah. yeah, that's not what I want. Right. You they're like frustrated or asking details and at the end of the day, like a prototype for both us when we talk to an LLM, but also for people who are our bosses. It's good to have like an actual example to see.

[00:55:08] Graham Lee: Yeah, so there used to be this phrase that sometimes a month in the library can save an hour in the lab. yeah, it's true. Just tr just try the thing out and see whether you were right or not.

[00:55:19] Leo Dion: were there any other patterns you wanted to cover before we move on?

[00:55:24] Graham Lee: I think that like that's a good overview of what the whole like pattern catalogue is about. Like that you know, going into the details of all of them is perhaps just going to glaze people's eyes over. Like yeah, the idea is that in each of these cases there is a thing a thing that many people are trying to do with an LLM and there is an answer to, well, here's how you can do that and then Here's some new problems you get because you've solved that problem.

[00:55:54] Leo Dion: Yeah, yeah, fair enough. are there any anti patterns you've seen that people do that you're like, No, that's not the way we're supposed to do it?

[00:56:04] Graham Lee: So like obviously the big anti pattern from a software engineering perspective is vibe coding. yeah, that I have no pro Right.

[00:56:15] Leo Dion: Okay, yeah.

[00:56:16] Graham Lee: So I think it was Andrej Karpathy came up with the with the term and it basically means just generating software by typing like what you want, you know, typing a problem specification into an LLM and and then getting software out and then just running that. So it means like not having an engineering process. and I have no problem with people doing that. I you know, I think that like this is one of the big moments in the history of computing where computing becomes an empowering tool that enables more people to do more things. But going back to what I was saying about responsibility, as software engineers, our responsibility is to make good quality software that does what our customers want and that benefits society. You can't guarantee that software that you've never like read or interacted with and was just generated by a language model has those attributes. Yeah, you can't guarantee that it has like the security qualities that people need, the performance qualities that it actually does what they want. but that was the cheap answer, right?

[00:57:27] Leo Dion: Yeah.

[00:57:28] Graham Lee: Saying the vibe coding is an anti pattern to a podcast for like developers is not really kind of giving anyone any new information. An interesting like anti pattern in the way that people are using these things in like in software development contexts is the rules file. So if you have like an AGENTS.md file or a CLAUDE.md file. Yeah, people put all of the things that they want the LLM to pay attention to when it's generating code in that. two problems. Like one is as this file gets big, it fills with thing like it fills up context and not every rule in there is relevant to every task you have. Like, you know, if you're doing kind of if you're doing changing the data schema, then you need your migration rules, you don't need your UI automation rules. and so you should separate those things out. Another thing that people do is they kind of give there was a workshop and you know I I'm not kind of like calling this out but there was there was a a workshop at the conference where someone was showing their rules file and it said, like, you are a staff developer at this, you know, company with like expertise in this areas. and they said, Well, why would I have a junior developer if I can have a staff developer for free? you know it's kinda

[00:58:50] Leo Dion: the whole you are thing, I think that's dated. I don't think that's a thing anymore.

[00:58:55] Graham Lee: Yeah, so it turns out that what that is good for, is increasing alignment. So if you have particular safety guardrails, you want your model to do things in particular ways, giving it a role makes it align better with those safety guardrails. But it actually reduces it like the accuracy with which it completes tasks. So if you look at kind of benchmarks like Terminal-Bench or SWE-bench or something, a a rules file that like gives a role to the model means doing getting a worse outcome on those benchmarks than not bothering. So again this goes back to the idea of a Bake Off and actually testing any of the changes you make.

[00:59:35] Leo Dion: Yeah, yeah, yeah, totally.

---
Axminster & UML
---

[00:59:37] Leo Dion: all right, so we're one hour in, and I think I should probably give you an opportunity to talk about your project. yeah. what have you been working on lately to help with this?

[00:59:50] Graham Lee: about the we've talked about the book that I've just actually been through all of the kind of technical and editorial review comments from the publisher on that. So that should be coming out. It's published by Apress, currently due to be out March twenty twenty seven. So, you know, maybe pre order it as an early Christmas present for all of your friends, but like you'll they'll have to wait until Easter to actually get it. But I've also been working on an app called Axminster. and Axminster is a way to visualize the conversation that you have with an LLM and to get some of this information out of Markdown files that you know we've already talked about the problem with like reading large code files. Reading large Markdown files is not much more fun. And getting that into a visual canvas so that you can kind of see what the different types that the model is going to generate are and you can change that, you can relate them together. And it's based on and you know you might be saying why are you talking about this in 2026, Graham? It's based on UML, which is a modeling language from the nineteen nineties. But it turns out that like UML has a lot of benefits. It has a well defined semantic structure. So the different lines and arrows and boxes on a UML diagram mean something precise in the context of software design. So if you have a UML diagram and then you feed that information to an LLM, the LLM knows exactly what you mean by all of those boxes and arrows. And so it can generate code that does exactly what you want. It also has a built-in constraint language, which is a way of encoding rules and correctness information directly into the diagram. So you can say, like if you have a state diagram, so you have a state machine that like has events that move a system between different states, and you have a guard that says like this transition only happens when this condition is true. You've basically put a constraint on the LLM so that when it generates code the code has that logic built into it as part of the design. So I think that this is a really powerful way to move beyond walls of Markdown files and chatbot interfaces and get into a partly visual, partly textual model of the software that we're building with AI tools.

[01:02:20] Leo Dion: So yeah, so basically you have like this MCP server that takes the diagrams and turns it into code and but also code can turn it into diagrams. and then how are the di how are the diagrams I mean know UML, but I mostly know it graphically. Is there like a language or do you use Mermaid or what what exactly are you doing?

[01:02:44] Graham Lee: No, so there's no kind of well, so the documents are stored in an XML format called XMI, which is a

[01:02:51] Leo Dion: Okay.

[01:02:52] Graham Lee: a standard format that's interchangeable. But no, the model doesn't work with the XMI files. The model works with the MCP server, which exposes a c so like let's back up for people who don't understand or don't haven't heard the phrase MCP, it's model context pro protocol. It is a server client protocol that's designed for the client to be a tool that has an LLM in it and for the server to provide capabilities or information to the LLM that's running in the client. So in this case it gives tools that allow the model to ask questions about the diagrams about your software design and to make changes to those diagrams and to that software design. So The way that round trip works is when you say, please generate code that conforms to this model, Axminster is running. You're looking at the model on the you know in the window if if you want. and the model is asking, okay, so what classes are in this design? And it gets a list of classes, and then it says, What attributes are on this class? So it gets a list of like instance variables and operations. What constraints are there? on this operation and then it gets the rules and then it can generate the code from there. So it is effectively it's not doing it in a visual format, but it's effectively reading the diagram.

[01:04:14] Leo Dion: Right, right. Yeah, I like that a lot. so this is a macOS app, SwiftUI. you wrote your own MCP server, right?

[01:04:24] Graham Lee: so like that is not actually that complex. Like MCP is just kind of JSON in, JSON out. yeah, and it's like they used to be in fact it's simpler than it used to be. The early version had like this handshake where the client says, Right, I I'm this version and I have these capabilities and the server said, I'm this version, I have these capabilities and then they agreed. That doesn't even happen anymore. So like there's there's two facilities I use. One is well yeah, what one is tools. So the client says list all the tools that available and the server responds with a list and there's a little description of what each one does and the LLM can read that and so it knows when to use each tool. And the other is prompts. So you can add like like agent skills give you the ability to kind of define custom reusable prompts for an LLM. An MCP server can contain a collection of prompts that are relevant to its activities. So yeah, there's like prompts for disclosing ambiguity in models for applying constraints to models based on business rules. So basically go you know, applying a bunch of these patterns that we've been talking about. There are built in prompts in Axminster so that your when you connect your like coding assistant to it, you can use those prompts from inside the chat box.

[01:05:45] Leo Dion: Okay. Yeah, that makes total sense. Did you learn anything during the development of this app that you were like, I didn't realize this is a certain AI pattern I've been using or anything about software development in this space?

[01:06:01] Graham Lee: I I learned a a lot. I have had because I'm an old school like Cocoa person who've like learned Objective-C and AppKit and everything, I'd kind of always gone, yeah, SwiftUI looks interesting. I already know how to solve this problem using these other tools, so I'm gonna go and do that. And on this thing I decided right, I am like going all in on SwiftUI, Swift 6, like you know, modern concurrency patterns, everything. so I had to learn how all of that works. particularly until Xcode 27, models integrating with Xcode didn't do a great job at doing idiomatic Swift. I don't know whether you found that. I certainly did. Like things like Swift Concurrency, they would just go, Hey, I've got a bunch of like async await errors, so I'm just gonna write @MainActor on everything and then hope that it all goes away. Yeah, so I had to learn

[01:06:53] Leo Dion: if it's even worse that it was using locks and stuff and it's like, Come on, seriously?

[01:06:58] Graham Lee: Yeah. had to learn how Swift Concurrency works and like make sure that I had a good model of the concurrency in my app, like so that I could express that to the LLM. yeah, I you know, I learned a ton about the technology. I learned a ton about App Store Review because like it it turns out that I have never been the admin on a App Store Connect account before. I've always been a developer and someone hasn't been the app so I haven't actually added an app to TestFlight or put an app into review before. yeah, we've all done anything we've done, we've all done for the first time, right? Like that that's nothing to be ashamed of.

[01:07:38] Leo Dion: Cool. so what's the status of Axminster right now?

[01:07:42] Graham Lee: so it went into review on Thursday. it's still awaiting review. so hopefully it'll be in the store by the time this this episode airs.

[01:07:53] Leo Dion: Is it closed source? What's the monetization?

[01:07:57] Graham Lee: Right. okay, so yeah, it is a a fixed upfront cost. I still haven't decided what that cost is gonna be. so I will work that out by the time we by the time it goes live. but yeah, like one time cost. the what was the other question?

[01:08:14] Leo Dion: think you answered it. It was just like yeah, it's all closed source I assume and

[01:08:18] Graham Lee: source. Right, yeah. So currently yes, I'm considering like open sourcing the part that represents the UML model itself, so the in memory model, so that other people can build r related tools. if you're you know, if your listeners are interested in that then yeah, they should get in contact and like let me know what they do with it and I'll see if there's interest in doing that.

[01:08:43] Leo Dion: Cool. anything else before we close out?

[01:08:48] Graham Lee: so yeah, I suppose like I just want to hit on the point that like this is somewhat related. The Axminster kind of visual approach is somewhat related to spec driven development. but I want to kind of close off the idea that we're going back to a world of waterfall development where we like, you know, spend ages writing project documentation and doing designs and things, and then only like after everyone in the company and all of the presidents and whatever have signed off that we actually get to write a line of code. that's not what I'm proposing at all. What I'm because the you know these generative tools are incredibly quick. What I'm proposing is having a little think before you write any software and thinking about what software you want to write and then encoding that thinking into like writing the software. so like, you know, I gave an example in my talk. where I did about an hour of planning before I write wrote some code. but I went through a bunch of alternative designs and the pros and cons of those and selected some and explored their like impacts and whatever in that hour using the LLM as a tool to enable that. And that's what I'm like that's the kind of workflow that Axminster is geared towards is the just like you know, spending an hour or so thinking about the problem you solve. So that you do a better solution.

[01:10:09] Leo Dion: So it's funny you mentioned that. This weekend, I've been putting together a PR for just a a spec document. And I decided I just figured I'd try it. Let's burn some tokens. I had it run. So I'm ta I'm working with Claude and then I'm having Claude review the PR. And it's interesting how like There's some things that I think were legitimate, but I almost felt I was getting to a point of analysis paralysis. Where it's like or over like one thing I I actually prompted in the last comment, the last PR, I'm like, we're getting a little too specific about edge cases, but also things that like will need to be a little bit flexible. Cause I've run into issues where I've created a spec document and it said, Well, you said you were gonna it was This is how it's supposed to be. And I'm like, Well, yeah, but that's before I knew about X, Y, and Z. It's like there needs to be a little bit of flexibility in these documents where it's like, No, no, no, no, no. this is this doesn't have to be in the document because it's really not that strict of a rule, so to speak.

[01:11:16] Graham Lee: Yeah, absolutely. So one of the patterns I have in the book is Defer the Details, which is just about saying, Yeah, we don't need to answer that. don't worry about it. Or like let's see let's see whether people complain when when we put that in TestFlight.

[01:11:29] Leo Dion: Yeah yeah yeah. Yep. Yeah, exactly. Let's f let's experiment and see what people say. Graham, thank you so much for coming on. This was awesome.

[01:11:38] Graham Lee: Absolute pleasure. It's I really enjoyed the conversation. Thank you.

[01:11:41] Leo Dion: Yeah, we'll put links to your website and blog posts, etc. Hopefully we'll have an App Store link in the show notes as well. anything else you want to plug before we close out?

[01:11:56] Graham Lee: nope, that sounds good. thanks again for the conversation.

[01:11:59] Leo Dion: Yeah. if people have any questions or follow-up, let us know. you can find me I'm at BrightDigit.com, leogdion on the socials. if you have it, definitely you should think about subscribing to the newsletter as well as the YouTube channel and like it and subscribe the podcast as well. thank you so much and I look forward to talking to you later. Bye everybody.