Weeknote 29
A bumper two weeknote. Which LLM and How do you decide, Open Recommendations, Vibe Coding tips, Mapping my Swims, interesting things.
I almost managed to get most of this out last week but then fell at the last hurdle. So it's a double weekenote!
What I did
- The first Monday was super busy as I bounced from one thing to another. It started with a chat with Stu Pearson which was lovely. I then raced out the door to meet a tiny charity in Sunderland to help with data/tech things. I got home just in time for a meeting with some folks from the Pro Bono Association about a prototype referral tool I'd previously built. Then had a catch up with Liz about OR and finally had an interview for some work we'd pitched for.
That's a lot for one day, or at least it is for me, lots to try and take in. I put so much into every meeting, conversation, workshop, really actively listening, considering, processing, that it is incredibly draining, and so that many things in one day I find hard. I know that some would say that is their day everyday, bouncing from one meeting to another. I'm definitely of the opinion that fewer, better meetings is the way. - After that I then spent a good chunk of this week on some deep work. I wrote the first pass of The Collective Resilience Guide and workbook for the OR Programme. That was really good to get into, really thinking through the overarching concepts, thinking about tools that could be useful and how they fit together. I made a lot of progress. I find things like this require me to think for a good while, and then big chunks of time immersed in it.
- I also spent a lot of time on some other deep work, which was related to this post Which LLM? And how do YOU decide? . It's something I've been thinking about and considering for a while, especially when building products, and decided to draft a framework beyond the standard bench-marking. I think there are many things to consider when building with LLM's and I feel there are perhaps even more to consider when building things within services related to the social purpose or public sector, and I've yet to see anything that begins to really explore it from that lens, so I wrote one. I've only had a first pass at it, but hopefully will have another run through next week before sharing.
- I had not one, but two conversations about Open Recommendations this week, which was nice. Firstly with Ed and then with Alex. It was nice to chat with Alex as it's the first time we've spoken and I'm a big fan of his and MySociety's work. (Also a big fan of Ed, but we get to talk more 😄) - I'm hoping that maybe there is a chance to work with one or both on Open Recs at some point in the future. Alex also encouraged me to finish my blog on the things I've considered and learned while building it (which links to the LLM evaluation framework above), as, per Alex "you'd have at least one avid reader" which is more than most of my blogs I imagine!
- Caught up with Tris about all things Open. Plan to put some time and effort into some action led community stuff next year
- Was meant to join a workshop being ran by Tom F in Sheffield exploring place based insight - what does governance, technology, ownership look like for this. Couldn't make it in the end due to weather. Will be interested to hear what came out of it.
- Finally met Wayne Murray in almost 3d (it was online). I'm a big fan of Wayne's so I enjoyed chatting about all things collaboration. Hope to get to work together next year on something. Wayne was as chilled and lovely as I expected.
Everyone wants to measure things, but wow can you actually measure transformation?
- Caught up with Alice which was also lovely. We talked about 'signals of transformation or change'. Really interesting conversation for me as I'm developing the Signals tool. We explore the question above about measuring transformation. Things like transformation (or systems change, or impact), where there isn't really a KPI for it is really hard. Our tools and systems aren't really built for this, which is where I'm hoping signals comes in, as we focus on surfacing signals of change. The chat with Alice helped reinforce how useful the tool could be and some ideas for development. We also talked a bit of running obvs.
- I ran a webinar on Vibe Coding. Write post about the top tips I shared during the session, which are applicable across vibe coding, but also more generally.
Tom Watson
- Ended the week with some disappointing news about a piece of work we had responded to. The feedback was we were a very very close second, but lost out on price. That's happened a few times this year. I don't think our day rate is very high or anything, but I am beginning to wonder if larger agencies can absorb some things through having more junior staff or just promise more that we do. It's a shame as the work was really interesting, but such is life.
None work stuff
Spent the weekend in Scotland, in the woods and the Lochs, eating and relaxing. Got my November dip in, which means I've only got December to complete on my year long challenge of a dip a month in a new location. This one was cold, and stunning.
To celebrate getting nearly to the end of the year, I decided to map my dips! So I started putting them into my own tool - Map my Patch. In doing so I noticed some things I could improve in the app so spent a few hours adding some useful features like adding the mapbox search plugin to allow users to search on the map for things. Which is handy...but not that handy when most of the things you are searching for are bodies of water! So I added the OS Names api so you can search for lots of other things. I might even add the OS vector tile set on a later update so users can switch to an OS map for geographic features. Also added some sharing features and a few other bits like adding options for storing images and documents in the surveys - solve your own problems as they say! Anyway, here are my dips for 2025 (just December to go)
Data For ActionToms Dips 2025
Interesting things
The disciplines theory of government Are legacy disciplines dragging the whole operation down? - good read this from James
TOOLBOXTOOLBOX - A curated list of the best business, design, and organisational change toolboxes built by some of the most influential companies, institutions and thinkers. - this is good - thanks to Doug for pointing this one out
What happens when funders invest in collaboration?
Introducing Semantic Model Sync in Hex - Governed metrics for trusted self-serve analytics
Referenced publications on systemic investing, measurement and learning - by Alan Hudon
tools.simonwillison - Miscellaneous HTML+JavaScript tools built mostly with the help of LLMs.
Schematic Interoperability Lives in the Architecture, Not the Models
Consequences Practical Tools for Understanding the Impacts of AI by Careful Industries - 🤩🤩👌
Making sense of emergence: Measuring impact to inform systemic investing (Part 1) - second entry for Alan Hudson this week!
Introducing a new AI metric to drive sustainability - this is very good. It proposes a metric for understanding the environmental impact of LLMs in the real world - mWh per 100 tokens. As they say in their paper, this is only one part of the puzzle, but it's nice for them to openly 'lift the lid' as they say. Some will tell you AI is destroying the world, some will say it's negligible or less harmful that a human typing. The answer, like most things, is somewhere inbetween, and unless we can open conversations about this, we will never get anywhere. Hmm, maybe I should fire up the linkblog on this one.