Category: LM Studio

Creating Documentation with AI


AI is great at producing documentation. Point it at a folder of text and images and get it to make sense of the contents and produce documentation, or better yet, as a Developer, point it at a folder of source code and get it to document it. You may even learn something about your own software that you had forgotten 🙂 It may even point out flaws or unexpected side effects, according to a friend, this has never happened to me … moving swiftly on …


I’ve developed a very simple prompt that I use which gives me great results and I usually use Claude Code with the Sonnet 4.6 model. However as local AI is improving rapidly I thought it would be fun and interesting to compare the output from Claude Code with that of a couple of local LLMs. It may currently only take a dollar or two to use Claude Code for this task but given that prompt costs are currently heavily subsidised, and even in 2026 it’s sometimes possible to be without a stable internet connection, it makes sense to try alternatives.


When using local LLMs for a job like this I am currently testing Cline (https://cline.bot/). This gives a Claude Code type experience within VS Code. A terminal version of Cline is also available but I had some TUI effects that I’ve not yet bothered to get to the bottom of. The VS Code installation however was flawless.


The Prompt

The prompt I used was this:

Write a professional and in depth combined user and developer documentation for this application. Output as a self contained html file named: appname_Documentation saved to the project/Documentation directory. Use graphs and charts if possible to make the document as visual as possible. Avoid dark backgrounds and if text is on a dark background make the contrast significant for easy readability. 


The Project

The project I was using to test the documentation creation is a simple Python application; 

PyTrain1 is a lightweight, command-line Python application that automates a three-stage data extraction pipeline. It connects to a Microsoft SQL Server database, runs a SELECT query against a customer table, and exports the results to a CSV file on the local machine, all in a single run with no user interaction required.

The application serves as a training exercise in combining SQLAlchemy, pyodbc, and pandas to extract and export relational data with minimal code. It runs as a single script with no web server, background process, or GUI component.

And yes, AI did write the above description, I couldn’t have done it better myself.


The Computer


2025 Mac Studio Max with 64 Gb RAM, 16 CPU cores and 40 GPU cores.


First Test: Sonnet 4.6


Second Test: Qwen3.6-35b-a3b-mlx


The image below shows the LLM loaded into LM Studio.


The image below is Cline running against Qwen in VS Code.


In this image, below, you can see the activity on my computer, with the memory usage and GPU usage shown plainly. Note I was also running a Windows 11 VM at the time.


Third Test: Gemma 4 3b


Fourth Test: Llama 3.3 70b Instruct


The Results


The results shown above don’t tell the whole story. The instant take away (as expected) is that Claude produced the highest quality documentation. What I didn’t expect is how good the output from Qwen was, and also how fast it was compared to Claude.


I expected better from Gemma. Gemma named the output file incorrectly, gave great feedback during the process, finished quickly but gave only a few lines of output. So, a failure. I will show some sample output later in this document, at least for those that created output … this brings me to Llama. 


Llama was not happy. I left it running for 15 minutes or so during which time it had numerous issues and errors and produced some strange commentary including telling me the temperature and humidity in San Francisco! I’m in the UK. I don’t know whether it had an issue with Cline but a big difference is that unlike the other local LLMs it is only a 4 bit quantisation so I expected it to perform slightly worse despite it’s higher parameter count. However in the end it was a complete failure at this task.


Sample Output from Sonnet


Sample Output from Qwen


Total Output from Gemma


Summary and Conclusion


Claude Code with Sonnet produces the highest quality documentation of all the models I tested. There is a price and time penalty however. The right local LLM that costs nothing to run and performs quickly is capable of creating accurate and useful documentation with no fuss and drama, and it will run on your (reasonably specced) laptop or desktop.


Local LLMs were considered a joke for real work not so long ago. I’m convinced the future is hybrid, local LLMs for most tasks with cloud models for the really tough stuff. If you’re interested run your own tests and see for yourself. 


Follow up Article


I have written a cross platform chat style application that allows me to switch between cloud and local models and see the difference between their outputs for the type of questions users often type in to ChatGPT or Claude. I can also experiment with pre-loading the prompt with a Speciality or a Personality as well as integrate a Prompt Library and pre-defined Information Layers. I’ll cover the results of these experiments in a future article. You can see the application in the image below.

Local AI: With LM Studio

Introduction

What is a local LLM? When people talk about AI they are often referring to the chatbots such as Claude, ChatGPT or Gemini. These chatbots are user interfaces for one or more Large Language Models. They are huge, advanced, highly trained AI models that sit in the cloud and serve thousands or more people at a time.

A local LLM is a smaller model that generally sits on your own computer or within your local network. These models are significantly smaller and therefore slower and less capable than their larger cousins. They do however have a few advantages that the larger models lack:

  • They are private and secure. They don’t generally send or receive information via the Internet.
  • They cost nothing to run, except for the electricity to power them.
  • They are yours and can be configured any way you want.

There are several different ways to run these local LLMs and several hardware platforms you can use. Software to run local LLMs include Ollama, vLLM and LM Studio. The hardware usually consists of a fast PC running Linux or Windows, a powerful GPU with at least 16GB of VRAM, or a Mac Studio. My preferred platform, and the one I bought for just this purpose is an M4 Max Mac Studio with 64GB of RAM. The fact that it uses very little electricity, is virtually silent, and doesn’t heat my office goes some way towards justifying the cost!

To be honest, go with your favourite platform, or what you already have. It is even possible to run smaller models on a laptop with some success. Keep your expectations realistic and you will have fun, learn a lot and maybe find that you don’t need to rely on subscriptions to the cloud models, or at least not as often.


Above you can see a high-spec PC configured for AI workloads, next to an Apple Mac Studio. Both great machines for what we need.

Getting Started

Once you have identified the hardware that you wish to use for your local LLM and confirmed you have enough space for the models you want to try (local LLM sizes vary from a few Mb to 100Gb+) you need to look at the software you are going to use. 

I started with Ollama and recently moved to LM Studio. I’ve not personally tried vLLM. I found LM Studio to be a good choice and for the time being I intend to stick with it. Screenshots and examples in this (and probably future articles) will be based on LM Studio.

All of the software choices are available to download for free and are all under active development. LM Studio can be downloaded from here:

https://lmstudio.ai/download

Select your operating system and download.

After a simple installation, you can run LM Studio and one of the first things you need to do is download one (or more) models. There are hundreds of different models available and you could start with those that you recognise either the model or the creator by name, or just pick one. This is the model selection window:

There is a lot of terminology relating to the file names of some of these models, often they include the manufacturer, the amount of parameters, the quantisation level etc. No need to worry about any of these for now, but going forward it would pay you to learn what these terms mean and how they could affect your experience.

The most important thing is the size of the model and whether it will operate comfortably within the amount of RAM (usually VRAM on Windows / Linux PCs) you have available. If the selected model will fit comfortably then you will be given the opportunity to download, if not LM Studio will warn you first, and you have the opportunity to download a different model.

The way RAM and VRAM is used differs significantly between PCs and Macs. Basically the Mac uses unified memory, meaning the total RAM is split by the operating system between GPU and CPU. On a PC you will have two values RAM and VRAM, usually you will have an adapter card containing your GPU and it will have a specific amount of RAM, your computer itself will have a separate amount of RAM. This is an important distinction at the moment but hopefully will become less relevant over time as some PC manufacturers are starting to introduce unified memory, or at least making hardware changes that give the appearance and functionality of unified memory.

Note: The GPU is the Graphics Processing Unit and the CPU is the computer’s Central Processing Unit. LLM calculations generally take place on the GPU because it is faster at performing the kind of numerical calculations required by the LLM.

Your First Prompt

Once you have downloaded a model, you need to select it before you can run a prompt against it.

Once you have a model selected, you can type a prompt in the same way that you would using ChatGPT or Claude.

Note that LM Studio is still in development and I have seen issues where you sometimes have to select the model twice in order for LM Studio to realise it’s loaded.

Below you can see two example chats that I tested against this model.


You have much more control over your models in LM Studio than you would have with most commercial models, so feel free to experiment. You can change parameters like temperature, context length, system prompts, and choice of model weights.

Summary

There are many additional things that you can do with your own hosted LLMs, including accessing them across your network or even running these large models across the internet using an integrated Tailscale VPN. In addition, this allows you to run larger models from a computer with much less VRAM, as the compute is carried out on your main computer, where the larger models reside. This technology is quite recent and is called LM Link.

There is also a fascinating technology called ‘Speculative Decoding’ which allows you to pair a smaller LLM (the Draft model) with a larger one. In many circumstances this can significantly increase performance.