Our colleague taught Qwen3.8-27B to write Forge apps that pass Atlassian's validator, and the weight

[#1]

Disclosure first: I work at LeanZero, we're a Marketplace Partner, and the model below is ours. My colleague Mihai Perdum built it, trained it and published the weights, and he's the one who measured every number here on tests we can rerun. I'm posting it because I think it's the most interesting thing we've done and it belongs in front of this community. Read the numbers as a builder's numbers and check them yourself.

What it is

Qwen3.8-27B is a strong open model that knows almost nothing about Atlassian Forge. Ask it for a Forge app and it invents a manifest format and imports packages that do not exist. Mihai taught it Forge, Jira, Confluence and Jira Service Management from real apps, the documentation, the OpenAPI specs and answers we gave on these forums, then published the result as open weights under Apache-2.0. It is the same model in the same file format, so anything that loads an mlx-community model loads it. It runs fully local on Apple Silicon, and training the 27B took 26 hours on one Mac Studio.

Models and adapter are all on Hugging Face under the account Mihai-LeanZero (search that name there): Q8-mlx (8-bit, about 31 GB, the one to use), Q6-mlx (about 21 GB), Q4-mlx (about 15 GB), the LoRA adapter alone (rank 32, 190 MB) and the untouched quantised base we trained from. All three builds ship the multi-token-prediction head as mtp.safetensors; goose local edition and Rapid-MLX use it for speculative decoding, while mlx-lm and LM Studio ignore it and load the model normally.

Measured, base against tuned

Same harness, same prompts, same machine. The base is the untouched quantised Qwen3.8-27B; tuned is what's on Hugging Face.

• Atlassian identifier probe, facts from before April 2026, thinking on: 15% to 69%.
• Same probe on facts after April 2026: 23% to 23%. No gain, because the training data stops there.
• Complete Forge app from a one-line brief whose manifest passes Atlassian's validator: 0 of 25 to 14 of 25.
• The whole app also compiling under tsc against the real @Forge/* types: 0 of 25 to 12 of 25.
• Answers still running at the 2048-token cap, across four sampling legs: 60/48/53/25% down to 25/15/28/13%.
• Genuine loops over 40 prompts times 4 legs: 2/1/3/4 down to 0/0/1/0.
• Needle recall at 4k, 32k and 128k context: 100% before and after.
• Speculative decoding draft acceptance: 53% before, 51% after, so the head survived the tuning.

What is still weak

A complete app from a one-line brief can still need one fix before forge deploy. With thinking off the usual one is a TypeScript typing error against @Forge/react or @Forge/bridge; the next is a resolver or resources key placed under modules, or a handler path that does not match the file. The post-April facts did not move at all. And the 4-bit build's retention was measured only by agreement with the 8-bit model and the identifier probe; the app-writing tasks were not rerun on it. This is a strong assistant, not a compiler. Keep Atlassian's manifest validator and tsc in the loop, which is exactly what our harness does.

How to run it

1. Download the Q8 folder from Hugging Face (Q6 or Q4 if you have less memory).
2. goose local edition: mount the folder in the MLX Engine view; the engine supervisor finds mtp.safetensors and turns speculative decoding on. Measured on the merged model, single stream: 1.26x faster decode on short context, 1.21x at 32k.
3. LM Studio: drop the folder into your models directory; it loads with the MLX engine like any mlx-community model.
4. mlx-lm: mlx_lm.generate --model <folder>, or load the adapter alone with --adapter-path after downloading it to a local folder, since mlx-lm resolves adapters from disk only.
5. Use thinking on for questions and thinking off for code generation. We measured the difference and it is large.

Why I'm posting it here

Because a 27B that actually knows Forge, running on a Mac with nothing leaving the machine, is the kind of thing this community should have access to, and open weights are how that happens. This is a premiere, the models went up on Hugging Face today. If you run the probes on your own setup and get different numbers, tell us, and if there's a test you'd want measured next, I'll pass it straight to Mihai.

[Aristóteles Quintanar]

Looks like a great idea.  I'll try to run it on any place ... Now I just need to search for some place where it could runs hahaha.  Seriously; nice initiative!

Thanks for that!

Aris

[Gabriela - LeanZero]

Hey Aris, thank you, that made my morning :) On where to run it, you'll need a Mac with Apple Silicon, because these are MLX builds. The 4-bit one is about 15 GB on disk, so a 24 GB machine runs it, the 8-bit wants roughly 32 GB free, and if you've got a Mac like that, LM Studio is the least fuss, you drop the folder in and it loads. If you don't have one, tell me what you do have and I'll ask Mihai whether the adapter can be applied to a build for it.