Qwen3.8 released!

[960design]

All I want for Christmas is a Studio M5 with 1.5TB of unified memory!

 

[startergo]

Yep prepare 50K for Christmas.

 

[BeatCrazy]

When can I expect a GGUF version that will will work well with my M3 Ultra 256GB?

 

[960design]

When can I expect a GGUF version that will will work well with my M3 Ultra 256GB?

Today! You will not be running a 2.4T parameter model on that rig, but you can certainly run the smaller qwen3.8-27B. https://huggingface.co/Qwen/Qwen3.8-27B

Unsloth has the GGUF version you requested, but I would strongly recommend looking for the MLX version to take full advantage of the M-series processors.
Intelligence Rating: 38! @ https://artificialanalysis.ai/models/qwen3-6-27b

Good luck!

 

[BeatCrazy]

Today! You will not be running a 2.4T parameter model on that rig, but you can certainly run the smaller qwen3.8-27B. https://huggingface.co/Qwen/Qwen3.8-27B

Unsloth has the GGUF version you requested, but I would strongly recommend looking for the MLX version to take full advantage of the M-series processors.
Intelligence Rating: 38! @ https://artificialanalysis.ai/models/qwen3-6-27b

Good luck!

Thanks. I use LM Studio as a front-end so I need a GGUF version.

 

[960design]

Thanks. I use LM Studio as a front-end so I need a GGUF version.

I have used LM Studio and had no issues with MLX models. ( I mostly use hermes now ).

 

[BeatCrazy]

I have used LM Studio and had no issues with MLX models. ( I mostly use hermes now ).View attachment 2652924

Ya, I'm dumb 🙂

I actually run that exact same model (MLX version) in LM Studio.

I also got the GGUF version of Qwen3.8 27B dialed in pretty well last night. This looks like it could be my go-to model for coding/troubleshooting/reasoning.

 

[citivolus]

What’s your config and how much memory are you guys using with 2.8? I was pushing the limit with 2.6 on my M4 Max with 36GB.

 

[BeatCrazy]

What’s your config and how much memory are you guys using with 2.8? I was pushing the limit with 2.6 on my M4 Max with 36GB.

I'm fortunate enough to have M3 Ultra w/256GB.

Qwen3.8 27b with other 'regular' stuff on my Mac, using ~100GB of memory.

 

[960design]

What’s your config and how much memory are you guys using with 2.8? I was pushing the limit with 2.6 on my M4 Max with 36GB.

Macbook M5 Max 128GB, I am running q8 (29.5GB): https://huggingface.co/mlx-community/Qwen3.8-27B-8bit