Convert an open-weight LLM. GGUF and MLX, step by step.
Turn downloadable model weights into files for local use. Follow our Azure and Apple Silicon conversion steps, with validation checks, reference code and FREE model downloads.
Read the full report for the conversion workflow, reference code and FREE model download links.
Know what each piece does
Weights are the learned numbers. A tokenizer turns text into numbered pieces. A runtime uses both to produce an answer.
GGUF and MLX
GGUF is a file format for llama.cpp-based runtimes. MLX is Apple’s framework; its quantized checkpoints use different packing rules.
BF16 and quantization
BF16 stores the original weights in a 16-bit format. Lower-bit recipes reduce storage while requiring their own quality checks.
Rent CUDA; run Metal on the Mac
The worked run used one full Azure A100 and an M4 Max Mac with 128 GiB unified memory.
Azure Virtual Machines
Standard_NC24ads_A100_v4: 24 virtual CPUs, 220 GiB host RAM and one A100. Persistent 1-TiB storage held the remote files.
Apple Silicon
Native BF16 supplied the MLX baseline. Each 8-, 6- and 4-bit affine model started independently from the official source.
The tokenizer must match
The pinned model needed a native Metaspace BPE patch in llama.cpp. Loading the tensors alone did not establish compatibility.
Preserve the source
22 pinned files, 14 weight shards, 471 tensors and all original parameters. Complete hashes identify the exact bytes.
Check the runtime
330 tokenizer cases and 1,980 comparisons had zero mismatches. The GGUF results require the pinned, patched runtime.
Start each GGUF conversion from BF16
The full BF16 GGUF is the weight source. An accepted CUDA importance matrix supplies the lower-bit calibration statistics.
BF16 and Q8_0
BF16 GGUF: 66.25 GB. Q8_0: 35.20 GB. Both preserve all logical tensors and the original metadata.
Q6_K and Q4_K_M
Q6_K: 27.18 GB. Q4_K_M: 20.06 GB. Each starts from BF16 with the accepted 79,872-token calibration.
Test more than a successful load
Structure, token IDs, parsed code and inert tool JSON, numerical comparisons, termination and full GPU weight placement each have a separate check.
Fixed correctness tests
Eight scored cases cover English, Chinese, code, tool JSON and reasoning termination. Two unscored adversarial cases record behavior.
Actual context prompts
Quantized GGUF and MLX passed 16K, 32K and 64K retrieval. BF16 passed 16K and 32K. The advertised 524,288-token (512K) context was not tested.
Build the Apple checkpoints independently
Affine quantization used group size 64. Native BF16, structural checks and independent parsing provided the MLX reference path.
Stored model sizes
8-bit: 35.21 GB. 6-bit: 26.93 GB. 4-bit: 18.65 GB. Smaller files do not eliminate context-cache memory.
Validation takes time
Conversion phases took about 13–17 seconds. Each 64K retrieval test took roughly 13 minutes on the stated Mac.
About $23–$32 for the operation
The estimate combines recorded compute, an operation-duration disk allocation and accepted delivery at the recorded retail rates.
Not an invoice or maximum bill. Retained disks, failed probes, taxes, disk transactions, staff time and optional services add cost.
Verify the bytes before shutdown
Hash the complete accepted remote files and local copies. Reload every delivered GGUF offline, then release the task-owned VM after its work is complete.
Local completion
The four GGUFs total 148.68 GB. Delivery includes the 27.17 MB calibration matrix. Complete transfer hashes and four offline reloads passed.
Azure completion
Verify PowerState/deallocated. A merely stopped VM may still bill compute; the retained persistent disk continues billing.
Convert, verify and deliver
Follow the source checks, tokenizer patch, GGUF and MLX conversions, runtime tests, verified transfer and Azure shutdown.
Included
The ZIP includes Python functions, calibration text, test fixtures and recorded results. No model weights or cloud credentials.
Measured limits
Quantized GGUF and MLX passed 16K, 32K and 64K retrieval. BF16 passed 16K and 32K. The publisher's 512K context is untested.
© 2026 Chander Dhall Methodworks, LLC. All rights reserved.
