Contact
Cum să cumpăr
Livrare
Consilier de cumpărături
Inside LLM Inference
From Silicon to Served Token - A Measured Guide to How Language Models Run on GPUs
engleză
57 b
Curând
Nou
Până la 30 de zile pentru returnare
How does a language model actually become served tokens on a GPU - and why is it so much slower than the hardware's headline TFLOPS promise?
Inside LLM Inference follows a single request through the entire stack: from a PyTorch call down through the CUDA runtime, the streaming multiprocessor, the warp, and the memory system, to the arithmetic cores that finally do the work - and back up to the throughput and cost you pay for.
Every claim is measured, not asserted. A production model runs on 2026's newest silicon - NVIDIA's Blackwell generation - across four inference engines (vLLM, SGLang, TensorRT-LLM, and Hugging Face Transformers), profiled to individual GPU kernels with hardware counters and cross-checked against the physics that predicts them.
You start from zero - Chapter 0 defines every term - and finish able to read a live server:
- classify a kernel as memory- or compute-bound, and prove it with a clock-sensitivity test
- size a KV-cache pool and the batch it admits from a model's own dimensions
- read sm_active and sm_occupancy off a running GPU and know what they mean
- choose an inference engine on evidence, not vibes
The closing chapters reach the 2026 state of the art: speculative decoding and self-drafting (EAGLE-3, multi-token prediction), the compressed KV cache of multi-head latent attention, prefill/decode disaggregation, mixture-of-experts routing, and the Blackwell-era FlashAttention-4 kernel.
One thesis runs throughout: serving language models is fundamentally a memory-bandwidth problem, and batch size is the price of admission to compute you have already paid for.
Whether you deploy models, optimize inference, or simply want to understand what your GPU is really doing between tokens, this book turns the black box into something you can measure, predict, and tune.
Informații despre carte
engleză
Dăruiește această carte chiar astăzi
Este foarte ușor
1 Adaugă cartea în coș și selectează Livrează ca un cadou 2 Îți vom trimite un voucher în schimb 3 Cartea va ajunge direct la adresa destinataruluiBună ziua! Sunt Libroamiko, consilierul dumneavoastră de cărți.
Cu ce vă pot ajuta?