Redesign site and optimize page performance
All checks were successful
Deploy Website / build-and-deploy (push) Successful in 23s

This commit is contained in:
Akshay Kolli
2026-07-18 19:57:44 -04:00
parent 2f2794f5d5
commit 9dda076b56
53 changed files with 747 additions and 277 deletions

View File

@@ -68,7 +68,7 @@ So the practical distinction is not "does this GPU support FP4 at all?" It does.
I still wanted to measure the card instead of just reading target tables. So I pulled CUTLASS and ran its NVFP4 matrix multiplication example on the RTX 5090.
{{< figure src="/images/1_blackwell_dc_vs_gf/5090_65536_cropped.png" alt="A screenshot of a CUTLASS NVFP4 matrix multiplication benchmark on an RTX 5090" />}}
{{< figure src="/images/1_blackwell_dc_vs_gf/5090_65536_cropped.png" width="236" height="77" alt="A screenshot of a CUTLASS NVFP4 matrix multiplication benchmark on an RTX 5090" />}}
That is over a petaflop of NVFP4 compute.
@@ -76,7 +76,7 @@ A petaflop means one quadrillion floating-point operations per second. For a des
But the next question is whether the Tensor Cores are being fed efficiently. Nsight Compute gives the more interesting picture.
{{< figure src="/images/1_blackwell_dc_vs_gf/geforce_ncu.png" alt="Nsight Compute showing register pressure and memory bottlenecks on a GeForce GPU" />}}
{{< figure src="/images/1_blackwell_dc_vs_gf/geforce_ncu.png" width="1974" height="807" alt="Nsight Compute showing register pressure and memory bottlenecks on a GeForce GPU" />}}
The short read is: memory is the bottleneck. The Tensor Cores can chew through math faster than the rest of the kernel can keep them supplied. Shared memory pressure shows up immediately.
@@ -86,9 +86,9 @@ That is exactly the problem the datacenter path is designed to address.
To compare against datacenter Blackwell, I rented a B200 instance on Vast.ai and ran the same kind of matrix multiplication with CUTLASS kernels targeting `sm_100a`.
{{< figure src="/images/1_blackwell_dc_vs_gf/nvtop_b200.png" alt="nvtop showing B200 GPU memory capacity" />}}
{{< figure src="/images/1_blackwell_dc_vs_gf/nvtop_b200.png" width="647" height="106" alt="nvtop showing B200 GPU memory capacity" />}}
{{< figure src="/images/1_blackwell_dc_vs_gf/b200_65536_cropped.png" alt="A CUTLASS benchmark result from a B200 GPU" />}}
{{< figure src="/images/1_blackwell_dc_vs_gf/b200_65536_cropped.png" width="300" height="108" alt="A CUTLASS benchmark result from a B200 GPU" />}}
That run gets past 2 petaflops, and I suspect better kernels can push it further. The point is not just the number. The point is that the B200 has access to the datacenter Blackwell path that the 5090 does not.

View File

@@ -31,7 +31,7 @@ That is awkward for a large GPU. A 5090 has a lot of compute, but a single decod
So raw GPU size does not always turn into efficient local inference.
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig5_autoregressive_decode.svg" alt="Autoregressive decoding diagram showing one model pass producing one accepted token at a time" />}}
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig5_autoregressive_decode.svg" width="1200" height="520" alt="Autoregressive decoding diagram showing one model pass producing one accepted token at a time" />}}
Speculative decoding tries to fix this shape.
@@ -49,7 +49,7 @@ That is why the Qwen3.6 MTP model caught my attention. `Qwen3.6-27B` is a dense
Large enough to be useful. Small enough to run locally.
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig6_mtp_decode.svg" alt="MTP speculative decoding diagram showing a lookahead head proposing several draft tokens and the target model verifying them" />}}
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig6_mtp_decode.svg" width="1200" height="560" alt="MTP speculative decoding diagram showing a lookahead head proposing several draft tokens and the target model verifying them" />}}
## The 5090 settings
@@ -84,7 +84,7 @@ The same power settings without MTP gave me **32.1 tok/s**.
So MTP was not a small change. It moved the setup from usable to comfortable.
{{< figure src="/images/2_qwen36_tokens_per_kwh/author_nvidia_mtp_x6_215w_report.png" alt="Author measurement screenshot showing Qwen3.6-27B MTP throughput and a 215 watt GPU power reading" />}}
{{< figure src="/images/2_qwen36_tokens_per_kwh/author_nvidia_mtp_x6_215w_report.png" width="1656" height="807" alt="Author measurement screenshot showing Qwen3.6-27B MTP throughput and a 215 watt GPU power reading" />}}
## The math
@@ -170,7 +170,7 @@ That is still fine for me. I would leave that running.
Could I push the GPU lower? Yes. I can get the 5090 closer to **150 W**. I need a proper sweep before I say where the best point is. For now, 215 W is the point I like. It is fast enough, and it does not feel wasteful.
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig1_tokens_per_day.png" alt="Bar chart comparing generated tokens per day across the measured NVIDIA run and M3 Ultra benchmark rows" />}}
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig1_tokens_per_day.png" width="1800" height="990" alt="Bar chart comparing generated tokens per day across the measured NVIDIA run and M3 Ultra benchmark rows" />}}
## The Mac comparison
@@ -190,7 +190,7 @@ Those are good numbers for agent work. The missing number is power.
Against my GPU only 215 W point, the M3 Ultra 80 core oQ4 MTP rows need to run around **95 to 104 W at the wall** to match the 5090 on generated tokens per kWh.
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig3_m3_break_even_power.png" alt="Bar chart showing the M3 Ultra wall-power levels needed to match the measured NVIDIA GPU-side tokens per kWh" />}}
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig3_m3_break_even_power.png" width="1800" height="990" alt="Bar chart showing the M3 Ultra wall-power levels needed to match the measured NVIDIA GPU-side tokens per kWh" />}}
Against my rough 275 W whole box estimate, the break even numbers move up.
@@ -211,7 +211,7 @@ That is why I do not want to turn this into a clean win for either side. The Mac
The real comparison needs outlet power during the same workload.
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig4_m3_power_sensitivity.png" alt="Line chart showing M3 Ultra tokens per kWh across different assumed wall-power levels" />}}
{{< figure src="/images/2_qwen36_tokens_per_kwh/fig4_m3_power_sensitivity.png" width="1800" height="990" alt="Line chart showing M3 Ultra tokens per kWh across different assumed wall-power levels" />}}
## What changed for me

View File

@@ -19,7 +19,7 @@ math: false
ClipBored is a small native macOS clipboard manager. It captures local clipboard history and opens a keyboard-first responsive bottom panel for search, sorting, copy, paste, pinning, deletion, and organization.
{{< figure src="/images/projects/clipbored/panel.png" alt="ClipBored clipboard panel snapshot." />}}
{{< figure src="/images/projects/clipbored/panel.png" width="3024" height="860" alt="ClipBored clipboard panel snapshot." />}}
## What It Does

View File

@@ -15,7 +15,7 @@ math: false
FeedMe is a small native SwiftUI feed reader for macOS and iOS. It is local-first, account-free, and built around a shared Swift package for feed parsing, OPML import/export, refresh orchestration, and SQLite persistence.
{{< figure src="/images/projects/feedme/demo.png" alt="FeedMe running with a local demo library on iOS." />}}
{{< figure src="/images/projects/feedme/demo.png" width="644" height="1400" alt="FeedMe running with a local demo library on iOS." />}}
## What It Does

View File

@@ -21,7 +21,7 @@ math: false
I Hate PDFs is a small native macOS PDF reader for local reading, highlighting, commenting, and review. It uses SwiftUI, AppKit, and PDFKit, keeps documents on your Mac, and avoids accounts, tracking, and cloud upload.
{{< figure src="/images/projects/ihatepdfs/default-reading.png" alt="I Hate PDFs default reading mode." />}}
{{< figure src="/images/projects/ihatepdfs/default-reading.png" width="2024" height="1568" alt="I Hate PDFs default reading mode." />}}
## What It Does

View File

@@ -17,7 +17,7 @@ math: false
Inspire Me is a small browser app that displays a changing stream of words over a soft animated atmosphere. It is built as a static web app with JavaScript, Canvas, CSS typography controls, and a local word corpus.
{{< figure src="/images/projects/inspire-me/preview.png" alt="Inspire Me showing a large word over a soft animated paper-like background." />}}
{{< figure src="/images/projects/inspire-me/preview.png" width="1440" height="900" alt="Inspire Me showing a large word over a soft animated paper-like background." />}}
## What It Does