This post walks through the whole path. Install, download a small model, serve it over HTTP, open the model file and read its guts, then pick a quantization on purpose instead of by habit. At the end you will find the arithmetic for working out whether a given model fits in your RAM before you download multi-gigabyte files.

Subscribe now

Running a language model on your own hardware costs nothing per token. It works on a plane or a train. It never sends your code to someone else’s machine and on Apple Silicon the setup takes about ten minutes, with llama.cpp doing all the heavy lifting.

Why Mac Minis are out of stock

If you’ve recently tried to buy a Mac Mini you might have noticed that even the most basic configurations are out of stock with at least a 4 week wait before you can get your hands in one. The reason for this is simple: LLMs.

When designing its own M family of Silicon chips, Apple made the unusual choice of having a unified memory pool between the CPU and the GPU so the entire memory pool is accessible to the GPU and there is no shuffle data around.

llama.cpp was introduced by Georgi Gerganov in 2023 as a way of simplifying model hosting and it was built from the onset to treat Apple Silicon as a first-class target. It uses Metal for GPU work and the Accelerate framework on the CPU side.

Other popular options such as Ollama and LM Studio build on top of llama.cpp and provide a (perhaps) friendlier user experience by hiding some of the flags that matter the most for our use case: context length, batch sizes, KV cache precision, sampling. Even if you prefer simpler approaches, understanding how the engine works pays off every time you need to do something beyond the basics.

The pipeline

The process we’ll follow to get up and running and pretty straightforward. We’ll build download the model GGUF file (the file containing all the weights) from HuggingFace, install llama.cpp and use it to setup a local service that will serve the model and that we can interact programmatically using an OpenAI client.

To better understand how these models work, we’ll also use gguf-dump to extract the metadata from the model files and llama-bench to measure the performance on our hardware. Here is a visual take on the whole pipeline.:

Installing llama.cpp

The first step is to install llama.cpp. The easy way is to just use homebrew:

brew install llama.cpp

But building it from source takes five minutes and produces a custom binary optimized for your exact machine:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release -j $(sysctl -n hw.ncpu)

After which, we should have llama-cli installed:

export PATH="$PWD/build/bin:$PATH"
llama-cli --version

Downloading a model

For the purposes of this post, we’ll use a relatively small model, a version of Gemma 3 with just 1B parameters. You can read all about it on the models HuggingFace page: ggml-org/gemma-3-1b-it-GGUF.

llama-cli can download the model directly from HuggingFace by using the -hf flag:

llama-cli -hf ggml-org/gemma-3-1b-it-GGUF

This will download 769MB into the hugging face cache directory and immediately start running the model:

Congratulations! You now have a running local model!

To make your life this easy, llama-cli had to make some choices for you. In particular, it decided where to download the files to (~/.cache/huggingface/hub/) and which quantization (Q4_K_M) of the model to use. We’ll discuss quantization later on, but for now you can think of it as being the size of the model.

You can have a lot more control over this process by using the HuggingFace CLI tool directly. In case you need to install it first, you can use:

curl -LsSf https://hf.co/cli/install.sh | bash

And then you’re ready to download exactly what you want and place it where you want it:

hf download ggml-org/gemma-3-1b-it-GGUF --include "*f16.gguf" --local-dir ~/models

This will download the F16 quantization of the model (about 2GB) and place the files in the directory you specified (the ~/models folder).

Now you can launch this new model version directly:

llama-cli \
  -m ~/models/gemma-3-1b-it-f16.gguf \
  -ngl 99 \
  -c 8192 \
  --jinja --color on 

Which brings us to the chat interface we encountered above.

Our choice of flags makes a difference:

• -ngl 99 pushes every layer onto the Metal GPU. The number 99 is a common shorthand for “all of them”.

• -c 8192 sets the context window size in tokens. This drives KV cache memory, covered later.

• --jinja applies the chat template stored inside the GGUF file. Skip it and your prompts get formatted wrong.

• --color on uses colors to separates your text from the model’s output.

Serving it

A simple chatting application on your command line, while useful, is still fairly limited. If you’re interested in building pipelines on top of your local models you’ll need to setup a local server that you can interact with programmatically just as you do with the OpenAI, Anthropic, Gemini, etc third party APIs.

Fortunately, this is a lot easier than it sounds, as llama-cpp ships with llama-server. All you have to do is:

llama-server \
  -m ~/models/gemma-3-1b-it-f16.gguf \
  -ngl 99 -fa on \
  -c 8192 -b 2048 -ub 2048 \
  --jinja \
  --host 127.0.0.1 --port 8080

to get a server running on port 8080 of your local machine. You can check that all is running as expected with a simple(-ish) API call:

curl -s http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "local",
    "messages": [{"role": "user", "content": "Explain KV caching in two sentences."}],
    "max_tokens": 200
  }' | jq -r '.choices[0].message.content'

Fortunately, llama-server follows the OpenAI API standard so any OpenAI client can natively speak with it as if it were the official OpenAI API:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="not-needed")

response = client.chat.completions.create(
    model="local",
    messages=[{"role": "user", "content": "Explain KV caching in two sentences."}],
)
print(response.choices[0].message.content)

What’s inside GGUF Files

A GGUF is a one stop shop for model information. It contains not only a weights blob but also a configuration, the tokenizer, the chat template, and the weights. That design choice is why llama-server -m model.gguf needs no other arguments.

The header carries some magic bytes, a format version (currently 3), a tensor count, and a metadata count. Files default to little-endian. Big-endian support arrived with version 3.

The metadata section is the interesting part. Keys are dotted lower-snake-case ASCII, capped at 65535 bytes. Values carry an explicit type tag covering the integer widths, floats, booleans, strings, and arrays. Older ggml formats used fixed-order untyped hyperparameters. The typed key-value store is what lets one format carry over a hundred model architectures without a version bump every time.

Tensor entries hold a name, a dimension count, the dimensions, a ggml type, and an offset into the data section. One trap for anyone parsing by hand: dimensions are stored in reverse order. A [4096, 11008] tensor shows up as [11008, 4096].

Tensor data sits at a 32-byte alignment boundary by default, overridable with the `general.alignment` key. Every tensor offset is a multiple of that alignment. The flat, aligned layout is what lets Metal read tensors straight out of a memory-mapped file with no extra copy. It is the reason a 5 GB model appears to load in under a second the second time you start the server.

All of this information can be easily extracted from the gguf file using the gguf Python package (installable with uv add gguf):

gguf-dump ~/models/gemma-3-1b-it-f16.gguf

to produce:

and you can use

gguf-dump --no-tensors ~/models/gemma-3-1b-it-f16.gguf

to skip all the verbose tensor information.

Naturally, all this information is also available programmatically from inside Python:

from gguf import GGUFReader

reader = GGUFReader("/Users/bgoncalves/models/gemma-3-1b-it-f16.gguf")

for key, field in reader.fields.items():
    if "tokens" in key or "scores" in key:   # skip the 32k-element arrays
        continue
    print(key, "=", field.contents())

for tensor in reader.tensors[:5]:
    print(f"{tensor.name:40s} {str(tensor.tensor_type):12s} {tensor.shape}")

You can edit metadata too. gguf-set-metadata writes a modified copy with keys added or removed. gguf-editor-gui opens a PySide6 window over the same data. This might come in handy when a third party re-uploaded quant ships a broken tokenizer.chat_template, and you need to patch it without re-downloading the whole file.

Quantization

We’ll dive into the intricacies of quantization in a future post, but in order to successfully choose the best local models for your application and hardware you need to grap the basic idea.

Large Language Models are massive memory hogs and the more parameters, the more memory you’ll need. State of the art models (like Kimi 3) have multi-trillion parameters, resulting in multiple terabytes requirements for VRAM. Quantization is a way of reducing the memory footprint of an LLM by lowering the numerical precision of its weights. Conceptually, it’s similar to how you might compress image files to reduce the file size.

By default, model weights are trained and stored in 16-bit Floating Point precision, known as FP16, making each parameter takes up 2 bytes of VRAM. At this precision, a 70-billion parameter model (like Llama-3 70B) requires over 140 GB of VRAM just to load into memory. In order to run it, you would need multiple (less than affordable) NVIDIA H100s, making it impossible for regular developers or consumers to run locally.

Quantization changes the math by converting those 16-bit floating-point numbers into 8-bit or 4-bit integers (INT8 or INT4), it drastically shrinks the model size. An INT4 version of that same 70B model only requires about 35–40 GB of VRAM, allowing it to run on consumer hardware or high-end MacBooks.

So what do all those letters after a model name mean? The first part of a GGUF/llama.cpp suffix always starts with a Q followed by a number which corresponds to the average number of bits used to store each weight. You might also see a _K_ followed by an S(mall), M(edium), or L(arge). The “K” indicates it uses K-quantization (block-wise mixed precision), where different layers of the neural network are quantized at different levels depending on how important they are to the model’s structure.

As the reduced size applies to every weight in the model, changing the quantization can have a huge impact on the memory requirements:

So, why not just use the smallest quantization possible? Because, just like your photos, the price you pay for compression is quality and Llama.cpp’s choice to go with Q4_K_M above is starting to make sense. It uses 4-bit per weight, with (K) smart, mixed-precision layer strategy and a medium profile. Essentially, a good default setting.

Memory Footprint

From the discussion above, we can start to work out exactly how much memory we’ll need to run our chosen model, Gemma 3 1B at full F16 precision. With F16, we need 2 bytes per weight at with 1B weights we it’s easy to see that we’ll need around 2GB of memory.

Unfortunately, that’s not the end of the memory story. We also need space for the Key-Value cache, a temporary memory storage technique that speeds up text generation by storing past key and value computations, preventing the model from recalculating previous words from scratch for every new token. The memory necessary for this depends directly on the context size so the larger the context, the larger the memory requirements.

Finally, you also need compute buffers to speed up some of the calculations. So, on an entry level 16GB Mac machine our 1B parameter model looks something like this:

Where it is clear that we are well within our limits, with room to spare, even without any quantization and on an entry level machine.

Now that you understand the basics of how to use llama.cpp you’re well on your way to explore the wonderful world of self hosting LLM models.

If you enjoyed this post, consider subscribing and sharing it with a colleague who is building agents, and let me know in the comments which primitive you’d want a deeper dive on next.

Subscribe now

In this post we continue exploring the finer points of Agentic Harnesses and build a full fledge Advanced Harness.

Subscribe now

The companion notebook is available on the LLMs for Data Science GitHub repository:

LLMs for Science GitHub

That Basic Harness loop is correct, but naive. A lone pilot in a well-built jet might win a dogfight, but nobody runs an air campaign that way. Real operations add mission planners who decide what sorties to fly before anyone takes off, squadrons that fly independent sorties in parallel, fuel budgets and bingo calls that force a return to base before the tanks run dry, flight recorders that make every mission reconstructible after the fact, and after-act…

Read more

Subscribe now

You can also find the companion notebook on the LLMs for Data Science GitHub repository:

You can also find the companion notebook on the LLMs for Data Science GitHub repository:

LLMs for Science GitHub

Every language model does one thing. It reads some words and guesses the next one. ChatGPT does it. Your phone keyboard does it. The model in this post does it too, and you can build this one by hand. It counts how often each three-word sequence is followed by a given word, then turns those counts into probabilities. That single idea, next-word prediction, sits at the center of every system in the field.

The build is short. We start with what an n-gram model is and where the idea came from. We load WikiText-103 and study it. We build the model, read off what it learned, and let it write. We close by set…

Read more

In this post, we’ll build a basic Harness from scratch to understand every moving part. By the end, you’ll have a working bare bones agent and a clear mental model of what agents like Claude Code are actually doing under the hood.

We’ll look at how to build a more sophisticated production-ready harness in a future post.

Subscribe now

You can also find the companion notebook on the LLMs for Data Science GitHub repository:

LLMs for Science GitHub

In the early 1950s, war raged over the Korean peninsula with American-made F-86 fighters being outmatched by the technically superior Russian-made MiG-15. Col John Boyd was the US Air Force fighter pilot in charge of trying to His main insight has proven useful ever since: You can make the combined US Pilot + F-86 system better than the Korean Pilot + MiG-15 system…

Read more

Each sentence is a triple in disguise: two entities and a verb tying them together. If we extract enough of them across enough articles, we get a knowledge graph: a structure we can query, visualize, and reason over.

This post walks through that pipeline end-to-end. We start from raw news text, run named entity recognition with spaCy, infer relationships, and analyze the resulting graph.

Subscribe now

As always, you can find the companion notebook on the Graphs for Data Science GitHub repository:

Graphs For Science GitHub

We use the AG News corpus, 120,000 articles across four categories (World, Sports, Business, Sci/Tech). For the sake of expediency, we sample 2,000 articles, balanced 500 per category. Loading is a single call to Hugging Face:

from datasets import load_dataset

dataset = load_dataset('ag_news', split='train')
df = dataset.to_pand…

Read more

Subscribe now

You can find the companion notebook on the Visualizarion for Data Science GitHub repository:

Visualization For Science GitHub

The Bureau of Transportation Statistics provides a wide range of datasets on US and global transportation infrastructure. We have used it previously here, here, and here.

The first step in our analysis is to extract the Top destinations for each state. We sort the outgoing destination of each state and extract the Top n:

Read more

Subscribe now

As always, you can find the companion notebook on the Graphs for Data Science GitHub repository:

Graphs For Data Science GitHub

In this post, we’re flipping our usual script, and instead of focusing on analyzing a graph, we’re going to learn from one. We’ll build a neural network to predict a molecule's solubility in water purely from the graph structure of the connections between atoms. The family of models that pulls this off goes by the name Graph Neural Networks (GNNs), and they’ve quietly become one of the most versatile tools in the modern ML toolkit.

For the sake of convenience (and a shiny new dataset), we’re using molecules as our playground. Still, the core ideas: message passing, neighborhood aggregation, and graph-level readout are applicable anywhere you encounter graph structure: social networks, road systems, knowledge…

Read more

1. Building AI Agents with LLMs, RAG, and Knowledge Graphs

Read more

If you would like a quick refresher before we get started, check out our previous posts in this series (especially the first 2):

1. Epidemic Modeling 101: Or why your CoVID19 exponential fits are wrong

2. All CoVID-19 models are wrong, but some are useful

3. Adding confidence intervals and stochastic effects to your CoVID-19 Models

4. Impact of Seasonal effects on CoVID-19

5. Competing CoVID-19 Strains

6. Network Structure, Super-spreaders and Contact Tracing

7. Network Models, the effect of degree correlations

8. How to model the effects of vaccination

9. The Impact of Age Structure on Epidemic Spreading

10. Meta-population Models

In all the models we have considered so far, we have kept the total population constant by making sure that all transitions between compartments balanced out. For e…

Read more

We hope your enjoy this post and don’t forget to

Subscribe now

so you never miss a post and

Share

to help us grow!

The US Census Bureau provides detailed shapefiles for various regions in the US. We download the shapefile for the 50 States. And load it with GeoPandas:

Read more

1. Natural Language Processing with Transformers

Read more

Features

• Homogeneously mixed Model: Easily define compartmental models by just specifying the transitions and rates

• Seasonality: Allow parameters to evolve in time

• Age Structure: Convert any compartmental model to an age-structured model by simply adding an age contact matrix and a the population of each age group

• Network Model: Easily define epidemic models running over an arbitrary network structure

• Meta-Population Models: Define metapopulation model connecting subpopulations through a travel matrix

Installation

You can install the latest version of Epidemik using pip:

pip install epidemik

Alternatively, you can download the source code from the GitHub repository and install it manually.

Documentation and Support

Checkout the documentation o…

Read more

Welcome to the 10th post of the Epidemic Modeling series. In this post, we go beyond single populations and start modeling the spread of a disease through multiple coupled populations. Our fundamental assumptions are that the disease structure is the same everywhere and that individuals can travel between any two populations with a given fixed probability.

If you would like a quick refresher before we get started, check out our previous posts in this series (especially the first 2):

1. Epidemic Modeling 101: Or why your CoVID19 exponential fits are wrong

2. All CoVID-19 models are wrong, but some are useful

3. Adding confidence intervals and stochastic effects to your CoVID-19 Models

4. Impact of Seasonal effects on CoVID-19

5. Competing CoVID-19 Strains

6. Network Structure, Super-spreaders and Contact Tracing

7. Network Models, the effect of degree correlations

8. How to model the effects of vaccination

9. The Impact of Age Structure on Epidemic Spreading

Read more

Subscribe now

Real-world networks are typically large and hard to visualize, leading to the development of sophisticated methods and algorithms, such as degree correlations, network structure, and modularity, to explore their structure systematically. In this post, we continue this exploration by learning about k-core decomposition.

The k-core of a graph is the largest subgraph where every node has degree of at least k. The k-core can easily computed by recursively removing every node of degree less than k. In Python, this can done with:

Read more

This is a conversation space in the Substack app that I set up exclusively for my subscribers — kind of like a group chat or live hangout. I’ll post short prompts, thoughts, and updates that come my way, and you can jump into the discussion.

To join our chat, you’ll need to download the Substack app, now available for both iOS and Android. Chats are sent via the app, not email, so turn on push notifications so you don’t miss conversation as it happens.

Read more

Welcome to the latest post on our long-running epidemic modeling series. In this post, we’ll cover how to introduce age structure into our compartmental models and its impact on how an epidemic spreads.

If you need a quick refresher, check out our previous posts in this series (especially the first 2):

1. Epidemic Modeling 101: Or why your CoVID19 exponential fits are wrong

2. All CoVID-19 models are wrong, but some are useful

3. Adding confidence intervals and stochastic effects to your CoVID-19 Models

4. Impact of Seasonal effects on CoVID-19

5. Competing CoVID-19 Strains

6. Network Structure, Super-spreaders and Contact Tracing

7. Network Models, the effect of degree correlations

8. How to model the effects of vaccination

Read more

2023 was a year full of great books, some just published and others that have been around for a while and that have just gotten to my hands. Below, you’ll find some of the ones I found the most impactful for my work and personal interests.

1. Computing the Climate 

A captivating journey into the synergy of climate science and computing, making it a must-read for anyone intrigued by the intersection of these fields. Easterbrook’s engaging writing style effortlessly demystifies complex concepts, ensuring accessibility for readers with diverse backgrounds. The book’s strength lies in its seamless blend of theoretical discussions with real-world examples, showcasing the instrumental role of computing in unraveling the intricacies of climate dynamics.

Easterbrook’s balanced perspective sets this book apart, acknowledging the uncertainties in climate science while underscoring the transformative impact of technological advancements. By delving into interdisciplinary connections with policy, economics, and environmental science, Easterbrook provides a holistic understanding of the challenges associated with climate change. This comprehensive approach educates and empowers readers to recognize the pivotal role of computational progress in shaping our collective response to climate-related issues.

In essence, “Computing the Climate” stands as a persuasive testament to the indispensable role of computing in climate research. Easterbrook’s skillful narrative not only informs but also inspires readers to grasp the significance of technological innovation in confronting the pressing challenges of our changing climate. This book is an essential addition to the literature, urging readers to actively engage in the ongoing dialogue surrounding the future of our planet.

Buy On Amazon

2. SQL for Data Analysis 

A must-have resource for anyone serious about data analysis and SQL. It equips you with the tools and knowledge to confidently tackle complex data analysis tasks. With its clear explanations, real-world examples, and comprehensive coverage of advanced topics, this book will undoubtedly become an invaluable asset in your data analysis journey. Whether you’re a data analyst, data scientist, or SQL enthusiast, this book will help you take your skills to the next level and transform data into actionable insights.

Buy On Amazon

3. Network Science with Python 

A highly recommended book for anyone interested in network analysis. It provides a comprehensive and accessible introduction to the topic. The book’s linear progression and friendly tone make it highly engaging and easy to follow. The author’s contagious enthusiasm and practical examples effectively communicate the power and importance of network analysis. The book covers various domains, including language and social media data mining, and explores the relationship between NLP and networks, an approach similar to our very own Graphs for Data Science substack. It emphasizes the value of actionable insights in the conversational AI domain and provides historical context and real-world use cases for NLP solutions. The book also introduces the Python packages used and dives into network science using the NetworkX library. It demonstrates how graphs can be used in machine learning and covers important concepts like betweenness centrality, page rank, and community detection with real-world applications. Overall, “Network Science with Python” is a well-written and comprehensive guide that offers practical insights and is suitable for readers of all levels.

Buy On Amazon

4. What is ChatGPT Doing… and Why Does It Work? 

This book offers a compelling exploration of ChatGPT’s success and the broader implications of artificial intelligence while delving into the inner workings of ChatGPT and its historical context.

The book stands out for its accessibility, making complex concepts understandable to non-computer scientists and non-mathematicians alike. Wolfram provides a high-level overview of ChatGPT’s components, such as embeddings and transformers, without getting bogged down in technical details.

While acknowledging ChatGPT’s successes, Wolfram candidly addresses its limitations, especially in real-world computational tasks. He critically examines ChatGPT’s reliance on neural network weightings and raises intriguing questions about intelligence, drawing parallels with biological evolution.

In conclusion, “What Is ChatGPT doing … and Why Does It Work?” is a thought-provoking read that demystifies AI for a wide audience. Wolfram’s expertise and candid exploration make it a must-read for those interested in AI’s frontiers and the complexities of intelligence.

Buy On Amazon

5. Causal Factor Investing 

 Causal Factor Investing introduces a unique approach by emphasizing causality, a factor often overlooked in traditional quantitative models. The book provides a refreshing perspective on financial markets, challenging conventional wisdom and offering a comprehensive guide to understanding and implementing causal factor investing.

What sets this book apart is its practicality. López de Prado delves into the theoretical underpinnings of causal factor investing and provides real-world examples and case studies. This hands-on approach ensures that readers can grasp and apply the concepts to their investment strategies. The author’s writing style is engaging and approachable, balancing academic rigor and practical applicability.

In a landscape where financial strategies constantly evolve, “Causal Factor Investing” stands out as a timely and relevant resource. López de Prado’s expertise shines through, making this book essential for both seasoned professionals and those new to quantitative finance. It’s a must-read for anyone serious about staying ahead in the dynamic world of investment, offering a thought-provoking and informative guide that will shape the future of quantitative finance.

Buy On Amazon

6. Data Science in Context 

Provides a comprehensive overview of Data Science, covering the technical aspects of the field and the ethical considerations and challenges it presents. It is structured to make it easy for readers to understand and absorb the information and offers recommendations for addressing ethical concerns. The authors focus on real-life examples from their various fields, such as healthcare and finance, to illustrate data science applications and their potential impact. Aimed at a wide audience, it includes data science novices and experienced professionals and is recommended for anyone with an interest in data science and its role in daily life and various industries.

Buy On Amazon

7. AI and Machine Learning for Coders 

 AI and Machine Learning for Coders  exceeds expectations with excellent explanations on how to code machine learning using TensorFlow and different ML techniques. The book covers various topics, including computer vision, natural language processing, and time series forecasting, and even includes a section on text generation.

The book is aimed specifically at coders with Python experience and explains how neural networks work at a high level without overwhelming readers with too much math. The author does an excellent job of explaining convolution, maxpooling, interpretability, bias/fairness, and Google’s AI principles.

Overall, anyone who wants to learn about deep learning using TensorFlow will find here an excellent resource that provides a solid foundation in deep learning and is suitable for hands-on practitioners without overwhelming them with math.

Buy On Amazon

8. Automate the Boring Stuff with Python 

 This is an exceptional book that is perfect for anyone who wants to learn how to use Python to automate mundane tasks. The author has done an excellent job of explaining complex concepts in a simple and easy-to-understand manner.

The book is well-organized and starts with the basics of Python programming and gradually progresses to more advanced topics such as web scraping, working with Excel spreadsheets, and sending automated emails. The real-life examples and practical exercises included in each chapter make it easy for readers to apply what they have learned and see the results.

One of the most remarkable aspects of this book is that it doesn’t require any prior programming experience, making it perfect for beginners. However, it is also an excellent resource for experienced programmers who want to learn how to automate repetitive tasks.

Overall, this is a must-read for anyone who wants to learn how to use Python for automating everyday tasks. The book is well-written, easy to follow, and the author’s sense of humor makes it an enjoyable read. I highly recommend this book to anyone who wants to learn Python programming and automate their boring tasks.

Buy On Amazon

9. Introduction to Algorithms 

Affectionately known as the bible of algorithms and over the years has proven to be an essential reading and, at over 1300 pages, a complete reference for anyone interested in gaining a broad understanding of algorithms. The content can at times be challenging but is presented in a fashion that is engaging and easily digestible. Exercises at the end of each chapter are expressly presented without the benefit of solutions but were carefully designed to help students to think algorithmically and thoroughly absorb the material presented.

Buy On Amazon

10. Advanced Algorithms and Data Structures 

 This book is a great resource for engineers who want to enhance their knowledge of algorithms and data structures without having to go back to traditional, academic-style textbooks. The author has a deep understanding of how to deliver high-quality code and each algorithm is thoroughly illustrated with pseudo-code and diagrams. The book is especially helpful for tackling contemporary problems such as multidimensional search, understanding caches better, classification, and graph theory.

The writing is friendly and approachable, making it relatively easy to understand, although some familiarity with math may be helpful. It is a book that can be read, skipped around, and returned to occasionally when needed. Overall, this book is strongly recommended for engineers looking to enhance their algorithm and data structure knowledge.

Buy On Amazon

What books kept you company throughout 2023? And which ones are you looking forward to in 2024? Leave the most interesting ones in the comment section below. And my fondest wishes for a 2024 full of great books.

Also, don’t forget to check out our reading list from previous years:

• Top 10 Books we read in 2022

• Top 10 Books we read in 2021

• Top 10 Books we read in 2020

We hope your enjoy this post and don’t forget to

Subscribe now

so you never miss a post and

Share

to help us grow! We’re looking forward to hearing your thoughts, so go ahead and….

Leave a comment

In a previous post, we explore the inverse correlation between vaccination rates and deaths. Here we explore the effect that vaccination has on disease prevalence, the number of infected individuals per capita.

The original visualization was published in the Wall Street Journal back on Feb 11th, 2015 and covers 7 different disease/vaccine pairs. For simplicity, we’ll focus on the figure for Measles, a childhood diesease that used to kill up to 2.6 million people per year, before the advent of va…

Read more

In this post, we will download GDP data from the Bureau of Economic Analysis (BEA) and use a Waterfall Chart to compare how it changed between the second quarter of 2019 and 2020. Waterfall Charts are 2D diagrams used to highlight how different components contribute to explaining a change in the value of a specific quantity, making them ideal for this specific application. McKinsey, a consulting firm, popularized Waterfall Charts and they’ve since gained popularity in financial applications.

The data we will be using can be found in Table 14 of this spreadsheet which can be downloaded from the BEA website.

The data can be downloaded directly into a pandas …

Read more

1. The Order of Time 

Carlo Rovelli explores the concept of time and how it has been understood by scientists and philosophers throughout history. It discusses advances in our understanding of time, such as Einstein’s theory of relativity, which proposed that time is affected by mass and speed. The book also explores the idea of a universe without time and discusses theories and models that have been developed to explain the universe in this way, such as Loop Quantum theory, the theory that Rovelli works on. The book is intellectually stimulating and interesting although it sometimes struggles to find the right balance between being accessible to a general audience and being a serious work of science.

Buy On Amazon

2. Code: The Hidden Language of Computer Hardware and Software  

This book provides a step-by-step timeline of how computers came to be, clearly and concisely. It takes you on a tour of what happens “behind” the pixels on your screen, from logical gates on up, without requiring a heavy technical background. Chapter, by chapter, introduces each concept and technology necessary to make modern computers work. By the end of it, you’ll have a detailed and intuitive understanding of how Computers really work and will be able to mode easily optimize how you write your own software. It might also make you want to learn to program in Assembly! A book that should be required reading for anyone interested in Computer Science.

Buy On Amazon

3. Interactive Dashboards and Data Apps with Plotly and Dash 

In our lives as data scientists and machine learning engineers, we are often called upon to develop Dashboards and other Data Drive apps to communicate results or to monitor the performance of models deployed to production. Plotly and Dash are the current State of the Art libraries for interactive visualizations with a web frontend. This book does a remarkable job of getting you up to speed with both of these libraries taking you from basic to advanced level through practical building blocks that you can immediately customize for your own use.

Buy On Amazon

4. DeFi and the Future of Finance 

Decentralized Finance (DeFi) is a new industry that aims to increase financial freedom in the world. The book explains the problems that DeFi solves for the current financial system, such as inefficiency, limited access, opacity, centralized control, and lack of interoperability. It then goes on to describe the solutions that DeFi offers, using examples and diagrams to illustrate the concepts. Finally, the book discusses the risks that DeFi faces, such as smart contract, governance, scaling, and regulatory risks. A highly recommended read for anyone interested in this burgeoning field and with some previous familiarity with financial concepts.

Buy On Amazon

5. Bayes’ Rule: A Tutorial Introduction to Bayesian Analysis 

Despite its simplicity, Bayes Rule is one of the most powerful theorems in probability theory. This relatively short book (170 pages) provides an excellent overview and hands-on introduction to Bayesian analysis. While the main text provides a strong foundation for the main ideas and concepts, the companion Python code gives you a leg up in introducing these techniques into your daily work. Overall, an excellent first introduction to the topic and a good refresher reference if you haven’t touched this area in some time.

Buy On Amazon

6. Causality 

Causal Inference is a lively and fast-developing area in Data Science that we believe has the potential to be truly revolutionary in coming years (you can get a quick overview of the main ideas in our Causal Inference series over at Medium). Judea Pearl is one of the most prominent founding fathers of this field that he introduces masterfully in this textbook. While the approach Pearl chooses is mathematically rigorous, thanks to his rich use of toy examples, the key ideas and concepts are easily grasped and adapted to real-world datasets. Causal Inference is a powerful arrow in any Data Scientist’s quiver and this is the ideal starting point if you’re interested in taking the first steps in this exciting area.

Buy On Amazon

7. The Practitioner’s Guide to Graph Data 

Graph Thinking and Graph Data are topics near and dear to our hearts here at D4Sci (checkout G4Sci if you haven’t yet) and this book does an excellent job of introducing both fundamental and advanced topics and techniques using practical real-world datasets and state of the art graph databases. The book is exceptionally well written and easy to follow, with practical “rules of thumb” generously sprinkled throughout along with practical examples that you can use to grok as the various concepts are they are introduced. A must-have for anyone interested in Graph Thinking and Graph Databases.

Buy On Amazon

8. Essential Math for Data Science 

Good mathematical foundations are paramount to being successful as a Data Scientist. In his recent book, T. Nield introduces the reader to enough of the mathematics underlying Data Science and Machine Learning to give you a head start in grokking both classic and state-of-the-art algorithms that you're likely to encounter in your career as a Data Scientist. This is a highly recommended read to anyone who wants to look inside the black boxes we use every day.

Buy On Amazon

9. The Unicorn Project 

This is a novel that focuses on the transformation required for companies to achieve sustained velocity and quality in their operations. The book explores the roles of development and business teams in this process and highlights the importance of communication and data-driven decision-making. The story is entertaining and a good illustration of industry standards in the field of IT despite (or perhaps because of) the occasional over-emphasis on long hours and burnout.

Buy On Amazon

10. Scientific Computing with Python 

 Fills an important gap in most Data Scientist's bookshelves. High-performance computing, especially scientific and numeric computing, is a fairly technical branch of computer science that most programmers aren’t familiar with. It requires a deep understanding of the finer points of the data structures being used and of their specific implementations. With this book, the authors have managed to demystify a highly complex subject with clear explanations and analysis that will appeal to developers of all levels. Naturally, some of the subjects could gain from being further developed, but in every project of this kind, a balance must be struck between breadth and depth. Overall, a useful book that I’ll refer to often.

Buy On Amazon

What books kept you company throughout 2022? And which ones are you looking forward to in 2023?

Also, don’t forget to check out our reading list from previous years

• Top 10 Books we read in 2021

• Top 10 Books we read in 2020