Crandi

Deep Learning From Scratch In Modern C Medium

Leo Migdal
-
deep learning from scratch in modern c medium

Almost a year ago, I found myself in a situation where I wanted to grow in my career and learn something new. I had a deep fascination with Machine Learning and more specifically Computer Vision. I could have reached for TensorFlow or PyTorch and started learning the high level use of these existing frameworks, but something in me needed to know how machine learning works under the hood. Not just the high-level concepts, but the actual mathematical operations, memory management, and the beautiful complexity that makes modern AI possible. So I did what any reasonable person would do: I decided to build a Convolutional Neural Network from scratch in pure C with no dependencies. Yes, you read that right.

Pure C. No frameworks, no shortcuts, no hand-holding. Just me, a text editor, and a deep desire to understand every single operation that makes neural networks tick. Building a neural network in C is like building a house with only a hammer and a chisel. It’s possible, but every single detail requires careful thought and implementation. You can’t just call model.fit(), you need to:

But here’s the thing: when you build something from scratch, you understand it completely. Every convolution operation, every backpropagation step, every memory allocation becomes a conscious choice. You don’t just know what the network does — you know how it does it, why it works, and where the bottlenecks are. It is needless to say how relevant machine learning frameworks are for research and industry. Due to their extensibility and flexibility, it is rare to find a project that does not use Google TensorFlow or Meta PyTorch nowadays. It may seem counter-intuitive to spend time coding machine learning algorithms from scratch without any base framework.

However, it is not. Coding the algorithms ourselves provides a clear and solid understanding of how the algorithms work and what the models are really doing. In this series, we will learn how to code the must-to-know deep learning algorithms such as convolutions, backpropagation, activation functions, optimizers, deep neural networks, and so on, using only plain and modern C++. We will begin our journey in this story by learning modern C++ language features and relevant programming details to code deep learning and machine learning models. What I cannot create, I do not understand. — Richard Feynman

This repository contains my experiments and implementations of deep learning concepts using the C programming language. So far, I have focused on building foundational components for Recurrent Neural Networks (RNNs) and Self-Attention mechanisms from scratch. This project is an attempt to delve into the core principles of deep learning by implementing neural network components without relying on high-level libraries or frameworks. It emphasizes understanding the underlying math and mechanics of deep learning. C provides direct control over memory and performance, making it an excellent choice for exploring the efficiency of neural network computations at a low level. This project aims to demonstrate that even advanced deep learning concepts can be implemented in C.

Run the example programs to test the implementations: Contributions are welcome! Feel free to open issues or submit pull requests for improvements or new features. An implementation of deep learning algorithms in pure C programming language, without any third-party dependencies. All written from scratch. Suitable for embedded systems or low-level coding projects where TensorFlow and PyTorch cannot be used.

The library currently contains the following units: No prerequisites needed. Just run make in the top folder. The following files will be created: Open the ./vs/deep-c.sln solution in Visual Studio and build/run the desired example. The provided saddle and pendulum examples should sufficiently demonstrate how to use the library.

Additionally, Doxygen can be used to build the API documentation from the source code comments. Alternatively, you may examine the comments in the MLPC and DDPGC header files. Artificial Intelligence in Plain English This project has been created as a part of subject A4M39GPU at Czech Technical University. As the title suggests, it is a deep learning framework written from scratch, using C/C++ and pure CUDA (no cuDDD). The reason why I started this project was pure curiosity as I've always found a reasonable satisfaction in re-inventing the wheel.

Experiencing an engineering problem entirely for oneself leads to a deep and intuitive understanding of the existing solutions and also it's great fun! The project can be built using CMake, and the basic usage can be seen in /src/src/main.cpp. The API functions are designed to be used similarly as in Python frameworks Keras and NumPy as those are great examples of highly functional yet intuitive interfaces. More detailed description of the framework and the design can be seen in the report.pdf. Here is a snippet from the /src/src/main.cpp demonstrating the basic usage. Special thanks go to the user wichtounete, whose code is used for loading the MNIST dataset, see the project repository https://github.com/wichtounet/mnist

People Also Search

Almost A Year Ago, I Found Myself In A Situation

Almost a year ago, I found myself in a situation where I wanted to grow in my career and learn something new. I had a deep fascination with Machine Learning and more specifically Computer Vision. I could have reached for TensorFlow or PyTorch and started learning the high level use of these existing frameworks, but something in me needed to know how machine learning works under the hood. Not just ...

Pure C. No Frameworks, No Shortcuts, No Hand-holding. Just Me,

Pure C. No frameworks, no shortcuts, no hand-holding. Just me, a text editor, and a deep desire to understand every single operation that makes neural networks tick. Building a neural network in C is like building a house with only a hammer and a chisel. It’s possible, but every single detail requires careful thought and implementation. You can’t just call model.fit(), you need to:

But Here’s The Thing: When You Build Something From Scratch,

But here’s the thing: when you build something from scratch, you understand it completely. Every convolution operation, every backpropagation step, every memory allocation becomes a conscious choice. You don’t just know what the network does — you know how it does it, why it works, and where the bottlenecks are. It is needless to say how relevant machine learning frameworks are for research and in...

However, It Is Not. Coding The Algorithms Ourselves Provides A

However, it is not. Coding the algorithms ourselves provides a clear and solid understanding of how the algorithms work and what the models are really doing. In this series, we will learn how to code the must-to-know deep learning algorithms such as convolutions, backpropagation, activation functions, optimizers, deep neural networks, and so on, using only plain and modern C++. We will begin our j...

This Repository Contains My Experiments And Implementations Of Deep Learning

This repository contains my experiments and implementations of deep learning concepts using the C programming language. So far, I have focused on building foundational components for Recurrent Neural Networks (RNNs) and Self-Attention mechanisms from scratch. This project is an attempt to delve into the core principles of deep learning by implementing neural network components without relying on h...