Safa Bayar
About Blog Categories Contact
Safa Bayar

Contents

System Programming: Low-Level C++ Development and System Integration

Safa Bayar included in Hobby Projects
   3207 words   16 minutes 

System Programming: Low-Level C++ Development and System Integration

In this post, I’ll share insights from my System Programming project, which demonstrates advanced low-level C++ development techniques, system integration, and performance optimization in system-level applications.

Project Overview

The System Programming project showcases comprehensive low-level programming skills using C++, focusing on system integration, memory management, performance optimization, and operating system interaction. This project demonstrates mastery of system-level programming concepts and modern C++ development practices.

Technical Architecture

Project Structure

SystemProgramming/
├── src/
│   ├── memory_management/
│   │   ├── custom_allocator.cpp
│   │   ├── memory_pool.cpp
│   │   └── smart_pointers.cpp
│   ├── file_system/
│   │   ├── file_operations.cpp
│   │   ├── directory_traversal.cpp
│   │   └── file_monitoring.cpp
│   ├── networking/
│   │   ├── tcp_server.cpp
│   │   ├── udp_client.cpp
│   │   └── socket_wrapper.cpp
│   ├── threading/
│   │   ├── thread_pool.cpp
│   │   ├── mutex_wrapper.cpp
│   │   └── async_operations.cpp
│   ├── performance/
│   │   ├── profiling.cpp
│   │   ├── optimization.cpp
│   │   └── benchmarking.cpp
│   └── utilities/
│       ├── logging.cpp
│       ├── configuration.cpp
│       └── error_handling.cpp
├── include/
├── tests/
├── benchmarks/
├── CMakeLists.txt
└── README.md

Core Components

Custom Memory Allocator

// src/memory_management/custom_allocator.cpp
#include <memory>
#include <unordered_map>
#include <mutex>
#include <algorithm>

namespace SystemProgramming {

class CustomAllocator {
private:
    struct Block {
        void* ptr;
        size_t size;
        bool isFree;
        Block* next;
        Block* prev;
    };
    
    void* memoryPool;
    size_t poolSize;
    Block* freeList;
    Block* usedList;
    std::mutex allocatorMutex;
    std::unordered_map<void*, Block*> blockMap;
    
    static constexpr size_t ALIGNMENT = 8;
    static constexpr size_t MIN_BLOCK_SIZE = 32;
    
public:
    explicit CustomAllocator(size_t poolSize = 1024 * 1024) 
        : poolSize(poolSize), freeList(nullptr), usedList(nullptr) {
        initializePool();
    }
    
    ~CustomAllocator() {
        std::free(memoryPool);
    }
    
    void* allocate(size_t size) {
        std::lock_guard<std::mutex> lock(allocatorMutex);
        
        // Align size to ALIGNMENT boundary
        size = alignSize(size);
        
        // Find suitable free block
        Block* block = findFreeBlock(size);
        if (!block) {
            return nullptr; // Out of memory
        }
        
        // Split block if it's too large
        if (block->size > size + sizeof(Block) + MIN_BLOCK_SIZE) {
            splitBlock(block, size);
        }
        
        // Move block from free list to used list
        moveToUsedList(block);
        
        blockMap[block->ptr] = block;
        return block->ptr;
    }
    
    void deallocate(void* ptr) {
        if (!ptr) return;
        
        std::lock_guard<std::mutex> lock(allocatorMutex);
        
        auto it = blockMap.find(ptr);
        if (it == blockMap.end()) {
            return; // Invalid pointer
        }
        
        Block* block = it->second;
        blockMap.erase(it);
        
        // Move block from used list to free list
        moveToFreeList(block);
        
        // Try to coalesce with adjacent free blocks
        coalesceBlocks(block);
    }
    
    size_t getUsedMemory() const {
        std::lock_guard<std::mutex> lock(allocatorMutex);
        size_t used = 0;
        
        Block* current = usedList;
        while (current) {
            used += current->size;
            current = current->next;
        }
        
        return used;
    }
    
    size_t getFreeMemory() const {
        std::lock_guard<std::mutex> lock(allocatorMutex);
        size_t free = 0;
        
        Block* current = freeList;
        while (current) {
            free += current->size;
            current = current->next;
        }
        
        return free;
    }
    
private:
    void initializePool() {
        memoryPool = std::aligned_alloc(ALIGNMENT, poolSize);
        if (!memoryPool) {
            throw std::bad_alloc();
        }
        
        // Create initial free block
        Block* initialBlock = static_cast<Block*>(memoryPool);
        initialBlock->ptr = static_cast<char*>(memoryPool) + sizeof(Block);
        initialBlock->size = poolSize - sizeof(Block);
        initialBlock->isFree = true;
        initialBlock->next = nullptr;
        initialBlock->prev = nullptr;
        
        freeList = initialBlock;
    }
    
    size_t alignSize(size_t size) {
        return (size + ALIGNMENT - 1) & ~(ALIGNMENT - 1);
    }
    
    Block* findFreeBlock(size_t size) {
        Block* current = freeList;
        while (current) {
            if (current->size >= size) {
                return current;
            }
            current = current->next;
        }
        return nullptr;
    }
    
    void splitBlock(Block* block, size_t size) {
        Block* newBlock = reinterpret_cast<Block*>(
            static_cast<char*>(block->ptr) + size);
        
        newBlock->ptr = static_cast<char*>(newBlock) + sizeof(Block);
        newBlock->size = block->size - size - sizeof(Block);
        newBlock->isFree = true;
        newBlock->next = block->next;
        newBlock->prev = block->prev;
        
        if (newBlock->next) {
            newBlock->next->prev = newBlock;
        }
        if (newBlock->prev) {
            newBlock->prev->next = newBlock;
        }
        
        block->size = size;
        
        if (block == freeList) {
            freeList = newBlock;
        }
    }
    
    void moveToUsedList(Block* block) {
        // Remove from free list
        if (block->prev) {
            block->prev->next = block->next;
        } else {
            freeList = block->next;
        }
        if (block->next) {
            block->next->prev = block->prev;
        }
        
        // Add to used list
        block->isFree = false;
        block->next = usedList;
        block->prev = nullptr;
        
        if (usedList) {
            usedList->prev = block;
        }
        usedList = block;
    }
    
    void moveToFreeList(Block* block) {
        // Remove from used list
        if (block->prev) {
            block->prev->next = block->next;
        } else {
            usedList = block->next;
        }
        if (block->next) {
            block->next->prev = block->prev;
        }
        
        // Add to free list
        block->isFree = true;
        block->next = freeList;
        block->prev = nullptr;
        
        if (freeList) {
            freeList->prev = block;
        }
        freeList = block;
    }
    
    void coalesceBlocks(Block* block) {
        // Coalesce with next block
        if (block->next && block->next->isFree) {
            Block* nextBlock = block->next;
            
            // Remove next block from free list
            if (nextBlock->prev) {
                nextBlock->prev->next = nextBlock->next;
            } else {
                freeList = nextBlock->next;
            }
            if (nextBlock->next) {
                nextBlock->next->prev = nextBlock->prev;
            }
            
            // Merge blocks
            block->size += nextBlock->size + sizeof(Block);
            block->next = nextBlock->next;
            if (nextBlock->next) {
                nextBlock->next->prev = block;
            }
        }
        
        // Coalesce with previous block
        if (block->prev && block->prev->isFree) {
            Block* prevBlock = block->prev;
            
            // Remove current block from free list
            if (block->prev) {
                block->prev->next = block->next;
            } else {
                freeList = block->next;
            }
            if (block->next) {
                block->next->prev = block->prev;
            }
            
            // Merge blocks
            prevBlock->size += block->size + sizeof(Block);
            prevBlock->next = block->next;
            if (block->next) {
                block->next->prev = prevBlock;
            }
        }
    }
};

// Custom allocator for STL containers
template<typename T>
class STLAllocator {
private:
    CustomAllocator* allocator;
    
public:
    using value_type = T;
    using pointer = T*;
    using const_pointer = const T*;
    using reference = T&;
    using const_reference = const T&;
    using size_type = std::size_t;
    using difference_type = std::ptrdiff_t;
    
    explicit STLAllocator(CustomAllocator* alloc) : allocator(alloc) {}
    
    template<typename U>
    STLAllocator(const STLAllocator<U>& other) : allocator(other.allocator) {}
    
    pointer allocate(size_type n) {
        return static_cast<pointer>(allocator->allocate(n * sizeof(T)));
    }
    
    void deallocate(pointer p, size_type n) {
        allocator->deallocate(p);
    }
    
    template<typename U>
    bool operator==(const STLAllocator<U>& other) const {
        return allocator == other.allocator;
    }
    
    template<typename U>
    bool operator!=(const STLAllocator<U>& other) const {
        return !(*this == other);
    }
};

} // namespace SystemProgramming

Thread Pool Implementation

// src/threading/thread_pool.cpp
#include <thread>
#include <queue>
#include <vector>
#include <mutex>
#include <condition_variable>
#include <future>
#include <functional>
#include <atomic>

namespace SystemProgramming {

class ThreadPool {
private:
    std::vector<std::thread> workers;
    std::queue<std::function<void()>> tasks;
    std::mutex queueMutex;
    std::condition_variable condition;
    std::atomic<bool> stop;
    
public:
    explicit ThreadPool(size_t numThreads = std::thread::hardware_concurrency()) 
        : stop(false) {
        
        for (size_t i = 0; i < numThreads; ++i) {
            workers.emplace_back([this] {
                while (true) {
                    std::function<void()> task;
                    
                    {
                        std::unique_lock<std::mutex> lock(queueMutex);
                        condition.wait(lock, [this] { return stop || !tasks.empty(); });
                        
                        if (stop && tasks.empty()) {
                            return;
                        }
                        
                        task = std::move(tasks.front());
                        tasks.pop();
                    }
                    
                    task();
                }
            });
        }
    }
    
    ~ThreadPool() {
        {
            std::unique_lock<std::mutex> lock(queueMutex);
            stop = true;
        }
        
        condition.notify_all();
        
        for (std::thread& worker : workers) {
            worker.join();
        }
    }
    
    template<typename F, typename... Args>
    auto enqueue(F&& f, Args&&... args) 
        -> std::future<typename std::result_of<F(Args...)>::type> {
        
        using return_type = typename std::result_of<F(Args...)>::type;
        
        auto task = std::make_shared<std::packaged_task<return_type()>>(
            std::bind(std::forward<F>(f), std::forward<Args>(args)...)
        );
        
        std::future<return_type> result = task->get_future();
        
        {
            std::unique_lock<std::mutex> lock(queueMutex);
            
            if (stop) {
                throw std::runtime_error("enqueue on stopped ThreadPool");
            }
            
            tasks.emplace([task] { (*task)(); });
        }
        
        condition.notify_one();
        return result;
    }
    
    size_t getTaskCount() const {
        std::unique_lock<std::mutex> lock(queueMutex);
        return tasks.size();
    }
    
    size_t getWorkerCount() const {
        return workers.size();
    }
};

// Advanced thread pool with priority queues
class PriorityThreadPool {
private:
    struct Task {
        std::function<void()> function;
        int priority;
        
        bool operator<(const Task& other) const {
            return priority < other.priority; // Higher priority first
        }
    };
    
    std::vector<std::thread> workers;
    std::priority_queue<Task> tasks;
    std::mutex queueMutex;
    std::condition_variable condition;
    std::atomic<bool> stop;
    
public:
    explicit PriorityThreadPool(size_t numThreads = std::thread::hardware_concurrency()) 
        : stop(false) {
        
        for (size_t i = 0; i < numThreads; ++i) {
            workers.emplace_back([this] {
                while (true) {
                    Task task;
                    
                    {
                        std::unique_lock<std::mutex> lock(queueMutex);
                        condition.wait(lock, [this] { return stop || !tasks.empty(); });
                        
                        if (stop && tasks.empty()) {
                            return;
                        }
                        
                        task = std::move(const_cast<Task&>(tasks.top()));
                        tasks.pop();
                    }
                    
                    task.function();
                }
            });
        }
    }
    
    ~PriorityThreadPool() {
        {
            std::unique_lock<std::mutex> lock(queueMutex);
            stop = true;
        }
        
        condition.notify_all();
        
        for (std::thread& worker : workers) {
            worker.join();
        }
    }
    
    template<typename F, typename... Args>
    auto enqueue(int priority, F&& f, Args&&... args) 
        -> std::future<typename std::result_of<F(Args...)>::type> {
        
        using return_type = typename std::result_of<F(Args...)>::type;
        
        auto task = std::make_shared<std::packaged_task<return_type()>>(
            std::bind(std::forward<F>(f), std::forward<Args>(args)...)
        );
        
        std::future<return_type> result = task->get_future();
        
        {
            std::unique_lock<std::mutex> lock(queueMutex);
            
            if (stop) {
                throw std::runtime_error("enqueue on stopped PriorityThreadPool");
            }
            
            tasks.emplace([task] { (*task)(); }, priority);
        }
        
        condition.notify_one();
        return result;
    }
};

} // namespace SystemProgramming

File System Operations

// src/file_system/file_operations.cpp
#include <fstream>
#include <filesystem>
#include <chrono>
#include <functional>

namespace SystemProgramming {

class FileOperations {
public:
    // Asynchronous file operations
    static std::future<std::string> readFileAsync(const std::string& filename) {
        return std::async(std::launch::async, [filename]() -> std::string {
            std::ifstream file(filename, std::ios::binary);
            if (!file) {
                throw std::runtime_error("Cannot open file: " + filename);
            }
            
            file.seekg(0, std::ios::end);
            size_t size = file.tellg();
            file.seekg(0, std::ios::beg);
            
            std::string content(size, '\0');
            file.read(&content[0], size);
            
            return content;
        });
    }
    
    static std::future<void> writeFileAsync(const std::string& filename, 
                                           const std::string& content) {
        return std::async(std::launch::async, [filename, content]() {
            std::ofstream file(filename, std::ios::binary);
            if (!file) {
                throw std::runtime_error("Cannot create file: " + filename);
            }
            
            file.write(content.data(), content.size());
        });
    }
    
    // Memory-mapped file operations
    class MemoryMappedFile {
    private:
        void* mappedMemory;
        size_t fileSize;
        int fileDescriptor;
        
    public:
        explicit MemoryMappedFile(const std::string& filename) 
            : mappedMemory(nullptr), fileSize(0), fileDescriptor(-1) {
            
            fileDescriptor = open(filename.c_str(), O_RDONLY);
            if (fileDescriptor == -1) {
                throw std::runtime_error("Cannot open file: " + filename);
            }
            
            struct stat fileStat;
            if (fstat(fileDescriptor, &fileStat) == -1) {
                close(fileDescriptor);
                throw std::runtime_error("Cannot get file stats: " + filename);
            }
            
            fileSize = fileStat.st_size;
            
            mappedMemory = mmap(nullptr, fileSize, PROT_READ, MAP_PRIVATE, 
                               fileDescriptor, 0);
            if (mappedMemory == MAP_FAILED) {
                close(fileDescriptor);
                throw std::runtime_error("Cannot map file to memory: " + filename);
            }
        }
        
        ~MemoryMappedFile() {
            if (mappedMemory != MAP_FAILED) {
                munmap(mappedMemory, fileSize);
            }
            if (fileDescriptor != -1) {
                close(fileDescriptor);
            }
        }
        
        const void* getData() const { return mappedMemory; }
        size_t getSize() const { return fileSize; }
        
        // Non-copyable
        MemoryMappedFile(const MemoryMappedFile&) = delete;
        MemoryMappedFile& operator=(const MemoryMappedFile&) = delete;
    };
    
    // File monitoring
    class FileMonitor {
    private:
        std::string watchPath;
        std::function<void(const std::string&)> callback;
        std::atomic<bool> running;
        std::thread monitorThread;
        
    public:
        FileMonitor(const std::string& path, 
                   std::function<void(const std::string&)> cb) 
            : watchPath(path), callback(cb), running(false) {}
        
        ~FileMonitor() {
            stop();
        }
        
        void start() {
            if (running) return;
            
            running = true;
            monitorThread = std::thread([this] {
                monitorLoop();
            });
        }
        
        void stop() {
            if (!running) return;
            
            running = false;
            if (monitorThread.joinable()) {
                monitorThread.join();
            }
        }
        
    private:
        void monitorLoop() {
            auto lastWriteTime = std::filesystem::last_write_time(watchPath);
            
            while (running) {
                try {
                    auto currentWriteTime = std::filesystem::last_write_time(watchPath);
                    
                    if (currentWriteTime != lastWriteTime) {
                        callback(watchPath);
                        lastWriteTime = currentWriteTime;
                    }
                    
                    std::this_thread::sleep_for(std::chrono::milliseconds(100));
                } catch (const std::exception& e) {
                    // Handle file system errors
                    std::this_thread::sleep_for(std::chrono::seconds(1));
                }
            }
        }
    };
    
    // Directory traversal
    static std::vector<std::string> traverseDirectory(const std::string& path, 
                                                     bool recursive = true) {
        std::vector<std::string> files;
        
        try {
            if (recursive) {
                for (const auto& entry : std::filesystem::recursive_directory_iterator(path)) {
                    if (entry.is_regular_file()) {
                        files.push_back(entry.path().string());
                    }
                }
            } else {
                for (const auto& entry : std::filesystem::directory_iterator(path)) {
                    if (entry.is_regular_file()) {
                        files.push_back(entry.path().string());
                    }
                }
            }
        } catch (const std::filesystem::filesystem_error& e) {
            throw std::runtime_error("Directory traversal error: " + std::string(e.what()));
        }
        
        return files;
    }
    
    // File operations with error handling
    static bool copyFile(const std::string& source, const std::string& destination) {
        try {
            std::filesystem::copy_file(source, destination, 
                                      std::filesystem::copy_options::overwrite_existing);
            return true;
        } catch (const std::filesystem::filesystem_error& e) {
            return false;
        }
    }
    
    static bool moveFile(const std::string& source, const std::string& destination) {
        try {
            std::filesystem::rename(source, destination);
            return true;
        } catch (const std::filesystem::filesystem_error& e) {
            return false;
        }
    }
    
    static bool deleteFile(const std::string& filename) {
        try {
            return std::filesystem::remove(filename);
        } catch (const std::filesystem::filesystem_error& e) {
            return false;
        }
    }
};

} // namespace SystemProgramming

Network Operations

// src/networking/tcp_server.cpp
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <fcntl.h>
#include <poll.h>

namespace SystemProgramming {

class TCPServer {
private:
    int serverSocket;
    int port;
    std::atomic<bool> running;
    std::thread serverThread;
    std::vector<std::thread> clientThreads;
    std::mutex clientMutex;
    
public:
    explicit TCPServer(int port) : port(port), running(false) {
        serverSocket = socket(AF_INET, SOCK_STREAM, 0);
        if (serverSocket == -1) {
            throw std::runtime_error("Failed to create socket");
        }
        
        // Set socket options
        int opt = 1;
        if (setsockopt(serverSocket, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt)) == -1) {
            throw std::runtime_error("Failed to set socket options");
        }
        
        // Make socket non-blocking
        int flags = fcntl(serverSocket, F_GETFL, 0);
        fcntl(serverSocket, F_SETFL, flags | O_NONBLOCK);
    }
    
    ~TCPServer() {
        stop();
        if (serverSocket != -1) {
            close(serverSocket);
        }
    }
    
    void start() {
        if (running) return;
        
        // Bind socket
        struct sockaddr_in address;
        address.sin_family = AF_INET;
        address.sin_addr.s_addr = INADDR_ANY;
        address.sin_port = htons(port);
        
        if (bind(serverSocket, (struct sockaddr*)&address, sizeof(address)) == -1) {
            throw std::runtime_error("Failed to bind socket");
        }
        
        // Listen for connections
        if (listen(serverSocket, 10) == -1) {
            throw std::runtime_error("Failed to listen on socket");
        }
        
        running = true;
        serverThread = std::thread([this] { serverLoop(); });
    }
    
    void stop() {
        if (!running) return;
        
        running = false;
        
        if (serverThread.joinable()) {
            serverThread.join();
        }
        
        // Close all client connections
        std::lock_guard<std::mutex> lock(clientMutex);
        for (auto& thread : clientThreads) {
            if (thread.joinable()) {
                thread.join();
            }
        }
        clientThreads.clear();
    }
    
private:
    void serverLoop() {
        struct pollfd pollFds[1];
        pollFds[0].fd = serverSocket;
        pollFds[0].events = POLLIN;
        
        while (running) {
            int result = poll(pollFds, 1, 1000); // 1 second timeout
            
            if (result > 0 && (pollFds[0].revents & POLLIN)) {
                // Accept new connection
                struct sockaddr_in clientAddress;
                socklen_t clientAddressLen = sizeof(clientAddress);
                
                int clientSocket = accept(serverSocket, 
                                        (struct sockaddr*)&clientAddress, 
                                        &clientAddressLen);
                
                if (clientSocket != -1) {
                    // Handle client in separate thread
                    std::lock_guard<std::mutex> lock(clientMutex);
                    clientThreads.emplace_back([this, clientSocket, clientAddress] {
                        handleClient(clientSocket, clientAddress);
                    });
                }
            }
        }
    }
    
    void handleClient(int clientSocket, const struct sockaddr_in& clientAddress) {
        char buffer[4096];
        
        while (running) {
            ssize_t bytesReceived = recv(clientSocket, buffer, sizeof(buffer) - 1, 0);
            
            if (bytesReceived <= 0) {
                break; // Client disconnected
            }
            
            buffer[bytesReceived] = '\0';
            
            // Process received data
            std::string response = processRequest(buffer);
            
            // Send response
            send(clientSocket, response.c_str(), response.length(), 0);
        }
        
        close(clientSocket);
    }
    
    virtual std::string processRequest(const std::string& request) {
        // Default implementation - echo the request
        return "Echo: " + request;
    }
};

// UDP Client implementation
class UDPClient {
private:
    int socketFd;
    struct sockaddr_in serverAddress;
    
public:
    UDPClient(const std::string& serverIP, int serverPort) {
        socketFd = socket(AF_INET, SOCK_DGRAM, 0);
        if (socketFd == -1) {
            throw std::runtime_error("Failed to create UDP socket");
        }
        
        serverAddress.sin_family = AF_INET;
        serverAddress.sin_port = htons(serverPort);
        inet_pton(AF_INET, serverIP.c_str(), &serverAddress.sin_addr);
    }
    
    ~UDPClient() {
        if (socketFd != -1) {
            close(socketFd);
        }
    }
    
    bool sendMessage(const std::string& message) {
        ssize_t bytesSent = sendto(socketFd, message.c_str(), message.length(), 0,
                                 (struct sockaddr*)&serverAddress, sizeof(serverAddress));
        
        return bytesSent == static_cast<ssize_t>(message.length());
    }
    
    std::string receiveMessage(size_t maxSize = 4096) {
        char buffer[maxSize];
        struct sockaddr_in fromAddress;
        socklen_t fromAddressLen = sizeof(fromAddress);
        
        ssize_t bytesReceived = recvfrom(socketFd, buffer, maxSize - 1, 0,
                                       (struct sockaddr*)&fromAddress, &fromAddressLen);
        
        if (bytesReceived > 0) {
            buffer[bytesReceived] = '\0';
            return std::string(buffer);
        }
        
        return "";
    }
};

} // namespace SystemProgramming

Performance Profiling

// src/performance/profiling.cpp
#include <chrono>
#include <unordered_map>
#include <mutex>
#include <fstream>
#include <iomanip>

namespace SystemProgramming {

class PerformanceProfiler {
private:
    struct ProfileData {
        std::chrono::high_resolution_clock::time_point startTime;
        std::chrono::nanoseconds totalTime{0};
        size_t callCount{0};
        std::chrono::nanoseconds minTime{std::chrono::nanoseconds::max()};
        std::chrono::nanoseconds maxTime{std::chrono::nanoseconds::min()};
    };
    
    std::unordered_map<std::string, ProfileData> profiles;
    std::mutex profileMutex;
    
public:
    class ScopedTimer {
    private:
        PerformanceProfiler* profiler;
        std::string functionName;
        std::chrono::high_resolution_clock::time_point startTime;
        
    public:
        ScopedTimer(PerformanceProfiler* prof, const std::string& name) 
            : profiler(prof), functionName(name) {
            startTime = std::chrono::high_resolution_clock::now();
        }
        
        ~ScopedTimer() {
            auto endTime = std::chrono::high_resolution_clock::now();
            auto duration = endTime - startTime;
            profiler->recordProfile(functionName, duration);
        }
    };
    
    void startProfile(const std::string& functionName) {
        std::lock_guard<std::mutex> lock(profileMutex);
        profiles[functionName].startTime = std::chrono::high_resolution_clock::now();
    }
    
    void endProfile(const std::string& functionName) {
        auto endTime = std::chrono::high_resolution_clock::now();
        
        std::lock_guard<std::mutex> lock(profileMutex);
        auto& profile = profiles[functionName];
        auto duration = endTime - profile.startTime;
        
        profile.totalTime += duration;
        profile.callCount++;
        profile.minTime = std::min(profile.minTime, duration);
        profile.maxTime = std::max(profile.maxTime, duration);
    }
    
    void recordProfile(const std::string& functionName, 
                      std::chrono::nanoseconds duration) {
        std::lock_guard<std::mutex> lock(profileMutex);
        auto& profile = profiles[functionName];
        
        profile.totalTime += duration;
        profile.callCount++;
        profile.minTime = std::min(profile.minTime, duration);
        profile.maxTime = std::max(profile.maxTime, duration);
    }
    
    void printReport() const {
        std::lock_guard<std::mutex> lock(profileMutex);
        
        std::cout << std::left << std::setw(30) << "Function Name"
                  << std::setw(15) << "Calls"
                  << std::setw(20) << "Total Time (ms)"
                  << std::setw(20) << "Avg Time (ms)"
                  << std::setw(20) << "Min Time (ms)"
                  << std::setw(20) << "Max Time (ms)" << std::endl;
        
        std::cout << std::string(105, '-') << std::endl;
        
        for (const auto& [name, data] : profiles) {
            double totalMs = std::chrono::duration<double, std::milli>(data.totalTime).count();
            double avgMs = totalMs / data.callCount;
            double minMs = std::chrono::duration<double, std::milli>(data.minTime).count();
            double maxMs = std::chrono::duration<double, std::milli>(data.maxTime).count();
            
            std::cout << std::left << std::setw(30) << name
                      << std::setw(15) << data.callCount
                      << std::setw(20) << std::fixed << std::setprecision(3) << totalMs
                      << std::setw(20) << avgMs
                      << std::setw(20) << minMs
                      << std::setw(20) << maxMs << std::endl;
        }
    }
    
    void saveReportToFile(const std::string& filename) const {
        std::lock_guard<std::mutex> lock(profileMutex);
        
        std::ofstream file(filename);
        if (!file) {
            throw std::runtime_error("Cannot create report file: " + filename);
        }
        
        file << "Performance Profile Report\n";
        file << "Generated at: " << std::chrono::system_clock::now().time_since_epoch().count() << "\n\n";
        
        file << std::left << std::setw(30) << "Function Name"
             << std::setw(15) << "Calls"
             << std::setw(20) << "Total Time (ms)"
             << std::setw(20) << "Avg Time (ms)"
             << std::setw(20) << "Min Time (ms)"
             << std::setw(20) << "Max Time (ms)" << std::endl;
        
        file << std::string(105, '-') << std::endl;
        
        for (const auto& [name, data] : profiles) {
            double totalMs = std::chrono::duration<double, std::milli>(data.totalTime).count();
            double avgMs = totalMs / data.callCount;
            double minMs = std::chrono::duration<double, std::milli>(data.minTime).count();
            double maxMs = std::chrono::duration<double, std::milli>(data.maxTime).count();
            
            file << std::left << std::setw(30) << name
                 << std::setw(15) << data.callCount
                 << std::setw(20) << std::fixed << std::setprecision(3) << totalMs
                 << std::setw(20) << avgMs
                 << std::setw(20) << minMs
                 << std::setw(20) << maxMs << std::endl;
        }
    }
    
    void reset() {
        std::lock_guard<std::mutex> lock(profileMutex);
        profiles.clear();
    }
};

// Global profiler instance
extern PerformanceProfiler g_profiler;

// Macro for easy profiling
#define PROFILE_FUNCTION() \
    PerformanceProfiler::ScopedTimer _timer(&g_profiler, __FUNCTION__)

#define PROFILE_SCOPE(name) \
    PerformanceProfiler::ScopedTimer _timer(&g_profiler, name)

} // namespace SystemProgramming

CMake Build System

CMakeLists.txt

cmake_minimum_required(VERSION 3.16)
project(SystemProgramming VERSION 1.0.0 LANGUAGES CXX)

# Set C++ standard
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

# Compiler flags
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wall -Wextra -Wpedantic")
    set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} -g -O0")
    set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} -O3 -DNDEBUG")
endif()

# Include directories
include_directories(include)

# Source files
set(SOURCES
    src/memory_management/custom_allocator.cpp
    src/memory_management/memory_pool.cpp
    src/file_system/file_operations.cpp
    src/file_system/directory_traversal.cpp
    src/networking/tcp_server.cpp
    src/networking/udp_client.cpp
    src/threading/thread_pool.cpp
    src/threading/mutex_wrapper.cpp
    src/performance/profiling.cpp
    src/performance/optimization.cpp
    src/utilities/logging.cpp
    src/utilities/configuration.cpp
)

# Create library
add_library(SystemProgrammingLib STATIC ${SOURCES})

# Link libraries
find_package(Threads REQUIRED)
target_link_libraries(SystemProgrammingLib Threads::Threads)

# Create executable
add_executable(SystemProgramming main.cpp)
target_link_libraries(SystemProgramming SystemProgrammingLib)

# Testing
enable_testing()
add_subdirectory(tests)

# Benchmarking
add_subdirectory(benchmarks)

# Installation
install(TARGETS SystemProgramming SystemProgrammingLib
        RUNTIME DESTINATION bin
        LIBRARY DESTINATION lib
        ARCHIVE DESTINATION lib)

install(DIRECTORY include/ DESTINATION include)

Testing Framework

Unit Tests

// tests/test_memory_management.cpp
#include <gtest/gtest.h>
#include "SystemProgramming/custom_allocator.h"

class CustomAllocatorTest : public ::testing::Test {
protected:
    void SetUp() override {
        allocator = std::make_unique<SystemProgramming::CustomAllocator>(1024 * 1024);
    }
    
    std::unique_ptr<SystemProgramming::CustomAllocator> allocator;
};

TEST_F(CustomAllocatorTest, BasicAllocation) {
    void* ptr = allocator->allocate(100);
    ASSERT_NE(ptr, nullptr);
    
    allocator->deallocate(ptr);
}

TEST_F(CustomAllocatorTest, MultipleAllocations) {
    std::vector<void*> pointers;
    
    for (int i = 0; i < 100; ++i) {
        void* ptr = allocator->allocate(50);
        ASSERT_NE(ptr, nullptr);
        pointers.push_back(ptr);
    }
    
    for (void* ptr : pointers) {
        allocator->deallocate(ptr);
    }
}

TEST_F(CustomAllocatorTest, MemoryUsage) {
    size_t initialFree = allocator->getFreeMemory();
    
    void* ptr = allocator->allocate(1000);
    ASSERT_NE(ptr, nullptr);
    
    size_t usedAfterAlloc = allocator->getUsedMemory();
    ASSERT_GT(usedAfterAlloc, 0);
    
    allocator->deallocate(ptr);
    
    size_t freeAfterDealloc = allocator->getFreeMemory();
    ASSERT_EQ(freeAfterDealloc, initialFree);
}

TEST_F(CustomAllocatorTest, STLAllocator) {
    using Allocator = SystemProgramming::STLAllocator<int>;
    Allocator stlAllocator(allocator.get());
    
    std::vector<int, Allocator> vec(stlAllocator);
    
    for (int i = 0; i < 1000; ++i) {
        vec.push_back(i);
    }
    
    ASSERT_EQ(vec.size(), 1000);
    
    for (int i = 0; i < 1000; ++i) {
        ASSERT_EQ(vec[i], i);
    }
}

Lessons Learned

System Programming

  • Memory Management: Efficient custom memory allocators and smart pointers
  • Threading: Advanced thread pool implementations and synchronization
  • File Systems: Asynchronous file operations and memory mapping
  • Networking: Low-level socket programming and server implementations

Performance Optimization

  • Profiling: Comprehensive performance profiling and analysis
  • Memory Optimization: Custom allocators and memory pool management
  • Concurrency: Efficient thread pool and async operation patterns
  • System Integration: Direct system call usage and optimization

C++ Development

  • Modern C++: Use of C++17 features and best practices
  • RAII: Resource management and exception safety
  • Templates: Generic programming and STL integration
  • Testing: Comprehensive unit testing and benchmarking

Future Enhancements

Advanced Features

  • Lock-Free Programming: Implement lock-free data structures
  • GPU Computing: CUDA integration for parallel processing
  • Real-Time Systems: Real-time scheduling and priority management
  • Distributed Systems: Network protocols and distributed algorithms

Performance Improvements

  • SIMD Instructions: Vectorized operations for performance
  • Cache Optimization: Cache-friendly data structures and algorithms
  • Memory Mapping: Advanced memory mapping techniques
  • Profiling Tools: Integration with external profiling tools

Conclusion

The System Programming project demonstrates comprehensive low-level C++ development skills and system integration expertise. Key achievements include:

  • Advanced Memory Management: Custom allocators and memory pool implementations
  • Concurrent Programming: Thread pools and synchronization primitives
  • System Integration: File system operations and network programming
  • Performance Optimization: Profiling tools and optimization techniques
  • Modern C++: Use of C++17 features and best practices
  • Testing: Comprehensive testing and benchmarking frameworks

The project is available on GitHub and serves as a comprehensive example of system-level programming and performance optimization.

This project represents my deep dive into system programming and demonstrates how low-level programming skills can be applied to create high-performance, efficient systems. The lessons learned here continue to influence my approach to performance-critical applications and system design.

Updated on 2025-04-05
 , , , , , , 
Back | Home