Consillum
Consilium: Building a Gradient-Free, Evolutionary Machine Learning Framework from ScratchIn the modern AI landscape, gradient descent and backpropagation are the undisputed kings of model training. But what happens when we abandon these traditional mechanisms and look to biological evolution for inspiration?Designed and engineered by software developer Emre Ercan, Consilium is a collaborative, evolutionary machine learning framework built completely from scratch. With zero external ML dependencies, this Python-based system evolves ensembles of neural specialists to predict tabular data, utilizing an interactive CLI and a data-agnostic Flask web dashboard.🧬 The Core Paradigm: Gradient-Free Evolutionary LearningUnlike standard neural networks that rely on frameworks like PyTorch or TensorFlow to compute gradients, Consilium's core training, optimization, and evaluation logic is built entirely using the Python standard library. The system trains neural models using pure evolutionary algorithms: mutation, crossover, and selection.Instead of a monolithic network, Consilium organizes an agent population (e.g., 144 agents) into distinct specialist groups:Numerical Specialists: Agents engineered with strong numerical weights but weak categorical embeddings.Categorical Specialists: Agents prioritizing strong categorical embeddings over numerical features.Interaction/Hybrid Specialists: Agents focused on capturing complex interactions between different feature types.Through Gradual Hybridization, outstanding agents earn "transfer rights" to crossbreed across these specialist groups, producing highly adaptable hybrid models.🧠 Under the Hood: Collaborative Penalty and Smart CrossoverWhat makes Consilium unique is that agents are never evaluated in isolation; they are evaluated inside an ensemble. This prevents agents from converging on identical, mediocre strategies.1. Collaborative Penalty MathFor each row in a batch, the ensemble output is calculated as the mean of its agents:$$\hat{y}_{ens} = \frac{1}{N}\sum_{i=1}^{N} \hat{y}_i$$If the ensemble predicts incorrectly, an individual agent $i$ is penalized using the following equation:$$\text{Penalty}_i = |\hat{y}_i - y| + \gamma \cdot (\hat{y}_{ens} - y) \cdot (\hat{y}_i - \hat{y}_{ens})$$Where $\gamma$ acts as the collaborative penalty weight. This mathematical constraint ensures that agents who skew the ensemble's prediction in the wrong direction are heavily penalized, actively forcing the emergence of complementary, specialized behaviors rather than redundant agents.2. Feature-Level Ablation & Smart CrossoverConsilium continuously measures the "expertise" of each agent via Feature-Level Ablation. By zeroing out individual features one at a time and measuring the resulting increase in error, the system dynamically calculates ablation scores.These scores are then transformed into a probability distribution utilized during Smart Crossover. When Agent A and Agent B breed, the child inherits weights at the feature level based on which parent understands that specific feature best:$$\text{Child Feature Weights}_f = \begin{cases} \text{Agent A Weights}_f & \text{with probability } \frac{\text{Score}_{A, f}}{\text{Score}_{A, f} + \text{Score}_{B, f}} \\ \text{Agent B Weights}_f & \text{otherwise} \end{cases}$$📊 Performance and the Data-Agnostic DashboardWhile pure evolution is notoriously difficult to optimize compared to gradient descent, Consilium stands its ground. Tested on the Polish car market dataset (~100,000 listings) to predict vehicle prices, the pure evolutionary model achieved a respectable 78.7% $R^2$ and a Mean Absolute Error of ~16,764 PLN, proving the viability of zero-dependency evolutionary algorithms against baselines like sklearn's Random Forest (93.9% $R^2$).To make this architecture accessible, Consilium ships with a Data-Agnostic Flask Web Dashboard. Users can simply upload any tabular CSV file, dynamically configure feature types, select a target variable, and train the population asynchronously. The dashboard provides real-time loss curves and allows users to interactively run inference on the evolved "Hall of Fame" ensemble directly from the browser.Engineered by Emre Ercan, Consilium is a testament to the power of fundamental computer science, proving that complex ML architectures can be built from the ground up without relying on black-box libraries.Explore the architecture, test the dashboard, and view the source code on my GitHub:👉 github.com/kairos8713/Consilium
