By admin 
Just two weeks after the groundbreaking release of Inkling, its inaugural open-source AI language model, Thinking Machines, the ambitious startup helmed by former OpenAI chief technology officer Mira Murati, has once again disrupted the artificial intelligence landscape with the introduction of Inkling-Small. This new iteration, launched today, achieves a remarkable feat: it offers performance levels that are virtually indistinguishable from its larger predecessor, often surpassing it on critical benchmarks, all while boasting a significantly reduced size and an accessible Apache 2.0 license. This strategic move by Thinking Machines, a company known for its substantial funding and focus on human-AI collaboration, signals a clear intent to democratize access to high-caliber multimodal AI for a broader range of enterprises.
Inkling-Small is engineered as a 276-billion-parameter multimodal reasoning model, a substantial but more manageable architecture compared to the original Inkling’s 975 billion parameters (internal model settings). Crucially, it operates with a permissive Apache 2.0 license, fostering widespread adoption and innovation. Its performance is so compelling that it registers a mere one-point difference from its larger sibling on the esteemed Artificial Analysis Intelligence Index, a testament to the efficiency of its design. The model’s capabilities are inherently multimodal, accepting text, image, and audio inputs and generating text outputs. Furthermore, it supports an expansive context window of up to one million tokens, enabling it to process and understand vast amounts of information within a single interaction.
The efficiency gains are particularly evident in Inkling-Small’s parameter utilization. While the original Inkling utilizes 41 billion active parameters per token, Inkling-Small achieves comparable, and in some cases superior, performance with only 12 billion active parameters per token. This dramatic reduction in active parameter usage, while preserving much of the flagship model’s prowess in coding, reasoning, and multimodal comprehension, is the cornerstone of Inkling-Small’s appeal to enterprises. The implication for businesses is not simply a smaller model; it’s a model that requires considerably less computational power for inference, leading to reduced operational costs and a smaller deployment footprint.
While Inkling-Small remains too resource-intensive for deployment on standard laptops or conventional workstations, its significantly reduced compute requirements make it a far more practical option for enterprises that possess some, but not extensive, in-house graphics processing unit (GPU) capabilities. The reduction in active parameters translates directly into lower inference costs and a more manageable hardware setup, broadening the accessibility for organizations that might have been deterred by the immense resource demands of larger models.
Thinking Machines has demonstrated its commitment to open access by releasing the full weights of Inkling-Small on Hugging Face, a prominent platform for AI model sharing. Additionally, the company has integrated support for fine-tuning the model through its Tinker model training application programming interface (API). This dual approach—providing raw model weights for advanced users and a streamlined API for easier integration—caters to a diverse range of developer needs.
To further incentivize adoption, Thinking Machines is currently offering a limited-time 50% discount on API pricing. This promotional offer brings the standard 64K-context Inkling-Small model’s API costs down to $0.58 per million prefill (input) tokens and $1.44 per million sampled (output) tokens. Training tokens are priced at $1.73 per million, with cached prefill requests available at a competitive $0.116 per million tokens. A more extensive 256K-context variant is also accessible, albeit at slightly higher rates, catering to applications that demand even deeper contextual understanding.
Nearly the Same Performance at a Quarter the Size: A Deep Dive into Benchmarks
The performance metrics for Inkling-Small are nothing short of remarkable. Artificial Analysis, a leading independent evaluator, assigned Inkling-Small an impressive score of 40 on its Intelligence Index. This is a mere one-point difference from Inkling’s score of 41. This near-parity is particularly astonishing given that Inkling-Small boasts 276 billion total parameters and 12 billion active parameters, while Inkling utilizes a colossal 975 billion total parameters and 41 billion active parameters. The Artificial Analysis report further underscores Inkling-Small’s significance, noting that no other open-weight model of its size or smaller has achieved a higher score on the index.
Beyond the aggregate score, Inkling-Small demonstrably outperforms its larger sibling on several critical evaluations. Thinking Machines reports that Inkling-Small achieves an 80.2% score on SWE-bench Verified, surpassing Inkling’s 77.6%. Similarly, on Terminal Bench 2.1, Inkling-Small scores 64.7%, edging out Inkling’s 63.8%. The model also shows superior performance on benchmarks such as SciCode, Humanity’s Last Exam, GPQA Diamond, and CritPt. These results highlight Inkling-Small’s advanced capabilities in coding, logical reasoning, and complex problem-solving.
However, this performance enhancement is not universally distributed. Inkling retains a clear advantage in areas requiring deep factual recall and certain agentic tasks. For instance, on the τ³-Banking benchmark, Inkling-Small scores 15.5%, a notable drop from Inkling’s 23.7%. Its AA Omniscience score is also negative, indicating weaker factual coverage. While Inkling-Small does report a slightly lower hallucination rate, this tradeoff is crucial for enterprises. Inkling-Small emerges as a highly attractive option for applications such as coding assistants, tool-use systems, retrieval-augmented generation (RAG), document analysis, and various multimodal workflows. Nevertheless, for high-stakes factual tasks where accuracy is paramount, organizations will still need to implement robust retrieval mechanisms, verification processes, and human oversight.
The Mechanics Behind the Efficiency: A Sparse Mixture-of-Experts Architecture
The remarkable efficiency of Inkling-Small is rooted in its sophisticated architecture: a sparse Mixture-of-Experts (MoE) model. According to the detailed model card published by Thinking Machines, the model’s 42-layer decoder is designed to route each token to a select group of specialized experts. Specifically, each token is directed to six out of 256 specialized experts, complemented by two shared experts that remain active for every token processed. This dynamic routing mechanism is the key to understanding the disparity between the model’s 276 billion total parameters and its 12 billion active parameters. The system effectively leverages a vast pool of learned knowledge, activating only a necessary fraction during each inference step, thereby optimizing computational resources.
Inkling-Small is also natively multimodal, meaning it processes different data modalities—images, audio, and text—within a unified framework. These diverse inputs are projected into a shared representation and jointly processed by the decoder, rather than relying on separate, external systems for each modality. This integrated approach enhances efficiency and coherence in multimodal tasks. Thinking Machines explicitly lists coding assistants, agentic applications, chatbots, RAG systems, and other multimodal applications as ideal use cases for Inkling-Small.
Furthermore, the model incorporates support for variable reasoning effort. This feature allows developers to dynamically adjust the model’s test-time compute based on the complexity of the task at hand. This granular control empowers engineering teams to strike an optimal balance between output quality, response latency, and operational cost across a spectrum of workloads.
"Small" is Relative: Still Demanding, but More Accessible
Despite its moniker, Inkling-Small is not a model designed for personal computers or standard developer workstations. The standard BF16 checkpoint requires a substantial 600 GB of aggregate GPU memory, with Thinking Machines recommending configurations such as 4x NVIDIA B300 GPUs or 8x NVIDIA H200 GPUs. For those seeking even greater efficiency, a quantized NVFP4 checkpoint reduces the requirement to approximately 180 GB of aggregate VRAM. This version can be deployed on a single NVIDIA B300 GPU in W4A4 mode or on two H200 GPUs in W4A16 mode.
These specifications effectively rule out typical laptops, MacBooks, desktop gaming PCs, and most developer workstations, which generally lack the necessary memory capacity. The practical deployment targets for Inkling-Small are therefore enterprise-grade GPU servers, cloud computing clusters, and specialized inference service providers. The "Small" designation is thus a relative term, signifying a significant reduction compared to the flagship Inkling, rather than a transition to consumer-grade hardware.
Nevertheless, the reduction is highly significant. A model that approaches the performance of a much larger system while demanding substantially less aggregate memory can lead to considerable savings in hosting costs, simplify capacity planning, and expand the pool of organizations capable of self-hosting such advanced AI capabilities. For businesses that prioritize data control, model behavior customization, and fine-tuning autonomy, this smaller footprint can be a more critical factor than achieving the absolute highest benchmark score. Moreover, its open-source nature ensures that rapid quantization efforts will likely emerge, making it even more accessible for resource-constrained environments, and potential blending with other models to further shrink its footprint for consumer-grade hardware.
Apache 2.0: The Gold Standard for Enterprise Open-Source AI
The licensing model of Inkling-Small, an Apache 2.0 release, is as strategically important as its performance metrics. Apache 2.0 is widely recognized as a gold standard for permissive open-source licenses in the software industry. It grants organizations broad latitude to use, modify, fine-tune, redistribute, and commercialize the model, including within proprietary products, provided they adhere to the license’s notice and attribution requirements. This level of legal flexibility significantly surpasses that offered by many custom "open" AI licenses, which often impose revenue thresholds, branding obligations, restrictive use cases, or separate conditions for large-scale commercial deployments.
This distinction is becoming increasingly pertinent as more AI companies release model weights without adopting conventional open-source licenses. For example, Chinese AI leader Moonshot recently made the weights of its Kimi K3 model available under a bespoke "open" license that includes additional commercial stipulations, diverging from the straightforward terms of Apache 2.0. For legal, procurement, and platform teams, the clarity and predictability of Apache 2.0 can materially simplify the adoption process. While it does not negate the need for due diligence concerning acceptable use policies, data provenance, regulatory compliance, and downstream safety obligations, it provides a solid foundation for building internal systems, launching commercial products, and maintaining customized versions of the model.
A Repeatable Model Development Pipeline: Thinking Machines’ Evolving Strategy
The swift release of Inkling-Small also highlights Thinking Machines’ maturation into a highly efficient and repeatable model development organization. Horace He, a researcher at Thinking Machines, contrasted the two launches on X, stating, "Whereas I felt like it took a village to release Inkling, Inkling-Small felt much more routine… We just took the pipeline used for Inkling, passed in a smaller model, and voila – new model! Inkling Small benefited quite a bit vs Inkling from some minor improvements, but there’s still so much more left in the tank…" This observation suggests that Thinking Machines is moving beyond treating each model release as a unique research project, instead building a robust and reusable pipeline encompassing pre-training, post-training, reinforcement learning, evaluation, and deployment.
The development of Inkling-Small incorporated several enhancements, including an improved pre-training data mix, adjustments to the machine-learning recipe, and on-policy distillation using the original Inkling as a teacher. The team further refined the model through two weeks of agentic coding reinforcement learning. Mira Murati echoed this sentiment in her own public remarks, emphasizing Inkling-Small’s comparable performance to Inkling at a quarter of the size and highlighting the immediate availability of its open and fine-tunable weights via Tinker.
Guidance for Enterprises and AI Builders Navigating Inkling-Small
Thinking Machines is providing comprehensive support for Inkling-Small’s deployment, distributing full BF16 and NVFP4 checkpoints and ensuring compatibility with popular inference frameworks such as SGLang, vLLM, TokenSpeed, Unsloth, and Hugging Face tooling. This multifaceted support ecosystem offers developers a variety of deployment pathways: leveraging an API, fine-tuning through the Tinker platform, engaging a third-party inference provider, or operating the model on private, on-premises infrastructure.
While Inkling-Small is not a model for casual individual download and local execution, it presents a compelling value proposition for businesses evaluating their AI strategy. It offers a strong compromise between cutting-edge performance and manageable resource requirements. For organizations weighing the benefits of a massive, flagship model against a more accessible open-weight system, Inkling-Small delivers nearly equivalent measured intelligence, superior performance on critical coding and reasoning tasks, reduced token pricing, a smaller hardware footprint, and a commercially permissive license.
The broader strategic implication of Inkling-Small’s release is perhaps the most significant. Thinking Machines is demonstrating that the success of Inkling was not a singular event. The company is rapidly iterating, compressing its model family, refining its training pipeline, and establishing a cadence for the routine production, improvement, and deployment of open-weight multimodal AI systems. This forward momentum signals a commitment to sustained innovation and accessibility in the rapidly evolving AI landscape.
