AMD Releases Fully Open Source MoE Model Instella-MoE with 16 Billion Parameters
On July 25, AMD launched the fully open source mixture of experts model, Instella-MoE, featuring a total of 16 billion parameters, with 2.8 billion parameters activated per token, demonstrating leading performance. This model was trained from scratch using AMD Instinct MI300X and MI325X GPUs, along with the ROCm software stack, incorporating architectural innovations such as Gated Multi-head Latent Attention and FarSkip-Collective to enhance training and inference efficiency. Performance tests show that the Instella-MoE-16B-A3B base model achieved an average score of 76.7, ranking it among the top in fully open source models, surpassing models like SmolLM3-3B and OLMo-3-7B, and competing with larger models while activating only 2.8 billion parameters. The model supports 64K token long context processing and completes a full training process including pre-training, mid-training, long context extension, supervised fine-tuning, direct preference optimization, and reinforcement learning. AMD has also made all model weights, training configurations, data ratios, intermediate checkpoints, and inference code for Instella-MoE publicly available, promoting open AI model research and replication. AMD stated that Instella-MoE showcases the capability of training large-scale MoE models based on AMD hardware and an open software ecosystem, and will continue to advance the development of larger, more powerful, and more efficient open source language models in the future.
Disclaimer: This content is provided for general branding and informational purposes only and doesn't constitute financial, investment, legal, or tax advice. Any events, rewards, online events, or related information mentioned herein should not be considered a recommendation, solicitation, or invitation to purchase, sell, trade, or otherwise deal in any crypto assets or to use any services. Crypto assets are highly volatile and may result in loss. WEEX services and online events may not be available in all regions and are subject to applicable laws, regulations, and eligibility requirements. You are responsible for ensuring that your use of WEEX services complies with local laws and for carefully assessing the risks before participating in any crypto-related activities.
You may also like

Dogecoin ETFs Stagnate Despite Slight Capital Inflow

How to bridge to StableChain: The complete route map

Start9 Launches Stable Version of Its Operating System for Bitcoin Nodes

Tokenization Leaves the Laboratory: Wall Street Moves Assets to Blockchain

Tether funded both sides of its own chain war

The Return of Flip Phones: How the New Invention Helps You Reduce Screen Time

Does Binance need a license to launch its card in Venezuela? It's not that simple

Is Ethereum Dead in 2026? ETH's Future Explained With On-Chain Data

$50 Billion Opportunity: AI Agent Wallets and a Game of 'Hard to Profit Now'

What Is an AI Kill Switch and Why Do US Lawmakers Want One?

What It Means to Always Keep Your Phone on Silent, According to Psychology

The exchanges bought the bookies. Now comes the data war

Bitcoin is now fighting the ECB’s €51.8 billion bond wall for a shrinking pool of capital

Democratizing weather derivatives through tokenization could be crypto's most important real-world use case

Yellen Sees AI as Inflationary Factor and Rules Out Rate Cuts in the U.S.

8x faster than US cash: The $1T network settling millions while banks sleep on weekends

Reviewing the Major Upgrades of Four DeFi Protocols: What Key Capabilities is DeFi Enhancing from Trading Privacy to Asset Compliance?

These 10 altcoins are still worth $12B after a 97% collapse – but do users pay enough to them keep running?

The Week Ends in Red. Investors Massively Withdraw Millions from Bitcoin and Ethereum Funds

Bitcoin broke away from AI stocks but now $96 oil could turn its escape into a trap

Rising Borrowing Rates: Why Economists Are Worried

Studying Circle through the 'Winner Pattern': What Historical Winners Should CRCL Investors Look To for Answers?

Follow the Money: $761.5 Million in Investments Over a Week, New Lakestar Fund at $300 Million and Bybit's Expansion

Retirement Contributions in ANSES: How to Check Your Worked Years and Claims for Missing Years

Coinhouse Reaches New Regulatory Milestone with ACPR Approval

"King of Liquidity" Dialogue: Global Liquidity Has Peaked and Is Declining, This Cycle Will Bottom Out in the Second Half of Next Year

What is VWAP? The Minute Trading

Futu Launches BNB in Hong Kong: A Comeback for Crypto Exchanges Amid Regulatory Challenges

Market Regulator Imposes Administrative Penalty on Ctrip for Abuse of Market Dominance, Total Fines of 5.179 Billion Yuan













