Mistral and Google Release Domain-Specific Foundation Models

오픈소스 | Wed Jul 01 2026 00:00:00 GMT+0000 (Coordinated Universal Time) | 2 sources

Mistral and Google released domain-specific foundation models including Leanstral 1.5 for Lean 4 theorem proving and TabFM for tabular data.

Analysis

[Mistral AI] released Leanstral 1.5 formal proof model ^[1]

• Optimized for Lean 4-based automated theorem proving and autoformalization
• 119B total parameters with 6.5B active
• Supports 256k context
• provided for free
• Supports various features including Function Calling
• Structured Outputs
• and FIM

[Google Research] announced TabFM, a zero-shot foundation model for tabular data ^[2]

• Handles classification and regression workflows in a single forward pass
• Uses in-context learning (ICL)
• eliminating the need for hyperparameter tuning
• Available on Hugging Face and GitHub repositories
• Aims to replace traditional tree-based algorithms such as XGBoost and AdaBoost

[TabFM architecture] designed hybrid attention tailored to 2D orderless tabular structure ^[2]

• Captures feature interactions with alternating attention across rows and columns
• Row compression that condenses each row's information into a single dense vector
• Applies Transformer to compressed embeddings to reduce computational cost
• Pre-trained on hundreds of millions of synthetic datasets
• planned integration into BigQuery

Sources

• [1] Leanstral 1.5 - Hacker News
• [2] TabFM: A zero-shot foundation model for tabular data - Hacker News