Treepai g1 is our second model, a base for later fine-tuned models. This card is taken from Appendix A of the research report.
| Model | Treepai g1, a 517.8M-parameter decoder-only transformer. Base (pretrained) model only. |
| Developer | Treepai, Inc., October 2026. |
| Availability | Research model. Weights, tokenizer and training code are not released. |
| Intended use | Research at Treepai on training and serving efficiency; a base for later fine-tuned models. |
| Out of scope | Any user-facing use, factual question answering, advice of any kind, and languages other than English. |
| Training data | 10.0B tokens of English text, each used once: 9.3B from FineWeb-Edu (educational score of 3 or higher) and 0.7B of filtered public-domain fiction. 6,646 documents and 33 books that overlapped with the nine benchmarks removed. |
| Evaluation | Nine zero-shot capability benchmarks, held-out bits per byte, and the CrowS-Pairs bias probe (Section 6 of the report). |
| Known risks | States false facts fluently; reproduces stereotypes more than g0 on the one probe we ran; not checked for memorized personal information. |
| Compute | 87 GPU-hours on one RTX 4090; at most about 39 kWh at the GPU. |
Results in brief
- 49.6% on five core benchmarks, ahead of GPT-2 medium (44.9%) and Pythia-410M (47.0%), and behind Qwen2.5-0.5B (54.4%) and SmolLM2-360M (58.3%), which were trained with hundreds of times more data.
- Strongest on science questions. Its training text is narrow: on Wikipedia text it is behind every finished model of its size.
- Fine-tuned to sort customer-support messages, it is level with Qwen2.5-0.5B fine-tuned the same way (95.2% and 95.6% of categories right).
- In a same-engine speed test it was the slowest of the four models compared.
Full results, methods and limits: the report page and the PDF.