Tanishq Mathew Abraham, Ph.D.

Rewriting Pre-Training Data Boosts LLM Performance in Math and Code

Introduces two openly licensed datasets:
1. SwallowCode (≈16.1 billion tokens) refines Python snippets from The-Stack-v2
2. SwallowMath (≈2.3 billion tokens) enhances Finemath-4+ by removing boilerplate,

Rewriting Pre-Training Data Boosts LLM Performance in Math and Code

Introduces two openly licensed datasets:
1. SwallowCode (≈16.1 billion tokens) refines Python snippets from The-Stack-v2
2. SwallowMath (≈2.3 billion tokens) enhances Finemath-4+ by removing boilerplate,