Nome Sobrenome

AI researcher · XPTO

I study how neural networks learn — and how to make them more efficient, more robust and easier to evaluate.

Get in touch See publications

São Paulo, Brazil, UTC−3

descending…

Optimization landscape: a loss surface drawn with points of light and contour lines, where an amber point descends to the minimum. Illustrative.

About

A little about my path and what drives my research.

I'm an AI researcher at XPTO, where I study how deep learning models are trained, compressed and evaluated. I'm drawn to problems at the boundary between theory and practice: understanding the geometry of the loss to design optimizers and evaluation protocols that keep working outside the lab.

Before XPTO, I earned a PhD in computer science and worked with research groups in academia and industry. I publish at venues such as NeurIPS, ICML and ACL, review for these communities, and try to turn research results into tools other teams use every day.

Position
Machine learning research at XPTO, since 2021
Education
PhD in computer science
Languages
Portuguese, English and Spanish
Publications
10 papers, 2021–2026
CV (PDF)

Three lines in the third person, for events and the press.

Research

Four lines of work that feed one another: understanding how models learn helps train them at lower cost and evaluate them more honestly.

  • Optimization for deep learning

    Training dynamics and loss geometry, toward optimizers that converge faster and need less hyperparameter tuning.

    Keywords:

    • adaptive optimizers
    • loss geometry
    • learning rates
    • large-scale training
    See 4 publications

  • Representation learning

    How models organize what they learn: self-supervision, latent spaces and transfer across tasks and domains.

    Keywords:

    • self-supervision
    • contrastive learning
    • latent spaces
    • transfer learning
    See 2 publications

  • Efficient language models

    Distillation, quantization and careful use of context to bring large models to small compute budgets — Portuguese included.

    Keywords:

    • distillation
    • quantization
    • long context
    • efficient inference
    See 2 publications

  • Evaluation and robustness

    Metrics and protocols that measure what matters: calibration, uncertainty and behavior under distribution shift.

    Keywords:

    • calibration
    • uncertainty
    • distribution shift
    • benchmarks
    See 2 publications

Publications

Peer-reviewed papers and preprints, newest first.

Fictional publications, for demonstration only.Updated October 2026

  1. arXiv Preprint Optimization

    Sharp Routers, Flat Experts: Probing the Loss Landscape of Mixture-of-Experts Models

    Nome Sobrenome, Inessa Petrova, Kukua Mensah

    arXiv preprint, 2026

    PDF
    View BibTeX 
    @misc{sobrenome2026sharp,
  title  = {Sharp Routers, Flat Experts: Probing the Loss Landscape of Mixture-of-Experts Models},
  author = {Sobrenome, Nome and Petrova, Inessa and Mensah, Kukua},
  year   = {2026},
  url    = {https://example.org/papers/moe-landscape-2026.pdf},
  note   = {Preprint},
}

  2. NeurIPS Oral Optimization

    Descending Gently: Curvature-Aware Warmup for Large-Batch Training

    Nome Sobrenome, Lívia Tanaka, Munachiso Okafor

    Advances in Neural Information Processing Systems, 2025

    PDF Code
    View BibTeX 
    @inproceedings{sobrenome2025descending,
  title     = {Descending Gently: Curvature-Aware Warmup for Large-Batch Training},
  author    = {Sobrenome, Nome and Tanaka, L{\'\i}via and Okafor, Munachiso},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2025},
  url       = {https://example.org/papers/curvature-warmup-2025.pdf},
}

  3. TMLR Journal Optimization

    Step-Size Schedules as Control Policies: A Unified View of Warmup, Decay and Restarts

    Nome Sobrenome, Cyprien Moreau

    Transactions on Machine Learning Research, 2024

    PDF
    View BibTeX 
    @article{sobrenome2024schedules,
  title   = {Step-Size Schedules as Control Policies: A Unified View of Warmup, Decay and Restarts},
  author  = {Sobrenome, Nome and Moreau, Cyprien},
  journal = {Transactions on Machine Learning Research},
  year    = {2024},
  url     = {https://example.org/papers/schedules-control-2024.pdf},
}

  4. ACL Efficiency

    Distilling Long Context into Compact Students for Retrieval-Heavy Tasks

    Nome Sobrenome, Hilke Weber, Graziela Duarte, Lívia Tanaka

    Annual Meeting of the Association for Computational Linguistics, 2024

    PDF Code
    View BibTeX 
    @inproceedings{sobrenome2024distilling,
  title     = {Distilling Long Context into Compact Students for Retrieval-Heavy Tasks},
  author    = {Sobrenome, Nome and Weber, Hilke and Duarte, Graziela and Tanaka, L{\'\i}via},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year      = {2024},
  url       = {https://example.org/papers/long-context-distillation-2024.pdf},
}

  5. NeurIPS Workshop Workshop Evaluation

    Paraphrase Drift: Stress-Testing Language Model Evaluations with Controlled Rewrites

    Nome Sobrenome, Suhail Haddad

    Workshop at the Conference on Neural Information Processing Systems, 2024

    PDF
    View BibTeX 
    @inproceedings{sobrenome2024paraphrase,
  title     = {Paraphrase Drift: Stress-Testing Language Model Evaluations with Controlled Rewrites},
  author    = {Sobrenome, Nome and Haddad, Suhail},
  booktitle = {Workshop at the Conference on Neural Information Processing Systems},
  year      = {2024},
  url       = {https://example.org/papers/paraphrase-drift-2024.pdf},
}

  6. ICML Optimization

    Gradient Noise as a Thermometer: Estimating the Effective Temperature of SGD

    Anselmo Ribeiro, Nome Sobrenome, Jonna Lindqvist

    International Conference on Machine Learning, 2023

    PDF
    View BibTeX 
    @inproceedings{ribeiro2023gradient,
  title     = {Gradient Noise as a Thermometer: Estimating the Effective Temperature of {SGD}},
  author    = {Ribeiro, Anselmo and Sobrenome, Nome and Lindqvist, Jonna},
  booktitle = {International Conference on Machine Learning},
  year      = {2023},
  url       = {https://example.org/papers/gradient-noise-2023.pdf},
}

  7. EMNLP Efficiency

    Low-Bit, High Fidelity: Quantization-Aware Distillation for Portuguese Language Models

    Telma Brandão, Nome Sobrenome, Firmino Castro

    Conference on Empirical Methods in Natural Language Processing, 2023

    PDF Code
    View BibTeX 
    @inproceedings{brandao2023lowbit,
  title     = {Low-Bit, High Fidelity: Quantization-Aware Distillation for {Portuguese} Language Models},
  author    = {Brand{\~a}o, Telma and Sobrenome, Nome and Castro, Firmino},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year      = {2023},
  url       = {https://example.org/papers/low-bit-portuguese-2023.pdf},
}

  8. ICLR Spotlight Representations

    Contrastive Views from Weak Supervision: Learning Representations from Noisy Metadata

    Revathi Iyer, Nome Sobrenome, Eneja Novak, Suhail Haddad

    International Conference on Learning Representations, 2022

    PDF
    View BibTeX 
    @inproceedings{iyer2022contrastive,
  title     = {Contrastive Views from Weak Supervision: Learning Representations from Noisy Metadata},
  author    = {Iyer, Revathi and Sobrenome, Nome and Novak, Eneja and Haddad, Suhail},
  booktitle = {International Conference on Learning Representations},
  year      = {2022},
  url       = {https://example.org/papers/weak-contrastive-2022.pdf},
}

  9. JMLR Journal Evaluation

    Calibration Under Shift: A Benchmark and Protocol for Uncertainty in Deployed Classifiers

    Munachiso Okafor, Nome Sobrenome, Bunta Nakamura

    Journal of Machine Learning Research, 2022

    PDF Code
    View BibTeX 
    @article{okafor2022calibration,
  title   = {Calibration Under Shift: A Benchmark and Protocol for Uncertainty in Deployed Classifiers},
  author  = {Okafor, Munachiso and Sobrenome, Nome and Nakamura, Bunta},
  journal = {Journal of Machine Learning Research},
  year    = {2022},
  url     = {https://example.org/papers/calibration-shift-2022.pdf},
}

  10. CVPR Representations

    Agreeing on Parts: Patch-Level Consistency for Self-Supervised Dense Features

    Nome Sobrenome, Plácido Almeida, Dobromir Kowalski

    IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021

    PDF
    View BibTeX 
    @inproceedings{sobrenome2021agreeing,
  title     = {Agreeing on Parts: Patch-Level Consistency for Self-Supervised Dense Features},
  author    = {Sobrenome, Nome and Almeida, Pl{\'a}cido and Kowalski, Dobromir},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year      = {2021},
  url       = {https://example.org/papers/patch-consistency-2021.pdf},
}

Full list on Google Scholar All as BibTeX (.bib)

Get in touch

Research collaborations, invitations to speak or sit on committees, press or opportunities: write here or through the other channels. I reply personally.

I usually reply within three business days.

Other channels

Google Scholar
Citation profile
LinkedIn
in/nome-sobrenome
GitHub
nome-sobrenome
ORCID
0000-0000-0000-0000