The Optimizer Challenge

The Spaces focused on Tig’s new Optimizer Challenge, a non-convex optimization benchmark targeting the core algorithmic problem underpinning modern AI training. Host and John (Speaker 2) discussed why this is Tig’s most consequential challenge yet, tracing its roots to the highly cited 2014 Adam optimizer and explaining how a next‑generation optimizer could dramatically cut training costs, speed up model improvement, and reduce overfitting—while not alone guaranteeing AGI without a missing “mother of all datasets” of inventive know‑how. John outlined the challenge’s formulation (led by researcher David Lou with Ying and Dan Adams), the verification hurdles, and the broader incentive and IP landscape that favors open publication and Tig’s model over traditional patent monetization. They explored licensing dynamics if Tig produces a breakthrough (likely industry‑wide adoption with transparent, non‑discriminatory terms) and how revenues would feed a flywheel for further open innovation. The conversation also covered Tig’s state‑of‑the‑art progress (e.g., QKP; continuing improvements by Thibaut Vidal), the upcoming vector search advanced‑rewards vote and two‑tier rewards process, and John’s recent essay on capturing the missing dataset of researchers’ problem‑solving strategies. The session closed with housekeeping, raffle winners, and invitations to join future Wednesday community calls.

Tig Community Call: Optimizer Challenge Deep Dive

Who spoke

• Host (Tig community moderator): Facilitated the discussion, framed questions, and highlighted implications for Tig’s model and ecosystem.
• John: Lead voice on technical, strategic, and economic framing of the Optimizer Challenge; referenced his recent essays on the “mother of all datasets.”

The Optimizer Challenge (aka Non‑Convex Optimization)

What it is and why it matters

• Core idea: Training modern AI (especially neural networks) is a non‑convex optimization problem—finding minima on a high‑dimensional landscape with many local dips and peaks. Unlike convex optimization, you can’t just follow the gradient to a guaranteed global minimum.
• Naming choice: Internally discussed, but avoided calling it the “Neural Network Optimizer” to prevent confusion with routine weight fine‑tuning. “Optimizer Challenge” emphasizes broader generality: non‑convex optimization applies beyond neural nets.
• Historical anchor: The 2014 Adam optimizer paper (≈220k citations) is one of the most influential in modern AI; Adam represented a step‑change vs previous optimizers. Since then, there have been refinements within the “Adam family,” but no generational leap on par with 2014.
• Tig’s intent: Incentivize the next generational shift in non‑convex optimization—an innovation that could accelerate AI and, by extension, have civilization‑scale impact.

Origin and design of the challenge

• Lead designer: David Lou (ex‑Meta; G‑Research), supported by Ying and Dan Adams—AI researchers at Tig.
• Development effort: ≈5 months (about 4× longer than earlier challenges) due to complexities such as generating benchmark instances without leaking the “answer” (a non‑issue in some other challenge types).
• Prior Tig AI‑relevant challenges: Hypergraph partitioning (with Dr. Soma), Vehicle Routing (with team including Tibó Vidal), among others.

What a better optimizer could change

• Training cost and speed: A materially better optimizer could reduce compute and power requirements, allowing faster, cheaper training. Even 10×–100× improvements would be transformative for industry viability, though not sufficient alone to deliver AGI.
• Power infrastructure reality: Without algorithmic breakthroughs, electricity and capex constraints become binding (lead times for power generation—especially nuclear—are long; John noted training costs for frontier AGI‑class systems today would be in the tens of billions of dollars in electricity alone).
• Generalization vs overfitting: Improved optimization could reduce overfitting, leading to better generalization—especially critical for novel, out‑of‑distribution problems (e.g., scientific research) where prior data doesn’t provide a “shortcut.”
• Scope beyond AI: Non‑convex optimization improvements also benefit many other optimization domains outside neural networks.

Will a breakthrough alone deliver AGI?

• John’s stance: A next‑gen optimizer would be a powerful accelerant but not a sufficient condition. He argues we also need a missing dataset—the “mother of all datasets”—to unlock genuine inventive reasoning.

The “Mother of All Datasets”: Inventive Know‑How

What’s missing from today’s AI training data

• Literature ≠ real research process: Papers present a clean, linear path from assumptions to results. Actual discovery is messy: getting stuck, reframing, abandoning paths, trying alternatives—the tacit “unsticking” strategies are largely absent from the written record.
• Consequence: Models trained on literature often “answer first, justify later,” mimicking paper structure rather than authentic problem‑solving. They lack procedural know‑how for working beyond the current state of the art.

Where this knowledge resides and why it’s hard to obtain

• Location: In the heads of many researchers—distributed across a long tail, not just the top percentile. Non‑consensus strategies from “non‑obvious” contributors often unlock progress.
• Centralized capture is impractical: No single lab or company can hire enough people or access this distributed tacit knowledge at scale.

Why Tig is positioned to help capture it

• Necessarily decentralized: Scale and dispersion of know‑how require a system that can involve thousands of contributors with proper incentives.
• Incentive design: Codifying tacit strategies is laborious; a mechanism that rewards and re‑rewards contributions is needed. John suggests Tig’s model will be a key piece of a broader strategy (detailed in his essays and upcoming follow‑ups) to elicit, validate, and utilize this dataset for algorithmic research.

Path to Impact and Licensing Dynamics

If Tig produces a state‑of‑the‑art optimizer

• Indispensability: Even a 2× efficiency gain in training would create overwhelming economic pressure for all AI companies to adopt the new optimizer; historic leaps (e.g., transformer‑era advances) have been far larger.
• Market behavior: In practice, all major players would need to license it. Pricing must balance maximizing reinvestment into open innovation (Tig’s flywheel) with avoiding undue friction that slows industry progress.
• Continuous lead: Open, high‑velocity improvement can yield a sustainable lead. The host cited QKP (Quadratic Knapsack Problem) becoming state of the art in July and being further improved with Tibó Vidal’s contributions. This agility positions Tig to maintain ongoing superiority if a breakthrough optimizer emerges.
• Open monopoly analogy: John noted how open projects can become unassailable through pace and participation (he cited Linux as a pattern for an open, non‑discriminatory regime and explained Tig’s licensing would follow set, non‑arbitrary terms).

Why inventors would submit to Tig rather than patent privately

• Enforcement reality in tech: Individual or academic inventors face severe asymmetry enforcing patents against large companies; norms, litigation costs, and timelines make direct licensing monetization rare.
• How big tech uses patents: Often as defensive portfolios and non‑aggression pacts rather than direct royalty extraction among competitors.
• Academic incentives: Open publication, citations, grants, reputational capital, and career advancement predominate; many landmark algorithms (e.g., Adam) were published without patents.
• Google’s exception: John discussed Google’s research as analogous to AT&T’s Bell Labs—strategically valuable for monopoly defense via public‑interest research, not direct commercialization of every output.
• Tig’s advantage for researchers:
  • Monetization without becoming an entrepreneur (avoids years of legal/operational overhead and the necessity to leave academia to run a company full‑time).
  • Transparent, open publication and community validation.
  • Revenues flow back into open innovation rewards—visible alignment of incentives with scientific progress.
• Net effect: For many researchers, contributing to Tig is a Pareto improvement vs the status quo; there’s effectively no comparable alternative platform (“no competitors”)—awareness is the main limiter.

Current Tig Progress and Pipeline

• Confirmed state of the art: QKP in July; reportedly already improved since initial SOTA confirmation.
• Strong performers: Vehicle Routing looks very promising (watch for future updates).
• Vector Search: Moving to an “Advanced Rewards” vote.
  • Two‑tier innovation rewards: Standard (implementation improvements) vs Advanced (patent‑grade, novel algorithmic strategies with unexpected results).
  • Process: Public scrutiny period (1 week, starting next week) followed by a token‑holder vote the week after; forum debate informs the decision.
• Teased but undisclosed: John expects another challenge may be approaching a state‑of‑the‑art advance, pending full testing.

Philosophical and contextual notes

• “Found” vs “created”: John prefers “discovered” for elegant mathematical structures (e.g., Adam) that feel uncovered rather than invented—akin to discovering constants or deep patterns (pi, prime number structure).
• Power trajectory: The host noted industry‑wide power and spend have been compounding at unsustainable rates (~400% YoY by his reference), underscoring urgency to prioritize algorithmic efficiency over brute‑force scaling.
• Essays and outreach: John’s recent X article (pinned) explains the “mother of all datasets” and will be followed by practical plans to codify and use it. The host recommended John’s Baseline interview as a concise “why Tig matters” explainer.

Administrative and community notes

• Calls: Weekly Tig community call, typically Wednesdays around 5pm UK; Spaces are recorded.
• Raffle winners this session: Mr Max and Tommy (instructions given to claim Tig rewards).

Key takeaways

• The Optimizer Challenge targets the most central algorithmic bottleneck in modern AI: non‑convex optimization.
• A generational leap in optimizers would drastically cut training costs/power and improve generalization, accelerating progress but not alone guaranteeing AGI.
• John argues a missing, decentralized “inventive know‑how” dataset is necessary to reach truly inventive AI; Tig’s model is an essential component to capture and operationalize it.
• If Tig delivers the next‑gen optimizer, its adoption would be industry‑wide and licensing‑driven, with revenue recycling into open innovation—potentially establishing a durable, open, de facto standard.
• Tig offers researchers a superior route to impact and monetization without leaving academia for entrepreneurship, addressing a longstanding market failure in scientific research incentives.
• Near‑term: Watch the Vector Search Advanced Rewards process and further SOTA announcements; follow John’s essays for the unfolding plan on the “mother of all datasets.”