Add Learnzy hypothesis autoresearch loop#2245
Add Learnzy hypothesis autoresearch loop#2245notebook-learnzy wants to merge 2 commits intogit:masterfrom
Conversation
Adds two directories:
autoresearch/ — karpathy/autoresearch cloned as reference implementation.
An autonomous ML research loop where an AI agent modifies train.py,
trains a GPT for 5 minutes, records val_bpb, and keeps or resets the
change. Used as the architectural blueprint for learnzy-autoresearch.
learnzy-autoresearch/ — Learnzy's hypothesis validation loop, modeled
1:1 on autoresearch but for scientific literature instead of ML training:
- sources.py : fixed infrastructure — OpenAlex API client +
evaluate_evidence() metric (analog to prepare.py)
- hypothesis.py: mutable search queries per hypothesis link
(analog to train.py, agent modifies this each run)
- agent.py : fixed orchestrator — one experiment per execution
- program.md : agent instructions (analog to program.md)
- .github/workflows/research.yml: GitHub Actions cron loop (10 min)
The loop searches peer-reviewed literature (OpenAlex, 250M+ works)
for evidence supporting the Focus Score hypothesis:
A: HRV → cognition
B: Sleep → cognition
C: Cognition → academic grades
D1/D2/D3: Focus Score → depression / anxiety / insomnia
Each run: GPT-4o proposes new search queries → OpenAlex fetches papers
→ GPT-4o-mini extracts effect sizes and study quality → evidence_score
computed → hypothesis.py committed if improved, reset if not.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Welcome to GitGitGadgetHi @notebook-learnzy, and welcome to GitGitGadget, the GitHub App to send patch series to the Git mailing list from GitHub Pull Requests. Please make sure that either:
You can CC potential reviewers by adding a footer to the PR description with the following syntax: NOTE: DO NOT copy/paste your CC list from a previous GGG PR's description, Also, it is a good idea to review the commit messages one last time, as the Git project expects them in a quite specific form:
It is in general a good idea to await the automated test ("Checks") in this Pull Request before contributing the patches, e.g. to avoid trivial issues such as unportable code. Contributing the patchesBefore you can contribute the patches, your GitHub username needs to be added to the list of permitted users. Any already-permitted user can do that, by adding a comment to your PR of the form Both the person who commented An alternative is the channel Once on the list of permitted usernames, you can contribute the patches to the Git mailing list by adding a PR comment If you want to see what email(s) would be sent for a After you submit, GitGitGadget will respond with another comment that contains the link to the cover letter mail in the Git mailing list archive. Please make sure to monitor the discussion in that thread and to address comments and suggestions (while the comments and suggestions will be mirrored into the PR by GitGitGadget, you will still want to reply via mail). If you do not want to subscribe to the Git mailing list just to be able to respond to a mail, you can download the mbox from the Git mailing list archive (click the curl -g --user "<EMailAddress>:<Password>" \
--url "imaps://imap.gmail.com/INBOX" -T /path/to/raw.txtTo iterate on your change, i.e. send a revised patch or patch series, you will first want to (force-)push to the same branch. You probably also want to modify your Pull Request description (or title). It is a good idea to summarize the revision by adding something like this to the cover letter (read: by editing the first comment on the PR, i.e. the PR description): To send a new iteration, just add another PR comment with the contents: Need help?New contributors who want advice are encouraged to join git-mentoring@googlegroups.com, where volunteers who regularly contribute to Git are willing to answer newbie questions, give advice, or otherwise provide mentoring to interested contributors. You must join in order to post or view messages, but anyone can join. You may also be able to find help in real time in the developer IRC channel, |
|
There is an issue in commit 5e969dc:
|
Bug 1: Papers from OpenAlex were getting filtered to 0 after LLM extraction because n defaulted to 0 and the min_sample_size=20 filter removed everything when the OpenAI API call failed or returned n=0. Fix: initialise n with cited_by_count//5 (proxy) so papers survive the filter even if extraction doesn't produce an explicit sample size. Add verbose pre/post-extraction counts and better API error logging. Bug 2: ensure_branch() was switching to autoresearch/<date> branch, causing 'tip of branch is behind' push failures on every second run. Fix: remove branch switching entirely — commit directly to whatever branch Actions checked out (main), same approach as autoresearch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
There is an issue in commit 5e969dc:
|
|
There is an issue in commit dcdeb64:
|
Summary
autoresearch/— karpathy/autoresearch cloned as reference implementation: an autonomous ML research loop where an AI agent modifiestrain.py, trains a GPT for 5 minutes, recordsval_bpb, and keeps or resets the changelearnzy-autoresearch/— Learnzy's hypothesis validation loop, modeled 1:1 on autoresearch but for scientific literature instead of ML trainingevidence_scorecomputed →hypothesis.pycommitted if improved, reset if notTest plan
python sources.pylocally — confirm OpenAlex returns papers andevidence_scoreprintspython agent.pylocally withOPENAI_API_KEYset — confirm full loop completes in <5 minworkflow_dispatchonnotebook-learnzy/learnzy-autoresearch— confirm Actions run cleanresults.tsvafter first run — confirm per-link scores are populated🤖 Generated with Claude Code