i haven't heard it dicussed yet but AI basically killed hackathons. pretty much anything you could possibly make at a hackathon in 2019 can be built better and faster by AI in 2025

this is bad code right?

probably 10x more people should be working on prompt optimization systems (we need a vLLM for promptopt), theory, new techniques, benchmarks. the whole kit and caboodle

here's some free alpha: if we do RL for too long after pretraining, we will surely overwrite parameters and start to forget things in the original instructGPT paper, their best model mixed RLHF with pretraining gradients to avoid exactly this model drift issue yet no one is doing this anymore. sure, it's one particular instantiation (gradient mixing) of a broader idea (avoiding forgetting) but seems like a greatly-overlooked line of thinking as we do more and more steps of RL for example see the recent ProRL paper. they're doing over 1000 steps of GRPO now with a non-trivial learning rate and no penalty for deviating from the original model. the circuits built inside the model during pretraining are surely starting to decay. and if not, they will after 10k or 100k RL steps i suspect this idea will come back around eventually; they're probably already doing this at the big labs

i'm looking for good examples of reasoning model generalization for example, a model incentivized via RL to think for a while and solve math problems gets better at creative writing is this common?

this seems really important: it is totally plausible that a model could get IMO gold without *any* reinforcement learning, given a perfectly-crafted prompt we just don't know, and lack tools to efficiently search through prompt space. glad to see at least someone is trying

you can't make this stuff up

hypothetical situation - i am an AI company that's reduced the cost of transferring and storing models to zero. i can serve each user their own model with no overhead what do i do? directly SFT user-specific models on their data? or RLHF on the chat ratings? something else?

the human brain reserves 40% of its processing exclusively for vision. modern LLMs somehow evolved without this entirely

This one dude I know (allegedly) poisoned the global AI training data corpus with self-propagating trigger-activated sleeper jailbreak payloads