Artificial Superintelligence feels like a technology we are suppose to unlock in 200-300 years. As a species we are absolutely not ready for it. This feels like giving a medieval peasant access to weapons grade uranium. The Mathematics and algorithms behind reinforcement learning are not incredibly difficult to understand meaning any civilisation that has discovered matrix algebra, probability theory and calculus could achieve this with compute. What we have learnt is what we call intelligence is substrate independent and can be scaled dramatically with compute. The upper bounds of this absolutely no one knows. I think George Hotz was on the right line when he said there needs an entire new field of scientific enquiry that has predictive power on a models capability and predictions on any upper bounds to what we call intelligence if they even exist. ASI will destroy any notions we have of current modern society. Our world in the 2100's will look at us as if we were primitive man. Exponential curves means every year progress compresses achieving in 1 year what it would take 10-100 years. Human brain is not built to handle this environment.