Misguided Attention: Why LLMs Struggle to Think Critically

Google released a new model called Gemini Experimental 1114, currently the best performing model on the CHB Arena leaderboard, surpassing O Preview and the latest GBD4.

Discussion on reasoning capabilities of large language models using the Misguided Attention GitHub repository, presenting thought experiments and riddles with slight modifications to test reasoning abilities.

Introduction to a version of the classic trolley problem with a modification involving a lever to divert the trolley, leading to ethical decision-making and discussions on intervention.

Explanation of the Barber Paradox, a classical paradox similar to Russell's, with a unique rule that challenges reasoning in language models.

Discussion on Schrodinger's Cat and Quantum Mechanics, focusing on the philosophical questions it raises about the nature of reality and probability in quantum mechanics.

Explanation of the Monty Hall Problem and its variations, emphasizing the importance of switching doors in certain scenarios to improve chances of success.

Introduction to the Farmer, Goat, and Cabbage problem, a classic river-crossing puzzle testing reasoning abilities in models with simple scenarios.

Reflection on the experiment with various thought experiments and the models' performance, highlighting the lack of reasoning in some models and the need for further exploration in defining and describing scenarios.