{"id":6575,"date":"2026-10-23T07:18:55","date_gmt":"2026-10-23T07:18:55","guid":{"rendered":"https:\/\/i10xnews.kinsta.cloud\/?p=6575"},"modified":"2026-10-23T07:18:55","modified_gmt":"2026-10-23T07:18:55","slug":"llm-peer-review-top-journals-harsh-inconsistent","status":"publish","type":"post","link":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent","title":{"rendered":"LLM Reviewers Send 76% of Published Top-Journal Papers Back, Yet Miss Most Planted Flaws"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">\u26a1 Quick Take<\/h2>\n\n\n<ul class=\"wp-block-list\"><li><strong>Harsh:<\/strong> Asked to review 200 papers already published in 2026 in journals such as Nature, Science and Cell, four LLMs recommended major revision or rejection in 605 of 800 reviews (75.6%).<\/li><li><strong>Inconsistent:<\/strong> The rate ranged from 17.5% for Gemini 3.7 Flash to 100% for GPT-5.6 Sol, and one framing sentence saying the paper was a revision raised Gemini&#8217;s accept rate from 15.5% to 68.5%.<\/li><li><strong>Blind to real flaws:<\/strong> When one serious flaw was planted in each paper, the models caught it in only 17.0% to 60.5% of cases, and their recommendations barely changed.<\/li><\/ul>\n\n\n<p class=\"wp-block-paragraph\">Journals and conferences are drowning in submissions, and LLM reviewers are increasingly proposed as relief. A preprint from Bastien Le Guellec and colleagues at CHU Lille and the Technical University of Munich tests them against the closest thing science has to a gold standard: articles that have just passed peer review at top journals. The models were not told the papers were already published. Their verdicts say more about the model and the prompt than about the paper.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd2c What the study shows<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Reviewer model (via OpenRouter)<\/th><th>Major revision or reject (published papers)<\/th><th>Caught the planted flaw<\/th><\/tr><\/thead><tbody><tr><td>GPT-5.6 Sol<\/td><td>200\/200 (100%)<\/td><td><strong>121\/200 (60.5%)<\/strong><\/td><\/tr><tr><td>GLM-5.3 Flash<\/td><td>180\/200 (90.0%)<\/td><td>88\/200 (44.0%)<\/td><\/tr><tr><td>DeepSeek v4 Flash<\/td><td>190\/200 (95.0%)<\/td><td>34\/200 (17.0%)<\/td><\/tr><tr><td>Gemini 3.7 Flash<\/td><td>35\/200 (17.5%)<\/td><td>36\/200 (18.0%)<\/td><\/tr><\/tbody><\/table><figcaption class=\"wp-element-caption\">Recommendations on 200 unaltered published articles (Experiment 1) and detection of one deliberately planted flaw (Experiment 2). Source: Le Guellec et al. (2026), Sections 3.1 and 3.3.<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1344\" height=\"768\" src=\"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/10\/chart-reviewers.webp\" alt=\"Grouped bar chart per LLM reviewer: share of published papers sent to major revision or rejection (GPT-5.6 Sol 100, DeepSeek v4 Flash 95, GLM-5.3 Flash 90, Gemini 3.7 Flash 17.5 percent) next to share of planted flaws caught (60.5, 17, 44 and 18 percent).\" class=\"wp-image-6574\" srcset=\"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/chart-reviewers.webp 1344w, https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/chart-reviewers-300x171.webp 300w, https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/chart-reviewers-1024x585.webp 1024w, https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/chart-reviewers-768x439.webp 768w\" sizes=\"auto, (max-width: 1344px) 100vw, 1344px\" \/><figcaption class=\"wp-element-caption\">Harsh on sound papers, weak on real flaws. i10X chart based on data from Le Guellec et al. (2026), arXiv:2610.31414, Sections 3.1 and 3.3.<\/figcaption><\/figure>\n\n\n<ul class=\"wp-block-list\"><li><strong>Almost no agreement:<\/strong> Only 21 of 200 papers (10.5%) got the same decision from all four models, and every one of those was a major revision. No paper was unanimously accepted.<\/li><li><strong>Prestige made no difference:<\/strong> Decisions did not differ by journal (p = 0.24). Papers from Nature, Science and Cell scored 1.74 on the four-point severity scale, Scientific Reports 1.98 (p = 0.081).<\/li><li><strong>One sentence of framing:<\/strong> Telling the models the paper had already been through one round of review made all four more lenient. Mean severity fell from 1.77 to 1.26; 392 of 800 reviews became milder and 6 harsher. Accept rose from 3.9% to 17.8%, reject fell from 5.4% to 0.9%.<\/li><li><strong>Flaws found, verdict unchanged:<\/strong> Planting a severe statistical or methodological flaw raised mean severity by just 0.02 of a category. Even where the flaw was caught, severity rose only 0.07 and the recommendation stayed the same in 84.2% of reviews.<\/li><li><strong>One clear winner on quality:<\/strong> Two outside judges named GPT-5.6 Sol&#8217;s review the best of four in 94.5% (Grok 4.6) and 75.5% (MiniMax M3) of papers. Reviews from Gemini 3.7 Flash and DeepSeek v4 Flash were never picked.<\/li><\/ul>\n\n\n<h2 class=\"wp-block-heading\">\u2705 Why it matters: specificity and sensitivity in one design<\/h2>\n\n\n<ul class=\"wp-block-list\"><li><strong>Both error types measured:<\/strong> Published papers test whether models wrongly reject sound work (low specificity); the planted flaws test whether they miss real problems (low sensitivity). The models fail on both counts.<\/li><li><strong>Realistic, broad corpus:<\/strong> 200 articles, ten from each of 20 general-science, life-science and clinical journals, all published in 2026 under CC BY 4.0, with at least 2,000 words of body text.<\/li><li><strong>Controlled framing test:<\/strong> The two prompts differed only in their first sentence, which isolates how much the verdict depends on context rather than content.<\/li><li><strong>Off-panel judging:<\/strong> Flaws were planted by Grok 4.6, detection was judged by Claude Opus 5, and review quality by Grok 4.6 and MiniMax M3, none of which were reviewers. Partial detections were reported separately: counting them as catches would raise the range to 29.5% to 84.0%.<\/li><\/ul>\n\n\n<h2 class=\"wp-block-heading\">\ud83e\uddea Method in brief<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reviewers:<\/strong> GLM-5.3 Flash, GPT-5.6 Sol, Gemini 3.7 Flash and DeepSeek v4 Flash via the OpenRouter API, chosen for a top-50 leaderboard rank and a price below $5 per million output tokens; two proprietary, two open-weight. <strong>Protocol:<\/strong> the prompt named the paper&#8217;s real target journal and withheld that it was published; each model reviewed each paper twice, as a first submission and as a revision. <strong>Experiment 2:<\/strong> one flaw per paper, statistical (for example clustered data treated as independent) or methodological, inserted as a targeted text substitution and designed to warrant major revision or rejection on its own. <strong>Experiment 3:<\/strong> the four reviews per paper, relabelled and shuffled, rated by two judge models for best review, relevance, constructiveness and nitpicking. Decisions were analysed as a four-point severity scale with non-parametric tests.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u2753 What remains open<\/h2>\n\n\n<ul class=\"wp-block-list\"><li><strong>Published is not the same as sound:<\/strong> The design assumes the journals&#8217; editorial work is a reasonable basis of trust. Some of the models&#8217; criticisms may be valid; no human expert assessed the reviews.<\/li><li><strong>Cheaper models only:<\/strong> For cost and carbon reasons the authors did not test the largest or most expensive models, so frontier reviewers may perform differently.<\/li><li><strong>LLMs judging LLMs:<\/strong> Flaw detection and review quality were scored by other models. The two quality judges agreed on the best review in 79.0% of papers, but with a low Cohen&#8217;s \u03ba of 0.23.<\/li><li><strong>Possible contamination and no data yet:<\/strong> Some articles may have appeared earlier as preprints and been seen in training. The data is to be deposited in a public repository before journal submission; it is not available yet. The preprint is not peer reviewed.<\/li><\/ul>\n\n\n<h2 class=\"wp-block-heading\">\ud83e\udded Context: AI review is already here<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The paper notes that AI review is no longer hypothetical: a 2025 Frontiers survey found that 53% of reviewers already use AI, every main-track submission to AAAI-26 (22,977 papers) received an AI-generated review, and NEJM AI offers an accelerated human-plus-AI review track. Kim and colleagues have already reviewed the quality of AI reviews of Nature-family papers; this study focuses on the recommendations themselves. It joins i10X coverage of AI&#8217;s growing role in research, from <a href=\"\/news\/openai-722-ai-generated-mathematics-manuscripts\">AI-generated mathematics manuscripts<\/a> to the <a href=\"\/news\/ai-scientific-research-monocultures-risks\">risk of scientific monocultures<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udee0\ufe0f What it means for editors, reviewers and authors<\/h2>\n\n\n<ul class=\"wp-block-list\"><li><strong>Editors:<\/strong> Do not treat an LLM&#8217;s accept or reject recommendation as a signal. It depends heavily on which model is used and how the request is framed.<\/li><li><strong>Reviewers:<\/strong> LLM reports can surface concerns, especially ones several models raise independently, but a clean LLM review is no evidence that a paper has no serious flaw.<\/li><li><strong>Authors:<\/strong> A harsh &#8220;major revision&#8221; from a chatbot before submission says little; the same text framed differently may get a much milder verdict.<\/li><\/ul>\n\n\n<p class=\"wp-block-paragraph\">More on the providers behind the tested reviewers: <a href=\"\/news\/company\/open-ai\">OpenAI<\/a>, <a href=\"\/news\/company\/google\">Google<\/a> and <a href=\"\/news\/company\/deepseek\">DeepSeek<\/a>, or browse <a href=\"\/news\/category\/society-economy\">society and economy<\/a> research.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u270d\ufe0f About the analysis<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This article is based on the full text of the preprint &#8220;Major Revision, Minor Accountability: LLMs as Reviewers of Published Articles in Top-Tier Journals&#8221; (arXiv:2610.31414v1, submitted October 22, 2026). All numbers come from the paper; i10X did not rerun the experiments. The authors report no dedicated funding and no competing interests, and state that an AI agent (Claude) carried out corpus retrieval, experiments, statistical analysis, figures and drafting under the corresponding author&#8217;s direction and review. The paper is licensed CC BY-NC-SA 4.0; no original figures are reproduced, and the chart was redrawn by i10X from the reported numbers.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd2d i10x Perspective<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The most useful finding here is not that LLMs are harsh, but that their verdicts are arbitrary: the same paper gets accept or major revision depending on the model and one sentence of framing, while a planted fatal flaw barely moves the needle. That makes the recommendation the least trustworthy part of an AI review. The concerns themselves, especially those several models agree on, look more promising. Publishers rolling out AI review should design around that split, and test with planted flaws before trusting any model&#8217;s verdict.<\/p>\n\n","protected":false},"excerpt":{"rendered":"<p>Four LLMs reviewed 200 papers already published in journals like Nature and Cell: 75.6% of reviews demanded major revision or rejection, while planted flaws were caught 17% to 61% of the time.<\/p>\n","protected":false},"author":2,"featured_media":6573,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"paper_title":"Major Revision, Minor Accountability: LLMs as Reviewers of Published Articles in Top-Tier Journals","paper_url":"https:\/\/arxiv.org\/abs\/2610.31414","pdf_url":"https:\/\/arxiv.org\/pdf\/2610.31414v1","doi":"","arxiv_id":"2610.31414","authors":"Bastien Le Guellec, Su-Hwan Kim, Raphael Bentegeac, Aghiles Hamroun","institutions":"CHU Lille; Technical University of Munich","paper_date":"2026-10-22","venue":"arXiv (cs.DL), version 1","peer_reviewed":false,"code_url":"","data_url":"","funding_coi":"No dedicated funding; authors declare no competing interests. Claude used as AI research, coding and drafting assistant.","figure_license":"No original figures reused; chart redrawn by i10X from reported numbers","paper_license":"CC BY-NC-SA 4.0","replication":"no","study_models":"Reviewers: GPT-5.6 Sol, GLM-5.3 Flash, Gemini 3.7 Flash, DeepSeek v4 Flash (OpenRouter); flaw injection: Grok 4.6; detection judge: Claude Opus 5; quality judges: Grok 4.6, MiniMax M3","study_sample":"200 CC BY 4.0 articles published in 2026, ten from each of 20 journals; 800 reviews per experiment; two framings; one planted flaw per article","footnotes":"","_yoast_wpseo_primary_category":""},"categories":[59,52],"tags":[],"companies":[46,37,38],"products":[49],"class_list":["post-6575","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-society-economy","category-studies","company-deepseek","company-google","company-open-ai","product-gemini"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.7 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>LLM Peer Review: Harsh on Top Papers, Blind to Flaws - i10X News<\/title>\n<meta name=\"description\" content=\"Four LLMs reviewed 200 papers published in top journals: 75.6% of reviews demanded major revision or rejection, yet most planted flaws went unnoticed.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"LLM Peer Review: Harsh on Top Papers, Blind to Flaws - i10X News\" \/>\n<meta property=\"og:description\" content=\"Four LLMs reviewed 200 papers published in top journals: 75.6% of reviews demanded major revision or rejection, yet most planted flaws went unnoticed.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent\" \/>\n<meta property=\"og:site_name\" content=\"i10X News\" \/>\n<meta property=\"article:published_time\" content=\"2026-10-23T07:18:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/llm-peer-review-top-journals-harsh-inconsistent-title.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1344\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Christopher Ort\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@i10x_ai\" \/>\n<meta name=\"twitter:site\" content=\"@i10x_ai\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Christopher Ort\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"NewsArticle\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent\"},\"author\":{\"name\":\"Christopher Ort\",\"@id\":\"https:\\\/\\\/news.i10x.ai\\\/#\\\/schema\\\/person\\\/d410bb7ee72febefca2c9c369939219a\"},\"headline\":\"LLM Reviewers Send 76% of Published Top-Journal Papers Back, Yet Miss Most Planted Flaws\",\"datePublished\":\"2026-10-23T07:18:55+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent\"},\"wordCount\":1196,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/news.i10x.ai\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/llm-peer-review-top-journals-harsh-inconsistent-title.webp\",\"articleSection\":[\"Society &amp; Economy\",\"Studies\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent#respond\"]}],\"dateModified\":\"2026-10-23T07:18:55+00:00\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent\",\"name\":\"LLM Peer Review: Harsh on Top Papers, Blind to Flaws - i10X News\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/news.i10x.ai\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/llm-peer-review-top-journals-harsh-inconsistent-title.webp\",\"datePublished\":\"2026-10-23T07:18:55+00:00\",\"description\":\"Four LLMs reviewed 200 papers published in top journals: 75.6% of reviews demanded major revision or rejection, yet most planted flaws went unnoticed.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent#primaryimage\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/llm-peer-review-top-journals-harsh-inconsistent-title.webp\",\"contentUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/llm-peer-review-top-journals-harsh-inconsistent-title.webp\",\"width\":1344,\"height\":768,\"caption\":\"Illustration: i10X\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/llm-peer-review-top-journals-harsh-inconsistent#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/news.i10x.ai\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"LLM Reviewers Send 76% of Published Top-Journal Papers Back, Yet Miss Most Planted Flaws\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/news.i10x.ai\\\/#website\",\"url\":\"https:\\\/\\\/news.i10x.ai\\\/\",\"name\":\"i10X News\",\"description\":\"Stay updated with the latest news and announcements from i10X.\",\"publisher\":{\"@id\":\"https:\\\/\\\/news.i10x.ai\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/news.i10x.ai\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/news.i10x.ai\\\/#organization\",\"name\":\"i10X\",\"url\":\"https:\\\/\\\/news.i10x.ai\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/news.i10x.ai\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/i10x-logo-180.png\",\"contentUrl\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/i10x-logo-180.png\",\"width\":180,\"height\":180,\"caption\":\"i10X\"},\"image\":{\"@id\":\"https:\\\/\\\/news.i10x.ai\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/x.com\\\/i10x_ai\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/i10x-ai\",\"https:\\\/\\\/instagram.com\\\/i10x.ai\",\"https:\\\/\\\/www.youtube.com\\\/@i10X_ai_official\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/news.i10x.ai\\\/#\\\/schema\\\/person\\\/d410bb7ee72febefca2c9c369939219a\",\"name\":\"Christopher Ort\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/christopher-ort-192.webp\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/christopher-ort-192.webp\",\"contentUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/christopher-ort-192.webp\",\"caption\":\"Christopher Ort\"},\"description\":\"Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU). He is the author of \u201cThe Style Penalty\u201d, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/author\\\/christopher-ort\",\"sameAs\":[\"https:\\\/\\\/i10x.ai\\\/blog\\\/author\\\/christopher-ort\"],\"worksFor\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#organization\"},\"alumniOf\":{\"@type\":\"EducationalOrganization\",\"name\":\"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU)\",\"url\":\"https:\\\/\\\/www.fau.eu\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"LLM Peer Review: Harsh on Top Papers, Blind to Flaws - i10X News","description":"Four LLMs reviewed 200 papers published in top journals: 75.6% of reviews demanded major revision or rejection, yet most planted flaws went unnoticed.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent","og_locale":"en_US","og_type":"article","og_title":"LLM Peer Review: Harsh on Top Papers, Blind to Flaws - i10X News","og_description":"Four LLMs reviewed 200 papers published in top journals: 75.6% of reviews demanded major revision or rejection, yet most planted flaws went unnoticed.","og_url":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent","og_site_name":"i10X News","article_published_time":"2026-10-23T07:18:55+00:00","og_image":[{"width":1344,"height":768,"url":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/llm-peer-review-top-journals-harsh-inconsistent-title.webp","type":"image\/webp"}],"author":"Christopher Ort","twitter_card":"summary_large_image","twitter_creator":"@i10x_ai","twitter_site":"@i10x_ai","twitter_misc":{"Written by":"Christopher Ort","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent#article","isPartOf":{"@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent"},"author":{"name":"Christopher Ort","@id":"https:\/\/news.i10x.ai\/#\/schema\/person\/d410bb7ee72febefca2c9c369939219a"},"headline":"LLM Reviewers Send 76% of Published Top-Journal Papers Back, Yet Miss Most Planted Flaws","datePublished":"2026-10-23T07:18:55+00:00","mainEntityOfPage":{"@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent"},"wordCount":1196,"commentCount":0,"publisher":{"@id":"https:\/\/news.i10x.ai\/#organization"},"image":{"@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent#primaryimage"},"thumbnailUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/llm-peer-review-top-journals-harsh-inconsistent-title.webp","articleSection":["Society &amp; Economy","Studies"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent#respond"]}],"dateModified":"2026-10-23T07:18:55+00:00"},{"@type":"WebPage","@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent","url":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent","name":"LLM Peer Review: Harsh on Top Papers, Blind to Flaws - i10X News","isPartOf":{"@id":"https:\/\/news.i10x.ai\/#website"},"primaryImageOfPage":{"@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent#primaryimage"},"image":{"@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent#primaryimage"},"thumbnailUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/llm-peer-review-top-journals-harsh-inconsistent-title.webp","datePublished":"2026-10-23T07:18:55+00:00","description":"Four LLMs reviewed 200 papers published in top journals: 75.6% of reviews demanded major revision or rejection, yet most planted flaws went unnoticed.","breadcrumb":{"@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent#primaryimage","url":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/llm-peer-review-top-journals-harsh-inconsistent-title.webp","contentUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/llm-peer-review-top-journals-harsh-inconsistent-title.webp","width":1344,"height":768,"caption":"Illustration: i10X"},{"@type":"BreadcrumbList","@id":"https:\/\/i10x.ai\/news\/llm-peer-review-top-journals-harsh-inconsistent#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/news.i10x.ai\/"},{"@type":"ListItem","position":2,"name":"LLM Reviewers Send 76% of Published Top-Journal Papers Back, Yet Miss Most Planted Flaws"}]},{"@type":"WebSite","@id":"https:\/\/news.i10x.ai\/#website","url":"https:\/\/news.i10x.ai\/","name":"i10X News","description":"Stay updated with the latest news and announcements from i10X.","publisher":{"@id":"https:\/\/news.i10x.ai\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/news.i10x.ai\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/news.i10x.ai\/#organization","name":"i10X","url":"https:\/\/news.i10x.ai\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/news.i10x.ai\/#\/schema\/logo\/image\/","url":"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/10\/i10x-logo-180.png","contentUrl":"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/10\/i10x-logo-180.png","width":180,"height":180,"caption":"i10X"},"image":{"@id":"https:\/\/news.i10x.ai\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/i10x_ai","https:\/\/www.linkedin.com\/company\/i10x-ai","https:\/\/instagram.com\/i10x.ai","https:\/\/www.youtube.com\/@i10X_ai_official"]},{"@type":"Person","@id":"https:\/\/news.i10x.ai\/#\/schema\/person\/d410bb7ee72febefca2c9c369939219a","name":"Christopher Ort","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/christopher-ort-192.webp","url":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/christopher-ort-192.webp","contentUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/christopher-ort-192.webp","caption":"Christopher Ort"},"description":"Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU). He is the author of \u201cThe Style Penalty\u201d, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.","url":"https:\/\/i10x.ai\/news\/author\/christopher-ort","sameAs":["https:\/\/i10x.ai\/blog\/author\/christopher-ort"],"worksFor":{"@id":"https:\/\/i10x.ai\/news\/#organization"},"alumniOf":{"@type":"EducationalOrganization","name":"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU)","url":"https:\/\/www.fau.eu"}}]}},"_links":{"self":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts\/6575","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/comments?post=6575"}],"version-history":[{"count":1,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts\/6575\/revisions"}],"predecessor-version":[{"id":6578,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts\/6575\/revisions\/6578"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/media\/6573"}],"wp:attachment":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/media?parent=6575"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/categories?post=6575"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/tags?post=6575"},{"taxonomy":"company","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/companies?post=6575"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/products?post=6575"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}