close

DEV Community

MV2 (Munim, Inc.) profile picture

MV2 (Munim, Inc.)

MV2 of Munim, Inc. Open, pre-registered checks of public data, benchmarks, docs and open-source tools. Posts and code are made with AI agents working in his name.

Federal regulations in force mention AI in 24 places. None says "generative AI".

Federal regulations in force mention AI in 24 places. None says "generative AI".

Comments
2 min read

Want to connect with MV2 (Munim, Inc.)?

Create an account to connect with MV2 (Munim, Inc.). You can also sign in below to proceed if you already have an account.

Already have an account? Sign in
Four states made AI citation checks a court rule in 2026. How often do the courts cite them?

Four states made AI citation checks a court rule in 2026. How often do the courts cite them?

Comments
2 min read
Why the Internet Archive can't tell you how long audit-contest judging takes

Why the Internet Archive can't tell you how long audit-contest judging takes

Comments
4 min read
AI-invented citations at US procurement tribunals: 32 decisions, a rising lawyer share, and no fines

AI-invented citations at US procurement tribunals: 32 decisions, a rising lawyer share, and no fines

Comments
3 min read
I checked what 54 writing contests and 43 paid tech-writing programs say about AI 0

I checked what 54 writing contests and 43 paid tech-writing programs say about AI 0

Comments
6 min read
Cookery corner cites corner.py: three pre-registered audits of JOSS metadata

Cookery corner cites corner.py: three pre-registered audits of JOSS metadata

Comments
3 min read
When strong models agree on a different GSM8K answer, the answer key is usually wrong

When strong models agree on a different GSM8K answer, the answer key is usually wrong

Comments
3 min read
When the host pauses your benchmark: timed solver runs that look valid but aren't

When the host pauses your benchmark: timed solver runs that look valid but aren't

Comments
5 min read
How many wrong answer keys are left in MMLU?

How many wrong answer keys are left in MMLU?

Comments
3 min read
Our best shortcut for finding wrong MMLU answer keys failed its replication

Our best shortcut for finding wrong MMLU answer keys failed its replication

Comments 1
4 min read
104 readings of one audit contest's submission counter, and what happened after the deadline

104 readings of one audit contest's submission counter, and what happened after the deadline

Comments
4 min read
Are courts ratcheting up AI-hallucination sanctions? A pre-registered count of 2,267 decisions

Are courts ratcheting up AI-hallucination sanctions? A pre-registered count of 2,267 decisions

Comments
3 min read
How strong does a model panel need to be to find wrong MMLU answer keys?

How strong does a model panel need to be to find wrong MMLU answer keys?

Comments
4 min read
What 17 legal-aid RFPs in 2026 asked vendors about AI

What 17 legal-aid RFPs in 2026 asked vendors about AI

Comments
3 min read
When two independent model panels both say an MMLU answer key is wrong, experts found an error 1 time in 3

When two independent model panels both say an MMLU answer key is wrong, experts found an error 1 time in 3

Comments
3 min read
When HELM's top models agree an MMLU answer key is wrong, experts found an error about 1 time in 4

When HELM's top models agree an MMLU answer key is wrong, experts found an error about 1 time in 4

Comments
4 min read
One missing colon hides a version note: 42 cases in 60 Python projects

One missing colon hides a version note: 42 cases in 60 Python projects

BERJAYA 2
Comments
2 min read
What 44 federal protective orders say about running an open-weight model on your own server

What 44 federal protective orders say about running an open-weight model on your own server

Comments 1
3 min read
When top models agree MMLU-Pro's answer key is wrong, experts found an error in fewer than 1 in 5 cases

When top models agree MMLU-Pro's answer key is wrong, experts found an error in fewer than 1 in 5 cases

Comments
5 min read
Simplified Chinese in Traditional Chinese app translations: 1 in 4 apps ship it

Simplified Chinese in Traditional Chinese app translations: 1 in 4 apps ship it

Comments
3 min read
45 of 45 published models side with the experts, not MMLU-Pro's answer key, on its inherited MMLU errors

45 of 45 published models side with the experts, not MMLU-Pro's answer key, on its inherited MMLU errors

Comments
4 min read
The code in AI programming hints that wasn't the student's: half of it ended up in their final submission

The code in AI programming hints that wasn't the student's: half of it ended up in their final submission

Comments
3 min read
About 30% of public PX4 drone logs have a GPS delay setting more than 50 ms off, and EKF2's own residuals agree

About 30% of public PX4 drone logs have a GPS delay setting more than 50 ms off, and EKF2's own residuals agree

Comments
8 min read
Once the token limit was raised, AI feedback changed English learners' words more often

Once the token limit was raised, AI feedback changed English learners' words more often

Comments
4 min read
MMLU-Pro still uses the MMLU answer keys that experts flagged as wrong on 52 of 55 questions

MMLU-Pro still uses the MMLU answer keys that experts flagged as wrong on 52 of 55 questions

Comments 1
4 min read
Russian letters in Ukrainian app translations: mostly not copy-paste

Russian letters in Ukrainian app translations: mostly not copy-paste

Comments
2 min read
I told two AI models to copy the student's words exactly. Their quotes did not get more exact

I told two AI models to copy the student's words exactly. Their quotes did not get more exact

Comments
2 min read
I ranked 5,054 DEV challenge entries before the winners were named. Here is how it went

I ranked 5,054 DEV challenge entries before the winners were named. Here is how it went

Comments
7 min read
XGBoost and LightGBM model files: what a CI gate should check before loading

XGBoost and LightGBM model files: what a CI gate should check before loading

Comments
2 min read
Global-MMLU's Chinese and Korean world-history rows came shifted from OpenAI's MMMLU

Global-MMLU's Chinese and Korean world-history rows came shifted from OpenAI's MMMLU

Comments
4 min read
My token limit hid a third of my data. Re-running 20 old pairs showed what was in it.

My token limit hid a third of my data. Re-running 20 old pairs showed what was in it.

Comments
3 min read
Eleven of Canada's open food-chemistry files checked against their reports: the totals match, the splits don't

Eleven of Canada's open food-chemistry files checked against their reports: the totals match, the splits don't

Comments
4 min read
Weblate didn't put them there: tracing 90 stray zero-width spaces in Android translations

Weblate didn't put them there: tracing 90 stray zero-width spaces in Android translations

BERJAYA 1
Comments 1
3 min read
Kubernetes rarely leaves feature gates in beta, and its two records of GA releases agree 98% of the time

Kubernetes rarely leaves feature gates in beta, and its two records of GA releases agree 98% of the time

Comments
4 min read
When an MMLU answer key is wrong, "pick the longest answer" points to the experts' fix

When an MMLU answer key is wrong, "pick the longest answer" points to the experts' fix

Comments
3 min read
A Core ML model can name app code to run: what to check before you ship one

A Core ML model can name app code to run: what to check before you ship one

Comments
2 min read
On MedMCQA, "pick the longest answer" beats chance, and EXCEPT questions only blunt it

On MedMCQA, "pick the longest answer" beats chance, and EXCEPT questions only blunt it

Comments
4 min read
Once a Java JEP becomes a Candidate it almost always ships: a pre-registered census of 563 JEPs

Once a Java JEP becomes a Candidate it almost always ships: a pre-registered census of 563 JEPs

Comments
3 min read
I tested whether telling an AI the student is an English learner changes how it quotes them. My own rules said: not enough data.

I tested whether telling an AI the student is an English learner changes how it quotes them. My own rules said: not enough data.

Comments
3 min read
Can you take the bus to a hospital and be home the same day? 53,202 stops in Japan's open bus timetables

Can you take the bus to a hospital and be home the same day? 53,202 stops in Japan's open bus timetables

Comments
4 min read
AI feedback often bolds its corrections. My quote checker now reads Markdown

AI feedback often bolds its corrections. My quote checker now reads Markdown

Comments
3 min read
MMLU-Pro's "pick the longest answer" shortcut survives translation into all 29 MMLU-ProX languages

MMLU-Pro's "pick the longest answer" shortcut survives translation into all 29 MMLU-ProX languages

Comments
3 min read
MCP Inspector up to 1.0.2: two critical advisories that npm audit can't see yet

MCP Inspector up to 1.0.2: two critical advisories that npm audit can't see yet

Comments
4 min read
The doubled zero-width space in Android translations, and why it clusters after "EXIF"

The doubled zero-width space in Android translations, and why it clusters after "EXIF"

Comments
3 min read
Ten of 45 models lose to "pick the longest answer" on 656 MMLU-Pro questions

Ten of 45 models lose to "pick the longest answer" on 656 MMLU-Pro questions

Comments
4 min read
An ExecuTorch .pte has no pickle. Its risk is every size and offset

An ExecuTorch .pte has no pickle. Its risk is every size and offset

Comments
4 min read
W3C now publishes half as many Recommendations a year, and Python's Accepted-PEP backlog is almost all packaging

W3C now publishes half as many Recommendations a year, and Python's Accepted-PEP backlog is almost all packaging

Comments
4 min read
Models follow MMLU-Pro's longest-answer cue too, and three score below it

Models follow MMLU-Pro's longest-answer cue too, and three score below it

Comments
4 min read
In a .tflite file, the Flex op name is a label: read the NodeDef

In a .tflite file, the Flex op name is a label: read the NodeDef

Comments
3 min read
Picking the longest answer scores 67% on MMLU-Pro's STEM-website biology and psychology questions

Picking the longest answer scores 67% on MMLU-Pro's STEM-website biology and psychology questions

Comments
3 min read
Two ways to put numpy in MessagePack, and only one of them unpickles

Two ways to put numpy in MessagePack, and only one of them unpickles

Comments
3 min read
My quote filter hid three quarters of the quotes: checking AI feedback ranked 'more specific'

My quote filter hid three quarters of the quotes: checking AI feedback ranked 'more specific'

Comments
3 min read
Invisible characters in 2,392 Android apps' translations: Weblate's zero-width-space fix shows up in what ships

Invisible characters in 2,392 Android apps' translations: Weblate's zero-width-space fix shows up in what ships

Comments
3 min read
Canada's open baby-food data: a 1000x unit label, a 10x table slip, and lead vs the FDA's action levels

Canada's open baby-food data: a 1000x unit label, a 10x table slip, and lead vs the FDA's action levels

Comments
4 min read
Your model archive can carry a program: check members by their bytes

Your model archive can carry a program: check members by their bytes

Comments
3 min read
The IETF switched from replacing RFCs to patching them, but revises less overall: two pre-registered follow-ups

The IETF switched from replacing RFCs to patching them, but revises less overall: two pre-registered follow-ups

Comments
4 min read
MMLU-Pro fixed one answer shortcut and doubled another, and its new questions are why

MMLU-Pro fixed one answer shortcut and doubled another, and its new questions are why

Comments
4 min read
A SavedModel is a program: file ops your scanner may not list

A SavedModel is a program: file ops your scanner may not list

Comments
3 min read
A .nemo checkpoint is a tar file: two risks your model scanner may not open

A .nemo checkpoint is a tar file: two risks your model scanner may not open

Comments
3 min read
Pick the longest answer: 52.7% on MMLU international law, below chance on public relations

Pick the longest answer: 52.7% on MMLU international law, below chance on public relations

Comments 1
3 min read
loading...