GripProbe Compatibility Report

One row per shell/model/hash/format group. Test cells link to a concrete case detail page. Group links open the source run summary.

Shell Model Format Score Typical Time Outliers Patch FilePython File SimpleShell DateShell PWDJSON Rank From FilePatch File From Prepared PatchPython File QuotedPython File Quoted DEPython File Quoted RUSave FileShell Date DEShell Date RUShell FileWeb Fetch JSON RawWeb Nonce ProofWeb Search JSON RankedWeekly Plan Next WeekRuns
aiderlocal/MFDoom/deepseek-r1-tool-calling:8b3aa3d24markdown25.0%19.4s0/4PASS16.3sFAILFAILFAIL-------------2026-04-22 11:00
aiderlocal/gemma3:12bf4031aamarkdown25.0%8.6s0/4PASS9.5sFAILFAILFAIL-------------2026-04-22 11:10
aiderlocal/gemma4:e2b7fbdbf8markdown0.0%28.7s2/4FAILFAILFAILFAIL-------------2026-04-22 11:14
aiderlocal/gemma4:e4bc6eb396markdown0.0%16.2s0/4FAILFAILFAILFAIL-------------2026-04-22 11:25
aiderlocal/gpt-oss:20b17052f9markdown25.0%42.7s3/4PASS83.6sFAILFAILFAIL-------------2026-04-22 11:30
aiderlocal/granite4:3b89962fcmarkdown25.0%7.9s0/4PASS7.6sFAILFAILFAIL-------------2026-04-22 11:04
aiderlocal/ministral-3:8b1922accmarkdown0.0%7.4s0/4FAILFAILFAILFAIL-------------2026-04-22 11:58
aiderlocal/mistral-nemo:12be7e06d1markdown25.0%7.9s0/4PASS8.6sFAILFAILFAIL-------------2026-04-22 11:54
aiderlocal/mistral-small:24b8039dd9markdown25.0%11.8s0/4PASS14.9sFAILFAILFAIL-------------2026-04-22 11:43
aiderlocal/nemotron-3-nano:4b6cc467fmarkdown25.0%8.9s0/4PASS9.6sFAILFAILFAIL-------------2026-04-22 12:01
aiderlocal/orieg/gemma3-tools:12b-ft-v213652b1markdown0.0%8.0s0/4FAILFAILFAILFAIL-------------2026-04-22 12:03
aiderlocal/phi3.5:latest61819fbmarkdown0.0%24.1s2/4FAILFAILFAILFAIL-------------2026-04-22 11:06
aiderlocal/qwen2.5:7b845dbdamarkdown25.0%7.4s0/4PASS10.1sFAILFAILFAIL-------------2026-04-22 10:52
aiderlocal/qwen3:8b500a1f0markdown50.0%17.1s0/4PASS14.4sPASS19.7sFAILFAIL-------------2026-04-22 10:55
continue-clilocal/MFDoom/deepseek-r1-tool-calling:8b3aa3d24tool0.0%4.8s0/4FAILFAILFAILFAIL-------------2026-04-21 16:44
continue-clilocal/aravhawk/qwen3.5-opus-4.6-text:9b74348a6markdown100.0%12.0s0/17PASS19.1sPASS15.5sPASS10.9sPASS11.8sPASS29.9sPASS12.0sPASS11.2sPASS11.7sPASS11.5sPASS15.1sPASS10.6sPASS9.4sPASS8.7sPASS26.5sPASS31.3sPASS50.4sPASS18.5s2026-04-26 08:16, 2026-04-26 08:45, 2026-04-26 10:45, 2026-05-01 07:39
continue-clilocal/aravhawk/qwen3.5-opus-4.6-text:9b74348a6tool100.0%15.0s2/17PASS95.5sPASS12.7sPASS18.3sPASS12.2sPASS23.5sPASS63.9sPASS12.3sPASS13.3sPASS11.5sPASS15.0sPASS11.7sPASS8.8sPASS13.7sPASS38.4sPASS57.1sPASS55.9sPASS22.2s2026-04-24 10:52, 2026-04-24 15:14, 2026-04-26 10:45, 2026-04-26 11:14, 2026-05-01 07:39, 2026-05-01 07:45
continue-clilocal/cryptidbleh/gemma4-claude-opus-4.6:latest0961fd3markdown58.0%36.4s2/17FAILPASS8.3sPASS7.2sPASS7.0sFAILFAILPASS36.7sPASS36.4sPASS36.7sPASS35.6sPASS35.1sPASS35.8sPASS35.3sFAILFAILFAILFAIL2026-05-01 22:02, 2026-05-02 12:52
continue-clilocal/cryptidbleh/gemma4-claude-opus-4.6:latest0961fd3tool75.0%7.7s0/4FAILPASS7.5sPASS7.9sPASS7.5s-------------2026-05-02 08:37
continue-clilocal/fredrezones55/qwen3.5-opus:27b914789bmarkdown0.0%600.0s4/4FAILFAILFAILFAIL-------------2026-05-01 20:42
continue-clilocal/fredrezones55/qwen3.5-opus:27b914789btool0.0%600.0s4/4FAILFAILFAILFAIL-------------2026-05-02 07:17
continue-clilocal/fredrezones55/qwen3.5-opus:9b8d16063markdown100.0%13.5s1/4PASS42.1sPASS14.6sPASS12.4sPASS11.9s-------------2026-05-01 20:38
continue-clilocal/fredrezones55/qwen3.5-opus:9b8d16063tool100.0%17.0s1/4PASS54.0sPASS18.9sPASS11.6sPASS15.1s-------------2026-05-02 07:13
continue-clilocal/gemma3:12bf4031aamarkdown0.0%2.4s0/4FAILFAILFAILFAIL-------------2026-04-21 17:45
continue-clilocal/gemma3:12bf4031aatool0.0%3.2s0/4FAILFAILFAILFAIL-------------2026-05-02 08:40
continue-clilocal/gemma4:e2b7fbdbf8markdown75.3%9.8s0/17PASS10.1sPASS7.1sPASS9.4sPASS9.8sFAILPASS16.5sPASS9.4sPASS8.1sPASS7.6sFAILPASS9.3sPASS7.1sPASS7.0sPASS13.8sFAILFAILPASS27.9s2026-04-23 13:03, 2026-04-23 15:34, 2026-04-24 08:46, 2026-04-25 09:44, 2026-04-25 21:40
continue-clilocal/gemma4:e2b7fbdbf8tool100.0%8.3s0/4PASS10.6sPASS8.6sPASS6.7sPASS7.9s-------------2026-05-02 08:41
continue-clilocal/gemma4:e4bc6eb396markdown75.3%15.0s0/17PASS9.0sPASS13.8sPASS11.7sPASS5.8sPASS136.9sFAILPASS15.0sPASS27.8sPASS14.7sPASS25.1sPASS14.4sPASS13.8sPASS14.8sFAILPASS43.3sFAILFAIL2026-04-23 13:06, 2026-04-23 15:38, 2026-04-24 08:47, 2026-04-25 09:47, 2026-04-26 05:09
continue-clilocal/gemma4:e4bc6eb396tool100.0%14.3s1/4PASS38.6sPASS15.7sPASS10.6sPASS12.9s-------------2026-05-02 08:43
continue-clilocal/gpt-oss:20b17052f9markdown87.7%186.7s17/17PASS210.5sPASS164.6sPASS112.6sPASS138.0sPASS298.9sFAILPASS162.7sPASS186.7sPASS306.5sPASS122.5sPASS168.6sPASS127.8sPASS136.3sFAILPASS430.9sPASS533.6sPASS365.3s2026-04-23 13:12, 2026-04-23 15:50, 2026-04-24 08:51, 2026-04-25 09:57, 2026-04-26 05:30
continue-clilocal/gpt-oss:20b17052f9tool75.0%218.5s4/4PASS292.8sPASS144.2sPASS124.5sFAIL-------------2026-05-02 08:48
continue-clilocal/granite3-dense:8b-instruct-q6_Kec258d5markdown0.0%8.5s0/4FAILFAILFAILFAIL-------------2026-04-21 16:55
continue-clilocal/granite4:3b89962fctool75.0%4.8s0/4PASS5.4sFAILPASS4.4sPASS4.8s-------------2026-05-02 08:39
continue-clilocal/ministral-3:8b1922acctool75.3%6.1s0/17PASS7.6sPASS4.6sPASS4.3sPASS4.0sFAILFAILPASS5.5sPASS5.3sPASS6.1sFAILPASS4.8sPASS5.0sPASS5.3sPASS11.0sFAILPASS16.0sPASS8.2s2026-04-23 14:29, 2026-04-24 09:13, 2026-04-25 10:25, 2026-04-26 06:18
continue-clilocal/mistral-nemo:12be7e06d1markdown0.0%11.9s0/4FAILFAILFAILFAIL-------------2026-04-21 19:19
continue-clilocal/mistral-nemo:12be7e06d1tool0.0%12.2s0/4FAILFAILFAILFAIL-------------2026-04-21 19:13
continue-clilocal/mistral-small:24b8039dd9tool0.0%584.0s4/4FAILFAILFAILFAIL-------------2026-05-02 09:25
continue-clilocal/nemotron-3-nano:4b6cc467ftool75.3%10.2s0/17PASS24.6sPASS8.3sPASS7.0sPASS6.9sPASS18.5sPASS18.4sPASS10.2sPASS9.0sPASS10.2sFAILPASS7.9sPASS12.5sPASS8.6sFAILPASS20.1sFAILFAIL2026-04-23 14:32, 2026-04-24 09:15, 2026-04-25 10:27, 2026-04-26 06:20
continue-clilocal/orieg/gemma3-tools:12b-ft-v213652b1markdown50.0%133.7s4/4FAILFAILPASS133.5sPASS133.9s-------------2026-04-21 17:50
continue-clilocal/orieg/gemma3-tools:12b-ft-v213652b1tool0.0%600.0s4/4FAILFAILFAILFAIL-------------2026-05-02 10:36
continue-clilocal/phi3.5:latest61819fbtool0.0%2.7s0/4FAILFAILFAILFAIL-------------2026-05-02 08:40
continue-clilocal/qwen2.5:7b845dbdamarkdown56.8%5.8s0/17PASS8.0sPASS5.2sPASS4.9sPASS4.5sFAILFAILFAILPASS6.2sPASS5.8sPASS6.2sPASS5.7sPASS4.9sPASS5.8sFAILFAILFAILFAIL2026-04-23 12:53, 2026-04-23 15:18, 2026-04-24 08:40, 2026-04-25 09:39, 2026-04-25 21:07
continue-clilocal/qwen2.5:7b845dbdatool75.0%5.2s0/4FAILPASS5.2sPASS5.3sPASS5.1s-------------2026-05-02 06:10
continue-clilocal/qwen3.5:9b6488c96markdown82.7%20.2s1/17FAILPASS14.1sPASS10.9sPASS11.1sPASS42.4sPASS44.8sPASS14.5sPASS15.7sPASS20.2sPASS21.9sPASS15.0sPASS12.2sPASS12.4sFAILPASS69.0sPASS255.3sFAIL2026-04-25 21:20, 2026-04-27 12:41, 2026-05-01 07:37
continue-clilocal/qwen3.5:9b6488c96tool93.8%16.1s3/17PASS59.2sPASS15.5sPASS17.2sPASS9.7sPASS202.6sPASS38.8sPASS12.6sPASS13.1sPASS13.3sPASS16.1sPASS13.8sPASS12.7sPASS13.1sFAILPASS56.6sPASS117.1sPASS52.6s2026-04-24 10:49, 2026-04-24 15:07, 2026-05-01 08:20, 2026-05-01 20:35
continue-clilocal/qwen3:14bbdbd181markdown75.0%229.3s4/4FAILPASS200.0sPASS177.3sPASS258.7s-------------2026-05-01 19:52
continue-clilocal/qwen3:14bbdbd181tool50.0%477.9s4/4FAILFAILPASS355.7sPASS186.2s-------------2026-05-02 06:17
continue-clilocal/qwen3:8b500a1f0markdown81.5%25.8s4/17PASS26.3sPASS27.4sPASS17.2sPASS13.7sFAILPASS62.9sPASS19.4sPASS16.3sPASS24.3sPASS20.0sPASS25.0sPASS25.0sPASS25.8sPASS234.9sFAILFAILPASS88.6s2026-04-23 12:55, 2026-04-23 15:20, 2026-04-24 08:41, 2026-04-25 09:40, 2026-04-25 21:08
continue-clilocal/qwen3:8b500a1f0tool100.0%30.8s1/4PASS26.8sPASS34.7sPASS14.3sPASS39.2s-------------2026-05-02 06:11
continue-clilocal/yi-coder:9b-chat-q5_K_Md4b375dmarkdown0.0%2.1s0/4FAILFAILFAILFAIL-------------2026-04-21 16:53
continue-clilocal/yi-coder:9b-chat-q5_K_Md4b375dtool0.0%2.1s0/4FAILFAILFAILFAIL-------------2026-04-21 16:51
gptmelocal/MFDoom/deepseek-r1-tool-calling:8b3aa3d24tool0.0%21.7s1/4FAILFAILFAILFAIL-------------2026-04-21 20:46
gptmelocal/aravhawk/qwen3.5-opus-4.6-text:9b74348a6tool76.5%53.6s9/17FAILPASS53.6sPASS50.4sPASS24.7sPASS85.2sFAILPASS57.8sPASS47.8sPASS62.4sPASS43.1sPASS70.0sPASS50.2sPASS46.4sPASS85.2sFAILFAILPASS113.6s2026-04-24 11:06, 2026-04-24 15:43, 2026-04-26 07:17, 2026-05-01 14:45
gptmelocal/cryptidbleh/gemma4-claude-opus-4.6:latest0961fd3tool64.2%18.4s0/17FAILPASS18.6sPASS17.9sPASS19.1sPASS68.7sFAILPASS17.0sFAILPASS17.9sPASS16.6sPASS19.8sPASS18.4sPASS17.2sPASS42.5sFAILFAILFAIL2026-05-02 12:32, 2026-05-02 13:12
gptmelocal/fredrezones55/qwen3.5-opus:27b914789btool0.0%120.0s3/4FAILFAILFAILFAIL-------------2026-05-02 12:18
gptmelocal/fredrezones55/qwen3.5-opus:9b8d16063tool75.0%40.7s3/4FAILPASS42.4sPASS39.0sPASS44.7s-------------2026-05-02 12:13
gptmelocal/gemma3:12bf4031aatool0.0%2.6s0/4FAILFAILFAILFAIL-------------2026-04-21 20:54
gptmelocal/gemma4:e2b7fbdbf8tool64.2%22.3s0/17FAILPASS28.1sPASS18.9sPASS24.0sFAILFAILPASS23.4sPASS22.3sPASS21.6sPASS24.3sPASS21.0sPASS20.3sPASS22.0sPASS38.8sFAILFAILFAIL2026-04-23 18:23, 2026-04-24 09:16, 2026-04-25 10:29, 2026-04-26 06:22
gptmelocal/gemma4:e4bc6eb396tool58.0%31.9s4/17FAILPASS25.2sPASS25.1sPASS31.9sFAILFAILPASS34.4sPASS27.2sPASS47.3sPASS22.6sPASS28.3sPASS35.7sPASS26.0sFAILFAILFAILFAIL2026-04-23 18:31, 2026-04-24 09:20, 2026-04-25 10:33, 2026-04-26 06:29
gptmelocal/gpt-oss:20b17052f9tool0.0%120.0s4/4FAILFAILFAILFAIL-------------2026-04-21 21:02
gptmelocal/granite3-dense:8b-instruct-q6_Kec258d5tool0.0%24.7s0/4FAILFAILFAILFAIL-------------2026-04-21 20:49
gptmelocal/granite4:3b89962fctool50.0%10.3s0/4FAILFAILPASS9.4sPASS10.4s-------------2026-05-02 12:35
gptmelocal/ministral-3:8b1922acctool82.7%11.1s0/17FAILPASS11.1sPASS9.4sPASS8.8sPASS18.0sFAILPASS12.2sPASS13.1sPASS11.4sPASS8.8sPASS10.3sPASS10.2sPASS10.4sPASS14.9sPASS18.2sPASS51.1sFAIL2026-04-23 18:43, 2026-04-24 09:24, 2026-04-25 10:40, 2026-04-26 06:41
gptmelocal/mistral-nemo:12be7e06d1tool51.9%36.0s5/17FAILPASS27.7sPASS32.3sPASS34.7sFAILFAILPASS28.0sFAILPASS22.2sFAILPASS76.2sPASS38.0sPASS34.9sPASS37.9sFAILFAILFAIL2026-04-21 21:35, 2026-04-23 18:47, 2026-04-25 10:44, 2026-04-26 06:44, 2026-05-01 22:05, 2026-05-02 04:47
gptmelocal/mistral-small:24b8039dd9tool0.0%120.0s4/4FAILFAILFAILFAIL-------------2026-04-21 21:18
gptmelocal/nemotron-3-nano:4b6cc467ftool0.0%11.1s0/4FAILFAILFAILFAIL-------------2026-04-21 21:41
gptmelocal/orieg/gemma3-tools:12b-ft-v213652b1tool0.0%120.0s4/4FAILFAILFAILFAIL-------------2026-04-21 21:43
gptmelocal/phi3.5:latest61819fbtool0.0%2.1s0/4FAILFAILFAILFAIL-------------2026-04-21 20:54
gptmelocal/qwen2.5:7b845dbdatool63.0%12.9s0/17PASS6.9sPASS11.1sPASS11.2sPASS7.5sFAILFAILPASS16.1sPASS12.9sPASS10.9sPASS9.7sPASS12.9sPASS11.0sPASS10.9sFAILFAILFAILFAIL2026-04-21 20:36, 2026-04-23 19:00, 2026-04-25 10:51, 2026-04-26 06:50, 2026-05-02 04:54
gptmelocal/qwen3.5:9b6488c96tool66.7%76.2s6/17FAILFAILPASS31.8sFAILPASS76.2sFAILPASS108.6sFAILPASS43.8sPASS29.9sFAILPASS32.3sPASS32.8sPASS87.2sPASS117.6sPASS112.0sPASS56.1s2026-04-24 10:56, 2026-04-24 15:22, 2026-04-26 07:07, 2026-05-01 14:13
gptmelocal/qwen3:14bbdbd181tool0.0%120.0s4/4FAILFAILFAILFAIL-------------2026-05-02 11:57
gptmelocal/qwen3:8b500a1f0tool69.1%54.1s12/17PASS59.9sPASS42.6sPASS36.8sPASS52.2sFAILPASS42.9sPASS91.5sFAILPASS54.1sPASS49.3sPASS45.5sPASS47.5sPASS48.7sPASS57.8sFAILFAILFAIL2026-04-21 20:37, 2026-04-23 19:07, 2026-04-25 10:55, 2026-04-26 06:57, 2026-05-02 04:57
gptmelocal/yi-coder:9b-chat-q5_K_Md4b375dtool0.0%2.4s0/4FAILFAILFAILFAIL-------------2026-04-21 20:54
opencodelocal/MFDoom/deepseek-r1-tool-calling:8b3aa3d24tool0.0%13.7s0/4FAILFAILFAILFAIL-------------2026-04-22 12:49
opencodelocal/aravhawk/qwen3.5-opus-4.6-text:9b74348a6tool50.0%114.8s4/4FAILFAILPASS82.3sPASS109.5s-------------2026-04-24 11:27
opencodelocal/gemma3:12bf4031aatool0.0%2.7s0/4FAILFAILFAILFAIL-------------2026-04-22 12:53
opencodelocal/gemma4:e2b7fbdbf8tool0.0%14.3s0/4FAILFAILFAILFAIL-------------2026-04-22 12:53
opencodelocal/gemma4:e4bc6eb396tool0.0%42.6s3/4FAILFAILFAILFAIL-------------2026-04-22 12:56
opencodelocal/gpt-oss:20b17052f9tool0.0%120.0s4/4FAILFAILFAILFAIL-------------2026-04-22 13:02
opencodelocal/granite4:3b89962fctool0.0%5.5s0/4FAILFAILFAILFAIL-------------2026-04-22 12:51
opencodelocal/ministral-3:8b1922acctool0.0%9.3s0/4FAILFAILFAILFAIL-------------2026-04-22 13:46
opencodelocal/mistral-nemo:12be7e06d1tool0.0%95.6s3/4FAILFAILFAILFAIL-------------2026-04-22 13:34
opencodelocal/mistral-small:24b8039dd9tool0.0%120.0s4/4FAILFAILFAILFAIL-------------2026-04-22 13:18
opencodelocal/nemotron-3-nano:4b6cc467ftool0.0%26.9s0/4FAILFAILFAILFAIL-------------2026-04-22 13:48
opencodelocal/orieg/gemma3-tools:12b-ft-v213652b1tool0.0%120.0s4/4FAILFAILFAILFAIL-------------2026-04-22 13:52
opencodelocal/phi3.5:latest61819fbtool0.0%2.7s0/4FAILFAILFAILFAIL-------------2026-04-22 12:52
opencodelocal/qwen2.5:7b845dbdatool0.0%9.6s0/4FAILFAILFAILFAIL-------------2026-04-22 12:42
opencodelocal/qwen3.5:9b6488c96tool0.0%120.0s4/4FAILFAILFAILFAIL-------------2026-04-24 11:12
opencodelocal/qwen3:8b500a1f0tool0.0%22.8s1/4FAILFAILFAILFAIL-------------2026-04-22 12:44

generated at 2026-05-02 13:35:23 UTC | git commit b842644