From b48d8881eeac0ffbaace4e2ff24e4cf7eab43b38 Mon Sep 17 00:00:00 2001 From: Alexey Efimchik Date: Thu, 12 Mar 2026 06:19:03 +0000 Subject: [PATCH] Add Limitations & Best Practices --- Limitations-%26-Best-Practices.md | 43 +++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 Limitations-%26-Best-Practices.md diff --git a/Limitations-%26-Best-Practices.md b/Limitations-%26-Best-Practices.md new file mode 100644 index 0000000..6eaabe6 --- /dev/null +++ b/Limitations-%26-Best-Practices.md @@ -0,0 +1,43 @@ +# Limitations & Best Practices + +## Limitations + +### Context & Understanding +- **No business logic awareness** — AI only sees code, not product requirements +- **Limited context window** — the model has its own context window, which can hold a limited amount of information. +- **No decision history** — does not store the history of previous executions, only reads existing information in Pull Request. +- **Hallucinations** — may confidently suggest incorrect solutions or non-existent APIs +- **Large PR degradation** — review quality drops significantly for PRs with 1000+ lines changed + +### Process Constraints +- **Does not replace humans** — final decision always belongs to the author and human reviewer +- **Slower on large PRs** — response time grows with the size of changes +- **No memory between PRs** — each review starts from scratch with no prior context +- **Trigger dependency** — only runs on configured events (`[no ai]` in title disables it) + +## Best Practices + +### Writing PRs for Better AI Review + +- **Keep PRs small and focused** — one logical change per PR, ideally under 400 lines +- **Write a clear PR description** — explain *what* and *why*, not just *what changed* +- **Add `[no ai]` to title** when the PR is trivial (version bumps, typo fixes, formatting) +- **Split refactoring from feature changes** — mixed PRs confuse both AI and humans + +### Working with AI Comments + +- **Treat AI comments as suggestions, not mandates** — use your judgment +- **Verify all suggested code** before applying — AI can introduce bugs +- **Use `@kodobot` in comments** to ask follow-up questions about specific lines +- **Don't blindly resolve AI comments** — acknowledge or explain why you disagree +- **Re-trigger review after major changes** by pushing a new commit + +### Security + +- **Never put secrets in PR descriptions or comments** — AI logs may capture them +- **Store `anthropic_api_key` and `gitea_token` in repository secrets**, not in workflow files +- **Audit AI comments on security-sensitive PRs** (auth, crypto, infra) with extra care + +### Team Workflow + +- **Establish a policy** on when to override or ignore AI suggestions