How I Auto-Generate Claude Skills From Conversation Logs at 3:30 AM

Lily Posted on Jul 20 • Originally published at dev.to Every night while I sleep, a batch job reads yesterday's conversation logs, distills the reusable procedures out of them, and writes them to disk as SKILL.md files. In my previous post I covered the lifecycle management of those auto-generated skills (the two-stage stale → archive retirement). This time I'm exposing the whole internal structure of the step that comes before that: the harvest batch that runs at 3:30 every night, reads the conversation logs, extracts only the reusable procedures, and writes them out as SKILL.md. Right now ~/.claude/skills/auto/ has 104 skills stacked up in it. Almost all of them were generated unattended by skill-harvest.sh. The problem: reading all 3,349 logs every night would torch the quota ~/Documents/my-knowledge-base/raw/conversations/ has accumulated 3,349 conversation logs as of today. If I fed all of them to an LLM every night, I could certainly harvest skill candidates. But at what cost? 3,349 logs × 10k tokens/log (average estimate) = 33 million tokens. Burning that on the Max quota every night would wipe out every other job. I kill this with two mechanisms. Watermark method: never re-read a file that has already been processed Budget cap: fix the per-run LLM spend at a ceiling of --max-budget-usd 1.20 Combining the two keeps each night's harvest to a fixed "3 diff logs, 45KB." The full pipeline: 5 phases ~/.claude/scripts/skill-harvest.sh runs in this flow. 1. ウォーターマーク読み込み → 差分ログを 3 本だけ取得 2. system-reminder ノイズを除去 → バイト上限でダイジェスト生成 3. claude -p ヘッドレス → ステージングに SKILL.md を書かせる 4. author:auto を保証 → AUTO ディレクトリへシェルがコピー 5. ウォーターマーク更新 → ログ追記して終了 Enter fullscreen mode Exit fullscreen mode There are only four parameters (from the top of skill-harvest.sh). MAX_LOGS=3 # 1回で扱うログ本数 PER_LOG_BYTES=15000 # ログ1本あたりの取り込み上限バイト BUDGET_USD=1.20 # 暴走防止コストキャップ TIMEOUT_SEC=600 # 壁時計タイムアウト Enter fullscreen mode Exit fullscreen mode The watermark method prevents duplicate processing ~/.claude/skills/auto/.harvest-watermark records the timestamp from when the previous harvest completed. # 前回以降に更新されたログを収集（初回は最新 MAX_LOGS 件） if [[ -f "$WM" ]]; then newlogs=("${(@f)$(find "$LOGS" -name '*.md' -newer "$WM" 2>/dev/null)}") else newlogs=("${(@f)$(ls -t "$LOGS"/*.md 2>/dev/null)}") fi # mtime 降順で上位 MAX_LOGS 件に絞る newlogs=("${(@f)$(ls -t "${newlogs[@]}" 2>/dev/null | head -$MAX_LOGS)}") Enter fullscreen mode Exit fullscreen mode find -newer "$WM" extracts only the files that are "newer than the watermark." Of the 3,349 logs, typically only 3–5 were updated between last night and this morning — so the set to process is smaller by orders of magnitude. Right before exiting, the watermark is always updated. touch "$WM" exit 0 Enter fullscreen mode Exit fullscreen mode You can verify this in the actual logs. [2026-07-05 03:30:05] harvest start: 3 log(s), digest= 18255B [2026-07-05 03:35:54] harvest done (exit 0, created=0) Enter fullscreen mode Exit fullscreen mode It processes about 3 logs and 18KB every night. Controlling runaway cost with a $1.20 budget cap The ceiling is passed to the --max-budget-usd flag of claude -p. "$CLAUDE" -p "$PROMPT" \ --model sonnet \ --permission-mode acceptEdits \ --allowedTools "Write Edit Read" \ --max-budget-usd "$BUDGET_USD" >> "$LOG" 2>&1 tags and skill/tool listing lines written in verbatim. Shoving those into the prompt lets noise unrelated to procedural knowledge take up the bulk of the tokens. grep -v -e 'system-reminder' -e '^- [a-z0-9].*:' "$f" 2>/dev/null | head -c $PER_LOG_BYTES Enter fullscreen mode Exit fullscreen mode There are two kinds of exclusions. system-reminder: removes entire lines containing '^- [a-z0-9].*:': removes the - skill-name: description style lines of the skill/tool listing This pulls in at most $PER_LOG_BYTES=15000 bytes from a single log. Combined with MAX_LOGS=3, what gets handed to the LLM in one harvest is fixed at a maximum of 45,000 bytes (about 45KB). No matter how many logs pile up, the prompt size doesn't change. Writing files securely via a staging directory Everything under ~/.claude/ can be write-protected while Claude Code is running. If you let claude -p write directly to ~/.claude/skills/auto/, it dies with permission denied. STAGING=$(mktemp -d -t skill-harvest-stg) ( cd "$STAGING" && perl -e 'alarm shift @ARGV; exec @ARGV' "$TIMEOUT_SEC" \ "$CLAUDE" -p "$PROMPT" \ --permission-mode acceptEdits \ --allowedTools "Write Edit Read" \ --max-budget-usd "$BUDGET_USD" >> "$LOG" 2>&1 /SKILL.md （カレントディレクトリ直下に作る。絶対パス禁止） Enter fullscreen mode Exit fullscreen mode After claude finishes writing, the shell side scans $STAGING and copies into the AUTO directory. for sd in "$STAGING"/*(/N); do [[ -f "$sd/SKILL.md" ]] || continue name="${sd:t}" [[ "$name" == .* ]] && continue # author: auto を保証（無ければ frontmatter 直後に挿入） grep -q '^author:[[:space:]]*auto' "$sd/SKILL.md" || python3 - "$sd/SKILL.md" StartCalendarInterval Hour3 Minute30 Enter fullscreen mode Exit fullscreen mode It launches at 3:30 every day. It runs with ProcessType: Background, and logs are emitted to ~/.claude/logs/com.lily.skill-harvest.log. Registration and an immediate test: launchctl load ~/Library/LaunchAgents/com.lily.skill-harvest.plist launchctl start com.lily.skill-harvest Enter fullscreen mode Exit fullscreen mode Pitfalls I hit launchd's PATH is minimal: in a bare launchd environment, claude isn't found. Unless you explicitly add nvm's bin and ~/.local/bin to the plist's EnvironmentVariables/PATH, it exits immediately Direct writes to ~/.claude get permission denied: it's protected while Claude Code is running. Worked around with staging + shell copy The timeout command isn't standard on macOS: substituted with perl -e 'alarm N; exec @ARGV' Forget to touch the watermark and it re-scans all logs every time: the touch "$WM" right before exit must always be placed immediately before exit 0 exit 142 is recorded looking the same as created=0: since a timeout can't be distinguished from a normal exit, get in the habit of telling them apart by the time gap between start and done in the log (normal: a few minutes, timeout: exactly 10 minutes) A skill with the same name already existing can't be overwritten: because [[ -e "$AUTO/$name" ]] skips the copy, updating an existing skill (patch) is handled on the prompt side by instructing "if it's a duplicate, Read the existing one and patch it" "A miss is normal," as seen in the real logs Here's this week's harvest log. [2026-07-05 03:30:05] harvest start: 3 log(s), digest= 18255B この会話ログ抜粋を確認しました。INDEX.md はファイル一覧のみで 手順的知識なし。tsuzuke-ios の 2 件は既存の xcodegen-project-yml-security-review スキルでカバー済み。 該当なし [2026-07-05 03:35:54] harvest done (exit 0, created=0) [2026-07-06 03:30:05] harvest start: 3 log(s), digest= 18255B 一回性レビュー＋抽出基準に非該当。よってファイルは作成しませんでした。 [2026-07-06 03:30:45] harvest done (exit 0, created=0) [2026-07-07 03:30:05] harvest start: 3 log(s), digest= 35008B [2026-07-07 03:32:28] CREATED: note-bokumolily-factual-review [2026-07-07 03:32:28] harvest done (exit 0, created=1) Enter fullscreen mode Exit fullscreen mode Four out of five days are created=0. A miss isn't an anomaly — it's proof the extraction criteria are working correctly. Duplicates of existing skills, one-off personal circumstances, and idle chat all get rejected. Because the LLM makes the rejection call autonomously, it doesn't churn out junk. Note: A created=0 harvest consumes almost no cost (Max quota). It's just enough to launch claude -p, read the prompt, and return "nothing applicable." "Misses being cheap" is part of why I can run this nightly batch every single night. Summary Watermark method: extract only the diff logs with find -newer, keyed off the mtime of .harvest-watermark. Even with 3,349 logs, it processes only 3 per night Double budget cap: --max-budget-usd 1.20 + perl alarm 600 puts a ceiling on both LLM spend and execution time Noise filter up front: adding grep -v 'system-reminder' as preprocessing compresses the prompt to a fixed 45KB Writing via staging: mktemp -d lets the shell mediate, avoiding write contention against ~/.claude A miss is normal: with tight extraction criteria, a run of created=0 days is healthy behavior I wrote about how skills crossing the 100 mark raised the importance of the curate side in the previous piece. Next time I plan to write about how these two batches piling up skills and memory started to make Claude Code slow to launch — auditing and reducing the amount of context injection. Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code. Follow along: Portfolio · X · GitHub*