Skip to content

Conversation

@srrk-GreenMan
Copy link
Collaborator

@srrk-GreenMan srrk-GreenMan commented Jan 6, 2026

  1. Claude 형님과 함께 vlm 추론 코드를 eval 부분에 저장하였습니다.
  2. Agentic 구조에서 한 번 고장나면 그동안 flow에서 만들었던 데이터가 날라가는 이슈를 방지하기 위해 thread_id와 함께 checkpointing 기술을 도입하였습니다.
  3. prompt에서 reasoning annotation에 해당하는 부분이 str가 아닌 테이블 형태로 집어넣는? 이상한 LLM들이 있어 str으로 넣도록 강제하였습니다.
  4. 모든 도메인에 대해서 우선적으로 간단한 프롬프팅을 진행하였고 notion에 있는 프롬프트로 대체할 예정입니다.

seokhyun Seo and others added 3 commits January 6, 2026 00:50
- Add LangGraph checkpointing for resumable execution
  - SQLite-based persistent checkpoints
  - Auto-generated thread IDs from image paths
  - Checkpoint management utilities (list, show, delete, clear)
  - Resume from checkpoint support

- Add CLI checkpointing options
  - --checkpoint: Enable checkpointing
  - --checkpoint-dir: Custom checkpoint directory
  - --thread-id: Custom thread ID
  - --resume: Resume from existing checkpoint
  - --list-checkpoints, --show-checkpoint, --delete-checkpoint, --clear-checkpoints
  - --resume-checkpoint: Direct resume from thread ID

- Fix bugs
  - Fix async kwargs mutation in api_pool.py (kwargs.pop modifying original dict)
  - Fix error list mutation in flow.py (state errors list being modified in place)
  - Move imports to top of file in flow.py (PEP8)
  - Remove global logging.basicConfig from api_pool.py (library code)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add eval/ module for Table QA evaluation
  - dataset.py: Load QA data, create eval datasets (JSON/JSONL)
  - metrics.py: Evaluation metrics (EM, F1, BLEU, Contains Match)
  - inference.py: Inference clients (vLLM, OpenAI, Anthropic/Claude)
  - evaluate.py: Main evaluation pipeline
  - cli.py: CLI interface for prepare/run/evaluate commands

- Add run_claude_public.sh for Claude Sonnet with public domain

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@srrk-GreenMan srrk-GreenMan linked an issue Jan 6, 2026 that may be closed by this pull request
@srrk-GreenMan
Copy link
Collaborator Author

@choijhyeok 님 혹시 시간 되실 때 확인해주세요! 이 버전에서 notion db 붙이는 기능이 도입되면 좋을 것 같습니다

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] vlm_inference

2 participants