Skip to main content

JSONL Session File Chunker

Splits large JSONL session files into processable chunks with smart boundary detection and overlap for conversation continuity.

Features:

  • Smart split point detection (via JSONLAnalyzer)
  • Overlap window generation
  • Chunk file creation
  • Chunk index generation
  • Progress tracking

Author: Claude + AZ1.AI License: MIT

File: session_chunker.py

Classes​

SessionChunk​

Represents a chunk of a session file

ChunkIndex​

Index of all chunks for a session

SessionChunker​

Split large JSONL session files into manageable chunks.

Functions​

split_session(target_chunk_lines, overlap_messages, dry_run)​

Split session into chunks with overlap.

create_chunk_file(start_line, end_line, chunk_id, overlap_start, overlap_end, split_quality, split_reason, dry_run)​

Extract chunk from session file.

create_chunk_index(chunks, structure, chunk_size_target, overlap_messages)​

Create index of all chunks.

cleanup_chunks()​

Remove all chunk files for this session

Usage​

python session_chunker.py