Index writer
index_writer
#
Functions:
Name | Description |
---|---|
generate_context |
Create fake metadata for the ith file. |
write_entry |
Each parallel worker writes a unique path and context to the index. |
IndexReadError
#
Bases: imgtools.io.writers.index_writer.IndexWriterError
Raised when reading the index file fails unexpectedly.
Use this when an exception occurs while attempting to read or parse the existing CSV file.
Source code in src/imgtools/io/writers/index_writer.py
IndexSchemaMismatchError
#
Bases: imgtools.io.writers.index_writer.IndexWriterError
Raised when the index file schema is missing required fields and merging columns is disabled.
Use this error to notify the caller that the existing index cannot accommodate the current row’s structure and merging is not allowed.
Source code in src/imgtools/io/writers/index_writer.py
IndexWriteError
#
Bases: imgtools.io.writers.index_writer.IndexWriterError
Raised when writing to the index file fails unexpectedly.
Use this when a CSV write operation fails during append or full rewrite of the index.
Source code in src/imgtools/io/writers/index_writer.py
IndexWriter
#
Handles safe and smart updates to a shared CSV file used as an index.
This class manages writing entries to a CSV index while avoiding problems like file corruption (from two writers editing at once), column mismatches, or missing data.
Think of this like a notebook where many writers might want to write down their output paths and metadata. This class is the referee: it waits for its turn (locking), makes sure the notebook has the right columns, and writes everything in order.
index_path : Path
Path to the CSV file that acts as a shared index.
lock_path : Path | None, optional
Path to a .lock
file that ensures one writer updates at a time.
If None, uses the index file path with .lock
added.
Methods:
Name | Description |
---|---|
write_entry |
Write one entry to the index file. Safe in parallel with full lock. |
Source code in src/imgtools/io/writers/index_writer.py
write_entry
#
write_entry(
path: pathlib.Path,
context: dict[str, typing.Any],
filepath_column: str = "path",
replace_existing: bool = False,
merge_columns: bool = True,
) -> None
Write one entry to the index file. Safe in parallel with full lock.
You give this a path and a dictionary of info. → It checks the index file. → If the path is already in there and you want to replace it, it does. → If your new info has different keys, it adds new columns (if allowed). → Then it saves the full table back to disk, safely.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
|
pathlib.Path
|
The file path that you want to record in the index. |
required |
|
dict[str, typing.Any]
|
Extra metadata (e.g. subject ID, date, label) to log alongside the path. |
required |
|
str
|
Name of the column to store the file path. Default is "path". |
"path"
|
|
bool
|
If True, update the row if one with the same path already exists. |
False
|
|
bool
|
If True, automatically add new columns if the context has fields the CSV didn't have yet. |
True
|
Source code in src/imgtools/io/writers/index_writer.py
IndexWriterError
#
Bases: Exception
Base exception for all IndexWriter-related errors.
This should be used to catch any general IndexWriter failure that does not fall under a more specific error type.
generate_context
#
Create fake metadata for the ith file.
Source code in src/imgtools/io/writers/index_writer.py
write_entry
#
Each parallel worker writes a unique path and context to the index.