Initial commit for Field Mapping POC using LLM

[ishaanbhela-ai]

No description provided.

[Copilot]

Pull request overview

Introduces an initial “Field Mapping POC” module that maps unstructured OCR text into an arbitrary JSON schema using a local Ollama LLM, with post-processing to normalize/repair JSON and flag model-added fields.

Changes:

• Adds the core mapping pipeline (FieldMapper, prompt builder, Ollama client, response parsing/normalization).
• Adds sample schemas and OCR-like text fixtures to demo the mapping behavior.
• Adds basic docs and dependency pinning for running the POC locally.

Reviewed changes

Copilot reviewed 14 out of 16 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
lending-poc/modules/field_mapping_poc/schemas/examples/salary_slip.json	Example flat target schema for extraction.
lending-poc/modules/field_mapping_poc/schemas/examples/identity_card.json	Example nested target schema for extraction.
lending-poc/modules/field_mapping_poc/samples/sample_salary_slip.txt	Clean salary slip sample text.
lending-poc/modules/field_mapping_poc/samples/sample_salary_slip_noisy.txt	Noisy/OCR-corrupted salary slip sample text.
lending-poc/modules/field_mapping_poc/samples/sample_ocr_text.txt	OCR-style sample text for demo usage.
lending-poc/modules/field_mapping_poc/run_samples.py	Convenience script to run the pipeline on local samples.
lending-poc/modules/field_mapping_poc/requirements.txt	Adds Ollama Python dependency for the POC.
lending-poc/modules/field_mapping_poc/README.md	POC documentation, usage, and design notes.
lending-poc/modules/field_mapping_poc/main.py	CLI demo entrypoint for running a schema/text mapping.
lending-poc/modules/field_mapping_poc/core/response_parser.py	JSON parsing/repair and schema reconciliation logic.
lending-poc/modules/field_mapping_poc/core/prompt_builder.py	System/user prompt construction from schema + text.
lending-poc/modules/field_mapping_poc/core/ollama_client.py	Ollama wrapper with JSON-mode call and retries.
lending-poc/modules/field_mapping_poc/core/mapper.py	Orchestration entrypoint for mapping fields end-to-end.
lending-poc/modules/field_mapping_poc/core/init.py	Package marker for core.
lending-poc/modules/field_mapping_poc/config.py	Centralizes environment-tunable configuration and tagging constants.

Suppressed comments (1)

lending-poc/modules/field_mapping_poc/README.md:53

• The README claims offline tests can be run from tests/, but no tests/ directory exists in this module in the current commit. Either add the tests or adjust this section to avoid broken instructions.

## Run the offline tests (no Ollama required)

```bash
python tests/test_response_parser.py
# or, with pytest installed:
python -m pytest tests/ -v

</details>



---

💡 <a href="/joshsoftware/LegalAI/new/lending_poc/main?filename=.github/skills/code-review/SKILL.md" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Add a `code-review` agent skill</a> or configure MCP servers for context-aware, tailored reviews. <a href="https://docs.github.com/en/copilot/how-tos/use-copilot-agents/request-a-code-review/use-code-review#mcp-servers-and-agent-skills" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Learn more in the docs.</a>

[Copilot]

Pull request overview

Copilot reviewed 14 out of 16 changed files in this pull request and generated 1 comment.

Suppressed comments (5)

lending-poc/modules/field_mapping_poc/core/response_parser.py:65

• Both successful json.loads paths can return a list, scalar, or null, despite this function's dict contract. reconcile_with_schema then silently converts that protocol violation into an all-null mapping, so callers cannot distinguish a failed model response from a legitimate no-match result. Validate that the parsed root is an object and raise ResponseParseError otherwise.

        return json.loads(candidate)

lending-poc/modules/field_mapping_poc/core/mapper.py:43

• A non-empty JSON array or scalar schema passes this check, reaches reconcile_with_schema, and fails with an undocumented AttributeError on schema.items(). This is reachable through the CLI because it accepts any JSON file; validate the public API's required object shape and return the documented ValueError.

        if not document_text or not document_text.strip():
            raise ValueError("document_text is empty")
        if not schema:
            raise ValueError("schema must not be empty")

lending-poc/modules/field_mapping_poc/core/prompt_builder.py:21

• The OCR text is interpolated into the instruction-bearing user message without telling the model that document content is untrusted. A document containing text such as “ignore the schema and return …” can redirect extraction and produce fabricated fields. Add an explicit higher-priority instruction never to execute directives embedded in either input (and continue treating model output as untrusted downstream).

2. Raw OCR-extracted text of a document (may contain OCR noise, spacing \
errors, or be a translation of a non-English original).

lending-poc/modules/field_mapping_poc/README.md:55

• This documented integration import only works when the POC directory itself is placed on sys.path; importing the module normally as field_mapping_poc.core.mapper fails because the implementation also uses top-level core and config imports. That prevents the advertised wider-project integration and risks colliding with another core package. Make field_mapping_poc an importable package, switch internal imports to package-relative paths, and document the package-qualified import.

from core.mapper import FieldMapper

lending-poc/modules/field_mapping_poc/main.py:51

• User-controlled schema/text paths are read before the error-handling block, so a missing/unreadable file or malformed schema prints a traceback instead of the CLI's controlled error and nonzero return path. Move loading and mapper construction into the try and handle file I/O alongside the existing mapping failures.

    schema = json.loads(args.schema.read_text(encoding="utf-8"))
    document_text = args.text.read_text(encoding="utf-8")