Turn Charts into LLM-Actionable Data: Introducing Chart Recognition in Upstage Document Parse

Minjee Kang

•

Products

•

January 23, 2025

Data-rich charts are pivotal in business documents, yet they remain among the most complex elements for traditional parsers to handle. From irregular layouts to diverse chart types, accurately parsing charts requires not only precision but also structural understanding at a deeper level.

Document Parse has earned widespread acclaim for its robust capabilities, but users have expressed a desire for additional support for images—specifically charts. Now, with chart recognition, Document Parse extends its capabilities beyond reading text and tables to accurately converting charts to HTML, which often hold the most critical information in documents.

With this capability, Document Parse transforms static chart visuals into structured text like HTML, bridging the gap between human-readable visuals and machine-readable insights.

A sample document including a pie chart, converted to HTML, using Upstage Document Parse.

Why chart recognition matters

Parsing documents often focuses on extracting text, tables, or basic images. Charts, however, are frequently overlooked due to their complexity. Attempting to use LLMs to interpret chart visuals directly can lead to inefficiencies, higher costs, and reduced accuracy.

Document Parse addresses this challenge with chart recognition, converting charts into structured data before they reach LLMs. This enhancement expands the range of recognized information, increases accuracy, and streamlines workflows for enterprise users.

Our internal benchmarks highlight the exceptional performance of Document Parse with chart recognition, consistently surpassing leading competitors in key areas. The test sets consist of 24 documents with images from diverse sources and domains, including financial reports and scientific documents.

To validate the capabilities of Document Parse with chart recognition, we conducted a two-step evaluation process that measures accuracy and speed while comparing its performance against competitors.

Chart-level intrinsic evaluation

The first stage measured the efficiency and accuracy of extracting data from charts using a test set exclusively composed of chart images, collected from diverse sources such as Common Crawl, arXiv, and Upstage's internal documents.

Document Parse excelled in both RMS F1 scores and processing time, highlighting its industry-leading capabilities.

Image-level extrinsic evaluation

In the second stage, we tested the ability of Document Parse to extract structured data from entire document images containing charts. The evaluation process mirrored real-world tasks, including question-answering (QA) assessments.

Key experimental results are:

Document Parse achieves the highest accuracy among competitors with a 6.49% increase in accuracy for the Chart-related Document QA task, when chart recognition is enabled.
Processing times are up to 13x faster compared to competitors like LlamaParse.