New solution evaluates the effectiveness of
large language models (LLMs) for financial and quantitative use
cases
NEW
YORK, April 3, 2024 /PRNewswire/ -- S&P
Global (NYSE: SPGI) is proud to announce the launch of S&P AI
Benchmarks by Kensho. This innovative benchmarking solution
assesses and ranks the quantitative reasoning abilities and
expertise of large language models (LLMs) across the business and
finance industry. The results of these assessments will be
displayed on a scoreboard, providing a transparent and
comprehensive view of the performance of different LLMs.
S&P AI Benchmarks by Kensho is set to overhaul the way
financial professionals evaluate which LLM to leverage for their
financial workflows. The solution measures and ranks how well LLMs
can solve complex quantitative reasoning questions, understand
business fundamentals, and extract relevant financial information
from documents. By rigorously validating the model's outputs,
S&P AI Benchmarks by Kensho offers accurate and reliable
assessments of LLM applications for financial professionals.
"We're delivering what started as an in-house tool to help
enhance our technology into our customers' hands," said
Bhavesh Dayalji, Chief AI Officer for S&P Global and
CEO of Kensho. "It serves as further evidence of S&P
Global's longstanding commitment to harnessing AI in ways that
consistently enhance our data and experiences for our
customers."
S&P AI Benchmarks by Kensho stands out due to its rigorous
validation process, developed in partnership with a diverse team of
specialists. This includes esteemed academics, seasoned
researchers, subject matter experts across various domains, and
financial professionals from across S&P Global's divisions, all
contributing to the tool's precision and efficacy.
"As GenAI and LLMs become pervasive across products and
services, it is critical that we keep accuracy and reliability at
the forefront of what we do," added Dayalji. "This solution is a
huge step forward in ensuring that we are building solutions our
customers can rely on."
One of the key advantages of S&P AI Benchmarks by Kensho is
its secure submission process. Financial professionals can
benchmark their LLMs without sharing any part of their model, only
its outputs. We use the submitted outputs to create the user's
score, which will be populated on the leaderboard. Any user who
wishes to have their scores removed from the leaderboard can
request so at any time.
S&P Global has been at the forefront of AI for the financial
services industry since acquiring Kensho in 2018. Since then, the
teams have worked closely together to deliver solutions for
customers using Kensho's core capabilities of machine learning,
natural language processing, and data discovery. S&P Global
aims to redefine the intelligence that powers global markets with
AI, embracing the transformative potential of GenAI across the
enterprise to drive innovation, speed up time to market, and build
sophisticated, intuitive experiences for customers.
For more information about S&P AI Benchmarks by Kensho
please visit benchmarks.kensho.com.
About S&P Global
S&P Global (NYSE: SPGI)
provides essential intelligence. We enable governments, businesses
and individuals with the right data, expertise and connected
technology so that they can make decisions with conviction. From
helping our customers assess new investments to guiding them
through ESG and energy transition across supply chains, we unlock
new opportunities, solve challenges, and accelerate progress for
the world.
We are widely sought after by many of the world's leading
organizations to provide credit ratings, benchmarks, analytics and
workflow solutions in the global capital, commodity and automotive
markets. With every one of our offerings, we help the world's
leading organizations plan for tomorrow, today. For more
information, visit www.spglobal.com.
Media Contacts
Roger Cabrera
S&P Global
roger.cabrera@spglobal.com
Madeline McSherry
Kensho
madeline.mcsherry@kensho.com
View original content to download
multimedia:https://www.prnewswire.com/news-releases/sp-global-redefines-financial-ai-benchmarking-with-sp-ai-benchmarks-by-kensho-beta-302106198.html
SOURCE S&P Global