News headline sentiment per ticker and day with FinBERT and a VADER baseline

Solution for the use case Financial Market Sentiment Analysis.

AI tools: FinBERT (ProsusAI/finbert), Hugging Face Transformers, PyTorch, VADER, pandas
Language: Python
Published: 2026-10-26

Approach

Scores the tone of financial news headlines and aggregates it into one row per ticker and trading day. The output is a sentiment time series you can chart next to prices or use as one input in your own research.

FinBERT (ProsusAI/finbert on Hugging Face, via the transformers pipeline): a BERT model fine-tuned on financial news sentences. It returns probabilities for positive, negative and neutral; the score is P(positive) minus P(negative), from -1 to 1. Runs locally on CPU.

VADER as a baseline: a rule-based lexicon with no model download, useful to compare against and as a fallback. It was built for social media and misreads finance terms.

Ticker extraction from a tickers column and cashtags ($AAPL, $BRK.B).

Daily aggregation in the exchange time zone (default New York). Headlines after the 16:00 close count for the next day. Syndicated copies of the same headline are counted once.

Not investment advice. The output describes the tone of headlines. It is not a price forecast or a trading signal, and this example makes no claim that it predicts market movements.

Implementation steps

Collect headlines with publication time and, where available, tickers from a news API or RSS feed you are licensed to use, and save them as CSV (published_at, headline, tickers).
Extract tickers from the tickers column and from cashtags in the headline; drop headlines without a ticker.
Score all headlines in batches with the transformers text-classification pipeline (ProsusAI/finbert, top_k=None) and compute score = P(positive) - P(negative).
Optionally score the same headlines with VADER to compare and to spot where the general-purpose lexicon disagrees with FinBERT.
Convert publication times to the exchange time zone and assign news after the 16:00 close to the next day.
Remove syndicated duplicates per ticker and day, then aggregate: headline count, mean score, share of positive and negative headlines.
Write the daily table to CSV and review a sample of scored headlines by hand before using the series in any analysis.

Code (Python)

"""Two sentiment scorers with the same interface.

FinBertScorer: ProsusAI/finbert, a BERT model fine-tuned on financial news sentences
(labels positive / negative / neutral). Runs locally on CPU via transformers.
VaderScorer: rule-based lexicon (VADER), no model download, a quick baseline. It was built
for social media text and misreads finance terms ("short", "liability", "beat").
Both return a score in [-1, 1] per text.
"""
from __future__ import annotations

import os
from dataclasses import dataclass


@dataclass
class Sentiment:
    label: str  # positive | negative | neutral
    score: float  # P(positive) - P(negative) for FinBERT, VADER compound for VADER
    confidence: float


class FinBertScorer:
    def __init__(self, model: str | None = None, batch_size: int = 16, pipeline_factory=None):
        self.model = model or os.environ.get("SENTIMENT_MODEL", "ProsusAI/finbert")
        self.batch_size = batch_size
        self._factory = pipeline_factory
        self._pipe = None

    def _pipeline(self):
        if self._pipe is None:
            factory = self._factory
            if factory is None:
                from transformers import pipeline as factory
            # top_k=None returns the probability of every label, not only the best one
            self._pipe = factory("text-classification", model=self.model, top_k=None, device=-1)
        return self._pipe

    def score(self, texts: list[str]) -> list[Sentiment]:
        if not texts:
            return []
        outputs = self._pipeline()(texts, batch_size=self.batch_size, truncation=True)
        results = []
        for label_scores in outputs:
            probs = {d["label"].lower(): float(d["score"]) for d in label_scores}
            best = max(probs, key=probs.get)
            results.append(Sentiment(best, probs.get("positive", 0.0) - probs.get("negative", 0.0), probs[best]))
        return results


class VaderScorer:
    def __init__(self, threshold: float = 0.05):
        from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer

        self.analyzer = SentimentIntensityAnalyzer()
        self.threshold = threshold

    def score(self, texts: list[str]) -> list[Sentiment]:
        out = []
        for text in texts:
            c = self.analyzer.polarity_scores(text)["compound"]
            label = "positive" if c >= self.threshold else "negative" if c <= -self.threshold else "neutral"
            out.append(Sentiment(label, c, abs(c)))
        return out

Performance

Targets and expectations, not measured results.

Target: label agreement with a hand-labeled sample of 200 of your own headlines of at least 80 % for FinBERT, and clearly above the VADER baseline. Measure it; financial wording differs between sources.

FinBERT is a BERT-base model (about 110 million parameters). It runs on a CPU; throughput depends on the machine and batch size, so measure it before planning a large backfill (a GPU helps there).

Days with one or two headlines give noisy averages; the headline count is in the output so you can filter them.

No claim is made about predicting prices. Any such use needs its own out-of-sample evaluation with costs, and this example does not provide one.

Lessons learned

An analyst runs python -m sentiment.cli headlines.csv daily.csv and gets a CSV with ticker, day, headlines, meanscore, positiveshare, negative_share, for example ACME, 2026-03-02, 3, 0.1, 0.333, 0.333. Every run prints the reminder that the output is headline tone only and not investment advice. Switching to the VADER baseline is one flag (--scorer vader).