<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/"
    xmlns:atom="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/" version="2.0">
    <channel>
        
        <title>
            <![CDATA[ Samuel Tosin - freeCodeCamp.org ]]>
        </title>
        <description>
            <![CDATA[ Browse thousands of programming tutorials written by experts. Learn Web Development, Data Science, DevOps, Security, and get developer career advice. ]]>
        </description>
        <link>https://www.freecodecamp.org/news/</link>
        <image>
            <url>https://cdn.freecodecamp.org/universal/favicons/favicon.png</url>
            <title>
                <![CDATA[ Samuel Tosin - freeCodeCamp.org ]]>
            </title>
            <link>https://www.freecodecamp.org/news/</link>
        </image>
        <generator>Eleventy</generator>
        <lastBuildDate>Tue, 17 Nov 2026 03:38:04 +0000</lastBuildDate>
        <atom:link href="https://www.freecodecamp.org/news/author/SamuelAspirin/rss.xml" rel="self" type="application/rss+xml" />
        <ttl>60</ttl>
        
            <item>
                <title>
                    <![CDATA[ How to Build an NLP Pipeline in Python to Analyze Therapy Progress Notes ]]>
                </title>
                <description>
                    <![CDATA[ Therapy progress notes are written records of what happened during a therapy session. They may include what a client said, what the therapist observed, the topics discussed, and plans for the next ses ]]>
                </description>
                <link>https://www.freecodecamp.org/news/build-nlp-pipeline-python-therapy-notes/</link>
                <guid isPermaLink="false">6afba1044c9d975934925ee6</guid>
                
                    <category>
                        <![CDATA[ Python ]]>
                    </category>
                
                    <category>
                        <![CDATA[ nlp ]]>
                    </category>
                
                    <category>
                        <![CDATA[ therapy ]]>
                    </category>
                
                    <category>
                        <![CDATA[ machine learning models ]]>
                    </category>
                
                <dc:creator>
                    <![CDATA[ Samuel Tosin ]]>
                </dc:creator>
                <pubDate>Tue, 17 Nov 2026 00:34:12 +0000</pubDate>
                <media:content url="https://cdn.hashnode.com/uploads/covers/5fc16e412cae9c5b190b6cdd/2573405d-995c-4738-862d-0fadc243446c.png" medium="image" />
                <content:encoded>
                    <![CDATA[ <p>Therapy progress notes are written records of what happened during a therapy session. They may include what a client said, what the therapist observed, the topics discussed, and plans for the next session. Over time, these notes can add up to a large amount of text.</p>
<p>Reading one note helps us understand one session. But what if we want to look at several notes? Can we use text analysis to see how the language changes from one session to another?</p>
<p>Natural Language Processing (NLP) is a way of using computers to work with human language. With Python, you can break text into sentences, pick out language features, and use models to classify sentiment and emotion. In this tutorial, you’ll use <a href="https://spacy.io/usage/processing-pipelines">spaCy’s language-processing pipeline</a> for text processing and <a href="https://huggingface.co/docs/transformers/main_classes/pipelines">Hugging Face Transformers</a> to run pretrained classification models.</p>
<p>You'll build a small NLP pipeline using fictional therapy notes. You’ll use spaCy to process the text, two ready-made Hugging Face models to classify sentiment and emotion, pandas to organize results by session, and Matplotlib to create charts.</p>
<p>You can find the complete project code in the <a href="https://github.com/Samuellyworld/therapeutic-alliance">GitHub repository</a>.</p>
<p>We are not trying to decide whether therapy is working. The goal is to learn how to turn written text into data that we can review, question, and display in charts.</p>
<p>At the end, you'll have a practical Python project using spaCy, Hugging Face Transformers, pandas, and Matplotlib to explore sentiment, emotion, and language patterns in fictional therapy notes.</p>
<p><strong>Note</strong>: This tutorial is for learning only and uses fictional text. The models have not been tested for clinical use. They cannot tell us whether therapy is working, identify problems in the therapist-client relationship, or replace a trained professional.</p>
<h2 id="heading-table-of-contents">Table of Contents</h2>
<ul>
<li><a href="#heading-what-nlp-can-and-cannot-tell-us">What NLP Can and Cannot Tell Us</a></li>
<li><a href="#heading-what-we-are-building">What We Are Building</a></li>
<li><a href="#heading-tools-and-models">Tools and Models</a></li>
<li><a href="#heading-set-up-the-python-project">Set Up the Python Project</a></li>
<li><a href="#heading-create-a-synthetic-dataset">Create a Synthetic Dataset</a></li>
<li><a href="#heading-preprocess-notes-with-spacy">Preprocess Notes with spaCy</a></li>
<li><a href="#heading-run-sentiment-analysis-with-distilbert">Run Sentiment Analysis with DistilBERT</a></li>
<li><a href="#heading-classify-emotions-with-distilroberta">Classify Emotions with DistilRoBERTa</a></li>
<li><a href="#heading-explore-linguistic-features">Explore Linguistic Features</a></li>
<li><a href="#heading-compare-results-across-sessions">Compare Results Across Sessions</a></li>
<li><a href="#heading-visualize-the-outputs">Visualize the Outputs</a></li>
<li><a href="#heading-connect-the-pipeline">Connect the Pipeline</a></li>
<li><a href="#heading-test-the-project">Test the Project</a></li>
<li><a href="#heading-limitations-and-ethical-considerations">Limitations and Ethical Considerations</a></li>
<li><a href="#heading-conclusion-and-further-learning">Conclusion and Further Learning</a></li>
<li><a href="#heading-references">References</a></li>
</ul>
<h2 id="heading-what-nlp-can-and-cannot-tell-us">What NLP Can and Cannot Tell Us</h2>
<p>Natural Language Processing (NLP) is a part of artificial intelligence that helps computers work with human language. Common NLP tasks include breaking text into words and sentences, identifying grammar, finding names and places, and classifying sentiment and emotion.</p>
<p>Each task does something different. Tokenization breaks text into smaller pieces, such as words. Part-of-speech tagging identifies how a word is used in a sentence, such as a noun or verb. Lemmatization maps a word to its base form, while named-entity recognition identifies items such as people, organizations, and places. These are examples of the components available in <a href="https://spacy.io/usage/processing-pipelines">spaCy’s processing pipelines</a>. Sentiment analysis estimates whether text sounds positive or negative based on examples the model learned from. Emotion classification gives scores for emotion categories the model learned during training.</p>
<p>These results do not tell us what a person is really feeling. A therapy note may contain the therapist’s own observations, a client’s exact words, or the therapist’s summary of what the client said. A model may not know who said something or understand the full situation. In this tutorial, we’ll talk about model predictions and language patterns, not a person’s emotions or progress in therapy.</p>
<h2 id="heading-what-we-are-building">What We Are Building</h2>
<p>Our project will analyze a small collection of fictional session notes and produce four types of results:</p>
<ul>
<li>A sentiment prediction for each sentence</li>
<li>Emotion predictions for each sentence, along with average scores</li>
<li>Simple language features, such as counts of selected pronouns and phrases like “maybe” or “I think”</li>
<li>A timeline and dashboard to compare the results across sessions</li>
</ul>
<p>The workflow is:</p>
<img src="https://cdn.hashnode.com/uploads/covers/61e7d0230204273bdcb1072f/b8ca06c5-214c-4ffd-bc14-0d7181354919.svg" alt="Figure 1 — Pipeline diagram (pipeline-miro.svg)  A five-stage NLP pipeline flowchart. Stage 01 (Input) receives fictional therapy notes as a validated CSV with client_id, session, and note fields. Stage 02 (Preprocess) uses spaCy with a blank pipeline and sentencizer to tokenize sentences, count words, and compute language ratios. The flow then splits in parallel into Stage 03A (Sentiment), which uses DistilBERT SST-2 to produce positive and negative scores, and Stage 03B (Emotion), which uses DistilRoBERTa to produce seven emotion scores using chunked weighted mean aggregation. A separate card below 03B lists the seven emotion categories: anger, disgust, fear, joy, neutral, sadness, and surprise. The two parallel branches merge into Stage 04 (Explore), which outputs CSV tables, Matplotlib charts, a portable HTML report, and a run manifest. A dashed line from Stage 02 feeds five linguistic features (word count, sentence count, mean sentence length, lexical diversity, first-person ratio) into a band at the bottom, joined to the final output by client and session key." style="display: block;" width="1500" height="860" loading="lazy">

<p><em>Figure 1. High-level view of the NLP workflow. The diagram also shows possible extensions, such as CSV and HTML reports, that are not implemented in every step of this tutorial.</em></p>
<p>We’ll keep the code in separate files so each part has one clear job. The project will not calculate a relationship-rupture score or label a clinical outcome.</p>
<h2 id="heading-tools-and-models">Tools and Models</h2>
<h3 id="heading-python">Python</h3>
<p>Python will connect each part of the project. It also has many libraries for working with data and language.</p>
<p><a href="https://spacy.io/usage">spaCy</a> is a Python library that helps us process text. Its <a href="https://spacy.io/usage/processing-pipelines">processing pipeline</a> can split text into tokens, assign part-of-speech tags, find base forms of words, and identify possible names or places. We’ll use its small English model, <a href="https://spacy.io/models/en#en_core_web_sm">en_core_web_sm</a>. This model works with general English, not specifically with therapy notes.</p>
<h3 id="heading-hugging-face-transformers"><a href="https://huggingface.co/docs/transformers">Hugging Face Transformers</a></h3>
<p>We’ll use two pretrained classification models:</p>
<p>The first model, <a href="https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english">distilbert/distilbert-base-uncased-finetuned-sst-2-english</a>, classifies text as positive or negative. Its <a href="https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english">model card</a> identifies it as a DistilBERT checkpoint fine-tuned on the Stanford Sentiment Treebank (SST-2), a dataset of general English sentences. DistilBERT itself is described in the research paper <a href="https://arxiv.org/abs/1910.01108">“DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter”</a>. Neither the checkpoint nor the original DistilBERT paper establishes that this model is suitable for therapy notes.</p>
<p>The second model, <a href="https://huggingface.co/j-hartmann/emotion-english-distilroberta-base">j-hartmann/emotion-english-distilroberta-base</a>, predicts seven categories: anger, disgust, fear, joy, neutral, sadness, and surprise. According to <a href="https://huggingface.co/j-hartmann/emotion-english-distilroberta-base">Jochen Hartmann’s model card</a>, it was trained on a balanced collection drawn from six datasets, including social media posts, student self-reports, and dialogue. It has not been validated for analysing therapy progress notes.</p>
<p>These models help us learn how text classification works. But because they were trained on general text, we cannot assume they will work well on therapy notes.</p>
<h3 id="heading-pandas-and-matplotlib">pandas and Matplotlib</h3>
<p><a href="https://pandas.pydata.org/docs/">pandas</a> will help us organize the results in a table called a DataFrame. <a href="https://matplotlib.org/stable/">Matplotlib</a> will help us turn the results into charts.</p>
<h3 id="heading-pytest">pytest</h3>
<p><a href="https://docs.pytest.org/">pytest</a> is a tool for checking that our code works as expected. We’ll use sample model results in some tests, so we can test the code without downloading the large model files.</p>
<h2 id="heading-set-up-the-python-project">Set Up the Python Project</h2>
<p>Create a folder for the project, then move into it:</p>
<pre><code class="language-bash">mkdir therapy_notes_nlp
cd therapy_notes_nlp
mkdir data src tests images
</code></pre>
<p>Your project folders and files should look like this:</p>
<pre><code class="language-text">therapy_notes_nlp/
├── data/
│   └── synthetic_notes.json
├── images/
├── src/
│   ├── __init__.py
│   ├── preprocessor.py
│   ├── sentiment.py
│   ├── emotions.py
│   ├── tracker.py
│   └── visualizer.py
├── tests/
│   ├── conftest.py
│   ├── test_analyzer.py
│   ├── test_emotions.py
│   ├── test_preprocessor.py
│   ├── test_sentiment.py
│   └── test_tracker.py
├── analyzer.py
├── main.py
├── pytest.ini
└── requirements.txt
</code></pre>
<p>Create a virtual environment and activate it. A virtual environment keeps this project’s Python packages separate from packages used by your other projects:</p>
<pre><code class="language-bash">python -m venv .venv

# Windows
.venv\Scripts\activate

# macOS or Linux
source .venv/bin/activate
</code></pre>
<p>Add the following packages to <code>requirements.txt</code>:</p>
<pre><code class="language-python">spacy&gt;=3.8,&lt;4.0
transformers&gt;=4.45,&lt;6.0
torch&gt;=2.2
pandas&gt;=2.2,&lt;3.0
matplotlib&gt;=3.8,&lt;4.0
pytest&gt;=8.0,&lt;10.0
</code></pre>
<p>Install the packages and download spaCy’s English language model:</p>
<pre><code class="language-bash">pip install -r requirements.txt
python -m spacy download en_core_web_sm
</code></pre>
<p>The first time you run the project, Hugging Face will download the model files. You can run the models on a CPU, but they may take longer than they would on a GPU.</p>
<h2 id="heading-create-a-synthetic-dataset">Create a Synthetic Dataset</h2>
<p>Therapy notes can contain private information. To protect people’s privacy, we’ll use made-up examples throughout this tutorial. This also means anyone can follow the project without the need to access real patient records.</p>
<p>Keep using fictional data for this tutorial. Do not replace it with real therapy notes unless you have the proper permission and safeguards, and have checked the laws and rules that apply to you.</p>
<p>spaCy can sometimes find names and other details in text, but it can also miss them. Removing a person’s name does not make a therapy note anonymous. For this tutorial, use only the fictional examples. Any real research project would need the right approval and privacy protections.</p>
<p>If you haven't already, create a file named <code>synthetic_notes.json</code> in the <code>data</code> folder:</p>
<pre><code class="language-json">[
  {
    "session": 1,
    "client_id": "SYNTHETIC_001",
    "note": "The fictional client described feeling overwhelmed by work demands. They said sleep had been difficult and they were unsure where to begin."
  },
  {
    "session": 2,
    "client_id": "SYNTHETIC_001",
    "note": "The fictional client discussed trying a new evening routine. They described one helpful moment but also mentioned ongoing worry."
  },
  {
    "session": 3,
    "client_id": "SYNTHETIC_001",
    "note": "The fictional client reported a stressful week and said they felt frustrated. They also identified one situation they handled differently."
  },
  {
    "session": 4,
    "client_id": "SYNTHETIC_001",
    "note": "The fictional client described feeling more hopeful about a work conversation. They remained concerned about maintaining the changes."
  },
  {
    "session": 5,
    "client_id": "SYNTHETIC_001",
    "note": "The fictional client questioned whether the current approach was useful and said they had expected a different pace. They wanted to discuss this concern."
  },
  {
    "session": 6,
    "client_id": "SYNTHETIC_001",
    "note": "The fictional client and therapist discussed the concern raised previously. The client described feeling heard, while noting that some questions remained unresolved."
  },
  {
    "session": 7,
    "client_id": "SYNTHETIC_001",
    "note": "The fictional client reflected on using a coping strategy during a difficult conversation. They described both relief and lingering nervousness."
  },
  {
    "session": 8,
    "client_id": "SYNTHETIC_001",
    "note": "The fictional client reviewed topics from the earlier sessions. They described changes they had noticed and identified challenges they still wanted to work on."
  }
]
</code></pre>
<p>Each item contains a session number, a made-up client ID, and a note. The examples include both positive and difficult experiences. We have not written them to force the models to give certain answers. We’ll run the models and examine the results they produce.</p>
<h2 id="heading-preprocess-notes-with-spacy">Preprocess Notes with spaCy</h2>
<p>Before we classify the text, we’ll use spaCy to split each note into sentences and collect some basic information about the words.</p>
<p>In <code>src/preprocessor.py:</code></p>
<pre><code class="language-python">from __future__ import annotations
from typing import Any
import spacy

MODEL_NAME = "en_core_web_sm"

def load_nlp(model_name: str = MODEL_NAME):
    return spacy.load(model_name)

_nlp = None

def get_nlp():
    global _nlp
    if _nlp is None:
        _nlp = load_nlp()
    return _nlp

def preprocess_note(text: str, nlp=None) -&gt; dict[str, Any]:
    if not isinstance(text, str):
        raise TypeError("text must be a string")
    if not text.strip():
        return {
            "tokens": [], "lemmas": [], "sentences": [],
            "pos_tags": [], "entities": [],
            "token_count": 0, "sentence_count": 0
        }

    doc = (nlp or get_nlp())(text)
    tokens = [t.text.lower() for t in doc if not t.is_punct and not t.is_space]
    lemmas = [
        t.lemma_.lower() for t in doc
        if not t.is_stop and not t.is_punct and not t.is_space and t.is_alpha
    ]
    sentences = [s.text.strip() for s in doc.sents if s.text.strip()]
    pos_tags = [
        {"text": t.text, "pos": t.pos_, "tag": t.tag_}
        for t in doc if not t.is_punct and not t.is_space
    ]
    entities = [{"text": e.text, "label": e.label_} for e in doc.ents]

    return {
        "tokens": tokens,
        "lemmas": lemmas,
        "sentences": sentences,
        "pos_tags": pos_tags,
        "entities": entities,
        "token_count": len(tokens),
        "sentence_count": len(sentences)
    }
</code></pre>
<p>The <code>_nlp = None</code> / <code>get_nlp()</code> pattern is worth pausing on. Loading a spaCy model takes a second or two, and you do not want that cost on every function call. By keeping <code>_nlp</code> as a module-level variable, the model loads once, and every call after that reuses it. The <code>(nlp or get_nlp())(text)</code> line might look odd at first, but it just means: use the model I was handed, or fall back to the default. This also makes the function easier to test later, since you can pass in a lightweight fake model instead of the real one.</p>
<p>One thing to notice: we pass the original sentences to the classifiers, not the cleaned tokens or lemmas. That is intentional. Words like "not" change a sentence's meaning, and stripping them before classification could quietly change what the model sees.</p>
<p>The entities field stores possible names, places, and other named items found by spaCy. It does not remove personal information or make the text anonymous.</p>
<h2 id="heading-run-sentiment-analysis-with-distilbert">Run Sentiment Analysis with DistilBERT</h2>
<p>Sentiment analysis is a way to estimate whether text is more positive or negative based on examples a model learned from. The DistilBERT model we selected was trained further on SST-2, a general dataset of English sentences. It returns a positive score and a negative score for each sentence.</p>
<p>In <code>src/sentiment.py</code>:</p>
<pre><code class="language-python">MODEL_ID = "distilbert/distilbert-base-uncased-finetuned-sst-2-english"
_pipeline = None

def get_pipeline():
    global _pipeline
    if _pipeline is None:
        from transformers import pipeline
        _pipeline = pipeline(
            "text-classification", model=MODEL_ID, top_k=None
        )
    return _pipeline

def _normalise_prediction(prediction):
    if isinstance(prediction, dict):
        return [prediction]
    if prediction and isinstance(prediction[0], list):
        return prediction[0]
    return prediction

def analyze_sentiment(sentences: list[str], classifier=None) -&gt; dict:
    clean = [s.strip() for s in sentences if isinstance(s, str) and s.strip()]
    if not clean:
        return {
            "sentence_results": [], "aggregate_score": None,
            "aggregate_label": None, "positive_ratio": 0.0,
            "negative_ratio": 0.0
        }

    model = classifier or get_pipeline()
    predictions = model(clean, truncation=True, batch_size=8)
    results = []

    for sentence, prediction in zip(clean, predictions):
        scores = _normalise_prediction(prediction)
        by_label = {
            item["label"].upper(): float(item["score"])
            for item in scores
        }
        positive = by_label.get("POSITIVE", 0.0)
        negative = by_label.get("NEGATIVE", 0.0)
        signed_score = positive - negative
        label = (
            "POSITIVE" if signed_score &gt; 0
            else "NEGATIVE" if signed_score &lt; 0
            else "BALANCED"
        )
        results.append({
            "sentence": sentence,
            "label": label,
            "positive_score": positive,
            "negative_score": negative,
            "signed_score": round(signed_score, 4)
        })

    values = [item["signed_score"] for item in results]
    aggregate = sum(values) / len(values)
    return {
        "sentence_results": results,
        "aggregate_score": round(aggregate, 4),
        "aggregate_label": (
            "POSITIVE" if aggregate &gt; 0
            else "NEGATIVE" if aggregate &lt; 0
            else "BALANCED"
        ),
        "positive_ratio": round(
            sum(item["label"] == "POSITIVE" for item in results) / len(results), 3
        ),
        "negative_ratio": round(
            sum(item["label"] == "NEGATIVE" for item in results) / len(results), 3
        )
    }
</code></pre>
<p><code>get_pipeline()</code> follows the same lazy-loading pattern as <code>get_nlp()</code>: the model loads once and is reused on every subsequent call. Setting <code>top_k=None</code> tells the Hugging Face pipeline to return scores for every label, not just the highest one. Without it, we would only get the winning label and could not compare both scores. <code>truncation=True</code> cuts any input longer than 512 tokens, which is the maximum the model can process. <code>batch_size=8</code> sends up to eight sentences to the model at once, which is faster than sending them one at a time.</p>
<p><code>_normalise_prediction()</code> exists because the pipeline can return results in different shapes depending on the version and configuration. It converts every shape into a consistent list of <code>{"label": ..., "score": ...}</code> dictionaries so the rest of the function always has the same format to work with.</p>
<p>For each sentence, <code>by_label</code> is a dictionary built from a comprehension that maps each uppercase label to its score. <code>signed_score</code> subtracts the negative score from the positive score, giving a value between -1 and +1. At the end, the function averages all signed scores to produce one <code>aggregate_score</code> for the session.</p>
<p>This index is simply a summary we created from the model's scores. It is not a medically approved measurement, a recovery score, or a measure of wellbeing. A score close to zero may also mean that positive and negative results balance each other out.</p>
<h2 id="heading-classify-emotions-with-distilroberta">Classify Emotions with DistilRoBERTa</h2>
<p>Sentiment and emotion classification are not the same. Sentiment gives a broad positive or negative result. Emotion classification looks at categories such as joy, sadness, anger, and fear.</p>
<p>In <code>src/emotions.py</code>:</p>
<pre><code class="language-python">MODEL_ID = "j-hartmann/emotion-english-distilroberta-base"
_pipeline = None

def get_pipeline():
    global _pipeline
    if _pipeline is None:
        from transformers import pipeline
        _pipeline = pipeline(
            "text-classification", model=MODEL_ID, top_k=None
        )
    return _pipeline

def _normalise_prediction(prediction):
    if isinstance(prediction, dict):
        return [prediction]
    if prediction and isinstance(prediction[0], list):
        return prediction[0]
    return prediction

def detect_emotions(sentences: list[str], classifier=None) -&gt; dict:
    clean = [s.strip() for s in sentences if isinstance(s, str) and s.strip()]
    if not clean:
        return {
            "per_sentence": [], "emotion_distribution": {},
            "session_dominant_emotion": None
        }

    model = classifier or get_pipeline()
    predictions = model(clean, truncation=True, batch_size=8)
    per_sentence = []
    totals = {}

    for sentence, prediction in zip(clean, predictions):
        scores = _normalise_prediction(prediction)
        score_map = {
            item["label"].lower(): float(item["score"])
            for item in scores
        }
        for label, score in score_map.items():
            totals[label] = totals.get(label, 0.0) + score
        dominant = max(score_map, key=score_map.get)
        per_sentence.append({
            "sentence": sentence,
            "dominant_emotion": dominant,
            "scores": score_map
        })

    distribution = {
        label: round(score / len(clean), 4)
        for label, score in totals.items()
    }
    dominant_session = max(distribution, key=distribution.get)
    return {
        "per_sentence": per_sentence,
        "emotion_distribution": distribution,
        "session_dominant_emotion": dominant_session
    }
</code></pre>
<p>The <code>totals</code> dictionary accumulates scores across sentences. For each sentence, the loop adds the score for every emotion label to the running total. Using <code>totals.get(label, 0.0)</code> safely handles the first time a label appears, starting it at zero rather than causing a key error.</p>
<p><code>max(score_map, key=score_map.get)</code> finds the label with the highest score in <code>score_map</code> for that sentence. The <code>key</code> argument tells <code>max()</code> to compare values in the dictionary rather than the keys themselves.</p>
<p>After processing all sentences, <code>distribution</code> divides each accumulated total by the number of sentences, giving a mean score per emotion across the whole session. <code>max(distribution, key=distribution.get)</code> applies the same pattern to find the session-level dominant emotion.</p>
<p>These results are the model's predictions. They do not confirm how anyone feels. The model was trained on general English text, while therapy notes are written for a different purpose and may use different language. We should not assume that the model will work reliably on them.</p>
<h2 id="heading-explore-linguistic-features">Explore Linguistic Features</h2>
<p>We can also count simple language features. For example, we can count how often first-person pronouns appear and look for phrases such as “maybe”, “perhaps”, and “I guess”. These are examples of hedging phrases, which people sometimes use when they are unsure or speaking cautiously.</p>
<p>Add this function to <code>src/preprocessor.py</code>:</p>
<pre><code class="language-python">def extract_linguistic_features(text: str) -&gt; dict:
    doc = get_nlp()(text)
    first_person = {
        "i", "me", "my", "mine", "myself", "we", "us",
        "our", "ours", "ourselves"
    }
    third_person = {
        "he", "him", "his", "himself", "she", "her", "hers",
        "herself", "they", "them", "their", "theirs",
        "themselves", "it", "its", "itself"
    }
    second_person = {"you", "your", "yours", "yourself", "yourselves"}

    pronouns = [token.text.lower() for token in doc if token.pos_ == "PRON"]
    first = sum(word in first_person for word in pronouns)
    third = sum(word in third_person for word in pronouns)
    second = sum(word in second_person for word in pronouns)
    total = first + third + second

    hedging_phrases = (
        "maybe", "perhaps", "possibly", "i guess", "not sure",
        "sort of", "kind of", "i think", "might"
    )
    lower = text.lower()
    hedging_count = sum(lower.count(phrase) for phrase in hedging_phrases)

    return {
        "first_person_count": first,
        "third_person_count": third,
        "second_person_count": second,
        "pronoun_count": total,
        "first_person_ratio": round(first / total, 3) if total else 0.0,
        "hedging_phrase_count": hedging_count
    }
</code></pre>
<p><code>token.pos_ == "PRON"</code> filters tokens by their part-of-speech tag. spaCy assigns this tag to words it identifies as pronouns. The three sets <code>first_person</code>, <code>third_person</code>, and <code>second_person</code> define which lowercase words belong to each group. The generator expression inside <code>sum()</code> checks whether each pronoun token's text belongs to that set and counts how many match.</p>
<p><code>first_person_ratio</code> divides the first-person count by the total pronoun count. The <code>if total else 0.0</code> guard prevents a division-by-zero error when no pronouns appear in the text.</p>
<p>For hedging phrases, <code>text.lower()</code> converts the whole note to lowercase before searching, so "Maybe" and "maybe" are treated the same. <code>lower.count(phrase)</code> returns the number of times that phrase appears as a substring. The result for each phrase is added to <code>hedging_count</code>.</p>
<p>These numbers are only counts. They do not explain why someone used a particular word or phrase. Their meaning depends on the context and the person who wrote the note. To understand whether these features are useful, we would need to compare them with carefully labelled examples. That is outside the scope of this tutorial.</p>
<h2 id="heading-compare-results-across-sessions">Compare Results Across Sessions</h2>
<p>Each session produces a set of results. We’ll use pandas to place the results in a table, with one row for each session. This makes it easier to compare the results over time.</p>
<p>In <code>src/tracker.py</code>:</p>
<pre><code class="language-python">import pandas as pd

def build_session_timeline(session_results: list[dict]) -&gt; pd.DataFrame:
    rows = []
    for result in session_results:
        sentiment = result["sentiment"]
        emotions = result["emotions"]
        linguistic = result["linguistic_features"]
        rows.append({
            "session": result["session"],
            "sentiment_index": sentiment["aggregate_score"],
            "sentiment_label": sentiment["aggregate_label"],
            "dominant_emotion": emotions["session_dominant_emotion"],
            "first_person_ratio": linguistic["first_person_ratio"],
            "hedging_phrase_count": linguistic["hedging_phrase_count"],
            "sentence_count": result["sentence_count"]
        })

    columns = [
        "session", "sentiment_index", "sentiment_label",
        "dominant_emotion", "first_person_ratio",
        "hedging_phrase_count", "sentence_count"
    ]
    if not rows:
        return pd.DataFrame(columns=columns)
    return pd.DataFrame(rows).sort_values("session").reset_index(drop=True)

def describe_sentiment_shift(df: pd.DataFrame) -&gt; dict:
    if df.empty:
        return {
            "session_count": 0,
            "first_half_mean": None,
            "second_half_mean": None,
            "difference": None,
            "description": "No sessions to compare"
        }

    midpoint = max(1, len(df) // 2)
    first = df.iloc[:midpoint]["sentiment_index"].dropna()
    second = df.iloc[midpoint:]["sentiment_index"].dropna()
    first_mean = float(first.mean()) if not first.empty else None
    second_mean = float(second.mean()) if not second.empty else None
    difference = (
        second_mean - first_mean
        if first_mean is not None and second_mean is not None
        else None
    )

    if difference is None:
        description = "Insufficient non-missing scores to compare halves"
    elif difference &gt; 0.05:
        description = "Higher average model sentiment index in the second half"
    elif difference &lt; -0.05:
        description = "Lower average model sentiment index in the second half"
    else:
        description = "Little difference between the two half-period averages"

    return {
        "session_count": len(df),
        "first_half_mean": first_mean,
        "second_half_mean": second_mean,
        "difference": round(difference, 4) if difference is not None else None,
        "description": description
    }
</code></pre>
<p><code>build_session_timeline()</code> loops through each session result, pulls the fields it needs from the nested dictionaries, and appends a flat dictionary as a row. <code>pd.DataFrame(rows)</code> turns the list of rows into a table. <code>.sort_values("session")</code> orders the rows by session number, and <code>.reset_index(drop=True)</code> replaces the index with a clean sequence starting at zero.</p>
<p>In <code>describe_sentiment_shift()</code>, <code>df.iloc[:midpoint]</code> selects the first half of rows by position. <code>.dropna()</code> removes any rows where <code>sentiment_index</code> is missing before calculating the mean, so a single missing value does not make the whole average <code>NaN</code>. The 0.05 threshold keeps the description from calling a very small numerical difference meaningful.</p>
<p>The description refers only to the model's sentiment index. We avoid labels such as "strong improvement" or "clinical deterioration" because a change in a model's score does not prove that someone's mental health or therapy progress has changed.</p>
<h2 id="heading-visualize-the-outputs">Visualize the Outputs</h2>
<p>Charts can make results easier to understand than a table alone. We’ll create four charts: the sentiment index, the most likely emotion predicted for each session, the share of first-person pronouns, and the number of hedging phrases.</p>
<p>In <code>src/visualizer.py</code>:</p>
<pre><code class="language-python">import matplotlib.pyplot as plt
import pandas as pd

def plot_dashboard(
    df: pd.DataFrame,
    output_path: str = "images/session_dashboard.png"
):
    if df.empty:
        raise ValueError("Cannot plot an empty session timeline")

    fig, axes = plt.subplots(2, 2, figsize=(12, 8))
    sessions = df["session"]

    axes[0, 0].plot(sessions, df["sentiment_index"], marker="o")
    axes[0, 0].axhline(0, linestyle="--", linewidth=1)
    axes[0, 0].set(
        title="Model sentiment index by session",
        xlabel="Session",
        ylabel="Positive score minus negative score"
    )

    emotion_codes = {
        name: i for i, name in enumerate(
            sorted(df["dominant_emotion"].dropna().unique())
        )
    }
    emotion_values = [
        emotion_codes.get(value, -1)
        for value in df["dominant_emotion"]
    ]
    axes[0, 1].scatter(sessions, emotion_values)
    axes[0, 1].set(
        title="Dominant predicted emotion",
        xlabel="Session",
        ylabel="Emotion category"
    )
    axes[0, 1].set_yticks(
        list(emotion_codes.values()), list(emotion_codes.keys())
    )

    axes[1, 0].plot(sessions, df["first_person_ratio"], marker="o")
    axes[1, 0].set(
        title="First-person pronoun share",
        xlabel="Session",
        ylabel="Share of counted pronouns",
        ylim=(0, 1)
    )

    axes[1, 1].bar(sessions, df["hedging_phrase_count"])
    axes[1, 1].set(
        title="Selected hedging phrases",
        xlabel="Session",
        ylabel="Phrase count"
    )

    fig.suptitle("Exploratory text-analysis outputs (synthetic data)")
    fig.tight_layout()
    fig.savefig(output_path, dpi=160, bbox_inches="tight")
    plt.close(fig)
    return output_path
</code></pre>
<p><code>plt.subplots(2, 2, figsize=(12, 8))</code> creates a figure with a 2 by 2 grid of subplots. It returns the figure object <code>fig</code> and a 2D array <code>axes</code> where <code>axes[row, col]</code> gives each chart.</p>
<p>The sentiment chart uses <code>axes[0, 0].axhline(0)</code> to draw a horizontal reference line at zero. This makes it easy to see at a glance which sessions produced a net-positive or net-negative index.</p>
<p>The emotion chart needs an extra step because Matplotlib scatter plots require numbers, not strings. <code>emotion_codes</code> is a dictionary that maps each unique emotion label to an integer using <code>enumerate</code>. <code>emotion_values</code> converts the dominant emotion column to those integers for plotting. <code>set_yticks()</code> then restores the original label names on the y-axis, so the chart shows words instead of numbers.</p>
<p><code>fig.tight_layout()</code> automatically adjusts spacing between the subplots so titles and labels do not overlap. <code>fig.savefig()</code> writes the finished figure to a file, and <code>plt.close(fig)</code> releases the memory the figure used.</p>
<p>[Figure 2: figure-2.png]</p>
<img src="https://cdn.hashnode.com/uploads/covers/61e7d0230204273bdcb1072f/3371ade6-3161-40e3-bf6c-5f969be7d0c9.png" alt="Figure 2 — Matplotlib output (demo-overview.png)  A three-panel chart showing NLP model outputs for two fictional therapy clients across five sessions each. The top panel is a line chart of sentiment (positive score) over sessions; fictional-A shows higher overall positivity with variation across sessions, while fictional-B trends lower. The middle panel is a heatmap of mean emotion scores per client across seven categories. Fictional-A shows the highest scores in joy (0.39) and fear (0.38), with near-zero disgust, sadness, and surprise. Fictional-B shows the highest score in sadness (0.39), followed by fear (0.23) and anger (0.20), with near-zero disgust and surprise. The bottom panel is a bar chart of mean sentence length per session, showing variation within and between clients." style="display: block;" width="1650" height="1950" loading="lazy">

<p><em>Figure 2. Example dashboard from fictional sessions. The scores and language counts are model outputs and text features, not measures of therapy progress.</em></p>
<p>Each chart shows a different result. The sentiment chart shows the index we calculated from the model scores. The emotion chart shows the category with the highest score in each session. The other two charts show counts or proportions taken directly from the text.</p>
<p>This dashboard is not a report on someone’s progress in therapy. It only displays the results produced by our code and the models we selected.</p>
<h2 id="heading-connect-the-pipeline">Connect the Pipeline</h2>
<p>Next, we’ll connect the different parts of the project using a Python class. This gives us one place to start when we want to analyse a single note or several notes.</p>
<p>In <code>analyzer.py</code>:</p>
<pre><code class="language-python">from src.preprocessor import preprocess_note, extract_linguistic_features
from src.sentiment import analyze_sentiment
from src.emotions import detect_emotions
from src.tracker import build_session_timeline, describe_sentiment_shift

class TherapyNoteAnalyzer:
    def analyze_session(self, note_data: dict) -&gt; dict:
        required = {"session", "client_id", "note"}
        missing = required - note_data.keys()
        if missing:
            raise ValueError(f"Missing required fields: {sorted(missing)}")

        processed = preprocess_note(note_data["note"])
        sentiment = analyze_sentiment(processed["sentences"])
        emotions = detect_emotions(processed["sentences"])
        linguistic = extract_linguistic_features(note_data["note"])

        return {
            "session": note_data["session"],
            "client_id": note_data["client_id"],
            "sentence_count": processed["sentence_count"],
            "sentiment": sentiment,
            "emotions": emotions,
            "linguistic_features": linguistic,
            "entity_candidates": processed["entities"]
        }

    def analyze_all(self, notes: list[dict]) -&gt; dict:
        if not notes:
            timeline = build_session_timeline([])
            return {
                "client_id": None,
                "session_results": [],
                "timeline": timeline,
                "sentiment_comparison": describe_sentiment_shift(timeline)
            }

        client_ids = {item.get("client_id") for item in notes}
        if len(client_ids) != 1:
            raise ValueError(
                "This example expects notes for one synthetic client at a time"
            )

        results = [self.analyze_session(item) for item in notes]
        timeline = build_session_timeline(results)
        return {
            "client_id": next(iter(client_ids)),
            "session_results": results,
            "timeline": timeline,
            "sentiment_comparison": describe_sentiment_shift(timeline)
        }
</code></pre>
<p>Finally, in <code>main.py</code>:</p>
<pre><code class="language-python">import json
from pathlib import Path
from analyzer import TherapyNoteAnalyzer
from src.visualizer import plot_dashboard

def main():
    with Path("data/synthetic_notes.json").open(encoding="utf-8") as file:
        notes = json.load(file)

    results = TherapyNoteAnalyzer().analyze_all(notes)
    print(results["timeline"].to_string(index=False))
    print("\nDescriptive comparison:", results["sentiment_comparison"])

    output = plot_dashboard(results["timeline"])
    print(f"\nSaved dashboard to {output}")
    print("Reminder: these model outputs are exploratory, not clinical assessments.")

if __name__ == "__main__":
    main()
</code></pre>
<p>Run the project from its root directory:</p>
<pre><code class="language-python">python main.py
</code></pre>
<p>The first time you run the script, it will download both Hugging Face models. It will then print a table of the session results and save a dashboard image in the images folder.</p>
<p>The results may differ depending on the model version and how the code is run. Always check the results from your own run. Say what the model predicted, rather than saying what a session “showed” or “revealed”. A model prediction is not proof of what happened.</p>
<h2 id="heading-test-the-project">Test the Project</h2>
<p>Testing helps us check whether our functions work with normal inputs and unusual cases, such as empty text. Passing these tests does not prove that a model is accurate or useful in a clinical setting.</p>
<p>For the sentiment and emotion tests, we’ll use fake classifiers that return results we choose in advance. This keeps the tests quick and means we do not need to download the model files.</p>
<p>A simplified sentiment test looks like this:</p>
<pre><code class="language-python">from src.sentiment import analyze_sentiment

class FakeSentiment:
    def __call__(self, texts, **kwargs):
        return [
            [
                {"label": "POSITIVE", "score": 0.8},
                {"label": "NEGATIVE", "score": 0.2}
            ],
            [
                {"label": "POSITIVE", "score": 0.1},
                {"label": "NEGATIVE", "score": 0.9}
            ]
        ][:len(texts)]

def test_sentiment_score_is_positive_minus_negative():
    result = analyze_sentiment(
        ["Good sentence.", "Bad sentence."],
        classifier=FakeSentiment()
    )
    assert result["sentence_results"][0]["signed_score"] == 0.6
    assert result["sentence_results"][1]["signed_score"] == -0.8
</code></pre>
<p><code>FakeSentiment</code> works as a drop-in replacement for the real model because it defines <code>__call__</code>. When Python sees <code>model(clean, ...)</code>, it calls <code>model.__call__(clean, ...)</code>, so any object with that method behaves like a callable. <code>FakeSentiment.__call__</code> returns a fixed list of scores regardless of the input text, which means the test checks the calculation logic without downloading the model.</p>
<p><code>[:len(texts)]</code> slices the return value to match the number of input sentences. This prevents the fake classifier from returning more results than there are inputs, which would cause the <code>zip()</code> in <code>analyze_sentiment</code> to behave unexpectedly.</p>
<p>Passing <code>classifier=FakeSentiment()</code> to <code>analyze_sentiment</code> overrides <code>get_pipeline()</code> for that call. This pattern is called dependency injection: the function accepts its dependency from outside rather than creating it internally, which makes the function easier to test and reuse.</p>
<p>The full project includes tests for empty inputs, sentence splitting, word handling, sentiment score calculations, emotion score summaries, sorting sessions, empty timelines, and checking that required input fields are present.</p>
<p>Run the test suite from the project root:</p>
<pre><code class="language-python">python -m pytest -q
</code></pre>
<p>The project archive contains 15 tests, and all 15 passed when the project was checked. Some tests use made-up model results, while others use a small spaCy setup to check selected text-processing steps. These tests do not measure model accuracy, confirm that the emotion labels are correct, or show how the models perform on real therapy notes.</p>
<h2 id="heading-limitations-and-ethical-considerations">Limitations and Ethical Considerations</h2>
<p>Because this project uses therapy-related text, we need to think carefully about privacy, the limits of the models, and the difference between language patterns and a professional clinical assessment.</p>
<h3 id="heading-general-domain-models-are-not-clinical-models">General-domain models are not clinical models</h3>
<p>The DistilBERT model was trained on SST-2, not on therapy notes. The DistilRoBERTa model was trained on a mix of general English text. We do not know how well either model works on clinical notes unless it is tested separately using suitable data and clear labels. The models may also handle formal writing, writing by people who speak English as an additional language, and different cultural ways of expressing distress poorly. Their training data was not designed to cover all these situations.</p>
<h3 id="heading-sentiment-is-not-therapeutic-progress">Sentiment is not therapeutic progress</h3>
<p>A positive sentence does not prove that a client is getting better. A negative sentence does not prove that therapy is failing. A note may contain the therapist’s view, the client’s own words, a difficult event, or a mix of these. The model may not be able to tell the difference.</p>
<h3 id="heading-linguistic-features-are-not-psychological-diagnoses">Linguistic features are not psychological diagnoses</h3>
<p>Pronoun counts and hedging phrases are simple features found in the text. A change in these counts does not prove that a client is withdrawing, avoiding a topic, losing trust, or having a problem with the therapist. We removed the rupture score from an earlier version of this project because its scoring rules and alert level had not been tested or approved for clinical use.</p>
<h3 id="heading-privacy-requirements-depend-on-jurisdiction">Privacy requirements depend on jurisdiction</h3>
<p>In the United States, HIPAA applies to certain healthcare organisations and their business partners. Other countries have their own data protection and healthcare privacy laws. Before using real records, an organisation must check the laws that apply, its own policies, the legal reason for using the data, any required consent, the services and vendors involved, security measures, and how long the data will be kept. This tutorial is not a substitute for that review. Even if HIPAA does not apply where you live, that does not mean it is safe or ethical to process real therapy notes.</p>
<h3 id="heading-human-oversight-and-transparency">Human oversight and transparency</h3>
<p>This project is not made for clinical decisions, and it should not be used as the basis for them. Any NLP tool considered for real healthcare work would need to be tested for that specific setting. It would also need privacy and bias reviews, clear documentation, and oversight from qualified professionals. People whose information is used should be told about it as required by the relevant rules.</p>
<h2 id="heading-conclusion-and-further-learning">Conclusion and Further Learning</h2>
<p>In this project, we built an NLP workflow with spaCy, Hugging Face Transformers, pandas, and Matplotlib. Using fictional therapy notes, we showed how to split text into sentences, classify sentiment and emotion, count language features, compare sessions, create charts, and test the code.</p>
<p>NLP can help turn large amounts of written text into data that we can organise and explore. But the results have limits, especially when we use a general-purpose model on a sensitive type of text it was not trained or tested to understand.</p>
<p>If you want to build on this project, here are a few ideas:</p>
<p>Try different sentiment models and compare their predictions. Note where they agree and where they differ.</p>
<p>Before making claims about model performance, create a dataset with human-checked labels and get the proper ethical approval to use it.</p>
<p>If you have correct labels to compare against, use measures such as precision, recall, F1-score, and a confusion matrix to check the model’s results.</p>
<p>Explore clinical NLP tools such as scispaCy. Remember that tools designed for medical text still need to be tested for the task you want to use them for.</p>
<p>Build a Streamlit app that lets people try the project with fictional examples only.</p>
<p>To learn more, read the spaCy documentation, the DistilBERT sentiment model card, and the DistilRoBERTa emotion model card. You can also read Edward S. Bordin’s 1979 paper, “The Generalizability of the Psychoanalytic Concept of the Working Alliance”, for background on the working relationship between a therapist and client.</p>
<p>NLP can help us study large amounts of text. But using it responsibly means understanding the data, knowing what a model was trained to do, and being clear about what its results cannot tell us.</p>
<p>If you'd like to explore the project further, you can check out the <a href="https://github.com/Samuellyworld/therapeutic-alliance">GitHub repository</a>. I've included the code and project structure there, so you can see how everything works. It's also a good starting point if you're curious about how Python and NLP apply to areas like healthcare, health tech, clinical NLP, and AI. There's still a lot to explore in this space, and I hope the project gives you a few ideas to build on.</p>
<h2 id="heading-references">References</h2>
<ul>
<li>Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance. <em>Psychotherapy: Theory, Research &amp; Practice, 16</em>(3), 252–260. <a href="https://doi.org/10.1037/h0085885">https://doi.org/10.1037/h0085885</a></li>
<li>Hartmann, J. (2022). <em>Emotion English DistilRoBERTa-base</em>. <a href="https://huggingface.co/j-hartmann/emotion-english-distilroberta-base">Hugging Face Model Hub</a>.</li>
<li>Hugging Face. (n.d.). <em>DistilBERT base uncased fine-tuned on SST-2</em>. <a href="https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english">Hugging Face Model Hub</a>.</li>
<li>Hugging Face. (n.d.). <em>Transformers documentation</em>. <a href="https://huggingface.co/docs/transformers">https://huggingface.co/docs/transformers</a>.</li>
<li>Matplotlib Development Team. (n.d.). <em>Matplotlib documentation</em>. <a href="https://matplotlib.org/stable/">https://matplotlib.org/stable/</a>.</li>
<li>pandas. (n.d.). <em>pandas documentation</em>. <a href="https://pandas.pydata.org/docs/">https://pandas.pydata.org/docs/</a>.</li>
<li>pytest Development Team. (n.d.). <em>pytest documentation</em>. <a href="https://docs.pytest.org/">https://docs.pytest.org/</a>.</li>
<li>Sanh, V., Debut, L., Chaumond, J., &amp; Wolf, T. (2019). <em>DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter</em>. <a href="https://arxiv.org/abs/1910.01108">arXiv:1910.01108</a>.</li>
<li>spaCy. (n.d.). <em>Usage documentation: Language processing pipelines</em>. <a href="https://spacy.io/usage/processing-pipelines">https://spacy.io/usage/processing-pipelines</a>.</li>
</ul>
 ]]>
                </content:encoded>
            </item>
        
    </channel>
</rss>
