Delivered for a government agency, reached through an enterprise partner, through Platform to Platform and Data to Data.
The challenge. A government agency had four separate streams of citizen opinion — news coverage, complaints, feedback forms and phone surveys — and no way to turn any of them into a comparable read on sentiment for a given ward.
Context
Between 2018 and 2019 we built a sentiment analytics platform for a government end user, delivered through an enterprise partner. It reads citizen sentiment across a city and reports it at ward level.
The questions it was built to answer are operational, not academic: how do residents of this ward regard the public services they receive, which areas are being described as unsafe, and where should a facility or an intervention go.
The problem
Agencies are not short of citizen opinion. They are short of comparability.
Opinion arrives through four channels that have nothing in common. News coverage on TV and in newspapers is edited, published at the city level, and about events rather than about services. Citizen complaints are specific, located, and skewed towards people motivated enough to complain. Feedback forms are structured but sparse and collected only where someone thought to collect them. Automated phone surveys and call logs are the broadest sample and the noisiest text.
Each channel has its own bias, its own geography and its own format. Read separately, each supports a different conclusion. There was no mechanism for turning all four into one number for a ward that a decision could be made against.
Why it is hard
The text is not standard. This is the substance of the problem. Citizen feedback in a large Indian city arrives in regional dialects, in transliteration, in code-switched sentences, and in colloquial constructions that off-the-shelf sentiment models score wrongly and confidently. A phrase that is idiomatic praise in one district reads as neutral or negative to a model trained on formal text. Sarcasm about a public service is common and is scored as positive.
Sentiment about a service is not sentiment about a topic. “The road was fixed quickly” and “The road was finally fixed” have the same topic and opposite implications for the agency. Sentence-level scoring has to survive that.
Geography has to be inferred, not read. A complaint has an address; a news article about a district does not; a phone survey has a caller region. Getting every channel down to a ward requires a different method per channel, and the error bars differ accordingly.
Aggregate sentiment is boring until it moves. A stable ward score is not actionable. The signal an agency can act on is a shift, which means the platform has to detect change rather than report level.
What we built
Multi-channel ingestion and normalisation. News, complaints, feedback forms and telephony feeds ingested in near real time and normalised into one internal text and metadata structure with a resolved geography.
A purpose-built Java NLP engine. Parsing, tokenisation, sentence-level positive, negative and neutral scoring, plus keyword and tone-indicator extraction. We wrote the engine rather than adopting a general-purpose sentiment library because the geography and tone requirements were specific enough that tuning someone else’s pipeline would have been the longer route.
A model ensemble, plus rules for the parts models get wrong. Naive Bayes, logistic regression and neural networks trained on the labelled output, and alongside them a rule-based scoring layer for regional dialects and colloquial phrasing. The rule layer is not a fallback for an immature model. It is the correct tool for a construction that appears frequently in one district and nowhere else, where there will never be enough labelled examples to learn it and where a linguist can write the rule in an afternoon. Retaining that layer is why the accuracy figure held across regions.
Bubble maps and drill-down at three levels. Processed sentiment integrated into the agency’s SAS analytics platform, rendering as bubble maps at city, district and ward level, with drill-down by time range, region and subject matter.
Anomaly alerts on sentiment shift. Sudden movement in a ward’s score raises an alert rather than waiting to be noticed in a monthly report.
Stack: Java with purpose-built NLP and ML libraries for processing, Python with scikit-learn and Weka for model training, SAS for visualisation and enterprise reporting, deployed across on-premise and hybrid cloud infrastructure.
Results
92% sentiment classification accuracy at sentence level across the four channels.
Four channels became one comparable score per ward, which is the outcome the agency wanted and the reason the normalisation layer took as long as the models.
Dialect and colloquial handling held up across regions, because the rule layer covers what the statistical models could not learn from the available labels.
Sentiment shifts raise alerts rather than appearing in retrospect.
What we would do differently
We would separate topic extraction from sentiment scoring rather than running them in one pass. Agencies want to know what the sentiment is about at least as much as whether it is negative, and retrofitting topic classification onto a pipeline built for polarity meant reprocessing history.
We would also carry per-channel confidence through to the map. A ward score assembled mostly from phone surveys and a ward score assembled mostly from located complaints are not equally reliable, and presenting both as the same size bubble understates how much the underlying sample matters.
Where this transfers
Normalising several biased channels into one comparable score for a geography, handling non-standard language with rules where models cannot learn it, and alerting on change rather than level is the same pipeline as brand and reputation monitoring, community trust measurement for policing, and customer experience analytics across reviews, surveys and support contacts.