Automations, proofs of concept (POCs) and research. A look at what we built, why it mattered and what we learned.
E-commerce · Multi-brand fashion retailer
~80% less time spent tagging new products
Product tags across 6 categories, ready for review
Tagged product photos in bulk, let staff correct mistakes and exported the results for Shopify.
READRetail store expansion · Hypermarket retailer
A ranked shortlist of new store locations
Potential to cut hours of initial location research
Ranked areas for new stores using local demand, costs, existing stores and nearby competitors.
READFinancial media · Business intelligence firm
A daily social sentiment score per company
Filtered for bots and sarcasm, traceable to tweets
Scored company-related tweets with a finance-trained model and turned them into a daily signal for analysts.
READMedia research · News aggregator
Less collecting. More analysis.
Relevant articles and structured findings in one place
Collected articles and organised their analysis around a research framework, with source links for human review.
READClinical research · Research team
From papers to comparison
Study details and results organised for researcher review
Screened studies, extracted results and ran combined analyses after approval, keeping researchers in control.
READHigh-performance computing · Academic lab
Tested across 1–32 processes
Measuring runtime and gains from extra computing power
Tested AI-written code that split established route-finding methods across processors on a supercomputer.
READA multi-brand fashion retailer had to tag thousands of new products a week before they could go live online. Tagging was manual, done in a spreadsheet, and falling behind.
A tagging tool was built around an open-source, fashion-specific image-text model, fine-tuned on the retailer's own past tags. For each product photo, it suggested a value for each of the retailer's six categories. A staff member checked them, fixed any mistakes, and exported a file ready for bulk import.
Weekly tagging time dropped by about 80%. The tool ran on a standard laptop, had no per-image fees, and improved from every correction.
Every new product needed six categories tagged before it could be listed:
These categories drive a store's menus and filters. A product without them can't be found. A product with the wrong ones gets found by the wrong shoppers.
The first three form a hierarchical taxonomy (each level narrows down the one above it). Casual Shoes can only sit under Footwear > Shoes.
Staff tagged each product by hand. They opened the photo from a folder, then typed six values into a spreadsheet row with the same image ID. At the retailer's volume, this added up to a full-time job. The team fitted it around other work, and new stock waited in a backlog instead of being on sale.
The options they had looked at each had a catch:
The retailer did have one strong asset: about 6,000 product images its staff had already tagged by hand, carefully and in exactly the retailer's format. This data became the most valuable input of the project.
A tagging tool with a human review step. Staff uploaded a batch of product photos. The model suggested a value for each of the six categories. A staff member scrolled through, fixed anything wrong, and exported the results. Every fix was saved and used to fine-tune the model further.
The model was first run as it comes, with the raw tag names as text. Broad categories like master category were already reliable. Season and gender were the weak spots, and article type confused close neighbours like shirts and tops.
This became the baseline. Every later change had to beat it on the test set to stay.
Bare values like "Men" or "Fall" were ambiguous to the model. A photo could match "Men" because a man was wearing the product, not because it was a men's product. "Fall" on its own could mean many things.
These values were rewritten as short product descriptions, built around the article type the model had already predicted. "Men" became "a men's shirt". "Fall" became "a shirt for autumn". A few phrasings per value were also averaged, a standard prompt engineering technique for this kind of model. Season and gender both improved.
The model understood fashion, but not this exact tag set.
The Fashion Product Images dataset on Kaggle uses almost the same six categories, on similar products. Its labels are noisy in places, season especially. Fine-tuning on it still moved the model closer to the retailer's way of tagging.
The retailer had clean data for about 6,000 product images, tagged by hand in the same spreadsheet format.
Fine-tuning on them gave the biggest improvement. These images showed exactly how this team applied each tag, including the judgment calls. Is a light jacket Winter or Fall? Is a plain white sneaker Men or Unisex? No public dataset could teach that.
The review dashboard was built with Streamlit. Each product got a card with the photo and six dropdowns showing the model's picks. Dropdowns only allowed approved values, and staff could type to search them. This kept the tag vocabulary clean.
The export followed Shopify's product CSV format, so it could go straight into a bulk import alongside the product details the retailer already had.
Every fix was logged with the product, category, the model's pick, the correct value, the model's confidence and the time. After every few thousand reviewed products, the model was fine-tuned again on these corrections.
The first attempt used the corrections alone. The model stopped making those exact mistakes, but got worse overall on the test set. Corrections were mostly hard cases, and fine-tuning only on them made the model lose what it already did well (catastrophic forgetting).
This was fixed with experience replay, which meant mixing a sample of the original 6,000 hand-tagged images into every fine-tuning run. Accuracy recovered and then went past its previous best.
A newly fine-tuned model only replaced the current one if it scored higher on the test set.
From the first baseline to the working tool, the build took about a month.
Weekly tagging time dropped by about 80%.
Checking a suggested row took a fraction of the time of tagging from scratch. Most products needed no edits. Season remained the category that needed fixing most often. A typical week's new stock should fit into a single working day.
Beyond time saved:
What this added beyond the options already available:
| Paid tagging tools | Frontier model API | Open-source model as is | This tool | |
|---|---|---|---|---|
| Uses the retailer's own tags | With an add-on or custom plan | Yes, given as instructions | Yes | Yes |
| Accurate enough to rely on | Varies, but mostly yes | Yes, with good instructions | No | Yes |
| Running cost | Per image or subscription | Low, per image | None | None |
| Product photos stay in-house | No | No | Yes | Yes |
| Learns from staff corrections | Varies | Some models, through paid fine-tuning | No | Yes |
Limitations
Way ahead
They made the biggest difference here. If you have none, a frontier AI model through an API may be the simpler start. Some of these can also learn from your corrections. OpenAI offers fine-tuning of GPT-4o on images, and Google offers fine-tuning of Gemini on Vertex AI. This happens on the provider's platform and at extra cost. The other route is to start with an open model and a public dataset, and let the review step build your tagged data over time.
No. Here, tagging and regular fine-tuning ran on the team's existing laptop (a MacBook Air with an Apple M3 chip and 16 GB of memory). A similar recent laptop should work too. A cloud GPU (like Google Colab) only helped speed up experiments during the build.
Most support custom tag sets, but usually as an add-on or custom plan priced through sales, not on basic plans. Product photos also go to the vendor. At this volume, a small in-house model was the cheaper fit and kept photos in-house. Depending on your needs, a paid tool may still be the simpler and better option. Not every business needs its own model.
Yes. They're accurate out of the box, cheap per image, and some can be fine-tuned on your own tagged photos. The trade-offs are that every photo goes to an outside provider, and the provider controls model updates and retirements. If neither matters to you, an API may well be the simpler and better choice. This retailer wanted an in-house model, so that's what was built.
It doesn't go live. Each new version should beat the current one on the same test set.
A hypermarket retailer needed a better way to decide where to open its next stores. Expansion planning leaned on spreadsheets, broker packs and experience. Only a small part of the map could be studied in detail.
A two-part pipeline was built. A machine learning model learned the pattern of where the retailer's stores already are and scored every area against it. An optimiser then turned those scores, plus estimated store financials, into a ranked shortlist of new locations, keeping new stores far enough apart to avoid stealing each other's customers. Everything was shown on interactive maps.
Planners could test scenarios in minutes and start every expansion discussion from an evidence-based shortlist instead of a blank map.
Choosing a new store location is one of the most expensive decisions a retailer makes. A good location pays back for decades. A poor one can drag down nearby stores too.
Planners weigh many questions, from road access, visibility and parking to lease terms and local planning rules. Most of those are checked on the ground. Four of them depend on data and are hard to answer at scale:
Planners answered these area by area. Each candidate took days of research, so only a handful could be studied properly. Promising areas outside the usual search patterns were easy to miss.
The options already available were:
The retailer did have one strong asset: a long history of where its stores had been opened, across many very different markets. That history held patterns no planner could spot by hand.
A pipeline in two parts. The first part, the location model, is a machine learning model. It learned the pattern of where the retailer's stores already are and scored every area on how closely it matched. The second part, the network optimiser, is not machine learning. It applied clear, adjustable rules to the areas scoring above a confidence threshold, estimated store financials for each, and picked the requested number of new locations while keeping them a minimum distance apart. Planners reviewed the shortlist and decided what to visit.
The location model was given a simple task. For each area, it had to predict whether the retailer already had a store there. The idea was that years of past opening decisions, made by experienced planners, carry patterns about what a good area looks like.
This framing has a known weakness. An area without a store isn't necessarily a bad area. It may simply never have been tried. With accurate store financials, the better approach is to train the model on how well stores perform instead (see the FAQs).
The location model's first results looked excellent on paper. They weren't. Most areas have no store, so a model that predicts "no store" almost everywhere still scores high on accuracy.
The three model types were then compared on a balanced sample, where some no-store areas were randomly left out (undersampling), so none of them could look good just by favouring "no store". All of them were judged on precision and recall rather than accuracy. A confidence threshold was exposed in the dashboard, so planners could choose between a short, high-confidence list and a longer, more exploratory one.
Early versions of the location model leaned heavily on population. Bigger places got higher scores, almost regardless of anything else. That is true but not useful, since every planner already knows big places have more shoppers.
Ratio features fixed much of this. Population per competitor store, income relative to rent and competitor pressure scores gave the model ways to tell apart two areas of the same size. Population stayed the strongest single driver, but no longer the only one.
Neighbouring areas tend to look alike. If a model is tested on areas scattered among the ones it learned from, it can do well just by recognising the neighbourhood, which makes it look better than it really is. So the location model was also trained on some regions and tested on regions it had never seen (spatial cross-validation).
Counting only exact matches also undersold the model. A pick counted as wrong if it landed in the postal code area next to a real store, even though both serve much the same shoppers. So a distance check was added. It measured how far each pick was from the nearest real store, and many "wrong" picks were within a few kilometres of one.
A second check looked at competitors. The share of picks with a competitor store nearby was close to the same share for the retailer's real stores, so the model had picked up how the retailer positions itself.
All of this was visible on the maps. Their key could toggle actual stores, predicted stores, correctly predicted stores and competitor stores. A confidence view shaded each area by how sure the model was, and hovering over an area showed its population and income.
Income and rent data were missing for some areas. Dropping those areas would have left holes in the map. Filling them silently would have made guesses look as solid as real figures.
The gaps were filled using CatBoost. It was trained on the areas where income and rent figures were known, using their other figures such as population, density and region, and then predicted the missing values. Every imputed value was flagged, and the optimiser applied an adjustable penalty to those areas, so a location built on estimates had to clear a higher bar.
A map of scores isn't a decision. The XGBoost location model's scores were passed to the network optimiser (because it performed the best), which turned them into one in three stages:
Planners controlled the inputs. These were how many stores to open, which format, whether to optimise for revenue, profit or market share, the minimum distance between stores, the budget, and whether to search everywhere or within a single region. The final shortlist appeared on a map with a ranked table, and could be downloaded.
From the first data pull to the working pipeline, the build took about two months.
Expansion planning could start from a ranked, explainable shortlist instead of a blank map.
With real store financials plugged in, the same shortlist should also carry much more reliable revenue and profit estimates.
What this added beyond the options already available:
| Manual analysis and consultants | Location intelligence platforms | Gravity model in GIS software | This pipeline | |
|---|---|---|---|---|
| Screens every area at once | Rarely, due to time | Yes | Usually one candidate per run | Yes |
| Learns from the retailer's own store history | Through experience | Varies by platform | Partly, weights can be calibrated from customer data | Yes |
| Rich data such as foot traffic | Varies | Yes, often their main strength | Only if added | Not in this POC |
| Shows why an area scored well | Yes | Varies | Yes | Yes |
| Ongoing cost | Fees per project | Subscription and data licences | Software licence | Low, but needs upkeep |
| Ready to use without building | Yes | Yes | Mostly | No, needs building |
Limitations
Way ahead
No. It decides which locations are worth visiting first.
Often, yes. They come with rich data such as foot traffic, ready-made scoring and support, and nothing needs building. A custom pipeline makes more sense when you want the scoring built on your own store history, full control over the logic, or lower long-term costs. Depending on your needs, a platform may well be the simpler and better choice.
Most serious teams combine several methods. Catchment analysis shows who lives within reach of a location. Sales forecasting models learn from existing stores' performance and apply it to candidates. Gravity models estimate how much a new store will take from nearby stores. Site visits come last. This pipeline covers the screening and shortlisting part well. With real store financials, its location model could become a sales forecasting model too.
Yes, if accurate figures are available. It is the most important upgrade to this pipeline. Real figures from existing stores are used once, to train the location model. Instead of learning whether a store exists in an area, it learns how much a store there sells (or earns). It then predicts those figures for every candidate area, and the optimiser works from those predictions instead of formula-based estimates. New locations don't have real financial figures yet, only predicted ones, so the quality of those predictions is what matters.
With such a model, the market potential stage in 2.2.6 would mostly no longer be needed, since the predictions already reflect local demand. A cannibalisation check would still be needed, though. A model that predicts a new store's own sales doesn't measure how much it takes from the retailer's nearby stores.
This POC didn't train on estimated financials, because the model would only have learned the assumptions behind the estimates.
To score a map, it has to be split into units. Postal code areas are a common choice because a lot of public data is published by postal code. But they were drawn for mail delivery, not shopping, and they vary hugely in size. Two alternatives are widely used. A hexagon grid such as Uber's H3 splits the map into equal-sized cells, which are easier to compare, if your data is available at that level. Drive-time catchments cover the area reachable within a set number of minutes by road, which is closer to how shoppers actually travel, but they need road network data. Many teams screen with hexagons and use drive-time catchments for the final shortlist.
Enough for the model to see a wide range of locations, good and bad. A retailer with only a handful of stores doesn't have enough history to learn from. In that case, comparing candidates to your best existing stores, a gravity model, or a location intelligence platform with data on comparable retailers is a better start.
Not on their own. Frontier models were used during the development of this pipeline, for example to help write and debug code, and they can speed up a lot of the work, such as cleaning data or summarising each shortlisted area for planners. But they aren't specialist tools for spatial analysis or for learning patterns from large tables of location and store data. The scoring needs purpose-built models trained on your own stores and local data, and people with the skills to build, check and maintain them. Frontier models make that work faster. They don't replace it.
A financial media and business intelligence firm wanted to know what social media was saying about listed companies, and whether it was worth paying attention to. Its analysts serve wealth managers and needed a consistent, daily measure they could rely on.
A pipeline was built that scored each tweet about a company as bullish, bearish or neutral with a finance-trained AI model. It filtered out bots and spam, flagged likely sarcasm, and combined everything into one sentiment score per company per day. A dashboard showed analysts how that score was moving, when it shifted sharply, and which tweets drove each shift.
Analysts got a daily, explainable read of social sentiment for each company, with no price quotes or trading calls. Testing showed the signal was better at warning that a stock was likely to make an unusually large move than at saying which way. Sarcasm remained the hardest problem, so likely sarcastic tweets were counted at a reduced weight, and their share was shown separately as a sign that a day's score was less reliable.
Investors talk about companies on social media all day. Some of it reacts to news, some of it runs ahead of news, and most of it is noise. Analysts serving wealth managers are regularly asked what the mood around a company is, and whether a sudden burst of chatter matters.
With thousands of tweets a day for a single large company, reading them is not feasible. Sampling a few gives an impression that can't be compared from one day to the next and is easily swayed by a handful of loud accounts.
What was needed was a daily measure of social sentiment for each company, an alert when it shifted sharply, and a way to see the tweets behind it. Equally important was an honest answer to whether that measure carried any useful information at all.
The options already available were:
The firm wanted a method it could see into and control, built on open-source models, before deciding whether to invest further or buy a feed.
A pipeline that took tweets mentioning a company, removed bots, spam and duplicates, and scored each remaining tweet as bullish, bearish or neutral. A second model flagged tweets that were likely sarcastic. The scores were then combined into one sentiment score per company per day, weighted by how much engagement each tweet got. A dashboard showed the score over time, highlighted sharp shifts and listed the tweets behind them. Separately, the score was tested against past market data to see what it could and couldn't tell analysts.
The first version scored tweets with VADER, a well-known word-list tool built for social media. It was quick to set up and gave a baseline to beat.
It struggled with finance. Words like "short", "puts", "bear" and "dip" carry a market meaning that a general word list doesn't know, and "buying the dip" reads as positive to an investor but not to a dictionary. Sarcasm got through untouched. On the labelled sample, VADER often got the label wrong, especially on bearish tweets.
Before any model could be compared, it needed an answer key, which was the labelled sample described in 2.1. Having a frontier model label a few thousand tweets once was cheap and fast, and checking its labels took far less time than labelling from scratch. Labelling millions of tweets the same way, every day, would not have been, so the frontier model built the answer key and an open-source model did the scoring.
Transformer models read a whole sentence at once, so they pick up context that word lists miss. Two open-source options were compared on the labelled sample. FinBERT, a widely used finance model (trained on financial news), was a clear step up from VADER but stumbled on tweet slang, symbols and emojis. FinTwitBERT-sentiment, trained on financial tweets, matched the labelled sample most closely, so it was the one used.
Its labels also suited the job. Bullish and bearish say what a tweet implies about the stock, which is what analysts care about. A tweet can be negative in tone ("shorts are going to get destroyed") and still be bullish.
A large share of tweets about big companies weren't really about them. Research on stock tweets has shown that groups of bots attach the symbols of popular companies to tweets promoting unrelated, low-value stocks (cashtag piggybacking). These tweets added volume and distorted the score.
Three simple rules removed most of it. Tweets listing many stock symbols at once were dropped, exact and near-duplicate tweets were collapsed into one, and no single account could count for more than a few tweets per company per day. Tweets that mentioned two or three companies were kept but counted at a reduced weight, since one sentiment label can't always apply to all of them. Every filtered tweet was logged with the reason, so the rules could be checked.
Sarcasm was the hardest problem. "Great, another recall. Can't wait for earnings" is bearish, but it opens like a cheerful tweet. The finance-trained model handled some of these from context, but not reliably, and sarcastic tweets cluster around bad news, which is when an accurate score matters most.
The irony model was added to flag them. Used as it was, it flagged far too much. It had been trained on general tweets, so excited investor posts ("to the moon 🚀") often looked ironic to it. Its threshold was raised using the sarcastic tweets in the labelled sample until most flags were correct.
The first fix then backfired. Flipping the label of every flagged tweet, from bullish to bearish and back, made the scores agree less with the labelled sample, not more. The finance-trained model had often already read a sarcastic tweet correctly from its context, so flipping turned right answers into wrong ones. Some flags were also still wrong. So flagged tweets kept their label but counted at a reduced weight. The share of sarcastic tweets was shown separately for each day, so analysts knew when the score was resting on shaky ground.
Each company got a daily score from −1 (all bearish) to +1 (all bullish). Tweets with more likes, retweets and replies counted for more, on a logarithmic scale, so one viral tweet couldn't outweigh hundreds of others. Low-confidence labels counted for less.
Two details mattered more than expected. Tweets posted after the US market closed at 4 pm New York time were counted towards the next trading day, since that is the first day they could affect trading. Counting them on the same day would have quietly let the score see the future. And a raw score meant little on its own, because some companies are always discussed more positively than others. So each day was also compared with the company's own last 30 trading days, and a shift was flagged when the score or the number of tweets moved well outside its usual range.
To test whether the score carried information, early versions forecast the next day's share price, with and without sentiment. The error looked impressively small. It wasn't. A forecast that simply repeats yesterday's price scored almost as well, because prices rarely move much in a day. Small errors on price levels say almost nothing about whether a model knows anything.
So the test was reframed around questions that a repeat-yesterday guess can't answer. Would the stock go up or down the next day, and would it move more than usual? Each question was tested with past market data alone, then with the sentiment score added, on 2020 data the models had never seen. That year included the market crash in spring, a tough test for any model.
The results were in line with long-standing research (message board activity predicts trading volume and volatility). Adding sentiment did little for predicting whether a stock would go up or down. It helped more with predicting whether it would make an unusually large move, in either direction. A sharp shift in the score or a burst of tweets showed that something was happening around a company, but not which way the market would take it, partly because prices tend to react to news faster than tweets can signal it. SHAP confirmed these were the sentiment inputs the models relied on. That shaped what the dashboard showed analysts, and what it didn't.
The dashboard showed sentiment, not prices. Analysts picked a company and a date range and saw:
New tweets could be uploaded as a CSV file. They were filtered, scored and combined in the same way, and the daily scores and flagged shifts could be downloaded. Some sources don't include engagement counts (the 2021 to 2022 dataset doesn't), so in that case every tweet counted equally.
From the first data pull to the working dashboard, the build took about three months, in two-week sprints.
Analysts got a daily, explainable read of social sentiment for each company, and an honest picture of what it could and couldn't tell them.
That gave the firm a clear basis for deciding whether to extend the pipeline to more companies and sources, or to buy a commercial feed.
What this added beyond the options already available:
| Social sentiment data feeds | Frontier model API | Word-list scoring | This pipeline | |
|---|---|---|---|---|
| Understands finance language and slang | Varies, but mostly yes | Yes | Limited | Yes |
| Handles sarcasm | Varies | Often, with good instructions | Rarely | Partly, flagged and down-weighted |
| Filters bots and spam | Usually | Only if built around it | Only if built around it | Yes, with simple rules |
| Covers many companies and platforms | Yes, often their main strength | Depends on the data supplied | Depends on the data supplied | Not in this POC |
| Full view of how scores are made | Limited | Partly, prompts are visible | Yes | Yes |
| Ongoing cost | Subscription | Low per tweet, adds up at volume | Very low | Low, but needs upkeep |
| Ready to use without building | Yes | No, needs a pipeline around it | Mostly | No, needs building |
Limitations
Way ahead
Not reliably on its own. In this project, and in much of the published research, sentiment said little about which way a stock would move. It was more useful as a sign that a stock was likely to make an unusually large move, in either direction. Treat it as one input among many, not a forecast.
Often, yes. Feeds cover thousands of companies and several platforms, include years of history and handle bot filtering for you, with nothing to build. A custom pipeline makes more sense when you want full control over the method, need to score your own sources, or want to test the idea before paying for a feed. Depending on your needs, a feed may well be the simpler and better choice.
Yes, and they are often better at sarcasm and context than smaller open-source models. The catches are cost at millions of tweets a day, and consistency over time, since a provider can update or retire a model and shift your historical scores. A common middle ground is the one in the way ahead, where an open-source model scores everything and a frontier model reads only the hardest cases. If volume is low, a frontier model alone may be the simpler choice.
For a quick, free, transparent first look, yes. For finance, it misses too much. Market terms like "short" or "bear" and most sarcasm get scored wrongly, so a finance-trained model is a better default.
Mostly by paying. The X API now has paid tiers, and the higher ones are expensive. Data vendors also resell social data. Reddit and StockTwits are useful additional sources, each with their own terms of access. Check the terms of use for whichever source you choose before building on it.
Less well. Fewer tweets mean a few accounts can swing the score, and smaller companies are more common targets for promotional bots. For them, it helps to require a minimum number of tweets before showing a score, and to look closely at who is posting.
Without labelled tweets, there is no way to tell whether a model's scores are right. Here, a frontier model labelled a couple of thousand tweets, spread across companies and time, and each label was checked by hand. That was enough to compare models and tune the sarcasm threshold. Checking matters, since frontier models also get sarcasm and finance slang wrong at times. More labels are needed to fine-tune a model, and sarcastic examples matter most, because they are rare.
A media research publication runs topic pages that show how outlets in different countries cover major stories, without reporting the news or giving its own opinion. Its editors spent much of each week finding, reading and sorting new articles and opinion pieces, in many languages, before any analysis could start.
A weekly automation was built. It found new articles on each topic from a curated list of outlets, grouped duplicate wire stories, and used frontier AI models to code each article against van Dijk's framework for critical discourse analysis. Every finding had to be backed by a quote that was checked against the article. After an editor approved them, the results were added to the topic pages on the publication's website automatically.
Editors spent far less time collecting and far more time on the analysis itself, with every finding traceable to its source.
The publication's topic pages, such as one on the war in Ukraine, set out how the story is told in different parts of the world. Readers can see how outlets in one country describe the same events compared with outlets elsewhere, for example Western media compared with media in Russia, China, India or the Middle East. The value lies in the editors' analysis of that coverage, not in the news itself.
Before any analysis, the coverage had to be gathered. For each topic, editors searched dozens of outlets, many in languages they didn't read, sorted news reports from opinion pieces, and noted who was quoted and how each side was described. That took up a large part of every week, and coverage leaned towards outlets that were easy to find and read in English.
What was needed was a weekly, consistent record of new coverage on each topic, coded the same way every time and checkable down to the sentence, so editors could start from the evidence instead of collecting it.
The options already available were:
The publication's editors were trained in discourse analysis and already applied a recognised framework by hand. That meant the method existed. It only needed to be automated and applied at scale.
A weekly job that searched for new articles on each topic from a curated list of outlets, extracted their text, and grouped copies of the same wire story. A frontier AI model then read each article in its original language and filled in a fixed set of fields based on van Dijk's framework, with a supporting quote for every finding. Code checked that each quote really appeared in the article. Editors reviewed the week's results in a review queue, and approved rows were added to the topic page on the publication's website, grouped by country and region. The editors' own analysis was written on top.
The first version searched news indexes by topic and took whatever came back. The results leaned heavily towards large English-language outlets, which are indexed more thoroughly and published in greater volume. For a page about how different parts of the world see a story, that skew defeated the purpose.
So the editors built the source list. They chose the outlets that mattered for each region, tagged each one, and decided how regions were grouped, including what counted as Western media. The automation then searched within that list and showed each week's coverage by country and region, so a region with few articles was visible as a gap instead of being drowned out. Which outlets count, and how they are grouped, stayed an editorial decision, not the automation's.
Many outlets don't write their own international coverage. They republish stories from news agencies such as Reuters, AP, AFP, TASS or Xinhua, often with a new headline and a few edits. Counted separately, one agency story could look like broad agreement across dozens of outlets.
Copies were grouped by comparing their embeddings, and each group was coded once. The list of outlets that ran it was kept, since which outlets choose to carry a given agency's story is itself part of the picture. Where an outlet had changed the headline or added its own paragraphs, those changes were coded separately, because that is often where an outlet's own angle shows.
A framework that editors apply from experience isn't something a model can follow as it is. It was written down as a codebook, a fixed list of questions with defined answer options and examples, as is standard in content analysis. For each article, the model recorded:
The model filled in these fields and nothing else. It didn't write a verdict on the article or decide whether its claims were true.
Early results read well but didn't always hold up. Now and then, the model described an emphasis that the article didn't contain, or attributed a quoted official's view to the outlet itself (hallucination). On a page about how media frames a conflict, one invented finding would undermine the rest.
So every finding needed a supporting quote, and code checked that the quote appeared word for word in the article. Findings that failed were sent back to the model once with the reason. If they failed again, they were flagged for an editor instead of being published. The codebook also separated what an outlet said in its own voice from what it quoted others saying, which removed most of the misattributions.
The publication already had past work to check against. About 200 articles had been analysed by hand by two different editors, each working independently. These were coded into the codebook's fields and used as the test sample. Comparing the model with each editor, and the editors with each other, showed how far the model could be trusted, and on which fields (inter-coder reliability).
Article type, actors and voices were reliable from the start. The ideological square was not. The model and the editors disagreed often, and the editors disagreed with each other more than expected too. The cause was the codebook. "Emphasised" had been left to judgement. It was redefined with concrete signs, such as placement in the headline or opening paragraph, repetition and the words chosen, plus worked examples for each option. After that, the model agreed with the editors about as closely as they agreed with each other, which is the usual bar in content analysis (as also found in recent research on AI-assisted coding). The same sample was kept to test any later change to the model or the instructions.
Once a week, the job ran for every topic page. It found new articles, grouped wire copies, coded the rest and checked the quotes. The results landed in a review queue, where editors could approve, edit or reject each row, with the article link and quotes beside it.
Approved rows went straight to the topic pages, which the readers saw grouped by country and region. Alongside them, a short factual overview of the week was drafted from the approved rows only, such as which regions covered the story most and which terms each used for the main events. Editors checked it and then wrote the analysis that the pages exist for.
From the first codebook draft to the first weekly update on the website, the build took about six weeks.
Less collecting. More analysis. Editors started each week from coded, checkable coverage instead of a blank search page.
What this added beyond the options already available:
| Media monitoring platforms | Manual collection | AI chat with web search | This automation | |
|---|---|---|---|---|
| Covers many countries and languages | Yes, often their main strength | Limited to the languages the team reads | Varies, sources differ between runs | Yes, from the editors' source list |
| Coded against a research framework | Usually sentiment and topics, some allow custom tags | Yes | Only with careful instructions | Yes, van Dijk's framework |
| Same method every week | Yes | Depends on who does it | Hard to keep consistent | Yes |
| Every finding backed by a checked quote | Varies | Yes | Links given, but claims need checking | Yes, checked in code |
| Ongoing cost | Subscription | Staff time | Low | Low API costs, but needs upkeep |
| Ready to use without building | Yes | Yes | Yes | No, needs building |
Limitations
Way ahead
No. It codes each article against a fixed framework and backs every finding with a quote. Deciding what the patterns mean, and how to write about them, is the editors' work.
It is one of the most widely used approaches to analysing ideology in news, and its focus on how "us" and "them" are portrayed suits coverage of conflicts and international stories. It is not the only option. Other approaches, such as framing analysis or Fairclough's critical discourse analysis, could be used by changing the codebook, with the rest of the automation unchanged.
Often, yes. If you need broad tracking of mentions, reach and sentiment, alerts and dashboards, a platform is ready to use and covers far more sources than a custom list. A custom automation makes more sense when your work follows a specific research framework, or the results need to appear on your own website. Depending on your needs, a platform may well be the simpler and better choice.
For a one-off overview, yes, and it takes minutes. For a weekly record, it is hard to keep consistent. The sources found differ between runs, the coding drifts unless the instructions are very precise, and quotes need checking by hand. The same frontier models did the coding here. What the automation adds is a fixed source list, a fixed codebook and automatic quote checks around them.
Showing headlines and links, with your own analysis, is generally the lowest-risk approach, and it is what was done here. Republishing article text or long extracts usually needs a licence, and the rules vary by country and publisher. Check the rules where you operate, for example the UK's guidance for media monitoring. This isn't legal advice.
Three ways. Every finding needs a quote, code checks that the quote is really in the article, and an editor approves every row before it is published. The model is also limited to the codebook's answer options, so it can't invent new categories.
A couple of hundred, coded independently by at least two people, was enough here to measure agreement and fix the codebook. The editors' agreement with each other matters as much as the model's, because it shows how clear the codebook is.
A clinical research team runs systematic reviews comparing treatments, for example one drug on its own, a second drug on its own, and the two combined. Most of the time went into finding eligible studies, screening them against the review's criteria and copying study details and results into spreadsheets, before any analysis could start.
An automation was built around frontier AI models. From the review's question (in PICO format) and its inclusion and exclusion criteria, it drafted a search strategy, searched the main medical databases and screened every record, with a reason for each decision. For each included study, it filled in a data extraction sheet with the study's design, participants, treatments and results, each value linked to the page and passage it came from. The sheet then ran the meta-analysis itself.
Researchers moved from collecting data to checking it and interpreting the results, while every decision stayed with them.
A systematic review answers a clinical question by finding every eligible study on it, extracting the same details from each, and, where the studies are similar enough, combining their results (meta-analysis). Systematic reviews sit at the top of most evidence hierarchies in medicine, and are also among the slowest kinds of research to produce. A single review can take many months.
The team's reviews start from a question framed as PICO (the population, the intervention, the comparison and the outcomes) plus inclusion and exclusion criteria, such as only randomised controlled trials (RCTs) in adults. From there, the work was largely manual:
The first three steps took most of the time, and the analysis and discussion, which is where the team's expertise matters most, got squeezed.
The options already available were:
The team wanted its existing method, criteria and spreadsheets kept, with the repetitive work automated around them.
An automation that took a review's PICO question and criteria as input. It drafted a search strategy for researchers to approve, ran it on the main medical databases and trial registries, and removed duplicates. A frontier AI model then screened every record against each criterion and gave a reason for every decision. For each included study, it extracted the study's details and results into the team's data extraction sheet, with the source page and passage for every value. Researchers checked and corrected the sheet, and the sheet ran the meta-analysis from the checked values.
A good search is broad on purpose. From the PICO question, the model drafted a search string for each database, combining keywords and synonyms with each database's controlled vocabulary (MeSH terms in PubMed), plus a standard filter for RCTs where the criteria required them.
The draft was never run unchecked. Researchers reviewed and edited every search string before it ran, as Cochrane's guidance on searching expects of any review. The model saved time on the first draft. The search itself remained the researchers' decision.
In a systematic review, missing an eligible study is far worse than reading an extra one. So screening was tuned to find every eligible study, at the cost of passing some ineligible ones through (high recall over high precision).
The first version was too strict. It excluded studies whose abstracts didn't mention a required detail, such as the comparison group or the follow-up length, even though the full text did. The fix was a third answer. For each criterion, the model answered "met", "not met" or "unclear from the abstract", and any record with an "unclear" went on to full-text screening instead of being excluded. Every exclusion came with the criterion it failed, so researchers could check them quickly, and they reviewed a sample of exclusions as well as every inclusion.
A single trial often produces several papers, such as the main results, a longer follow-up, an analysis of a subgroup and a conference abstract. Counting each as a separate study would count the same participants more than once and distort the meta-analysis.
Records were grouped by their trial registration number where one existed, and by matching authors, treatments and participant numbers where it didn't. Each trial became one row in the sheet, with all of its papers linked and the main publication marked as the primary source.
Most of the results in a trial paper sit in tables, and that is where early errors appeared. In trials with three or more groups, the model sometimes took a value from the wrong column. It also mixed up measures that look alike, such as the standard deviation and the standard error, which can change a combined result considerably.
Three changes fixed most of this. Each value had to come with the group it belonged to, the type of measure (mean, median, standard deviation, standard error, confidence interval) and the page and passage it came from. Code then checked that every extracted number really appeared in the paper's text, and flagged any that didn't. And the model only copied numbers as reported. It never converted or calculated anything.
Calculations were kept out of the AI model entirely and done with formulas, which give the same answer every time and can be checked line by line.
Once researchers had checked a study's values, the sheet converted them into a common form where needed, using the formulas in the Cochrane Handbook, for example a standard deviation from a standard error. It then calculated each study's effect (such as the difference in average change between groups, or the risk ratio for side effects) and combined them with a random-effects model, which allows for real differences between studies. It also showed how much the results varied between studies, and drew a forest plot. The formulas were checked against the same analysis run in metafor, a widely used meta-analysis package for R.
Before it was used on a new review, the automation was run on a review the team had already completed by hand, using the same question and criteria. That gave a clear test. Did it find every study the team had included, and did its extracted values match theirs?
It found all of the included studies, along with a few extra borderline records that screening correctly passed on for a closer look. Most extracted values matched. The mismatches were almost all the table and measure-type errors described in 2.2.4, and were rare after those fixes. The finished review stayed as a test set, so any change to the model or instructions could be checked against it.
From the first test search to the first live review, the build took about two months.
From papers to comparison. Researchers started from a filled-in extraction sheet and a working meta-analysis, and spent their time checking values and interpreting the results.
What this added beyond the options already available:
| Systematic review software | AI research assistants | Manual review | This automation | |
|---|---|---|---|---|
| Runs the database searches | Mostly imports results from searches run elsewhere | Yes, mainly in their own index | Yes, by hand | Yes, once researchers approve the strategy |
| Helps with screening | Yes, often with AI ranking | Yes | Done entirely by people | Yes, with a reason per criterion |
| Fills in your own extraction template | Yes, mostly typed in by people | Usually their own table format | Yes | Yes, with a source for every value |
| Runs the meta-analysis | Some do | Varies | Separately, in statistics software | Yes, in the sheet |
| Established in published reviews | Yes, widely used | Growing, with validation and reporting | Yes, the reference standard | Growing, with validation and reporting |
| Ready to use without building | Yes | Yes | Yes | No, needs building |
Limitations
Way ahead
Yes, with care. Cochrane and other evidence synthesis organisations accept AI use when authors stay accountable, keep human oversight and report exactly how AI was used. That is why every decision here was logged and checked by a researcher.
Not on its own. Standard practice is for two people to screen and extract independently. Here, the AI model acted as one of them, with a researcher checking its work and a second researcher resolving disagreements. Whether that is acceptable for a given review is a decision for the review team and the journal.
Often, yes. It is widely used, well understood by journals and built for teams working together. Many teams will find it is all they need. An automation like this makes more sense when searching and data extraction take up most of your time, or when you want to keep your own spreadsheets and templates. The two can also be combined.
For scoping a topic or a quick overview, yes, and faster. For a formal review, check how complete its search is, whether you can export every decision, and whether its extracted values can be traced back to the source. Depending on your needs, it may well be the simpler choice.
Because calculations should give the same answer every time and be easy to check. AI models can make arithmetic slips and may not show their working. Formulas in a spreadsheet, or a statistics package, do neither.
It was run on a review the team had already completed by hand. It had to find the same studies and extract the same values before it was used on anything new. On every new review, researchers still checked every included study and every value.
An academic research lab wanted to know how well an early AI coding system could take established algorithms and turn them into parallel code, the kind that splits work across many processors on a supercomputer. Writing such code is specialist work, and code that runs is not the same as code that gets faster with more processors.
An experiment was set up around three well-known heuristics for the travelling salesman problem, a classic route-finding problem. The AI was asked to implement them using the two standard tools for parallel programming, OpenMP and MPI. The code was then run on the lab's high-performance computing system. First, several thread counts were tested to find the best one. Then one heuristic, Nearest Addition, was run with 1, 2, 4, 8, 16 and 32 processes.
The result was a measured picture of how the AI-written code behaved as computing power was added, through runtime, speedup and scaling efficiency, instead of a judgement based on reading the code.
Supercomputers get their power from running many processors at once. To use that power, code has to be written to split its work between them, keep them coordinated and combine their results (parallel programming). It is specialist work. Mistakes can produce wrong answers, and code that runs correctly can still get slower, not faster, as processors are added, because they spend more time coordinating than computing.
AI coding systems were becoming capable of writing code from a description. The lab wanted to look past whether the code looked right, and see how it actually behaved when run. In short:
Could an AI coding system implement established algorithms with standard parallel programming tools, and how did the resulting code's runtime change as more computing power was added?
The aim was not to find new algorithms or to solve the travelling salesman problem in a new way. The algorithms were chosen in advance, and were well known. What was being tested was the AI's implementation of them, and how that implementation performed.
Three established heuristics for the travelling salesman problem were given to an AI coding system, which was asked to implement them using OpenMP and MPI. Researchers reviewed the generated code and ran it on the lab's high-performance computing system. The experiment then had two stages. The first tested several OpenMP thread counts to find the best one. The second fixed that thread count and ran the Nearest Addition implementation with an increasing number of MPI processes, recording how long each run took.
Using established heuristics was a deliberate choice. Their behaviour is well documented, so anything unexpected in a run could be traced to the implementation, not to the method itself. It also kept the question narrow. The AI was asked to implement known methods, not to design new ones.
These heuristics are also a demanding test for parallel code. Each one builds a route one city at a time, and each step depends on the route so far, so the steps can't run side by side. Only the work within each step, searching for the next city to add, can be split between processors. Every step then ends with all of them agreeing on the best candidate before the next one can begin. That pattern of small pieces of work followed by coordination is where parallel code often struggles.
The AI coding system was asked to implement the three heuristics with OpenMP for multithreading, and to produce an MPI version for distributed computing. The researchers reviewed the generated code and ran it on the HPC system.
Code that compiles isn't necessarily correct, and parallel code has failure modes of its own. Two threads updating the same value at the same moment (a race condition) can produce a different result on each run. So each parallel version was checked to produce a valid route that visited every city exactly once, and the same route as the single-process version, before any timing was trusted.
More threads aren't automatically faster. Each extra thread brings overhead for starting it, sharing out work and waiting for the others, and past a certain point that overhead outweighs the gain. The best number depends on the code and on the machine.
So the first stage ran the implementation with several OpenMP thread counts and compared their runtimes. The fastest configuration was then fixed for the next stage, so that any change in runtime there came from the number of MPI processes alone.
The second stage focused on Nearest Addition. With the thread count fixed, the MPI version was run with 1, 2, 4, 8, 16 and 32 processes, and the runtime of each run was recorded. Doubling the processes at each step is the usual way to test scaling, since it shows clearly whether each doubling of resources still pays off.
A note on terms. TSP problems are often described in "nodes", meaning cities, while supercomputers are also made of "nodes", meaning machines. Here, the 1 to 32 refers to MPI processes, not machines. Several processes can run on the same machine.
The runtimes were turned into the two standard measures of parallel performance, each plotted against the number of processes:
Some loss of efficiency is expected in any parallel code. Part of every program can't be split (Amdahl's law), and coordination costs grow with the number of processes. For heuristics like these, problem size matters a lot. With only a few cities, each step has very little work to share, so coordination can easily cost more than it saves. Larger inputs give each process more work between coordination steps. The curves showed where adding processes stopped paying off for this implementation.
A measured view of how AI-written parallel code behaved as computing power was added, instead of a judgement based on reading it.
The specific measurements are withheld for confidentiality.
How this approach compares with other ways of producing parallel code:
| Written by an HPC programmer | Automatic parallelisation by the compiler | AI-written code, untested | AI-written code, benchmarked | |
|---|---|---|---|---|
| Time to a first parallel version | Slowest | Fast, but mainly for simple loops | Fast | Fast to write, plus time to test |
| Expertise needed | High | Low | Low to write, high to judge | Enough to review and benchmark |
| Splits work across machines (MPI) | Yes | No, usually within one machine | Yes, if asked | Yes |
| Evidence that it scales | Only if benchmarked | Only if benchmarked | None | Yes, measured from 1 to 32 processes |
| Best achievable performance | Usually the highest | Limited | Unknown | Not compared with expert code here |
Limitations
Way ahead
No. The travelling salesman problem was only the workload. The AI implemented three well-known heuristics chosen by the researchers. It didn't invent any methods, and the experiment measured speed, not route quality.
AI can write code that runs in parallel. Whether it scales has to be measured, because parallel code can look right and still slow down as processors are added. That measurement is what this experiment was about.
OpenMP splits work between threads on one machine that share the same memory. MPI splits work between processes that each have their own memory and exchange messages, so they can run across many machines. Large HPC programs often use both.
So that only one thing changed at a time. Fixing the best thread count first meant any change in runtime in the MPI stage came from the number of processes alone.
Quite possibly, since they have improved a lot. But that is a claim to test, not assume. The same experiment can be rerun with current tools for a direct comparison.
As a fast first draft, it can help. Before relying on it, check that results match a trusted version and measure how it scales on your own hardware. For performance-critical code, an HPC programmer's review is still worth it.
