The Pipeline That Outgrew Its Home
It started in a Google Sheet. A simple Apps Script that called Gemini, generated an article, and pushed it to WordPress via the REST API. It worked beautifully — for about three months. Then the volume increased, the content got more complex, the optimization requirements multiplied, and suddenly I was running a production content pipeline inside a spreadsheet.
Google Apps Script has a six-minute execution limit. My pipeline was hitting it on every run. The script would timeout mid-publish, leaving half-written articles in WordPress and orphaned rows in the Sheet. I was spending more time debugging the pipeline than using it.
The migration to Cloud Run was not optional. It was survival.
What the Original Pipeline Did
The Apps Script pipeline was elegantly simple. A Google Sheet held rows of keyword targets, each with a topic, a target site, and a content brief. The script would iterate through rows marked “ready,” call Gemini via the Vertex AI API to generate an article, format it as HTML, add SEO metadata, and publish it to WordPress using the REST API with Application Password authentication.
It also logged results back to the Sheet — post ID, publish date, word count, and status. This gave me a running ledger of every article the pipeline had ever produced. At its peak, the Sheet had over 300 rows spanning eight different WordPress sites.
The problem was not the logic. The logic was sound. The problem was the execution environment. Apps Script was never designed to run content pipelines that make multiple API calls, process large text payloads, and handle error recovery across external services.
The Cloud Run Architecture
The new pipeline runs on Google Cloud Run as a containerized service. It is triggered by a Cloud Scheduler cron job or by manual invocation through the proxy. The container pulls the content queue from Notion (replacing the Google Sheet), generates articles through the Vertex AI API, optimizes them through the SEO/AEO/GEO framework, and publishes through the WordPress proxy.
The key architectural change was moving from synchronous to asynchronous processing. Apps Script runs everything in sequence — one article at a time, blocking on each API call. Cloud Run processes articles in parallel, with independent error handling for each one. If article three fails, articles four through fifteen still publish successfully.
Error recovery was the other major upgrade. Apps Script has no retry logic beyond what you manually code into try-catch blocks. Cloud Run has built-in retry policies, dead letter queues, and structured logging. When something fails, I know exactly what failed, why, and whether it recovered on retry.
The Migration Strategy
I did not do a big-bang migration. I ran both systems in parallel for two weeks. The Apps Script pipeline continued handling three low-volume sites while I migrated the high-volume sites to Cloud Run one at a time. Each migration followed the same pattern: verify credentials on the new system, publish one test article, compare the output to an Apps Script article from the same site, and then switch over.
The parallel period caught three bugs that would have caused data loss in a direct cutover. One was a character encoding issue where Cloud Run’s UTF-8 handling differed from Apps Script’s. Another was a timezone mismatch in the publish timestamps. The third was a subtle difference in how the two systems handled WordPress category IDs.
Every bug was caught because I had a production comparison running side by side. This is the only safe way to migrate a content pipeline: never trust the new system until it proves itself against the old one.
What Changed After Migration
Publishing speed went from 45 minutes for a batch of ten articles to under eight minutes. Error rate dropped from roughly 15 percent (mostly timeouts) to under 2 percent. And the pipeline now handles 18 sites without modification — the same container, the same code, different credential sets pulled from the site registry.
The biggest win was not speed. It was confidence. With Apps Script, every batch run was a gamble. Would it timeout? Would it leave orphaned posts? Would the Sheet get corrupted? With Cloud Run, I trigger the pipeline and walk away. It either succeeds completely or fails cleanly with a detailed error log.
Lessons for Anyone Running Production Pipelines in Spreadsheets
First: if your spreadsheet pipeline takes more than 60 seconds to run, it is already too big for a spreadsheet. Start planning the migration now, not when it breaks.
Second: always run parallel before cutting over. The bugs you catch in parallel mode are the bugs that would have cost you data in production.
Third: structured logging is not optional. When your pipeline publishes to external services, you need to know exactly what happened on every run. Spreadsheet logs are fragile. Cloud logging is permanent and searchable.
Fourth: the migration is an opportunity to fix everything you tolerated in the original system. Do not just port the code. Redesign the architecture for the new environment.
FAQ
How much does Cloud Run cost compared to Apps Script?
Apps Script is free but limited. Cloud Run costs roughly -30 per month at my volume, which is negligible compared to the time saved from fewer failures and faster execution.
Do you still use Google Sheets anywhere in the pipeline?
No. Notion replaced the Sheet as the content queue. The Sheet was a good prototype but a poor production database.
How long did the full migration take?
Three weeks from first Cloud Run deployment to full cutover. The parallel running period was the longest phase.
Leave a Reply