In February 2025, Gartner published a prediction that should make any founder pause before paying for another AI tool. Through 2026, it said, organizations will abandon 60% of AI projects that are not backed by AI-ready data. The same research found that 63% of organizations either lack, or aren't sure they have, the right data management practices for AI (Gartner press release, February 26, 2025, based on a mid-2024 survey of data management leaders).

So the model usually isn't what breaks. The data is. Sales numbers live in one tool, product usage in another, support tickets in a third, and nobody fully trusts the monthly report because two dashboards show two different revenue totals.

Sorting out that mess is the job a data warehouse was built for, and Amazon Redshift is one of the oldest and most widely used options. AWS announced it in late 2012, and in a November 2025 product announcement the company said it is used by tens of thousands of customers.

This practical guide covers what Redshift does, how it fits into AI work, what it costs, and how it behaves when data shows up late, arrives twice, contradicts itself, or pours in faster than planned. No database background needed. Technical terms get a short explanation right where they appear.

PART 1  

What a data warehouse does, and where Redshift fits

Start with a simple distinction. The database behind your app (PostgreSQL or MySQL, for example) is built for lots of small, quick actions: save this order, update that password, fetch one customer's profile. It handles those very well.

Now ask that same database, "What was our average order value by region, by month, for the last three years?" That question touches millions of rows at once. Running it on your live app database can slow the app down for real customers while it grinds through the work.

A data warehouse is a separate system built for those big questions. You copy data from your app, payment tool, CRM, and logs into it, and analysts, dashboards, and AI tools ask their questions there instead.

AWS has a few analytics services (Athena, for instance, queries files sitting in storage), but Redshift is the dedicated AWS data warehouse: a system that stays ready, keeps data in a structure tuned for analysis, and answers heavy questions fast.

Three design choices explain most of its speed.

▪       Columnar storage. A typical app database stores data row by row, like a stack of complete order forms. Redshift stores it column by column, so every "order total" value sits together. If a question needs only 3 columns out of 50, Redshift reads only those 3. Values of the same kind stored side by side also compress very well, which saves storage and reading time.

▪       Massively parallel processing, usually shortened to MPP. Your data is split across several machines called compute nodes, and each node is split again into slices. A query gets broken into pieces, every slice works on its own share at the same time, and a leader node combines the results.

▪       Zone maps. Redshift stores data in 1 MB blocks and keeps a note of the smallest and largest value in each one. Ask for last week's orders, and any block holding only 2023 orders gets skipped without being read.

Think of one librarian checking every shelf, compared with twenty librarians who each take a few shelves and only read the spine labels they need.

PART 2  

How one question travels through Redshift

Here's the path a typical query takes. Every dashboard tile and every AI-generated report runs through the same steps, so this is the backbone of all Redshift analytics.

1.      A person, a dashboard, or an AI agent sends a request in SQL, the standard language for asking databases questions.

2.      The leader node reads it, checks statistics about the tables, and builds a plan: which tables to scan, in what order to join them, and where the needed data physically sits.

3.      That plan gets compiled into code the compute nodes can run. In March 2026, AWS said a new compilation method made first-run queries up to 7 times faster for dashboards and ETL jobs, switched on by default at no extra charge.

4.      Each slice scans its own portion of the data and skips blocks it doesn't need.

5.      If a join needs rows that live on different nodes, Redshift moves data across the network between them. This step is often the slowest part of a query, and it's where table design matters most.

6.      Slices send back partial results, and the leader node assembles the final answer.

PART 3  

Provisioned clusters or Serverless?

You can run Amazon Redshift in two ways, and the choice mostly comes down to how steady your workload is.

With a provisioned cluster, you choose a node type and how many nodes you want, and you pay by the hour while the cluster runs. You can pause it when nobody needs it, and reserved nodes (a one or three year commitment) bring the hourly price down. Current RA3 nodes keep compute and storage apart: frequently used data stays on fast local drives, while the rest sits in Amazon S3 under a feature called Redshift Managed Storage. That separation means you can add storage without paying for more compute.

With Redshift Serverless, there are no nodes to choose. Capacity is measured in Redshift Processing Units, or RPUs, and each RPU provides 16 GB of memory. You only pay while queries are running, billed per second with a 60-second minimum. In the US East (N. Virginia) region the price is $0.375 per RPU-hour, so the smallest 4-RPU setup costs $1.50 for each hour of active work (AWS, June 2025). Since February 2026, three-year Serverless Reservations have offered up to 45% off, with no upfront payment.

The entry-level 4-RPU option comes with real limits: up to 32 TB of managed storage, a maximum of 100 columns per table, and 64 GB of memory in total. Those numbers come straight from AWS's announcement, and the column limit catches people out (more on that in Part 6).

PART 4  

What "the AI era" changes for a warehouse

A warehouse's core job hasn't changed: collect data from many places and make it easy to question. What has changed is who's asking, and what they do with the answers. Three shifts stand out.

AI projects need joined, trustworthy history

A churn prediction model needs product usage, billing history, and support contacts for the same customers, with the same definition of "active customer" across all three. Pulling that together is classic warehouse work, and it's exactly the gap behind the Gartner warning at the top of this article.

SQL users want to use models without leaving SQL

Redshift ML lets you train a model with a SQL command (CREATE MODEL). Behind the scenes, Redshift sends the data to Amazon SageMaker, AWS's machine learning service, and brings back a function you can call in ordinary queries. Since October 2024, Redshift can also connect to large language models in Amazon Bedrock with a command called CREATE EXTERNAL MODEL. AWS lists tasks such as translation, summarization, and sentiment analysis, using models including Anthropic Claude, Amazon Titan, Llama, and Mistral. AWS states that this adds no Redshift charge beyond normal Bedrock pricing.

Here's a realistic use. A support team has 50,000 ticket descriptions. One SQL statement asks a language model to tag each ticket's topic and sentiment and saves the results to a table. Join that table with revenue data, and you can see which complaint types come from your biggest accounts. That's Redshift analytics reaching into text that used to sit unread.

One caution: every row is a separate model call, so cost and Bedrock rate limits apply. Test on 500 rows before running all 50,000.

AI agents now query warehouses directly

In August 2026, AWS connected Redshift to its Agent Toolkit. AI coding agents such as Claude Code, Kiro, and Cursor can now explore schemas, write queries, troubleshoot, and even plan migrations to Redshift through an MCP server (MCP, or Model Context Protocol, is a standard way for AI tools to talk to outside systems). AWS also offers Amazon Q generative SQL inside the Redshift query editor, which turns a typed question into a draft SQL query.

For non-technical teams this lowers the barrier a lot. It also raises the stakes on clear table names, column descriptions, and permissions, because an agent only knows what those labels tell it and can only touch what you allow.

Messy formats and open tables

AI work brings in JSON from apps, event logs, and API responses. Redshift stores this kind of nested data in a column type called SUPER, and in March 2026 it added nine array functions (ARRAY_CONTAINS, ARRAY_SORT, and others) for searching and reshaping that data in ordinary SQL.

Redshift can also read and, since November 2025, write Apache Iceberg tables. Iceberg is an open table format stored in S3 that other engines like Athena, EMR, and Spark can use too. In practice, your warehouse and your data lake can share one copy of a table instead of keeping two that drift apart. AWS's launch blog quotes the insurance analytics firm Verisk reporting 30% faster query aggregations with this setup. Keep in mind that the claim comes from AWS's own blog about its own product.

PART 5  

Three ways to get data in

Data has to arrive before anyone can analyze it. Any AWS data warehouse setup needs a loading plan, and Redshift offers three main routes. Most companies use more than one.

The first is batch loading with the COPY command, which pulls files from Amazon S3. It's the fastest way to load large volumes because every slice loads its share of files at the same time. AWS's own guidance is to split big loads into several files, ideally a multiple of the number of slices, so no slice sits idle.

The second is zero-ETL integration. ETL stands for extract, transform, load, the traditional process of copying data out of a source, cleaning it, and loading it into a warehouse. Zero-ETL means AWS handles the copying for you, with no pipeline code to maintain. Current sources listed in AWS documentation include Aurora MySQL and Aurora PostgreSQL, RDS for MySQL, PostgreSQL, and Oracle, DynamoDB, and business apps such as Salesforce, SAP, ServiceNow, and Zendesk.

The third is streaming ingestion. Redshift can read directly from Amazon Kinesis Data Streams and Amazon MSK (managed Apache Kafka) into what's called a streaming materialized view, a saved query result that refreshes as new data arrives. AWS says a single refresh can take in hundreds of megabytes per second. Since August 2026, refreshes of Kinesis-connected streaming views can also run on concurrency scaling capacity, which keeps the main warehouse free for other work.

PART 6  

When the data misbehaves

Feature lists make every warehouse sound tidy. Real data isn't, and this is where projects succeed or quietly fail.

Data gaps

Late data is the most common gap. A mobile app records events offline and uploads them hours later when the phone reconnects. A "yesterday's signups" dashboard built at 6 a.m. will be short, and it stays wrong forever if the pipeline only appends new rows once. The fix is to keep two timestamps on every event, when it happened and when it loaded, and to rebuild a rolling window (say, the last three days) each night using Redshift's MERGE command, which updates existing rows and inserts new ones in one statement.

Missing values cause quieter damage. SQL's AVG function ignores NULLs (empty values). If failed payments get stored as NULL instead of 0, your average payment looks higher than it really is. Decide, column by column, whether "missing" means zero, unknown, or "not applicable," and write that rule down.

Some gaps come from the integration itself. For Aurora MySQL and RDS for MySQL zero-ETL sources, AWS's documentation tells you to add primary keys to tables that lack them. A source table without one won't replicate the way you expect, and nothing on your dashboard will tell you that table is missing. Wide tables are another trap: a Salesforce object with 140 fields won't fit under the 100-column limit of a 4-RPU Serverless setup.

Conflicting signals

Your payment provider says you have 1,204 paying subscribers. Your CRM says 1,251. Redshift can't decide which one is right, because both numbers are accurate copies of what each system recorded. The difference usually comes from timing, refunds, trials, or test accounts. Pick one system as the source of truth for each metric, document the choice, and build a small reconciliation table that flags gaps bigger than an agreed threshold, such as 1%.

There's also a Redshift-specific trap here. Primary key, unique, and foreign key constraints are informational only, so Redshift doesn't enforce them. Only NOT NULL is enforced. The query planner may still trust a declared primary key when choosing how to run a query, so if duplicate rows slip in, some queries can return wrong answers without any error. Run your own duplicate checks after every load.

Streams add their own duplicates. Kinesis producers retry failed sends, so the same event can arrive twice. Give every event a unique ID at the source, then keep one copy per ID using a window function like ROW_NUMBER().

Time zones round out the list. Redshift's TIMESTAMP type carries no time zone, while TIMESTAMPTZ converts values to UTC for storage. Mix the two carelessly and your "lunchtime sales spike" may actually be breakfast in another country.

Real-time decisions

"Real time" means different things to different teams, so get specific before you build anything. Redshift streaming gives near real-time results, typically seconds to minutes behind depending on how often views refresh. That works well for an operations screen showing late deliveries by zone, updated every minute. It doesn't suit decisions that must happen in milliseconds, like blocking a fraudulent card at checkout. Those belong in your app or a stream processor, with Redshift reviewing the results afterward.

Short query acceleration helps here by moving small queries into an express lane, so a dashboard tile doesn't wait behind a 20-minute transformation job.

Exceptions and edge cases

A few edge cases surprise teams again and again.

▪       Python user-defined functions are gone. Redshift stopped allowing new Python UDFs in late 2025, and existing ones reached end of support after June 30, 2026. Any old pipeline that depends on them needs rewriting as SQL UDFs or Lambda UDFs (functions that call AWS Lambda).

▪       Old drivers can fail to connect. Since January 31, 2026, Redshift requires TLS 1.2 or higher for connections. An aging BI tool or an old JDBC driver can suddenly stop working after years of no trouble.

▪       Distribution skew can wreck performance. Say you distribute an orders table by customer_id, but guest checkouts have no customer ID. Millions of NULL rows land on a single slice, that one slice does most of the work, and the whole query runs only as fast as the busiest slice.

▪       Deleted rows linger. Redshift marks deleted rows and cleans them up later with VACUUM, which now mostly runs automatically in the background. Tables with heavy updates can still bloat and slow down between cleanups.

▪       Statistics go stale after big loads. The planner relies on table statistics, and after a large load those numbers can be outdated, leading to poor plans. Automatic ANALYZE usually catches up, but running ANALYZE right after a major load avoids a slow morning.

▪       Iceberg writes have limits. When write support launched in November 2025, it covered append-only workloads (adding new rows). Check the current documentation before planning updates or deletes on Iceberg tables from Redshift.

Behavior under pressure and at scale

Take a typical Monday at 9 a.m. Two hundred people open dashboards at once while the overnight data load runs late. Here's what Redshift does, and what you can control.

On a provisioned cluster, workload management (WLM) places queries into queues. When queues back up, concurrency scaling can add temporary clusters within seconds, then remove them once demand drops. Usage beyond your free credits is billed per second at your cluster's on-demand rate. On Serverless, the warehouse scales up from your base RPUs, and a maximum capacity setting puts a ceiling on how far it can grow.

Query priorities let ETL run below the CEO's dashboard, or the reverse during month-end close. Query monitoring rules can log or stop queries that scan or run far too long, protecting everyone else from one runaway request.

Memory is the other pressure point. A huge join that doesn't fit in memory spills to disk and slows sharply. For a heavy report that runs every morning, a materialized view that pre-computes it is often cheaper than more capacity.

For larger teams, data sharing lets one warehouse handle loading and transformation while a separate one serves dashboards. Both read the same live data without copying it, so heavy loading jobs can't starve dashboard users. That's how Redshift analytics setups usually stay responsive as headcount grows.

The most common failure at scale isn't a crash. It's the bill. Automatic scaling works for bad queries too, so set cost alerts on day one.

PART 7  

The market behind cloud warehouses

Interest in big data warehousing keeps rising as companies push AI projects into production. The research firms agree on fast growth, but they don't agree on the size of the market, and the gap is large.

All three measure the "data warehouse as a service" market, yet their 2025 estimates differ by almost two times, mostly because each firm counts different things. Treat any single figure with care. The growth rate, roughly 20 to 23% a year across all three, is the more dependable takeaway.

Data quality is also expensive. Gartner has estimated that poor data quality costs organizations an average of USD 12.9 million a year (Gartner, 2021). It's an older figure, and averages hide wide variation, but it explains why clean data gets budget before new dashboards do.

PART 8  

Redshift compared with other cloud warehouses

Here's how Redshift lines up against the three alternatives that come up most often. Prices change often, so the table shows how each one charges rather than exact rates.

The honest summary for big data warehousing buyers: if most of your data already lives in AWS, Redshift avoids data transfer fees and lets you use zero-ETL. If you're committed to more than one cloud, Snowflake or Databricks will usually be simpler to run.

PART 9  

Is Redshift right for you?

A quick checklist helps more than another feature list.

Redshift is probably a good fit if:

▪       Your app already runs on AWS, with data in S3, Aurora, RDS, or DynamoDB.

▪       Your team knows SQL, or plans to rely on SQL-friendly BI tools.

▪       You run regular reporting and want AI features (Redshift ML, Bedrock calls) close to the data.

▪       You expect data to grow from gigabytes into terabytes over the next year or two.

Think twice if:

▪       Your data fits comfortably in a few gigabytes. A read replica of your existing PostgreSQL database may answer your questions for years at a fraction of the cost.

▪       Your company runs on Azure or Google Cloud, or has a firm multi-cloud policy.

▪       Your main need is instant, per-request decisions inside your app.

For a small team trying an AWS data warehouse for the first time, a sensible first month looks like this. In week one, start a 4-RPU Serverless workgroup and set usage limits. In week two, connect one source through zero-ETL or a COPY load. In week three, build the three reports people ask for most and agree on metric definitions. In week four, add data quality checks for duplicates, NULLs, and row counts before you invite AI tools in.

Conclusion

The AI tools grabbing headlines all depend on something unglamorous: one place where company data is collected, cleaned, and agreed upon. Amazon Redshift has filled that role for more than a decade, and recent additions such as Bedrock calls from SQL, Iceberg writes, streaming ingestion, and AI agent access have pulled it closer to day-to-day AI work.

None of that removes the hard parts. Late events, duplicate records, two systems quoting different revenue, and a Monday morning traffic spike all still need deliberate design. Teams that plan for those cases get a warehouse their AI tools can trust. Teams that skip them get faster wrong answers.

If your data already lives in AWS, start small with Serverless, connect one source, set your cost limits, and write down your metric definitions before anything else. That's how big data warehousing pays off in practice for a startup or a growing office team.