Configuration Advisor (POC and benchmark tuning)
The Configuration Advisor helps you tune a StarRocks / CelerData cluster for a POC or benchmark, review a table design, and work out why a query failed or ran slowly. Ask it in AI Chat. It works from a rulebook: rules (ids like CR-001) that recommend configuration and table-design changes, the StarRocks parameters they use (checked against the source code, with defaults per release), and the evidence behind them, both past tuning cases with measured results (IV-…) and guidance from playbooks and docs (GD-…).
It covers statistics, Data Cache, table design (buckets, partitions, sort keys), parallelism, joins, memory and spill, and materialized views, on shared-nothing, shared-data and external-catalog (Iceberg, Hive, …) setups. Every rule is a draft until an engineer reviews it.
Next to the rules, the advisor runs configuration observations: fixed checks for common POC problems the rulebook doesn't cover yet (a global pipeline_dop pinned too low, a short query_timeout, statistics collection switched off, single-bucket or skewed tables, inactive MVs, and more). They are always labelled not approved rules.
You can use it in several ways:
- Ask questions about rules, settings and past cases. No cluster needed.
- Evaluate a POC cluster. It takes a read-only snapshot of the cluster and checks every rule and observation against it.
- Review a table design. Paste a
CREATE TABLE,CREATE MATERIALIZED VIEWorALTER TABLEand it is checked line by line, with or without a cluster. - Look at a cluster as it runs now: its build, deployment, warehouses, nodes and cores, and which database holds a table.
It only recommends. It never changes a setting, never runs a benchmark query, never creates a table or MV, and never asks for a password.
What to Tell It​
The advisor takes the snapshot first and works with what your message already says. For anything you leave out it states the default it used instead of asking. It asks only what would change the answer (at most three questions, see AI Chat), never for something your message already gives, and never again for something you said you'd rather not give. For a pre-flight check that is usually one question: what the evaluation must prove, and against which engine. You get a sharper answer if your first message already has the details below.
| Input | What to give | When you leave it out |
|---|---|---|
| Cluster | The cluster name (or UUID) | Required. An admin must have registered the cluster with the advisor first; you never type a password into chat. |
| Databases (and tables, if you know them) | e.g. tpcds_1t, or tpcds_1t.store_sales, tpcds_1t.date_dim | Table checks (statistics, buckets, partitions, skew, MVs) need them. With databases only, a table match is approximate until you confirm the table is part of the benchmark. If you name tables without a database, the assistant can look up which database holds them. |
| Benchmark goal | latency, throughput, concurrency, load speed or stability | Taken from your wording (dashboards or p95 mean latency, "N concurrent" means concurrency, ingest means load speed). Otherwise rules are checked for any goal, and the ones that fire that way are marked GOAL ASSUMED. |
| Concurrency | How many queries run at the same time | Assumed 1 (serial), and said so |
| Cold runs scored? | Yes if the first (cold-cache) runs count toward the score | Assumed not scored, and said so |
| Session defaults? | Yes if the benchmark sessions run with the cluster defaults: no SET statements, no session.* user properties | Session-variable rules stay approximate or unknown |
| Warehouse (shared-data) | The warehouse the benchmark runs on | default_warehouse when the cluster has several, and said so |
Optional, when they apply:
- the login the benchmark runs as;
- the kind of workload: dashboards (repeated queries), ad hoc queries, ingest, or one-at-a-time batch runs;
- tables that receive loads during the benchmark (for mixed load and query runs);
- the join conditions from your queries (
orders.customer_id = customers.customer_id), for the colocation and skew checks; - whether the source system deduplicated rows (a ClickHouse ReplacingMergeTree, or upserts), for the table-model check;
- whether the benchmark is running right now at its target concurrency (this enables the under-load checks: node CPU, memory and running queries).
You never need to give the version or the deployment (shared-nothing or shared-data): both are read from the cluster's nodes. If you state a deployment that contradicts the nodes, the advisor uses what the nodes say and tells you.
A message you can copy​
Evaluate the POC on poc-retail-01.
- Benchmark databases: tpcds_1t (tables: tpcds_1t.store_sales, tpcds_1t.date_dim, tpcds_1t.item)
- Goal: latency
- Concurrency: 1 (serial power run)
- Cold runs scored: no, we report the best of 3 warm runs
- Benchmark sessions use the cluster defaults: yes, no SET statements
Example Questions​
Evaluating a POC cluster​
- "Evaluate the POC on poc-retail-01: databases
tpcds_1t, goal latency, concurrency 1, cold runs not scored, sessions use the defaults." - "Same snapshot, but the customer now cares about throughput at 32 concurrent queries. What changes?" (The answer is re-evaluated without touching the cluster again.)
- "Which of the fired rules should we do first?"
- "Why is CR-001 only approximate?"
- "Show me the values you observed behind CR-005."
- "Which global settings differ from their defaults on poc-retail-01?"
- "Show me every
query_queue%variable from that snapshot." - "The benchmark is running right now at 64 concurrent queries. Take another snapshot under load."
- "We also stream loads into
orders_rtduring the run. Does that change anything?"
Table design and slow or failing queries​
- "Review this CREATE TABLE for the
orderstable on poc-retail-01." (paste the DDL) - "We have no queries yet. Propose a table design for this schema." (Sort and distribution keys stay as placeholders until you share 3 to 5 representative queries.)
- "This dashboard query runs every minute. Draft a materialized view for it."
- "Would adding a bloom filter on
customer_idrewrite the table?" - "This query fails with 'Memory of process exceed limit'. What should we change?"
- "Which database holds
lineitemandorderson poc-retail-01?" - "What is poc-retail-01 running right now?"
Learning about rules and settings (no cluster needed)​
- "Which rules apply to a 4.0 shared-data cluster with a latency goal?"
- "List the table-design rules."
- "Explain CR-004. When is spill safe to turn on?"
- "Is
pipeline_dopsafe to change at runtime on 3.5, and what is its default?" - "Does
enable_scan_datacacheneed a restart?" - "Which rule covers benchmark tables with no statistics, and how do I apply it?"
Learning from past POCs​
- "Has anyone seen 'BE has overloaded with N pipeline drivers' in a POC?"
- "What did we change last time joins were slow on a shared-data TPC-DS run?"
- "Show me the evidence behind CR-003."
What an Evaluation Looks Like​
A shortened, illustrative example. The cluster is made up; the rule text is CR-001's real wording, and your counts will differ.
You: Evaluate the POC on poc-retail-01. Databases tpcds_1t, goal latency, concurrency 1, cold runs not scored, sessions use the defaults.
Assistant: (takes a read-only snapshot: versions, nodes per warehouse, settings, users, Data Cache, tables, partitions and tablets, statistics, MVs, recent loads; this usually takes about 20 seconds) Snapshot of poc-retail-01 (3.5.21-ee, shared-data, read from the nodes; 4 CN in default_warehouse). These rules fired, in the order to consider them:
- CR-001 — Benchmark tables have no column statistics: run ANALYZE after the final load, before timed runs. Why:
store_salesanditemhave no column statistics, so plans fall back to default estimates. Evidence: 4 past POCs where this helped (strong). Risk: low; no restart. Apply:ANALYZE TABLE db.t (k1, k2, filter_col) WITH SYNC MODE;for small or medium tables,ANALYZE SAMPLE TABLE db.big_t WITH ASYNC MODE;for large or wide ones, then wait forSHOW ANALYZE STATUSto show FINISH. Verify:EXPLAIN COSTSshows ESTIMATE for the flagged columns; re-run the timed query and compare with the baseline. Roll back:DROP STATS <t>.- …
Configuration observations (not approved rules):
- High:
pipeline_dop = 1(global) is below the automatic value of 8 for these nodes: every query runs one pipeline driver per fragment. Apply:SET GLOBAL pipeline_dop = 0;(0 = automatic). Roll back:SET GLOBAL pipeline_dop = 1;. No restart.Two results are approximate: they matched tables in
tpcds_1tthat may not be part of the benchmark. Some rules are unknown: they need an answer, or per-query evidence that isn't collected yet. The assistant lists the exact questions.You: The benchmark tables are store_sales, date_dim, item and customer.
Assistant: (re-evaluates the same snapshot with your answer, without touching the cluster again) Both approximate results are now exact…
The top three fired rules come with their apply, verify and roll-back steps right away; ask about any other rule for its full text.
Reading the Results​
| Result | Meaning |
|---|---|
| Fired | The rule's conditions hold on your cluster. You get the why (with the values observed), the evidence, the risk, whether a restart is needed, and how to apply, verify and roll it back. |
| Not fired | Checked, and the conditions don't hold. |
| Unknown | Data or an answer is missing. The assistant lists the exact questions that would settle it, and re-evaluates without a new snapshot once you answer. Missing data never counts as "not fired". |
| Approximate | It matched a table somewhere in your benchmark databases. Confirm that table is part of the benchmark (or name the tables) to make it exact. |
| GOAL ASSUMED | It fired for a goal you didn't state. Tell the assistant what the POC must prove to confirm or drop it. |
| Suppressed | Fired, but a guardrail says not to apply it in your situation. The reason is shown. |
| Not applicable | The rule targets another version or deployment. |
| Not covered by the snapshot | You named a table the snapshot didn't collect (another database). Its checks are unknown, not clean: the assistant takes a new snapshot that includes it. |
| Observation | A fixed check outside the rulebook, labelled not approved rule, with its evidence and, for a change, its apply, roll-back and restart lines. A table rebuild always comes with its cost: rows and GB rewritten, an estimated time, extra storage until the old copy is dropped, and a pause in writes until the swap. |
| Pre-flight checks to review | Nothing representative is known to have run on the cluster (no running queries, no recent loads, query history off or empty; when the snapshot cannot tell, it says so and holds the steps back the same way) and you stated no goal or workload. Findings are listed as checks to review, without SET or apply steps. Run the benchmark (or say what it must prove) and ask again. |
- Evidence is "N past cases helped", or "doc-backed: no POC outcome yet" when only documentation supports the rule. Guidance is never presented as a measured result.
- Risk: high means a restart, a cluster-wide config change or a data rewrite. That's often fine in a POC, but it's stated plainly.
- When several rules fire, they come in the order to apply them. Rules that must run after another, or that conflict with one another, are called out.
- A value computed for your cluster (for example a
pipeline_dopderived from the nodes' cores) replaces the rule's example value, and the answer says so. - Advice drawn from past cases rather than an approved rule is labelled that way.
Table Design Reviews​
When you paste or ask for DDL, the advisor checks every CREATE TABLE, CREATE MATERIALIZED VIEW and ALTER TABLE with fixed rules (labelled local checks, not approved rules), and checks the DDL it proposes the same way. <placeholders> are fine anywhere a name, type or value goes. The checks cover:
- Statements that would fail: key columns not first or out of order; Primary Key, Unique or Aggregate tables whose partition or distribution columns aren't key columns; nullable or unsupported Primary Key types; FLOAT or DOUBLE keys and partition columns; features newer than the cluster's version.
- Design: money in DOUBLE / FLOAT (use DECIMAL), times stored as text (use DATE / DATETIME), no partitioning despite time columns,
BUCKETS 1on a large table, partitions x buckets that leave tablets under 100 MB, colocation groups with different bucket counts or key types, and a Duplicate Key table over data the source deduplicated (sums and counts would include every row version). - No queries yet: concrete sort or distribution keys are flagged; the assistant keeps them as placeholders and asks for 3 to 5 representative queries.
- Materialized views: qualified names, a
REFRESHclause (manual is the default), andPARTITION BYonly over a partitioned base table. - What an ALTER costs: whether it rewrites the table's data (a bloom filter or index change does, and so does removing one later) or only changes metadata, and whether a config change needs a restart.
For a repeated aggregate query, the advisor can draft a materialized view: qualified names, REFRESH ASYNC, the query's grouping and filter columns, PARTITION BY only when a base table is partitioned on a column the MV carries, the refresh cost from the snapshot, the storage estimate (or "unknown, needs EXPLAIN"), and the verify step: after the first refresh, EXPLAIN the original query and check the plan reads the MV.
Looking at a Cluster​
For a registered cluster, the assistant can read in a few seconds what is running now: the StarRocks build, the deployment (read from the nodes), the cluster's local time against UTC, the frontends, and per warehouse its state, nodes (CN or BE, alive or not), cores, memory limit and tablets, plus the warehouse it treats as the POC warehouse and its automatic pipeline_dop (half a node's cores). CPU and memory figures are instant readings, not a history. It can also find which database holds a table name (and which tables a view reads), and show a table's SHOW CREATE TABLE.
Settings It Won't Give Values For​
During an evaluation the assistant gives values and apply steps only for settings on an approved list. For any other setting it marks the answer OFF-LIST: it explains what the setting does, gives no value or steps, and points you to PhoenixAI support. This holds even when a past case changed that setting. Two exceptions: restoring a setting someone changed back to its default (with the roll-back), and a setting an observation reports with its own apply, roll-back and restart lines. For a failing query, query_mem_limit and new_planner_agg_stage can be suggested as a workaround for one session (SET), never cluster-wide.
Snapshots​
- Each snapshot belongs to the chat conversation that took it. When you ask for "recent snapshots" you see this conversation's; ask for all of them to see the others.
- A snapshot over 2 hours old is marked STALE: the cluster may have changed, so the assistant takes a new one before quoting current values.
- SQL text collected from the cluster (running queries, connections, loads) is stored with its literal values masked, and emails and phone numbers are removed from other text.
- You can ask for the observed values behind any result: specific variables or configs by name (wildcards with
%), one database or table, more columns, or the next page of rows.
Limits​
- Only registered clusters can be evaluated or looked at. Ask an admin to add the cluster's read-only login on the server (Admin Guide → Configuration). Never paste credentials into chat.
- Per-query evidence (
EXPLAINplans and query profiles) isn't collected yet, so rules that depend on the benchmark queries stay unknown, and the assistant does not name a single root cause from configuration alone. Have your benchmark SQL or query ids ready for when it is. - It runs metadata queries only (
SHOW,DESC,information_schema). It never reads user tables, never changes settings and never runs the benchmark. - Reading other users' properties needs a privilege the login may not have; then the per-user check says it wasn't run.
- Rules are drafts until an engineer reviews them, and observations, DDL checks and MV drafts are local checks that are not approved rules. Treat recommendations as a starting point and verify each one with its verify step.
See also: AI Chat for general tips, and Known Issues for errors and crashes you hit during a POC.