Skip to main content

Known Issues

Hit an error, a crash or a strange log line? Ask whether it's a known issue. The assistant checks it against curated known-issue cards: StarRocks / CelerData bugs and configuration problems built from support tickets, R&D on-call logs, Jira and pull requests. Each card says:

  • what is happening, in plain language;
  • how to confirm it is really this issue;
  • the fixes, split into verified (applied, and the problem stopped) and recommended by engineers, not verified;
  • what not to try (mitigations that were applied and failed);
  • the first release on each line that contains the code fix, computed from git.

You can use it in two places:

  • AI Chat: ask in plain English and paste the error.
  • AI Triage → an issue → Known Issues: buttons that check the issue's alerts, scan the cluster's logs, look up a pasted error, or check an upgrade. See AI Triage.

Matching is done by code, not guessed by the model, and the assistant only recommends. It never changes the cluster.

What to Give It​

Usually just the cluster name. The assistant finds the rest itself:

  • The version. It looks up the cluster's StarRocks build, checks whether that build already has the fix, and names the exact release to upgrade to. You never need to type the version. If you do paste one from SELECT current_version(), such as 4.0.16-ee-0e1dddd, it is read as the release 4.0.16-ee: a commit id is not a hotfix build.
  • The logs. It can read the cluster's recent logs live: the FE leader's fe.warn.log, and be.WARNING / cn.WARNING plus the crash output be.out / cn.out on up to three BE / CN nodes. Ask it to scan the logs of the cluster, optionally for a time window ("last 2 hours", "last 24h").

Paste the error text only when:

  • you already have it in hand (a copied stack, a customer's message), which is quicker than a scan;
  • it comes from a client application and never reached the cluster's logs;
  • it happened earlier than a scan reads (a scan tails recent lines, not the whole history);
  • the live scan is unavailable (see Tips and Limits).

Paste it verbatim: exact text matches reliably, while a description like "queries felt slow" usually does not. Crash stacks work best pasted whole, from the *** Aborted at / PC: @ line down through the stack frames.

As soon as you send the message, before the assistant starts writing, the server checks what you pasted against the cards:

  • A log excerpt (three or more log lines, or a stack) is scanned whole, with crash stacks kept together. The assistant never picks one line out of it.
  • One or two error lines (an exception, an ERROR 1064 … message) are looked up directly.
  • A table named as database.table: when the cluster is named and set up for read-only SQL (as for the Configuration Advisor), the assistant reads that table's definition (SHOW CREATE TABLE) rather than a full metadata snapshot, so it can check a card's conditions against the table.

Lines in a paste that read like instructions to the assistant are left out of these checks.

Example Questions​

"Is this a known issue?"​

  • "BEs on acme-prod keep restarting with exit 139. be.out has this, is it a known issue?" followed by the pasted stack.
  • "Is this known? Memory of process exceed limit. ... publish version task for tablet 12345"
  • "A customer's dashboards fail with 'StarRocks planner use long time 3000 ms in logical phase'. Have we seen this, and what should they do?"
  • "We get Couldn't open transport for 10.0.1.5:9020 on the BEs of acme-prod. Known?"

Scan a cluster's logs​

  • "Scan the logs of acme-prod for known issues."
  • "Scan only the BE logs of acme-prod from the last 2 hours."
  • "Here are the last 300 lines of fe.warn.log from acme-prod. Any known issues?" followed by the pasted lines. A long paste is scanned line by line, with crash stacks kept together.

Versions and upgrades​

  • "Is KI-0324 fixed in 3.5.21-ee?"
  • "Our build is 4.0.14-ee-0824. Does it have the fix for that crash?"
  • "If we upgrade acme-prod to 3.5.21-ee, which known issues would that fix? Only high-confidence ones."
  • "Which high-confidence issues are still open in 4.0.16-ee?"

Browse the cards​

  • "Known issues about routine load on shared-data."
  • "Show the configuration issues for compaction."
  • "Which cards have a disputed fix?"
  • "Show the resolution options for KI-0069." or "Where did KI-0001 come from?" (its sources)

Tell it whether it was right​

  • "Yes, that was it: the crashes were the runtime-filter bug."
  • "No, it turned out to be the disk filling up."

A match from the assistant's own lookups and scans has an id (KM-…). Your answer is recorded for the knowledge-base owners to improve the cards. It only records what you say, and only from a later message: never in the same reply that first shows you the match.

What the Answers Look Like​

Shortened examples from real runs (cluster names changed).

An exact match on a cluster below the fix:

You: BEs on acme-prod keep restarting (exit 139). be.out has: PC: @ … starrocks::RuntimeFilterProbeDescriptor::set_runtime_filter(…) … Is this a known issue?

Assistant: Yes, an exact match: KI-0324, BE SIGSEGV in RuntimeFilterProbeDescriptor::set_runtime_filter when receiving a shared global runtime filter (high confidence, 2 incidents).

  • What is happening: for joins across several BEs, installing the merged global runtime filter can race with query scheduling and crash the BE. It is not an out-of-memory problem.
  • How to confirm: the same PC and stack in be.out, pods exiting 139, memory well under limits.
  • Version: acme-prod runs 3.5.7-ee. Not fixed: upgrade to 3.5.9-ee or later on 3.5.x.
  • Verified fixes: none. Recommended by engineers, not verified: SET GLOBAL enable_global_runtime_filter = false;, which makes big joins scan more data.
  • Upgrade path: 3.5.9-ee or later.

One error text, several causes:

You: Dashboards fail with "StarRocks planner use long time 3000 ms in logical phase". Have we seen this?

Assistant: Yes, but three cards share this exact text, so the text alone can't decide: too many partitions and tablets (KI-0069), MV-rewrite analysis on complex MVs (KI-0295), or an HTTP proxy capturing internal gRPC on shared-data (KI-0371). To tell them apart, please check … (one test per card) … Which cluster is it?

The signature is there, but the build already has the fix:

You: A CN on acme-stage (4.0.14-ee) crashed with starrocks::serde::read_raw … Should they upgrade?

Assistant: The stack matches KI-0004, but 4.0.14-ee already contains its fix (first in 4.0.12-ee), so this card probably does not apply: a different cause, or an incomplete fix. Don't upgrade on this evidence alone. Here is how to tell which cause it is … and these mitigations were tried before and did not help: …

Reading the Answer​

MatchMeaning
ExactThe card's signature text is in what you gave. It is a known issue; still run the how to confirm check.
Exact, on a short phraseOnly a few generic words of the signature matched. The assistant checks the card's Applies when conditions against what you described before naming it. If one clearly fails, it calls the card a near match and says which condition fails.
PossibleOnly similar wording. Treat it as a lead and check the evidence the card names before acting.
Signature on fixed versionThe text matches a card, but your build already has that fix, so the card probably doesn't apply. The answer gives the two possibilities, (a) real data or a different cause, (b) an incomplete fix or a regression, and the check that tells them apart. The card's workaround is offered only as a stopgap while the cause is found, and you are never told to upgrade to a release you already run.
No matchNo card covers it. The assistant searches the knowledge base instead. It may explain what the message itself says, labelled general knowledge, not from the KB, unverified: never a known issue, a cause or a fix.
Version verdictMeaning
Not fixedYour build is older than the fix. The answer names the exact release to upgrade to.
Already fixedYour build contains the fix. If it is marked fix disputed, the issue has recurred on fixed builds before.
No fix on this lineNo release on your line has the fix. The answer names the release on the next line.
Hotfix buildYour build is a hotfix or custom build (a date or tag after the release, such as 4.0.14-ee-0824) that may already carry a backported fix. The assistant points you to the card's affected versions and notes instead of recommending an upgrade. It never claims a build carries a fix unless the card says so. A release followed only by its commit id is not a hotfix build.
Configuration issueNo release fixes it: apply the configuration or operational options.
No released fixFixed upstream but not in a release yet, or fixed in another component.

Also in each answer:

  • Confidence and incident count: high means at least two customer incidents plus a confirmed fix: a linked code fix that git finds in a release, or a configuration change that was applied and worked. Low means the fix or workaround is unconfirmed.
  • Applies when: the card's conditions (for example a table type or a deployment shape), each marked as holds, fails or unchecked. A condition about the cluster's shape (node count, shared-data, warehouses, external catalogs) counts as checked only against the live cluster, not billing records.
  • The fix PR: the pull request marked as the fix. Related PRs are labelled related, not the fix and are never cited as the fix.
  • Two extra warnings when a card carries them: the fix changes results or behaviour (what shifts after the upgrade), and wrong results with no error (confirm it by comparing results, not by looking for an error).
  • Range checks: for an error like 3524918272 conflict with range of (-2147483648, 2147483647), the answer states whether the value is outside the range and which integer type it is (here INT), with the numbers quoted as written.
  • Settings it suggests are checked against the Configuration Advisor's verified parameter list; the answer says when one isn't in it.
  • Confirming queries are valid StarRocks SQL. Where it can, the assistant asks you to re-run your own query under the card's conditions rather than writing a new one.
  • A card records where a fix landed, not where the bug began. "Not fixed on your version" does not mean your version has the bug.

When the answer is not a confirmed known issue, the assistant lists what to collect next: the query id, its profile, the table DDL, the session sql_mode, when it failed (with the time zone), and any runtime or JDK build named in the error.

Tips and Limits​

  • Paste exact text. Plain-language symptoms rarely match; the assistant then falls back to the knowledge base.
  • Name the cluster whenever you can, so you get a version verdict instead of a list of fix versions.
  • Live log scans read the cluster's logs through the Admin Console. The app reaches it through a temporary relay on an admin's laptop, so a scan works only while that laptop is awake and on the VPN. When it isn't, the assistant says the relay isn't answering and stops trying it in that conversation for about 15 minutes. Paste the log lines instead; pasted text is scanned the same way.
  • In a scan, only strong exact matches count as found. An exact match on a short generic phrase gets its own section, as in a lookup: a candidate until the card's Applies when conditions are checked. Similar matches are listed as possible, and routine warnings that match no card are listed separately so you can search the knowledge base for them.
  • It never applies a change and never runs anything on the cluster to test a fix. Replacing FE jars, changing the JDK or runtime, and FE/BE configuration changes go through CelerData support; the assistant never gives them as steps for you to run.
  • It does the lookups itself, including fetching a card's resolution options. It never asks you to run one of its internal tools.

See also: AI Chat, AI Triage, and Configuration Advisor for tuning a POC.