Shell Testcase Runner (CTP)

The cubrid-testcases-private-ex repository containing test cases must also be present. CUBRID itself does not need to be pre-installed.

Execution Steps

1. Request the build URL and install CUBRID

A build URL is required. If not provided, ask:

"A CUBRID build file URL is required. Please provide the build URL."

Install CUBRID and capture output for error checking:

sh ~/cubrid-testtools/CTP/common/script/run_cubrid_install <build_url> 2>&1 | tee /tmp/cubrid_install.log

Important: run_cubrid_install always returns 0 even on failure (prints [ERROR] instead) and removes $HOME/CUBRID before downloading. Verify by two checks:

# Check 1: Look for [ERROR] in install output
grep '\[ERROR\]' /tmp/cubrid_install.log

# Check 2: Verify CUBRID binary works
source ~/.cubrid.sh
cubrid --version

If either check fails, stop and show the [ERROR] lines from /tmp/cubrid_install.log. Ask the user to verify the URL.

2. Identify the test case

Determine the full path to the .sh file. Shell testcases follow this structure:

{test_name}/cases/{test_name}.sh

If the user gives a partial path, CBRD issue number, or test name, locate the actual file:

find ~/cubrid-testcases-private-ex/shell -name "<pattern>.sh" -path "*/cases/*"

3. Read the test script before running

Read the script to understand what it tests, what databases/services it uses, and any special requirements (e.g., WINDOWS_NOT_SUPPORTED). This context is essential for diagnosing failures.

4. Prepare the environment

source ~/.cubrid.sh
export init_path=$HOME/cubrid-testtools/CTP/shell/init_path

Verify both cubrid --version and ls $init_path/init.sh succeed. If either fails, stop and report what is missing.

5. Execute the test

Run from the cases/ directory (tests use relative paths for helpers/answer files). Use timeout of 300s:

cd /path/to/test_name/cases/
timeout 300 sh test_name.sh 2>&1 | tee /tmp/shell_runone_output.log
EXIT_CODE=$?

Exit code 124 means timeout.

6. Check the result

cat /path/to/test_name/cases/test_name.result

Result lines: test_name-1 : OK (pass) or test_name-1 : NOK (fail). Report each line.

7. Failure analysis (when NOK)

Perform a systematic investigation to give the user an actionable diagnosis.

Step A: Check the execution output

Review /tmp/shell_runone_output.log for CUBRID utility errors, csql errors, shell errors, or assertion failures (write_nok).

Step B: Check CUBRID server error logs

cat $CUBRID/log/server/*.err 2>/dev/null

Step C: Check broker logs if relevant

ls -la $CUBRID/log/broker/*.err 2>/dev/null

Step D: Check for core dumps

ls -la /path/to/test_name/cases/core* 2>/dev/null
ls -la $CUBRID/core* 2>/dev/null

A core dump indicates a server crash.

Step E: Compare actual vs expected output

If the test uses file-based comparison, diff the .answer file against the result file.

diff expected_file actual_file

Step F: Synthesize the diagnosis

Present the failure analysis:

1. Result: PASS or FAIL (with the exact result line)
2. What the test does: Brief summary from the script's header comment
3. Where it failed: Which phase (setup/test/verify/cleanup) and which specific command or check
4. Root cause: What went wrong and why, based on the log evidence
5. Relevant logs: Key error messages (quote the actual log lines)
6. Suggestion: What to fix — is it a test script bug, a CUBRID bug, or an environment issue?

8. Cleanup verification

After the test finishes, check for leftovers:

cubrid server status 2>/dev/null
ps -ef | grep cub_ | grep -v grep

If leftover processes or databases exist, warn the user and offer cleanup.

Handling Common Issues

• Test hangs: Check for unbounded loops, stuck server ($CUBRID/log/server/*.err), or lock waits (cubrid lockdb).
• Environment not set: Run source ~/.cubrid.sh and export init_path=$HOME/cubrid-testtools/CTP/shell/init_path.
• Permission denied: chmod +x test_name.sh
• Answer file missing: The test needs answer mode first (change init test to init answer in the script).

Output Format

On success:

[PASS] test_name
  - Result: test_name-1 : OK
  - Summary: <what the test verified>

On failure:

[FAIL] test_name
  - Result: test_name-1 : NOK
  - Summary: <what the test verified>
  - Failed at: <phase and command>
  - Root cause: <diagnosis>
  - Key logs:
    <relevant error lines>
  - Suggestion: <what to fix>